Files R1229_0 ... R1229_3 (written by TapeServer without data compression)

Timestamp data per FEE per 10s (i.e. 1 ch = 10s)

s3501-s3524 ADC data
s3601-s3624 PAUSE
s3701-s3724 RESUME
s3801-s3824 SYNC

Timestamp data per FEE per 10s (i.e. 1 ch = 10s)

s3501-s3524 ADC data
s3601-s3624 PAUSE
s3701-s3724 RESUME
s3801-s3824 SYNC

Little evidence of data loss or high dead time

See https://elog.ph.ed.ac.uk/AIDA/555

Date files May2017/R1_0 to May2017/R_79 are an extended (c. 3.8 day) background run with
BNC PB-4 Pulser OFF and and a relatively high slow comparator threshold 0x64 to reduce
data volume.

Attachment 1 - DSSSD#1-6 m_p versus m_n 

Attachment 2 - DSSSD#1-6 x versus y hit patterns

Attachment 3 - DSSSD#1-6 E_p versus E_n (dispersion ~20-23keV/channel)

Attachment 4 - DSSSD#1-6 E_p versus E_n (with tags)

Attachment 5 - 2D contour levels & colours for all 2D spectra above

Attachment 6 - MIDASsort subroutine

Attachment 7 - MIDASsort namelist I/O variables

Attachments 8-10 - ADC offset calculation logs etc

Attachment 11 - spectrum titles

MIDASsort calculates channel and ADC data as follows

       ch = channel + ( module - 1 ) * 64 + ( range * 2048 )
       adc_data = INT( RSHIFT( ABS( adc_data - 32768 ), 3 ) - offset(ch) + 0.5 )

       where

       channel = FEE channel # (0-63)
       module = FEE module # (1-24)
       range = LEC or HEC (LEC = 0, HEC = 1)

ADC offsets were calculated using the pulser walkthrough data of files May2017/R4
and command

cd ~/td/calibration
offsets /homes/npg/Spectra/R4 100 1400 15 10

where the peakfind peak search parameters are limits 100-1400 channels, fwhm=15 channels and acceptance=10%

Probably ~10% of ADC offsets need manual checking/adjustment

The 2D spectra (attachments 1-4) were produced for events with

multiplicities 0 < m_p < 8 *and* 0 < m_n < 8
all x-y combinations incremented

no clustering, gain matching or software thresholds

All DSSSDs show Uranium 4n+2 decay sequence background spectra from the intrinsic activity of the DSSSDs
and airborne decay products (e.g. 222Rn). Not clear why statistics are higher for DSSSD#1 cf. DSSSD#2-6.
One possibility is the activity of the aluminized Mylar window just upstream of DSSSD#1? Surrounding air
and Al snout would presumably affect DSSSD#1 and #6 about the same.

Analysis of Mar2017/R3_0 ... R3_9 by MIDASsort

*** ENTRY finish

 *** data items: ********** ( -48528.10 Hz)
 *** ADC events:  908055140 (  26136.76 Hz)
 *** time warps:          0 (      0.00 Hz)
 *** DSSSD # 1 count:       162 old count:       162 dt:  50309.28 s  LEC rate:      0.00 Hz
 *** DSSSD # 2 count:        69 old count:        69 dt:  50309.28 s  LEC rate:      0.00 Hz
 *** DSSSD # 3 count:        62 old count:        62 dt:  50309.28 s  LEC rate:      0.00 Hz
 *** DSSSD # 4 count:        70 old count:        70 dt:  50309.28 s  LEC rate:      0.00 Hz
 *** DSSSD # 5 count:        53 old count:        53 dt:  50309.28 s  LEC rate:      0.00 Hz
 *** DSSSD # 6 count:        62 old count:        62 dt:  50309.28 s  LEC rate:      0.00 Hz
 *** DSSSD # 1 count:         0 old count:         0 dt:  50309.28 s  HEC rate:      0.00 Hz
 *** DSSSD # 2 count:         0 old count:         0 dt:  50309.28 s  HEC rate:      0.00 Hz
 *** DSSSD # 3 count:         0 old count:         0 dt:  50309.28 s  HEC rate:      0.00 Hz
 *** DSSSD # 4 count:         0 old count:         0 dt:  50309.28 s  HEC rate:      0.00 Hz
 *** DSSSD # 5 count:         0 old count:         0 dt:  50309.28 s  HEC rate:      0.00 Hz
 *** DSSSD # 6 count:         0 old count:         0 dt:  50309.28 s  HEC rate:      0.00 Hz

 *** ENTRY finish ends

Background run (pulser off) with slow comparator thresholds ~1000keV
to check for intrinsic and extrinsic alpha activity in DSSSDs # 1-6
- see attachments 1-4 & 8

Spectra show alpha events distributed across the active areas of all 
DSSSDs - DSSSD #1 has largest number of alpha events

Analysis routine implantdecay.f, spectrum titles and sort variables
- see attachments 5-7

Note change of FEE64 configuration cf. September-December 2016.

Attachment 2 - analysis from *end* of file AIDA75_1

2D spectrum - FEE # versus timestamp (100us/channel)

1D spectrum - projection of timestamp data for FEE # = 6

Conclude data synchronised at beginning and end of AIDA75_1. 
Period of pulser data ~36ch = 3.6ms => pulser frequency ~ 280Hz

Attachment 3 - analysis around ts=0x18bacd75c4 +/- 20000000 (dec)

Conclude FEE # 6 stops producing ADC data c. ch 353

Attachment 4 - analysis around ts=0x18cac6144b +/- 20000000 (dec)

Conclude FEE # 6 starts producing ADC data c. ch 28

Spectrum s4100 y-channel calculated as follows

C        IF ( ts.GE.z'18bacd75c4' - 20000000
C----67---
C     +       .AND.
C     +       ts.LE.z'18bacd75c4' + 20000000 ) THEN
        IF ( ts.GE.z'18cac6144b' - 20000000
C----67---
     +       .AND.
     +       ts.LE.z'18cac6144b' + 20000000 ) THEN
C
C
C       Increment ts spectra per FEE64 module

        j = MOD( INT( ts*10.0D-09*1.0D+04 ), 4096 )
        IF ( j.EQ.0 ) THEN
         DO k = 0, 4095
          DO l = 1, 31
           CALL set2d( 4100, l, k, 0 )
          ENDDO
         ENDDO
        ENDIF
        CALL inc2d( 4100, module, j )

        ENDIF

See https://elog.ph.ed.ac.uk/AIDA/324

Analysis of first and last data file - see attachments 1-2
Rates per FEE64 ~20k data items/s

analyser2.f program - see attachment 3

To compile the analyser2.f program (SL6)

gfortran analyser2.f -o analyser2

analyser2 <filename> to produce summary statistics
analyser2 v <filename> to produce verbose analysis

ADC multiplicity as a function of time (1 channel = 0.1ms) for file R1_114 - see attachment 4
Pulser running at ~270Hz clearly identifiable

Conclusion
DAQ ran overnight without stalling
DAQ did not lose synchronisation

Decrease comparator thresholds to increase rates to experiment-realistic levels ~50k data items/s/FEE64

When using the T9 system and running with too much rate in one FEE64 so the system is unbalanced.

I found a good way to understand this and to be able to operate the controls of the FEE64s was to place the Merger in "Pause" mode.

Then the data is extracted from the busy FEE64 but it can still be controlled.

When there is unbalance in the system it is possible for one or more queue in the Merger to become full and so the communication with that FEE64 is not possible.

Placing the Merger in Pause mode will mean all the data is discarded, the links run at full speed and action can be taken on the problem FEE64(s) to help balance the system out better.

I hope this helps .... it is highly likely that this is already being done.....

Perhaps a wise sequence when starting the whole DAQ is to operate with the Merger in Pause mode and  only switch it to merging when the user is happy with the rates.

In the statistics window of the FEE64 there is an indication of the data being transferred over the link.

When deciding how to run the experiment thought needs to be given to the various maximum rates. The rate from the FEE64 is 500k events/sec. If running with 26 FEE64 then this goes up to 13M events/sec. An event is 64 bits.

I'm not sure what the maximum rates are in the Merger and the TapeServer.

Analysis of tape folder /TapeData/BG18OCT23

Using Go4 https://github.com/cej25/DESPEC_NOV23

Figure 1: Hit pattern (XY)
Figure 2: Energy 1D Spectra, alpha peaks visible
Figure 3: Energy X vs Energy Y
Figure 4: 1D Strip (0-386 = X 0-386, 386-512 = Y 0-128) and Energy

Some gaps at 112-114, 121-128 = aida05 
Hot channels at 368, 381, 382 = aida06 (high rate seen)
Gap at 435-450 = aida04 (ASIC#4 which is faulty)

Generally 1-10 counts per strip, 1 ch wide gaps may be statistics or issues

 This guide is to help with the filter pcb assembly and installation into the AIDA power supply.

During 'SETUP' the ASIC readout clock ( 500KHz ) is synchronised to the SYNC pulse in V8.10 and later versions of the VHDL.

Attached is a short run for verification.

I have taken videos with an Infrared camera of the power being supplied to one isolated ASIC on a mezzanine.

The power supply is a bench supply with current limit set to 3A.

The centre of the ASIC glowed white hot for a short time and then the whole ASIC shows as hot. Ths the short is on the ASIC and not the bond wires IMO.

I am unable to globally publish the videos at present as I only have a private google drive account.

Please contact Tom Davinson for further details.

Attachment 1 - DHCP configuration file /etc/dhcpd.conf

Attachment 2 - DHCP startup messages

Attachment 3 - FEE64 startup messages from /var/log/messages

Note

1) dhcpd service is not listed by services tool which we have usually used to restart dhcp following
changes to the configuration. Currently we restart dhcpd as follows

kill -SIGTERM <pid dhcpd>
/usr/local/sbin/dhcpd

2) significant delays (~6 and ~9 minutes) starting aida07 and aida05, otherwise appears to be normal startup

> 
> Attachment 1 - DHCP configuration file /etc/dhcpd.conf
> 
> Attachment 2 - DHCP startup messages
> 
> Attachment 3 - FEE64 startup messages from /var/log/messages
> 
> Note
> 
> 1) dhcpd service is not listed by services tool which we have usually used to restart dhcp following
> changes to the configuration. Currently we restart dhcpd as follows
> 
> kill -SIGTERM <pid dhcpd>
> /usr/local/sbin/dhcpd
> 
> 2) significant delays (~6 and ~9 minutes) starting aida07 and aida05, otherwise appears to be normal startup

Regarding startup of 5 and 7. what is the report from the console log monitor?

OH setup AIDA external clock using  Tektronix AFG3102 - settings as follows

CH1 ON - amplitude +5V, phase 0n deg, offset 0V, frequency 50.000000MHz

JA setup DTAS/BigRIPS clock using Tektronix AFG3102 - settings as follows

CH2 ON - amplitude -0.8V, phase 0n deg, offset 0V, frequency 25.000000MHz

Unfortunately the AGF3102 was set to CH1 frequency = CH2 frequency and this changed the AIDA external
clock frequency from 50MHz to 25MHz.

This stopped the DAQ. Interestingly it was possible to Resync successfully and *all* system wide
checks were OK *except* Check Clock Status which produced errors - see attachment 1.

CH1 & CH2 now set to 50MHz and 25MHz respectively.

AIDA DAQ now running OK.

 I actually topped off the chiller with water tonight as I am returning to Scotland (and then off to CERN) within hours.

I added 4 liters on around July 5th, to move the marker from a little under the halfway mark to over the halfway mark.  After 2.x weeks, it was still sitting a bit shy of under the halfway mark.  This means that in a humid and hot Japanese month like July, AIDA doesn't evaporate more than something like 2 liters in one week. 

As it's now topped off, I think it's good to go for something like several months.

I also note that I use a faucet in the old Nishina Center, rather than passing radiation control, to find a water tap.  It's maybe more of a hike but it avoids going through radiation control.

Anyway, the point of my elog post is that the AIDA chiller will not be running out of water any time soon, probably not before October, though local members at RIKEN might check it each month etc, or on request.  I left the bucket and the large-volume container right sitting by it.  I still don't know where the funnel is, so I robbed (borrowed) a kind of 1 liter handled jug with a beaker from a place, and returned it.

10.11 Detector bias OFF
      FEE64 power OFF
      BNC PB-4 OFF
      2x NIM bins OFF - fan tray OFF
      Julabo FL11006 chiller OFF - 3ph mains switch OFF
      USB ac mains relay OFF - flow interlock enabled - USB disconnected 

      Raspberry Pis ON
      Tektronix AFG3102C ON

Viewing AIDA status on Thursday morning 16/11/2023. 

Rates and histograms are observed with bPlast HV connected but not powered. The LIMO cables to the booster cards of bPlast are connected and the TAMEX crate is powered.

The following details the various tests of bPlast configurations and AIDA response:

1. The rates and status are the same as yesterday evening (15/11/2023) where we had the same configuration (bPlast connected to HV but not powered). (Attachment 1-3)

2. The rates and status are the same as in 1. when trying a different power supply but it is not turned on (Attachment 4-5). The R&S HMP4040 HV supply is a linearly regulated supply while the new supply is suspected to be switching regulated (?).

3. The rates and status are the same as in 1.,2. if the HV supply is connected. In fact they seem to increase when disconnecting the HV supply. (Attachment 6).

4. The rates and status are brought back to acceptable levels when disconnecting all the LIMOs of the bPlast booster board to TAMEX. (Attachment 7-8).

5. The rates and status are fairly worse when connecting a single LIMO (attachment 9) and when connecting the Mesytec module that powers the booster board (attachment 10).

6. The rates and status increase a little favourably when connecting a single LIMO to an odd numbered channel. (Attachment 11).

7. The rates and status are dramatically worse when connecting two LIMOs and a Mesytec module to the bPlast booster card. (Attachment 12).


The next tests will be dismounting the snout and trying to shield the bPlast ribbon cables with copper tape.

Tests resumed after lunch 14:00:00:

- The bPlast ribbon cables running to the booster boards were insulated with copper tape and kapton tape, and for the moment grounded freely with a tinned copper braid. A multimeter check of the ribbon cable showed that it is isolated from the snout as desired.

- Remounting the snout we found the following result on the AIDA rates and hit patterns (Attachment 15 & 16). The rates and hit patterns appear to be the same as it was before bPlast was mounted in the snout. The connector cable of AIDA01 was found to be broken for ASICs 1&2 (3&4 are still connected), and the noise on AIDA04 has increased slightly relative to previous tests.


8. The wave forms of the AIDA modules show regular structure. There are some high frequency 'beats' present in some of the AIDA modules (Attachment 13-14). The waveform of AIDA01 has the pulser signal (25 Hz), but has no noise as ASIC 1&2 is missing.

9. The next test carried out was that all the ribbon cables ground were connected to the separate grounds on the item frame and on the frame carrying the booster cards. The bPlast ribbon cables were also plugged into the booster board. (attachment 17-20).

10. The next test we connect all the ribbon cables to the same ground on the item frame and the ribbon cables were disconnected from the booster cards. (attachment 21 -24).

11. The test thereafter we connect all the grounds together but not grounded to anything, just floating in air. The ribbons were still disconnected from the booster cards. Rates were observed to decreased back to normal levels from the previous grounding configuration. (Attachment 25-28)

12. The final test carried out from the grounding tests was connecting the bPlast ribbon cables with floating air ground to the booster cards mounted around the snout. The rates and wave forms observed are back to normal and transience in some of the wave forms from (Attachment 20-28) have gone away after adjusting the LED and wave form capture threshold from 8900 -> 8000/8200. The rates and wave forms are back to the same point when we started the tests after lunch (Attachment 29-32).  

The next series of tests involved connecting and/or powering the bPlast HV and LIMOs individually to see their input to AIDA. Each of the tests were carried out individually, resetting back to the configuration from (Attachment 29-32, test 12.): 

13. The first test carried out was connecting a LIMO from the TAMEX card to the long side booster card of the bPlast. (Attachment 33-36). The rates, wave form and hit patterns seem to be dramatically affected by simply connecting 1 LIMO to the long side booster card.

14. The second test was connecting the HV supply of the booster card on the short side (right relative to the beam axis) and powering it with the R&S HMP4040 supply. Results are given by (Attachments 37-40). Rates, hit pattern and wave forms change and increase drastically.

15. The third test was simply connecting the booster card HV supply (R&S HMP4040) but not powering it at all. Results are given by (attachment 41-44). Simply connecting the HV supply is introducing a considerable amount of noise into AIDA. (Maybe grounding the supply, or changing its position relative to the DESPEC setup is a next possible test).

16. The fourth test was biasing the SiPMs on the short side booster card ie. right side of the bPlast with the Mesytec module 29.00 V @ 4 uA. The results are given by (attachment 45-48). The rates, hit patterns and wave forms return to more reasonable levels, however, it is clear that biasing the SiPMs is still introducing a great deal of noise into some of the ASICs based on the proximity. eg. aida03 ASIC1. 

17. The fifth and final test of the day was under-biasing the SiPMs of the long side of the bPlast with the Mesytec module 26.80 V @ 19 uA. The results are given by (attachment 1-4 in AIDA tests 16/11/2023 cont.). The rates, hit patterns and wave forms are the same as that which was carried out in test 16.


The tests wrap up with suspicion to the HV supply of the booster cards, whether it is grounded or not and its position in the setup. The grounding of the booster cards is also seen to affect AIDA substantially. Isolating and grounding the ribbon cables has proved to be helpful when biasing the SiPMs as the noise in AIDA is reduced in comparison to previous tests. The wave forms when connecting the HV supply seems to show a 100 kHz signal (HV supply cross-talk?).

We restarted AIDA (a10277ttachment 6) this morning and checked rates (attachments 3/4) and waveforms (attachments 1/2). Temperatures are shown in attachment 5.
It seems consistent with what was observed yesterday.

AIDA stopped and platform moved.

The power cable from USB main power relay was connected to the wall and not the DESPEC 19" racks as expected. It has been corrected : it is now connected to the DESPEC 19" rack, has it should be.
AIDA was restarted after that. bPlast was disconnected from 2 booster cards.
We checked the rates (attachments 7,10 and 11 [linear scale]). They are quite similar, although aida01 rate has increased (maybe a cable issue), and rate from aida07 significantly decreased (it was really low yesterday as well).
The waveforms (attachments 8 & 9) are similar as the ones obtained earlier.

bPlast has been reconnected to all booster cards. Rate increased a little (attachment 12) but broadly similar to that observed first thing this morning.
Then 2 booster cards were grounded, and the rates went up (attachments 13-16)

Afterwards, one power cable was connected to the booster card on the short side, increasing the noise, as observed yesterday (attachments 17-20)
 
The Rohde Schwarz PSU output was referenced to PSU ground ( 3mm cable to ground socket front panel ). It did not reduced much the noise (attachment 21)
Finally, the power supply was switched on, increasing the noise in AIDA (attachment 22).


After the meeting, we have chosen to reconnect bPlast to the booster : now all LIMOs are connected to the booster cards, the cards are powered and the SiPM are biased. The alarm for the short side has triggered (not full bias)
The Rohde & Scwarz PSU outputs is till referenced to its chassis ground.
See attachments 23-26 for the rates and waveforms in AIDA


14.57 JG/JB added additional black cloth to reduce nPlast SiPM dark current : now it is around 16uA. Updated noise observations

per FEE64 1.8.W spectra 20us FSR - attachments 27-28
per FEE64 1Rate spectra - attachment 29
ADC data item stats - attachment 30

aida02 asics #2-4 remain noisy c. kHz/channel 
Most other p+n FEE64s are OK - the usual 'hot' channels but most other channel rates low 

15.07 JG removed ground reference for Rohde & Schwarz PSU +4V output to bPlast booster cards
      ADC data items atst - attachment 31
      Most FEE64s have significantly higher rates so the ground reference is doing 'something'

15.12 JG re-ground ref'd Rohde & Schwarz PSU +4V output to bPlast booster cards
      ADC data item stats - attachment 32 
      Rates for most FEE64s are reduced - some significantly - PSU ground ref is doing something reproducible 

15.25 Christoph Scheidenberger reports FRS magnets ON

16.01 JG/JB TAMEX & NIM bin electrically isolated from AIDA stand/DESPEC platform. Screens of flat cables from  bPlast to booster cards grounded to snout.
      ADC data item stats - attachment 33
      per FEE64 1.8.W spectra 20us FSR - attachments 36-37
      per FEE64 Rate spectra - attachments 34-35


16.20 TAMEX & NIM bin electrically isolated from AIDA stand/DESPEC platform. Screens of flat cables from  bPlast to booster cards now connected together but electrically floating

     ADC data item stats - attachment 40
      per FEE64 1.8.W spectra 20us FSR - attachments 38-39
      per FEE64 Rate spectra - attachment 41
      Noted improvement in most FEE64 rates - aida04 improved significantly


16.31 TAMEX & NIM bin electrically isolated from AIDA stand/DESPEC platform. Screens of flat cables from  bPlast to booster cards now connected together and grounded to NIM bin chassis.
      ADC data item stats - attachment 42
      per FEE64 Rate spectra - attachment 43-44
      per FEE64 1.8.W spectra 20us FSR - attachments 44-46
      Continued, imcremental improvement in FEE64 rates 

Screenshot uploads as I hit the 50 image upload limit.