AIDA GELINA BRIKEN nToF CRIB ISOLDE CIRCE nTOFCapture DESPEC DTAS EDI_PSA 179Ta CARME StellarModelling DCF K40
  DESPEC, Page 14 of 37  ELOG logo
ID Date Author Subjectdown
  447   Sat May 14 06:57:21 2022 MS, OHSaturday 14 May
08:00 Took over the shift from Philippos
      In system wide checks aida09 fails ASIC clock check but it is bit 6 which is ok
  448   Sat May 14 09:00:21 2022 MS, OHSaturday 14 May

10:00

  449   Sat May 14 10:59:52 2022 MS, OHSaturday 14 May

12:00

  450   Sat May 14 13:01:58 2022 MS, OHSaturday 14 May

14:00

  451   Sat May 14 15:35:24 2022 BA, AASaturday 14 May
  452   Sat May 14 17:34:40 2022 BA, AASaturday 14 May

FEE64 module aida09 global clocks failed, 6
 Clock status test result: Passed 13, Failed 1

Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable

 

FEE64 module aida02 failed
FEE64 module aida06 failed
FEE64 module aida09 failed
FEE64 module aida10 failed
FEE64 module aida13 failed
Calibration test result: Passed 9, Failed 5

If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module

White Rabbit error counter test result: Passed 14, Failed 0

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

 

FPGA Timestamp error counter test result: Passed 14, Failed 0
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last

 

  453   Sat May 14 19:39:13 2022 BA, AASaturday 14 May

FEE64 module aida09 global clocks failed, 6
 Clock status test result: Passed 13, Failed 1

Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable

 

FEE64 module aida02 failed
FEE64 module aida06 failed
FEE64 module aida09 failed
FEE64 module aida10 failed
FEE64 module aida13 failed
Calibration test result: Passed 9, Failed 5

If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module

 

White Rabbit error counter test result: Passed 14, Failed 0

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

 

FPGA Timestamp error counter test result: Passed 14, Failed 0
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last

  454   Sat May 14 21:34:11 2022 BA, AASaturday 14 May

FEE64 module aida09 global clocks failed, 6
 Clock status test result: Passed 13, Failed 1

Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable

 

FEE64 module aida02 failed
FEE64 module aida06 failed
FEE64 module aida09 failed
FEE64 module aida10 failed
FEE64 module aida13 failed
Calibration test result: Passed 9, Failed 5

If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module

 

White Rabbit error counter test result: Passed 14, Failed 0

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

 

FPGA Timestamp error counter test result: Passed 14, Failed 0
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last

  705   Sat Jun 14 11:20:45 2025 TDSaturday 14 June 2025
12.21 BNC PB-5 settings
      Amplitude 1.0V
      Attenuator x1
      Frequency 22Hz
      Polarity +
      Tau_d 1ms
      Tail pulse
      Clamp ON
      Pulser output connected to p+n test daisy chain - n+n test daisy chain not connected 

      per DSSSD green/yellow ground daisy chain was removed yesterday

      FEE64 adaptor PCB grounds connected by heavy duty braid wrapped in copper foil to AIDA support assembly frame
      Snout connected to AIDA support assembly frame by heavy duty braid bolted to frame and connected to snout by adhesive copper foil.

      DSSSD - FEE64 adaptor PCB cabling (cannot connect some ribbon cables because cable flipped and connector key now incorrectly oriented)

                    aida10 aida14 aida13
                      ox     xo     oo
                    aida09 aida01 aida05
                      oo     oo     xo
      aida06 aida02                      aida04 aida08    *Beam into screen*
        ox     oo                          oo     oo
                    aida15 aida03 aida12
                      oo     oo     ox
                    aida11 aida07 aida16 
                      ox     oo     xo

      o = connected
      x = not connected
      e.g. J1 J2 connected = oo, J1 only connected = ox

      Adaptor PCB top/component side
      E     J1
      R
      N
      I     J2

      Adaptor PCB 
       LK1 fitted aida02, aida04, aida06, aida08
       LK3 fitted aida03, aida07

      DSSSD bias & leakage current - attachment 1
      FEE64 temps OK - attachment 2
      All system wide checks OK *except* WR decoder status aida02 - attachment 3
      WR timestamps OK - attachment 4
      ADC data item stats - attachment 5

      per FEE64 Rate spectra - attachments 6-7
       note ASICs with per channel rates of 22Hz (pulser)
       note *all* FEE64 slow comparator 0x64 ( *note* subsequently found ASIC #2-4 of a number of FEE64s were set to 0x0 )

      per p+n FEE64 1.8.W spectra - 20us, 200us, 2ms and 20ms FSR - attachments 8-11
      per p+n FEE64 1.8.W spectra - 20ms FSR - expanded y-scale = 2000 x 122uV/LSB = 0.244V FSR - attachment 12

      per n+n FEE64 1.8.W spectra - 20ms, 2ms, 200us and 20us FSR - attachments 13-16

      per p+n FEE64 1.8.L spectra - attachment 17
       pulser peak widths
        aida05, aida14, aida16 c. 15-20 ch FWHM ( c. 10-15keV FWHM ) - cable from DSSSD not connected to adaptor PCB
        aida09 c. 120 ch FWHM - electronic noise c. 2x worse than configuration used for 2023/4 tests
        all other FEE64s *very* noisy/unstable


14.45 Found some FEE64s with slow comparator 0x64 for ASIC#1 and 0x0 for ASICs #2-4
      Slow comparator 0x64 now set for all ASICs of all FEE64s

      ASIC settings saved to 2025Jun14-15.00.09

      ADC data item stats - attachment 18

      per FEE64 Rate spectra - attachments 19-20
       note ASICs with per channel rates of 22Hz (pulser)
       note *all* FEE64 slow comparator 0x64
       note aida10 to be connected to MSL type BB7 - ignore for time being

      per FEE64 1.8.W spectra - 20us FSR - attachments 21-22

      per p+n FEE64 1.8.L spectra - attachments 23-24
       pulser peak widths
        aida05, aida14, aida16 c. 20 ch FWHM - cable from DSSSD not connected to adaptor PCB
        aida09 c. 120 ch FWHM - electronic noise c. 2x worse than configuration used for 2023/4 tests
  676   Sat Dec 14 11:52:42 2024 TDSaturday 14 December
11:00 Visual inspection FEE64 adaptor PCBs & cabling
      aida02 ground from/to other FEE64s disconnected - re-connected
      aida01, aida09, aida12 - ground cabling screws to Lemo 00.250 housings loose - tightemed

12:50 Cooling water pressure & temperature OK - attachments 1  2

12:59 relay #1 power ON

13:01 relay #2 power ON

13:07 aida06 starts - panic during startup, automatic restart following 3 minute timeout

      DAQ reset, setup

      Check ASIC Control - browser tab timeout 
      AIDA MIDAS HTTPD server console log - attachment 3

      Appears to have restored ASIC settings 2024Dec13-17.02.45 saved yesterday
    
      aida10 ASICs #1 & #2 positive input, ASICs #3 & #4 negative input
      slow comparator 0xa (all p+n junction FEE64s and aida10), 0xf (n+n Ohmic FEE64s)

13:28 tar ASIC settings - attachment 4

[npg@aidas-gsi]$ cd /MIDAS/FEE_ASIC
[npg@aidas-gsi FEE_ASIC]$ tar cvf /tmp/FEE_ASIC.tar .

       System wide checks

       Sync ASIC clocks - attachment 5

       Clock, ADC calibration, WR decoder, FPGA timestamp, PLL checks - attachments 6-11
        all OK *except* aida02 WR decoder error

       WR timestamps OK - attachment 12
       FEE64 temps OK - attachment 13
       
13:45 Detector bias ON - attachment 14      

      BNC PB-5 pulser settings - attachment 15
      Pulser connected to all p+n junction FEE64s *except* aida10

      ADC, DISC, PAUSE and MBS correlation scaler stats - attachments 16-19
       aida02 rate significantly lower than yesterday - https://elog.gsi.de/despec/Implantation+Stack/8
       high rates observed for aida08, aida11 and aida14 - which are not connected to a DSSSD!

      per FEE64 Rate spectra - attachment 20

      per p+n junction FEE64 1.8.L spectra - attachment 21
       aida09 pulser peak width 56 ch FWHM = 39 keV FWHM
       consistent electronic noise for all p+n junction FEE64s (cabling+DSSSD)
       electronic noise of p+n junction FEE64s (cabling *only*) higher and more variable cf. https://elog.gsi.de/despec/Implantation+Stack/8 attachment 5

      per p+n junction FEE64 1.8.W spectra 20us FSR - attachments 22-23

      per n+n Ohmic FEE64 1.8.W spectra 20us FSR - attachment 24 

      WR timestamps OK - attachment 25

14:43 DAQ STOP
      Data transfer enabled
      Select Tape Server -> Next Run
      DAQ GO  data file R3

      Merger, Tape Server - attachments 26-27
       data transfer rate c. 900k data items/s cf c. 300k data items/s yesterday https://elog.gsi.de/despec/Implantation+Stack/8

16:45 DAQ STOP

      Data transfer disabled 

      Detector bias OFF

      FEE64 power OFF
  196   Sat Mar 13 17:00:49 2021 TDSaturday 13 March 18.00-00.00 UTC+1
18.00 DAQ continues OK - file R46_64

      ASIC settings 2019Dec19-16.19.51
      DSSSD#1 slow comparator 0xa
      DSSSD#2 slow comparator 0xa
      DSSSD#3 slow comparator 0xd

      BNC PB-5 Pulser 
      Amplitude1.0V
      Attenuation x1
      Frequency 2Hz
      tau_d 1ms
      - polarity
      Delay 250ns, tail pulse

18.02 System wide checks OK *except*

FEE64 module aida09 global clocks failed, 6
 Clock status test result: Passed 11, Failed 1

Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1' 
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable

FEE64 module aida09 failed
Calibration test result: Passed 11, Failed 1
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module

      FEE64 Temperatures OK - attachment 1
      Good event statistics OK - attachment 2
      Detector bias & leakage currents OK - attachment 3
      Merger OK - 4.6M data items/s
      TapeServer OK - 16Mb/s

      All histograms zero'd

17.15 Most recent messages in merger server terminal session

MERGE Data Link (2322): bad timestamp  6 3 0xc1b57e35 0x0ba1685c 0x0000ef3fbba1685c 0x166bef3fbba1685c 0x166bef3fbba1879c
MERGE Data Link (2322): bad timestamp  6 3 0xc1a77dc2 0x0ba1702c 0x0000ef3fbba1702c 0x166bef3fbba1702c 0x166bef3fbba1879c
MERGE Data Link (2322): bad timestamp  6 3 0xc1bf7f66 0x0ba1702c 0x0000ef3fbba1702c 0x166bef3fbba1702c 0x166bef3fbba1879c
Transfer Error - : Broken pipe
Error 32
1: send() failed:
TCP transfer library version 4.0
1: TCP socket send buffer was 16384 - now 249856
1: TCP socket receive buffer was 87380 - now 249856
1: TCP socket created OK - now connecting to localhost port 10305
1: Connected to localhost port 10305


22.05 DAQ continues OK - file R46_168

FEE64 module aida09 global clocks failed, 6
 Clock status test result: Passed 11, Failed 1

Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1' 
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable

FEE64 module aida09 failed
Calibration test result: Passed 11, Failed 1
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module

		 Base 		Current 	Difference
aida07 fault 	 0x82a0 : 	 0x82a2 : 	 2  
White Rabbit error counter test result: Passed 11, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

			 Base 		Current 		Difference
aida07 fault 	 0x2 : 	 0x3 : 	 1  
FPGA Timestamp error counter test result: Passed 11, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last

	
Returned 0 0 0 0 0 0 0 0 0 0 0 0  
Mem(KB) :	4	8	16	32	64	128	256	512	1k	2k	4k
aida01 :     18      6      7      5      1      3      1      3      3      4     10   : 54856
aida02 :     19      6      4      3      4      4      2      3      3      3      6   : 36892
aida03 :     22      8      1      2      1      2      1      3      3      3      6   : 36136
aida04 :     24      8      4      2      1      3      2      4      1      4     15   : 73952
aida05 :     37     14      9      8      7      3      2     18      7      4      7   : 55252
aida06 :     28     12     16      4      4      2      1      4      3      2      5   : 31056
aida07 :     28      9      2      2      2      2      1      3      3      3      6   : 36248
aida08 :     21     12      6      4      3      4      2      3      3      3      6   : 36948
aida09 :     18      5      2      1      1      3      2      2      3      3      6   : 35952
aida10 :     17      8      4      5      3      3      1      3      3      3      6   : 36516
aida11 :     21      8      5      0      0      4      3      3      2      3      6   : 35812
aida12 :     13     11      3      3    


      FEE64 Temperatures OK - attachment 4
      Good event statistics OK - attachment 5
      Detector bias & leakage currents OK - attachment 6
      Merger OK - 4.4M data items/s
      TapeServer OK - 15Mb/s

22.17 Rate spectra - attachments 7 & 8
      p+n junction HEC spectra - attachment 9

      Merger server error messages since 17.15

MERGE Data Link (2316): bad timestamp  0 3 0x80300000 0x0164e7f8 0x000000000164e7f8 0x166200000164e7f8 0x166bffffffa7654e
MERGE Data Link (2324): bad timestamp  8 3 0x88300000 0x0534e25c 0x000000000534e25c 0x166200000534e25c 0x166bffffed8383fc
MERGE Data Link (2322): bad timestamp  6 3 0xc1a07f16 0x0070e9cc 0x000001e74070e9cc 0x166c01e74070e9cc 0x166c01e74071090c
MERGE Data Link (2322): bad timestamp  6 3 0xc1b37895 0x0070e9cc 0x000001e74070e9cc 0x166c01e74070e9cc 0x166c01e74071090c
MERGE Data Link (2322): bad timestamp  6 3 0xc19f7e41 0x0070f19c 0x000001e74070f19c 0x166c01e74070f19c 0x166c01e74071090c
MERGE Data Link (2322): bad timestamp  6 3 0xc1a27e22 0x0070f19c 0x000001e74070f19c 0x166c01e74070f19c 0x166c01e74071090c
MERGE Data Link (2322): bad timestamp  6 3 0xc1b47399 0x0070f19c 0x000001e74070f19c 0x166c01e74070f19c 0x166c01e74071090c
MERGE Data Link (2322): bad timestamp  6 3 0xc1ab7dad 0x0070f96c 0x000001e74070f96c 0x166c01e74070f96c 0x166c01e74071090c
MERGE Data Link (2322): bad timestamp  6 3 0xc1b564f9 0x0070f96c 0x000001e74070f96c 0x166c01e74070f96c 0x166c01e74071090c
MERGE Data Link (2322): bad timestamp  6 3 0xc1b60177 0x0071013c 0x000001e74071013c 0x166c01e74071013c 0x166c01e74071090c
MERGE Data Link (2322): bad timestamp  6 3 0xc1a57e95 0x00e2208c 0x000001e700e2208c 0x166c01e700e2208c 0x166c01e74071090c
MERGE Data Link (2322): bad timestamp  6 3 0xc1b81491 0x00e2208c 0x000001e700e2208c 0x166c01e700e2208c 0x166c01e74071090c
MERGE Data Link (2322): bad timestamp  6 3 0xc1ac7e4f 0x00e2285c 0x000001e700e2285c 0x166c01e700e2285c 0x166c01e74071090c
MERGE Data Link (2322): bad timestamp  6 3 0xc1b90000 0x00e2285c 0x000001e700e2285c 0x166c01e700e2285c 0x166c01e74071090c
MERGE Data Link (2322): bad timestamp  6 3 0xc1ba0c64 0x00e2302c 0x000001e700e2302c 0x166c01e700e2302c 0x166c01e74071090c
MERGE Data Link (2322): bad timestamp  6 3 0xc1bb6ea8 0x00e237fc 0x000001e700e237fc 0x166c01e700e237fc 0x166c01e74071090c
MERGE Data Link (2322): bad timestamp  6 3 0xc1a87dc6 0x00e23fcc 0x000001e700e23fcc 0x166c01e700e23fcc 0x166c01e74071090c
MERGE Data Link (2322): bad timestamp  6 3 0xc1bc0641 0x00e23fcc 0x000001e700e23fcc 0x166c01e700e23fcc 0x166c01e74071090c
  365   Sat Jun 12 13:56:00 2021 TDSaturday 12 June
14.11 DAQ continues file S496/R32_697
      High data rate from c. 06.00 this morning
      ASIC check 

      System wide checks OK *except*

		 Base 		Current 	Difference
aida01 fault 	 0x8944 : 	 0x8948 : 	 4  
aida02 fault 	 0x9b36 : 	 0x9b3a : 	 4  
aida03 fault 	 0xf26b : 	 0xf26f : 	 4  
aida04 fault 	 0x57a0 : 	 0x57a4 : 	 4  
aida05 fault 	 0x4d3c : 	 0x4d3e : 	 2  
aida05 : WR status 0x10
 aida06 fault 	 0x640c : 	 0x640e : 	 2  
aida07 fault 	 0x255e : 	 0x2560 : 	 2  
aida08 fault 	 0xffbd : 	 0xffbf : 	 2  
White Rabbit error counter test result: Passed 8, Failed 8

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

			 Base 		Current 		Difference
aida12 fault 	 0x0 : 	 0x21be : 	 8638  
aida13 fault 	 0x0 : 	 0x2294 : 	 8852  
FPGA Timestamp error counter test result: Passed 14, Failed 2
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last

Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0  
Mem(KB) :	4	8	16	32	64	128	256	512	1k	2k	4k
aida01 :     33     10      9      2      5      4      2      2      2      4      6   : 37604
aida02 :     46     12      7      2      4      4      3      2      2      4      6   : 37832
aida03 :     34     10      9      5      4      3      1      3      3      3      6   : 36744
aida04 :     35     12     10      1      3      4      2      3      3      3      6   : 36972
aida05 :     37      9      8      3      4      4      2      2      2      4      6   : 37564
aida06 :     34     10      5      5      4      2      2      3      2      4      6   : 37832
aida07 :     31     11      1      7      4      2      3      2      2      4      6   : 37572
aida08 :     40      7      4      5      3      4      3      2      2      4      6   : 37752
aida09 :     39      9      8      4      1      3      3      2      2      4      6   : 37540
aida10 :     43     12      8      4      4      3      3      3      1      4      6   : 37260
aida11 :     34     14     10      5      3      4      3      3      1      4      6   : 37368
aida12 :     46     12      3      4      4      4      3      3      1      4      6   : 37320
aida13 :     41     11      5      4      5      3      3      3      1      4      6   : 37260
aida14 :     39     10      2      6      4      4      3      3      1      4      6   : 37324
aida15 :     19     12      4      3      2      2      3      2      2      4      6   : 37324
aida16 :     45     10      6      4      5      3      3      3      1      4      6   : 37284

Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same

 FEE : aida01 =>   Options file size is 1026 	 Last changed Fri Jun 04 09:30:42 CEST 2021
 FEE : aida02 =>   Options file size is 1025 	 Last changed Sun May 23 00:19:21 CEST 2021
 FEE : aida03 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:43:50 CEST 2021
 FEE : aida04 =>   Options file size is 1025 	 Last changed Fri May 14 16:54:56 CEST 2021
 FEE : aida05 =>   Options file size is 1025 	 Last changed Mon May 17 06:25:41 CEST 2021
 FEE : aida06 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:43:59 CEST 2021
 FEE : aida07 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:02 CEST 2021
 FEE : aida08 =>   Options file size is 1025 	 Last changed Sun May 23 00:16:54 CEST 2021
 FEE : aida09 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:08 CEST 2021
 FEE : aida10 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida11 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida12 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida13 =>   Options file size is 1025 	 Last changed Fri May 07 19:40:34 CEST 2021
 FEE : aida14 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida15 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida16 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021

      Grafana - DSSSD bias & leakage current - most recent 7 days - attachment 1
      Lost activity monitor - attachment 2
      1.8.W spectra - 20us FSR - attachments 3 & 4
      ADC data items - attachment 5
      FEE64 temperatures OK - attachment 6
      DSSSD bias & leakage currents OK - attachment 7
      Merger/Tape Server/Merger statistics - attachment 8
       no merger errors reported since previous restart

15.05 analysis of file S496/R32_698 - attachment 9
      max dead time 0.05% aida04

15.33 aida15 rebooted - see https://elog.ph.ed.ac.uk/DESPEC/365

      successful DAQ stop
      aida15 DAQ reset, setup, go 
      successful DAQ go 
  690   Sat Jan 11 14:05:03 2025 TDSaturday 11 January 2024
14.37 Detector bias & leakage current OK - attachment 1
  
      FEE64 temperatures OK - attachment 2

      WR timestamps - attachment 3
       aida06 out of sync

      global clock status - attachment 4
       aida06 fails

      ADC calibration - attachment 5
       all fail - probably due to ASIC synchronise *after* ADC calibration

      WR decoder & FPGA TS status - attachments 6-7 
       aida06 status 0xd

      DAQ run control - attachment 8
       aida06 undefined => aida06 has rebooted

      Merger & Tape Server - attachments 9-10
       merger, tape server and data transfer have stopped

      See https://elog.ph.ed.ac.uk/DESPEC/691
       aidao06 system console - aida06 rebooted sometime yesterday


*** Recovery

For FEE64 aida06 *only*

DAQ reset
DAQ setup


For all FEE64s

DAQ stop
Restart merger, merger setup, merger go 
 Following two merger restart cycles all FEE64s stopped OK
Recovery complete

       All system wide checks OK

       Disable data transfer, DAQ go

       per FEE64 rate spectra = attachment 11
        aida04 high rates

       per FEE64 1.8.W spectra - 20us FSR - attachments 12-13

       DAQ stop, enable data transfer, DAQ go

       Merger and server consoles - attachment 14
        merger and tape server (no storage mode) running OK but no data transfer to MBS server

       WR timestamps OK - attachment 15

       ADC and DISC data item stats - attachment 16-17
        aida04 & aida16 high rates - all others appear to be OK
        ASIC check load for all aida16 ASICs changes rate from c. 420k to 0 
        ASIC check load for all aida04 ASICs changes rate from c. 200k to c. 120k

18.00 No data transfer to MBS - nothing useful being donme at the moment - will place system in safe state

      DAQ stop
      Detector bias OFF
      FEE64 power OFF
  214   Sat Apr 10 12:15:54 2021 OH, TDSaturday 10 April
12:37 DAQ found crashed. Had crashed at some time following 7:50am following previous statistics update
      Unable to recover the FEEs so forced to power cycle
      Following power cycle statistics much improved from yesterday.
      In the waveforms the lower frequency component that was prevalent yesterday is no longer visible.
      The waveforms are not perfect by any means though
      Also note that following an ASIC synchronisation all of the FEEs lose ADC calibration and must be manually recalibrated.

16.42 UTC+1

      per FEE64 1.8.W spectra

      20us FSR attachments 7 & 8
      200us FSR attachments 9 & 10
      2ms FSR attachments 11 & 12
      20ms FSR attachments 15 & 16


      rate spectra attachment 13
      good events statistics attachment 14
  634   Sat Jun 1 12:48:54 2024 TDSaturday 1 June
13.26 DSSSD bias & leakage current - attachments 1-3

      FEE64 temps OK - attachment 4
       aida02 asic sensor u/s

      All system wide checks OK *except* WR decoder status - attachment 5

      ADC data item stats - attachment 6

      per FEE64 Rate spectra - attachments 7-8

      per FEE64 1.8.W spectra - 20us FSR - attachments 9-10

      WR timestamps OK - attachment 11

13.59 Data file S181/R1
      alpha background
      BNC PB-5 disconnected?
      slow comparator 0x64

      ADC data item stats - attachment 12
      Merger, TapeServer etc - attachments 13-14

14.02 All histograms & stats zero'd



16.19 DSSSD bias & leakage current - attachment 15

      FEE64 temps OK - attachment 16

      ADC data item stats - attachment 17

      per FEE64 Stat spectra - attachment 18


19.16 DSSSD bias & leakage current - attachments 19-20

      FEE64 temps OK - attachment 21

      ADC data item stats - attachment 22

      per FEE64 Stat spectra - attachment 23

      2x Check ASIC Control


21.42 DSSSD bias & leakage current - attachment 24

      FEE64 temps OK - attachment 25

      ADC data item stats - attachment 26

      per FEE64 Stat spectra - attachment 27

00.48 DSSSD bias & leakage current - attachment 28

      FEE64 temps OK - attachment 29

      ADC data item stats - attachment 30

      per FEE64 Stat spectra - attachment 31
  233   Sat Apr 17 07:28:02 2021 LPG, TDSaturday 08:00 - 12:00
08:30 CEST

HV and leakage currents: elog:233/1
Detector rates: elog:233/2
Temperatures: elog:233/3

WR status:


FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida10 failed
Calibration test result: Passed 9, Failed 3

If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module



FPGA status:


             Base         Current         Difference
aida12 fault      0x0 :      0x1 :      1  
FPGA Timestamp error counter test result: Passed 11, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last





09:20 CEST

Stats: ok!
DB: No faults found
ucesb: ok!


09:40 CEST

Beam will be stopped in order to increase intensity. Expected to be around 3-4 hours.
For now, we still get implants when Beam spill is on. Seems to be fluctuating as they play around.

10:00 CEST

Still getting beam, it is fluctuating in intensity

Stats: ok!
DB: No faults found
ucesb: ok!

HV and leakage currents: elog:233/4
Detector rates: elog:233/5
Temperatures: elog:233/6

Clock check: ok!
ADC calibration:
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida10 failed
Calibration test result: Passed 9, Failed 3
WR check: ok!
FPGA check:
Base Current Difference
aida12 fault 0x0 : 0x1 : 1
FPGA Timestamp error counter test result: Passed 11, Failed 1
Memory check: ok!


10:40 CEST

Stats: ok!
DB: No faults found
ucesb: ok!

11:00 CEST

Stats: ok!
DB: No faults found
ucesb: ok!


11:30 CEST

Stats: ok!
DB: No faults found
ucesb: ok!

12:00 BST

Stats: ok!
DB: No faults found
ucesb: ok!

HV and leakage currents: elog:233/7
Detector rates: elog:233/8
Temperatures: elog:233/9


Clock check: ok!
ADC calibration:
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida10 failed
Calibration test result: Passed 9, Failed 3
WR check:
Base Current Difference
aida05 fault 0x1a52 : 0x1a53 : 1
aida06 fault 0x4f3e : 0x4f3f : 1
aida07 fault 0x3bcd : 0x3bce : 1
aida08 fault 0xc7c7 : 0xc7c8 : 1
White Rabbit error counter test result: Passed 8, Failed 4
FPGA check:
Base Current Difference
aida09 fault 0x0 : 0x1 : 1
aida12 fault 0x0 : 0x2 : 2
Memory check: ok!
  237   Sun Apr 18 00:57:36 2021 BA, MASanday 18 April 00.00-08.00

02:01 Beam has stopped at 01:37 and returend at 01:43 for few min and then stopped again and not knowen how it will take until it is back

AIDA scalers attached 1

statistic attached 2

temretuer attached 3

bias attached 4

Clock check ok

ADC check :

FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida10 failed
Calibration test result: Passed 9, Failed 3

If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module


         Base         Current     Difference
aida05 fault      0x1a52 :      0x1a55 :      3  
aida06 fault      0x4f3e :      0x4f41 :      3  
aida07 fault      0x3bcd :      0x3bea :      29  
aida08 fault      0xc7c7 :      0xc7ca :      3  
White Rabbit error counter test result: Passed 8, Failed 4

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

 

   
             Base         Current         Difference
aida09 fault      0x0 :      0x1 :      1  
aida12 fault      0x0 :      0x9 :      9  
FPGA Timestamp error counter test result: Passed 10, Failed 2
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last

 

   
Returned 0 0 0 0 0 0 0 0 0 0 0 0  
Mem(KB) :    4    8    16    32    64    128    256    512    1k    2k    4k
aida01 :     32      8      3      1      0      3      1      3      3      3      6   : 36240
aida02 :     11      7      9      3      1      2      2      4      2      3      6   : 35988
aida03 :     32      4     13      5      5      4      3      3      2      3      6   : 36432
aida04 :     26      7      6      1      2      3      2      4      2      3      6   : 36128
aida05 :     18      8      8      7      4      2      2      2      2      4      6   : 37352
aida06 :     24     11      3      1      2      5      1      3      3      3      6   : 36616
aida07 :     14      5      3      3      3      2      3      2      4      3      6   : 37296
aida08 :     21      7      1      5      0      1      2      3      3      3      6   : 36284
aida09 :      0      7      1      1      1      3      2      2      3      3      6   : 35880
aida10 :     22      5      3      4      1      2      2      2      3      3      6   : 35952
aida11 :      4      3      1      1      2      2      3      3      2      3      6   : 35544
aida12 :     22     10      5      4      4      4      1      3      3      3      6   : 36728

 

02:19  beam is back

03:58 The beam has not been stable yet

           The rate reach 1.5 kHz, they will contact FRS team to lower the intensity of the beam

 

AIDA scalers attached 8

statistic attached 7

temretuer attached 6

bias attached 5

 

   
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida10 failed
Calibration test result: Passed 9, Failed 3

 

 

 

If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module

 

   
         Base         Current     Difference
aida05 fault      0x1a52 :      0x1a55 :      3  
aida06 fault      0x4f3e :      0x4f41 :      3  
aida07 fault      0x3bcd :      0x3beb :      30  
aida08 fault      0xc7c7 :      0xc7ca :      3  
White Rabbit error counter test result: Passed 8, Failed 4

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

 

   
             Base         Current         Difference
aida09 fault      0x0 :      0x1 :      1  
aida12 fault      0x0 :      0x9 :      9  
FPGA Timestamp error counter test result: Passed 10, Failed 2
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last

   
Returned 0 0 0 0 0 0 0 0 0 0 0 0  
Mem(KB) :    4    8    16    32    64    128    256    512    1k    2k    4k
aida01 :     21      8      6      2      1      2      1      3      3      3      6   : 36212
aida02 :     24     14     14      2      1      2      2      4      2      3      6   : 36144
aida03 :     31      5     10      5      5      4      2      3      2      3      6   : 36132
aida04 :     12     11     12      2      2      2      3      4      2      3      6   : 36360
aida05 :     23      8      5      7      4      2      2      2      2      4      6   : 37324
aida06 :     21     14     14      1      3      5      1      3      3      3      6   : 36868
aida07 :     15      9      5      0      3      2      3      2      4      3      6   : 37268
aida08 :     21     11     10      3      0      1      2      3      3      3      6   : 36396
aida09 :      5      7      5      1      1      3      1      3      3      3      6   : 36220
aida10 :     15     13     12      3      2      1      2      2      3      3      6   : 36036
aida11 :     13     10      1      0      2      2      2      4      2      3      6   : 35860
aida12 :     29      5      4      5      5      3      1      3      3      3      6   : 36668

 

05: 07  The rate was a bout 1500 and 2000, we contacted Oscar he said (if it's just bursts it should be ok), so they decided to do nothing.

 

06:26 

AIDA scalers attached 9

statistic attached 10

temretuer attached 11

bias attached 12

 

Clock status test result: Passed 12, Failed 0

Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable

 

FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida10 failed
Calibration test result: Passed 9, Failed 3

If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module

 

         Base         Current     Difference
aida01 fault      0x7685 :      0x7686 :      1  
aida02 fault      0x941c :      0x941d :      1  
aida03 fault      0x7cd6 :      0x7cd7 :      1  
aida04 fault      0xb86c :      0xb86d :      1  
aida05 fault      0x1a52 :      0x1a59 :      7  
aida06 fault      0x4f3e :      0x4f45 :      7  
aida07 fault      0x3bcd :      0x3bf9 :      44  
aida08 fault      0xc7c7 :      0xc7ce :      7  
aida09 fault      0xb33a :      0xb33b :      1  
White Rabbit error counter test result: Passed 3, Failed 9

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

 

             Base         Current         Difference
aida09 fault      0x0 :      0x1 :      1  
aida12 fault      0x0 :      0xa :      10  
FPGA Timestamp error counter test result: Passed 10, Failed 2
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last

 

Returned 0 0 0 0 0 0 0 0 0 0 0 0  
Mem(KB) :    4    8    16    32    64    128    256    512    1k    2k    4k
aida01 :     17     12      6      2      1      2      1      3      3      3      6   : 36228
aida02 :     19     11     12      3      2      2      2      4      2      3      6   : 36164
aida03 :     25      8     12      5      4      4      3      3      2      3      6   : 36356
aida04 :     22     19     11      3      1      2      3      4      2      3      6   : 36416
aida05 :     35      6      5      6      3      2      2      2      3      3      6   : 36236
aida06 :     12     11     11      2      3      4      1      3      3      3      6   : 36664
aida07 :     18      6      2      1      3      2      3      2      3      3      6   : 36216
aida08 :     27     10      8      3      0      1      2      3      3      3      6   : 36380
aida09 :     18      6      6      1      0      2      2      2      3      3      6   : 35832
aida10 :      3     14     10      3      1      1      2      3      3      3      6   : 36412
aida11 :      1      3      2      0      2      2      2      4      2      3      6   : 35772
aida12 :      0      5      6      3      3      4      1      3      3      3      6   : 36520

 

07:57  The beam stopped and they said there is a water leak !

 

 

 

 

 

  565   Mon Apr 8 11:18:19 2024 TDS505 offline analysis data file R3_150
DRG quotes ( from Elog ) S505 FEE64 configuration as

"Configuration: Single/Narrow - 8 FEEs, 2 DSSDs

DSSD 1:
 X 0-63    =  3 (HV -ve)
 X 64-127  =  1
 Y 0-63    =  4 (HV 0v) 
 Y 64-127  =  2 
DSSD 2:
 X 0-63    =  7 (HV -ve)
 X 64-127  =  5
 Y 0-63    =  8 (HV 0v)
 Y 64-127  =  6 


Scalers:
1 - Pulser
2 - n/c
3 - Time Machine Original
4 - Time Machine Delayed
5 - n/c
6 - n/c
7 - SC41 L
8 - SC41 R"

LEC (20MeV FSR) spectra

attachment 1 m_p versus m_n DSSSD#1 & DSSSD#2
attachment 2 E_p versus E_n DSSSD#1 & DSSSD#2

HEC (20GeV FSR) spectra

attachment 3 m_p versus m_n DSSSD#1 & DSSSD#2
attachment 4 E_p versus E_n DSSSD#1 & DSSSD#2

attachment variables.dat ( ADC offsets, e_diff windows & FEE64 onfiguration ) NAMELIST I/O format
  670   Tue Nov 5 15:19:35 2024 TDS505 offline analysis R5_780 - R5_814
Offline analysis of data files S505/R5_780 - R5_814 (corresponding to MBS data files 73-74) 

Can find an analysis of alpha background runs at RIBF, RIKEN for comparison at https://elog.ph.ed.ac.uk/AIDA/816

Beam off - background runs without sources

MBS 73 27.6.22 09:33-11:33 CEST https://elog.gsi.de/despec/S505/204

MBS 74 27.6.22 11:34-13:34 CEST https://elog.gsi.de/despec/S505/206

MBS 75 27.6.22 13:35-16:29 CEST https://elog.gsi.de/despec/S505/209

ADC offsets per https://elog.ph.ed.ac.uk/DESPEC/556

FEE64 configuration

FEE64   a b c d
DSSSD#1 3 4 1 2
DSSSD#2 7 8 5 6

p+n junction FE64s odd numbered


Data analysis assumes

- all LEC ADC data channels with valid ADC offset included (474 of 512 channels)
- no clustering
- no p+n junction side - n+n Ohmic side correlation time gates
- valid LEC events
   0 < p+n junction side multiplicity < 8 
   and
   0 < n+n Ohmic side multiplicity < 8 


Attachments 1-2 - per DSSSD p+n versus n+n multiplicity

Attachment 3 - per DSSSD x versus y

Attachments 4-5 - per DSSSD p+n versus n+n energy (20keV/channel nominal)
                  all combinations of per DSSSD p+n junction and n+n Ohmic energies 
                  with projection of data within window onto x and y axes

                  too many events for natural (U decay series) background
                  off leading diagonal correlations anomalous
                  transverse width of leading diagonal correlation wider than expected - ADC offsets OK?
                 

Attachment 6 - per FEE64 WR timestamp (32.768us/channel)
               FEE64 sync test using pulser data - looks OK

Attachments 7-14 - per FEE64 ADC spectra (5.6keV/channel nominal)
                   note common x/y scale - pulser peak height proxy for peak width

per FEE64 1.8.L pulser peak widths (ch FWHM)

1 11.43
2 16.58
3 12.52
4 17.54
5 9.10
6 12.22
7 17.03
8 16.09

3 of 4 p+n junction FEE64 good (<70keV), 1 of 4 n+n Ohmic FEE64 good - all others < 100keV
Pulser peak indicates noise/gain/offset stable throughout background runs


Attachments 15-22 - per FEE64 ADC spectra (5.6kleV/channel nominal)
                    observe broad peak-like structures (c. 2.8MeV) in first channel of most ASICs? 

Attachment 23 per DSSSD p+n versus n+n energy (20keV/channel nominal)
              valid LEC events
              p+n junction side multiplicity = 1 n+n Ohmic side multiplicity = 1

Attachment 24 p+n FEE64 #0 energy versus each of n+n FEE64s (#1, #3, #5 and #7) energies (20keV/channel nominal)
              valid LEC events
              p+n junction side multiplicity = 1 n+n Ohmic side multiplicity = 1

Attachments 23 & 24 do not unambiguously identify which FEE64s are attached to which DSSSD
                  
  556   Tue Apr 2 18:37:21 2024 TDS505 ADC offsets
S505 ADC offsets using pulser walkthrough data from data file R1

ch = channel + ( module * 64 ) + ( range * 2048 )

adc_data( ch ) = INT( RSHIFT( ABS( adc_data( ch ) - 32768 ), 3 ) - offset( ch ) + 0.5 )
ELOG V3.1.3-7933898