AIDA GELINA BRIKEN nToF CRIB ISOLDE CIRCE nTOFCapture DESPEC DTAS EDI_PSA 179Ta CARME StellarModelling DCF K40
  DESPEC, Page 14 of 36  ELOG logo
  ID Date Author Subjectdown
  453   Sat May 14 19:39:13 2022 BA, AASaturday 14 May

FEE64 module aida09 global clocks failed, 6
 Clock status test result: Passed 13, Failed 1

Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable

 

FEE64 module aida02 failed
FEE64 module aida06 failed
FEE64 module aida09 failed
FEE64 module aida10 failed
FEE64 module aida13 failed
Calibration test result: Passed 9, Failed 5

If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module

 

White Rabbit error counter test result: Passed 14, Failed 0

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

 

FPGA Timestamp error counter test result: Passed 14, Failed 0
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last

Attachment 1: Stat_2022-05-14_20-36-15.png
Stat_2022-05-14_20-36-15.png
Attachment 2: Temp_2022-05-14_20-35-19.png
Temp_2022-05-14_20-35-19.png
Attachment 3: Leakage_2022-05-14_20-33-32.png
Leakage_2022-05-14_20-33-32.png
  454   Sat May 14 21:34:11 2022 BA, AASaturday 14 May

FEE64 module aida09 global clocks failed, 6
 Clock status test result: Passed 13, Failed 1

Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable

 

FEE64 module aida02 failed
FEE64 module aida06 failed
FEE64 module aida09 failed
FEE64 module aida10 failed
FEE64 module aida13 failed
Calibration test result: Passed 9, Failed 5

If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module

 

White Rabbit error counter test result: Passed 14, Failed 0

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

 

FPGA Timestamp error counter test result: Passed 14, Failed 0
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last

Attachment 1: Rates_2022-05-14_22-32-44.png
Rates_2022-05-14_22-32-44.png
Attachment 2: Temp_2022-05-14_22-31-37.png
Temp_2022-05-14_22-31-37.png
Attachment 3: Leakg_2022-05-14_22-30-28.png
Leakg_2022-05-14_22-30-28.png
  705   Sat Jun 14 11:20:45 2025 TDSaturday 14 June 2025
12.21 BNC PB-5 settings
      Amplitude 1.0V
      Attenuator x1
      Frequency 22Hz
      Polarity +
      Tau_d 1ms
      Tail pulse
      Clamp ON
      Pulser output connected to p+n test daisy chain - n+n test daisy chain not connected 

      per DSSSD green/yellow ground daisy chain was removed yesterday

      FEE64 adaptor PCB grounds connected by heavy duty braid wrapped in copper foil to AIDA support assembly frame
      Snout connected to AIDA support assembly frame by heavy duty braid bolted to frame and connected to snout by adhesive copper foil.

      DSSSD - FEE64 adaptor PCB cabling (cannot connect some ribbon cables because cable flipped and connector key now incorrectly oriented)

                    aida10 aida14 aida13
                      ox     xo     oo
                    aida09 aida01 aida05
                      oo     oo     xo
      aida06 aida02                      aida04 aida08    *Beam into screen*
        ox     oo                          oo     oo
                    aida15 aida03 aida12
                      oo     oo     ox
                    aida11 aida07 aida16 
                      ox     oo     xo

      o = connected
      x = not connected
      e.g. J1 J2 connected = oo, J1 only connected = ox

      Adaptor PCB top/component side
      E     J1
      R
      N
      I     J2

      Adaptor PCB 
       LK1 fitted aida02, aida04, aida06, aida08
       LK3 fitted aida03, aida07

      DSSSD bias & leakage current - attachment 1
      FEE64 temps OK - attachment 2
      All system wide checks OK *except* WR decoder status aida02 - attachment 3
      WR timestamps OK - attachment 4
      ADC data item stats - attachment 5

      per FEE64 Rate spectra - attachments 6-7
       note ASICs with per channel rates of 22Hz (pulser)
       note *all* FEE64 slow comparator 0x64 ( *note* subsequently found ASIC #2-4 of a number of FEE64s were set to 0x0 )

      per p+n FEE64 1.8.W spectra - 20us, 200us, 2ms and 20ms FSR - attachments 8-11
      per p+n FEE64 1.8.W spectra - 20ms FSR - expanded y-scale = 2000 x 122uV/LSB = 0.244V FSR - attachment 12

      per n+n FEE64 1.8.W spectra - 20ms, 2ms, 200us and 20us FSR - attachments 13-16

      per p+n FEE64 1.8.L spectra - attachment 17
       pulser peak widths
        aida05, aida14, aida16 c. 15-20 ch FWHM ( c. 10-15keV FWHM ) - cable from DSSSD not connected to adaptor PCB
        aida09 c. 120 ch FWHM - electronic noise c. 2x worse than configuration used for 2023/4 tests
        all other FEE64s *very* noisy/unstable


14.45 Found some FEE64s with slow comparator 0x64 for ASIC#1 and 0x0 for ASICs #2-4
      Slow comparator 0x64 now set for all ASICs of all FEE64s

      ASIC settings saved to 2025Jun14-15.00.09

      ADC data item stats - attachment 18

      per FEE64 Rate spectra - attachments 19-20
       note ASICs with per channel rates of 22Hz (pulser)
       note *all* FEE64 slow comparator 0x64
       note aida10 to be connected to MSL type BB7 - ignore for time being

      per FEE64 1.8.W spectra - 20us FSR - attachments 21-22

      per p+n FEE64 1.8.L spectra - attachments 23-24
       pulser peak widths
        aida05, aida14, aida16 c. 20 ch FWHM - cable from DSSSD not connected to adaptor PCB
        aida09 c. 120 ch FWHM - electronic noise c. 2x worse than configuration used for 2023/4 tests
Attachment 1: Screenshot_from_2025-06-14_12-20-58.png
Screenshot_from_2025-06-14_12-20-58.png
Attachment 2: Screenshot_from_2025-06-14_12-21-03.png
Screenshot_from_2025-06-14_12-21-03.png
Attachment 3: Screenshot_from_2025-06-14_12-22-20.png
Screenshot_from_2025-06-14_12-22-20.png
Attachment 4: Screenshot_from_2025-06-14_12-22-49.png
Screenshot_from_2025-06-14_12-22-49.png
Attachment 5: Screenshot_from_2025-06-14_12-23-31.png
Screenshot_from_2025-06-14_12-23-31.png
Attachment 6: Screenshot_from_2025-06-14_12-23-39.png
Screenshot_from_2025-06-14_12-23-39.png
Attachment 7: Screenshot_from_2025-06-14_12-23-49.png
Screenshot_from_2025-06-14_12-23-49.png
Attachment 8: Screenshot_from_2025-06-14_12-34-18.png
Screenshot_from_2025-06-14_12-34-18.png
Attachment 9: Screenshot_from_2025-06-14_12-35-16.png
Screenshot_from_2025-06-14_12-35-16.png
Attachment 10: Screenshot_from_2025-06-14_12-35-53.png
Screenshot_from_2025-06-14_12-35-53.png
Attachment 11: Screenshot_from_2025-06-14_12-36-22.png
Screenshot_from_2025-06-14_12-36-22.png
Attachment 12: Screenshot_from_2025-06-14_12-36-48.png
Screenshot_from_2025-06-14_12-36-48.png
Attachment 13: Screenshot_from_2025-06-14_12-41-57.png
Screenshot_from_2025-06-14_12-41-57.png
Attachment 14: Screenshot_from_2025-06-14_12-42-22.png
Screenshot_from_2025-06-14_12-42-22.png
Attachment 15: Screenshot_from_2025-06-14_12-42-45.png
Screenshot_from_2025-06-14_12-42-45.png
Attachment 16: Screenshot_from_2025-06-14_12-43-05.png
Screenshot_from_2025-06-14_12-43-05.png
Attachment 17: Screenshot_from_2025-06-14_12-47-43.png
Screenshot_from_2025-06-14_12-47-43.png
Attachment 18: Screenshot_from_2025-06-14_14-46-16.png
Screenshot_from_2025-06-14_14-46-16.png
Attachment 19: Screenshot_from_2025-06-14_14-46-24.png
Screenshot_from_2025-06-14_14-46-24.png
Attachment 20: Screenshot_from_2025-06-14_14-46-42.png
Screenshot_from_2025-06-14_14-46-42.png
Attachment 21: Screenshot_from_2025-06-14_14-47-05.png
Screenshot_from_2025-06-14_14-47-05.png
Attachment 22: Screenshot_from_2025-06-14_14-47-24.png
Screenshot_from_2025-06-14_14-47-24.png
Attachment 23: Screenshot_from_2025-06-14_14-55-39.png
Screenshot_from_2025-06-14_14-55-39.png
Attachment 24: Screenshot_from_2025-06-14_14-55-57.png
Screenshot_from_2025-06-14_14-55-57.png
Attachment 25: 20250614_130606.jpg
20250614_130606.jpg
Attachment 26: 20250614_132851.jpg
20250614_132851.jpg
Attachment 27: 20250614_132846.jpg
20250614_132846.jpg
Attachment 28: 20250614_132812.jpg
20250614_132812.jpg
Attachment 29: 20250614_130553.jpg
20250614_130553.jpg
Attachment 30: 20250614_132820.jpg
20250614_132820.jpg
  676   Sat Dec 14 11:52:42 2024 TDSaturday 14 December
11:00 Visual inspection FEE64 adaptor PCBs & cabling
      aida02 ground from/to other FEE64s disconnected - re-connected
      aida01, aida09, aida12 - ground cabling screws to Lemo 00.250 housings loose - tightemed

12:50 Cooling water pressure & temperature OK - attachments 1  2

12:59 relay #1 power ON

13:01 relay #2 power ON

13:07 aida06 starts - panic during startup, automatic restart following 3 minute timeout

      DAQ reset, setup

      Check ASIC Control - browser tab timeout 
      AIDA MIDAS HTTPD server console log - attachment 3

      Appears to have restored ASIC settings 2024Dec13-17.02.45 saved yesterday
    
      aida10 ASICs #1 & #2 positive input, ASICs #3 & #4 negative input
      slow comparator 0xa (all p+n junction FEE64s and aida10), 0xf (n+n Ohmic FEE64s)

13:28 tar ASIC settings - attachment 4

[npg@aidas-gsi]$ cd /MIDAS/FEE_ASIC
[npg@aidas-gsi FEE_ASIC]$ tar cvf /tmp/FEE_ASIC.tar .

       System wide checks

       Sync ASIC clocks - attachment 5

       Clock, ADC calibration, WR decoder, FPGA timestamp, PLL checks - attachments 6-11
        all OK *except* aida02 WR decoder error

       WR timestamps OK - attachment 12
       FEE64 temps OK - attachment 13
       
13:45 Detector bias ON - attachment 14      

      BNC PB-5 pulser settings - attachment 15
      Pulser connected to all p+n junction FEE64s *except* aida10

      ADC, DISC, PAUSE and MBS correlation scaler stats - attachments 16-19
       aida02 rate significantly lower than yesterday - https://elog.gsi.de/despec/Implantation+Stack/8
       high rates observed for aida08, aida11 and aida14 - which are not connected to a DSSSD!

      per FEE64 Rate spectra - attachment 20

      per p+n junction FEE64 1.8.L spectra - attachment 21
       aida09 pulser peak width 56 ch FWHM = 39 keV FWHM
       consistent electronic noise for all p+n junction FEE64s (cabling+DSSSD)
       electronic noise of p+n junction FEE64s (cabling *only*) higher and more variable cf. https://elog.gsi.de/despec/Implantation+Stack/8 attachment 5

      per p+n junction FEE64 1.8.W spectra 20us FSR - attachments 22-23

      per n+n Ohmic FEE64 1.8.W spectra 20us FSR - attachment 24 

      WR timestamps OK - attachment 25

14:43 DAQ STOP
      Data transfer enabled
      Select Tape Server -> Next Run
      DAQ GO  data file R3

      Merger, Tape Server - attachments 26-27
       data transfer rate c. 900k data items/s cf c. 300k data items/s yesterday https://elog.gsi.de/despec/Implantation+Stack/8

16:45 DAQ STOP

      Data transfer disabled 

      Detector bias OFF

      FEE64 power OFF
Attachment 1: 20241214_125048.jpg
20241214_125048.jpg
Attachment 2: 20241214_125104.jpg
20241214_125104.jpg
Attachment 3: asic_check.log
executing do_PreReset
 ::MASTERTS namespace already exists
done executing do_PreReset
executing do_PostSetup
done executing do_PostSetup 
ASIC.tcl :- CheckASIC fetched file /MIDAS/FEE_ASIC => aida01
checking asic 1
Loaded ASIC 1 at 0x0000401c for FEE aida01
Second load re-activated at 0x0000401c  for FEE aida01 
Shifting out finished at 0 counts 
Read the control register from FEE aida01 ASIC 1
checking asic 2
Loaded ASIC 2 at 0x0000405c for FEE aida01
Second load re-activated at 0x0000405c  for FEE aida01 
Shifting out finished at 0 counts 
Read the control register from FEE aida01 ASIC 2
checking asic 3
Loaded ASIC 3 at 0x0000409c for FEE aida01
Second load re-activated at 0x0000409c  for FEE aida01 
Shifting out finished at 0 counts 
Read the control register from FEE aida01 ASIC 3
checking asic 4
Loaded ASIC 4 at 0x000040dc for FEE aida01
Second load re-activated at 0x000040dc  for FEE aida01 
Shifting out finished at 0 counts 
Read the control register from FEE aida01 ASIC 4
ASIC.tcl :- CheckASIC fetched file /MIDAS/FEE_ASIC => aida02
checking asic 1
Loaded ASIC 1 at 0x0000401c for FEE aida02
Second load re-activated at 0x0000401c  for FEE aida02 
Shifting out finished at 0 counts 
Read the control register from FEE aida02 ASIC 1
checking asic 2
Loaded ASIC 2 at 0x0000405c for FEE aida02
Second load re-activated at 0x0000405c  for FEE aida02 
Shifting out finished at 0 counts 
Read the control register from FEE aida02 ASIC 2
checking asic 3
Loaded ASIC 3 at 0x0000409c for FEE aida02
Second load re-activated at 0x0000409c  for FEE aida02 
Shifting out finished at 0 counts 
Read the control register from FEE aida02 ASIC 3
checking asic 4
Loaded ASIC 4 at 0x000040dc for FEE aida02
Second load re-activated at 0x000040dc  for FEE aida02 
Shifting out finished at 0 counts 
Read the control register from FEE aida02 ASIC 4
ASIC.tcl :- CheckASIC fetched file /MIDAS/FEE_ASIC => aida03
checking asic 1
Loaded ASIC 1 at 0x0000401c for FEE aida03
Second load re-activated at 0x0000401c  for FEE aida03 
Shifting out finished at 0 counts 
Read the control register from FEE aida03 ASIC 1
checking asic 2
Loaded ASIC 2 at 0x0000405c for FEE aida03
Second load re-activated at 0x0000405c  for FEE aida03 
Shifting out finished at 0 counts 
Read the control register from FEE aida03 ASIC 2
checking asic 3
Loaded ASIC 3 at 0x0000409c for FEE aida03
Second load re-activated at 0x0000409c  for FEE aida03 
Shifting out finished at 0 counts 
Read the control register from FEE aida03 ASIC 3
checking asic 4
Loaded ASIC 4 at 0x000040dc for FEE aida03
Second load re-activated at 0x000040dc  for FEE aida03 
Shifting out finished at 0 counts 
Read the control register from FEE aida03 ASIC 4
ASIC.tcl :- CheckASIC fetched file /MIDAS/FEE_ASIC => aida04
checking asic 1
Loaded ASIC 1 at 0x0000401c for FEE aida04
Second load re-activated at 0x0000401c  for FEE aida04 
Shifting out finished at 0 counts 
Read the control register from FEE aida04 ASIC 1
checking asic 2
Loaded ASIC 2 at 0x0000405c for FEE aida04
Second load re-activated at 0x0000405c  for FEE aida04 
Shifting out finished at 0 counts 
Read the control register from FEE aida04 ASIC 2
checking asic 3
Loaded ASIC 3 at 0x0000409c for FEE aida04
Second load re-activated at 0x0000409c  for FEE aida04 
Shifting out finished at 0 counts 
Read the control register from FEE aida04 ASIC 3
checking asic 4
Loaded ASIC 4 at 0x000040dc for FEE aida04
Second load re-activated at 0x000040dc  for FEE aida04 
Shifting out finished at 0 counts 
Read the control register from FEE aida04 ASIC 4
ASIC.tcl :- CheckASIC fetched file /MIDAS/FEE_ASIC => aida05
checking asic 1
Loaded ASIC 1 at 0x0000401c for FEE aida05
Second load re-activated at 0x0000401c  for FEE aida05 
Shifting out finished at 0 counts 
Read the control register from FEE aida05 ASIC 1
checking asic 2
Loaded ASIC 2 at 0x0000405c for FEE aida05
Second load re-activated at 0x0000405c  for FEE aida05 
Shifting out finished at 0 counts 
Read the control register from FEE aida05 ASIC 2
checking asic 3
Loaded ASIC 3 at 0x0000409c for FEE aida05
Second load re-activated at 0x0000409c  for FEE aida05 
Shifting out finished at 0 counts 
Read the control register from FEE aida05 ASIC 3
checking asic 4
Loaded ASIC 4 at 0x000040dc for FEE aida05
Second load re-activated at 0x000040dc  for FEE aida05 
Shifting out finished at 0 counts 
Read the control register from FEE aida05 ASIC 4
ASIC.tcl :- CheckASIC fetched file /MIDAS/FEE_ASIC => aida06
checking asic 1
Loaded ASIC 1 at 0x0000401c for FEE aida06
Second load re-activated at 0x0000401c  for FEE aida06 
Shifting out finished at 0 counts 
Read the control register from FEE aida06 ASIC 1
checking asic 2
Loaded ASIC 2 at 0x0000405c for FEE aida06
Second load re-activated at 0x0000405c  for FEE aida06 
Shifting out finished at 0 counts 
Read the control register from FEE aida06 ASIC 2
checking asic 3
Loaded ASIC 3 at 0x0000409c for FEE aida06
Second load re-activated at 0x0000409c  for FEE aida06 
Shifting out finished at 0 counts 
Read the control register from FEE aida06 ASIC 3
checking asic 4
Loaded ASIC 4 at 0x000040dc for FEE aida06
Second load re-activated at 0x000040dc  for FEE aida06 
Shifting out finished at 0 counts 
Read the control register from FEE aida06 ASIC 4
ASIC.tcl :- CheckASIC fetched file /MIDAS/FEE_ASIC => aida07
checking asic 1
Loaded ASIC 1 at 0x0000401c for FEE aida07
Second load re-activated at 0x0000401c  for FEE aida07 
Shifting out finished at 0 counts 
Read the control register from FEE aida07 ASIC 1
checking asic 2
Loaded ASIC 2 at 0x0000405c for FEE aida07
Second load re-activated at 0x0000405c  for FEE aida07 
Shifting out finished at 0 counts 
Read the control register from FEE aida07 ASIC 2
checking asic 3
Loaded ASIC 3 at 0x0000409c for FEE aida07
Second load re-activated at 0x0000409c  for FEE aida07 
Shifting out finished at 0 counts 
Read the control register from FEE aida07 ASIC 3
checking asic 4
Loaded ASIC 4 at 0x000040dc for FEE aida07
Second load re-activated at 0x000040dc  for FEE aida07 
Shifting out finished at 0 counts 
Read the control register from FEE aida07 ASIC 4
ASIC.tcl :- CheckASIC fetched file /MIDAS/FEE_ASIC => aida08
checking asic 1
Loaded ASIC 1 at 0x0000401c for FEE aida08
Second load re-activated at 0x0000401c  for FEE aida08 
Shifting out finished at 0 counts 
Read the control register from FEE aida08 ASIC 1
checking asic 2
Loaded ASIC 2 at 0x0000405c for FEE aida08
Second load re-activated at 0x0000405c  for FEE aida08 
Shifting out finished at 0 counts 
Read the control register from FEE aida08 ASIC 2
checking asic 3
Loaded ASIC 3 at 0x0000409c for FEE aida08
Second load re-activated at 0x0000409c  for FEE aida08 
Shifting out finished at 0 counts 
Read the control register from FEE aida08 ASIC 3
checking asic 4
Loaded ASIC 4 at 0x000040dc for FEE aida08
Second load re-activated at 0x000040dc  for FEE aida08 
Shifting out finished at 0 counts 
Read the control register from FEE aida08 ASIC 4
ASIC.tcl :- CheckASIC fetched file /MIDAS/FEE_ASIC => aida09
checking asic 1
Loaded ASIC 1 at 0x0000401c for FEE aida09
Second load re-activated at 0x0000401c  for FEE aida09 
Shifting out finished at 0 counts 
Read the control register from FEE aida09 ASIC 1
checking asic 2
Loaded ASIC 2 at 0x0000405c for FEE aida09
Second load re-activated at 0x0000405c  for FEE aida09 
Shifting out finished at 0 counts 
Read the control register from FEE aida09 ASIC 2
checking asic 3
Loaded ASIC 3 at 0x0000409c for FEE aida09
Second load re-activated at 0x0000409c  for FEE aida09 
Shifting out finished at 0 counts 
Read the control register from FEE aida09 ASIC 3
checking asic 4
Loaded ASIC 4 at 0x000040dc for FEE aida09
Second load re-activated at 0x000040dc  for FEE aida09 
Shifting out finished at 0 counts 
Read the control register from FEE aida09 ASIC 4
ASIC.tcl :- CheckASIC fetched file /MIDAS/FEE_ASIC => aida10
checking asic 1
Loaded ASIC 1 at 0x0000401c for FEE aida10
Second load re-activated at 0x0000401c  for FEE aida10 
Shifting out finished at 0 counts 
Read the control register from FEE aida10 ASIC 1
checking asic 2
Loaded ASIC 2 at 0x0000405c for FEE aida10
Second load re-activated at 0x0000405c  for FEE aida10 
Shifting out finished at 0 counts 
Read the control register from FEE aida10 ASIC 2
checking asic 3
Loaded ASIC 3 at 0x0000409c for FEE aida10
Second load re-activated at 0x0000409c  for FEE aida10 
Shifting out finished at 0 counts 
Read the control register from FEE aida10 ASIC 3
checking asic 4
Loaded ASIC 4 at 0x000040dc for FEE aida10
Second load re-activated at 0x000040dc  for FEE aida10 
Shifting out finished at 0 counts 
Read the control register from FEE aida10 ASIC 4
ASIC.tcl :- CheckASIC fetched file /MIDAS/FEE_ASIC => aida11
checking asic 1
Loaded ASIC 1 at 0x0000401c for FEE aida11
Second load re-activated at 0x0000401c  for FEE aida11 
Shifting out finished at 0 counts 
Read the control register from FEE aida11 ASIC 1
checking asic 2
Loaded ASIC 2 at 0x0000405c for FEE aida11
Second load re-activated at 0x0000405c  for FEE aida11 
Shifting out finished at 0 counts 
Read the control register from FEE aida11 ASIC 2
checking asic 3
Loaded ASIC 3 at 0x0000409c for FEE aida11
Second load re-activated at 0x0000409c  for FEE aida11 
Shifting out finished at 0 counts 
Read the control register from FEE aida11 ASIC 3
checking asic 4
Loaded ASIC 4 at 0x000040dc for FEE aida11
Second load re-activated at 0x000040dc  for FEE aida11 
Shifting out finished at 0 counts 
Read the control register from FEE aida11 ASIC 4
ASIC.tcl :- CheckASIC fetched file /MIDAS/FEE_ASIC => aida12
checking asic 1
Loaded ASIC 1 at 0x0000401c for FEE aida12
Second load re-activated at 0x0000401c  for FEE aida12 
Shifting out finished at 0 counts 
Read the control register from FEE aida12 ASIC 1
checking asic 2
Loaded ASIC 2 at 0x0000405c for FEE aida12
Second load re-activated at 0x0000405c  for FEE aida12 
Shifting out finished at 0 counts 
Read the control register from FEE aida12 ASIC 2
checking asic 3
Loaded ASIC 3 at 0x0000409c for FEE aida12
Second load re-activated at 0x0000409c  for FEE aida12 
Shifting out finished at 0 counts 
Read the control register from FEE aida12 ASIC 3
checking asic 4
Loaded ASIC 4 at 0x000040dc for FEE aida12
Second load re-activated at 0x000040dc  for FEE aida12 
Shifting out finished at 0 counts 
Read the control register from FEE aida12 ASIC 4
ASIC.tcl :- CheckASIC fetched file /MIDAS/FEE_ASIC => aida13
checking asic 1
Loaded ASIC 1 at 0x0000401c for FEE aida13
Second load re-activated at 0x0000401c  for FEE aida13 
Shifting out finished at 0 counts 
Read the control register from FEE aida13 ASIC 1
checking asic 2
Loaded ASIC 2 at 0x0000405c for FEE aida13
Second load re-activated at 0x0000405c  for FEE aida13 
Shifting out finished at 0 counts 
Read the control register from FEE aida13 ASIC 2
checking asic 3
Loaded ASIC 3 at 0x0000409c for FEE aida13
Second load re-activated at 0x0000409c  for FEE aida13 
Shifting out finished at 0 counts 
Read the control register from FEE aida13 ASIC 3
checking asic 4
Loaded ASIC 4 at 0x000040dc for FEE aida13
Second load re-activated at 0x000040dc  for FEE aida13 
Shifting out finished at 0 counts 
Read the control register from FEE aida13 ASIC 4
ASIC.tcl :- CheckASIC fetched file /MIDAS/FEE_ASIC => aida14
checking asic 1
Loaded ASIC 1 at 0x0000401c for FEE aida14
Second load re-activated at 0x0000401c  for FEE aida14 
Shifting out finished at 0 counts 
Read the control register from FEE aida14 ASIC 1
checking asic 2
Loaded ASIC 2 at 0x0000405c for FEE aida14
Second load re-activated at 0x0000405c  for FEE aida14 
Shifting out finished at 0 counts 
Read the control register from FEE aida14 ASIC 2
checking asic 3
Loaded ASIC 3 at 0x0000409c for FEE aida14
Second load re-activated at 0x0000409c  for FEE aida14 
Shifting out finished at 0 counts 
Read the control register from FEE aida14 ASIC 3
checking asic 4
Loaded ASIC 4 at 0x000040dc for FEE aida14
Second load re-activated at 0x000040dc  for FEE aida14 
Shifting out finished at 0 counts 
Read the control register from FEE aida14 ASIC 4
ASIC.tcl :- CheckASIC fetched file /MIDAS/FEE_ASIC => aida15
... 379 more lines ...
Attachment 4: FEE_ASIC.tar
Attachment 5: Screenshot_from_2024-12-14_13-30-48.png
Screenshot_from_2024-12-14_13-30-48.png
Attachment 6: Screenshot_from_2024-12-14_13-30-54.png
Screenshot_from_2024-12-14_13-30-54.png
Attachment 7: Screenshot_from_2024-12-14_13-31-02.png
Screenshot_from_2024-12-14_13-31-02.png
Attachment 8: Screenshot_from_2024-12-14_13-35-22.png
Screenshot_from_2024-12-14_13-35-22.png
Attachment 9: Screenshot_from_2024-12-14_13-35-28.png
Screenshot_from_2024-12-14_13-35-28.png
Attachment 10: Screenshot_from_2024-12-14_13-35-33.png
Screenshot_from_2024-12-14_13-35-33.png
Attachment 11: Screenshot_from_2024-12-14_13-36-06.png
Screenshot_from_2024-12-14_13-36-06.png
Attachment 12: Screenshot_from_2024-12-14_13-36-27.png
Screenshot_from_2024-12-14_13-36-27.png
Attachment 13: Screenshot_from_2024-12-14_13-36-42.png
Screenshot_from_2024-12-14_13-36-42.png
Attachment 14: Screenshot_from_2024-12-14_13-44-29.png
Screenshot_from_2024-12-14_13-44-29.png
Attachment 15: Screenshot_from_2024-12-14_13-46-41.png
Screenshot_from_2024-12-14_13-46-41.png
Attachment 16: Screenshot_from_2024-12-14_13-54-58.png
Screenshot_from_2024-12-14_13-54-58.png
Attachment 17: Screenshot_from_2024-12-14_13-55-19.png
Screenshot_from_2024-12-14_13-55-19.png
Attachment 18: Screenshot_from_2024-12-14_13-55-33.png
Screenshot_from_2024-12-14_13-55-33.png
Attachment 19: Screenshot_from_2024-12-14_13-56-00.png
Screenshot_from_2024-12-14_13-56-00.png
Attachment 20: Screenshot_from_2024-12-14_14-10-08.png
Screenshot_from_2024-12-14_14-10-08.png
Attachment 21: Screenshot_from_2024-12-14_14-14-18.png
Screenshot_from_2024-12-14_14-14-18.png
Attachment 22: Screenshot_from_2024-12-14_14-26-30.png
Screenshot_from_2024-12-14_14-26-30.png
Attachment 23: Screenshot_from_2024-12-14_14-27-16.png
Screenshot_from_2024-12-14_14-27-16.png
Attachment 24: Screenshot_from_2024-12-14_14-29-24.png
Screenshot_from_2024-12-14_14-29-24.png
Attachment 25: Screenshot_from_2024-12-14_14-42-40.png
Screenshot_from_2024-12-14_14-42-40.png
Attachment 26: Screenshot_from_2024-12-14_14-43-51.png
Screenshot_from_2024-12-14_14-43-51.png
Attachment 27: Screenshot_from_2024-12-14_14-43-58.png
Screenshot_from_2024-12-14_14-43-58.png
  196   Sat Mar 13 17:00:49 2021 TDSaturday 13 March 18.00-00.00 UTC+1
18.00 DAQ continues OK - file R46_64

      ASIC settings 2019Dec19-16.19.51
      DSSSD#1 slow comparator 0xa
      DSSSD#2 slow comparator 0xa
      DSSSD#3 slow comparator 0xd

      BNC PB-5 Pulser 
      Amplitude1.0V
      Attenuation x1
      Frequency 2Hz
      tau_d 1ms
      - polarity
      Delay 250ns, tail pulse

18.02 System wide checks OK *except*

FEE64 module aida09 global clocks failed, 6
 Clock status test result: Passed 11, Failed 1

Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1' 
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable

FEE64 module aida09 failed
Calibration test result: Passed 11, Failed 1
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module

      FEE64 Temperatures OK - attachment 1
      Good event statistics OK - attachment 2
      Detector bias & leakage currents OK - attachment 3
      Merger OK - 4.6M data items/s
      TapeServer OK - 16Mb/s

      All histograms zero'd

17.15 Most recent messages in merger server terminal session

MERGE Data Link (2322): bad timestamp  6 3 0xc1b57e35 0x0ba1685c 0x0000ef3fbba1685c 0x166bef3fbba1685c 0x166bef3fbba1879c
MERGE Data Link (2322): bad timestamp  6 3 0xc1a77dc2 0x0ba1702c 0x0000ef3fbba1702c 0x166bef3fbba1702c 0x166bef3fbba1879c
MERGE Data Link (2322): bad timestamp  6 3 0xc1bf7f66 0x0ba1702c 0x0000ef3fbba1702c 0x166bef3fbba1702c 0x166bef3fbba1879c
Transfer Error - : Broken pipe
Error 32
1: send() failed:
TCP transfer library version 4.0
1: TCP socket send buffer was 16384 - now 249856
1: TCP socket receive buffer was 87380 - now 249856
1: TCP socket created OK - now connecting to localhost port 10305
1: Connected to localhost port 10305


22.05 DAQ continues OK - file R46_168

FEE64 module aida09 global clocks failed, 6
 Clock status test result: Passed 11, Failed 1

Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1' 
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable

FEE64 module aida09 failed
Calibration test result: Passed 11, Failed 1
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module

		 Base 		Current 	Difference
aida07 fault 	 0x82a0 : 	 0x82a2 : 	 2  
White Rabbit error counter test result: Passed 11, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

			 Base 		Current 		Difference
aida07 fault 	 0x2 : 	 0x3 : 	 1  
FPGA Timestamp error counter test result: Passed 11, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last

	
Returned 0 0 0 0 0 0 0 0 0 0 0 0  
Mem(KB) :	4	8	16	32	64	128	256	512	1k	2k	4k
aida01 :     18      6      7      5      1      3      1      3      3      4     10   : 54856
aida02 :     19      6      4      3      4      4      2      3      3      3      6   : 36892
aida03 :     22      8      1      2      1      2      1      3      3      3      6   : 36136
aida04 :     24      8      4      2      1      3      2      4      1      4     15   : 73952
aida05 :     37     14      9      8      7      3      2     18      7      4      7   : 55252
aida06 :     28     12     16      4      4      2      1      4      3      2      5   : 31056
aida07 :     28      9      2      2      2      2      1      3      3      3      6   : 36248
aida08 :     21     12      6      4      3      4      2      3      3      3      6   : 36948
aida09 :     18      5      2      1      1      3      2      2      3      3      6   : 35952
aida10 :     17      8      4      5      3      3      1      3      3      3      6   : 36516
aida11 :     21      8      5      0      0      4      3      3      2      3      6   : 35812
aida12 :     13     11      3      3    


      FEE64 Temperatures OK - attachment 4
      Good event statistics OK - attachment 5
      Detector bias & leakage currents OK - attachment 6
      Merger OK - 4.4M data items/s
      TapeServer OK - 15Mb/s

22.17 Rate spectra - attachments 7 & 8
      p+n junction HEC spectra - attachment 9

      Merger server error messages since 17.15

MERGE Data Link (2316): bad timestamp  0 3 0x80300000 0x0164e7f8 0x000000000164e7f8 0x166200000164e7f8 0x166bffffffa7654e
MERGE Data Link (2324): bad timestamp  8 3 0x88300000 0x0534e25c 0x000000000534e25c 0x166200000534e25c 0x166bffffed8383fc
MERGE Data Link (2322): bad timestamp  6 3 0xc1a07f16 0x0070e9cc 0x000001e74070e9cc 0x166c01e74070e9cc 0x166c01e74071090c
MERGE Data Link (2322): bad timestamp  6 3 0xc1b37895 0x0070e9cc 0x000001e74070e9cc 0x166c01e74070e9cc 0x166c01e74071090c
MERGE Data Link (2322): bad timestamp  6 3 0xc19f7e41 0x0070f19c 0x000001e74070f19c 0x166c01e74070f19c 0x166c01e74071090c
MERGE Data Link (2322): bad timestamp  6 3 0xc1a27e22 0x0070f19c 0x000001e74070f19c 0x166c01e74070f19c 0x166c01e74071090c
MERGE Data Link (2322): bad timestamp  6 3 0xc1b47399 0x0070f19c 0x000001e74070f19c 0x166c01e74070f19c 0x166c01e74071090c
MERGE Data Link (2322): bad timestamp  6 3 0xc1ab7dad 0x0070f96c 0x000001e74070f96c 0x166c01e74070f96c 0x166c01e74071090c
MERGE Data Link (2322): bad timestamp  6 3 0xc1b564f9 0x0070f96c 0x000001e74070f96c 0x166c01e74070f96c 0x166c01e74071090c
MERGE Data Link (2322): bad timestamp  6 3 0xc1b60177 0x0071013c 0x000001e74071013c 0x166c01e74071013c 0x166c01e74071090c
MERGE Data Link (2322): bad timestamp  6 3 0xc1a57e95 0x00e2208c 0x000001e700e2208c 0x166c01e700e2208c 0x166c01e74071090c
MERGE Data Link (2322): bad timestamp  6 3 0xc1b81491 0x00e2208c 0x000001e700e2208c 0x166c01e700e2208c 0x166c01e74071090c
MERGE Data Link (2322): bad timestamp  6 3 0xc1ac7e4f 0x00e2285c 0x000001e700e2285c 0x166c01e700e2285c 0x166c01e74071090c
MERGE Data Link (2322): bad timestamp  6 3 0xc1b90000 0x00e2285c 0x000001e700e2285c 0x166c01e700e2285c 0x166c01e74071090c
MERGE Data Link (2322): bad timestamp  6 3 0xc1ba0c64 0x00e2302c 0x000001e700e2302c 0x166c01e700e2302c 0x166c01e74071090c
MERGE Data Link (2322): bad timestamp  6 3 0xc1bb6ea8 0x00e237fc 0x000001e700e237fc 0x166c01e700e237fc 0x166c01e74071090c
MERGE Data Link (2322): bad timestamp  6 3 0xc1a87dc6 0x00e23fcc 0x000001e700e23fcc 0x166c01e700e23fcc 0x166c01e74071090c
MERGE Data Link (2322): bad timestamp  6 3 0xc1bc0641 0x00e23fcc 0x000001e700e23fcc 0x166c01e700e23fcc 0x166c01e74071090c
Attachment 1: Screenshot_2021-03-13_Temperature_and_status_scan_aidas-gsi.png
Screenshot_2021-03-13_Temperature_and_status_scan_aidas-gsi.png
Attachment 2: Screenshot_2021-03-13_Statistics_aidas-gsi.png
Screenshot_2021-03-13_Statistics_aidas-gsi.png
Attachment 3: 32.png
32.png
Attachment 4: Screenshot_2021-03-13_Temperature_and_status_scan_aidas-gsi(1).png
Screenshot_2021-03-13_Temperature_and_status_scan_aidas-gsi(1).png
Attachment 5: Screenshot_2021-03-13_Statistics_aidas-gsi(1).png
Screenshot_2021-03-13_Statistics_aidas-gsi(1).png
Attachment 6: 33.png
33.png
Attachment 7: Screenshot_2021-03-13_Spectrum_Browser_aidas-gsi.png
Screenshot_2021-03-13_Spectrum_Browser_aidas-gsi.png
Attachment 8: Screenshot_2021-03-13_Spectrum_Browser_aidas-gsi(1).png
Screenshot_2021-03-13_Spectrum_Browser_aidas-gsi(1).png
Attachment 9: Screenshot_2021-03-13_Spectrum_Browser_aidas-gsi(2).png
Screenshot_2021-03-13_Spectrum_Browser_aidas-gsi(2).png
  365   Sat Jun 12 13:56:00 2021 TDSaturday 12 June
14.11 DAQ continues file S496/R32_697
      High data rate from c. 06.00 this morning
      ASIC check 

      System wide checks OK *except*

		 Base 		Current 	Difference
aida01 fault 	 0x8944 : 	 0x8948 : 	 4  
aida02 fault 	 0x9b36 : 	 0x9b3a : 	 4  
aida03 fault 	 0xf26b : 	 0xf26f : 	 4  
aida04 fault 	 0x57a0 : 	 0x57a4 : 	 4  
aida05 fault 	 0x4d3c : 	 0x4d3e : 	 2  
aida05 : WR status 0x10
 aida06 fault 	 0x640c : 	 0x640e : 	 2  
aida07 fault 	 0x255e : 	 0x2560 : 	 2  
aida08 fault 	 0xffbd : 	 0xffbf : 	 2  
White Rabbit error counter test result: Passed 8, Failed 8

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

			 Base 		Current 		Difference
aida12 fault 	 0x0 : 	 0x21be : 	 8638  
aida13 fault 	 0x0 : 	 0x2294 : 	 8852  
FPGA Timestamp error counter test result: Passed 14, Failed 2
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last

Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0  
Mem(KB) :	4	8	16	32	64	128	256	512	1k	2k	4k
aida01 :     33     10      9      2      5      4      2      2      2      4      6   : 37604
aida02 :     46     12      7      2      4      4      3      2      2      4      6   : 37832
aida03 :     34     10      9      5      4      3      1      3      3      3      6   : 36744
aida04 :     35     12     10      1      3      4      2      3      3      3      6   : 36972
aida05 :     37      9      8      3      4      4      2      2      2      4      6   : 37564
aida06 :     34     10      5      5      4      2      2      3      2      4      6   : 37832
aida07 :     31     11      1      7      4      2      3      2      2      4      6   : 37572
aida08 :     40      7      4      5      3      4      3      2      2      4      6   : 37752
aida09 :     39      9      8      4      1      3      3      2      2      4      6   : 37540
aida10 :     43     12      8      4      4      3      3      3      1      4      6   : 37260
aida11 :     34     14     10      5      3      4      3      3      1      4      6   : 37368
aida12 :     46     12      3      4      4      4      3      3      1      4      6   : 37320
aida13 :     41     11      5      4      5      3      3      3      1      4      6   : 37260
aida14 :     39     10      2      6      4      4      3      3      1      4      6   : 37324
aida15 :     19     12      4      3      2      2      3      2      2      4      6   : 37324
aida16 :     45     10      6      4      5      3      3      3      1      4      6   : 37284

Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same

 FEE : aida01 =>   Options file size is 1026 	 Last changed Fri Jun 04 09:30:42 CEST 2021
 FEE : aida02 =>   Options file size is 1025 	 Last changed Sun May 23 00:19:21 CEST 2021
 FEE : aida03 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:43:50 CEST 2021
 FEE : aida04 =>   Options file size is 1025 	 Last changed Fri May 14 16:54:56 CEST 2021
 FEE : aida05 =>   Options file size is 1025 	 Last changed Mon May 17 06:25:41 CEST 2021
 FEE : aida06 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:43:59 CEST 2021
 FEE : aida07 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:02 CEST 2021
 FEE : aida08 =>   Options file size is 1025 	 Last changed Sun May 23 00:16:54 CEST 2021
 FEE : aida09 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:08 CEST 2021
 FEE : aida10 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida11 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida12 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida13 =>   Options file size is 1025 	 Last changed Fri May 07 19:40:34 CEST 2021
 FEE : aida14 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida15 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida16 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021

      Grafana - DSSSD bias & leakage current - most recent 7 days - attachment 1
      Lost activity monitor - attachment 2
      1.8.W spectra - 20us FSR - attachments 3 & 4
      ADC data items - attachment 5
      FEE64 temperatures OK - attachment 6
      DSSSD bias & leakage currents OK - attachment 7
      Merger/Tape Server/Merger statistics - attachment 8
       no merger errors reported since previous restart

15.05 analysis of file S496/R32_698 - attachment 9
      max dead time 0.05% aida04

15.33 aida15 rebooted - see https://elog.ph.ed.ac.uk/DESPEC/365

      successful DAQ stop
      aida15 DAQ reset, setup, go 
      successful DAQ go 
Attachment 1: Screenshot_2021-06-12_AIDA_Alerting_-_Grafana.png
Screenshot_2021-06-12_AIDA_Alerting_-_Grafana.png
Attachment 2: Screenshot_2021-06-12_Lost_activity_monitor_aidas-gsi.png
Screenshot_2021-06-12_Lost_activity_monitor_aidas-gsi.png
Attachment 3: Screenshot_2021-06-12_Spectrum_Browser_aidas-gsi(1).png
Screenshot_2021-06-12_Spectrum_Browser_aidas-gsi(1).png
Attachment 4: Screenshot_2021-06-12_Spectrum_Browser_aidas-gsi.png
Screenshot_2021-06-12_Spectrum_Browser_aidas-gsi.png
Attachment 5: Screenshot_2021-06-12_Statistics_aidas-gsi.png
Screenshot_2021-06-12_Statistics_aidas-gsi.png
Attachment 6: Screenshot_2021-06-12_Temperature_and_status_scan_aidas-gsi.png
Screenshot_2021-06-12_Temperature_and_status_scan_aidas-gsi.png
Attachment 7: 998.png
998.png
Attachment 8: 999.png
999.png
Attachment 9: R32_698
 *** TDR format 3.3.0 analyser - TD - May 2021
 *** ERROR: READ I/O error:       5002
                   blocks:      32000
          ADC data format:   44970784 (   15685.0 Hz)
        Other data format:  216949222 (   75668.1 Hz)
 Sample trace data format:          0 (       0.0 Hz)
         Undefined format:          0 (       0.0 Hz)
   Other data format type:      PAUSE:         62 (       0.0 Hz)
                               RESUME:         62 (       0.0 Hz)
                              SYNC100:      42680 (      14.9 Hz)
                              WR48-63:      42680 (      14.9 Hz)
                           FEE64 disc:     985985 (     343.9 Hz)
                             MBS info:  215877753 (   75294.4 Hz)
                           Other info:          0 (       0.0 Hz)

   ADC data range bit set:          0 (       0.0 Hz)

                Timewarps:        ADC:          0 (       0.0 Hz)
                                PAUSE:          0 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:          0 (       0.0 Hz)
                              WR48-63:          0 (       0.0 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:          0 (       0.0 Hz)
                            Undefined:          0 (       0.0 Hz)
                         Sample trace:          0 (       0.0 Hz)

 *** Timestamp elapsed time:     2867.115 s
 FEE  elapsed dead time(s) elapsed idle time(s)
  0                0.000                0.000
  1                0.000             2299.418
  2                0.673                0.000
  3                1.368                0.000
  4                0.000                0.000
  5                0.000             2495.101
  6                0.000                0.000
  7                0.000             2566.695
  8                0.000                0.000
  9                0.000                0.000
 10                0.000                0.000
 11                0.000                0.000
 12                0.000                0.000
 13                0.000                0.000
 14                0.000                0.000
 15                0.000                0.000
 16                0.000                0.000
 17                0.000                0.000
 18                0.000                0.000
 19                0.000                0.000
 20                0.000                0.000
 21                0.000                0.000
 22                0.000                0.000
 23                0.000                0.000
 24                0.000                0.000
 25                0.000                0.000
 26                0.000                0.000
 27                0.000                0.000
 28                0.000                0.000
 29                0.000                0.000
 30                0.000                0.000
 31                0.000                0.000
 32                0.000                0.000

 *** Statistics
 FEE  ADC Data Other Data     Sample  Undefined      Pause     Resume    SYNC100    WR48-63       Disc        MBS      Other   HEC Data
  0        467          2          0          0          0          0          1          1          0          0          0          0
  1      72420         30          0          0          0          0         15         15          0          0          0          0
  2        935  107990737          0          0         33         33      19385      19385          0  107951901          0          0
  3   38882631  108955999          0          0         29         29      22052      22052     985985  107925852          0          0
  4        496          0          0          0          0          0          0          0          0          0          0          0
  5    3224207       1336          0          0          0          0        668        668          0          0          0          0
  6       1520          0          0          0          0          0          0          0          0          0          0          0
  7    2784837       1118          0          0          0          0        559        559          0          0          0          0
  8        473          0          0          0          0          0          0          0          0          0          0          0
  9        173          0          0          0          0          0          0          0          0          0          0          0
 10        554          0          0          0          0          0          0          0          0          0          0          0
 11        317          0          0          0          0          0          0          0          0          0          0          0
 12        752          0          0          0          0          0          0          0          0          0          0          0
 13        204          0          0          0          0          0          0          0          0          0          0          0
 14        302          0          0          0          0          0          0          0          0          0          0          0
 15        496          0          0          0          0          0          0          0          0          0          0          0
 16          0          0          0          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0          0          0          0

 *** Timewarps
 FEE       ADC      Pause     Resume    SYNC100    WR48-63       Disc        MBS  Undefined    Samples
  0          0          0          0          0          0          0          0          0          0
  1          0          0          0          0          0          0          0          0          0
  2          0          0          0          0          0          0          0          0          0
  3          0          0          0          0          0          0          0          0          0
  4          0          0          0          0          0          0          0          0          0
  5          0          0          0          0          0          0          0          0          0
  6          0          0          0          0          0          0          0          0          0
  7          0          0          0          0          0          0          0          0          0
  8          0          0          0          0          0          0          0          0          0
  9          0          0          0          0          0          0          0          0          0
 10          0          0          0          0          0          0          0          0          0
 11          0          0          0          0          0          0          0          0          0
 12          0          0          0          0          0          0          0          0          0
 13          0          0          0          0          0          0          0          0          0
 14          0          0          0          0          0          0          0          0          0
 15          0          0          0          0          0          0          0          0          0
 16          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0

 *** Program elapsed time:   29.945s ( 1068.615 blocks/s,  66.788 Mb/s)
  690   Sat Jan 11 14:05:03 2025 TDSaturday 11 January 2024
14.37 Detector bias & leakage current OK - attachment 1
  
      FEE64 temperatures OK - attachment 2

      WR timestamps - attachment 3
       aida06 out of sync

      global clock status - attachment 4
       aida06 fails

      ADC calibration - attachment 5
       all fail - probably due to ASIC synchronise *after* ADC calibration

      WR decoder & FPGA TS status - attachments 6-7 
       aida06 status 0xd

      DAQ run control - attachment 8
       aida06 undefined => aida06 has rebooted

      Merger & Tape Server - attachments 9-10
       merger, tape server and data transfer have stopped

      See https://elog.ph.ed.ac.uk/DESPEC/691
       aidao06 system console - aida06 rebooted sometime yesterday


*** Recovery

For FEE64 aida06 *only*

DAQ reset
DAQ setup


For all FEE64s

DAQ stop
Restart merger, merger setup, merger go 
 Following two merger restart cycles all FEE64s stopped OK
Recovery complete

       All system wide checks OK

       Disable data transfer, DAQ go

       per FEE64 rate spectra = attachment 11
        aida04 high rates

       per FEE64 1.8.W spectra - 20us FSR - attachments 12-13

       DAQ stop, enable data transfer, DAQ go

       Merger and server consoles - attachment 14
        merger and tape server (no storage mode) running OK but no data transfer to MBS server

       WR timestamps OK - attachment 15

       ADC and DISC data item stats - attachment 16-17
        aida04 & aida16 high rates - all others appear to be OK
        ASIC check load for all aida16 ASICs changes rate from c. 420k to 0 
        ASIC check load for all aida04 ASICs changes rate from c. 200k to c. 120k

18.00 No data transfer to MBS - nothing useful being donme at the moment - will place system in safe state

      DAQ stop
      Detector bias OFF
      FEE64 power OFF
Attachment 1: Screenshot_from_2025-01-11_14-37-29.png
Screenshot_from_2025-01-11_14-37-29.png
Attachment 2: Screenshot_from_2025-01-11_14-55-21.png
Screenshot_from_2025-01-11_14-55-21.png
Attachment 3: Screenshot_from_2025-01-11_14-56-44.png
Screenshot_from_2025-01-11_14-56-44.png
Attachment 4: Screenshot_from_2025-01-11_14-57-14.png
Screenshot_from_2025-01-11_14-57-14.png
Attachment 5: Screenshot_from_2025-01-11_14-57-37.png
Screenshot_from_2025-01-11_14-57-37.png
Attachment 6: Screenshot_from_2025-01-11_14-58-05.png
Screenshot_from_2025-01-11_14-58-05.png
Attachment 7: Screenshot_from_2025-01-11_14-58-25.png
Screenshot_from_2025-01-11_14-58-25.png
Attachment 8: Screenshot_from_2025-01-11_14-59-24.png
Screenshot_from_2025-01-11_14-59-24.png
Attachment 9: Screenshot_from_2025-01-11_15-00-00.png
Screenshot_from_2025-01-11_15-00-00.png
Attachment 10: Screenshot_from_2025-01-11_15-00-31.png
Screenshot_from_2025-01-11_15-00-31.png
Attachment 11: Screenshot_from_2025-01-11_15-29-32.png
Screenshot_from_2025-01-11_15-29-32.png
Attachment 12: Screenshot_from_2025-01-11_15-30-35.png
Screenshot_from_2025-01-11_15-30-35.png
Attachment 13: Screenshot_from_2025-01-11_15-31-29.png
Screenshot_from_2025-01-11_15-31-29.png
Attachment 14: Screenshot_from_2025-01-11_15-35-14.png
Screenshot_from_2025-01-11_15-35-14.png
Attachment 15: Screenshot_from_2025-01-11_15-38-24.png
Screenshot_from_2025-01-11_15-38-24.png
Attachment 16: Screenshot_from_2025-01-11_15-27-28.png
Screenshot_from_2025-01-11_15-27-28.png
Attachment 17: Screenshot_from_2025-01-11_15-26-39.png
Screenshot_from_2025-01-11_15-26-39.png
  214   Sat Apr 10 12:15:54 2021 OH, TDSaturday 10 April
12:37 DAQ found crashed. Had crashed at some time following 7:50am following previous statistics update
      Unable to recover the FEEs so forced to power cycle
      Following power cycle statistics much improved from yesterday.
      In the waveforms the lower frequency component that was prevalent yesterday is no longer visible.
      The waveforms are not perfect by any means though
      Also note that following an ASIC synchronisation all of the FEEs lose ADC calibration and must be manually recalibrated.

16.42 UTC+1

      per FEE64 1.8.W spectra

      20us FSR attachments 7 & 8
      200us FSR attachments 9 & 10
      2ms FSR attachments 11 & 12
      20ms FSR attachments 15 & 16


      rate spectra attachment 13
      good events statistics attachment 14
Attachment 1: 210410_1237_Aida_crash.png
210410_1237_Aida_crash.png
Attachment 2: 210410_1242_Crash.png
210410_1242_Crash.png
Attachment 3: 210410_1251_aida06_drop.png
210410_1251_aida06_drop.png
Attachment 4: 210410_1312_Waveform1.png
210410_1312_Waveform1.png
Attachment 5: 210304_1313_Waveform2.png
210304_1313_Waveform2.png
Attachment 6: 210410_1318_Stats.png
210410_1318_Stats.png
Attachment 7: Screenshot_2021-04-10_Spectrum_Browser_aidas-gsi.png
Screenshot_2021-04-10_Spectrum_Browser_aidas-gsi.png
Attachment 8: Screenshot_2021-04-10_Spectrum_Browser_aidas-gsi(5).png
Screenshot_2021-04-10_Spectrum_Browser_aidas-gsi(5).png
Attachment 9: Screenshot_2021-04-10_Spectrum_Browser_aidas-gsi(1).png
Screenshot_2021-04-10_Spectrum_Browser_aidas-gsi(1).png
Attachment 10: Screenshot_2021-04-10_Spectrum_Browser_aidas-gsi(4).png
Screenshot_2021-04-10_Spectrum_Browser_aidas-gsi(4).png
Attachment 11: Screenshot_2021-04-10_Spectrum_Browser_aidas-gsi(2).png
Screenshot_2021-04-10_Spectrum_Browser_aidas-gsi(2).png
Attachment 12: Screenshot_2021-04-10_Spectrum_Browser_aidas-gsi(3).png
Screenshot_2021-04-10_Spectrum_Browser_aidas-gsi(3).png
Attachment 13: Screenshot_2021-04-10_Spectrum_Browser_aidas-gsi(6).png
Screenshot_2021-04-10_Spectrum_Browser_aidas-gsi(6).png
Attachment 14: Screenshot_2021-04-10_Statistics_aidas-gsi.png
Screenshot_2021-04-10_Statistics_aidas-gsi.png
Attachment 15: Screenshot_2021-04-10_Spectrum_Browser_aidas-gsi(7).png
Screenshot_2021-04-10_Spectrum_Browser_aidas-gsi(7).png
Attachment 16: Screenshot_2021-04-10_Spectrum_Browser_aidas-gsi(8).png
Screenshot_2021-04-10_Spectrum_Browser_aidas-gsi(8).png
  634   Sat Jun 1 12:48:54 2024 TDSaturday 1 June
13.26 DSSSD bias & leakage current - attachments 1-3

      FEE64 temps OK - attachment 4
       aida02 asic sensor u/s

      All system wide checks OK *except* WR decoder status - attachment 5

      ADC data item stats - attachment 6

      per FEE64 Rate spectra - attachments 7-8

      per FEE64 1.8.W spectra - 20us FSR - attachments 9-10

      WR timestamps OK - attachment 11

13.59 Data file S181/R1
      alpha background
      BNC PB-5 disconnected?
      slow comparator 0x64

      ADC data item stats - attachment 12
      Merger, TapeServer etc - attachments 13-14

14.02 All histograms & stats zero'd



16.19 DSSSD bias & leakage current - attachment 15

      FEE64 temps OK - attachment 16

      ADC data item stats - attachment 17

      per FEE64 Stat spectra - attachment 18


19.16 DSSSD bias & leakage current - attachments 19-20

      FEE64 temps OK - attachment 21

      ADC data item stats - attachment 22

      per FEE64 Stat spectra - attachment 23

      2x Check ASIC Control


21.42 DSSSD bias & leakage current - attachment 24

      FEE64 temps OK - attachment 25

      ADC data item stats - attachment 26

      per FEE64 Stat spectra - attachment 27

00.48 DSSSD bias & leakage current - attachment 28

      FEE64 temps OK - attachment 29

      ADC data item stats - attachment 30

      per FEE64 Stat spectra - attachment 31
Attachment 1: Screenshot_from_2024-06-01_13-26-48.png
Screenshot_from_2024-06-01_13-26-48.png
Attachment 2: Screenshot_from_2024-06-01_13-27-24.png
Screenshot_from_2024-06-01_13-27-24.png
Attachment 3: Screenshot_from_2024-06-01_13-27-47.png
Screenshot_from_2024-06-01_13-27-47.png
Attachment 4: Screenshot_from_2024-06-01_13-40-22.png
Screenshot_from_2024-06-01_13-40-22.png
Attachment 5: Screenshot_from_2024-06-01_13-43-09.png
Screenshot_from_2024-06-01_13-43-09.png
Attachment 6: Screenshot_from_2024-06-01_13-44-41.png
Screenshot_from_2024-06-01_13-44-41.png
Attachment 7: Screenshot_from_2024-06-01_13-45-21.png
Screenshot_from_2024-06-01_13-45-21.png
Attachment 8: Screenshot_from_2024-06-01_13-45-49.png
Screenshot_from_2024-06-01_13-45-49.png
Attachment 9: Screenshot_from_2024-06-01_13-47-30.png
Screenshot_from_2024-06-01_13-47-30.png
Attachment 10: Screenshot_from_2024-06-01_13-48-20.png
Screenshot_from_2024-06-01_13-48-20.png
Attachment 11: Screenshot_from_2024-06-01_13-51-48.png
Screenshot_from_2024-06-01_13-51-48.png
Attachment 12: Screenshot_from_2024-06-01_13-53-14.png
Screenshot_from_2024-06-01_13-53-14.png
Attachment 13: Screenshot_from_2024-06-01_14-00-38.png
Screenshot_from_2024-06-01_14-00-38.png
Attachment 14: Screenshot_from_2024-06-01_14-01-08.png
Screenshot_from_2024-06-01_14-01-08.png
Attachment 15: Screenshot_from_2024-06-01_16-19-23.png
Screenshot_from_2024-06-01_16-19-23.png
Attachment 16: Screenshot_from_2024-06-01_16-27-28.png
Screenshot_from_2024-06-01_16-27-28.png
Attachment 17: Screenshot_from_2024-06-01_16-28-06.png
Screenshot_from_2024-06-01_16-28-06.png
Attachment 18: Screenshot_from_2024-06-01_16-29-07.png
Screenshot_from_2024-06-01_16-29-07.png
Attachment 19: Screenshot_from_2024-06-01_19-18-17.png
Screenshot_from_2024-06-01_19-18-17.png
Attachment 20: Screenshot_from_2024-06-01_19-15-57.png
Screenshot_from_2024-06-01_19-15-57.png
Attachment 21: Screenshot_from_2024-06-01_19-18-48.png
Screenshot_from_2024-06-01_19-18-48.png
Attachment 22: Screenshot_from_2024-06-01_19-19-11.png
Screenshot_from_2024-06-01_19-19-11.png
Attachment 23: Screenshot_from_2024-06-01_19-20-03.png
Screenshot_from_2024-06-01_19-20-03.png
Attachment 24: Screenshot_from_2024-06-01_21-41-48.png
Screenshot_from_2024-06-01_21-41-48.png
Attachment 25: Screenshot_from_2024-06-01_21-42-24.png
Screenshot_from_2024-06-01_21-42-24.png
Attachment 26: Screenshot_from_2024-06-01_21-42-49.png
Screenshot_from_2024-06-01_21-42-49.png
Attachment 27: Screenshot_from_2024-06-01_21-43-54.png
Screenshot_from_2024-06-01_21-43-54.png
Attachment 28: Screenshot_from_2024-06-02_00-48-44.png
Screenshot_from_2024-06-02_00-48-44.png
Attachment 29: Screenshot_from_2024-06-02_00-49-22.png
Screenshot_from_2024-06-02_00-49-22.png
Attachment 30: Screenshot_from_2024-06-02_00-49-48.png
Screenshot_from_2024-06-02_00-49-48.png
Attachment 31: Screenshot_from_2024-06-02_00-50-53.png
Screenshot_from_2024-06-02_00-50-53.png
  233   Sat Apr 17 07:28:02 2021 LPG, TDSaturday 08:00 - 12:00
08:30 CEST

HV and leakage currents: elog:233/1
Detector rates: elog:233/2
Temperatures: elog:233/3

WR status:


FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida10 failed
Calibration test result: Passed 9, Failed 3

If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module



FPGA status:


             Base         Current         Difference
aida12 fault      0x0 :      0x1 :      1  
FPGA Timestamp error counter test result: Passed 11, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last





09:20 CEST

Stats: ok!
DB: No faults found
ucesb: ok!


09:40 CEST

Beam will be stopped in order to increase intensity. Expected to be around 3-4 hours.
For now, we still get implants when Beam spill is on. Seems to be fluctuating as they play around.

10:00 CEST

Still getting beam, it is fluctuating in intensity

Stats: ok!
DB: No faults found
ucesb: ok!

HV and leakage currents: elog:233/4
Detector rates: elog:233/5
Temperatures: elog:233/6

Clock check: ok!
ADC calibration:
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida10 failed
Calibration test result: Passed 9, Failed 3
WR check: ok!
FPGA check:
Base Current Difference
aida12 fault 0x0 : 0x1 : 1
FPGA Timestamp error counter test result: Passed 11, Failed 1
Memory check: ok!


10:40 CEST

Stats: ok!
DB: No faults found
ucesb: ok!

11:00 CEST

Stats: ok!
DB: No faults found
ucesb: ok!


11:30 CEST

Stats: ok!
DB: No faults found
ucesb: ok!

12:00 BST

Stats: ok!
DB: No faults found
ucesb: ok!

HV and leakage currents: elog:233/7
Detector rates: elog:233/8
Temperatures: elog:233/9


Clock check: ok!
ADC calibration:
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida10 failed
Calibration test result: Passed 9, Failed 3
WR check:
Base Current Difference
aida05 fault 0x1a52 : 0x1a53 : 1
aida06 fault 0x4f3e : 0x4f3f : 1
aida07 fault 0x3bcd : 0x3bce : 1
aida08 fault 0xc7c7 : 0xc7c8 : 1
White Rabbit error counter test result: Passed 8, Failed 4
FPGA check:
Base Current Difference
aida09 fault 0x0 : 0x1 : 1
aida12 fault 0x0 : 0x2 : 2
Memory check: ok!
Attachment 1: 20.png
20.png
Attachment 2: Screenshot_2021-04-17_Statistics_aidas-gsi.png
Screenshot_2021-04-17_Statistics_aidas-gsi.png
Attachment 3: Screenshot_2021-04-17_Temperature_and_status_scan_aidas-gsi.png
Screenshot_2021-04-17_Temperature_and_status_scan_aidas-gsi.png
Attachment 4: 21.png
21.png
Attachment 5: Screenshot_2021-04-17_Statistics_aidas-gsi(1).png
Screenshot_2021-04-17_Statistics_aidas-gsi(1).png
Attachment 6: Screenshot_2021-04-17_Temperature_and_status_scan_aidas-gsi(1).png
Screenshot_2021-04-17_Temperature_and_status_scan_aidas-gsi(1).png
Attachment 7: 22.png
22.png
Attachment 8: Screenshot_2021-04-17_Statistics_aidas-gsi(2).png
Screenshot_2021-04-17_Statistics_aidas-gsi(2).png
Attachment 9: Screenshot_2021-04-17_Temperature_and_status_scan_aidas-gsi(2).png
Screenshot_2021-04-17_Temperature_and_status_scan_aidas-gsi(2).png
  237   Sun Apr 18 00:57:36 2021 BA, MASanday 18 April 00.00-08.00

02:01 Beam has stopped at 01:37 and returend at 01:43 for few min and then stopped again and not knowen how it will take until it is back

AIDA scalers attached 1

statistic attached 2

temretuer attached 3

bias attached 4

Clock check ok

ADC check :

FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida10 failed
Calibration test result: Passed 9, Failed 3

If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module


         Base         Current     Difference
aida05 fault      0x1a52 :      0x1a55 :      3  
aida06 fault      0x4f3e :      0x4f41 :      3  
aida07 fault      0x3bcd :      0x3bea :      29  
aida08 fault      0xc7c7 :      0xc7ca :      3  
White Rabbit error counter test result: Passed 8, Failed 4

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

 

   
             Base         Current         Difference
aida09 fault      0x0 :      0x1 :      1  
aida12 fault      0x0 :      0x9 :      9  
FPGA Timestamp error counter test result: Passed 10, Failed 2
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last

 

   
Returned 0 0 0 0 0 0 0 0 0 0 0 0  
Mem(KB) :    4    8    16    32    64    128    256    512    1k    2k    4k
aida01 :     32      8      3      1      0      3      1      3      3      3      6   : 36240
aida02 :     11      7      9      3      1      2      2      4      2      3      6   : 35988
aida03 :     32      4     13      5      5      4      3      3      2      3      6   : 36432
aida04 :     26      7      6      1      2      3      2      4      2      3      6   : 36128
aida05 :     18      8      8      7      4      2      2      2      2      4      6   : 37352
aida06 :     24     11      3      1      2      5      1      3      3      3      6   : 36616
aida07 :     14      5      3      3      3      2      3      2      4      3      6   : 37296
aida08 :     21      7      1      5      0      1      2      3      3      3      6   : 36284
aida09 :      0      7      1      1      1      3      2      2      3      3      6   : 35880
aida10 :     22      5      3      4      1      2      2      2      3      3      6   : 35952
aida11 :      4      3      1      1      2      2      3      3      2      3      6   : 35544
aida12 :     22     10      5      4      4      4      1      3      3      3      6   : 36728

 

02:19  beam is back

03:58 The beam has not been stable yet

           The rate reach 1.5 kHz, they will contact FRS team to lower the intensity of the beam

 

AIDA scalers attached 8

statistic attached 7

temretuer attached 6

bias attached 5

 

   
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida10 failed
Calibration test result: Passed 9, Failed 3

 

 

 

If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module

 

   
         Base         Current     Difference
aida05 fault      0x1a52 :      0x1a55 :      3  
aida06 fault      0x4f3e :      0x4f41 :      3  
aida07 fault      0x3bcd :      0x3beb :      30  
aida08 fault      0xc7c7 :      0xc7ca :      3  
White Rabbit error counter test result: Passed 8, Failed 4

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

 

   
             Base         Current         Difference
aida09 fault      0x0 :      0x1 :      1  
aida12 fault      0x0 :      0x9 :      9  
FPGA Timestamp error counter test result: Passed 10, Failed 2
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last

   
Returned 0 0 0 0 0 0 0 0 0 0 0 0  
Mem(KB) :    4    8    16    32    64    128    256    512    1k    2k    4k
aida01 :     21      8      6      2      1      2      1      3      3      3      6   : 36212
aida02 :     24     14     14      2      1      2      2      4      2      3      6   : 36144
aida03 :     31      5     10      5      5      4      2      3      2      3      6   : 36132
aida04 :     12     11     12      2      2      2      3      4      2      3      6   : 36360
aida05 :     23      8      5      7      4      2      2      2      2      4      6   : 37324
aida06 :     21     14     14      1      3      5      1      3      3      3      6   : 36868
aida07 :     15      9      5      0      3      2      3      2      4      3      6   : 37268
aida08 :     21     11     10      3      0      1      2      3      3      3      6   : 36396
aida09 :      5      7      5      1      1      3      1      3      3      3      6   : 36220
aida10 :     15     13     12      3      2      1      2      2      3      3      6   : 36036
aida11 :     13     10      1      0      2      2      2      4      2      3      6   : 35860
aida12 :     29      5      4      5      5      3      1      3      3      3      6   : 36668

 

05: 07  The rate was a bout 1500 and 2000, we contacted Oscar he said (if it's just bursts it should be ok), so they decided to do nothing.

 

06:26 

AIDA scalers attached 9

statistic attached 10

temretuer attached 11

bias attached 12

 

Clock status test result: Passed 12, Failed 0

Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable

 

FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida10 failed
Calibration test result: Passed 9, Failed 3

If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module

 

         Base         Current     Difference
aida01 fault      0x7685 :      0x7686 :      1  
aida02 fault      0x941c :      0x941d :      1  
aida03 fault      0x7cd6 :      0x7cd7 :      1  
aida04 fault      0xb86c :      0xb86d :      1  
aida05 fault      0x1a52 :      0x1a59 :      7  
aida06 fault      0x4f3e :      0x4f45 :      7  
aida07 fault      0x3bcd :      0x3bf9 :      44  
aida08 fault      0xc7c7 :      0xc7ce :      7  
aida09 fault      0xb33a :      0xb33b :      1  
White Rabbit error counter test result: Passed 3, Failed 9

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

 

             Base         Current         Difference
aida09 fault      0x0 :      0x1 :      1  
aida12 fault      0x0 :      0xa :      10  
FPGA Timestamp error counter test result: Passed 10, Failed 2
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last

 

Returned 0 0 0 0 0 0 0 0 0 0 0 0  
Mem(KB) :    4    8    16    32    64    128    256    512    1k    2k    4k
aida01 :     17     12      6      2      1      2      1      3      3      3      6   : 36228
aida02 :     19     11     12      3      2      2      2      4      2      3      6   : 36164
aida03 :     25      8     12      5      4      4      3      3      2      3      6   : 36356
aida04 :     22     19     11      3      1      2      3      4      2      3      6   : 36416
aida05 :     35      6      5      6      3      2      2      2      3      3      6   : 36236
aida06 :     12     11     11      2      3      4      1      3      3      3      6   : 36664
aida07 :     18      6      2      1      3      2      3      2      3      3      6   : 36216
aida08 :     27     10      8      3      0      1      2      3      3      3      6   : 36380
aida09 :     18      6      6      1      0      2      2      2      3      3      6   : 35832
aida10 :      3     14     10      3      1      1      2      3      3      3      6   : 36412
aida11 :      1      3      2      0      2      2      2      4      2      3      6   : 35772
aida12 :      0      5      6      3      3      4      1      3      3      3      6   : 36520

 

07:57  The beam stopped and they said there is a water leak !

 

 

 

 

 

Attachment 1: 20210418_0154.png
20210418_0154.png
Attachment 2: 20210418_0152_rate.png
20210418_0152_rate.png
Attachment 3: 20210418_0150_tem.png
20210418_0150_tem.png
Attachment 4: 20210418_0150_bias.png
20210418_0150_bias.png
Attachment 5: 20210418_0407.png
20210418_0407.png
Attachment 6: 20210418_0409.png
20210418_0409.png
Attachment 7: 20210418_0411.png
20210418_0411.png
Attachment 8: 20210418_0413.png
20210418_0413.png
Attachment 9: 18042021_0722_AIDA.png
18042021_0722_AIDA.png
Attachment 10: 18042021_0615_rate.png
18042021_0615_rate.png
Attachment 11: 18042021_0616_temp.png
18042021_0616_temp.png
Attachment 12: 18042021_0620_bias.png
18042021_0620_bias.png
  565   Mon Apr 8 11:18:19 2024 TDS505 offline analysis data file R3_150
DRG quotes ( from Elog ) S505 FEE64 configuration as

"Configuration: Single/Narrow - 8 FEEs, 2 DSSDs

DSSD 1:
 X 0-63    =  3 (HV -ve)
 X 64-127  =  1
 Y 0-63    =  4 (HV 0v) 
 Y 64-127  =  2 
DSSD 2:
 X 0-63    =  7 (HV -ve)
 X 64-127  =  5
 Y 0-63    =  8 (HV 0v)
 Y 64-127  =  6 


Scalers:
1 - Pulser
2 - n/c
3 - Time Machine Original
4 - Time Machine Delayed
5 - n/c
6 - n/c
7 - SC41 L
8 - SC41 R"

LEC (20MeV FSR) spectra

attachment 1 m_p versus m_n DSSSD#1 & DSSSD#2
attachment 2 E_p versus E_n DSSSD#1 & DSSSD#2

HEC (20GeV FSR) spectra

attachment 3 m_p versus m_n DSSSD#1 & DSSSD#2
attachment 4 E_p versus E_n DSSSD#1 & DSSSD#2

attachment variables.dat ( ADC offsets, e_diff windows & FEE64 onfiguration ) NAMELIST I/O format
Attachment 1: Screenshot_from_2024-03-29_20-21-07.png
Screenshot_from_2024-03-29_20-21-07.png
Attachment 2: Screenshot_from_2024-03-29_20-21-45.png
Screenshot_from_2024-03-29_20-21-45.png
Attachment 3: Screenshot_from_2024-03-29_20-22-29.png
Screenshot_from_2024-03-29_20-22-29.png
Attachment 4: Screenshot_from_2024-03-29_20-23-06.png
Screenshot_from_2024-03-29_20-23-06.png
Attachment 5: variables.dat
  $variables
  offset(   0) =    28.32
  offset(   1) =    17.71
  offset(   2) =    -9.75
  offset(   3) =    -5.24
  offset(   4) =   -10.84
  offset(   5) =     0.13
  offset(   6) =    28.59
  offset(   7) =    28.54
  offset(   8) =     0.83
  offset(   9) =   -12.15
  offset(  10) =    29.36
  offset(  11) =   -50.47
  offset(  12) =    10.03
  offset(  13) =   -26.90
  offset(  14) =    12.49
  offset(  15) =    10.57
  offset(  16) =     9.29
  offset(  17) =     1.09
  offset(  18) =    21.86
  offset(  19) =    -0.32
  offset(  20) =    22.59
  offset(  21) = -9999.99
  offset(  22) =   -20.22
  offset(  23) =     6.61
  offset(  24) =    -8.80
  offset(  25) =    -1.15
  offset(  26) =    13.08
  offset(  27) =    -9.03
  offset(  28) =    -2.67
  offset(  29) =    -9.42
  offset(  30) =     8.07
  offset(  31) =    19.01
  offset(  32) =     6.86
  offset(  33) =    15.16
  offset(  34) =    -3.89
  offset(  35) =   -18.80
  offset(  36) =    17.79
  offset(  37) =    16.45
  offset(  38) =    -0.74
  offset(  39) =     0.69
  offset(  40) =   -14.59
  offset(  41) =   -10.02
  offset(  42) =    11.38
  offset(  43) =    -2.57
  offset(  44) =    15.48
  offset(  45) =    14.57
  offset(  46) =     2.95
  offset(  47) =    30.70
  offset(  48) =   -12.55
  offset(  49) =   -15.61
  offset(  50) =    -6.61
  offset(  51) =    -7.10
  offset(  52) =   -33.20
  offset(  53) =    21.42
  offset(  54) =    -4.07
  offset(  55) =   -18.23
  offset(  56) =     8.83
  offset(  57) =   -22.74
  offset(  58) =    11.47
  offset(  59) =    10.67
  offset(  60) =   -28.65
  offset(  61) =    15.29
  offset(  62) =     5.76
  offset(  63) =   -11.57
  offset(  64) =     1.95
  offset(  65) = -9999.99
  offset(  66) =    22.52
  offset(  67) =    42.10
  offset(  68) =    -0.20
  offset(  69) =     9.78
  offset(  70) =   -11.48
  offset(  71) =    -0.03
  offset(  72) =   -12.50
  offset(  73) =    25.08
  offset(  74) =    12.77
  offset(  75) =    -2.76
  offset(  76) =     7.90
  offset(  77) =     9.85
  offset(  78) =     5.20
  offset(  79) =   -13.91
  offset(  80) =    18.82
  offset(  81) =    -8.25
  offset(  82) =    -8.40
  offset(  83) =    13.67
  offset(  84) =   -10.37
  offset(  85) =    28.32
  offset(  86) =    11.42
  offset(  87) =     2.74
  offset(  88) =   -24.88
  offset(  89) =    11.28
  offset(  90) =   -26.69
  offset(  91) =     0.91
  offset(  92) =     8.04
  offset(  93) =     7.64
  offset(  94) =    10.10
  offset(  95) =   -20.71
  offset(  96) =     6.72
  offset(  97) =   -12.63
  offset(  98) =    10.27
  offset(  99) =    16.05
  offset( 100) =     8.96
  offset( 101) =    33.57
  offset( 102) =    28.81
  offset( 103) =    23.49
  offset( 104) =     7.96
  offset( 105) =     6.07
  offset( 106) =    21.57
  offset( 107) =    13.06
  offset( 108) =     0.82
  offset( 109) =    -3.55
  offset( 110) =    -5.21
  offset( 111) =    11.48
  offset( 112) =    16.21
  offset( 113) =    -0.10
  offset( 114) =     3.87
  offset( 115) =    19.76
  offset( 116) =    -9.11
  offset( 117) =    24.64
  offset( 118) =     1.82
  offset( 119) = -9999.99
  offset( 120) =    22.80
  offset( 121) =    -7.37
  offset( 122) =   -18.32
  offset( 123) =   -24.74
  offset( 124) =    -4.71
  offset( 125) =    19.64
  offset( 126) =    53.61
  offset( 127) =     6.57
  offset( 128) =     3.21
  offset( 129) =     7.20
  offset( 130) =     5.55
  offset( 131) =    -3.40
  offset( 132) =   -16.40
  offset( 133) =    -1.71
  offset( 134) =   -14.85
  offset( 135) =   -29.53
  offset( 136) =     0.68
  offset( 137) =    -0.41
  offset( 138) =    17.59
  offset( 139) =   -28.92
  offset( 140) =    -8.79
  offset( 141) =    11.69
  offset( 142) =    -5.80
  offset( 143) =    -8.21
  offset( 144) =    13.90
  offset( 145) =    11.27
  offset( 146) =   -17.02
  offset( 147) =    -8.19
  offset( 148) =     6.14
  offset( 149) =   -13.13
  offset( 150) =    15.41
  offset( 151) =     0.12
  offset( 152) =    -5.02
  offset( 153) =     4.25
  offset( 154) =     3.84
  offset( 155) =    13.34
  offset( 156) =    16.79
  offset( 157) =   -36.21
  offset( 158) =    -2.18
  offset( 159) =    12.94
  offset( 160) =     2.05
  offset( 161) =   -17.31
  offset( 162) =    18.38
  offset( 163) =    -7.17
  offset( 164) =     6.59
  offset( 165) =   -13.36
  offset( 166) =   -12.11
  offset( 167) =    -3.91
  offset( 168) =    10.97
  offset( 169) =   -25.72
  offset( 170) =   -11.53
  offset( 171) =     1.94
  offset( 172) =   -12.76
  offset( 173) =    18.55
  offset( 174) =    13.49
  offset( 175) =    -9.45
  offset( 176) =   -10.19
  offset( 177) =    18.69
  offset( 178) =    24.49
  offset( 179) =    30.44
  offset( 180) =    41.95
  offset( 181) =     1.55
  offset( 182) =    20.88
  offset( 183) =     4.76
  offset( 184) =    15.80
  offset( 185) =    12.37
  offset( 186) =   -13.03
  offset( 187) =    15.64
  offset( 188) =    -8.44
  offset( 189) =    -3.02
  offset( 190) =    10.57
  offset( 191) =     6.88
  offset( 192) =   -17.81
  offset( 193) =    -5.77
  offset( 194) =   -18.29
  offset( 195) =    17.62
  offset( 196) =   -30.66
  offset( 197) =    -7.37
  offset( 198) =    36.32
  offset( 199) =     3.42
  offset( 200) =   -21.92
  offset( 201) =    15.88
  offset( 202) =   -17.33
  offset( 203) =    -2.71
  offset( 204) =   -21.06
  offset( 205) =    -4.53
  offset( 206) =    -1.59
  offset( 207) =    -9.54
  offset( 208) =    -0.04
  offset( 209) =    23.57
  offset( 210) =    13.83
  offset( 211) =    -9.28
  offset( 212) =    12.42
  offset( 213) =    23.85
  offset( 214) =     7.90
  offset( 215) =    -7.29
  offset( 216) =    26.18
  offset( 217) =    33.05
  offset( 218) =    32.10
  offset( 219) =    -8.10
  offset( 220) =    16.16
  offset( 221) =     3.94
  offset( 222) =   -18.22
  offset( 223) =   -16.84
  offset( 224) =     5.79
  offset( 225) =    21.07
  offset( 226) =    45.18
  offset( 227) =   -13.85
  offset( 228) =     6.76
  offset( 229) =    13.59
  offset( 230) =     8.28
  offset( 231) =    -9.32
  offset( 232) =    21.30
  offset( 233) =     7.02
  offset( 234) =    24.11
  offset( 235) =    16.26
  offset( 236) =    11.46
  offset( 237) =     2.16
  offset( 238) =     7.19
  offset( 239) =    18.45
  offset( 240) =     6.65
  offset( 241) =     6.56
  offset( 242) =   -33.59
  offset( 243) =    14.71
  offset( 244) =    21.28
  offset( 245) =    -0.73
  offset( 246) =   -10.29
  offset( 247) =    28.50
  offset( 248) =   -11.37
  offset( 249) =   -18.92
  offset( 250) =    23.12
  offset( 251) =     8.87
  offset( 252) =     9.64
  offset( 253) =    16.43
  offset( 254) =    -2.70
  offset( 255) =    26.14
  offset( 256) =    -4.99
  offset( 257) =   -10.88
  offset( 258) =    27.41
  offset( 259) =    -3.48
  offset( 260) =   -11.18
  offset( 261) =    12.60
  offset( 262) =    22.19
  offset( 263) =   -20.16
  offset( 264) =    20.32
  offset( 265) =     9.66
  offset( 266) =    13.30
  offset( 267) =   -12.64
  offset( 268) =    53.69
  offset( 269) =    23.15
  offset( 270) =    43.60
  offset( 271) =    48.58
  offset( 272) =   -13.12
  offset( 273) =   -12.57
  offset( 274) =    35.19
  offset( 275) =    14.84
  offset( 276) =    33.05
  offset( 277) =    13.73
  offset( 278) =    45.45
  offset( 279) =    55.49
  offset( 280) =    48.13
  offset( 281) =     3.86
  offset( 282) =    28.44
  offset( 283) =    23.73
  offset( 284) =    11.53
  offset( 285) =   -19.46
  offset( 286) =    10.81
  offset( 287) =    17.23
  offset( 288) =   -13.85
  offset( 289) =    30.17
  offset( 290) =    10.70
  offset( 291) =    28.06
  offset( 292) =   -14.35
  offset( 293) =    23.46
  offset( 294) =    15.16
  offset( 295) =   -25.15
  offset( 296) =   -15.03
  offset( 297) =   -24.74
  offset( 298) =    -5.94
... 228 more lines ...
  670   Tue Nov 5 15:19:35 2024 TDS505 offline analysis R5_780 - R5_814
Offline analysis of data files S505/R5_780 - R5_814 (corresponding to MBS data files 73-74) 

Can find an analysis of alpha background runs at RIBF, RIKEN for comparison at https://elog.ph.ed.ac.uk/AIDA/816

Beam off - background runs without sources

MBS 73 27.6.22 09:33-11:33 CEST https://elog.gsi.de/despec/S505/204

MBS 74 27.6.22 11:34-13:34 CEST https://elog.gsi.de/despec/S505/206

MBS 75 27.6.22 13:35-16:29 CEST https://elog.gsi.de/despec/S505/209

ADC offsets per https://elog.ph.ed.ac.uk/DESPEC/556

FEE64 configuration

FEE64   a b c d
DSSSD#1 3 4 1 2
DSSSD#2 7 8 5 6

p+n junction FE64s odd numbered


Data analysis assumes

- all LEC ADC data channels with valid ADC offset included (474 of 512 channels)
- no clustering
- no p+n junction side - n+n Ohmic side correlation time gates
- valid LEC events
   0 < p+n junction side multiplicity < 8 
   and
   0 < n+n Ohmic side multiplicity < 8 


Attachments 1-2 - per DSSSD p+n versus n+n multiplicity

Attachment 3 - per DSSSD x versus y

Attachments 4-5 - per DSSSD p+n versus n+n energy (20keV/channel nominal)
                  all combinations of per DSSSD p+n junction and n+n Ohmic energies 
                  with projection of data within window onto x and y axes

                  too many events for natural (U decay series) background
                  off leading diagonal correlations anomalous
                  transverse width of leading diagonal correlation wider than expected - ADC offsets OK?
                 

Attachment 6 - per FEE64 WR timestamp (32.768us/channel)
               FEE64 sync test using pulser data - looks OK

Attachments 7-14 - per FEE64 ADC spectra (5.6keV/channel nominal)
                   note common x/y scale - pulser peak height proxy for peak width

per FEE64 1.8.L pulser peak widths (ch FWHM)

1 11.43
2 16.58
3 12.52
4 17.54
5 9.10
6 12.22
7 17.03
8 16.09

3 of 4 p+n junction FEE64 good (<70keV), 1 of 4 n+n Ohmic FEE64 good - all others < 100keV
Pulser peak indicates noise/gain/offset stable throughout background runs


Attachments 15-22 - per FEE64 ADC spectra (5.6kleV/channel nominal)
                    observe broad peak-like structures (c. 2.8MeV) in first channel of most ASICs? 

Attachment 23 per DSSSD p+n versus n+n energy (20keV/channel nominal)
              valid LEC events
              p+n junction side multiplicity = 1 n+n Ohmic side multiplicity = 1

Attachment 24 p+n FEE64 #0 energy versus each of n+n FEE64s (#1, #3, #5 and #7) energies (20keV/channel nominal)
              valid LEC events
              p+n junction side multiplicity = 1 n+n Ohmic side multiplicity = 1

Attachments 23 & 24 do not unambiguously identify which FEE64s are attached to which DSSSD
                  
Attachment 1: Screenshot_from_2024-11-06_10-16-55.png
Screenshot_from_2024-11-06_10-16-55.png
Attachment 2: Screenshot_from_2024-11-06_10-18-43.png
Screenshot_from_2024-11-06_10-18-43.png
Attachment 3: Screenshot_from_2024-11-06_10-19-38.png
Screenshot_from_2024-11-06_10-19-38.png
Attachment 4: Screenshot_from_2024-11-06_10-21-13.png
Screenshot_from_2024-11-06_10-21-13.png
Attachment 5: Screenshot_from_2024-11-06_10-23-42.png
Screenshot_from_2024-11-06_10-23-42.png
Attachment 6: Screenshot_from_2024-11-06_10-47-04.png
Screenshot_from_2024-11-06_10-47-04.png
Attachment 7: Screenshot_from_2024-11-06_10-49-16.png
Screenshot_from_2024-11-06_10-49-16.png
Attachment 8: Screenshot_from_2024-11-06_10-50-17.png
Screenshot_from_2024-11-06_10-50-17.png
Attachment 9: Screenshot_from_2024-11-06_10-50-59.png
Screenshot_from_2024-11-06_10-50-59.png
Attachment 10: Screenshot_from_2024-11-06_10-51-48.png
Screenshot_from_2024-11-06_10-51-48.png
Attachment 11: Screenshot_from_2024-11-06_10-52-49.png
Screenshot_from_2024-11-06_10-52-49.png
Attachment 12: Screenshot_from_2024-11-06_10-54-12.png
Screenshot_from_2024-11-06_10-54-12.png
Attachment 13: Screenshot_from_2024-11-06_10-55-07.png
Screenshot_from_2024-11-06_10-55-07.png
Attachment 14: Screenshot_from_2024-11-06_10-55-56.png
Screenshot_from_2024-11-06_10-55-56.png
Attachment 15: Screenshot_from_2024-11-06_11-01-47.png
Screenshot_from_2024-11-06_11-01-47.png
Attachment 16: Screenshot_from_2024-11-06_11-02-48.png
Screenshot_from_2024-11-06_11-02-48.png
Attachment 17: Screenshot_from_2024-11-06_11-04-21.png
Screenshot_from_2024-11-06_11-04-21.png
Attachment 18: Screenshot_from_2024-11-06_11-05-09.png
Screenshot_from_2024-11-06_11-05-09.png
Attachment 19: Screenshot_from_2024-11-06_11-06-16.png
Screenshot_from_2024-11-06_11-06-16.png
Attachment 20: Screenshot_from_2024-11-06_11-07-16.png
Screenshot_from_2024-11-06_11-07-16.png
Attachment 21: Screenshot_from_2024-11-06_11-08-07.png
Screenshot_from_2024-11-06_11-08-07.png
Attachment 22: Screenshot_from_2024-11-06_11-09-09.png
Screenshot_from_2024-11-06_11-09-09.png
Attachment 23: Screenshot_from_2024-11-08_03-36-38.png
Screenshot_from_2024-11-08_03-36-38.png
Attachment 24: Screenshot_from_2024-11-08_03-39-13.png
Screenshot_from_2024-11-08_03-39-13.png
  556   Tue Apr 2 18:37:21 2024 TDS505 ADC offsets
S505 ADC offsets using pulser walkthrough data from data file R1

ch = channel + ( module * 64 ) + ( range * 2048 )

adc_data( ch ) = INT( RSHIFT( ABS( adc_data( ch ) - 32768 ), 3 ) - offset( ch ) + 0.5 )
Attachment 1: S505_calibration_data.txt
  $variables
  offset(   0) =    28.32
  offset(   1) =    17.71
  offset(   2) =    -9.75
  offset(   3) =    -5.24
  offset(   4) =   -10.84
  offset(   5) =     0.13
  offset(   6) =    28.59
  offset(   7) =    28.54
  offset(   8) =     0.83
  offset(   9) =   -12.15
  offset(  10) =    29.36
  offset(  11) =   -50.47
  offset(  12) =    10.03
  offset(  13) =   -26.90
  offset(  14) =    12.49
  offset(  15) =    10.57
  offset(  16) =     9.29
  offset(  17) =     1.09
  offset(  18) =    21.86
  offset(  19) =    -0.32
  offset(  20) =    22.59
  offset(  21) = -9999.99
  offset(  22) =   -20.22
  offset(  23) =     6.61
  offset(  24) =    -8.80
  offset(  25) =    -1.15
  offset(  26) =    13.08
  offset(  27) =    -9.03
  offset(  28) =    -2.67
  offset(  29) =    -9.42
  offset(  30) =     8.07
  offset(  31) =    19.01
  offset(  32) =     6.86
  offset(  33) =    15.16
  offset(  34) =    -3.89
  offset(  35) =   -18.80
  offset(  36) =    17.79
  offset(  37) =    16.45
  offset(  38) =    -0.74
  offset(  39) =     0.69
  offset(  40) =   -14.59
  offset(  41) =   -10.02
  offset(  42) =    11.38
  offset(  43) =    -2.57
  offset(  44) =    15.48
  offset(  45) =    14.57
  offset(  46) =     2.95
  offset(  47) =    30.70
  offset(  48) =   -12.55
  offset(  49) =   -15.61
  offset(  50) =    -6.61
  offset(  51) =    -7.10
  offset(  52) =   -33.20
  offset(  53) =    21.42
  offset(  54) =    -4.07
  offset(  55) =   -18.23
  offset(  56) =     8.83
  offset(  57) =   -22.74
  offset(  58) =    11.47
  offset(  59) =    10.67
  offset(  60) =   -28.65
  offset(  61) =    15.29
  offset(  62) =     5.76
  offset(  63) =   -11.57
  offset(  64) =     1.95
  offset(  65) = -9999.99
  offset(  66) =    22.52
  offset(  67) =    42.10
  offset(  68) =    -0.20
  offset(  69) =     9.78
  offset(  70) =   -11.48
  offset(  71) =    -0.03
  offset(  72) =   -12.50
  offset(  73) =    25.08
  offset(  74) =    12.77
  offset(  75) =    -2.76
  offset(  76) =     7.90
  offset(  77) =     9.85
  offset(  78) =     5.20
  offset(  79) =   -13.91
  offset(  80) =    18.82
  offset(  81) =    -8.25
  offset(  82) =    -8.40
  offset(  83) =    13.67
  offset(  84) =   -10.37
  offset(  85) =    28.32
  offset(  86) =    11.42
  offset(  87) =     2.74
  offset(  88) =   -24.88
  offset(  89) =    11.28
  offset(  90) =   -26.69
  offset(  91) =     0.91
  offset(  92) =     8.04
  offset(  93) =     7.64
  offset(  94) =    10.10
  offset(  95) =   -20.71
  offset(  96) =     6.72
  offset(  97) =   -12.63
  offset(  98) =    10.27
  offset(  99) =    16.05
  offset( 100) =     8.96
  offset( 101) =    33.57
  offset( 102) =    28.81
  offset( 103) =    23.49
  offset( 104) =     7.96
  offset( 105) =     6.07
  offset( 106) =    21.57
  offset( 107) =    13.06
  offset( 108) =     0.82
  offset( 109) =    -3.55
  offset( 110) =    -5.21
  offset( 111) =    11.48
  offset( 112) =    16.21
  offset( 113) =    -0.10
  offset( 114) =     3.87
  offset( 115) =    19.76
  offset( 116) =    -9.11
  offset( 117) =    24.64
  offset( 118) =     1.82
  offset( 119) = -9999.99
  offset( 120) =    22.80
  offset( 121) =    -7.37
  offset( 122) =   -18.32
  offset( 123) =   -24.74
  offset( 124) =    -4.71
  offset( 125) =    19.64
  offset( 126) =    53.61
  offset( 127) =     6.57
  offset( 128) =     3.21
  offset( 129) =     7.20
  offset( 130) =     5.55
  offset( 131) =    -3.40
  offset( 132) =   -16.40
  offset( 133) =    -1.71
  offset( 134) =   -14.85
  offset( 135) =   -29.53
  offset( 136) =     0.68
  offset( 137) =    -0.41
  offset( 138) =    17.59
  offset( 139) =   -28.92
  offset( 140) =    -8.79
  offset( 141) =    11.69
  offset( 142) =    -5.80
  offset( 143) =    -8.21
  offset( 144) =    13.90
  offset( 145) =    11.27
  offset( 146) =   -17.02
  offset( 147) =    -8.19
  offset( 148) =     6.14
  offset( 149) =   -13.13
  offset( 150) =    15.41
  offset( 151) =     0.12
  offset( 152) =    -5.02
  offset( 153) =     4.25
  offset( 154) =     3.84
  offset( 155) =    13.34
  offset( 156) =    16.79
  offset( 157) =   -36.21
  offset( 158) =    -2.18
  offset( 159) =    12.94
  offset( 160) =     2.05
  offset( 161) =   -17.31
  offset( 162) =    18.38
  offset( 163) =    -7.17
  offset( 164) =     6.59
  offset( 165) =   -13.36
  offset( 166) =   -12.11
  offset( 167) =    -3.91
  offset( 168) =    10.97
  offset( 169) =   -25.72
  offset( 170) =   -11.53
  offset( 171) =     1.94
  offset( 172) =   -12.76
  offset( 173) =    18.55
  offset( 174) =    13.49
  offset( 175) =    -9.45
  offset( 176) =   -10.19
  offset( 177) =    18.69
  offset( 178) =    24.49
  offset( 179) =    30.44
  offset( 180) =    41.95
  offset( 181) =     1.55
  offset( 182) =    20.88
  offset( 183) =     4.76
  offset( 184) =    15.80
  offset( 185) =    12.37
  offset( 186) =   -13.03
  offset( 187) =    15.64
  offset( 188) =    -8.44
  offset( 189) =    -3.02
  offset( 190) =    10.57
  offset( 191) =     6.88
  offset( 192) =   -17.81
  offset( 193) =    -5.77
  offset( 194) =   -18.29
  offset( 195) =    17.62
  offset( 196) =   -30.66
  offset( 197) =    -7.37
  offset( 198) =    36.32
  offset( 199) =     3.42
  offset( 200) =   -21.92
  offset( 201) =    15.88
  offset( 202) =   -17.33
  offset( 203) =    -2.71
  offset( 204) =   -21.06
  offset( 205) =    -4.53
  offset( 206) =    -1.59
  offset( 207) =    -9.54
  offset( 208) =    -0.04
  offset( 209) =    23.57
  offset( 210) =    13.83
  offset( 211) =    -9.28
  offset( 212) =    12.42
  offset( 213) =    23.85
  offset( 214) =     7.90
  offset( 215) =    -7.29
  offset( 216) =    26.18
  offset( 217) =    33.05
  offset( 218) =    32.10
  offset( 219) =    -8.10
  offset( 220) =    16.16
  offset( 221) =     3.94
  offset( 222) =   -18.22
  offset( 223) =   -16.84
  offset( 224) =     5.79
  offset( 225) =    21.07
  offset( 226) =    45.18
  offset( 227) =   -13.85
  offset( 228) =     6.76
  offset( 229) =    13.59
  offset( 230) =     8.28
  offset( 231) =    -9.32
  offset( 232) =    21.30
  offset( 233) =     7.02
  offset( 234) =    24.11
  offset( 235) =    16.26
  offset( 236) =    11.46
  offset( 237) =     2.16
  offset( 238) =     7.19
  offset( 239) =    18.45
  offset( 240) =     6.65
  offset( 241) =     6.56
  offset( 242) =   -33.59
  offset( 243) =    14.71
  offset( 244) =    21.28
  offset( 245) =    -0.73
  offset( 246) =   -10.29
  offset( 247) =    28.50
  offset( 248) =   -11.37
  offset( 249) =   -18.92
  offset( 250) =    23.12
  offset( 251) =     8.87
  offset( 252) =     9.64
  offset( 253) =    16.43
  offset( 254) =    -2.70
  offset( 255) =    26.14
  offset( 256) =    -4.99
  offset( 257) =   -10.88
  offset( 258) =    27.41
  offset( 259) =    -3.48
  offset( 260) =   -11.18
  offset( 261) =    12.60
  offset( 262) =    22.19
  offset( 263) =   -20.16
  offset( 264) =    20.32
  offset( 265) =     9.66
  offset( 266) =    13.30
  offset( 267) =   -12.64
  offset( 268) =    53.69
  offset( 269) =    23.15
  offset( 270) =    43.60
  offset( 271) =    48.58
  offset( 272) =   -13.12
  offset( 273) =   -12.57
  offset( 274) =    35.19
  offset( 275) =    14.84
  offset( 276) =    33.05
  offset( 277) =    13.73
  offset( 278) =    45.45
  offset( 279) =    55.49
  offset( 280) =    48.13
  offset( 281) =     3.86
  offset( 282) =    28.44
  offset( 283) =    23.73
  offset( 284) =    11.53
  offset( 285) =   -19.46
  offset( 286) =    10.81
  offset( 287) =    17.23
  offset( 288) =   -13.85
  offset( 289) =    30.17
  offset( 290) =    10.70
  offset( 291) =    28.06
  offset( 292) =   -14.35
  offset( 293) =    23.46
  offset( 294) =    15.16
  offset( 295) =   -25.15
  offset( 296) =   -15.03
  offset( 297) =   -24.74
  offset( 298) =    -5.94
... 215 more lines ...
  217   Tue Apr 13 14:24:48 2021 TDS460 information
aida-gsi Anydesk 897170655

DESPEC remote working - https://sf.gsi.de/d/a4fb2134e06a450ca777/

S460 shift plan - https://docs.google.com/spreadsheets/d/1hjNM5xdPoxq0MfNE5ILnNrHm2QKW7Pgp1UKF7p70ghQ/edit#gid=1097596683

Mattermost - https://mattermost.gsi.de/despec/channels/aida

DESPEC Zoom meeting

Zoom Meeting https://gsi-fair.zoom.us/j/94404952361
Passcode: s460
  174   Fri Mar 5 23:45:41 2021 TDS452 information
aida-gsi Anydesk 897170655

DESPEC remote working - https://sf.gsi.de/d/a4fb2134e06a450ca777/

S452 shift checks - attachment 1

S452 shift plan - PDF attachment 2 - Google doc https://docs.google.com/spreadsheets/d/1hjNM5xdPoxq0MfNE5ILnNrHm2QKW7Pgp1UKF7p70ghQ/edit#gid=1745844766

Mattermost - https://mattermost.gsi.de/despec/channels/experiment-s452-nearline-sort

DESPEC Zoom meeting

Join Zoom Meeting
https://gsi-fair.zoom.us/j/98980822569

Meeting ID: 989 8082 2569
Passcode: S452

FRS Zoom meeting 

Meeting-ID: 973 4809 5114

password: FRS-BT2021

Kenncode: 456657
Attachment 1: S452__Shifts_checkpoints_v1_(1).pdf
S452__Shifts_checkpoints_v1_(1).pdf S452__Shifts_checkpoints_v1_(1).pdf S452__Shifts_checkpoints_v1_(1).pdf S452__Shifts_checkpoints_v1_(1).pdf S452__Shifts_checkpoints_v1_(1).pdf S452__Shifts_checkpoints_v1_(1).pdf S452__Shifts_checkpoints_v1_(1).pdf
Attachment 2: S452_Shift_Plan.pdf
S452_Shift_Plan.pdf
  170   Thu Mar 4 11:03:04 2021 NHS452 Operating Notes
Please note the following "quirks" currently present in S452 that are important

a) Strips/Thresholds
Thresholds are 0xA, 0xA, 0x10 [Maybe changed depending on rate, etc]
Some strips are very noisy - damage?

b) Powering FEE64s
Due to a misbehaving USB hub(?) or cable only 7 FEE TTYs are monitored
aida04 and aida05 can take minutes to start, unsure why.
aida05 is monitored on nnrpi1 and hence when aida05 is ready it is safe to assume all 12 are ready
Watch the logs as normal

c) Slow control & nnrpi2
For the Grafana script to read the DSSD HV, it must be ran in a screen session
If nnrpi2 is restarted please run the HV as a screen by Putty SSHing (localhost 22)
and running
screen -S caenhv /dev/ttyACM0
if screen was just closed you can do
screen -x caenhv
to reconnect

The pulser ideally should be started the same
screen -S pulser /dev/ttyUSB0
To allow remote changing by NH/GSI
The pulser is sent to all subsystems for correlation and should be kept on at 2 Hz for the experiment
IN the pulser putty you can check the settings by typing
display settings
And general commands:
help

The NIM crate can be monitored /dev/ttyUSB2 but you should be careful
It is very easy to turn off the NIM crate if you do
I run it in a tmux session with (
tmux attach -r
) to have it "readonly"

There is a small script on nnrpi2
./check_usbs.sh
to see which ttyUSB is which device
As they can change if the pi is rebooted

d) Scaler Readout
If powercycled FEEs, enable the scaler in Local Controls page by setting BuTiS interface control to 0x3

To enable scaler readout after GOing the run please open the Correlation Control page
(Act on all FEE64s) and set control to 0x1
This has to be done every time the FEE64s are set to GOing

e) New Tape Server
For waveform readout in future experiments, the tape server is in "Expert" mode which is harder to use
Please read the "How to use" instructions by clicking the button on the bottom right
Or in a terminal
cat /MIDAS/TclHttpd/Html/ETape/1.txt

f) GSI/MBS
If GSI team ask to restart AIDA MBS you may need to restart the MBS relay so it can reconnect

To start MBS in a terminal
ssh despec@x86l-94
cd mbsrun/s452/aida_to_mbs
mbs
@startup
It should report it connected to AIDA-3 - if not check the MBS relay is running or restart the relay

Please check the MBS rate command
rate -nst -rst -nev -rev -ndata -rdata -st
That the "Streams" column does not frequently (ever) have 0 empty streams
If so AIDA data may be getting dropped and should be looked into
If another DESPEC DAQ has failed this will eventually happen and is not a problem
But if it occurs in normal operation it is concerning

Do NOT stop the AIDA run without talking to GSI team - if no AIDA data the whole MBS DAQ stops
and blocks all other subsystems. It is best to keep AIDA ticking over constantly and let
MBS start/stop files when relevant.
Of course if things have to stop it is OK but inform GSI shifters so they can take action
  481   Tue Jun 14 12:15:33 2022 NHS450 Archived
All MIDAS files for S450 (/TapeData/S450) have been copied to lustre and archived 
Attachment 1: s450_aida_tape.txt
R1_0.gz: (0x00000009) exists archived, archive_id:13
R1_100.gz: (0x00000009) exists archived, archive_id:13
R1_101.gz: (0x00000009) exists archived, archive_id:13
R1_102.gz: (0x00000009) exists archived, archive_id:13
R1_103.gz: (0x00000009) exists archived, archive_id:13
R1_104.gz: (0x00000009) exists archived, archive_id:13
R1_105.gz: (0x00000009) exists archived, archive_id:13
R1_106.gz: (0x00000009) exists archived, archive_id:13
R1_107.gz: (0x00000009) exists archived, archive_id:13
R1_108.gz: (0x00000009) exists archived, archive_id:13
R1_109.gz: (0x00000009) exists archived, archive_id:13
R1_10.gz: (0x00000009) exists archived, archive_id:13
R1_110.gz: (0x00000009) exists archived, archive_id:13
R1_111.gz: (0x00000009) exists archived, archive_id:13
R1_112.gz: (0x00000009) exists archived, archive_id:13
R1_113.gz: (0x00000009) exists archived, archive_id:13
R1_114.gz: (0x00000009) exists archived, archive_id:13
R1_115.gz: (0x00000009) exists archived, archive_id:13
R1_116.gz: (0x00000009) exists archived, archive_id:13
R1_117.gz: (0x00000009) exists archived, archive_id:13
R1_118.gz: (0x00000009) exists archived, archive_id:13
R1_119.gz: (0x00000009) exists archived, archive_id:13
R1_11.gz: (0x00000009) exists archived, archive_id:13
R1_120.gz: (0x00000009) exists archived, archive_id:13
R1_121.gz: (0x00000009) exists archived, archive_id:13
R1_122.gz: (0x00000009) exists archived, archive_id:13
R1_123.gz: (0x00000009) exists archived, archive_id:13
R1_124.gz: (0x00000009) exists archived, archive_id:13
R1_125.gz: (0x00000009) exists archived, archive_id:13
R1_126.gz: (0x00000009) exists archived, archive_id:13
R1_127.gz: (0x00000009) exists archived, archive_id:13
R1_128.gz: (0x00000009) exists archived, archive_id:13
R1_129.gz: (0x00000009) exists archived, archive_id:13
R1_12.gz: (0x00000009) exists archived, archive_id:13
R1_130.gz: (0x00000009) exists archived, archive_id:13
R1_131.gz: (0x00000009) exists archived, archive_id:13
R1_132.gz: (0x00000009) exists archived, archive_id:13
R1_133.gz: (0x00000009) exists archived, archive_id:13
R1_134.gz: (0x00000009) exists archived, archive_id:13
R1_135.gz: (0x00000009) exists archived, archive_id:13
R1_136.gz: (0x00000009) exists archived, archive_id:13
R1_137.gz: (0x00000009) exists archived, archive_id:13
R1_138.gz: (0x00000009) exists archived, archive_id:13
R1_139.gz: (0x00000009) exists archived, archive_id:13
R1_13.gz: (0x00000009) exists archived, archive_id:13
R1_140.gz: (0x00000009) exists archived, archive_id:13
R1_141.gz: (0x00000009) exists archived, archive_id:13
R1_142.gz: (0x00000009) exists archived, archive_id:13
R1_143.gz: (0x00000009) exists archived, archive_id:13
R1_144.gz: (0x00000009) exists archived, archive_id:13
R1_145.gz: (0x00000009) exists archived, archive_id:13
R1_146.gz: (0x00000009) exists archived, archive_id:13
R1_147.gz: (0x00000009) exists archived, archive_id:13
R1_148.gz: (0x00000009) exists archived, archive_id:13
R1_149.gz: (0x00000009) exists archived, archive_id:13
R1_14.gz: (0x00000009) exists archived, archive_id:13
R1_150.gz: (0x00000009) exists archived, archive_id:13
R1_151.gz: (0x00000009) exists archived, archive_id:13
R1_152.gz: (0x00000009) exists archived, archive_id:13
R1_153.gz: (0x00000009) exists archived, archive_id:13
R1_154.gz: (0x00000009) exists archived, archive_id:13
R1_155.gz: (0x00000009) exists archived, archive_id:13
R1_156.gz: (0x00000009) exists archived, archive_id:13
R1_157.gz: (0x00000009) exists archived, archive_id:13
R1_158.gz: (0x00000009) exists archived, archive_id:13
R1_159.gz: (0x00000009) exists archived, archive_id:13
R1_15.gz: (0x00000009) exists archived, archive_id:13
R1_160.gz: (0x00000009) exists archived, archive_id:13
R1_161.gz: (0x00000009) exists archived, archive_id:13
R1_162.gz: (0x00000009) exists archived, archive_id:13
R1_163.gz: (0x00000009) exists archived, archive_id:13
R1_164.gz: (0x00000009) exists archived, archive_id:13
R1_165.gz: (0x00000009) exists archived, archive_id:13
R1_166.gz: (0x00000009) exists archived, archive_id:13
R1_167.gz: (0x00000009) exists archived, archive_id:13
R1_168.gz: (0x00000009) exists archived, archive_id:13
R1_169.gz: (0x00000009) exists archived, archive_id:13
R1_16.gz: (0x00000009) exists archived, archive_id:13
R1_170.gz: (0x00000009) exists archived, archive_id:13
R1_171.gz: (0x00000009) exists archived, archive_id:13
R1_172.gz: (0x00000009) exists archived, archive_id:13
R1_173.gz: (0x00000009) exists archived, archive_id:13
R1_174.gz: (0x00000009) exists archived, archive_id:13
R1_175.gz: (0x00000009) exists archived, archive_id:13
R1_176.gz: (0x00000009) exists archived, archive_id:13
R1_177.gz: (0x00000009) exists archived, archive_id:13
R1_178.gz: (0x00000009) exists archived, archive_id:13
R1_179.gz: (0x00000009) exists archived, archive_id:13
R1_17.gz: (0x00000009) exists archived, archive_id:13
R1_180.gz: (0x00000009) exists archived, archive_id:13
R1_181.gz: (0x00000009) exists archived, archive_id:13
R1_182.gz: (0x00000009) exists archived, archive_id:13
R1_183.gz: (0x00000009) exists archived, archive_id:13
R1_184.gz: (0x00000009) exists archived, archive_id:13
R1_185.gz: (0x00000009) exists archived, archive_id:13
R1_186.gz: (0x00000009) exists archived, archive_id:13
R1_187.gz: (0x00000009) exists archived, archive_id:13
R1_188.gz: (0x00000009) exists archived, archive_id:13
R1_189.gz: (0x00000009) exists archived, archive_id:13
R1_18.gz: (0x00000009) exists archived, archive_id:13
R1_190.gz: (0x00000009) exists archived, archive_id:13
R1_191.gz: (0x00000009) exists archived, archive_id:13
R1_192.gz: (0x00000009) exists archived, archive_id:13
R1_193.gz: (0x00000009) exists archived, archive_id:13
R1_194.gz: (0x00000009) exists archived, archive_id:13
R1_195.gz: (0x00000009) exists archived, archive_id:13
R1_196.gz: (0x00000009) exists archived, archive_id:13
R1_197.gz: (0x00000009) exists archived, archive_id:13
R1_198.gz: (0x00000009) exists archived, archive_id:13
R1_199.gz: (0x00000009) exists archived, archive_id:13
R1_19.gz: (0x00000009) exists archived, archive_id:13
R1_1.gz: (0x00000009) exists archived, archive_id:13
R1_200.gz: (0x00000009) exists archived, archive_id:13
R1_201.gz: (0x00000009) exists archived, archive_id:13
R1_202.gz: (0x00000009) exists archived, archive_id:13
R1_203.gz: (0x00000009) exists archived, archive_id:13
R1_204.gz: (0x00000009) exists archived, archive_id:13
R1_205.gz: (0x00000009) exists archived, archive_id:13
R1_206.gz: (0x00000009) exists archived, archive_id:13
R1_207.gz: (0x00000009) exists archived, archive_id:13
R1_208.gz: (0x00000009) exists archived, archive_id:13
R1_209.gz: (0x00000009) exists archived, archive_id:13
R1_20.gz: (0x00000009) exists archived, archive_id:13
R1_210.gz: (0x00000009) exists archived, archive_id:13
R1_211.gz: (0x00000009) exists archived, archive_id:13
R1_212.gz: (0x00000009) exists archived, archive_id:13
R1_213.gz: (0x00000009) exists archived, archive_id:13
R1_214.gz: (0x00000009) exists archived, archive_id:13
R1_215.gz: (0x00000009) exists archived, archive_id:13
R1_216.gz: (0x00000009) exists archived, archive_id:13
R1_217.gz: (0x00000009) exists archived, archive_id:13
R1_218.gz: (0x00000009) exists archived, archive_id:13
R1_219.gz: (0x00000009) exists archived, archive_id:13
R1_21.gz: (0x00000009) exists archived, archive_id:13
R1_220.gz: (0x00000009) exists archived, archive_id:13
R1_221.gz: (0x00000009) exists archived, archive_id:13
R1_222.gz: (0x00000009) exists archived, archive_id:13
R1_223.gz: (0x00000009) exists archived, archive_id:13
R1_224.gz: (0x00000009) exists archived, archive_id:13
R1_225.gz: (0x00000009) exists archived, archive_id:13
R1_226.gz: (0x00000009) exists archived, archive_id:13
R1_227.gz: (0x00000009) exists archived, archive_id:13
R1_228.gz: (0x00000009) exists archived, archive_id:13
R1_229.gz: (0x00000009) exists archived, archive_id:13
R1_22.gz: (0x00000009) exists archived, archive_id:13
R1_230.gz: (0x00000009) exists archived, archive_id:13
R1_231.gz: (0x00000009) exists archived, archive_id:13
R1_232.gz: (0x00000009) exists archived, archive_id:13
R1_233.gz: (0x00000009) exists archived, archive_id:13
R1_234.gz: (0x00000009) exists archived, archive_id:13
R1_235.gz: (0x00000009) exists archived, archive_id:13
R1_236.gz: (0x00000009) exists archived, archive_id:13
R1_237.gz: (0x00000009) exists archived, archive_id:13
R1_238.gz: (0x00000009) exists archived, archive_id:13
R1_239.gz: (0x00000009) exists archived, archive_id:13
R1_23.gz: (0x00000009) exists archived, archive_id:13
R1_240.gz: (0x00000009) exists archived, archive_id:13
R1_241.gz: (0x00000009) exists archived, archive_id:13
R1_242.gz: (0x00000009) exists archived, archive_id:13
R1_243.gz: (0x00000009) exists archived, archive_id:13
R1_244.gz: (0x00000009) exists archived, archive_id:13
R1_245.gz: (0x00000009) exists archived, archive_id:13
R1_246.gz: (0x00000009) exists archived, archive_id:13
R1_247.gz: (0x00000009) exists archived, archive_id:13
R1_248.gz: (0x00000009) exists archived, archive_id:13
R1_249.gz: (0x00000009) exists archived, archive_id:13
R1_24.gz: (0x00000009) exists archived, archive_id:13
R1_250.gz: (0x00000009) exists archived, archive_id:13
R1_251.gz: (0x00000009) exists archived, archive_id:13
R1_252.gz: (0x00000009) exists archived, archive_id:13
R1_253.gz: (0x00000009) exists archived, archive_id:13
R1_254.gz: (0x00000009) exists archived, archive_id:13
R1_255.gz: (0x00000009) exists archived, archive_id:13
R1_256.gz: (0x00000009) exists archived, archive_id:13
R1_257.gz: (0x00000009) exists archived, archive_id:13
R1_258.gz: (0x00000009) exists archived, archive_id:13
R1_259.gz: (0x00000009) exists archived, archive_id:13
R1_25.gz: (0x00000009) exists archived, archive_id:13
R1_260.gz: (0x00000009) exists archived, archive_id:13
R1_261.gz: (0x00000009) exists archived, archive_id:13
R1_262.gz: (0x00000009) exists archived, archive_id:13
R1_263.gz: (0x00000009) exists archived, archive_id:13
R1_264.gz: (0x00000009) exists archived, archive_id:13
R1_265.gz: (0x00000009) exists archived, archive_id:13
R1_266.gz: (0x00000009) exists archived, archive_id:13
R1_267.gz: (0x00000009) exists archived, archive_id:13
R1_268.gz: (0x00000009) exists archived, archive_id:13
R1_269.gz: (0x00000009) exists archived, archive_id:13
R1_26.gz: (0x00000009) exists archived, archive_id:13
R1_270.gz: (0x00000009) exists archived, archive_id:13
R1_271.gz: (0x00000009) exists archived, archive_id:13
R1_272.gz: (0x00000009) exists archived, archive_id:13
R1_273.gz: (0x00000009) exists archived, archive_id:13
R1_274.gz: (0x00000009) exists archived, archive_id:13
R1_275.gz: (0x00000009) exists archived, archive_id:13
R1_276.gz: (0x00000009) exists archived, archive_id:13
R1_277.gz: (0x00000009) exists archived, archive_id:13
R1_278.gz: (0x00000009) exists archived, archive_id:13
R1_279.gz: (0x00000009) exists archived, archive_id:13
R1_27.gz: (0x00000009) exists archived, archive_id:13
R1_280.gz: (0x00000009) exists archived, archive_id:13
R1_281.gz: (0x00000009) exists archived, archive_id:13
R1_282.gz: (0x00000009) exists archived, archive_id:13
R1_283.gz: (0x00000009) exists archived, archive_id:13
R1_284.gz: (0x00000009) exists archived, archive_id:13
R1_285.gz: (0x00000009) exists archived, archive_id:13
R1_286.gz: (0x00000009) exists archived, archive_id:13
R1_287.gz: (0x00000009) exists archived, archive_id:13
R1_288.gz: (0x00000009) exists archived, archive_id:13
R1_289.gz: (0x00000009) exists archived, archive_id:13
R1_28.gz: (0x00000009) exists archived, archive_id:13
R1_290.gz: (0x00000009) exists archived, archive_id:13
R1_291.gz: (0x00000009) exists archived, archive_id:13
R1_292.gz: (0x00000009) exists archived, archive_id:13
R1_293.gz: (0x00000009) exists archived, archive_id:13
R1_294.gz: (0x00000009) exists archived, archive_id:13
R1_295.gz: (0x00000009) exists archived, archive_id:13
R1_296.gz: (0x00000009) exists archived, archive_id:13
R1_297.gz: (0x00000009) exists archived, archive_id:13
R1_298.gz: (0x00000009) exists archived, archive_id:13
R1_299.gz: (0x00000009) exists archived, archive_id:13
R1_29.gz: (0x00000009) exists archived, archive_id:13
R1_2.gz: (0x00000009) exists archived, archive_id:13
R1_300.gz: (0x00000009) exists archived, archive_id:13
R1_301.gz: (0x00000009) exists archived, archive_id:13
R1_302.gz: (0x00000009) exists archived, archive_id:13
R1_303.gz: (0x00000009) exists archived, archive_id:13
R1_304.gz: (0x00000009) exists archived, archive_id:13
R1_305.gz: (0x00000009) exists archived, archive_id:13
R1_306.gz: (0x00000009) exists archived, archive_id:13
R1_307.gz: (0x00000009) exists archived, archive_id:13
R1_308.gz: (0x00000009) exists archived, archive_id:13
R1_309.gz: (0x00000009) exists archived, archive_id:13
R1_30.gz: (0x00000009) exists archived, archive_id:13
R1_310.gz: (0x00000009) exists archived, archive_id:13
R1_311.gz: (0x00000009) exists archived, archive_id:13
R1_312.gz: (0x00000009) exists archived, archive_id:13
R1_313.gz: (0x00000009) exists archived, archive_id:13
R1_314.gz: (0x00000009) exists archived, archive_id:13
R1_315.gz: (0x00000009) exists archived, archive_id:13
R1_316.gz: (0x00000009) exists archived, archive_id:13
R1_317.gz: (0x00000009) exists archived, archive_id:13
R1_318.gz: (0x00000009) exists archived, archive_id:13
R1_319.gz: (0x00000009) exists archived, archive_id:13
R1_31.gz: (0x00000009) exists archived, archive_id:13
R1_320.gz: (0x00000009) exists archived, archive_id:13
R1_321.gz: (0x00000009) exists archived, archive_id:13
R1_322.gz: (0x00000009) exists archived, archive_id:13
R1_323.gz: (0x00000009) exists archived, archive_id:13
R1_324.gz: (0x00000009) exists archived, archive_id:13
R1_325.gz: (0x00000009) exists archived, archive_id:13
R1_326.gz: (0x00000009) exists archived, archive_id:13
R1_327.gz: (0x00000009) exists archived, archive_id:13
R1_328.gz: (0x00000009) exists archived, archive_id:13
R1_329.gz: (0x00000009) exists archived, archive_id:13
R1_32.gz: (0x00000009) exists archived, archive_id:13
R1_330.gz: (0x00000009) exists archived, archive_id:13
R1_331.gz: (0x00000009) exists archived, archive_id:13
R1_332.gz: (0x00000009) exists archived, archive_id:13
R1_333.gz: (0x00000009) exists archived, archive_id:13
R1_334.gz: (0x00000009) exists archived, archive_id:13
R1_335.gz: (0x00000009) exists archived, archive_id:13
R1_336.gz: (0x00000009) exists archived, archive_id:13
R1_337.gz: (0x00000009) exists archived, archive_id:13
R1_338.gz: (0x00000009) exists archived, archive_id:13
R1_339.gz: (0x00000009) exists archived, archive_id:13
R1_33.gz: (0x00000009) exists archived, archive_id:13
R1_340.gz: (0x00000009) exists archived, archive_id:13
R1_341.gz: (0x00000009) exists archived, archive_id:13
R1_342.gz: (0x00000009) exists archived, archive_id:13
R1_343.gz: (0x00000009) exists archived, archive_id:13
R1_344.gz: (0x00000009) exists archived, archive_id:13
R1_345.gz: (0x00000009) exists archived, archive_id:13
R1_346.gz: (0x00000009) exists archived, archive_id:13
R1_347.gz: (0x00000009) exists archived, archive_id:13
R1_348.gz: (0x00000009) exists archived, archive_id:13
R1_349.gz: (0x00000009) exists archived, archive_id:13
R1_34.gz: (0x00000009) exists archived, archive_id:13
R1_350.gz: (0x00000009) exists archived, archive_id:13
R1_351.gz: (0x00000009) exists archived, archive_id:13
R1_352.gz: (0x00000009) exists archived, archive_id:13
R1_353.gz: (0x00000009) exists archived, archive_id:13
R1_354.gz: (0x00000009) exists archived, archive_id:13
R1_355.gz: (0x00000009) exists archived, archive_id:13
R1_356.gz: (0x00000009) exists archived, archive_id:13
R1_357.gz: (0x00000009) exists archived, archive_id:13
R1_358.gz: (0x00000009) exists archived, archive_id:13
R1_359.gz: (0x00000009) exists archived, archive_id:13
R1_35.gz: (0x00000009) exists archived, archive_id:13
R1_360.gz: (0x00000009) exists archived, archive_id:13
R1_361.gz: (0x00000009) exists archived, archive_id:13
R1_362.gz: (0x00000009) exists archived, archive_id:13
R1_363.gz: (0x00000009) exists archived, archive_id:13
R1_364.gz: (0x00000009) exists archived, archive_id:13
R1_365.gz: (0x00000009) exists archived, archive_id:13
R1_366.gz: (0x00000009) exists archived, archive_id:13
R1_367.gz: (0x00000009) exists archived, archive_id:13
R1_368.gz: (0x00000009) exists archived, archive_id:13
R1_369.gz: (0x00000009) exists archived, archive_id:13
R1_36.gz: (0x00000009) exists archived, archive_id:13
... 2178 more lines ...
  521   Sun Oct 1 11:47:07 2023 TDS4 cooling water
The photographs show the cooling water controls and temperature/pressure gauges outside S4 and the connections used by AIDA within S4.
Attachment 1: 1000007356.jpg
1000007356.jpg
Attachment 2: 1000007355.jpg
1000007355.jpg
  58   Mon Aug 5 12:57:48 2019 NHS4 Conditions

Temp: 26 C
Humidity: 50%
Dew Point: ~15 C

There are two temperature dials outside S4, one shows exactly 20 C and one shows nearer 16 C.

Trying to clarify this discrepancy as 16 C is too cold at the moment.

In contact with developer of cooling system to confirm apparent 16 C after the heat exchanger

Today's conditions:

Temp: 26.9 C
Humidity: 48.7%
Dewpoint 15.2 C

ELOG V3.1.4-unknown