00:10 DAQ continues OK - file R30_372
ASIC settings 2019Dec19-16.19.51
DSSSD#1 slow comparator 0xa
DSSSD#2 slow comparator 0xa
DSSSD#3 slow comparator 0xd
BNC PB-5 Pulser
Amplitude1.0V
Attenuation x1
Frequency 2Hz
tau_d 1ms
- polarity
Delay 250ns, tail pulse
00:17 System wide checks all OK
FEE64 Temperatures ok - attachment 1
Good event statistics ok - attachment 2
detector bias and leakage currents ok - attachment 3
00:20 DESPEC on run 145
00:24 Merger ok ~45M items/s
Tapeserver ok ~15MB/s
01:00 aida07 crashed
recovered, but all FEE64 crash shortly after
called OH - stop DAQ, TapeService, Merger
01:20 all FEE64 powercycled, AIDA restart
01:30 AIDA recovered, DAQ now running - writing to R33
system wide checks ok *except*
FEE64 module aida09 global clocks failed, 6
Clock status test result: Passed 11, Failed 1
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable
FEE64 module aida07 failed
FEE64 module aida09 failed
Calibration test result: Passed 10, Failed 2
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
01:52 FEE64 Temperatures ok - attachment 4
-had to reload a few times to work, but otherwise ok
Good event statistics - attachment 5
-aida05 and aida08 running faster than before
detector bias and leakage currents ok - attachment 6
02:04 writing to file R33_34
Data forwarding to MBS ok
AIDA ASIC settings ok
02:11 beam off
02:13 attempted to recalibrate aida07 and aida09 in FADC Align and Control - calibration still fails
02:32 beam back - writing to file R33_48
04:30 DESPEC having issues with Go4 crashing
ucesb reports AIDA/FATIMA/bplast timewarp events
System wide checks:
Clock error:
FEE64 module aida09 global clocks failed, 6
Clock status test result: Passed 11, Failed 1
Calibration:
FEE64 module aida07 failed
FEE64 module aida09 failed
Calibration test result: Passed 10, Failed 2
White Rabbit:
Base Current Difference
aida07 fault 0xcdd1 : 0xcdd2 : 1
White Rabbit error counter test result: Passed 11, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
FEE64 Temperatures ok - attachment 7
Good event statistics ok - attachment 8
detector bias and leakage currents ok - attachment 9
04:52 analyzer output for R33_113 - attachment 10
no timewarps
aida06 dead time? (ignore idle time and rates)
05:02 Merger 5M data items/s
TapeServer 17 MB/s
05:15 DESPEC believe issue with bplast TAMEX causing ucesb issues/Go4 crash
They disable bplast and FATIMA TAMEX histograms in their online analysis - ucesb/go4 much more stable now
05:25 rates spectra - attachment 11
05:29 error message in MBS relay terminal - otherwise data forwarding to MBS ok - attachment 12
06:29 system wide checks:
Clock error:
FEE64 module aida09 global clocks failed, 6
Clock status test result: Passed 11, Failed 1
Calibration:
FEE64 module aida07 failed
FEE64 module aida09 failed
Calibration test result: Passed 10, Failed 2
White Rabbit:
Base Current Difference
aida05 fault 0x7a56 : 0x7a58 : 2
aida06 fault 0x65bb : 0x65bd : 2
aida07 fault 0xcdd1 : 0xcdd4 : 3
aida08 fault 0x2ab3 : 0x2ab5 : 2
White Rabbit error counter test result: Passed 8, Failed 4
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
rest ok
06:32 FEE64 Temperatures ok - attachment 13
Good event statistics ok - attachment 14
detector bias and leakage currents ok - attachment 15
06:41 Merger 5.1M data items/s
TapeServer 17 MB/s
Data forwarding to MBS ok
08:54 restart MBS relay, requested by NH
|