 Fri May 14 07:05:08 2021, CA, Friday 14th May 08:00 - 16:00 shift 24x
|
08:00 CA takes over
08:28 Problem at UNILAC - no beam
09:30 all system wide checks ok *except*
all FEE64 fail ADC calibration (no waveforms)
WR decoder status:
Base Current Difference
aida01 fault 0xf932 : 0xf933 : 1
aida02 fault 0x62ec : 0x62ed : 1
aida03 fault 0x8679 : 0x867a : 1
aida04 fault 0xf0e4 : 0xf0e5 : 1
aida05 fault 0x9db8 : 0x9dc2 : 10
aida06 fault 0x7f18 : 0x7f19 : 1
aida07 fault 0xdd2c : 0xdd2d : 1
aida08 fault 0x1557 : 0x1558 : 1
White Rabbit error counter test result: Passed 8, Failed 8
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
FPGA timestamp errors:
Base Current Difference
aida05 fault 0x0 : 0x2 : 2
FPGA Timestamp error counter test result: Passed 15, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
09:36 FEE64 Temperatures ok - attachment 1
Detector bias / leakage currents ok - attachment 2
Statistics - attachments 3-8
09:50 beam is back - writing to file R4_265
09:56 FRS adjusting degrader settings (S4) - temporarily remove to check counts in scintillator vs AIDA
10:10 FRS increase degrader thickness
10.28 all histograms, stats and merger stats zero'd
10.50 all system wide checks ok *except*
Base Current Difference
aida01 fault 0xf932 : 0xf933 : 1
aida02 fault 0x62ec : 0x62ed : 1
aida03 fault 0x8679 : 0x867a : 1
aida04 fault 0xf0e4 : 0xf0e5 : 1
aida05 fault 0x9db8 : 0x9dc2 : 10
aida06 fault 0x7f18 : 0x7f19 : 1
aida07 fault 0xdd2c : 0xdd2d : 1
aida08 fault 0x1557 : 0x1558 : 1
White Rabbit error counter test result: Passed 8, Failed 8
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Base Current Difference
aida05 fault 0x0 : 0x3 : 3
FPGA Timestamp error counter test result: Passed 15, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 3 2 2 2 2 2 2 4 3 3 6 : 36860
aida02 : 7 3 5 1 2 5 2 4 3 3 6 : 37284
aida03 : 7 5 3 1 4 4 2 3 3 3 6 : 36756
aida04 : 3 1 20 4 2 3 3 5 1 3 6 : 36052
aida05 : 19 25 26 6 2 2 3 2 2 3 6 : 35828
aida06 : 1 3 16 1 3 4 1 5 1 3 6 : 35580
aida07 : 6 4 1 2 3 3 2 3 3 3 6 : 36552
aida08 : 8 5 2 3 2 3 2 2 2 4 6 : 37064
aida09 : 9 7 1 2 2 4 2 3 3 3 6 : 36652
aida10 : 2 5 3 1 1 4 2 3 3 3 6 : 36544
aida11 : 16 4 4 2 1 2 2 3 3 3 6 : 36384
aida12 : 2 1 1 3 1 2 2 3 3 3 6 : 36288
aida13 : 0 1 1 2 0 4 3 2 3 3 6 : 36184
aida14 : 6 3 2 0 2 3 2 3 3 3 6 : 36432
aida15 : 25 8 2 1 2 3 1 2 2 4 6 : 36836
aida16 : 3 5 1 2 2 2 3 2 3 3 6 : 36100
Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same
FEE : aida01 => Options file size is 1026 Last changed Thu May 13 05:49:42 CEST 2021
FEE : aida02 => Options file size is 1014 Last changed Thu Apr 29 14:43:46 CEST 2021
FEE : aida03 => Options file size is 1014 Last changed Thu Apr 29 14:43:50 CEST 2021
FEE : aida04 => Options file size is 1014 Last changed Thu Apr 29 14:43:53 CEST 2021
FEE : aida05 => Options file size is 1014 Last changed Thu Apr 29 14:43:55 CEST 2021
FEE : aida06 => Options file size is 1014 Last changed Thu Apr 29 14:43:59 CEST 2021
FEE : aida07 => Options file size is 1014 Last changed Thu Apr 29 14:44:02 CEST 2021
FEE : aida08 => Options file size is 1025 Last changed Wed May 05 12:15:54 CEST 2021
FEE : aida09 => Options file size is 1014 Last changed Thu Apr 29 14:44:08 CEST 2021
FEE : aida10 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida11 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida12 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida13 => Options file size is 1025 Last changed Fri May 07 19:40:34 CEST 2021
FEE : aida14 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida15 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida16 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
10.50 DAQ continues S496/R4_304
11:18 FEE64 Temperatures ok - attachment 9
Detector bias & leakage currents ok - attachment 10
statistics - attachments 11-16
15:00 FEE64 avg. CPU usage (%)
1 52
2 55
3 55
4 60
5 93
6 50
7 53
8 55
9 55
10 51
11 50
12 67
13 56
14 55
15 55
16 58
15:08 all system wide checks ok *except*
all FEE64 fail ADC calibration (no waveform)
WR decoder status:
Base Current Difference
aida01 fault 0xf932 : 0xf933 : 1
aida02 fault 0x62ec : 0x62ed : 1
aida03 fault 0x8679 : 0x867a : 1
aida04 fault 0xf0e4 : 0xf0e5 : 1
aida05 fault 0x9db8 : 0x9dc3 : 11
aida06 fault 0x7f18 : 0x7f19 : 1
aida07 fault 0xdd2c : 0xdd2d : 1
aida08 fault 0x1557 : 0x1558 : 1
White Rabbit error counter test result: Passed 8, Failed 8
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
FPGA errors:
Base Current Difference
aida05 fault 0x0 : 0x3 : 3
FPGA Timestamp error counter test result: Passed 15, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
15:10 Temperatures ok - attachment 17
Detector bias / leakage current ok - attachment 18
Statistics - attachments 18 -24 |
|
Tue May 18 23:11:40 2021, CA, Wednesday May 19th 00:00 - 08:00 14x
|
00:00 CA takes over
LS has performed usual checks, see previous Elog entry - all ok
00:30 stats ok
R14_470 deadtime at 4.4%
02:10 system wide checks ok except:
all ADC fail calibration (expected, no waveforms)
WR decoder status
Base Current Difference
aida05 fault 0x500 : 0x52f : 47
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
02:12 FEE64 Temps ok - attachment 1
Stats ok - attachment 2
detector bias / leakage currents ok - attachment 3
dead-times ok - R14_497 max 4% - attachment 4
03:27 DAQ continues ok - writing to file R14_517
04:07 system wide checks ok except:
all ADC fail calibration (expected, no waveforms)
WR decoder status
Base Current Difference
aida05 fault 0x500 : 0x537 : 55
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
FEE64 Temps ok - attachment 6
Stats ok - attachment 5
detector bias / leakage currents ok - attachment 7
dead-times ok - R14_526 max 4% - attachment 8
05:27 bad timestamp error in merger terminal - attachment 9
06:05 all system wide checks ok except;
all ADC fail calibration (expected, no waveforms)
WR decoder status
Base Current Difference
aida05 fault 0x500 : 0x53d : 61
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
06:07 FEE64 Temps ok - attachment 10
Stats ok - attachment 11
detector bias / leakage currents ok - attachment 12
dead-times ok - R14_558 max 3.6% - attachment 13
07:01 another bad timestamp error in merger terminal - attachment 14 |
|
Wed May 19 09:06:58 2021, CA, May 16th 00:00 - 08:00 16x
|
16th May 2021 - 00:00 - 08:00 Shift
Author: CA
00:15 System wide checks;
FEE64 module aida06 global clocks failed, 6
FEE64 module aida09 global clocks failed, 6
Clock status test result: Passed 14, Failed 2
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable
FEE64 module aida09 failed
Calibration test result: Passed 15, Failed 1
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
Base Current Difference
aida05 fault 0x714c : 0x7154 : 8
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Base Current Difference
aida05 fault 0x0 : 0x3 : 3
aida12 fault 0x0 : 0x3 : 3
aida13 fault 0x0 : 0x4d : 77
FPGA Timestamp error counter test result: Passed 13, Failed 3
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 19 10 3 1 2 3 2 3 2 3 7 : 39660
aida02 : 29 5 3 1 1 3 2 3 2 3 7 : 39596
aida03 : 15 7 1 1 2 3 2 3 2 3 7 : 39588
aida04 : 44 26 19 5 2 4 3 3 2 2 7 : 38608
aida05 : 20 10 3 0 1 4 2 2 3 3 7 : 40208
aida06 : 27 6 2 1 2 4 2 4 2 3 7 : 40284
aida07 : 25 5 3 1 2 3 2 3 2 3 7 : 39644
aida08 : 23 8 2 3 1 4 2 3 2 3 7 : 39772
aida09 : 23 10 3 2 3 4 1 3 2 3 7 : 39644
aida10 : 18 10 3 2 3 2 2 2 2 4 7 : 41160
aida11 : 16 7 3 2 1 2 2 3 2 3 7 : 39464
aida12 : 21 4 7 1 3 3 3 3 2 3 7 : 40004
aida13 : 25 6 3 2 1 3 1 4 2 3 7 : 39876
aida14 : 20 8 10 1 1 2 2 2 2 4 7 : 41104
aida15 : 26 6 3 3 0 4 3 2 2 3 7 : 39464
aida16 : 9 8 2 4 1 3 1 4 2 3 7 : 39876
00:30 Stats ok - 210516_0030_stats.png
Temps ok - 210516_0030_temps.png
Bias ok - 210516_0030_bias.png
00:38 analyzer output R7_178 - Deadtime at 7.3% - 210516_R7_178_analysis.txt
02:21 system wide checks as above, except;
Base Current Difference
aida05 fault 0x714c : 0x7159 : 13
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Base Current Difference
aida05 fault 0x0 : 0x7 : 7
aida12 fault 0x0 : 0x3 : 3
aida13 fault 0x0 : 0x4d : 77
FPGA Timestamp error counter test result: Passed 13, Failed 3
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
02:28 Stats ok - 210516_0230_stats.png
Temps ok - 210516_0230_temps.png
Bias ok - 210516_0230_bias.png
02:33 analyzer output R7_216 - Deadtime at 6.5% - 210516_R7_216_analysis.txt
04:36 system wide checks same as above, except;
Base Current Difference
aida05 fault 0x714c : 0x715c : 16
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Base Current Difference
aida05 fault 0x0 : 0x9 : 9
aida12 fault 0x0 : 0x3 : 3
aida13 fault 0x0 : 0x4d : 77
FPGA Timestamp error counter test result: Passed 13, Failed 3
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
04:37 Stats ok - 210516_0430_stats.png
Temps ok - 210516_0430_temps.png
Bias ok - 210516_0430_bias.png
04:44 analyzer output R7_260 - Deadtime at 6.9% - 210516_R7_260_analysis.txt
06:38 system wide checks same as before, except;
Base Current Difference
aida01 fault 0xf294 : 0xf296 : 2
aida02 fault 0xd8ec : 0xd8ee : 2
aida03 fault 0xf001 : 0xf003 : 2
aida04 fault 0xd992 : 0xd994 : 2
aida05 fault 0x714c : 0x7161 : 21
aida06 fault 0x5a49 : 0x5a4a : 1
aida07 fault 0x5aca : 0x5acb : 1
aida08 fault 0xb92e : 0xb92f : 1
White Rabbit error counter test result: Passed 8, Failed 8
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
06:40 Stats ok - 210516_0640_stats.png
Temps ok - 210516_0640_temps.png
Bias ok - 210516_0640_bias.png
06:45 analyzer output R7_298 - Deadtime at 5.8% - 210516_R7_298_analysis.txt |
|
Thu May 20 07:03:02 2021, CA, Thursday May 20th 08:00 - 16:00 shift 40x
|
08:00 CA takes over
08:04 DAQ continues ok - writing to file R14_989
09:30 all system wide checks ok *except* WR decoder status;
Base Current Difference
aida05 fault 0x500 : 0x56a : 106
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
09:34 temperatures ok - attachment 1
stats ok - attachment 2
bias/leakage currents ok - attachment 3
R14_1016 analysis - max deadtime ~4.3% - attachment 4
11:30 all system wide checks ok *except* WR decoder status;
Base Current Difference
aida05 fault 0x500 : 0x56b : 107
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
11:34 temperatures ok - attachment 5
stats ok - attachment 6
bias/leakage currents ok - attachment 7
R14_1052 analysis - max deadtime ~3.6% - attachment 8
13:05 beam optimisation currently taking place
DAQ continues ok and writing to file R14_1085
13:17 no beam
13:28 all system wide checks ok, WR decoder status difference same as entry at 11:30.
13:31 temperatures ok - attachment 9
stats ok - attachment 10
bias/leakage currents ok - attachment 11
R14_1085 analysis (beam off) - max deadtime ~1.34% - attachment 12
13:37 beam is back - rate spectra - attachment 13
DAQ ok - writing to file R14_1088
13:43 analysis of R14_1088 - max deadtime back up to ~4.5% - attachment 14
14:21 beam off - issue with UNILAC
DESPEC also report issues with their DAQ
AIDA continues to forward data to MBS ok - lower rates while beam off
14.30 1.8.H spectra - attachments 16-18
1.8.L spectra - attachments 19-22
aida01 pulser peak width 99 ch FWHM
Rate spectra - attachment 23
Grafana - DSSSD bias & leakage currents past 7 days - attachment 24
DSSSD bias & leakage currents - attachment 25
Merger/Tape Server/Merger stats - attachment 26
14.41 All histograms & stats zero'd
14.55 Lost Activity Monitors - attachments 27, 28 & 35
Stats - ADC data items, disc, good wave, WR 28-47, pause, correlation scaler
15:00 beam is back
15:05 AIDA writing to file R14_1108
ADC data items - attachment 36
15:27 system wide checks ok *except* WR decoder status
Base Current Difference
aida05 fault 0x500 : 0x56c : 108
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
15:30 temperatures ok - attachment 37
stats ok - attachment 38
bias/leakage currents ok - attachment 39
R14_1113 analysis (beam off) - max deadtime ~4% - attachment 40 |
|
Fri May 21 15:03:48 2021, CA, Friday 21st May 16:00 - 00:00 shift 17x
|
16:00 CA takes over
DSSD1 n+n sides 0x20 on ASICs 1-3 and 0x64 on ASIC 4
DSSD2 n+n sides 0x1b on ASICs 1-3 and 0x64 on ASIC 4
PULSER SETTINGS
---------------------------
Pulse is ON
Positive Tail Pulse
Trigger Source is Internal Clock
Trigger Threshold is 3.5
Amplitude : 2.0 Volts
Rep Rate : 2.0 hZ
Delay : 250.0 ns
Fall Time : 1 ms
Attenuation : 1
Display is : Volts
Equivalent keV is : 200.0
Ramp Start at 0.01 Volts
Ramp Stop at 9.99 Volts
Ramp Start at 1.0 keV
Ramp Stop at 999.0 keV
Ramp Time is 60 seconds
# Ramp Cycles is 1
16:14 system wide checks all ok *except*
Base Current Difference
aida05 fault 0x500 : 0x570 : 112
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
16:17 temperatures ok - attachment 1
statistics ok - attachment 2
detector bias / leakage currents ok - attachment 3
analysis of R14_1625 - FEE7 deadtime at 4.5%
18:06 system wide checks all ok *except*
Base Current Difference
aida05 fault 0x500 : 0x571 : 113
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
18:07 temperatures ok - attachment 4
statistics ok - attachment 5
detector bias / leakage currents ok - attachment 6
analysis of R14_1662 - FEE7 deadtime at 4.3%
19:07 DAQ continues ok - writing to file R14_1682
20:07 system wide checks all ok *except*
Base Current Difference
aida05 fault 0x500 : 0x572 : 114
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
20:10 temperatures ok - attachment 7
statistics ok - attachment 8
detector bias / leakage currents ok - attachment 9
analysis of R14_1703 - max deadtime at 4.0% - attachment 10
20:35 rate spectra - attachment 11
HEC spectra - attachment 12 & 13
20:36 all histograms zeroed
20:37 DAQ continues ok - writing to file R14_1712
20:45 /media/1e121361-83d3-4825-b6ae-9700b07e0ca7 at 94% use.
~ 12 hours until 100% at 10 MB/s data rate, not accounting for compression of older files
21:12 no beam - writing to file R14_1723
21:17 beam is back
22:07 system wide checks all ok - WR decoder fault same as entry at 20:07
22:08 temperatures ok - attachment 14
statistics ok - attachment 15
detector bias / leakage currents ok - attachment 16
analysis of R14_1740 - max deadtime at 4.2% - attachment 17
|
|
Sun Apr 28 15:09:34 2024, Betool Alayed, 16:0-00:00 28 April 2024
|
16:00 aida04 not producing data
power cycle all FEE64s to recover DAQ
16.20 analysis data file R21_668
max deadtime 17% (aida04), 9% (aida02), 2% (aida06) all others < 1%
no timewarps
HEC data item rate 1.9kHz
|
|
Sun Apr 28 16:26:47 2024, Betool Alayed, 16:00-00:00 Sunday 28 April 10x
|
19:24 pm screenshots
and
17:18 pm screenshots |
|
Sun Apr 28 20:35:57 2024, Betool Alayed, 16:0-00:00 28 April 2024 10x
|
21:30 screenshots
and
23:24 screenshots |
|
Wed Jun 12 15:41:07 2024, Betool Alayed, 16:00-00:00 Wed 12 Jun 24 27x
|
5pm full checks:
DSSSD bias & leakage current ok - attachment # 1-2
FEE64 temperatures ok - attachment # 3
ADC data item stats - attachment # 4
Merger ok - Attachement # 5
Tape service - attachement # 6
7pm full checks:
DSSSD bias & leakage current ok - attachment # 7-8
FEE64 temperatures ok - attachment # 9
ADC data item stats - attachment # 10
Merger ok - Attachement # 11
Tape service - attachement # 12
ucesb - attachment # 13
9pm full checks:
DSSSD bias & leakage current ok - attachment# 14-15
FEE64 temperatures ok - attachment # 16
ADC data item stats - attachment # 17
Merger ok - Attachement # 18
Tape service - attachement # 19
ucesb - attachment # 20
11pm full checks:
DSSSD bias & leakage current ok - attachment# 21-22
FEE64 temperatures ok - attachment # 23
ADC data item stats - attachment # 24
Merger ok - Attachement # 25
Tape service - attachement # 26
ucesb - attachment # 27 |
|
Sun Apr 18 00:57:36 2021, BA, MA, Sanday 18 April 00.00-08.00 12x
|
02:01 Beam has stopped at 01:37 and returend at 01:43 for few min and then stopped again and not knowen how it will take until it is back
AIDA scalers attached 1
statistic attached 2
temretuer attached 3
bias attached 4
Clock check ok
ADC check :
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida10 failed
Calibration test result: Passed 9, Failed 3
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
Base Current Difference
aida05 fault 0x1a52 : 0x1a55 : 3
aida06 fault 0x4f3e : 0x4f41 : 3
aida07 fault 0x3bcd : 0x3bea : 29
aida08 fault 0xc7c7 : 0xc7ca : 3
White Rabbit error counter test result: Passed 8, Failed 4
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Base Current Difference
aida09 fault 0x0 : 0x1 : 1
aida12 fault 0x0 : 0x9 : 9
FPGA Timestamp error counter test result: Passed 10, Failed 2
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
Returned 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 32 8 3 1 0 3 1 3 3 3 6 : 36240
aida02 : 11 7 9 3 1 2 2 4 2 3 6 : 35988
aida03 : 32 4 13 5 5 4 3 3 2 3 6 : 36432
aida04 : 26 7 6 1 2 3 2 4 2 3 6 : 36128
aida05 : 18 8 8 7 4 2 2 2 2 4 6 : 37352
aida06 : 24 11 3 1 2 5 1 3 3 3 6 : 36616
aida07 : 14 5 3 3 3 2 3 2 4 3 6 : 37296
aida08 : 21 7 1 5 0 1 2 3 3 3 6 : 36284
aida09 : 0 7 1 1 1 3 2 2 3 3 6 : 35880
aida10 : 22 5 3 4 1 2 2 2 3 3 6 : 35952
aida11 : 4 3 1 1 2 2 3 3 2 3 6 : 35544
aida12 : 22 10 5 4 4 4 1 3 3 3 6 : 36728
02:19 beam is back
03:58 The beam has not been stable yet
The rate reach 1.5 kHz, they will contact FRS team to lower the intensity of the beam
AIDA scalers attached 8
statistic attached 7
temretuer attached 6
bias attached 5
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida10 failed
Calibration test result: Passed 9, Failed 3
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
Base Current Difference
aida05 fault 0x1a52 : 0x1a55 : 3
aida06 fault 0x4f3e : 0x4f41 : 3
aida07 fault 0x3bcd : 0x3beb : 30
aida08 fault 0xc7c7 : 0xc7ca : 3
White Rabbit error counter test result: Passed 8, Failed 4
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Base Current Difference
aida09 fault 0x0 : 0x1 : 1
aida12 fault 0x0 : 0x9 : 9
FPGA Timestamp error counter test result: Passed 10, Failed 2
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
Returned 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 21 8 6 2 1 2 1 3 3 3 6 : 36212
aida02 : 24 14 14 2 1 2 2 4 2 3 6 : 36144
aida03 : 31 5 10 5 5 4 2 3 2 3 6 : 36132
aida04 : 12 11 12 2 2 2 3 4 2 3 6 : 36360
aida05 : 23 8 5 7 4 2 2 2 2 4 6 : 37324
aida06 : 21 14 14 1 3 5 1 3 3 3 6 : 36868
aida07 : 15 9 5 0 3 2 3 2 4 3 6 : 37268
aida08 : 21 11 10 3 0 1 2 3 3 3 6 : 36396
aida09 : 5 7 5 1 1 3 1 3 3 3 6 : 36220
aida10 : 15 13 12 3 2 1 2 2 3 3 6 : 36036
aida11 : 13 10 1 0 2 2 2 4 2 3 6 : 35860
aida12 : 29 5 4 5 5 3 1 3 3 3 6 : 36668
05: 07 The rate was a bout 1500 and 2000, we contacted Oscar he said (if it's just bursts it should be ok), so they decided to do nothing.
06:26
AIDA scalers attached 9
statistic attached 10
temretuer attached 11
bias attached 12
Clock status test result: Passed 12, Failed 0
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida10 failed
Calibration test result: Passed 9, Failed 3
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
Base Current Difference
aida01 fault 0x7685 : 0x7686 : 1
aida02 fault 0x941c : 0x941d : 1
aida03 fault 0x7cd6 : 0x7cd7 : 1
aida04 fault 0xb86c : 0xb86d : 1
aida05 fault 0x1a52 : 0x1a59 : 7
aida06 fault 0x4f3e : 0x4f45 : 7
aida07 fault 0x3bcd : 0x3bf9 : 44
aida08 fault 0xc7c7 : 0xc7ce : 7
aida09 fault 0xb33a : 0xb33b : 1
White Rabbit error counter test result: Passed 3, Failed 9
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Base Current Difference
aida09 fault 0x0 : 0x1 : 1
aida12 fault 0x0 : 0xa : 10
FPGA Timestamp error counter test result: Passed 10, Failed 2
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
Returned 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 17 12 6 2 1 2 1 3 3 3 6 : 36228
aida02 : 19 11 12 3 2 2 2 4 2 3 6 : 36164
aida03 : 25 8 12 5 4 4 3 3 2 3 6 : 36356
aida04 : 22 19 11 3 1 2 3 4 2 3 6 : 36416
aida05 : 35 6 5 6 3 2 2 2 3 3 6 : 36236
aida06 : 12 11 11 2 3 4 1 3 3 3 6 : 36664
aida07 : 18 6 2 1 3 2 3 2 3 3 6 : 36216
aida08 : 27 10 8 3 0 1 2 3 3 3 6 : 36380
aida09 : 18 6 6 1 0 2 2 2 3 3 6 : 35832
aida10 : 3 14 10 3 1 1 2 3 3 3 6 : 36412
aida11 : 1 3 2 0 2 2 2 4 2 3 6 : 35772
aida12 : 0 5 6 3 3 4 1 3 3 3 6 : 36520
07:57 The beam stopped and they said there is a water leak !
|
|
Thu May 12 21:43:49 2022, BA, MA, Rates
|
Statistic check (screenshot attached).
Temperatures OK (screenshot attached).
Bias and leakage currents OK (same screenshot as temps).
|
|
Thu May 12 22:49:39 2022, BA, MA, Low Rates
|
The rate is very low in all aida, not sure . |
|
Sat May 14 15:35:24 2022, BA, AA, Saturday 14 May
|
| |
|
Sat May 14 17:34:40 2022, BA, AA, Saturday 14 May
|
FEE64 module aida09 global clocks failed, 6
Clock status test result: Passed 13, Failed 1
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable
FEE64 module aida02 failed
FEE64 module aida06 failed
FEE64 module aida09 failed
FEE64 module aida10 failed
FEE64 module aida13 failed
Calibration test result: Passed 9, Failed 5
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
White Rabbit error counter test result: Passed 14, Failed 0
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
FPGA Timestamp error counter test result: Passed 14, Failed 0
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
|
|
Sat May 14 19:39:13 2022, BA, AA, Saturday 14 May
|
FEE64 module aida09 global clocks failed, 6
Clock status test result: Passed 13, Failed 1
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable
FEE64 module aida02 failed
FEE64 module aida06 failed
FEE64 module aida09 failed
FEE64 module aida10 failed
FEE64 module aida13 failed
Calibration test result: Passed 9, Failed 5
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
White Rabbit error counter test result: Passed 14, Failed 0
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
FPGA Timestamp error counter test result: Passed 14, Failed 0
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last |
|
Sat May 14 21:34:11 2022, BA, AA, Saturday 14 May
|
FEE64 module aida09 global clocks failed, 6
Clock status test result: Passed 13, Failed 1
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable
FEE64 module aida02 failed
FEE64 module aida06 failed
FEE64 module aida09 failed
FEE64 module aida10 failed
FEE64 module aida13 failed
Calibration test result: Passed 9, Failed 5
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
White Rabbit error counter test result: Passed 14, Failed 0
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
FPGA Timestamp error counter test result: Passed 14, Failed 0
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last |
|
Sat Jun 25 17:06:42 2022, BA, AA, Saturday 25 June 2022 16:00-00:00 9x
|
Took over the night shift from Magda
18:00 : attachments 1-3
20:00 : attachments 4-6
22:00 : attachments 7-9
00:00 : attachments 10-12
|
|
Wed Jun 22 23:18:45 2022, AM, OH, TD, MA, Thursday 23 June 19x
|
00:00 TD Noticed on the last stats uploaded that AIDA08 had stopped sending signals.
Attempted to recover restarting merger but this caused 01 to drop out.
00:10 DAQ recovered and AIDA01 recovered with a reboot
01:00 Attachments 1-4, white rabbit failures on aida07 and aida08, otherwise all good
03:00 Attachments 5-9, aida07 and aida08 failures on white rabbit and fpga timestamp, otherwise all good
05:00 Attachments 10-14, aida07 and aida08 failures on white rabbit and fpga timestamp continue, otherwise all good
07:00 Attachments 15-19, aida07 and aida08 failures on white rabbit and fpga timestamp continue, otherwise all good |
|
Mon May 16 23:02:54 2022, AM, Tuesday 17th May 00:00-08:00 12x
|
01:00: Attachments 1-3, System checks good
03:00: Attachments 4-6, System checks good
05:00: Attachments 7-9, System checks good
07:00: Attachments 10-12, System checks good |
|
Fri Jun 24 00:34:55 2022, AM, Friday 24 June 00:00-08:00 20x
|
01:30 Attachments 1-5, white rabbit and fpga timestamp failures, otherwise all good
03:30 Attachments 11-15, white rabbit and fpga timestamp failures, otherwise all good
05:30 Attachments 6-10, white rabbit and fpga timestamp failures, otherwise all good
07:30 Attachments 16-20, white rabbit and fpga timestamp failures, otherwise all good |
|