 Thu Nov 6 12:58:09 2025, NH, 207Bi source test - data taking 6x
|
Restart AIDA
aida06 took three attempts to boot today :(
All system checks OK, ASICs aligned
Run DAQ without merger - looks okay, all FEEs working
When connecting to merger aida15 drops connection, solution restart merger
Aida13 not sending data - do an ASIC check/load
Finally all DAQs send data, merger is happy with all 16 FEEs
Open run R6 on disk |
|
Thu May 13 17:05:42 2021, Muneerah, MArc, Thu 13 May 04:00-00:00
|
19:08 System Check
Rates from ucesb below 1 kHz
Atachment 1 Stats
Atachment 2 Temperatures
Atachment 3 Voltages
System wide checks are all ok except
ADC calibration all failed
FEE64 module aida01 failed
FEE64 module aida02 failed
FEE64 module aida03 failed
FEE64 module aida04 failed
FEE64 module aida05 failed
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida08 failed
FEE64 module aida09 failed
FEE64 module aida10 failed
FEE64 module aida11 failed
FEE64 module aida12 failed
FEE64 module aida13 failed
FEE64 module aida14 failed
FEE64 module aida15 failed
FEE64 module aida16 failed
Calibration test result: Passed 0, Failed 16
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
White rabbit
Base Current Difference
aida01 fault 0xf932 : 0xf933 : 1
aida02 fault 0x62ec : 0x62ed : 1
aida03 fault 0x8679 : 0x867a : 1
aida04 fault 0xf0e4 : 0xf0e5 : 1
aida05 fault 0x9db8 : 0x9dba : 2
aida06 fault 0x7f18 : 0x7f19 : 1
aida07 fault 0xdd2c : 0xdd2d : 1
aida08 fault 0x1557 : 0x1558 : 1
White Rabbit error counter test result: Passed 8, Failed 8
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
FPGA all passed
Memory information
Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 6 2 2 1 2 4 3 4 2 3 7 : 40424
aida02 : 3 4 2 2 1 2 1 3 3 3 7 : 40140
aida03 : 10 10 0 3 2 4 2 4 2 3 7 : 40280
aida04 : 3 2 0 2 2 3 2 3 1 4 7 : 40540
aida05 : 4 9 2 3 2 4 3 3 1 4 7 : 41048
aida06 : 8 6 1 1 2 3 2 2 3 3 7 : 40064
aida07 : 1 4 0 1 1 3 1 3 3 3 7 : 40196
aida08 : 1 4 2 2 2 4 3 4 2 3 7 : 40452
aida09 : 8 5 2 2 1 2 1 3 3 3 7 : 40168
aida10 : 22 5 4 2 1 3 1 4 2 3 7 : 39872
aida11 : 2 1 2 1 2 2 3 3 2 3 7 : 39632
aida12 : 8 5 2 2 1 3 1 4 2 3 7 : 39784
aida13 : 18 8 2 1 0 2 3 3 2 3 7 : 39624
aida14 : 23 12 2 2 1 2 2 4 2 3 7 : 40028
aida15 : 3 5 3 2 2 3 1 4 2 3 7 : 39844
aida16 : 0 2 1 2 1 4 2 4 2 3 7 : 40096
|
|
Wed May 19 09:12:52 2021, Muneerah, Betool, LS, 17th May 00:00-12:00 17x
|
Auther: Muneerah and Betool
02:00 The dead time have been checked every 30 min from the start of the shift, it seemes Ok
Statistics ok - 210517_0202_Stats
Temperatures ok - 210517_0202_Temp
Bias and leakage ok - 210517_0200_Bias
02:05 System wide checks:
Clock status test result: Passed 16, Failed 0
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable
FEE64 module aida02 failed
FEE64 module aida03 failed
FEE64 module aida06 failed
FEE64 module aida10 failed
Calibration test result: Passed 12, Failed 4
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
Base Current Difference
aida05 fault 0x1552 : 0x155d : 11
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Base Current Difference
aida05 fault 0x0 : 0x2 : 2
FPGA Timestamp error counter test result: Passed 15, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 5 8 3 3 3 5 2 2 2 3 7 : 39460
aida02 : 22 4 5 0 3 2 2 3 2 3 7 : 39560
aida03 : 21 10 2 1 1 3 2 4 2 3 7 : 40100
aida04 : 18 7 2 0 3 3 3 2 2 3 7 : 39392
aida05 : 16 6 5 1 2 4 3 2 2 3 7 : 39520
aida06 : 18 6 6 2 2 2 2 3 2 3 7 : 39576
aida07 : 12 8 6 1 2 3 1 3 2 3 7 : 39408
aida08 : 13 11 4 0 1 3 3 2 2 3 7 : 39308
aida09 : 10 8 4 2 3 4 2 3 2 3 7 : 39848
aida10 : 16 8 2 1 2 4 2 3 1 4 7 : 40768
aida11 : 26 10 2 2 1 2 2 3 2 3 7 : 39512
aida12 : 16 9 2 1 2 4 2 2 2 3 7 : 39240
aida13 : 15 9 3 2 2 4 2 2 2 3 7 : 39284
aida14 : 25 10 2 3 3 3 2 3 2 3 7 : 39796
aida15 : 4 2 2 1 1 3 2 2 2 3 7 : 38944
aida16 : 16 7 3 3 1 3 3 2 2 3 7 : 39368
04:00 Statistics ok - 210517_0403_Stats
Temperatures ok - 210517_0403_Temp
Bias and leakage ok - 210517_0400_Bias
04:05 System wide checks:
Clock status test result: Passed 16, Failed 0
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable
FEE64 module aida02 failed
FEE64 module aida03 failed
FEE64 module aida06 failed
FEE64 module aida10 failed
Calibration test result: Passed 12, Failed 4
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
Base Current Difference
aida05 fault 0x1552 : 0x1560 : 14
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Base Current Difference
aida05 fault 0x0 : 0x2 : 2
FPGA Timestamp error counter test result: Passed 15, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 15 7 5 2 3 5 2 2 2 3 7 : 39492
aida02 : 23 6 7 2 1 3 2 3 2 3 7 : 39676
aida03 : 17 9 3 2 0 3 2 3 2 3 7 : 39548
aida04 : 16 4 1 2 1 3 3 3 2 3 7 : 39792
aida05 : 22 7 3 3 1 4 3 3 2 3 7 : 40032
aida06 : 18 6 7 3 1 3 2 3 2 3 7 : 39688
aida07 : 18 7 4 1 2 4 1 4 2 3 7 : 40032
aida08 : 19 5 4 1 1 3 3 3 2 3 7 : 39828
aida09 : 10 6 7 1 3 4 2 2 2 3 7 : 39336
aida10 : 12 10 2 3 1 4 2 3 1 4 7 : 40768
aida11 : 10 9 4 3 2 1 2 3 2 3 7 : 39440
aida12 : 8 7 6 1 2 4 2 3 2 3 7 : 39768
aida13 : 11 2 6 1 2 3 2 3 2 3 7 : 39612
aida14 : 27 7 4 2 3 3 2 3 2 3 7 : 39780
aida15 : 4 2 3 1 1 3 2 2 2 3 7 : 38960
aida16 : 18 8 4 2 1 4 2 2 2 3 7 : 39240
06:27 aida07 has dropped out and lost connection with the merger. Powercycling AIDA
06:48 Restarted system but the noise was extremely high. Equivalent to yesterday 15:00. Restarting again.
07:01 it is working fine.
07:40 Screen shots after the restart
Statistics ok - 210517_0742_Stats
Temperatures ok - 210517_0743_Temp
Bias and leakage ok - 210517_0740_Bias
System wide checks
FEE64 module aida06 global clocks failed, 6
Clock status test result: Passed 15, Failed 1
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable
FEE64 module aida01 failed
FEE64 module aida02 failed
FEE64 module aida03 failed
FEE64 module aida04 failed
FEE64 module aida05 failed
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida08 failed
FEE64 module aida09 failed
FEE64 module aida10 failed
FEE64 module aida11 failed
FEE64 module aida12 failed
FEE64 module aida13 failed
FEE64 module aida14 failed
FEE64 module aida15 failed
FEE64 module aida16 failed
Calibration test result: Passed 0, Failed 16
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
Base Current Difference
aida05 fault 0x61da : 0x61dd : 3
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
FPGA Timestamp error counter test result: Passed 16, Failed 0
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 20 6 3 1 1 3 1 3 3 3 7 : 40336
aida02 : 11 6 0 1 1 5 2 4 2 3 7 : 40252
aida03 : 0 5 2 3 3 3 3 4 2 3 7 : 40424
aida04 : 2 4 2 1 2 4 3 3 2 3 7 : 39912
aida05 : 11 7 5 1 2 2 4 2 3 3 7 : 40532
aida06 : 20 6 0 2 3 2 1 3 3 3 7 : 40320
aida07 : 15 5 1 2 2 3 2 2 3 3 7 : 40116
aida08 : 2 3 4 1 3 3 2 2 3 3 7 : 40128
aida09 : 18 5 1 3 2 4 2 2 3 3 7 : 40288
aida10 : 11 6 3 4 2 3 3 2 2 4 7 : 41484
aida11 : 4 4 5 2 2 2 3 3 2 3 7 : 39744
aida12 : 18 7 2 3 2 2 4 3 2 3 7 : 40064
aida13 : 2 2 1 3 0 3 2 4 2 3 7 : 39944
aida14 : 16 4 5 3 3 3 3 2 2 4 7 : 41552
aida15 : 9 3 1 1 2 3 2 3 3 3 7 : 40556
aida16 : 16 5 1 2 1 2 3 4 2 3 7 : 40184
Date:17/05/2021
Elog for shift: 08.00 to 12.00
Author: Lewis Sexton
Screenshots saved to: /tmp/Elog_210517_0800_1200
***** 08.30 *****
Checked R12_34 through analyser, dead time seems okay:
*** Timestamp elapsed time: 232.154 s
FEE elapsed dead time(s) elapsed idle time(s)
0 0.021 0.000
1 2.781 0.000
2 0.849 0.000
3 5.686 0.000
4 0.000 25.816
5 0.017 0.000
6 0.177 0.000
7 1.349 0.000
8 1.791 0.000
9 0.000 91.660
10 6.194 0.000
11 0.320 0.000
12 1.425 0.000
13 0.000 72.999
14 0.052 0.000
15 1.729 0.000
16 0.000 0.000
17 0.000 0.000
18 0.000 0.000
19 0.000 0.000
20 0.000 0.000
21 0.000 0.000
22 0.000 0.000
23 0.000 0.000
24 0.000 0.000
25 0.000 0.000
26 0.000 0.000
27 0.000 0.000
28 0.000 0.000
29 0.000 0.000
30 0.000 0.000
31 0.000 0.000
32 0.000 0.000
Stats checked seem okay (0833_Stats.png)
Merger rate ~3M items/s
Tape server ~12 MB/s
***** 09.00 *****
Checked R12_48 through analyser, dead time seems okay:
*** Timestamp elapsed time: 170.459 s
FEE elapsed dead time(s) elapsed idle time(s)
0 0.196 0.000
1 6.294 0.000
2 0.992 0.000
3 8.944 0.000
4 0.024 0.000
5 0.001 0.000
6 0.605 0.000
7 2.096 0.000
8 1.654 0.000
9 0.000 31.395
10 6.175 0.000
11 2.524 0.000
12 2.076 0.000
13 0.000 10.337
14 0.103 0.000
15 2.203 0.000
16 0.000 0.000
17 0.000 0.000
18 0.000 0.000
19 0.000 0.000
20 0.000 0.000
21 0.000 0.000
22 0.000 0.000
23 0.000 0.000
24 0.000 0.000
25 0.000 0.000
26 0.000 0.000
27 0.000 0.000
28 0.000 0.000
29 0.000 0.000
30 0.000 0.000
31 0.000 0.000
32 0.000 0.000
Stats checked seem okay (0902_Stats.png)
***** 09.30 *****
AIDA crash aida05 displaying 0 in stats
Power cycle performed with help of TD
Writing to R13 now
MBS relay restared
AIDA back up at 10.20
Ran analyser for R13_16
*** Timestamp elapsed time: 168.059 s
FEE elapsed dead time(s) elapsed idle time(s)
0 0.047 0.000
1 2.282 0.000
2 0.080 0.000
3 0.806 0.000
4 0.000 0.000
5 0.260 0.000
6 0.030 0.000
7 1.471 0.000
8 67.467 0.000
9 80.552 0.000
10 0.057 0.000
11 0.000 11.671
12 1.191 0.000
13 0.080 0.000
14 0.091 0.000
15 0.202 0.000
16 0.000 0.000
17 0.000 0.000
18 0.000 0.000
19 0.000 0.000
20 0.000 0.000
21 0.000 0.000
22 0.000 0.000
23 0.000 0.000
24 0.000 0.000
25 0.000 0.000
26 0.000 0.000
27 0.000 0.000
28 0.000 0.000
29 0.000 0.000
30 0.000 0.000
31 0.000 0.000
32 0.000 0.000
High deadtime in aida09 and aida10, changed slow comparator threshold to 0xe (previously 0xc)
Ran analyser for R13_21 for comparision
*** Timestamp elapsed time: 149.082 s
FEE elapsed dead time(s) elapsed idle time(s)
0 0.000 0.000
1 3.580 0.000
2 0.000 0.000
3 2.800 0.000
4 0.000 0.000
5 0.514 0.000
6 0.140 0.000
7 1.372 0.000
8 55.544 0.000
9 66.588 0.000
10 1.065 0.000
11 0.000 10.010
12 0.681 0.000
13 0.044 0.000
14 0.070 0.000
15 0.400 0.000
16 0.000 0.000
17 0.000 0.000
18 0.000 0.000
19 0.000 0.000
20 0.000 0.000
21 0.000 0.000
22 0.000 0.000
23 0.000 0.000
24 0.000 0.000
25 0.000 0.000
26 0.000 0.000
27 0.000 0.000
28 0.000 0.000
29 0.000 0.000
30 0.000 0.000
31 0.000 0.000
32 0.000 0.000
FEE temp slightly high in aida04, 0.44 above limit of 65 (FEETemps_1042.png)
Merger rate ~5M items/s
Tape server ~17 MB/s
Raised slow comparator in aida09 and aida10 to 0xf, analyse R13_26 for comparision
*** Timestamp elapsed time: 122.147 s
FEE elapsed dead time(s) elapsed idle time(s)
0 0.027 0.000
1 19.945 0.000
2 0.601 0.000
3 21.555 0.000
4 0.042 0.000
5 3.226 0.000
6 1.211 0.000
7 9.674 0.000
8 52.817 0.000
9 54.972 0.000
10 1.848 0.000
11 0.008 0.000
12 2.034 0.000
13 0.278 0.000
14 0.018 0.000
15 0.634 0.000
16 0.000 0.000
17 0.000 0.000
18 0.000 0.000
19 0.000 0.000
20 0.000 0.000
21 0.000 0.000
22 0.000 0.000
23 0.000 0.000
24 0.000 0.000
25 0.000 0.000
26 0.000 0.000
27 0.000 0.000
28 0.000 0.000
29 0.000 0.000
30 0.000 0.000
31 0.000 0.000
32 0.000 0.000
high deatime now also seen in aida02 and aida04
Several powercycles performed as high rates after restart, analysis of R14_13 showed low deadtime for all FEEs
Back running as of 10.45
Analyser of R14_15 to double check
*** Timestamp elapsed time: 205.006 s
FEE elapsed dead time(s) elapsed idle time(s)
0 0.000 40.176
1 0.438 0.000
2 0.199 0.000
3 2.927 0.000
4 0.000 140.347
5 1.282 0.000
6 0.150 0.000
7 2.097 0.000
8 1.673 0.000
9 0.000 87.749
10 1.150 0.000
11 0.489 0.000
12 0.374 0.000
13 0.048 20.995
14 0.251 0.000
15 2.441 0.000
16 0.000 0.000
17 0.000 0.000
18 0.000 0.000
19 0.000 0.000
20 0.000 0.000
21 0.000 0.000
22 0.000 0.000
23 0.000 0.000
24 0.000 0.000
25 0.000 0.000
26 0.000 0.000
27 0.000 0.000
28 0.000 0.000
29 0.000 0.000
30 0.000 0.000
31 0.000 0.000
32 0.000 0.000
12:04 During access Sultan added an additional ground cable and also noted the bias cable going into FEE12 was loose. He pushed it in solidly and the rates appear to have dropped across the FEEs
Date:17/05/2021
Elog for shift: 08.00 to 12.00
Author: Lewis Sexton
Screenshots saved to: /tmp/Elog_210517_0800_1200
***** 08.30 *****
Checked R12_34 through analyser, dead time seems okay:
*** Timestamp elapsed time: 232.154 s
FEE elapsed dead time(s) elapsed idle time(s)
0 0.021 0.000
1 2.781 0.000
2 0.849 0.000
3 5.686 0.000
4 0.000 25.816
5 0.017 0.000
6 0.177 0.000
7 1.349 0.000
8 1.791 0.000
9 0.000 91.660
10 6.194 0.000
11 0.320 0.000
12 1.425 0.000
13 0.000 72.999
14 0.052 0.000
15 1.729 0.000
16 0.000 0.000
17 0.000 0.000
18 0.000 0.000
19 0.000 0.000
20 0.000 0.000
21 0.000 0.000
22 0.000 0.000
23 0.000 0.000
24 0.000 0.000
25 0.000 0.000
26 0.000 0.000
27 0.000 0.000
28 0.000 0.000
29 0.000 0.000
30 0.000 0.000
31 0.000 0.000
32 0.000 0.000
Stats checked seem okay (0833_Stats.png)
Merger rate ~3M items/s
Tape server ~12 MB/s
***** 09.00 *****
Checked R12_48 through analyser, dead time seems okay:
*** Timestamp elapsed time: 170.459 s
FEE elapsed dead time(s) elapsed idle time(s)
0 0.196 0.000
1 6.294 0.000
2 0.992 0.000
3 8.944 0.000
4 0.024 0.000
5 0.001 0.000
6 0.605 0.000
7 2.096 0.000
8 1.654 0.000
9 0.000 31.395
10 6.175 0.000
11 2.524 0.000
12 2.076 0.000
13 0.000 10.337
14 0.103 0.000
15 2.203 0.000
16 0.000 0.000
17 0.000 0.000
18 0.000 0.000
19 0.000 0.000
20 0.000 0.000
21 0.000 0.000
22 0.000 0.000
23 0.000 0.000
24 0.000 0.000
25 0.000 0.000
26 0.000 0.000
27 0.000 0.000
28 0.000 0.000
29 0.000 0.000
30 0.000 0.000
31 0.000 0.000
32 0.000 0.000
Stats checked seem okay (0902_Stats.png)
***** 09.30 *****
AIDA crash aida05 displaying 0 in stats
Power cycle performed with help of TD
Writing to R13 now
MBS relay restared
AIDA back up at 10.20
Ran analyser for R13_16
*** Timestamp elapsed time: 168.059 s
FEE elapsed dead time(s) elapsed idle time(s)
0 0.047 0.000
1 2.282 0.000
2 0.080 0.000
3 0.806 0.000
4 0.000 0.000
5 0.260 0.000
6 0.030 0.000
7 1.471 0.000
8 67.467 0.000
9 80.552 0.000
10 0.057 0.000
11 0.000 11.671
12 1.191 0.000
13 0.080 0.000
14 0.091 0.000
15 0.202 0.000
16 0.000 0.000
17 0.000 0.000
18 0.000 0.000
19 0.000 0.000
20 0.000 0.000
21 0.000 0.000
22 0.000 0.000
23 0.000 0.000
24 0.000 0.000
25 0.000 0.000
26 0.000 0.000
27 0.000 0.000
28 0.000 0.000
29 0.000 0.000
30 0.000 0.000
31 0.000 0.000
32 0.000 0.000
High deadtime in aida09 and aida10, changed slow comparator threshold to 0xe (previously 0xc)
Ran analyser for R13_21 for comparision
*** Timestamp elapsed time: 149.082 s
FEE elapsed dead time(s) elapsed idle time(s)
0 0.000 0.000
1 3.580 0.000
2 0.000 0.000
3 2.800 0.000
4 0.000 0.000
5 0.514 0.000
6 0.140 0.000
7 1.372 0.000
8 55.544 0.000
9 66.588 0.000
10 1.065 0.000
11 0.000 10.010
12 0.681 0.000
13 0.044 0.000
14 0.070 0.000
15 0.400 0.000
16 0.000 0.000
17 0.000 0.000
18 0.000 0.000
19 0.000 0.000
20 0.000 0.000
21 0.000 0.000
22 0.000 0.000
23 0.000 0.000
24 0.000 0.000
25 0.000 0.000
26 0.000 0.000
27 0.000 0.000
28 0.000 0.000
29 0.000 0.000
30 0.000 0.000
31 0.000 0.000
32 0.000 0.000
FEE temp slightly high in aida04, 0.44 above limit of 65 (FEETemps_1042.png)
Merger rate ~5M items/s
Tape server ~17 MB/s
Raised slow comparator in aida09 and aida10 to 0xf, analyse R13_26 for comparision
*** Timestamp elapsed time: 122.147 s
FEE elapsed dead time(s) elapsed idle time(s)
0 0.027 0.000
1 19.945 0.000
2 0.601 0.000
3 21.555 0.000
4 0.042 0.000
5 3.226 0.000
6 1.211 0.000
7 9.674 0.000
8 52.817 0.000
9 54.972 0.000
10 1.848 0.000
11 0.008 0.000
12 2.034 0.000
13 0.278 0.000
14 0.018 0.000
15 0.634 0.000
16 0.000 0.000
17 0.000 0.000
18 0.000 0.000
19 0.000 0.000
20 0.000 0.000
21 0.000 0.000
22 0.000 0.000
23 0.000 0.000
24 0.000 0.000
25 0.000 0.000
26 0.000 0.000
27 0.000 0.000
28 0.000 0.000
29 0.000 0.000
30 0.000 0.000
31 0.000 0.000
32 0.000 0.000
high deatime now also seen in aida02 and aida04
Several powercycles performed as high rates after restart, analysis of R14_13 showed low deadtime for all FEEs
Back running as of 10.45
Analyser of R14_15 to double check
*** Timestamp elapsed time: 205.006 s
FEE elapsed dead time(s) elapsed idle time(s)
0 0.000 40.176
1 0.438 0.000
2 0.199 0.000
3 2.927 0.000
4 0.000 140.347
5 1.282 0.000
6 0.150 0.000
7 2.097 0.000
8 1.673 0.000
9 0.000 87.749
10 1.150 0.000
11 0.489 0.000
12 0.374 0.000
13 0.048 20.995
14 0.251 0.000
15 2.441 0.000
16 0.000 0.000
17 0.000 0.000
18 0.000 0.000
19 0.000 0.000
20 0.000 0.000
21 0.000 0.000
22 0.000 0.000
23 0.000 0.000
24 0.000 0.000
25 0.000 0.000
26 0.000 0.000
27 0.000 0.000
28 0.000 0.000
29 0.000 0.000
30 0.000 0.000
31 0.000 0.000
32 0.000 0.000
12:04 During access Sultan added an additional ground cable and also noted the bias cable going into FEE12 was loose. He pushed it in solidly and the rates appear to have dropped across the FEEs
12:52 Check FPGA Timestamp errors gives following error message:
Got the error Server Internal Error
while trying to obtain /AIDA/Check/Check.tml.
can't read "TS_Base_Collected": no such variable while executing "if { $TS_Base_Collected == 0 } { collect_TS_Base }" (procedure "do_TS_errors" line 8) invoked from within "do_TS_errors " ("TSERRORS" arm line 1) invoked from within "switch -glob $w { RESET {set started 0} CLEAR {EmptyLog} ELOG {PrintLog} ..." (procedure "do_click" line 23) invoked from within "do_click" invoked from within "if {$started != 0} { variable JS "" variable LogFlag if {$LogFlag == 1} {InsertLog "Last Updated: [clock format [clock seconds] -form..." (file "Check.tcl" line 8) invoked from within "source Check.tcl" (in namespace eval "::Check" script line 6) invoked from within "namespace eval Check { global env source [file join $env(MIDASBASE) TclHttpd tcl Common common.tcl] source [file join $env(MIDASB..." invoked from within "subst { [Doc_Dynamic] <! [global Httpd; upvar #0 Httpd[set Httpd(currentSocket)] data; set ClientIPAddress $data(ipaddr); set MyInfo $data(self)] >..." ("uplevel" body line 1) invoked from within "uplevel #0 [list subst $script]" (procedure "SubstFile" line 12) invoked from within "SubstFile /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119/TclHttpd/Html/AIDA/Check/Check.tml {}" ("uplevel" body line 1) invoked from within "uplevel 1 [list SubstFile $path $interp]" (procedure "Subst_File" line 7) invoked from within "Subst_File $template $interp" invoked from within "TemplateInstantiate $sock $path {} $suffix {} $Template(templateInterp)" (procedure "Doc_application/x-tcl-template" line 9) invoked from within "$cmd $path $suffix $sock" (procedure "Doc_Handle" line 20) invoked from within "Doc_Handle $prefix $path $suffix $sock" (procedure "DocDomain" line 44) invoked from within "DocDomain / /MIDAS/TclHttpd/Html sock8 AIDA/Check/Check.tml" ("eval" body line 1) invoked from within "eval $Url(command,$prefix) [list $sock $suffix]"
Tcl Call Trace
6: DocSubstSystemFile sock8 error 500 can't\ read\ "\;TS_Base_Collected"\;:\ no\ such\ variable\n\ \ \ \ while\ executing\n"\;if\ \{\ \$TS_Base_Collected\ ==\ 0\ \}\ \{\ collect_TS_Base\ \}"\;\n\ \ \ \ (procedure\ "\;do_TS_errors"\;\ line\ 8)\n\ \ \ \ invoked\ from\ within\n"\;do_TS_errors\ "\;\n\ \ \ \ ("\;TSERRORS"\;\ arm\ line\ 1)\n\ \ \ \ invoked\ from\ within\n"\;switch\ -glob\ \$w\ \{\n\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ RESET\ \ \ \ \{set\ started\ 0\}\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ CLEAR\ \ \ \ \{EmptyLog\}\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ ELOG\ \ \ \ \ \{PrintLog\}\n\ \ \ \ \ \ \ \ \ ..."\;\n\ \ \ \ (procedure\ "\;do_click"\;\ line\ 23)\n\ \ \ \ invoked\ from\ within\n"\;do_click"\;\n\ \ \ \ invoked\ from\ within\n"\;if\ \{\$started\ !=\ 0\}\ \{\n\n\ \ \ \ \ variable\ JS\ "\;"\;\n\ \ \ \ \ variable\ LogFlag\n\n\ \ \ \ \ if\ \{\$LogFlag\ ==\ 1\}\ \{InsertLog\ "\;Last\ Updated:\ [clock\ format\ [clock\ seconds\]\ -form..."\;\n\ \ \ \ (file\ "\;Check.tcl"\;\ line\ 8)\n\ \ \ \ invoked\ from\ within\n"\;source\ Check.tcl"\;\n\ \ \ \ (in\ namespace\ eval\ "\;::Check"\;\ script\ line\ 6)\n\ \ \ \ invoked\ from\ within\n"\;namespace\ eval\ Check\ \{\n\ \ \ \ \ \ \ global\ env\n\ \ \ \ \ \ \ source\ [file\ join\ \$env(MIDASBASE)\ TclHttpd\ tcl\ Common\ common.tcl\]\n\ \ \ \ \ \ \ source\ [file\ join\ \$env(MIDASB..."\;\n\ \ \ \ invoked\ from\ within\n"\;subst\ \{\n\n[Doc_Dynamic\]\n\n\n<\;!\n[global\ Httpd\;\ upvar\ #0\ Httpd[set\ Httpd(currentSocket)\]\ data\;\ set\ ClientIPAddress\ \$data(ipaddr)\;\ set\ MyInfo\ \$data(self)\]\n>\;..."\;\n\ \ \ \ ("\;uplevel"\;\ body\ line\ 1)\n\ \ \ \ invoked\ from\ within\n"\;uplevel\ #0\ [list\ subst\ \$script\]"\;\n\ \ \ \ (procedure\ "\;SubstFile"\;\ line\ 12)\n\ \ \ \ invoked\ from\ within\n"\;SubstFile\ /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119/TclHttpd/Html/AIDA/Check/Check.tml\ \{\}"\;\n\ \ \ \ ("\;uplevel"\;\ body\ line\ 1)\n\ \ \ \ invoked\ from\ within\n"\;uplevel\ 1\ [list\ SubstFile\ \$path\ \$interp\]"\;\n\ \ \ \ (procedure\ "\;Subst_File"\;\ line\ 7)\n\ \ \ \ invoked\ from\ within\n"\;Subst_File\ \$template\ \$interp"\;\n\ \ \ \ invoked\ from\ within\n"\;TemplateInstantiate\ \$sock\ \$path\ \{\}\ \$suffix\ \{\}\ \$Template(templateInterp)"\;\n\ \ \ \ (procedure\ "\;Doc_application/x-tcl-template"\;\ line\ 9)\n\ \ \ \ invoked\ from\ within\n"\;\$cmd\ \$path\ \$suffix\ \$sock"\;\n\ \ \ \ (procedure\ "\;Doc_Handle"\;\ line\ 20)\n\ \ \ \ invoked\ from\ within\n"\;Doc_Handle\ \$prefix\ \$path\ \$suffix\ \$sock"\;\n\ \ \ \ (procedure\ "\;DocDomain"\;\ line\ 44)\n\ \ \ \ invoked\ from\ within\n"\;DocDomain\ /\ /MIDAS/TclHttpd/Html\ sock8\ AIDA/Check/Check.tml"\;\n\ \ \ \ ("\;eval"\;\ body\ line\ 1)\n\ \ \ \ invoked\ from\ within\n"\;eval\ \$Url(command,\$prefix)\ [list\ \$sock\ \$suffix\]"\;
5: Doc_Error sock8 can't\ read\ \"TS_Base_Collected\":\ no\ such\ variable\n\ \ \ \ while\ executing\n\"if\ \{\ \$TS_Base_Collected\ ==\ 0\ \}\ \{\ collect_TS_Base\ \}\"\n\ \ \ \ (procedure\ \"do_TS_errors\"\ line\ 8)\n\ \ \ \ invoked\ from\ within\n\"do_TS_errors\ \"\n\ \ \ \ (\"TSERRORS\"\ arm\ line\ 1)\n\ \ \ \ invoked\ from\ within\n\"switch\ -glob\ \$w\ \{\n\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ RESET\ \ \ \ \{set\ started\ 0\}\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ CLEAR\ \ \ \ \{EmptyLog\}\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ ELOG\ \ \ \ \ \{PrintLog\}\n\ \ \ \ \ \ \ \ \ ...\"\n\ \ \ \ (procedure\ \"do_click\"\ line\ 23)\n\ \ \ \ invoked\ from\ within\n\"do_click\"\n\ \ \ \ invoked\ from\ within\n\"if\ \{\$started\ !=\ 0\}\ \{\n\n\ \ \ \ \ variable\ JS\ \"\"\n\ \ \ \ \ variable\ LogFlag\n\n\ \ \ \ \ if\ \{\$LogFlag\ ==\ 1\}\ \{InsertLog\ \"Last\ Updated:\ [clock\ format\ [clock\ seconds\]\ -form...\"\n\ \ \ \ (file\ \"Check.tcl\"\ line\ 8)\n\ \ \ \ invoked\ from\ within\n\"source\ Check.tcl\"\n\ \ \ \ (in\ namespace\ eval\ \"::Check\"\ script\ line\ 6)\n\ \ \ \ invoked\ from\ within\n\"namespace\ eval\ Check\ \{\n\ \ \ \ \ \ \ global\ env\n\ \ \ \ \ \ \ source\ [file\ join\ \$env(MIDASBASE)\ TclHttpd\ tcl\ Common\ common.tcl\]\n\ \ \ \ \ \ \ source\ [file\ join\ \$env(MIDASB...\"\n\ \ \ \ invoked\ from\ within\n\"subst\ \{\n\n[Doc_Dynamic\]\n\n\n<!\n[global\ Httpd\;\ upvar\ #0\ Httpd[set\ Httpd(currentSocket)\]\ data\;\ set\ ClientIPAddress\ \$data(ipaddr)\;\ set\ MyInfo\ \$data(self)\]\n>...\"\n\ \ \ \ (\"uplevel\"\ body\ line\ 1)\n\ \ \ \ invoked\ from\ within\n\"uplevel\ #0\ [list\ subst\ \$script\]\"\n\ \ \ \ (procedure\ \"SubstFile\"\ line\ 12)\n\ \ \ \ invoked\ from\ within\n\"SubstFile\ /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119/TclHttpd/Html/AIDA/Check/Check.tml\ \{\}\"\n\ \ \ \ (\"uplevel\"\ body\ line\ 1)\n\ \ \ \ invoked\ from\ within\n\"uplevel\ 1\ [list\ SubstFile\ \$path\ \$interp\]\"\n\ \ \ \ (procedure\ \"Subst_File\"\ line\ 7)\n\ \ \ \ invoked\ from\ within\n\"Subst_File\ \$template\ \$interp\"\n\ \ \ \ invoked\ from\ within\n\"TemplateInstantiate\ \$sock\ \$path\ \{\}\ \$suffix\ \{\}\ \$Template(templateInterp)\"\n\ \ \ \ (procedure\ \"Doc_application/x-tcl-template\"\ line\ 9)\n\ \ \ \ invoked\ from\ within\n\"\$cmd\ \$path\ \$suffix\ \$sock\"\n\ \ \ \ (procedure\ \"Doc_Handle\"\ line\ 20)\n\ \ \ \ invoked\ from\ within\n\"Doc_Handle\ \$prefix\ \$path\ \$suffix\ \$sock\"\n\ \ \ \ (procedure\ \"DocDomain\"\ line\ 44)\n\ \ \ \ invoked\ from\ within\n\"DocDomain\ /\ /MIDAS/TclHttpd/Html\ sock8\ AIDA/Check/Check.tml\"\n\ \ \ \ (\"eval\"\ body\ line\ 1)\n\ \ \ \ invoked\ from\ within\n\"eval\ \$Url(command,\$prefix)\ [list\ \$sock\ \$suffix\]\"
4: Url_Unwind sock8 can't\ read\ \"TS_Base_Collected\":\ no\ such\ variable\n\ \ \ \ while\ executing\n\"if\ \{\ \$TS_Base_Collected\ ==\ 0\ \}\ \{\ collect_TS_Base\ \}\"\n\ \ \ \ (procedure\ \"do_TS_errors\"\ line\ 8)\n\ \ \ \ invoked\ from\ within\n\"do_TS_errors\ \"\n\ \ \ \ (\"TSERRORS\"\ arm\ line\ 1)\n\ \ \ \ invoked\ from\ within\n\"switch\ -glob\ \$w\ \{\n\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ RESET\ \ \ \ \{set\ started\ 0\}\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ CLEAR\ \ \ \ \{EmptyLog\}\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ ELOG\ \ \ \ \ \{PrintLog\}\n\ \ \ \ \ \ \ \ \ ...\"\n\ \ \ \ (procedure\ \"do_click\"\ line\ 23)\n\ \ \ \ invoked\ from\ within\n\"do_click\"\n\ \ \ \ invoked\ from\ within\n\"if\ \{\$started\ !=\ 0\}\ \{\n\n\ \ \ \ \ variable\ JS\ \"\"\n\ \ \ \ \ variable\ LogFlag\n\n\ \ \ \ \ if\ \{\$LogFlag\ ==\ 1\}\ \{InsertLog\ \"Last\ Updated:\ [clock\ format\ [clock\ seconds\]\ -form...\"\n\ \ \ \ (file\ \"Check.tcl\"\ line\ 8)\n\ \ \ \ invoked\ from\ within\n\"source\ Check.tcl\"\n\ \ \ \ (in\ namespace\ eval\ \"::Check\"\ script\ line\ 6)\n\ \ \ \ invoked\ from\ within\n\"namespace\ eval\ Check\ \{\n\ \ \ \ \ \ \ global\ env\n\ \ \ \ \ \ \ source\ [file\ join\ \$env(MIDASBASE)\ TclHttpd\ tcl\ Common\ common.tcl\]\n\ \ \ \ \ \ \ source\ [file\ join\ \$env(MIDASB...\"\n\ \ \ \ invoked\ from\ within\n\"subst\ \{\n\n[Doc_Dynamic\]\n\n\n<!\n[global\ Httpd\;\ upvar\ #0\ Httpd[set\ Httpd(currentSocket)\]\ data\;\ set\ ClientIPAddress\ \$data(ipaddr)\;\ set\ MyInfo\ \$data(self)\]\n>...\"\n\ \ \ \ (\"uplevel\"\ body\ line\ 1)\n\ \ \ \ invoked\ from\ within\n\"uplevel\ #0\ [list\ subst\ \$script\]\"\n\ \ \ \ (procedure\ \"SubstFile\"\ line\ 12)\n\ \ \ \ invoked\ from\ within\n\"SubstFile\ /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119/TclHttpd/Html/AIDA/Check/Check.tml\ \{\}\"\n\ \ \ \ (\"uplevel\"\ body\ line\ 1)\n\ \ \ \ invoked\ from\ within\n\"uplevel\ 1\ [list\ SubstFile\ \$path\ \$interp\]\"\n\ \ \ \ (procedure\ \"Subst_File\"\ line\ 7)\n\ \ \ \ invoked\ from\ within\n\"Subst_File\ \$template\ \$interp\"\n\ \ \ \ invoked\ from\ within\n\"TemplateInstantiate\ \$sock\ \$path\ \{\}\ \$suffix\ \{\}\ \$Template(templateInterp)\"\n\ \ \ \ (procedure\ \"Doc_application/x-tcl-template\"\ line\ 9)\n\ \ \ \ invoked\ from\ within\n\"\$cmd\ \$path\ \$suffix\ \$sock\"\n\ \ \ \ (procedure\ \"Doc_Handle\"\ line\ 20)\n\ \ \ \ invoked\ from\ within\n\"Doc_Handle\ \$prefix\ \$path\ \$suffix\ \$sock\"\n\ \ \ \ (procedure\ \"DocDomain\"\ line\ 44)\n\ \ \ \ invoked\ from\ within\n\"DocDomain\ /\ /MIDAS/TclHttpd/Html\ sock8\ AIDA/Check/Check.tml\"\n\ \ \ \ (\"eval\"\ body\ line\ 1)\n\ \ \ \ invoked\ from\ within\n\"eval\ \$Url(command,\$prefix)\ [list\ \$sock\ \$suffix\]\" NONE
3: Url_DeferredDispatch / AIDA/Check/Check.tml sock8 buffer {}
2: HttpdReadPost sock8 buffer 16384 {Url_DeferredDispatch / AIDA/Check/Check.tml}
1: HttpdReadPostGlobal sock8 Httpds
Dead time check:
[npg@aidas-gsi S496]$ ~/analyser/analyser R14_24
*** TDR format 3.3.0 analyser - TD - May 2021
*** ERROR: READ I/O error: 5002
blocks: 32000
ADC data format: 252995858 ( 451517.9 Hz)
Other data format: 8924143 ( 15926.8 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 85 ( 0.2 Hz)
RESUME: 85 ( 0.2 Hz)
SYNC100: 34087 ( 60.8 Hz)
WR48-63: 34087 ( 60.8 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 8855799 ( 15804.8 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 0 ( 0.0 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
WR48-63: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
*** Timestamp elapsed time: 560.323 s
FEE elapsed dead time(s) elapsed idle time(s)
0 0.000 349.960
1 0.002 0.000
2 0.000 5.974
3 1.105 0.000
4 0.000 447.454
5 0.000 2.809
6 0.000 16.727
7 0.137 0.000
8 0.090 0.000
9 0.000 429.897
10 0.461 0.000
11 0.018 0.000
12 0.000 8.985
13 0.000 466.311
14 0.000 15.095
15 0.346 0.000
13:24 Beam is off at the moment...
13:43 Changing spill structure from 1.5s/3.5s to 1s.
13:50
[npg@aidas-gsi S496]$ ~/analyser/analyser R14_29
*** TDR format 3.3.0 analyser - TD - May 2021
*** ERROR: READ I/O error: 5002
blocks: 32000
ADC data format: 251763810 ( 930342.1 Hz)
Other data format: 10156190 ( 37530.1 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 205 ( 0.8 Hz)
RESUME: 205 ( 0.8 Hz)
SYNC100: 33009 ( 122.0 Hz)
WR48-63: 33009 ( 122.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 10089762 ( 37284.7 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 527077 ( 1947.7 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
WR48-63: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
*** Timestamp elapsed time: 270.614 s
FEE elapsed dead time(s) elapsed idle time(s)
0 0.034 6.803
1 8.461 0.000
2 0.569 0.000
3 11.664 0.000
4 0.024 26.416
5 0.854 0.000
6 0.069 0.000
7 0.741 0.000
8 1.336 0.000
9 0.000 75.926
10 0.353 0.000
11 0.105 0.000
12 0.070 0.000
13 0.000 93.585
14 0.017 0.000
15 0.121 0.000
15:06 Spill structure back to 1.5 on 2s off. |
|
Thu May 13 00:51:40 2021, Muneerah and OH, Thu 13 May 00:00-08:00 7x
|
01:50 Beam on
The leakage current and the voltages been monitered every half an hour and logged in the excel sheet, looks decreasing.
Tempetratures OK
Rates OK
Attachment 1 Voltages
Atachment 2 Rates
Atachment 3 Temperatures
Attachment 4 ucesb rates less than 1 kHz
System wide check are ok except:
Base Current Difference
aida05 fault 0x36ca : 0x36ce : 4
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Base Current Difference
aida13 fault 0xa : 0xe9 : 223
FPGA Timestamp error counter test result: Passed 15, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 0 5 3 0 1 4 3 3 3 3 6 : 36760
aida02 : 7 9 0 2 2 4 2 3 3 3 6 : 36644
aida03 : 1 3 2 1 2 1 2 3 3 3 6 : 36188
aida04 : 3 1 2 0 2 3 2 3 3 3 6 : 36404
aida05 : 20 9 3 2 2 3 2 2 3 3 6 : 36104
aida06 : 4 10 0 2 2 4 2 2 3 3 6 : 36128
aida07 : 3 1 2 1 3 3 2 3 3 3 6 : 36500
aida08 : 2 4 3 2 3 4 2 3 2 3 6 : 35672
aida09 : 1 6 3 3 1 4 2 3 2 3 6 : 35588
aida10 : 5 6 4 2 1 3 2 3 2 3 6 : 35460
aida11 : 23 8 2 2 2 3 3 4 2 3 6 : 36348
aida12 : 8 4 2 2 2 3 2 4 2 3 6 : 36000
aida13 : 2 5 3 1 2 4 3 3 2 3 6 : 35840
aida14 : 3 6 2 2 2 3 2 4 2 3 6 : 35996
aida15 : 2 4 3 3 0 4 1 3 3 3 6 : 36280
aida16 : 7 7 5 1 3 4 2 3 2 3 6 : 35716
<strong>05:00 OH Took over at 04:00</strong>
System wide checks ok except
Base Current Difference
aida05 fault 0x36ca : 0x36d0 : 6
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Base Current Difference
aida05 fault 0x0 : 0x1 : 1
aida13 fault 0xa : 0xe9 : 223
FPGA Timestamp error counter test result: Passed 14, Failed 2
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
05:41 AIDA 05 has dropped out. Will restart the system
05:58 DAQ recovered from powercycle
p+n strips at 0xa
n+n strips at 0x20
ADC control register at 0xFF
All waveforms turned off in LED
Statistics - attachment 5
Correlation scaler in FEE 3 and 4 are mostly tracking 1:1
06:34 Local users have been unable to recover MBS
And have been unable to contact local experts
07:22 Bias and leakage currents ok - attachment 6
Stats attachment -7 |
|
Thu May 20 01:00:30 2021, Muneerah, Thu May 20 00:00-08:00 13x
|
02:00 System check
The ucesb has stopped around 00:26, the team in the lab ask us to restart AIDA! Called OH for this but the team in the lab figured out thats no need to restart AIDA!
The rates on ucesb got back to work by around 01:12
Deat Time check ok R14_870, attachment 1
Statistics are ok attachment 2
Temperatures are ok attacment 3
Voltages are ok attachment 4
system wide check
All ok except
ADC calibration all failed
FEE64 module aida01 failed
FEE64 module aida02 failed
FEE64 module aida03 failed
FEE64 module aida04 failed
FEE64 module aida05 failed
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida08 failed
FEE64 module aida09 failed
FEE64 module aida10 failed
FEE64 module aida11 failed
FEE64 module aida12 failed
FEE64 module aida13 failed
FEE64 module aida14 failed
FEE64 module aida15 failed
FEE64 module aida16 failed
Calibration test result: Passed 0, Failed 16
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
White rabbite
1 failed
Base Current Difference
aida05 fault 0x500 : 0x569 : 105
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
memory information
Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 20 8 8 5 0 6 1 2 2 2 7 : 37296
aida02 : 19 8 11 2 3 4 2 2 3 3 6 : 36412
aida03 : 14 6 3 4 1 3 1 3 3 3 6 : 36312
aida04 : 25 12 12 4 3 4 1 4 3 3 6 : 37316
aida05 : 4 8 3 4 1 2 2 4 3 3 6 : 36928
aida06 : 42 12 8 7 2 3 3 4 3 3 6 : 37736
aida07 : 24 6 7 3 3 3 1 3 2 2 7 : 37536
aida08 : 10 15 7 2 1 4 2 3 3 3 6 : 36752
aida09 : 29 13 6 6 4 3 3 3 3 3 6 : 37244
aida10 : 18 12 7 2 2 3 3 2 2 4 6 : 37464
aida11 : 20 8 0 5 5 4 2 3 3 3 6 : 36976
aida12 : 7 2 4 3 2 3 2 2 4 3 6 : 37068
aida13 : 2 5 5 4 2 3 2 3 3 3 6 : 36608
aida14 : 9 13 8 2 2 2 3 3 2 4 6 : 37836
aida15 : 24 12 12 1 0 3 2 3 3 3 6 : 36640
aida16 : 12 9 9 2 1 3 2 4 3 3 6 : 37128
6:45 System check
Temperatures, Voltages, Histograms, Statistics, dead time are all ok attachments 5, 6, 7, 8, 9
System wide check are all ok except:
ADC calibration all failed
FEE64 module aida01 failed
FEE64 module aida02 failed
FEE64 module aida03 failed
FEE64 module aida04 failed
FEE64 module aida05 failed
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida08 failed
FEE64 module aida09 failed
FEE64 module aida10 failed
FEE64 module aida11 failed
FEE64 module aida12 failed
FEE64 module aida13 failed
FEE64 module aida14 failed
FEE64 module aida15 failed
FEE64 module aida16 failed
Calibration test result: Passed 0, Failed 16
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
White rabbit 1 failed
Base Current Difference
aida05 fault 0x500 : 0x569 : 105
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
memory information
Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 20 11 3 5 0 6 1 2 3 2 7 : 38264
aida02 : 14 9 9 2 3 3 3 2 3 3 6 : 36496
aida03 : 13 7 3 5 2 3 2 3 3 3 6 : 36668
aida04 : 30 15 13 4 3 4 1 3 4 3 6 : 37888
aida05 : 17 9 3 3 1 2 2 4 3 3 6 : 36956
aida06 : 40 16 5 6 2 3 2 3 2 4 6 : 37936
aida07 : 3 2 3 2 3 4 2 2 2 2 7 : 37196
aida08 : 17 10 6 2 1 3 3 3 3 3 6 : 36852
aida09 : 29 11 7 6 3 3 2 4 3 3 6 : 37436
aida10 : 1 10 5 1 1 3 3 2 2 4 6 : 37252
aida11 : 30 9 0 5 4 4 2 3 3 3 6 : 36960
aida12 : 19 10 5 3 2 3 2 2 4 3 6 : 37196
aida13 : 22 9 6 3 2 3 3 3 3 3 6 : 36960
aida14 : 19 6 9 2 2 3 3 2 2 4 6 : 37452
aida15 : 6 8 11 2 1 3 2 3 3 3 6 : 36616
aida16 : 19 14 5 3 1 3 1 4 3 3 6 : 36908
07:33 system check
Deat time, voltages, statistics, temeratuers are all ok, attachment 10, 11, 12, 13
System wide check
all ok except
ADC all failed
White rabbit
1 failed
Base Current Difference
aida05 fault 0x500 : 0x569 : 105
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
memory information
Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 22 9 4 6 0 5 1 2 2 2 7 : 37152
aida02 : 19 6 13 1 3 3 4 2 3 3 6 : 36780
aida03 : 23 5 6 4 2 3 1 3 3 3 6 : 36452
aida04 : 24 11 14 3 3 4 2 3 3 3 6 : 37048
aida05 : 17 3 6 3 1 2 2 3 3 3 6 : 36444
aida06 : 36 15 4 9 2 4 2 4 3 3 6 : 37608
aida07 : 20 8 5 3 3 4 2 2 2 2 7 : 37376
aida08 : 21 11 10 2 1 4 2 2 2 4 6 : 37324
aida09 : 27 14 7 6 3 3 2 4 3 3 6 : 37452
aida10 : 25 9 5 2 2 3 3 2 2 4 6 : 37436
aida11 : 30 14 2 4 5 3 2 3 3 3 6 : 36936
aida12 : 22 3 5 3 2 3 2 3 3 3 6 : 36640
aida13 : 10 7 1 4 2 3 3 2 3 3 6 : 36336
aida14 : 13 9 12 2 3 3 2 3 2 4 6 : 37820
aida15 : 20 8 11 3 0 3 2 3 3 3 6 : 36640
aida16 : 23 6 6 2 0 2 1 3 4 3 6 : 37164
|
|
Thu May 20 23:26:11 2021, Marc Labiche (ML), system wide checks 9x
|
System Wide check at begining of the night shift:
-----
Clock status test result: Passed 16, Failed 0
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable
-----
ADC calibration:
FEE64 module aida01 failed
FEE64 module aida02 failed
FEE64 module aida03 failed
FEE64 module aida04 failed
FEE64 module aida05 failed
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida08 failed
FEE64 module aida09 failed
FEE64 module aida10 failed
FEE64 module aida11 failed
FEE64 module aida12 failed
FEE64 module aida13 failed
FEE64 module aida14 failed
FEE64 module aida15 failed
FEE64 module aida16 failed
Calibration test result: Passed 0, Failed 16
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
-----
White rabbit status:
Base Current Difference
aida05 fault 0x500 : 0x56d : 109
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
-----
FPGA Timestamp error counter test result: Passed 16, Failed 0
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
-----
Memory info from FEE cards
Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 20 7 7 6 1 5 2 2 3 3 6 : 36472
aida02 : 19 5 4 4 4 3 3 3 2 3 6 : 36020
aida03 : 22 10 2 2 5 2 2 3 3 3 6 : 36680
aida04 : 12 7 1 0 4 3 1 4 3 3 6 : 36856
aida05 : 16 7 5 3 1 3 2 3 3 3 6 : 36584
aida06 : 20 7 5 6 2 4 4 2 3 3 6 : 36888
aida07 : 13 7 5 2 3 2 1 3 2 2 7 : 37308
aida08 : 18 7 5 3 0 2 3 3 3 3 6 : 36656
aida09 : 21 6 4 4 1 4 2 3 3 3 6 : 36740
aida10 : 21 10 2 3 2 3 2 2 2 4 6 : 37156
aida11 : 23 2 4 4 3 2 2 3 2 4 6 : 37612
aida12 : 24 1 4 3 2 2 1 2 4 3 6 : 36744
aida13 : 26 9 4 2 1 4 3 2 3 3 6 : 36464
aida14 : 19 9 9 2 3 4 1 2 2 4 6 : 37156
aida15 : 10 7 10 0 1 3 1 3 3 3 6 : 36288
aida16 : 28 7 3 2 1 3 3 3 3 3 6 : 36824 |
|
Fri Jun 24 15:04:06 2022, Marc, Friday 24th June - evening shift 11x
|
16:05 - Last checked was at 15:30. (see previous entry. All running smoothly.
Next wide check will be in about an hour.
17:00
Stats -ok - attachment 1
Temperatures ok - attachement 2
Leakage current ok but - attachment 3
ucesb screen-shot - attachment 4
Wide check completed. Nothing different.
WR status decoder status:
Base Current Difference
aida07 fault 0xc53d : 0xc547 : 10
aida08 fault 0xf1be : 0xf1e9 : 43
White Rabbit error counter test result: Passed 6, Failed 2
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
FPGA timestamp::
Base Current Difference
aida07 fault 0x2a : 0x2c : 2
FPGA Timestamp error counter test result: Passed 7, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
Note that the leakage current is ok but has increased since yesterday , See graphana - attachement 5 - Prossibly due to higher beam intensity.
19:20 -
Stats ok - attachment 6
Temp ok - attachment 7
Leakage current ok - attachement 8
Wide check completed and same output as above.
22:00
Stats ok - attachment 9
Temp ok - attachment 10
Leakage current ok - attachement 11
Wide check completed and same output as above.
|
|
Sun Jun 26 23:04:30 2022, Marc, new shift - Monday 27 June 0:00 to 8:00 12x
|
0:00 Experiment continues to run smoothly. AIDA DAQ has been rather stable. Leakage current has gone up for the past two days in both DSSDs (combination of high-Z and high-intensity beam and external high temperature).
Stats & Temperatures (VIRTEX,PSU, ASICs) all ok.
At 0:30
Stats ok - Attachment 1
Temp ok - Attachement 2
HV-LC -Attachment 3
At 2:20
Stats ok - Attachment 4
Temp ok - Attachement 5
HV-LC -Attachment 6
Wide Checks:
Clock status test result: Passed 8, Failed 0
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable
ADC Calibration (same as before):
FEE64 module aida01 failed
FEE64 module aida02 failed
FEE64 module aida03 failed
FEE64 module aida04 failed
FEE64 module aida05 failed
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida08 failed
Calibration test result: Passed 0, Failed 8
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
WR decoder status:
Base Current Difference
aida07 fault 0xc53d : 0xc5c9 : 140
aida08 fault 0xf1be : 0xf2b2 : 244
White Rabbit error counter test result: Passed 6, Failed 2
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
FPGA timestamp check:
Base Current Difference
aida07 fault 0x2a : 0x41 : 23
FPGA Timestamp error counter test result: Passed 7, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
At 4:15:
Stats ok - Attachment 7
Temp ok - Attachement 8
HV-LC -Attachment 9
Wide Checks: No change
At 7:15: (no beam since ~6am -> background run)
Stats ok - Attachment 10
Temp ok - Attachement 11
HV-LC -Attachment 12
Wide Checks: No change |
|
Thu Apr 25 15:02:50 2024, Marc, 16:00 - 0:00 Thursday 25 April 8x
|
16:20:
Spill length:4s (was 3.5s before) - Extraction time 1.5s (see attachement #1)
16.45 Checks:
DSSSD bias & leakage current - Grafana - attachments 2
leakage current ramping, spill micro structure observable
FEE64 temperatures OK - attachment 3
ADC data item stats - attachment 5
per FEE64 Rate spectra - attachment 4
Merger etc - attachment 6
ucesb - attachment 7
XX.XX Checks
DSSSD bias & leakage current - Grafana - attachments X
FEE64 temperatures OK - attachment X
ADC data item stats - attachment X
per FEE64 Rate spectra - attachment X
Merger etc - attachment X
ucesb - attachment X |
|
Fri Apr 26 23:04:51 2024, Marc, 00:00-08:00 Saturday 27 April 12x
|
It's a new day, it's a new shift and I'm feeeeeliiiiing goooood !!!!!
0.15 Checks:
DSSSD bias & leakage current - Grafana - attachments 1-2
leakage current dropped a bit since the evening shift ~22:00.
FEE64 temperatures OK - attachment 3
ADC data item stats - attachment 4
per FEE64 Rate spectra - attachment 5
Merger etc - attachment 6
ucesb - attachment 7
2:20 Nothing new to report. All good. Leakage currents continue to go down slowly. See attachement 8 to 12
3:30 Still running smoothly.
5:20 AIDA 4 stop counting in the last 20 min. I informed the main DAQ and they started a new run without AIDA.
DAQ restarted OK. Main DAQ informed.
|
|
Sat Apr 27 23:04:42 2024, Marc, 00:00-08:00 Sunday 28 April 12x
|
Encore un matin, un matin pour ..... a shift !!!ramping up again ! S4 temperature not really going down either: 25C
0:40 AIDA04 is down. I informed the DAQ shift crew and attempting to restart AIDA04.
After rebooting AIDA04 all seemed going until I reached the merger check. No data was being merged.
I realise then that AIDA01 data transfer was not active. I did not spot it when I enabled the data transfer for all.
I stopped the daq and rebooted AIDA01 following all the instructions in the ELOG and this worked.
We are now back in business with AIDA back in main DAQ at 01:40
01:45 full checks:
Bias: AIDA-Graphana - Attachment 1
Leakage current ramping up again ! S4 temperaturenot really going down either: 25C
FEE64 temperatures OK - attachment 2
ADC data item stats - attachment 3
per FEE64 Rate spectra - attachment 4
Merger etc - attachment 5
ucesb - attachment 6
03:00 full checks
No change to be highlighted. All pretty much the same as during previous full checks.
05:00 full checks
Expt still running very smoothly.
Bias: AIDA-Graphana - Attachment 7-8
Leakage current going down slightly
FEE64 temperatures OK - attachment 9
ADC data item stats - attachment 10
ramping up again ! S4 temperaturenot really going down either: 25C
per FEE64 Rate spectra - attachment 11
Merger etc - attachment 12
ucesb - attachment 13
07:15 all running smoothly.
07:20 Spoke too fast AIDA DAQ is down. DAQ shift crew informed.
07:40: AIDA10 was rebooted successfully and AIDA is now back in the main DAQ.
|
|
Sun Apr 28 22:52:06 2024, Marc, 0:00-08:00 Monday 29 April 7x
|
Just another manic magic Monday, ....
Taking over Betool's AIDA shift. All is good.
This shift's first hour has been eventless. Temperatures, Rates, bias, all look good.
01:15: AIDA04 is down again. DAQ shift crew informed.
01:27: AIDA04 is back and shift crew has been informed.
01:30 Checks:
Bias and leakage current - Attachment 1-2
Temperatures - Attachment 3
ADC Data item - Attachment 4
Rate Spectra - Attachment 5
UCESB - Attachment 6
Timestamp checks - Attachement 7
06:00 It has run very smoothly.
|
|
Mon Jun 10 15:07:24 2024, Marc, 16:00-00:00 shift Monday 10/06/24 31x
|
Evening shift
5pm full checks:
DSSSD bias & leakage current ok - attachment # 1-2
FEE64 temperatures ok - attachment # 3
ADC data item stats - attachment # 4
per FEE64 Rate spectra - attachment # 5
Merger ok - Attachement # 6
Tape service - attachement # 7
7pm full checks:
DSSSD bias & leakage current ok - attachment # 8-9
FEE64 temperatures ok - attachment # 10
ADC data item stats - attachment # 11
per FEE64 Rate spectra - attachment # 12
Merger ok - Attachement # 13
Tape service - attachement # 14
ucesb - attachment # 15
9pm full checks:
DSSSD bias & leakage current ok - attachment# 16-17
FEE64 temperatures ok - attachment # 18
ADC data item stats - attachment # 19
per FEE64 Rate spectra - attachment # 20
Merger ok - Attachement # 21
Tape service - attachement # 22
ucesb - attachment # 23
9pm full checks:
DSSSD bias & leakage current ok - attachment# 24-25
FEE64 temperatures ok - attachment # 26
ADC data item stats - attachment # 27
per FEE64 Rate spectra - attachment # 28
Merger ok - Attachement # 29
Tape service - attachement # 30
ucesb - attachment # 31
|
|
Wed Jun 12 23:00:16 2024, Marc, 00:00-08:00 Thursday 13 June 23x
|
Starting a new night shift - All good
1am full check:
DSSSD bias & leakage current ok - attachment # 1-2
FEE64 temperatures ok - attachment # 3
ADC data item stats - attachment # 4
Merger ok - Attachement # 5
Tape service - attachement # 6
2:30am: AIDA02 FEE stopped working. DESPEC main DAQ was alerted and remobe AIDA from timestitcher while I was restarting AIDA DAQ.
2:45am: AIDA02 FEE is back and AIDA has been added to the timestitcher again in the main DAQ.
The AIDA runs at the time of the crash to look at are R7_348, 349.
All spectra were zeroed at this time and checked.
4:00 am full check:
DSSSD bias & leakage current ok - attachment # 7-8
FEE64 temperatures ok - attachment # 9
ADC data item stats - attachment # 10
Merger ok - attachement # 11
Tape service - attachement # 12
Spectra - attachement # 13
ucesb - attachement # 14
6:00am full check:
DSSSD bias & leakage current ok - attachment # 15-16
FEE64 temperatures ok - attachment # 17
ADC data item stats - attachment # 18
Merger ok - attachement # 19
Tape service - attachement # 20
Spectra - attachement # 21
ucesb - attachement # 22
6:30 the nearline/offline analysis team reported possible noise issue in DSSD#2 and asked if the threshold were changed at any point. I can't find any mention of changes in the elog.
I've not done a setup of the all DAQ when AIDA02 fee crashes earlier this morning. I just did a setup of the merger. Thus I don't think the threshold have changed when I stopeed and restarted the DAQ.
I can see that currently slow comparator threshod are set to 0xa for most of the DSSD2 fees except for AIDA06 which is set to 0xf. I'm not sure this is the thershold we would need to increase. Something to check during the next shift.
In the UCESB window I can see that the number of decay in DSSD2 is double the number of decay in DSSD1 while the number of implant is similar. Looking at other shift it looks that this was alsready the case in prvious shifts.
07.06 analysis of data file S181/R7_457 - attachment 23
max deadtime 5.8% aida08, all other FEE64s < 1.4%
LEC data rate 1.788M, HEC data rate 4.3k
|
|
Sun Apr 28 06:57:59 2024, Magda Satrazani, 08:00-16:00 Sunday 28 April 36x
|
Took over the shift from Marc.
10.51 Added screenshots to illustrate erroneous data from Grafana (AIDA) - attachments 18-19
Compare to the CAEN HV control terminal (attachment 15) which shows the expected values. Have email'd Nic & Helena. TD
10.55 Noticed that the CAEN HV control terminal is showing the corerct data but on the wrong line of the display (attachment 15). Enter commands q (quit) and d (display) to restart display - the correct data now appears on the correct line (attachment 20).
Grafana (AIDA) is now reporting data correctly too (attachment 21).
Summary - CAEN HV terminal display issue ( c. 10.05 - 10.55 )
Note the correct data/values continued to be shown. Leakage current shown correctly by both Grafana and CAEN HV control throughout.
Conclude this was a display issue - not a change in the operating parameters of the CAEN N1419ET.. Data should be OK.
14.45 analysis data file R21_500 - attachment 36
max deadtime 22% (aida11), 16% (aida04), 7% (aida02), 6% (aida08), 6% (aida06) , all others < 1%
no timewarps
HEC data item rate 2.7kHz
|
|
Thu Jun 13 06:49:54 2024, Magda Satrazani, 08:00 - 16:00 Thursday 13 June 2024 40x
|
Took over the shift from Marc.
08:00 FULL CHECK
DSSSD bias & leakage current ok - attachment # 1-2
FEE64 temperatures ok - attachment # 3
ADC data item stats - attachment # 4
Merger ok - Attachement # 5
Tape service - attachement # 6
10:00 FULL CHECK
DSSSD bias & leakage current ok - attachment # 7-8
FEE64 temperatures ok - attachment # 9
ADC data item stats - attachment # 10
Merger ok - Attachement # 11
Tape service - attachement # 12
12:00 FULL CHECK
DSSSD bias & leakage current ok - attachment # 13-14
FEE64 temperatures ok - attachment # 15
ADC data item stats - attachment # 16
Merger ok - Attachement # 17
Tape service - attachement # 18
12.34 analysis data file S181/R7_610 - attachment 19
max deadtime 8.7% (aida08), all other FEE64s < 2.7%
LEC data rate 2.000M, HEC data rate 4.7k
12.52 per p+n FEE64 1.8.L spectra - attachment 20
aida09 pulser peak width 63 ch FWHM
per FEE64 1.8.H spectra - attachments 21-22
per FEE64 1.8.W spectra - 20us FSR - attachments 23-24
13.00 all histograms zero'd
14:00 FULL CHECK
DSSSD bias & leakage current ok - attachment # 19-20
FEE64 temperatures ok - attachment # 21
ADC data item stats - attachment # 22
Merger ok - Attachement # 23
Tape service - attachement # 24
13.01 Beam OFF for S4 access
ADC data item stats - attachment 25
per FEE64 Rate spectra - attachment 26
14.12 analysis data file R7_654 - attachment 27
max deadtime 3.9% (aida08), all other FEE64s < 3%
LEC data rate 1.817M, HEC data rate 2.5k
14.00 data file R7_649 - attachment 28
max deadtime 4.5% (aida08), all other FEE64s < 3%
LEC data rate 1.756M, HEC data rate 2.5k
16:00 FULL CHECK
DSSSD bias & leakage current ok - attachment # 25-26
FEE64 temperatures ok - attachment # 27
ADC data item stats - attachment # 28
Merger ok - Attachement # 29
Tape service - attachement # 30 |
|
Sat May 14 06:57:21 2022, MS, OH, Saturday 14 May
|
08:00 Took over the shift from Philippos
In system wide checks aida09 fails ASIC clock check but it is bit 6 which is ok |
|
Sat May 14 09:00:21 2022, MS, OH, Saturday 14 May
|
10:00 |
|
Sat May 14 10:59:52 2022, MS, OH, Saturday 14 May
|
12:00 |
|
Sat May 14 13:01:58 2022, MS, OH, Saturday 14 May
|
14:00 |
|