AIDA
GELINA
BRIKEN
nToF
CRIB
ISOLDE
CIRCE
nTOFCapture
DESPEC
DTAS
EDI_PSA
179Ta
CARME
StellarModelling
DCF
K40
DESPEC
Draft saved at 00:00:00
Fields marked with
*
are required
Entry time:
Wed May 19 09:12:52 2021
Author
*
:
Subject
*
:
Auther: Muneerah and Betool 02:00 The dead time have been checked every 30 min from the start of the shift, it seemes Ok Statistics ok - 210517_0202_Stats Temperatures ok - 210517_0202_Temp Bias and leakage ok - 210517_0200_Bias 02:05 System wide checks: Clock status test result: Passed 16, Failed 0 Understand status as follows Status bit 3 : firmware PLL that creates clocks from external clock not locked Status bit 2 : always logic '1' Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock If all these bits are not set then the operation of the firmware is unreliable FEE64 module aida02 failed FEE64 module aida03 failed FEE64 module aida06 failed FEE64 module aida10 failed Calibration test result: Passed 12, Failed 4 If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module Base Current Difference aida05 fault 0x1552 : 0x155d : 11 White Rabbit error counter test result: Passed 15, Failed 1 Understand the status reports as follows:- Status bit 3 : White Rabbit decoder detected an error in the received data Status bit 2 : Firmware registered WR error, no reload of Timestamp Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR Base Current Difference aida05 fault 0x0 : 0x2 : 2 FPGA Timestamp error counter test result: Passed 15, Failed 1 If any of these counts are reported as in error The ASIC readout system has detected a timeslip. That is the timestamp read from the time FIFO is not younger than the last Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k aida01 : 5 8 3 3 3 5 2 2 2 3 7 : 39460 aida02 : 22 4 5 0 3 2 2 3 2 3 7 : 39560 aida03 : 21 10 2 1 1 3 2 4 2 3 7 : 40100 aida04 : 18 7 2 0 3 3 3 2 2 3 7 : 39392 aida05 : 16 6 5 1 2 4 3 2 2 3 7 : 39520 aida06 : 18 6 6 2 2 2 2 3 2 3 7 : 39576 aida07 : 12 8 6 1 2 3 1 3 2 3 7 : 39408 aida08 : 13 11 4 0 1 3 3 2 2 3 7 : 39308 aida09 : 10 8 4 2 3 4 2 3 2 3 7 : 39848 aida10 : 16 8 2 1 2 4 2 3 1 4 7 : 40768 aida11 : 26 10 2 2 1 2 2 3 2 3 7 : 39512 aida12 : 16 9 2 1 2 4 2 2 2 3 7 : 39240 aida13 : 15 9 3 2 2 4 2 2 2 3 7 : 39284 aida14 : 25 10 2 3 3 3 2 3 2 3 7 : 39796 aida15 : 4 2 2 1 1 3 2 2 2 3 7 : 38944 aida16 : 16 7 3 3 1 3 3 2 2 3 7 : 39368 04:00 Statistics ok - 210517_0403_Stats Temperatures ok - 210517_0403_Temp Bias and leakage ok - 210517_0400_Bias 04:05 System wide checks: Clock status test result: Passed 16, Failed 0 Understand status as follows Status bit 3 : firmware PLL that creates clocks from external clock not locked Status bit 2 : always logic '1' Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock If all these bits are not set then the operation of the firmware is unreliable FEE64 module aida02 failed FEE64 module aida03 failed FEE64 module aida06 failed FEE64 module aida10 failed Calibration test result: Passed 12, Failed 4 If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module Base Current Difference aida05 fault 0x1552 : 0x1560 : 14 White Rabbit error counter test result: Passed 15, Failed 1 Understand the status reports as follows:- Status bit 3 : White Rabbit decoder detected an error in the received data Status bit 2 : Firmware registered WR error, no reload of Timestamp Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR Base Current Difference aida05 fault 0x0 : 0x2 : 2 FPGA Timestamp error counter test result: Passed 15, Failed 1 If any of these counts are reported as in error The ASIC readout system has detected a timeslip. That is the timestamp read from the time FIFO is not younger than the last Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k aida01 : 15 7 5 2 3 5 2 2 2 3 7 : 39492 aida02 : 23 6 7 2 1 3 2 3 2 3 7 : 39676 aida03 : 17 9 3 2 0 3 2 3 2 3 7 : 39548 aida04 : 16 4 1 2 1 3 3 3 2 3 7 : 39792 aida05 : 22 7 3 3 1 4 3 3 2 3 7 : 40032 aida06 : 18 6 7 3 1 3 2 3 2 3 7 : 39688 aida07 : 18 7 4 1 2 4 1 4 2 3 7 : 40032 aida08 : 19 5 4 1 1 3 3 3 2 3 7 : 39828 aida09 : 10 6 7 1 3 4 2 2 2 3 7 : 39336 aida10 : 12 10 2 3 1 4 2 3 1 4 7 : 40768 aida11 : 10 9 4 3 2 1 2 3 2 3 7 : 39440 aida12 : 8 7 6 1 2 4 2 3 2 3 7 : 39768 aida13 : 11 2 6 1 2 3 2 3 2 3 7 : 39612 aida14 : 27 7 4 2 3 3 2 3 2 3 7 : 39780 aida15 : 4 2 3 1 1 3 2 2 2 3 7 : 38960 aida16 : 18 8 4 2 1 4 2 2 2 3 7 : 39240 06:27 aida07 has dropped out and lost connection with the merger. Powercycling AIDA 06:48 Restarted system but the noise was extremely high. Equivalent to yesterday 15:00. Restarting again. 07:01 it is working fine. 07:40 Screen shots after the restart Statistics ok - 210517_0742_Stats Temperatures ok - 210517_0743_Temp Bias and leakage ok - 210517_0740_Bias System wide checks FEE64 module aida06 global clocks failed, 6 Clock status test result: Passed 15, Failed 1 Understand status as follows Status bit 3 : firmware PLL that creates clocks from external clock not locked Status bit 2 : always logic '1' Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock If all these bits are not set then the operation of the firmware is unreliable FEE64 module aida01 failed FEE64 module aida02 failed FEE64 module aida03 failed FEE64 module aida04 failed FEE64 module aida05 failed FEE64 module aida06 failed FEE64 module aida07 failed FEE64 module aida08 failed FEE64 module aida09 failed FEE64 module aida10 failed FEE64 module aida11 failed FEE64 module aida12 failed FEE64 module aida13 failed FEE64 module aida14 failed FEE64 module aida15 failed FEE64 module aida16 failed Calibration test result: Passed 0, Failed 16 If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module Base Current Difference aida05 fault 0x61da : 0x61dd : 3 White Rabbit error counter test result: Passed 15, Failed 1 Understand the status reports as follows:- Status bit 3 : White Rabbit decoder detected an error in the received data Status bit 2 : Firmware registered WR error, no reload of Timestamp Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR FPGA Timestamp error counter test result: Passed 16, Failed 0 If any of these counts are reported as in error The ASIC readout system has detected a timeslip. That is the timestamp read from the time FIFO is not younger than the last Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k aida01 : 20 6 3 1 1 3 1 3 3 3 7 : 40336 aida02 : 11 6 0 1 1 5 2 4 2 3 7 : 40252 aida03 : 0 5 2 3 3 3 3 4 2 3 7 : 40424 aida04 : 2 4 2 1 2 4 3 3 2 3 7 : 39912 aida05 : 11 7 5 1 2 2 4 2 3 3 7 : 40532 aida06 : 20 6 0 2 3 2 1 3 3 3 7 : 40320 aida07 : 15 5 1 2 2 3 2 2 3 3 7 : 40116 aida08 : 2 3 4 1 3 3 2 2 3 3 7 : 40128 aida09 : 18 5 1 3 2 4 2 2 3 3 7 : 40288 aida10 : 11 6 3 4 2 3 3 2 2 4 7 : 41484 aida11 : 4 4 5 2 2 2 3 3 2 3 7 : 39744 aida12 : 18 7 2 3 2 2 4 3 2 3 7 : 40064 aida13 : 2 2 1 3 0 3 2 4 2 3 7 : 39944 aida14 : 16 4 5 3 3 3 3 2 2 4 7 : 41552 aida15 : 9 3 1 1 2 3 2 3 3 3 7 : 40556 aida16 : 16 5 1 2 1 2 3 4 2 3 7 : 40184 Date:17/05/2021 Elog for shift: 08.00 to 12.00 Author: Lewis Sexton Screenshots saved to: /tmp/Elog_210517_0800_1200 ***** 08.30 ***** Checked R12_34 through analyser, dead time seems okay: *** Timestamp elapsed time: 232.154 s FEE elapsed dead time(s) elapsed idle time(s) 0 0.021 0.000 1 2.781 0.000 2 0.849 0.000 3 5.686 0.000 4 0.000 25.816 5 0.017 0.000 6 0.177 0.000 7 1.349 0.000 8 1.791 0.000 9 0.000 91.660 10 6.194 0.000 11 0.320 0.000 12 1.425 0.000 13 0.000 72.999 14 0.052 0.000 15 1.729 0.000 16 0.000 0.000 17 0.000 0.000 18 0.000 0.000 19 0.000 0.000 20 0.000 0.000 21 0.000 0.000 22 0.000 0.000 23 0.000 0.000 24 0.000 0.000 25 0.000 0.000 26 0.000 0.000 27 0.000 0.000 28 0.000 0.000 29 0.000 0.000 30 0.000 0.000 31 0.000 0.000 32 0.000 0.000 Stats checked seem okay (0833_Stats.png) Merger rate ~3M items/s Tape server ~12 MB/s ***** 09.00 ***** Checked R12_48 through analyser, dead time seems okay: *** Timestamp elapsed time: 170.459 s FEE elapsed dead time(s) elapsed idle time(s) 0 0.196 0.000 1 6.294 0.000 2 0.992 0.000 3 8.944 0.000 4 0.024 0.000 5 0.001 0.000 6 0.605 0.000 7 2.096 0.000 8 1.654 0.000 9 0.000 31.395 10 6.175 0.000 11 2.524 0.000 12 2.076 0.000 13 0.000 10.337 14 0.103 0.000 15 2.203 0.000 16 0.000 0.000 17 0.000 0.000 18 0.000 0.000 19 0.000 0.000 20 0.000 0.000 21 0.000 0.000 22 0.000 0.000 23 0.000 0.000 24 0.000 0.000 25 0.000 0.000 26 0.000 0.000 27 0.000 0.000 28 0.000 0.000 29 0.000 0.000 30 0.000 0.000 31 0.000 0.000 32 0.000 0.000 Stats checked seem okay (0902_Stats.png) ***** 09.30 ***** AIDA crash aida05 displaying 0 in stats Power cycle performed with help of TD Writing to R13 now MBS relay restared AIDA back up at 10.20 Ran analyser for R13_16 *** Timestamp elapsed time: 168.059 s FEE elapsed dead time(s) elapsed idle time(s) 0 0.047 0.000 1 2.282 0.000 2 0.080 0.000 3 0.806 0.000 4 0.000 0.000 5 0.260 0.000 6 0.030 0.000 7 1.471 0.000 8 67.467 0.000 9 80.552 0.000 10 0.057 0.000 11 0.000 11.671 12 1.191 0.000 13 0.080 0.000 14 0.091 0.000 15 0.202 0.000 16 0.000 0.000 17 0.000 0.000 18 0.000 0.000 19 0.000 0.000 20 0.000 0.000 21 0.000 0.000 22 0.000 0.000 23 0.000 0.000 24 0.000 0.000 25 0.000 0.000 26 0.000 0.000 27 0.000 0.000 28 0.000 0.000 29 0.000 0.000 30 0.000 0.000 31 0.000 0.000 32 0.000 0.000 High deadtime in aida09 and aida10, changed slow comparator threshold to 0xe (previously 0xc) Ran analyser for R13_21 for comparision *** Timestamp elapsed time: 149.082 s FEE elapsed dead time(s) elapsed idle time(s) 0 0.000 0.000 1 3.580 0.000 2 0.000 0.000 3 2.800 0.000 4 0.000 0.000 5 0.514 0.000 6 0.140 0.000 7 1.372 0.000 8 55.544 0.000 9 66.588 0.000 10 1.065 0.000 11 0.000 10.010 12 0.681 0.000 13 0.044 0.000 14 0.070 0.000 15 0.400 0.000 16 0.000 0.000 17 0.000 0.000 18 0.000 0.000 19 0.000 0.000 20 0.000 0.000 21 0.000 0.000 22 0.000 0.000 23 0.000 0.000 24 0.000 0.000 25 0.000 0.000 26 0.000 0.000 27 0.000 0.000 28 0.000 0.000 29 0.000 0.000 30 0.000 0.000 31 0.000 0.000 32 0.000 0.000 FEE temp slightly high in aida04, 0.44 above limit of 65 (FEETemps_1042.png) Merger rate ~5M items/s Tape server ~17 MB/s Raised slow comparator in aida09 and aida10 to 0xf, analyse R13_26 for comparision *** Timestamp elapsed time: 122.147 s FEE elapsed dead time(s) elapsed idle time(s) 0 0.027 0.000 1 19.945 0.000 2 0.601 0.000 3 21.555 0.000 4 0.042 0.000 5 3.226 0.000 6 1.211 0.000 7 9.674 0.000 8 52.817 0.000 9 54.972 0.000 10 1.848 0.000 11 0.008 0.000 12 2.034 0.000 13 0.278 0.000 14 0.018 0.000 15 0.634 0.000 16 0.000 0.000 17 0.000 0.000 18 0.000 0.000 19 0.000 0.000 20 0.000 0.000 21 0.000 0.000 22 0.000 0.000 23 0.000 0.000 24 0.000 0.000 25 0.000 0.000 26 0.000 0.000 27 0.000 0.000 28 0.000 0.000 29 0.000 0.000 30 0.000 0.000 31 0.000 0.000 32 0.000 0.000 high deatime now also seen in aida02 and aida04 Several powercycles performed as high rates after restart, analysis of R14_13 showed low deadtime for all FEEs Back running as of 10.45 Analyser of R14_15 to double check *** Timestamp elapsed time: 205.006 s FEE elapsed dead time(s) elapsed idle time(s) 0 0.000 40.176 1 0.438 0.000 2 0.199 0.000 3 2.927 0.000 4 0.000 140.347 5 1.282 0.000 6 0.150 0.000 7 2.097 0.000 8 1.673 0.000 9 0.000 87.749 10 1.150 0.000 11 0.489 0.000 12 0.374 0.000 13 0.048 20.995 14 0.251 0.000 15 2.441 0.000 16 0.000 0.000 17 0.000 0.000 18 0.000 0.000 19 0.000 0.000 20 0.000 0.000 21 0.000 0.000 22 0.000 0.000 23 0.000 0.000 24 0.000 0.000 25 0.000 0.000 26 0.000 0.000 27 0.000 0.000 28 0.000 0.000 29 0.000 0.000 30 0.000 0.000 31 0.000 0.000 32 0.000 0.000 12:04 During access Sultan added an additional ground cable and also noted the bias cable going into FEE12 was loose. He pushed it in solidly and the rates appear to have dropped across the FEEs Date:17/05/2021 Elog for shift: 08.00 to 12.00 Author: Lewis Sexton Screenshots saved to: /tmp/Elog_210517_0800_1200 ***** 08.30 ***** Checked R12_34 through analyser, dead time seems okay: *** Timestamp elapsed time: 232.154 s FEE elapsed dead time(s) elapsed idle time(s) 0 0.021 0.000 1 2.781 0.000 2 0.849 0.000 3 5.686 0.000 4 0.000 25.816 5 0.017 0.000 6 0.177 0.000 7 1.349 0.000 8 1.791 0.000 9 0.000 91.660 10 6.194 0.000 11 0.320 0.000 12 1.425 0.000 13 0.000 72.999 14 0.052 0.000 15 1.729 0.000 16 0.000 0.000 17 0.000 0.000 18 0.000 0.000 19 0.000 0.000 20 0.000 0.000 21 0.000 0.000 22 0.000 0.000 23 0.000 0.000 24 0.000 0.000 25 0.000 0.000 26 0.000 0.000 27 0.000 0.000 28 0.000 0.000 29 0.000 0.000 30 0.000 0.000 31 0.000 0.000 32 0.000 0.000 Stats checked seem okay (0833_Stats.png) Merger rate ~3M items/s Tape server ~12 MB/s ***** 09.00 ***** Checked R12_48 through analyser, dead time seems okay: *** Timestamp elapsed time: 170.459 s FEE elapsed dead time(s) elapsed idle time(s) 0 0.196 0.000 1 6.294 0.000 2 0.992 0.000 3 8.944 0.000 4 0.024 0.000 5 0.001 0.000 6 0.605 0.000 7 2.096 0.000 8 1.654 0.000 9 0.000 31.395 10 6.175 0.000 11 2.524 0.000 12 2.076 0.000 13 0.000 10.337 14 0.103 0.000 15 2.203 0.000 16 0.000 0.000 17 0.000 0.000 18 0.000 0.000 19 0.000 0.000 20 0.000 0.000 21 0.000 0.000 22 0.000 0.000 23 0.000 0.000 24 0.000 0.000 25 0.000 0.000 26 0.000 0.000 27 0.000 0.000 28 0.000 0.000 29 0.000 0.000 30 0.000 0.000 31 0.000 0.000 32 0.000 0.000 Stats checked seem okay (0902_Stats.png) ***** 09.30 ***** AIDA crash aida05 displaying 0 in stats Power cycle performed with help of TD Writing to R13 now MBS relay restared AIDA back up at 10.20 Ran analyser for R13_16 *** Timestamp elapsed time: 168.059 s FEE elapsed dead time(s) elapsed idle time(s) 0 0.047 0.000 1 2.282 0.000 2 0.080 0.000 3 0.806 0.000 4 0.000 0.000 5 0.260 0.000 6 0.030 0.000 7 1.471 0.000 8 67.467 0.000 9 80.552 0.000 10 0.057 0.000 11 0.000 11.671 12 1.191 0.000 13 0.080 0.000 14 0.091 0.000 15 0.202 0.000 16 0.000 0.000 17 0.000 0.000 18 0.000 0.000 19 0.000 0.000 20 0.000 0.000 21 0.000 0.000 22 0.000 0.000 23 0.000 0.000 24 0.000 0.000 25 0.000 0.000 26 0.000 0.000 27 0.000 0.000 28 0.000 0.000 29 0.000 0.000 30 0.000 0.000 31 0.000 0.000 32 0.000 0.000 High deadtime in aida09 and aida10, changed slow comparator threshold to 0xe (previously 0xc) Ran analyser for R13_21 for comparision *** Timestamp elapsed time: 149.082 s FEE elapsed dead time(s) elapsed idle time(s) 0 0.000 0.000 1 3.580 0.000 2 0.000 0.000 3 2.800 0.000 4 0.000 0.000 5 0.514 0.000 6 0.140 0.000 7 1.372 0.000 8 55.544 0.000 9 66.588 0.000 10 1.065 0.000 11 0.000 10.010 12 0.681 0.000 13 0.044 0.000 14 0.070 0.000 15 0.400 0.000 16 0.000 0.000 17 0.000 0.000 18 0.000 0.000 19 0.000 0.000 20 0.000 0.000 21 0.000 0.000 22 0.000 0.000 23 0.000 0.000 24 0.000 0.000 25 0.000 0.000 26 0.000 0.000 27 0.000 0.000 28 0.000 0.000 29 0.000 0.000 30 0.000 0.000 31 0.000 0.000 32 0.000 0.000 FEE temp slightly high in aida04, 0.44 above limit of 65 (FEETemps_1042.png) Merger rate ~5M items/s Tape server ~17 MB/s Raised slow comparator in aida09 and aida10 to 0xf, analyse R13_26 for comparision *** Timestamp elapsed time: 122.147 s FEE elapsed dead time(s) elapsed idle time(s) 0 0.027 0.000 1 19.945 0.000 2 0.601 0.000 3 21.555 0.000 4 0.042 0.000 5 3.226 0.000 6 1.211 0.000 7 9.674 0.000 8 52.817 0.000 9 54.972 0.000 10 1.848 0.000 11 0.008 0.000 12 2.034 0.000 13 0.278 0.000 14 0.018 0.000 15 0.634 0.000 16 0.000 0.000 17 0.000 0.000 18 0.000 0.000 19 0.000 0.000 20 0.000 0.000 21 0.000 0.000 22 0.000 0.000 23 0.000 0.000 24 0.000 0.000 25 0.000 0.000 26 0.000 0.000 27 0.000 0.000 28 0.000 0.000 29 0.000 0.000 30 0.000 0.000 31 0.000 0.000 32 0.000 0.000 high deatime now also seen in aida02 and aida04 Several powercycles performed as high rates after restart, analysis of R14_13 showed low deadtime for all FEEs Back running as of 10.45 Analyser of R14_15 to double check *** Timestamp elapsed time: 205.006 s FEE elapsed dead time(s) elapsed idle time(s) 0 0.000 40.176 1 0.438 0.000 2 0.199 0.000 3 2.927 0.000 4 0.000 140.347 5 1.282 0.000 6 0.150 0.000 7 2.097 0.000 8 1.673 0.000 9 0.000 87.749 10 1.150 0.000 11 0.489 0.000 12 0.374 0.000 13 0.048 20.995 14 0.251 0.000 15 2.441 0.000 16 0.000 0.000 17 0.000 0.000 18 0.000 0.000 19 0.000 0.000 20 0.000 0.000 21 0.000 0.000 22 0.000 0.000 23 0.000 0.000 24 0.000 0.000 25 0.000 0.000 26 0.000 0.000 27 0.000 0.000 28 0.000 0.000 29 0.000 0.000 30 0.000 0.000 31 0.000 0.000 32 0.000 0.000 12:04 During access Sultan added an additional ground cable and also noted the bias cable going into FEE12 was loose. He pushed it in solidly and the rates appear to have dropped across the FEEs 12:52 Check FPGA Timestamp errors gives following error message: Got the error Server Internal Error while trying to obtain /AIDA/Check/Check.tml. can't read "TS_Base_Collected": no such variable while executing "if { $TS_Base_Collected == 0 } { collect_TS_Base }" (procedure "do_TS_errors" line 8) invoked from within "do_TS_errors " ("TSERRORS" arm line 1) invoked from within "switch -glob $w { RESET {set started 0} CLEAR {EmptyLog} ELOG {PrintLog} ..." (procedure "do_click" line 23) invoked from within "do_click" invoked from within "if {$started != 0} { variable JS "" variable LogFlag if {$LogFlag == 1} {InsertLog "Last Updated: [clock format [clock seconds] -form..." (file "Check.tcl" line 8) invoked from within "source Check.tcl" (in namespace eval "::Check" script line 6) invoked from within "namespace eval Check { global env source [file join $env(MIDASBASE) TclHttpd tcl Common common.tcl] source [file join $env(MIDASB..." invoked from within "subst { [Doc_Dynamic] <! [global Httpd; upvar #0 Httpd[set Httpd(currentSocket)] data; set ClientIPAddress $data(ipaddr); set MyInfo $data(self)] >..." ("uplevel" body line 1) invoked from within "uplevel #0 [list subst $script]" (procedure "SubstFile" line 12) invoked from within "SubstFile /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119/TclHttpd/Html/AIDA/Check/Check.tml {}" ("uplevel" body line 1) invoked from within "uplevel 1 [list SubstFile $path $interp]" (procedure "Subst_File" line 7) invoked from within "Subst_File $template $interp" invoked from within "TemplateInstantiate $sock $path {} $suffix {} $Template(templateInterp)" (procedure "Doc_application/x-tcl-template" line 9) invoked from within "$cmd $path $suffix $sock" (procedure "Doc_Handle" line 20) invoked from within "Doc_Handle $prefix $path $suffix $sock" (procedure "DocDomain" line 44) invoked from within "DocDomain / /MIDAS/TclHttpd/Html sock8 AIDA/Check/Check.tml" ("eval" body line 1) invoked from within "eval $Url(command,$prefix) [list $sock $suffix]" Tcl Call Trace 6: DocSubstSystemFile sock8 error 500 can't\ read\ "\;TS_Base_Collected"\;:\ no\ such\ variable\n\ \ \ \ while\ executing\n"\;if\ \{\ \$TS_Base_Collected\ ==\ 0\ \}\ \{\ collect_TS_Base\ \}"\;\n\ \ \ \ (procedure\ "\;do_TS_errors"\;\ line\ 8)\n\ \ \ \ invoked\ from\ within\n"\;do_TS_errors\ "\;\n\ \ \ \ ("\;TSERRORS"\;\ arm\ line\ 1)\n\ \ \ \ invoked\ from\ within\n"\;switch\ -glob\ \$w\ \{\n\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ RESET\ \ \ \ \{set\ started\ 0\}\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ CLEAR\ \ \ \ \{EmptyLog\}\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ ELOG\ \ \ \ \ \{PrintLog\}\n\ \ \ \ \ \ \ \ \ ..."\;\n\ \ \ \ (procedure\ "\;do_click"\;\ line\ 23)\n\ \ \ \ invoked\ from\ within\n"\;do_click"\;\n\ \ \ \ invoked\ from\ within\n"\;if\ \{\$started\ !=\ 0\}\ \{\n\n\ \ \ \ \ variable\ JS\ "\;"\;\n\ \ \ \ \ variable\ LogFlag\n\n\ \ \ \ \ if\ \{\$LogFlag\ ==\ 1\}\ \{InsertLog\ "\;Last\ Updated:\ \[clock\ format\ \[clock\ seconds\]\ -form..."\;\n\ \ \ \ (file\ "\;Check.tcl"\;\ line\ 8)\n\ \ \ \ invoked\ from\ within\n"\;source\ Check.tcl"\;\n\ \ \ \ (in\ namespace\ eval\ "\;::Check"\;\ script\ line\ 6)\n\ \ \ \ invoked\ from\ within\n"\;namespace\ eval\ Check\ \{\n\ \ \ \ \ \ \ global\ env\n\ \ \ \ \ \ \ source\ \[file\ join\ \$env(MIDASBASE)\ TclHttpd\ tcl\ Common\ common.tcl\]\n\ \ \ \ \ \ \ source\ \[file\ join\ \$env(MIDASB..."\;\n\ \ \ \ invoked\ from\ within\n"\;subst\ \{\n\n\[Doc_Dynamic\]\n\n\n<\;!\n\[global\ Httpd\;\ upvar\ #0\ Httpd\[set\ Httpd(currentSocket)\]\ data\;\ set\ ClientIPAddress\ \$data(ipaddr)\;\ set\ MyInfo\ \$data(self)\]\n>\;..."\;\n\ \ \ \ ("\;uplevel"\;\ body\ line\ 1)\n\ \ \ \ invoked\ from\ within\n"\;uplevel\ #0\ \[list\ subst\ \$script\]"\;\n\ \ \ \ (procedure\ "\;SubstFile"\;\ line\ 12)\n\ \ \ \ invoked\ from\ within\n"\;SubstFile\ /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119/TclHttpd/Html/AIDA/Check/Check.tml\ \{\}"\;\n\ \ \ \ ("\;uplevel"\;\ body\ line\ 1)\n\ \ \ \ invoked\ from\ within\n"\;uplevel\ 1\ \[list\ SubstFile\ \$path\ \$interp\]"\;\n\ \ \ \ (procedure\ "\;Subst_File"\;\ line\ 7)\n\ \ \ \ invoked\ from\ within\n"\;Subst_File\ \$template\ \$interp"\;\n\ \ \ \ invoked\ from\ within\n"\;TemplateInstantiate\ \$sock\ \$path\ \{\}\ \$suffix\ \{\}\ \$Template(templateInterp)"\;\n\ \ \ \ (procedure\ "\;Doc_application/x-tcl-template"\;\ line\ 9)\n\ \ \ \ invoked\ from\ within\n"\;\$cmd\ \$path\ \$suffix\ \$sock"\;\n\ \ \ \ (procedure\ "\;Doc_Handle"\;\ line\ 20)\n\ \ \ \ invoked\ from\ within\n"\;Doc_Handle\ \$prefix\ \$path\ \$suffix\ \$sock"\;\n\ \ \ \ (procedure\ "\;DocDomain"\;\ line\ 44)\n\ \ \ \ invoked\ from\ within\n"\;DocDomain\ /\ /MIDAS/TclHttpd/Html\ sock8\ AIDA/Check/Check.tml"\;\n\ \ \ \ ("\;eval"\;\ body\ line\ 1)\n\ \ \ \ invoked\ from\ within\n"\;eval\ \$Url(command,\$prefix)\ \[list\ \$sock\ \$suffix\]"\; 5: Doc_Error sock8 can't\ read\ \"TS_Base_Collected\":\ no\ such\ variable\n\ \ \ \ while\ executing\n\"if\ \{\ \$TS_Base_Collected\ ==\ 0\ \}\ \{\ collect_TS_Base\ \}\"\n\ \ \ \ (procedure\ \"do_TS_errors\"\ line\ 8)\n\ \ \ \ invoked\ from\ within\n\"do_TS_errors\ \"\n\ \ \ \ (\"TSERRORS\"\ arm\ line\ 1)\n\ \ \ \ invoked\ from\ within\n\"switch\ -glob\ \$w\ \{\n\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ RESET\ \ \ \ \{set\ started\ 0\}\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ CLEAR\ \ \ \ \{EmptyLog\}\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ ELOG\ \ \ \ \ \{PrintLog\}\n\ \ \ \ \ \ \ \ \ ...\"\n\ \ \ \ (procedure\ \"do_click\"\ line\ 23)\n\ \ \ \ invoked\ from\ within\n\"do_click\"\n\ \ \ \ invoked\ from\ within\n\"if\ \{\$started\ !=\ 0\}\ \{\n\n\ \ \ \ \ variable\ JS\ \"\"\n\ \ \ \ \ variable\ LogFlag\n\n\ \ \ \ \ if\ \{\$LogFlag\ ==\ 1\}\ \{InsertLog\ \"Last\ Updated:\ \[clock\ format\ \[clock\ seconds\]\ -form...\"\n\ \ \ \ (file\ \"Check.tcl\"\ line\ 8)\n\ \ \ \ invoked\ from\ within\n\"source\ Check.tcl\"\n\ \ \ \ (in\ namespace\ eval\ \"::Check\"\ script\ line\ 6)\n\ \ \ \ invoked\ from\ within\n\"namespace\ eval\ Check\ \{\n\ \ \ \ \ \ \ global\ env\n\ \ \ \ \ \ \ source\ \[file\ join\ \$env(MIDASBASE)\ TclHttpd\ tcl\ Common\ common.tcl\]\n\ \ \ \ \ \ \ source\ \[file\ join\ \$env(MIDASB...\"\n\ \ \ \ invoked\ from\ within\n\"subst\ \{\n\n\[Doc_Dynamic\]\n\n\n<!\n\[global\ Httpd\;\ upvar\ #0\ Httpd\[set\ Httpd(currentSocket)\]\ data\;\ set\ ClientIPAddress\ \$data(ipaddr)\;\ set\ MyInfo\ \$data(self)\]\n>...\"\n\ \ \ \ (\"uplevel\"\ body\ line\ 1)\n\ \ \ \ invoked\ from\ within\n\"uplevel\ #0\ \[list\ subst\ \$script\]\"\n\ \ \ \ (procedure\ \"SubstFile\"\ line\ 12)\n\ \ \ \ invoked\ from\ within\n\"SubstFile\ /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119/TclHttpd/Html/AIDA/Check/Check.tml\ \{\}\"\n\ \ \ \ (\"uplevel\"\ body\ line\ 1)\n\ \ \ \ invoked\ from\ within\n\"uplevel\ 1\ \[list\ SubstFile\ \$path\ \$interp\]\"\n\ \ \ \ (procedure\ \"Subst_File\"\ line\ 7)\n\ \ \ \ invoked\ from\ within\n\"Subst_File\ \$template\ \$interp\"\n\ \ \ \ invoked\ from\ within\n\"TemplateInstantiate\ \$sock\ \$path\ \{\}\ \$suffix\ \{\}\ \$Template(templateInterp)\"\n\ \ \ \ (procedure\ \"Doc_application/x-tcl-template\"\ line\ 9)\n\ \ \ \ invoked\ from\ within\n\"\$cmd\ \$path\ \$suffix\ \$sock\"\n\ \ \ \ (procedure\ \"Doc_Handle\"\ line\ 20)\n\ \ \ \ invoked\ from\ within\n\"Doc_Handle\ \$prefix\ \$path\ \$suffix\ \$sock\"\n\ \ \ \ (procedure\ \"DocDomain\"\ line\ 44)\n\ \ \ \ invoked\ from\ within\n\"DocDomain\ /\ /MIDAS/TclHttpd/Html\ sock8\ AIDA/Check/Check.tml\"\n\ \ \ \ (\"eval\"\ body\ line\ 1)\n\ \ \ \ invoked\ from\ within\n\"eval\ \$Url(command,\$prefix)\ \[list\ \$sock\ \$suffix\]\" 4: Url_Unwind sock8 can't\ read\ \"TS_Base_Collected\":\ no\ such\ variable\n\ \ \ \ while\ executing\n\"if\ \{\ \$TS_Base_Collected\ ==\ 0\ \}\ \{\ collect_TS_Base\ \}\"\n\ \ \ \ (procedure\ \"do_TS_errors\"\ line\ 8)\n\ \ \ \ invoked\ from\ within\n\"do_TS_errors\ \"\n\ \ \ \ (\"TSERRORS\"\ arm\ line\ 1)\n\ \ \ \ invoked\ from\ within\n\"switch\ -glob\ \$w\ \{\n\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ RESET\ \ \ \ \{set\ started\ 0\}\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ CLEAR\ \ \ \ \{EmptyLog\}\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ ELOG\ \ \ \ \ \{PrintLog\}\n\ \ \ \ \ \ \ \ \ ...\"\n\ \ \ \ (procedure\ \"do_click\"\ line\ 23)\n\ \ \ \ invoked\ from\ within\n\"do_click\"\n\ \ \ \ invoked\ from\ within\n\"if\ \{\$started\ !=\ 0\}\ \{\n\n\ \ \ \ \ variable\ JS\ \"\"\n\ \ \ \ \ variable\ LogFlag\n\n\ \ \ \ \ if\ \{\$LogFlag\ ==\ 1\}\ \{InsertLog\ \"Last\ Updated:\ \[clock\ format\ \[clock\ seconds\]\ -form...\"\n\ \ \ \ (file\ \"Check.tcl\"\ line\ 8)\n\ \ \ \ invoked\ from\ within\n\"source\ Check.tcl\"\n\ \ \ \ (in\ namespace\ eval\ \"::Check\"\ script\ line\ 6)\n\ \ \ \ invoked\ from\ within\n\"namespace\ eval\ Check\ \{\n\ \ \ \ \ \ \ global\ env\n\ \ \ \ \ \ \ source\ \[file\ join\ \$env(MIDASBASE)\ TclHttpd\ tcl\ Common\ common.tcl\]\n\ \ \ \ \ \ \ source\ \[file\ join\ \$env(MIDASB...\"\n\ \ \ \ invoked\ from\ within\n\"subst\ \{\n\n\[Doc_Dynamic\]\n\n\n<!\n\[global\ Httpd\;\ upvar\ #0\ Httpd\[set\ Httpd(currentSocket)\]\ data\;\ set\ ClientIPAddress\ \$data(ipaddr)\;\ set\ MyInfo\ \$data(self)\]\n>...\"\n\ \ \ \ (\"uplevel\"\ body\ line\ 1)\n\ \ \ \ invoked\ from\ within\n\"uplevel\ #0\ \[list\ subst\ \$script\]\"\n\ \ \ \ (procedure\ \"SubstFile\"\ line\ 12)\n\ \ \ \ invoked\ from\ within\n\"SubstFile\ /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119/TclHttpd/Html/AIDA/Check/Check.tml\ \{\}\"\n\ \ \ \ (\"uplevel\"\ body\ line\ 1)\n\ \ \ \ invoked\ from\ within\n\"uplevel\ 1\ \[list\ SubstFile\ \$path\ \$interp\]\"\n\ \ \ \ (procedure\ \"Subst_File\"\ line\ 7)\n\ \ \ \ invoked\ from\ within\n\"Subst_File\ \$template\ \$interp\"\n\ \ \ \ invoked\ from\ within\n\"TemplateInstantiate\ \$sock\ \$path\ \{\}\ \$suffix\ \{\}\ \$Template(templateInterp)\"\n\ \ \ \ (procedure\ \"Doc_application/x-tcl-template\"\ line\ 9)\n\ \ \ \ invoked\ from\ within\n\"\$cmd\ \$path\ \$suffix\ \$sock\"\n\ \ \ \ (procedure\ \"Doc_Handle\"\ line\ 20)\n\ \ \ \ invoked\ from\ within\n\"Doc_Handle\ \$prefix\ \$path\ \$suffix\ \$sock\"\n\ \ \ \ (procedure\ \"DocDomain\"\ line\ 44)\n\ \ \ \ invoked\ from\ within\n\"DocDomain\ /\ /MIDAS/TclHttpd/Html\ sock8\ AIDA/Check/Check.tml\"\n\ \ \ \ (\"eval\"\ body\ line\ 1)\n\ \ \ \ invoked\ from\ within\n\"eval\ \$Url(command,\$prefix)\ \[list\ \$sock\ \$suffix\]\" NONE 3: Url_DeferredDispatch / AIDA/Check/Check.tml sock8 buffer {} 2: HttpdReadPost sock8 buffer 16384 {Url_DeferredDispatch / AIDA/Check/Check.tml} 1: HttpdReadPostGlobal sock8 Httpds Dead time check: [npg@aidas-gsi S496]$ ~/analyser/analyser R14_24 *** TDR format 3.3.0 analyser - TD - May 2021 *** ERROR: READ I/O error: 5002 blocks: 32000 ADC data format: 252995858 ( 451517.9 Hz) Other data format: 8924143 ( 15926.8 Hz) Sample trace data format: 0 ( 0.0 Hz) Undefined format: 0 ( 0.0 Hz) Other data format type: PAUSE: 85 ( 0.2 Hz) RESUME: 85 ( 0.2 Hz) SYNC100: 34087 ( 60.8 Hz) WR48-63: 34087 ( 60.8 Hz) FEE64 disc: 0 ( 0.0 Hz) MBS info: 8855799 ( 15804.8 Hz) Other info: 0 ( 0.0 Hz) ADC data range bit set: 0 ( 0.0 Hz) Timewarps: ADC: 0 ( 0.0 Hz) PAUSE: 0 ( 0.0 Hz) RESUME: 0 ( 0.0 Hz) SYNC100: 0 ( 0.0 Hz) WR48-63: 0 ( 0.0 Hz) FEE64 disc: 0 ( 0.0 Hz) MBS info: 0 ( 0.0 Hz) Undefined: 0 ( 0.0 Hz) Sample trace: 0 ( 0.0 Hz) *** Timestamp elapsed time: 560.323 s FEE elapsed dead time(s) elapsed idle time(s) 0 0.000 349.960 1 0.002 0.000 2 0.000 5.974 3 1.105 0.000 4 0.000 447.454 5 0.000 2.809 6 0.000 16.727 7 0.137 0.000 8 0.090 0.000 9 0.000 429.897 10 0.461 0.000 11 0.018 0.000 12 0.000 8.985 13 0.000 466.311 14 0.000 15.095 15 0.346 0.000 13:24 Beam is off at the moment... 13:43 Changing spill structure from 1.5s/3.5s to 1s. 13:50 [npg@aidas-gsi S496]$ ~/analyser/analyser R14_29 *** TDR format 3.3.0 analyser - TD - May 2021 *** ERROR: READ I/O error: 5002 blocks: 32000 ADC data format: 251763810 ( 930342.1 Hz) Other data format: 10156190 ( 37530.1 Hz) Sample trace data format: 0 ( 0.0 Hz) Undefined format: 0 ( 0.0 Hz) Other data format type: PAUSE: 205 ( 0.8 Hz) RESUME: 205 ( 0.8 Hz) SYNC100: 33009 ( 122.0 Hz) WR48-63: 33009 ( 122.0 Hz) FEE64 disc: 0 ( 0.0 Hz) MBS info: 10089762 ( 37284.7 Hz) Other info: 0 ( 0.0 Hz) ADC data range bit set: 527077 ( 1947.7 Hz) Timewarps: ADC: 0 ( 0.0 Hz) PAUSE: 0 ( 0.0 Hz) RESUME: 0 ( 0.0 Hz) SYNC100: 0 ( 0.0 Hz) WR48-63: 0 ( 0.0 Hz) FEE64 disc: 0 ( 0.0 Hz) MBS info: 0 ( 0.0 Hz) Undefined: 0 ( 0.0 Hz) Sample trace: 0 ( 0.0 Hz) *** Timestamp elapsed time: 270.614 s FEE elapsed dead time(s) elapsed idle time(s) 0 0.034 6.803 1 8.461 0.000 2 0.569 0.000 3 11.664 0.000 4 0.024 26.416 5 0.854 0.000 6 0.069 0.000 7 0.741 0.000 8 1.336 0.000 9 0.000 75.926 10 0.353 0.000 11 0.105 0.000 12 0.070 0.000 13 0.000 93.585 14 0.017 0.000 15 0.121 0.000 15:06 Spill structure back to 1.5 on 2s off.
Encoding
:
HTML
ELCode
plain
Suppress Email notification
Resubmit as new entry
Attachment 1:
210517_0200_Bias.png
Original size: 480x303
Attachment 2:
210517_0202_stats.png
Original size: 1906x452
Attachment 3:
210517_0202_temp.png
Original size: 1912x530
Attachment 4:
210517_0400_Bias.png
Original size: 470x309
Attachment 5:
210517_0403_stats.png
Original size: 1913x471
Attachment 6:
210517_0403_temp.png
Original size: 1914x520
Attachment 7:
210517_0740_Bias.png
Original size: 492x335
Attachment 8:
210517_0742_Stats.png
Original size: 1901x533
Attachment 9:
210517_0743_Temp.png
Original size: 1887x585
Attachment 10:
0833_Stats.png
Original size: 1916x549
Attachment 11:
0902_Stats.png
Original size: 1914x552
Attachment 12:
1241_Stats.png
Original size: 1915x465
Attachment 13:
1347_Stats.png
Original size: 1912x458
Attachment 14:
FEETemps_1042.png
Original size: 1919x557
Attachment 15:
FEETemps_1246.png
Original size: 1918x539
Attachment 16:
FEETemps_1350.png
Original size: 1915x538
Attachment 17:
leakage_currents_1302.png
Original size: 502x344
Attachment 18:
Drop attachments here...
Draft saved at 00:00:00
ELOG V3.1.4-unknown