AIDA
GELINA
BRIKEN
nToF
CRIB
ISOLDE
CIRCE
nTOFCapture
DESPEC
DTAS
EDI_PSA
179Ta
CARME
StellarModelling
DCF
K40
DESPEC
Draft saved at 00:00:00
Fields marked with
*
are required
Entry time:
Tue Apr 1 03:25:30 2025
Author
*
:
Subject
*
:
> Auther: Muneerah and Betool > 02:00 The dead time have been checked every 30 min from the start of the shift, it seemes Ok > Statistics ok - 210517_0202_Stats > Temperatures ok - 210517_0202_Temp > Bias and leakage ok - 210517_0200_Bias > 02:05 System wide checks: > Clock status test result: Passed 16, Failed 0 > > Understand status as follows > Status bit 3 : firmware PLL that creates clocks from external clock not locked > Status bit 2 : always logic '1' > Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock > Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock > If all these bits are not set then the operation of the firmware is unreliable > > FEE64 module aida02 failed > FEE64 module aida03 failed > FEE64 module aida06 failed > FEE64 module aida10 failed > Calibration test result: Passed 12, Failed 4 > > If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module > > Base Current Difference > aida05 fault 0x1552 : 0x155d : 11 > White Rabbit error counter test result: Passed 15, Failed 1 > > Understand the status reports as follows:- > Status bit 3 : White Rabbit decoder detected an error in the received data > Status bit 2 : Firmware registered WR error, no reload of Timestamp > Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR > > > Base Current Difference > aida05 fault 0x0 : 0x2 : 2 > FPGA Timestamp error counter test result: Passed 15, Failed 1 > If any of these counts are reported as in error > The ASIC readout system has detected a timeslip. > That is the timestamp read from the time FIFO is not younger than the last > > Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 > Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k > aida01 : 5 8 3 3 3 5 2 2 2 3 7 : 39460 > aida02 : 22 4 5 0 3 2 2 3 2 3 7 : 39560 > aida03 : 21 10 2 1 1 3 2 4 2 3 7 : 40100 > aida04 : 18 7 2 0 3 3 3 2 2 3 7 : 39392 > aida05 : 16 6 5 1 2 4 3 2 2 3 7 : 39520 > aida06 : 18 6 6 2 2 2 2 3 2 3 7 : 39576 > aida07 : 12 8 6 1 2 3 1 3 2 3 7 : 39408 > aida08 : 13 11 4 0 1 3 3 2 2 3 7 : 39308 > aida09 : 10 8 4 2 3 4 2 3 2 3 7 : 39848 > aida10 : 16 8 2 1 2 4 2 3 1 4 7 : 40768 > aida11 : 26 10 2 2 1 2 2 3 2 3 7 : 39512 > aida12 : 16 9 2 1 2 4 2 2 2 3 7 : 39240 > aida13 : 15 9 3 2 2 4 2 2 2 3 7 : 39284 > aida14 : 25 10 2 3 3 3 2 3 2 3 7 : 39796 > aida15 : 4 2 2 1 1 3 2 2 2 3 7 : 38944 > aida16 : 16 7 3 3 1 3 3 2 2 3 7 : 39368 > > 04:00 Statistics ok - 210517_0403_Stats > Temperatures ok - 210517_0403_Temp > Bias and leakage ok - 210517_0400_Bias > 04:05 System wide checks: > Clock status test result: Passed 16, Failed 0 > > Understand status as follows > Status bit 3 : firmware PLL that creates clocks from external clock not locked > Status bit 2 : always logic '1' > Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock > Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock > If all these bits are not set then the operation of the firmware is unreliable > > FEE64 module aida02 failed > FEE64 module aida03 failed > FEE64 module aida06 failed > FEE64 module aida10 failed > Calibration test result: Passed 12, Failed 4 > > If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module > > > Base Current Difference > aida05 fault 0x1552 : 0x1560 : 14 > White Rabbit error counter test result: Passed 15, Failed 1 > > Understand the status reports as follows:- > Status bit 3 : White Rabbit decoder detected an error in the received data > Status bit 2 : Firmware registered WR error, no reload of Timestamp > Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR > > > Base Current Difference > aida05 fault 0x0 : 0x2 : 2 > FPGA Timestamp error counter test result: Passed 15, Failed 1 > If any of these counts are reported as in error > The ASIC readout system has detected a timeslip. > That is the timestamp read from the time FIFO is not younger than the last > > > Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 > Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k > aida01 : 15 7 5 2 3 5 2 2 2 3 7 : 39492 > aida02 : 23 6 7 2 1 3 2 3 2 3 7 : 39676 > aida03 : 17 9 3 2 0 3 2 3 2 3 7 : 39548 > aida04 : 16 4 1 2 1 3 3 3 2 3 7 : 39792 > aida05 : 22 7 3 3 1 4 3 3 2 3 7 : 40032 > aida06 : 18 6 7 3 1 3 2 3 2 3 7 : 39688 > aida07 : 18 7 4 1 2 4 1 4 2 3 7 : 40032 > aida08 : 19 5 4 1 1 3 3 3 2 3 7 : 39828 > aida09 : 10 6 7 1 3 4 2 2 2 3 7 : 39336 > aida10 : 12 10 2 3 1 4 2 3 1 4 7 : 40768 > aida11 : 10 9 4 3 2 1 2 3 2 3 7 : 39440 > aida12 : 8 7 6 1 2 4 2 3 2 3 7 : 39768 > aida13 : 11 2 6 1 2 3 2 3 2 3 7 : 39612 > aida14 : 27 7 4 2 3 3 2 3 2 3 7 : 39780 > aida15 : 4 2 3 1 1 3 2 2 2 3 7 : 38960 > aida16 : 18 8 4 2 1 4 2 2 2 3 7 : 39240 > > > 06:27 aida07 has dropped out and lost connection with the merger. Powercycling AIDA > 06:48 Restarted system but the noise was extremely high. Equivalent to yesterday 15:00. Restarting again. > 07:01 it is working fine. > 07:40 Screen shots after the restart > Statistics ok - 210517_0742_Stats > Temperatures ok - 210517_0743_Temp > Bias and leakage ok - 210517_0740_Bias > > System wide checks > > > FEE64 module aida06 global clocks failed, 6 > Clock status test result: Passed 15, Failed 1 > > Understand status as follows > Status bit 3 : firmware PLL that creates clocks from external clock not locked > Status bit 2 : always logic '1' > Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock > Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock > If all these bits are not set then the operation of the firmware is unreliable > > > FEE64 module aida01 failed > FEE64 module aida02 failed > FEE64 module aida03 failed > FEE64 module aida04 failed > FEE64 module aida05 failed > FEE64 module aida06 failed > FEE64 module aida07 failed > FEE64 module aida08 failed > FEE64 module aida09 failed > FEE64 module aida10 failed > FEE64 module aida11 failed > FEE64 module aida12 failed > FEE64 module aida13 failed > FEE64 module aida14 failed > FEE64 module aida15 failed > FEE64 module aida16 failed > Calibration test result: Passed 0, Failed 16 > > If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module > > > Base Current Difference > aida05 fault 0x61da : 0x61dd : 3 > White Rabbit error counter test result: Passed 15, Failed 1 > > Understand the status reports as follows:- > Status bit 3 : White Rabbit decoder detected an error in the received data > Status bit 2 : Firmware registered WR error, no reload of Timestamp > Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR > > > > FPGA Timestamp error counter test result: Passed 16, Failed 0 > If any of these counts are reported as in error > The ASIC readout system has detected a timeslip. > That is the timestamp read from the time FIFO is not younger than the last > > > > Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 > Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k > aida01 : 20 6 3 1 1 3 1 3 3 3 7 : 40336 > aida02 : 11 6 0 1 1 5 2 4 2 3 7 : 40252 > aida03 : 0 5 2 3 3 3 3 4 2 3 7 : 40424 > aida04 : 2 4 2 1 2 4 3 3 2 3 7 : 39912 > aida05 : 11 7 5 1 2 2 4 2 3 3 7 : 40532 > aida06 : 20 6 0 2 3 2 1 3 3 3 7 : 40320 > aida07 : 15 5 1 2 2 3 2 2 3 3 7 : 40116 > aida08 : 2 3 4 1 3 3 2 2 3 3 7 : 40128 > aida09 : 18 5 1 3 2 4 2 2 3 3 7 : 40288 > aida10 : 11 6 3 4 2 3 3 2 2 4 7 : 41484 > aida11 : 4 4 5 2 2 2 3 3 2 3 7 : 39744 > aida12 : 18 7 2 3 2 2 4 3 2 3 7 : 40064 > aida13 : 2 2 1 3 0 3 2 4 2 3 7 : 39944 > aida14 : 16 4 5 3 3 3 3 2 2 4 7 : 41552 > aida15 : 9 3 1 1 2 3 2 3 3 3 7 : 40556 > aida16 : 16 5 1 2 1 2 3 4 2 3 7 : 40184 > > Date:17/05/2021 > Elog for shift: 08.00 to 12.00 > Author: Lewis Sexton > Screenshots saved to: /tmp/Elog_210517_0800_1200 > > ***** 08.30 ***** > > Checked R12_34 through analyser, dead time seems okay: > > *** Timestamp elapsed time: 232.154 s > FEE elapsed dead time(s) elapsed idle time(s) > 0 0.021 0.000 > 1 2.781 0.000 > 2 0.849 0.000 > 3 5.686 0.000 > 4 0.000 25.816 > 5 0.017 0.000 > 6 0.177 0.000 > 7 1.349 0.000 > 8 1.791 0.000 > 9 0.000 91.660 > 10 6.194 0.000 > 11 0.320 0.000 > 12 1.425 0.000 > 13 0.000 72.999 > 14 0.052 0.000 > 15 1.729 0.000 > 16 0.000 0.000 > 17 0.000 0.000 > 18 0.000 0.000 > 19 0.000 0.000 > 20 0.000 0.000 > 21 0.000 0.000 > 22 0.000 0.000 > 23 0.000 0.000 > 24 0.000 0.000 > 25 0.000 0.000 > 26 0.000 0.000 > 27 0.000 0.000 > 28 0.000 0.000 > 29 0.000 0.000 > 30 0.000 0.000 > 31 0.000 0.000 > 32 0.000 0.000 > > Stats checked seem okay (0833_Stats.png) > Merger rate ~3M items/s > Tape server ~12 MB/s > > ***** 09.00 ***** > > Checked R12_48 through analyser, dead time seems okay: > > *** Timestamp elapsed time: 170.459 s > FEE elapsed dead time(s) elapsed idle time(s) > 0 0.196 0.000 > 1 6.294 0.000 > 2 0.992 0.000 > 3 8.944 0.000 > 4 0.024 0.000 > 5 0.001 0.000 > 6 0.605 0.000 > 7 2.096 0.000 > 8 1.654 0.000 > 9 0.000 31.395 > 10 6.175 0.000 > 11 2.524 0.000 > 12 2.076 0.000 > 13 0.000 10.337 > 14 0.103 0.000 > 15 2.203 0.000 > 16 0.000 0.000 > 17 0.000 0.000 > 18 0.000 0.000 > 19 0.000 0.000 > 20 0.000 0.000 > 21 0.000 0.000 > 22 0.000 0.000 > 23 0.000 0.000 > 24 0.000 0.000 > 25 0.000 0.000 > 26 0.000 0.000 > 27 0.000 0.000 > 28 0.000 0.000 > 29 0.000 0.000 > 30 0.000 0.000 > 31 0.000 0.000 > 32 0.000 0.000 > > Stats checked seem okay (0902_Stats.png) > > ***** 09.30 ***** > > AIDA crash aida05 displaying 0 in stats > Power cycle performed with help of TD > Writing to R13 now > MBS relay restared > AIDA back up at 10.20 > > Ran analyser for R13_16 > > *** Timestamp elapsed time: 168.059 s > FEE elapsed dead time(s) elapsed idle time(s) > 0 0.047 0.000 > 1 2.282 0.000 > 2 0.080 0.000 > 3 0.806 0.000 > 4 0.000 0.000 > 5 0.260 0.000 > 6 0.030 0.000 > 7 1.471 0.000 > 8 67.467 0.000 > 9 80.552 0.000 > 10 0.057 0.000 > 11 0.000 11.671 > 12 1.191 0.000 > 13 0.080 0.000 > 14 0.091 0.000 > 15 0.202 0.000 > 16 0.000 0.000 > 17 0.000 0.000 > 18 0.000 0.000 > 19 0.000 0.000 > 20 0.000 0.000 > 21 0.000 0.000 > 22 0.000 0.000 > 23 0.000 0.000 > 24 0.000 0.000 > 25 0.000 0.000 > 26 0.000 0.000 > 27 0.000 0.000 > 28 0.000 0.000 > 29 0.000 0.000 > 30 0.000 0.000 > 31 0.000 0.000 > 32 0.000 0.000 > > High deadtime in aida09 and aida10, changed slow comparator threshold to 0xe (previously 0xc) > Ran analyser for R13_21 for comparision > > *** Timestamp elapsed time: 149.082 s > FEE elapsed dead time(s) elapsed idle time(s) > 0 0.000 0.000 > 1 3.580 0.000 > 2 0.000 0.000 > 3 2.800 0.000 > 4 0.000 0.000 > 5 0.514 0.000 > 6 0.140 0.000 > 7 1.372 0.000 > 8 55.544 0.000 > 9 66.588 0.000 > 10 1.065 0.000 > 11 0.000 10.010 > 12 0.681 0.000 > 13 0.044 0.000 > 14 0.070 0.000 > 15 0.400 0.000 > 16 0.000 0.000 > 17 0.000 0.000 > 18 0.000 0.000 > 19 0.000 0.000 > 20 0.000 0.000 > 21 0.000 0.000 > 22 0.000 0.000 > 23 0.000 0.000 > 24 0.000 0.000 > 25 0.000 0.000 > 26 0.000 0.000 > 27 0.000 0.000 > 28 0.000 0.000 > 29 0.000 0.000 > 30 0.000 0.000 > 31 0.000 0.000 > 32 0.000 0.000 > > FEE temp slightly high in aida04, 0.44 above limit of 65 (FEETemps_1042.png) > > Merger rate ~5M items/s > Tape server ~17 MB/s > > Raised slow comparator in aida09 and aida10 to 0xf, analyse R13_26 for comparision > > *** Timestamp elapsed time: 122.147 s > FEE elapsed dead time(s) elapsed idle time(s) > 0 0.027 0.000 > 1 19.945 0.000 > 2 0.601 0.000 > 3 21.555 0.000 > 4 0.042 0.000 > 5 3.226 0.000 > 6 1.211 0.000 > 7 9.674 0.000 > 8 52.817 0.000 > 9 54.972 0.000 > 10 1.848 0.000 > 11 0.008 0.000 > 12 2.034 0.000 > 13 0.278 0.000 > 14 0.018 0.000 > 15 0.634 0.000 > 16 0.000 0.000 > 17 0.000 0.000 > 18 0.000 0.000 > 19 0.000 0.000 > 20 0.000 0.000 > 21 0.000 0.000 > 22 0.000 0.000 > 23 0.000 0.000 > 24 0.000 0.000 > 25 0.000 0.000 > 26 0.000 0.000 > 27 0.000 0.000 > 28 0.000 0.000 > 29 0.000 0.000 > 30 0.000 0.000 > 31 0.000 0.000 > 32 0.000 0.000 > > high deatime now also seen in aida02 and aida04 > > Several powercycles performed as high rates after restart, analysis of R14_13 showed low deadtime for all FEEs > Back running as of 10.45 > Analyser of R14_15 to double check > > *** Timestamp elapsed time: 205.006 s > FEE elapsed dead time(s) elapsed idle time(s) > 0 0.000 40.176 > 1 0.438 0.000 > 2 0.199 0.000 > 3 2.927 0.000 > 4 0.000 140.347 > 5 1.282 0.000 > 6 0.150 0.000 > 7 2.097 0.000 > 8 1.673 0.000 > 9 0.000 87.749 > 10 1.150 0.000 > 11 0.489 0.000 > 12 0.374 0.000 > 13 0.048 20.995 > 14 0.251 0.000 > 15 2.441 0.000 > 16 0.000 0.000 > 17 0.000 0.000 > 18 0.000 0.000 > 19 0.000 0.000 > 20 0.000 0.000 > 21 0.000 0.000 > 22 0.000 0.000 > 23 0.000 0.000 > 24 0.000 0.000 > 25 0.000 0.000 > 26 0.000 0.000 > 27 0.000 0.000 > 28 0.000 0.000 > 29 0.000 0.000 > 30 0.000 0.000 > 31 0.000 0.000 > 32 0.000 0.000 > > > 12:04 During access Sultan added an additional ground cable and also noted the bias cable going into FEE12 was loose. He pushed it in solidly and the rates appear to have dropped across the FEEs > > Date:17/05/2021 > Elog for shift: 08.00 to 12.00 > Author: Lewis Sexton > Screenshots saved to: /tmp/Elog_210517_0800_1200 > > ***** 08.30 ***** > > Checked R12_34 through analyser, dead time seems okay: > > *** Timestamp elapsed time: 232.154 s > FEE elapsed dead time(s) elapsed idle time(s) > 0 0.021 0.000 > 1 2.781 0.000 > 2 0.849 0.000 > 3 5.686 0.000 > 4 0.000 25.816 > 5 0.017 0.000 > 6 0.177 0.000 > 7 1.349 0.000 > 8 1.791 0.000 > 9 0.000 91.660 > 10 6.194 0.000 > 11 0.320 0.000 > 12 1.425 0.000 > 13 0.000 72.999 > 14 0.052 0.000 > 15 1.729 0.000 > 16 0.000 0.000 > 17 0.000 0.000 > 18 0.000 0.000 > 19 0.000 0.000 > 20 0.000 0.000 > 21 0.000 0.000 > 22 0.000 0.000 > 23 0.000 0.000 > 24 0.000 0.000 > 25 0.000 0.000 > 26 0.000 0.000 > 27 0.000 0.000 > 28 0.000 0.000 > 29 0.000 0.000 > 30 0.000 0.000 > 31 0.000 0.000 > 32 0.000 0.000 > > Stats checked seem okay (0833_Stats.png) > Merger rate ~3M items/s > Tape server ~12 MB/s > > ***** 09.00 ***** > > Checked R12_48 through analyser, dead time seems okay: > > *** Timestamp elapsed time: 170.459 s > FEE elapsed dead time(s) elapsed idle time(s) > 0 0.196 0.000 > 1 6.294 0.000 > 2 0.992 0.000 > 3 8.944 0.000 > 4 0.024 0.000 > 5 0.001 0.000 > 6 0.605 0.000 > 7 2.096 0.000 > 8 1.654 0.000 > 9 0.000 31.395 > 10 6.175 0.000 > 11 2.524 0.000 > 12 2.076 0.000 > 13 0.000 10.337 > 14 0.103 0.000 > 15 2.203 0.000 > 16 0.000 0.000 > 17 0.000 0.000 > 18 0.000 0.000 > 19 0.000 0.000 > 20 0.000 0.000 > 21 0.000 0.000 > 22 0.000 0.000 > 23 0.000 0.000 > 24 0.000 0.000 > 25 0.000 0.000 > 26 0.000 0.000 > 27 0.000 0.000 > 28 0.000 0.000 > 29 0.000 0.000 > 30 0.000 0.000 > 31 0.000 0.000 > 32 0.000 0.000 > > Stats checked seem okay (0902_Stats.png) > > ***** 09.30 ***** > > AIDA crash aida05 displaying 0 in stats > Power cycle performed with help of TD > Writing to R13 now > MBS relay restared > AIDA back up at 10.20 > > Ran analyser for R13_16 > > *** Timestamp elapsed time: 168.059 s > FEE elapsed dead time(s) elapsed idle time(s) > 0 0.047 0.000 > 1 2.282 0.000 > 2 0.080 0.000 > 3 0.806 0.000 > 4 0.000 0.000 > 5 0.260 0.000 > 6 0.030 0.000 > 7 1.471 0.000 > 8 67.467 0.000 > 9 80.552 0.000 > 10 0.057 0.000 > 11 0.000 11.671 > 12 1.191 0.000 > 13 0.080 0.000 > 14 0.091 0.000 > 15 0.202 0.000 > 16 0.000 0.000 > 17 0.000 0.000 > 18 0.000 0.000 > 19 0.000 0.000 > 20 0.000 0.000 > 21 0.000 0.000 > 22 0.000 0.000 > 23 0.000 0.000 > 24 0.000 0.000 > 25 0.000 0.000 > 26 0.000 0.000 > 27 0.000 0.000 > 28 0.000 0.000 > 29 0.000 0.000 > 30 0.000 0.000 > 31 0.000 0.000 > 32 0.000 0.000 > > High deadtime in aida09 and aida10, changed slow comparator threshold to 0xe (previously 0xc) > Ran analyser for R13_21 for comparision > > *** Timestamp elapsed time: 149.082 s > FEE elapsed dead time(s) elapsed idle time(s) > 0 0.000 0.000 > 1 3.580 0.000 > 2 0.000 0.000 > 3 2.800 0.000 > 4 0.000 0.000 > 5 0.514 0.000 > 6 0.140 0.000 > 7 1.372 0.000 > 8 55.544 0.000 > 9 66.588 0.000 > 10 1.065 0.000 > 11 0.000 10.010 > 12 0.681 0.000 > 13 0.044 0.000 > 14 0.070 0.000 > 15 0.400 0.000 > 16 0.000 0.000 > 17 0.000 0.000 > 18 0.000 0.000 > 19 0.000 0.000 > 20 0.000 0.000 > 21 0.000 0.000 > 22 0.000 0.000 > 23 0.000 0.000 > 24 0.000 0.000 > 25 0.000 0.000 > 26 0.000 0.000 > 27 0.000 0.000 > 28 0.000 0.000 > 29 0.000 0.000 > 30 0.000 0.000 > 31 0.000 0.000 > 32 0.000 0.000 > > FEE temp slightly high in aida04, 0.44 above limit of 65 (FEETemps_1042.png) > > Merger rate ~5M items/s > Tape server ~17 MB/s > > Raised slow comparator in aida09 and aida10 to 0xf, analyse R13_26 for comparision > > *** Timestamp elapsed time: 122.147 s > FEE elapsed dead time(s) elapsed idle time(s) > 0 0.027 0.000 > 1 19.945 0.000 > 2 0.601 0.000 > 3 21.555 0.000 > 4 0.042 0.000 > 5 3.226 0.000 > 6 1.211 0.000 > 7 9.674 0.000 > 8 52.817 0.000 > 9 54.972 0.000 > 10 1.848 0.000 > 11 0.008 0.000 > 12 2.034 0.000 > 13 0.278 0.000 > 14 0.018 0.000 > 15 0.634 0.000 > 16 0.000 0.000 > 17 0.000 0.000 > 18 0.000 0.000 > 19 0.000 0.000 > 20 0.000 0.000 > 21 0.000 0.000 > 22 0.000 0.000 > 23 0.000 0.000 > 24 0.000 0.000 > 25 0.000 0.000 > 26 0.000 0.000 > 27 0.000 0.000 > 28 0.000 0.000 > 29 0.000 0.000 > 30 0.000 0.000 > 31 0.000 0.000 > 32 0.000 0.000 > > high deatime now also seen in aida02 and aida04 > > Several powercycles performed as high rates after restart, analysis of R14_13 showed low deadtime for all FEEs > Back running as of 10.45 > Analyser of R14_15 to double check > > *** Timestamp elapsed time: 205.006 s > FEE elapsed dead time(s) elapsed idle time(s) > 0 0.000 40.176 > 1 0.438 0.000 > 2 0.199 0.000 > 3 2.927 0.000 > 4 0.000 140.347 > 5 1.282 0.000 > 6 0.150 0.000 > 7 2.097 0.000 > 8 1.673 0.000 > 9 0.000 87.749 > 10 1.150 0.000 > 11 0.489 0.000 > 12 0.374 0.000 > 13 0.048 20.995 > 14 0.251 0.000 > 15 2.441 0.000 > 16 0.000 0.000 > 17 0.000 0.000 > 18 0.000 0.000 > 19 0.000 0.000 > 20 0.000 0.000 > 21 0.000 0.000 > 22 0.000 0.000 > 23 0.000 0.000 > 24 0.000 0.000 > 25 0.000 0.000 > 26 0.000 0.000 > 27 0.000 0.000 > 28 0.000 0.000 > 29 0.000 0.000 > 30 0.000 0.000 > 31 0.000 0.000 > 32 0.000 0.000 > > > 12:04 During access Sultan added an additional ground cable and also noted the bias cable going into FEE12 was loose. He pushed it in solidly and the rates appear to have dropped across the FEEs > > > 12:52 Check FPGA Timestamp errors gives following error message: > > Got the error Server Internal Error > while trying to obtain /AIDA/Check/Check.tml. > can't read "TS_Base_Collected": no such variable while executing "if { $TS_Base_Collected == 0 } { collect_TS_Base }" (procedure "do_TS_errors" line 8) invoked from within "do_TS_errors " ("TSERRORS" arm line 1) invoked from within "switch -glob $w { RESET {set started 0} CLEAR {EmptyLog} ELOG {PrintLog} ..." (procedure "do_click" line 23) invoked from within "do_click" invoked from within "if {$started != 0} { variable JS "" variable LogFlag if {$LogFlag == 1} {InsertLog "Last Updated: [clock format [clock seconds] -form..." (file "Check.tcl" line 8) invoked from within "source Check.tcl" (in namespace eval "::Check" script line 6) invoked from within "namespace eval Check { global env source [file join $env(MIDASBASE) TclHttpd tcl Common common.tcl] source [file join $env(MIDASB..." invoked from within "subst { [Doc_Dynamic] <! [global Httpd; upvar #0 Httpd[set Httpd(currentSocket)] data; set ClientIPAddress $data(ipaddr); set MyInfo $data(self)] >..." ("uplevel" body line 1) invoked from within "uplevel #0 [list subst $script]" (procedure "SubstFile" line 12) invoked from within "SubstFile /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119/TclHttpd/Html/AIDA/Check/Check.tml {}" ("uplevel" body line 1) invoked from within "uplevel 1 [list SubstFile $path $interp]" (procedure "Subst_File" line 7) invoked from within "Subst_File $template $interp" invoked from within "TemplateInstantiate $sock $path {} $suffix {} $Template(templateInterp)" (procedure "Doc_application/x-tcl-template" line 9) invoked from within "$cmd $path $suffix $sock" (procedure "Doc_Handle" line 20) invoked from within "Doc_Handle $prefix $path $suffix $sock" (procedure "DocDomain" line 44) invoked from within "DocDomain / /MIDAS/TclHttpd/Html sock8 AIDA/Check/Check.tml" ("eval" body line 1) invoked from within "eval $Url(command,$prefix) [list $sock $suffix]" > Tcl Call Trace > 6: DocSubstSystemFile sock8 error 500 can't\ read\ "\;TS_Base_Collected"\;:\ no\ such\ variable\n\ \ \ \ while\ executing\n"\;if\ \{\ \$TS_Base_Collected\ ==\ 0\ \}\ \{\ collect_TS_Base\ \}"\;\n\ \ \ \ (procedure\ "\;do_TS_errors"\;\ line\ 8)\n\ \ \ \ invoked\ from\ within\n"\;do_TS_errors\ "\;\n\ \ \ \ ("\;TSERRORS"\;\ arm\ line\ 1)\n\ \ \ \ invoked\ from\ within\n"\;switch\ -glob\ \$w\ \{\n\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ RESET\ \ \ \ \{set\ started\ 0\}\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ CLEAR\ \ \ \ \{EmptyLog\}\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ ELOG\ \ \ \ \ \{PrintLog\}\n\ \ \ \ \ \ \ \ \ ..."\;\n\ \ \ \ (procedure\ "\;do_click"\;\ line\ 23)\n\ \ \ \ invoked\ from\ within\n"\;do_click"\;\n\ \ \ \ invoked\ from\ within\n"\;if\ \{\$started\ !=\ 0\}\ \{\n\n\ \ \ \ \ variable\ JS\ "\;"\;\n\ \ \ \ \ variable\ LogFlag\n\n\ \ \ \ \ if\ \{\$LogFlag\ ==\ 1\}\ \{InsertLog\ "\;Last\ Updated:\ \[clock\ format\ \[clock\ seconds\]\ -form..."\;\n\ \ \ \ (file\ "\;Check.tcl"\;\ line\ 8)\n\ \ \ \ invoked\ from\ within\n"\;source\ Check.tcl"\;\n\ \ \ \ (in\ namespace\ eval\ "\;::Check"\;\ script\ line\ 6)\n\ \ \ \ invoked\ from\ within\n"\;namespace\ eval\ Check\ \{\n\ \ \ \ \ \ \ global\ env\n\ \ \ \ \ \ \ source\ \[file\ join\ \$env(MIDASBASE)\ TclHttpd\ tcl\ Common\ common.tcl\]\n\ \ \ \ \ \ \ source\ \[file\ join\ \$env(MIDASB..."\;\n\ \ \ \ invoked\ from\ within\n"\;subst\ \{\n\n\[Doc_Dynamic\]\n\n\n<\;!\n\[global\ Httpd\;\ upvar\ #0\ Httpd\[set\ Httpd(currentSocket)\]\ data\;\ set\ ClientIPAddress\ \$data(ipaddr)\;\ set\ MyInfo\ \$data(self)\]\n>\;..."\;\n\ \ \ \ ("\;uplevel"\;\ body\ line\ 1)\n\ \ \ \ invoked\ from\ within\n"\;uplevel\ #0\ \[list\ subst\ \$script\]"\;\n\ \ \ \ (procedure\ "\;SubstFile"\;\ line\ 12)\n\ \ \ \ invoked\ from\ within\n"\;SubstFile\ /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119/TclHttpd/Html/AIDA/Check/Check.tml\ \{\}"\;\n\ \ \ \ ("\;uplevel"\;\ body\ line\ 1)\n\ \ \ \ invoked\ from\ within\n"\;uplevel\ 1\ \[list\ SubstFile\ \$path\ \$interp\]"\;\n\ \ \ \ (procedure\ "\;Subst_File"\;\ line\ 7)\n\ \ \ \ invoked\ from\ within\n"\;Subst_File\ \$template\ \$interp"\;\n\ \ \ \ invoked\ from\ within\n"\;TemplateInstantiate\ \$sock\ \$path\ \{\}\ \$suffix\ \{\}\ \$Template(templateInterp)"\;\n\ \ \ \ (procedure\ "\;Doc_application/x-tcl-template"\;\ line\ 9)\n\ \ \ \ invoked\ from\ within\n"\;\$cmd\ \$path\ \$suffix\ \$sock"\;\n\ \ \ \ (procedure\ "\;Doc_Handle"\;\ line\ 20)\n\ \ \ \ invoked\ from\ within\n"\;Doc_Handle\ \$prefix\ \$path\ \$suffix\ \$sock"\;\n\ \ \ \ (procedure\ "\;DocDomain"\;\ line\ 44)\n\ \ \ \ invoked\ from\ within\n"\;DocDomain\ /\ /MIDAS/TclHttpd/Html\ sock8\ AIDA/Check/Check.tml"\;\n\ \ \ \ ("\;eval"\;\ body\ line\ 1)\n\ \ \ \ invoked\ from\ within\n"\;eval\ \$Url(command,\$prefix)\ \[list\ \$sock\ \$suffix\]"\; > 5: Doc_Error sock8 can't\ read\ \"TS_Base_Collected\":\ no\ such\ variable\n\ \ \ \ while\ executing\n\"if\ \{\ \$TS_Base_Collected\ ==\ 0\ \}\ \{\ collect_TS_Base\ \}\"\n\ \ \ \ (procedure\ \"do_TS_errors\"\ line\ 8)\n\ \ \ \ invoked\ from\ within\n\"do_TS_errors\ \"\n\ \ \ \ (\"TSERRORS\"\ arm\ line\ 1)\n\ \ \ \ invoked\ from\ within\n\"switch\ -glob\ \$w\ \{\n\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ RESET\ \ \ \ \{set\ started\ 0\}\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ CLEAR\ \ \ \ \{EmptyLog\}\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ ELOG\ \ \ \ \ \{PrintLog\}\n\ \ \ \ \ \ \ \ \ ...\"\n\ \ \ \ (procedure\ \"do_click\"\ line\ 23)\n\ \ \ \ invoked\ from\ within\n\"do_click\"\n\ \ \ \ invoked\ from\ within\n\"if\ \{\$started\ !=\ 0\}\ \{\n\n\ \ \ \ \ variable\ JS\ \"\"\n\ \ \ \ \ variable\ LogFlag\n\n\ \ \ \ \ if\ \{\$LogFlag\ ==\ 1\}\ \{InsertLog\ \"Last\ Updated:\ \[clock\ format\ \[clock\ seconds\]\ -form...\"\n\ \ \ \ (file\ \"Check.tcl\"\ line\ 8)\n\ \ \ \ invoked\ from\ within\n\"source\ Check.tcl\"\n\ \ \ \ (in\ namespace\ eval\ \"::Check\"\ script\ line\ 6)\n\ \ \ \ invoked\ from\ within\n\"namespace\ eval\ Check\ \{\n\ \ \ \ \ \ \ global\ env\n\ \ \ \ \ \ \ source\ \[file\ join\ \$env(MIDASBASE)\ TclHttpd\ tcl\ Common\ common.tcl\]\n\ \ \ \ \ \ \ source\ \[file\ join\ \$env(MIDASB...\"\n\ \ \ \ invoked\ from\ within\n\"subst\ \{\n\n\[Doc_Dynamic\]\n\n\n<!\n\[global\ Httpd\;\ upvar\ #0\ Httpd\[set\ Httpd(currentSocket)\]\ data\;\ set\ ClientIPAddress\ \$data(ipaddr)\;\ set\ MyInfo\ \$data(self)\]\n>...\"\n\ \ \ \ (\"uplevel\"\ body\ line\ 1)\n\ \ \ \ invoked\ from\ within\n\"uplevel\ #0\ \[list\ subst\ \$script\]\"\n\ \ \ \ (procedure\ \"SubstFile\"\ line\ 12)\n\ \ \ \ invoked\ from\ within\n\"SubstFile\ /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119/TclHttpd/Html/AIDA/Check/Check.tml\ \{\}\"\n\ \ \ \ (\"uplevel\"\ body\ line\ 1)\n\ \ \ \ invoked\ from\ within\n\"uplevel\ 1\ \[list\ SubstFile\ \$path\ \$interp\]\"\n\ \ \ \ (procedure\ \"Subst_File\"\ line\ 7)\n\ \ \ \ invoked\ from\ within\n\"Subst_File\ \$template\ \$interp\"\n\ \ \ \ invoked\ from\ within\n\"TemplateInstantiate\ \$sock\ \$path\ \{\}\ \$suffix\ \{\}\ \$Template(templateInterp)\"\n\ \ \ \ (procedure\ \"Doc_application/x-tcl-template\"\ line\ 9)\n\ \ \ \ invoked\ from\ within\n\"\$cmd\ \$path\ \$suffix\ \$sock\"\n\ \ \ \ (procedure\ \"Doc_Handle\"\ line\ 20)\n\ \ \ \ invoked\ from\ within\n\"Doc_Handle\ \$prefix\ \$path\ \$suffix\ \$sock\"\n\ \ \ \ (procedure\ \"DocDomain\"\ line\ 44)\n\ \ \ \ invoked\ from\ within\n\"DocDomain\ /\ /MIDAS/TclHttpd/Html\ sock8\ AIDA/Check/Check.tml\"\n\ \ \ \ (\"eval\"\ body\ line\ 1)\n\ \ \ \ invoked\ from\ within\n\"eval\ \$Url(command,\$prefix)\ \[list\ \$sock\ \$suffix\]\" > 4: Url_Unwind sock8 can't\ read\ \"TS_Base_Collected\":\ no\ such\ variable\n\ \ \ \ while\ executing\n\"if\ \{\ \$TS_Base_Collected\ ==\ 0\ \}\ \{\ collect_TS_Base\ \}\"\n\ \ \ \ (procedure\ \"do_TS_errors\"\ line\ 8)\n\ \ \ \ invoked\ from\ within\n\"do_TS_errors\ \"\n\ \ \ \ (\"TSERRORS\"\ arm\ line\ 1)\n\ \ \ \ invoked\ from\ within\n\"switch\ -glob\ \$w\ \{\n\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ RESET\ \ \ \ \{set\ started\ 0\}\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ CLEAR\ \ \ \ \{EmptyLog\}\n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ ELOG\ \ \ \ \ \{PrintLog\}\n\ \ \ \ \ \ \ \ \ ...\"\n\ \ \ \ (procedure\ \"do_click\"\ line\ 23)\n\ \ \ \ invoked\ from\ within\n\"do_click\"\n\ \ \ \ invoked\ from\ within\n\"if\ \{\$started\ !=\ 0\}\ \{\n\n\ \ \ \ \ variable\ JS\ \"\"\n\ \ \ \ \ variable\ LogFlag\n\n\ \ \ \ \ if\ \{\$LogFlag\ ==\ 1\}\ \{InsertLog\ \"Last\ Updated:\ \[clock\ format\ \[clock\ seconds\]\ -form...\"\n\ \ \ \ (file\ \"Check.tcl\"\ line\ 8)\n\ \ \ \ invoked\ from\ within\n\"source\ Check.tcl\"\n\ \ \ \ (in\ namespace\ eval\ \"::Check\"\ script\ line\ 6)\n\ \ \ \ invoked\ from\ within\n\"namespace\ eval\ Check\ \{\n\ \ \ \ \ \ \ global\ env\n\ \ \ \ \ \ \ source\ \[file\ join\ \$env(MIDASBASE)\ TclHttpd\ tcl\ Common\ common.tcl\]\n\ \ \ \ \ \ \ source\ \[file\ join\ \$env(MIDASB...\"\n\ \ \ \ invoked\ from\ within\n\"subst\ \{\n\n\[Doc_Dynamic\]\n\n\n<!\n\[global\ Httpd\;\ upvar\ #0\ Httpd\[set\ Httpd(currentSocket)\]\ data\;\ set\ ClientIPAddress\ \$data(ipaddr)\;\ set\ MyInfo\ \$data(self)\]\n>...\"\n\ \ \ \ (\"uplevel\"\ body\ line\ 1)\n\ \ \ \ invoked\ from\ within\n\"uplevel\ #0\ \[list\ subst\ \$script\]\"\n\ \ \ \ (procedure\ \"SubstFile\"\ line\ 12)\n\ \ \ \ invoked\ from\ within\n\"SubstFile\ /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119/TclHttpd/Html/AIDA/Check/Check.tml\ \{\}\"\n\ \ \ \ (\"uplevel\"\ body\ line\ 1)\n\ \ \ \ invoked\ from\ within\n\"uplevel\ 1\ \[list\ SubstFile\ \$path\ \$interp\]\"\n\ \ \ \ (procedure\ \"Subst_File\"\ line\ 7)\n\ \ \ \ invoked\ from\ within\n\"Subst_File\ \$template\ \$interp\"\n\ \ \ \ invoked\ from\ within\n\"TemplateInstantiate\ \$sock\ \$path\ \{\}\ \$suffix\ \{\}\ \$Template(templateInterp)\"\n\ \ \ \ (procedure\ \"Doc_application/x-tcl-template\"\ line\ 9)\n\ \ \ \ invoked\ from\ within\n\"\$cmd\ \$path\ \$suffix\ \$sock\"\n\ \ \ \ (procedure\ \"Doc_Handle\"\ line\ 20)\n\ \ \ \ invoked\ from\ within\n\"Doc_Handle\ \$prefix\ \$path\ \$suffix\ \$sock\"\n\ \ \ \ (procedure\ \"DocDomain\"\ line\ 44)\n\ \ \ \ invoked\ from\ within\n\"DocDomain\ /\ /MIDAS/TclHttpd/Html\ sock8\ AIDA/Check/Check.tml\"\n\ \ \ \ (\"eval\"\ body\ line\ 1)\n\ \ \ \ invoked\ from\ within\n\"eval\ \$Url(command,\$prefix)\ \[list\ \$sock\ \$suffix\]\" NONE > 3: Url_DeferredDispatch / AIDA/Check/Check.tml sock8 buffer {} > 2: HttpdReadPost sock8 buffer 16384 {Url_DeferredDispatch / AIDA/Check/Check.tml} > 1: HttpdReadPostGlobal sock8 Httpds > > Dead time check: > > [npg@aidas-gsi S496]$ ~/analyser/analyser R14_24 > *** TDR format 3.3.0 analyser - TD - May 2021 > *** ERROR: READ I/O error: 5002 > blocks: 32000 > ADC data format: 252995858 ( 451517.9 Hz) > Other data format: 8924143 ( 15926.8 Hz) > Sample trace data format: 0 ( 0.0 Hz) > Undefined format: 0 ( 0.0 Hz) > Other data format type: PAUSE: 85 ( 0.2 Hz) > RESUME: 85 ( 0.2 Hz) > SYNC100: 34087 ( 60.8 Hz) > WR48-63: 34087 ( 60.8 Hz) > FEE64 disc: 0 ( 0.0 Hz) > MBS info: 8855799 ( 15804.8 Hz) > Other info: 0 ( 0.0 Hz) > > ADC data range bit set: 0 ( 0.0 Hz) > > Timewarps: ADC: 0 ( 0.0 Hz) > PAUSE: 0 ( 0.0 Hz) > RESUME: 0 ( 0.0 Hz) > SYNC100: 0 ( 0.0 Hz) > WR48-63: 0 ( 0.0 Hz) > FEE64 disc: 0 ( 0.0 Hz) > MBS info: 0 ( 0.0 Hz) > Undefined: 0 ( 0.0 Hz) > Sample trace: 0 ( 0.0 Hz) > > *** Timestamp elapsed time: 560.323 s > FEE elapsed dead time(s) elapsed idle time(s) > 0 0.000 349.960 > 1 0.002 0.000 > 2 0.000 5.974 > 3 1.105 0.000 > 4 0.000 447.454 > 5 0.000 2.809 > 6 0.000 16.727 > 7 0.137 0.000 > 8 0.090 0.000 > 9 0.000 429.897 > 10 0.461 0.000 > 11 0.018 0.000 > 12 0.000 8.985 > 13 0.000 466.311 > 14 0.000 15.095 > 15 0.346 0.000 > > 13:24 Beam is off at the moment... > > 13:43 Changing spill structure from 1.5s/3.5s to 1s. > > 13:50 > > [npg@aidas-gsi S496]$ ~/analyser/analyser R14_29 > *** TDR format 3.3.0 analyser - TD - May 2021 > *** ERROR: READ I/O error: 5002 > blocks: 32000 > ADC data format: 251763810 ( 930342.1 Hz) > Other data format: 10156190 ( 37530.1 Hz) > Sample trace data format: 0 ( 0.0 Hz) > Undefined format: 0 ( 0.0 Hz) > Other data format type: PAUSE: 205 ( 0.8 Hz) > RESUME: 205 ( 0.8 Hz) > SYNC100: 33009 ( 122.0 Hz) > WR48-63: 33009 ( 122.0 Hz) > FEE64 disc: 0 ( 0.0 Hz) > MBS info: 10089762 ( 37284.7 Hz) > Other info: 0 ( 0.0 Hz) > > ADC data range bit set: 527077 ( 1947.7 Hz) > > Timewarps: ADC: 0 ( 0.0 Hz) > PAUSE: 0 ( 0.0 Hz) > RESUME: 0 ( 0.0 Hz) > SYNC100: 0 ( 0.0 Hz) > WR48-63: 0 ( 0.0 Hz) > FEE64 disc: 0 ( 0.0 Hz) > MBS info: 0 ( 0.0 Hz) > Undefined: 0 ( 0.0 Hz) > Sample trace: 0 ( 0.0 Hz) > > *** Timestamp elapsed time: 270.614 s > FEE elapsed dead time(s) elapsed idle time(s) > 0 0.034 6.803 > 1 8.461 0.000 > 2 0.569 0.000 > 3 11.664 0.000 > 4 0.024 26.416 > 5 0.854 0.000 > 6 0.069 0.000 > 7 0.741 0.000 > 8 1.336 0.000 > 9 0.000 75.926 > 10 0.353 0.000 > 11 0.105 0.000 > 12 0.070 0.000 > 13 0.000 93.585 > 14 0.017 0.000 > 15 0.121 0.000 > > 15:06 Spill structure back to 1.5 on 2s off.
Encoding
:
HTML
ELCode
plain
Suppress Email notification
Attachment 1:
Drop attachments here...
Draft saved at 00:00:00
ELOG V3.1.4-unknown