AIDA GELINA BRIKEN nToF CRIB ISOLDE CIRCE nTOFCapture DESPEC DTAS EDI_PSA 179Ta CARME StellarModelling DCF K40
  DESPEC, Page 20 of 37  ELOG logo
New entries since:Thu Jan 1 01:00:00 1970
ID Date Author Subjectdown
  249   Mon Apr 19 19:19:18 2021 CBMonday 19 April 20:00-24:00 shift
20:19 
Statistics OK (attach 1)
No faults found
Ucesb OK (attach 2)
Temps OK (attach 3)
Bias OK (attach 4)

System-wide checks - as per last check
aida09 fails clock (OK)
aida07 and 09 fail calibration (OK)

		 Base 		Current 	Difference
aida05 fault 	 0xdb9a : 	 0xdb9d : 	 3  
aida06 fault 	 0xb74f : 	 0xb752 : 	 3  
aida07 fault 	 0xd052 : 	 0xd05f : 	 13  
aida08 fault 	 0xe4c1 : 	 0xe4c4 : 	 3  
aida09 fault 	 0xaf4b : 	 0xaf4d : 	 2  
White Rabbit error counter test result: Passed 7, Failed 5

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
	
			 Base 		Current 		Difference
aida12 fault 	 0x0 : 	 0x29 : 	 41  
FPGA Timestamp error counter test result: Passed 11, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last

	
Returned 0 0 0 0 0 0 0 0 0 0 0 0  
Mem(KB) :	4	8	16	32	64	128	256	512	1k	2k	4k
aida01 :      0      3      2      2      2      5      2      3      2      3      6   : 35704
aida02 :     19      4      1      1      1      2      2      3      3      3      6   : 36316
aida03 :     12      9      2      1      2      2      2      3      3      3      6   : 36408
aida04 :     16      3      5      4      2      4      2      3      3      3      6   : 36776
aida05 :     41     10     10      3      3      3      3      3      3      3      6   : 37172
aida06 :     25     21      9      3      2      3      1      4      3      3      6   : 37116
aida07 :     10      7      1      2      1      3      2      2      3      3      6   : 35952
aida08 :     33      7      5      1      1      2      2      3      3      3      6   : 36460
aida09 :     12      7      4      2      2      2      2      2      2      4      6   : 36968
aida10 :     16      6      2      1      1      4      3      3      2      3      6   : 35824
aida11 :      2      0      1      2      2      2      2      4      2      3      6   : 35800
aida12 :     15      6      2      0      2      4      2      2      3      3      6   : 36108



20:48 Started compressing *old* runs on Screen 2, Workspace 5
nice -n10 gzip -v R[...]_*

21:00 Stats OK, UCESB OK, No faults

21:07 Go4 spectra erased by remote shifters.

21:15 Local shifters report correlations lost. Beam may be intermittently removed for checks.

21:30 Still no beam. DAQ issues locally. AIDA MBS link had to be restarted due to operator mistake.




21:37 Beam back. Stats OK, UCESB OK, No faults found.

System wide checks:
Clock - aida09 fails

All modules now fail ADC calibration
FEE64 module aida01 failed
FEE64 module aida02 failed
FEE64 module aida03 failed
FEE64 module aida04 failed
FEE64 module aida05 failed
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida08 failed
FEE64 module aida09 failed
FEE64 module aida10 failed
FEE64 module aida11 failed
FEE64 module aida12 failed
Calibration test result: Passed 0, Failed 12

	 Base 		Current 	Difference
aida05 fault 	 0xdb9a : 	 0xdb9d : 	 3  
aida06 fault 	 0xb74f : 	 0xb752 : 	 3  
aida07 fault 	 0xd052 : 	 0xd061 : 	 15  
aida08 fault 	 0xe4c1 : 	 0xe4c4 : 	 3  
aida09 fault 	 0xaf4b : 	 0xaf4d : 	 2  
White Rabbit error counter test result: Passed 7, Failed 5

	
			 Base 		Current 		Difference
aida12 fault 	 0x0 : 	 0x36 : 	 54  
FPGA Timestamp error counter test result: Passed 11, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last

	
Returned 0 0 0 0 0 0 0 0 0 0 0 0  
Mem(KB) :	4	8	16	32	64	128	256	512	1k	2k	4k
aida01 :      2      6      3      1      2      4      2      4      2      3      6   : 36104
aida02 :      2      3      0      2      2      4      2      2      3      3      6   : 36064
aida03 :      6      3      3      3      2      2      2      2      4      3      6   : 36928
aida04 :     15      5      8      4      3      3      2      3      3      3      6   : 36772
aida05 :     10      5      8      4      4      4      3      2      3      3      6   : 36688
aida06 :     14     17     10      3      3      3      1      4      3      3      6   : 37120
aida07 :      3      4      1      1      1      2      2      2      3      3      6   : 35740
aida08 :     12      5      7      2      2      4      1      2      3      3      6   : 35976
aida09 :      0      6      2      0      2      2      2      2      2      4      6   : 36816
aida10 :      4      2      5      2      1      4      2      4      2      3      6   : 36080
aida11 :     14      3      2      3      1      2      2      4      2      3      6   : 35856
aida12 :      1      2      0      2      1      4      1      3      3      3      6   : 36244

22:05 Stats OK (attach 5)
Temps OK (attach 6)
Bias OK (attach 7)

System wide checks as before - see above.

22:13 ADC calibration - Only 06,07 and 09 now fail
(TD re-calibrated remotely)

22:53 Beam off to allow access to S4 to reboot VME crate (akash).


22:58 Started compressing *old* runs on Screen 2, Workspace 5
nice -n10 gzip -v R[1-2][0-9]_* R[3][0-2]_*
aka everything *except* R33_*


23:11: Beam is back. Stats and UCESB as before.

23:24: Stats OK (attach 8)
Temps OK (attach 9)
Bias OK (attach 10)
UCESB OK (attach 11)
System-wide checks
Clock: 06 fails
Calibration: 06,07,09 fail

	 Base 		Current 	Difference
aida05 fault 	 0xdb9a : 	 0xdb9d : 	 3  
aida06 fault 	 0xb74f : 	 0xb752 : 	 3  
aida07 fault 	 0xd052 : 	 0xd066 : 	 20  
aida08 fault 	 0xe4c1 : 	 0xe4c4 : 	 3  
aida09 fault 	 0xaf4b : 	 0xaf4d : 	 2  
White Rabbit error counter test result: Passed 7, Failed 5

	 Base 		Current 	Difference
aida05 fault 	 0xdb9a : 	 0xdb9d : 	 3  
aida06 fault 	 0xb74f : 	 0xb752 : 	 3  
aida07 fault 	 0xd052 : 	 0xd066 : 	 20  
aida08 fault 	 0xe4c1 : 	 0xe4c4 : 	 3  
aida09 fault 	 0xaf4b : 	 0xaf4d : 	 2  
White Rabbit error counter test result: Passed 7, Failed 5

Returned 0 0 0 0 0 0 0 0 0 0 0 0  
Mem(KB) :	4	8	16	32	64	128	256	512	1k	2k	4k
aida01 :     15      8      2      2      3      4      3      3      2      3      6   : 35996
aida02 :      8     12      4      1      2      4      3      2      3      3      6   : 36448
aida03 :      2      4      2      4      3      3      1      3      3      3      6   : 36360
aida04 :      7      8      4      4      2      4      3      3      3      3      6   : 37020
aida05 :      3      5      7      4      3      3      3      3      3      3      6   : 36964
aida06 :      2      6      8      2      3      4      2      3      3      3      6   : 36792
aida07 :      8      4      5      2      0      3      1      2      3      4      6   : 37712
aida08 :     26     11      3      1      1      3      1      3      3      3      6   : 36304
aida09 :     22      4      2      2      3      4      1      3      3      3      6   : 36504
aida10 :     21      6      6      2      0      4      2      3      3      3      6   : 36644
aida11 :     22      7      2      3      2      1      2      4      2      3      6   : 35856
aida12 :     16      9      2      2      3      3      2      2      3      3      6   : 36136
Attachment 1: Stats1.png
Stats1.png
Attachment 2: Ucesb1.png
Ucesb1.png
Attachment 3: Temps1.png
Temps1.png
Attachment 4: Bias1.png
Bias1.png
Attachment 5: Stats2.png
Stats2.png
Attachment 6: Temp2.png
Temp2.png
Attachment 7: Bias2.png
Bias2.png
Attachment 8: Stats3.png
Stats3.png
Attachment 9: Temps3.png
Temps3.png
Attachment 10: Bias3.png
Bias3.png
Attachment 11: Ucesb2.png
Ucesb2.png
  331   Wed May 19 11:24:55 2021 RDPMonday 19 April 12 noon shift
System checks gave 11 passed/1 failed, except for ADC timimgs: 10 passed/2 failed.

FEE64 module aida09 global clocks failed, 6
 Clock status test result: Passed 11, Failed 1

Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1' 
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable

FEE64 module aida07 failed
FEE64 module aida09 failed
Calibration test result: Passed 10, Failed 2


		 Base 		Current 	Difference
aida07 fault 	 0xd052 : 	 0xd056 : 	 4  
White Rabbit error counter test result: Passed 11, Failed 1

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR


			 Base 		Current 		Difference
aida12 fault 	 0x0 : 	 0x16 : 	 22  
FPGA Timestamp error counter test result: Passed 11, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last

If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module


Returned 0 0 0 0 0 0 0 0 0 0 0 0  
Mem(KB) :	4	8	16	32	64	128	256	512	1k	2k	4k
aida01 :      2      6      2      0      5      5      1      2      3      3      6   : 36120
aida02 :     25     10      0      4      3      5      2      3      3      3      6   : 36980
aida03 :     22     12      5      2      1      3      2      3      3      3      6   : 36616
aida04 :     16      3      3      3      4      4      4      2      3      3      6   : 36840
aida05 :     14      5      6      2      3      4      4      2      3      3      6   : 36800
aida06 :     30     10      6      3      3      4      2      3      3      3      6   : 36936
aida07 :     27     11      1      0      0      3      2      3      3      3      6   : 36436
aida08 :     38      7      7      3      3      2      2      3      3      3      6   : 36704
aida09 :      4      6      4      2      4      4      1      4      3      3      6   : 37056
aida10 :     21     10      3      3      1      5      1      3      3      3      6   : 36596
aida11 :     24      8      2      1      4      3      2      4      2      3      6   : 36192
aida12 :     31     12      4      3      5      5      2      2      3      3      6   : 36668

Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same

 FEE : aida01 =>   Options file size is 1025 	 Last changed Mon Apr 19 10:32:22 CEST 2021
 FEE : aida02 =>   Options file size is 1014 	 Last changed Fri Apr 16 00:56:20 CEST 2021
 FEE : aida03 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:04 CEST 2021
 FEE : aida04 =>   Options file size is 1025 	 Last changed Sat Apr 17 06:07:36 CEST 2021
 FEE : aida05 =>   Options file size is 1025 	 Last changed Fri Apr 16 00:53:25 CEST 2021
 FEE : aida06 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:04 CEST 2021
 FEE : aida07 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:04 CEST 2021
 FEE : aida08 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:04 CEST 2021
 FEE : aida09 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:05 CEST 2021
 FEE : aida10 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:06 CEST 2021
 FEE : aida11 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:05 CEST 2021
 FEE : aida12 =>   Options file size is 1025 	 Last changed Mon Apr 19 09:05:25 CEST 2021

There should be a separate elog note about changing one of the bias voltages.

All histograms zeroed at 16:13.

16.20 DAQ continues file NULL/R33_156
      aida09 HEC fast comparator 0x2->0x1
      DSSSD#3 bias -100->-120V
Attachment 1: Screenshot_2021-04-19_Temperature_and_status_scan_aidas-gsi(3).png
Screenshot_2021-04-19_Temperature_and_status_scan_aidas-gsi(3).png
Attachment 2: Screenshot_2021-04-19_Tape_Service_(Expert)_aidas-gsi.png
Screenshot_2021-04-19_Tape_Service_(Expert)_aidas-gsi.png
Attachment 3: 62.png
62.png
  244   Mon Apr 19 06:46:15 2021 MSMonday 19 April 07:47-12:00 2021

07:47

    
Clock status test result: Passed 12, Failed 0

Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1' 
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable

    
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida10 failed
Calibration test result: Passed 9, Failed 3

If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module

 

    
         Base         Current     Difference
aida01 fault      0x7686 :      0x768a :      4  
aida02 fault      0x941d :      0x9421 :      4  
aida03 fault      0x7cd7 :      0x7cdb :      4  
aida04 fault      0xb86d :      0xb871 :      4  
aida05 fault      0x1a59 :      0x1a61 :      8  
aida06 fault      0x4f45 :      0x4f4d :      8  
aida07 fault      0x3bfc :      0x3c46 :      74  
aida08 fault      0xc7ce :      0xc7d5 :      7  
aida09 fault      0xb33b :      0xb33c :      1  
White Rabbit error counter test result: Passed 3, Failed 9

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

 

        
             Base         Current         Difference
aida09 fault      0x1 :      0x2 :      1  
aida12 fault      0xa :      0x35 :      43  
FPGA Timestamp error counter test result: Passed 10, Failed 2
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last

 

    
Returned 0 0 0 0 0 0 0 0 0 0 0 0  
Mem(KB) :    4    8    16    32    64    128    256    512    1k    2k    4k
aida01 :     19     18     10      7      1      3      2      4      2      3      6   : 36380
aida02 :     11     15      5      3      2      3      2      3      1      4      6   : 36692
aida03 :     15     11     13      3      4      3      3      3      2      3      6   : 36164
aida04 :     22     14     20      5      1      3      4      3      2      3      6   : 36456
aida05 :      7      7      5      2      1      2      3      4      2      3      6   : 36132
aida06 :     21     14      6      2      3      3      1      2      2      4      6   : 37028
aida07 :     11      8      7      4      2      3      2      2      4      3      6   : 37212
aida08 :     18     10      7      5      3      1      4      3      2      3      6   : 36072
aida09 :     24     23     18      3      2      3      2      4      2      3      6   : 36504
aida10 :      6     19      5      2      2      2      2      3      2      3      6   : 35520
aida11 :     22     15      7      2      1      3      2      3      2      3      6   : 35648
aida12 :     35     13      8      2      5      3      1      3      3      3      6   : 36724

 

At 8:30 DAQ and Temperature Scan stopped working properly.

 

 

11:00

It was noted that at the time that AIDA dropped out ~8:30 the beam also dropped out. It seems a bit of a coincidence

TD and MS powercycled the FEEs after the first crash and restarted MIDAS but not the merger. This recovered the FEEs but did not re-establish the links between the FEEs and the Merger.

Another powercycle was performed this time with a full reset of the merger and the links were restored. Upon restoring there was a large amount of noise in across all FEEs on average a 50% increase across all FEEs but in DSSD a factor of 4-6 increase was common.

In the waveforms large 100kHz transiets could be seen - attachment 4

A further power-cycle was performed by OH. The rates following this powercycle are better than just before the power cycle but have not recovered to pre-glitch levels.

The waveforms here are much improved though - attachment 5 and 6

 

 

11:45

FEE64 module aida09 global clocks failed, 6
 Clock status test result: Passed 11, Failed 1

Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1' 
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable

 

 

FEE64 module aida01 failed
FEE64 module aida02 failed
FEE64 module aida03 failed
FEE64 module aida04 failed
FEE64 module aida05 failed
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida08 failed
FEE64 module aida09 failed
FEE64 module aida10 failed
FEE64 module aida11 failed
FEE64 module aida12 failed
Calibration test result: Passed 0, Failed 12

If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module

 

 

    
White Rabbit error counter test result: Passed 12, Failed 0

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

 

FPGA Timestamp error counter test result: Passed 12, Failed 0
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last

 

    
Returned 0 0 0 0 0 0 0 0 0 0 0 0  
Mem(KB) :    4    8    16    32    64    128    256    512    1k    2k    4k
aida01 :      1      2      2      1      2      4      2      4      2      3      7   : 40148
aida02 :      5      4      2      2      1      2      2      3      2      3      7   : 39380
aida03 :      3      2      1      1      1      3      1      4      2      3      7   : 39692
aida04 :      4      2      1      0      1      3      3      3      2      3      7   : 39664
aida05 :      2      7      2      0      2      2      3      4      2      3      7   : 40160
aida06 :      2      3      2      2      2      2      1      3      3      3      7   : 40192
aida07 :     23      7      2      0      0      3      2      4      2      3      7   : 39988
aida08 :     23     11      4      0      1      3      1      4      2      3      7   : 39860
aida09 :      1      3      1      1      2      4      1      3      1      4      7   : 40396
aida10 :      1      3      3      1      1      4      1      2      3      3      7   : 39852
aida11 :      3      0      2      3      1      2      3      2      2      3      7   : 39116
aida12 :      5      8      0      3      1      4      2      3      2      3      7   : 39668

 

 

 

 

 

 

 

 

Attachment 1: Screenshot_2021-04-19_Statistics_aidas-gsi.png
Screenshot_2021-04-19_Statistics_aidas-gsi.png
Attachment 2: Screenshot_2021-04-19_Temperature_and_status_scan_aidas-gsi.png
Screenshot_2021-04-19_Temperature_and_status_scan_aidas-gsi.png
Attachment 3: 42.png
42.png
Attachment 4: Screenshot_2021-04-19_Spectrum_Browser_aidas-gsi(1).png
Screenshot_2021-04-19_Spectrum_Browser_aidas-gsi(1).png
Attachment 5: 210319_1054_Layout7.png
210319_1054_Layout7.png
Attachment 6: 210419_1054_Layout8.png
210419_1054_Layout8.png
Attachment 7: Screenshot_2021-04-19_Statistics_aidas-gsi(1).png
Screenshot_2021-04-19_Statistics_aidas-gsi(1).png
Attachment 8: Screenshot_2021-04-19_Temperature_and_status_scan_aidas-gsi(1).png
Screenshot_2021-04-19_Temperature_and_status_scan_aidas-gsi(1).png
Attachment 9: 43.png
43.png
  320   Mon May 17 15:14:35 2021 LJWMonday 17th May 2021 16:00-20:00

16:25

Informed at shift handover that there has been no beam.

Statistics  - See attachment 1

Temperatures - See attachment 2

Bias & Leakage currents - See attachment 3

System Wide checks - All passed, EXCEPT:

ADC Calibration

   
FEE64 module aida01 failed
FEE64 module aida02 failed
FEE64 module aida03 failed
FEE64 module aida04 failed
FEE64 module aida05 failed
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida08 failed
FEE64 module aida09 failed
FEE64 module aida10 failed
FEE64 module aida11 failed
FEE64 module aida12 failed
FEE64 module aida13 failed
FEE64 module aida14 failed
FEE64 module aida15 failed
FEE64 module aida16 failed
Calibration test result: Passed 0, Failed 16

If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module

 

White Rabbit decoder status:

   
         Base         Current     Difference
aida05 fault      0x4da :      0x4dd :      3  
White Rabbit error counter test result: Passed 15, Failed 1

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

 

18:00

Usecb scalars now showing DSSD implant rates.

Statistics  - See attachment 4

Temperatures - See attachment 5

Bias & Leakage currents - See attachment 6

System Wide checks - All passed, EXCEPT:

ADC Calibration

   
FEE64 module aida01 failed
FEE64 module aida02 failed
FEE64 module aida03 failed
FEE64 module aida04 failed
FEE64 module aida05 failed
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida08 failed
FEE64 module aida09 failed
FEE64 module aida10 failed
FEE64 module aida11 failed
FEE64 module aida12 failed
FEE64 module aida13 failed
FEE64 module aida14 failed
FEE64 module aida15 failed
FEE64 module aida16 failed
Calibration test result: Passed 0, Failed 16

If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module

White rabbit decoder status

   
         Base         Current     Difference
aida05 fault      0x4da :      0x4e1 :      7  
White Rabbit error counter test result: Passed 15, Failed 1

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WRx

 

19:45

Usecb scalars have continued to show DSSD implant rates since the last recorded elog check.

Statistics  - See attachment 7

Temperatures - See attachment 8

Bias & Leakage currents - See attachment 9

System Wide checks - All passed, EXCEPT:

ADC Calibration:

   
FEE64 module aida01 failed
FEE64 module aida02 failed
FEE64 module aida03 failed
FEE64 module aida04 failed
FEE64 module aida05 failed
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida08 failed
FEE64 module aida09 failed
FEE64 module aida10 failed
FEE64 module aida11 failed
FEE64 module aida12 failed
FEE64 module aida13 failed
FEE64 module aida14 failed
FEE64 module aida15 failed
FEE64 module aida16 failed
Calibration test result: Passed 0, Failed 16

If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module


White Rabbit decoder status:

     Base         Current     Difference
aida05 fault      0x4da :      0x4e1 :      7  
White Rabbit error counter test result: Passed 15, Failed 1

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

 

19:50 - Just as completing checks, notced usesb rates not showing DSSD implants again for several minutes

19:55 - Beam now back again - DSSD implants showing

 

 

Attachment 1: 170521_1.png
170521_1.png
Attachment 2: 170521_2.png
170521_2.png
Attachment 3: 170521_3.png
170521_3.png
Attachment 4: 170521_4.png
170521_4.png
Attachment 5: 170521_5.png
170521_5.png
Attachment 6: 170521_6.png
170521_6.png
Attachment 7: 170521_7.png
170521_7.png
Attachment 8: 170521_8.png
170521_8.png
Attachment 9: 170521_9.png
170521_9.png
  321   Mon May 17 19:05:23 2021 LSMonday 17th May 20.00-24.00
20.30
Checked stats, seem okay (Fig 1)
Ran R14_99 through analyser, dead times all good (Fig 2)
Merger rate 1.8M items/s
Tape server 6.4MB/s

21.00
Checked stats seem okay (Fig 3)
Ran R14_106 through analyser, dead times all good (Fig 4)

21.30
Checked stats seem okay (Fig 5)
Ran R14_112 through analyser, dead times all good (Fig 6)

22.00
Stats okay (Fig 7)
FEE temps okay (Fig 8)
Rate spectra (Fig 9)
Leakage currents (Fig 10)

System wide checks:
Clock status all passed

White rabbit 1 fail:
		 Base 		Current 	Difference
aida05 fault 	 0x4da : 	 0x4e7 : 	 13  
White Rabbit error counter test result: Passed 15, Failed 1

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

FPGA timestamp all passed

Merger rate 2.4M items/s
Tape server 6.3MB/s

Ran R14_119 through analyser, dead times all good (Fig 11)

22.30
Checked stats seem okay (Fig 12)
Ran R14_123 through analyser, dead times all good (Fig 13)

23.00
Checked stats seem okay (Fig 14)
Ran R14_129 through analyser, dead times all good (Fig 15)

23.30
Checked stats seem okay (Fig 16)
Ran R14_135 through analyser, dead times all good (Fig 17)

24.00
Stats okay (Fig 18)
FEE temps okay (Fig 19)
Leakage currents (Fig 20)

System wide checks:
Clock status all passed

White rabbit 1 fail:
		 Base 		Current 	Difference
aida05 fault 	 0x4da : 	 0x4eb : 	 17  
White Rabbit error counter test result: Passed 15, Failed 1

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

FPGA timestamp all passed

Merger rate 2.5M items/s
Tape server 6.5MB/s

Ran R14_142 through analyser, dead times all good (Fig 21)
Attachment 1: 210517_Stats_2026.png
210517_Stats_2026.png
Attachment 2: 210517_Deadtime_R14_99.png
210517_Deadtime_R14_99.png
Attachment 3: 210517_Stats_2101.png
210517_Stats_2101.png
Attachment 4: 210517_Deadtimes_R14_106.png
210517_Deadtimes_R14_106.png
Attachment 5: 210517_Stats_2128.png
210517_Stats_2128.png
Attachment 6: 210517_Deadtimes_R14_112.png
210517_Deadtimes_R14_112.png
Attachment 7: 210517_Stats_2159.png
210517_Stats_2159.png
Attachment 8: 210517_FEEtemps_2159.png
210517_FEEtemps_2159.png
Attachment 9: 210517_Spectra_2159.png
210517_Spectra_2159.png
Attachment 10: 210517_Bias_2159.png
210517_Bias_2159.png
Attachment 11: 210517_Deadtimes_R14_119.png
210517_Deadtimes_R14_119.png
Attachment 12: 210517_Stats_2230.png
210517_Stats_2230.png
Attachment 13: 210517_Deadtimes_R14_123.png
210517_Deadtimes_R14_123.png
Attachment 14: 210517_Stats_2300.png
210517_Stats_2300.png
Attachment 15: 210517_Deadtimes_R14_129.png
210517_Deadtimes_R14_129.png
Attachment 16: 210517_Stats_2328.png
210517_Stats_2328.png
Attachment 17: 210517_Deadtimes_R14_135.png
210517_Deadtimes_R14_135.png
Attachment 18: 210517_Stats_2359.png
210517_Stats_2359.png
Attachment 19: 210517_FEEtemps_2359.png
210517_FEEtemps_2359.png
Attachment 20: 210517_bias_2359.png
210517_bias_2359.png
Attachment 21: 210517_deadtime_R14_142_1.png
210517_deadtime_R14_142_1.png
  666   Mon Jun 17 02:13:21 2024 TDMonday 17 June
03.11 DSSSD bias & leakage current OK - attachment 1

      FEE64 temperatures OK - attachment 2

      ADC data item stats OK - attachment 3

      per FEE64 Rate spectra - attachment 4

05.33 DSSSD bias & leakage current OK - attachment 5

      FEE64 temperatures OK - attachment 6

      ADC data item stats OK - attachment 7

      per FEE64 Rate spectra - attachment 8

      per FEE64 1.8.L spectra - attachments 9-11
       aida09 pulser walkthrough 60 ch FWHM

      per FEE64 1.8.W spectra - 20us FSR - attachments 12-14

      Merger, TapeServer etc - attachments 15-16

05.40 analysis data file R9_1042 - attachment 17

      Check ASIC load x2

      aida02 zero rate
      restart Merger

05.55 Data file S181/R10
      Pulser walkthrough

      All histograms zero'd

      BNC PB-5
      Amplitudes 10.0-1.0 @ 1.0 step
      Attenuation x10
      Polarity +
      tau_d 1ms
      Frequency 25Hz

      per p+n FEE64 1.8.L spectra - attachment 18

      per FEE64 Rate spectra - attachment 19

06.43 TapeServer no storage mode

09.14 Data file S181/R11
      Pulser walkthrough

      All histograms zero'd

      BNC PB-5
      Amplitudes 10.0-1.0 @ 1.0 step
      Attenuation x10
      Polarity -
      tau_d 1ms
      Frequency 25Hz

09.52 TapeServer no storage mode

      per n+n FEE64 1.8.L spectra - attachment 20

15.18 DEFGAS/FATIMA arrays moved downstream to permit installation of calibration source at centre of each array - increased distance to AIDA/bPlas => lower rate


15.16 DSSSD bias & leakage current OK - attachments 21-22

      FEE64 temperatures OK - attachment 23

      ADC data item stats OK - attachment 24
       aida02 zero rate

      per FEE64 Rate spectra - attachment 25

      Merger restart
      ADC data item stats OK - attachment 26
       7x < 20k, max 207k aida16

15.54 bPlast debiased.


16.29 Data file S181/R12
      bPlas OFF but remiains cabled/connected/grounded

16.30 DSSSD bias & leakage current OK - attachment 27

      FEE64 temperatures OK - attachment 28

      ADC data item stats OK - attachment 29

      per FEE64 Rate spectra - attachment 30

      per FEE64 1.8.W spectra - 20us, 200us, 2ms, 20ms FSR - see attachments 31-39
       note - not all FEE64s updated spectra for all settings

      ADC data item stats OK - attachment 40
       rates change cf. attachment 29
       sources installed/moved/removed, people moving around platform?

      per FEE64 Rate spectra - attachment 41

16.57 DSSSD bias OFF
      FEE64 power OFF

16.34 analysis data file S181/R12_7
      max deadtime 2.2% (aida08)
      ADC data rate 1.330M, HEC data rate 2.1k

16.52 analysis data file S181/R12_12
      max deadtime 1.7% (aida08)
      ADC data rate 945k, HEC data rate 1.8k
Attachment 1: Screenshot_from_2024-06-17_03-11-38.png
Screenshot_from_2024-06-17_03-11-38.png
Attachment 2: Screenshot_from_2024-06-17_03-12-05.png
Screenshot_from_2024-06-17_03-12-05.png
Attachment 3: Screenshot_from_2024-06-17_03-12-29.png
Screenshot_from_2024-06-17_03-12-29.png
Attachment 4: Screenshot_from_2024-06-17_03-12-53.png
Screenshot_from_2024-06-17_03-12-53.png
Attachment 5: Screenshot_from_2024-06-17_05-33-18.png
Screenshot_from_2024-06-17_05-33-18.png
Attachment 6: Screenshot_from_2024-06-17_05-33-47.png
Screenshot_from_2024-06-17_05-33-47.png
Attachment 7: Screenshot_from_2024-06-17_05-34-22.png
Screenshot_from_2024-06-17_05-34-22.png
Attachment 8: Screenshot_from_2024-06-17_05-34-53.png
Screenshot_from_2024-06-17_05-34-53.png
Attachment 9: Screenshot_from_2024-06-17_05-36-26.png
Screenshot_from_2024-06-17_05-36-26.png
Attachment 10: Screenshot_from_2024-06-17_05-38-40.png
Screenshot_from_2024-06-17_05-38-40.png
Attachment 11: Screenshot_from_2024-06-17_05-39-53.png
Screenshot_from_2024-06-17_05-39-53.png
Attachment 12: Screenshot_from_2024-06-17_05-40-29.png
Screenshot_from_2024-06-17_05-40-29.png
Attachment 13: Screenshot_from_2024-06-17_05-41-12.png
Screenshot_from_2024-06-17_05-41-12.png
Attachment 14: Screenshot_from_2024-06-17_05-42-07.png
Screenshot_from_2024-06-17_05-42-07.png
Attachment 15: Screenshot_from_2024-06-17_05-42-35.png
Screenshot_from_2024-06-17_05-42-35.png
Attachment 16: Screenshot_from_2024-06-17_05-43-06.png
Screenshot_from_2024-06-17_05-43-06.png
Attachment 17: R9_1042
 *** TDR format 3.3.0 analyser - TD - May 2021
 *** ERROR: READ I/O error:       5002
                   blocks:      32000
          ADC data format:  259119494 ( 1239652.5 Hz)
        Other data format:    2800506 (   13397.9 Hz)
 Sample trace data format:          0 (       0.0 Hz)
         Undefined format:          0 (       0.0 Hz)
   Other data format type:      PAUSE:        509 (       2.4 Hz)
                               RESUME:        509 (       2.4 Hz)
                              SYNC100:      32779 (     156.8 Hz)
                              WR48-63:      32779 (     156.8 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:    2733930 (   13079.4 Hz)
                           Other info:          0 (       0.0 Hz)

   ADC data range bit set:     425638 (    2036.3 Hz)

                Timewarps:        ADC:          0 (       0.0 Hz)
                                PAUSE:          0 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:          0 (       0.0 Hz)
                              WR48-63:          0 (       0.0 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:          0 (       0.0 Hz)
                            Undefined:          0 (       0.0 Hz)
                         Sample trace:          0 (       0.0 Hz)

 *** Timestamp elapsed time:      209.026 s
 FEE  elapsed dead time(s) elapsed idle time(s)
  0                0.013                0.000
  1                0.000                0.000
  2                0.006                0.000
  3                0.367                0.000
  4                0.000                0.000
  5                0.343                0.000
  6                0.000                0.000
  7                4.268                0.000
  8                0.000                0.000
  9                2.321                0.000
 10                0.126                0.000
 11                0.000                0.000
 12                0.000                0.000
 13                0.030                0.000
 14                0.000                0.000
 15                1.478                0.000
 16                0.000                0.000
 17                0.000                0.000
 18                0.000                0.000
 19                0.000                0.000
 20                0.000                0.000
 21                0.000                0.000
 22                0.000                0.000
 23                0.000                0.000
 24                0.000                0.000
 25                0.000                0.000
 26                0.000                0.000
 27                0.000                0.000
 28                0.000                0.000
 29                0.000                0.000
 30                0.000                0.000
 31                0.000                0.000
 32                0.000                0.000

 *** Statistics
 FEE  ADC Data Other Data     Sample  Undefined      Pause     Resume    SYNC100    WR48-63       Disc        MBS      Other   HEC Data
  0    9607668       8795          0          0         21         21       1240       1240          0       6273          0          0
  1     568341        148          0          0          0          0         74         74          0          0          0          0
  2   11592594    1368472          0          0          2          2       1710       1710          0    1365048          0          0
  3   20057241       5226          0          0         32         32       2581       2581          0          0          0          0
  4    4038921       1042          0          0          0          0        521        521          0          0          0          0
  5   16834727       4288          0          0         27         27       2117       2117          0          0          0          0
  6    2895864      13265          0          0          0          0        361        361          0      12543          0          0
  7   58542191      15052          0          0        171        171       7355       7355          0          0          0     425638
  8    6363514       1604          0          0          0          0        802        802          0          0          0          0
  9   47066112    1362366          0          0        140        140       6010       6010          0    1350066          0          0
 10   19083458       4678          0          0         24         24       2315       2315          0          0          0          0
 11    2437006        602          0          0          0          0        301        301          0          0          0          0
 12    3839787        956          0          0          0          0        478        478          0          0          0          0
 13   12055396       3054          0          0         12         12       1515       1515          0          0          0          0
 14    3699941        872          0          0          0          0        436        436          0          0          0          0
 15   40436733      10086          0          0         80         80       4963       4963          0          0          0          0
 16          0          0          0          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0          0          0          0

 *** Timewarps
 FEE       ADC      Pause     Resume    SYNC100    WR48-63       Disc        MBS  Undefined    Samples
  0          0          0          0          0          0          0          0          0          0
  1          0          0          0          0          0          0          0          0          0
  2          0          0          0          0          0          0          0          0          0
  3          0          0          0          0          0          0          0          0          0
  4          0          0          0          0          0          0          0          0          0
  5          0          0          0          0          0          0          0          0          0
  6          0          0          0          0          0          0          0          0          0
  7          0          0          0          0          0          0          0          0          0
  8          0          0          0          0          0          0          0          0          0
  9          0          0          0          0          0          0          0          0          0
 10          0          0          0          0          0          0          0          0          0
 11          0          0          0          0          0          0          0          0          0
 12          0          0          0          0          0          0          0          0          0
 13          0          0          0          0          0          0          0          0          0
 14          0          0          0          0          0          0          0          0          0
 15          0          0          0          0          0          0          0          0          0
 16          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0

 *** Program elapsed time:   69.695s (  459.141 blocks/s,  28.696 Mb/s)
Attachment 18: Screenshot_from_2024-06-17_06-44-16.png
Screenshot_from_2024-06-17_06-44-16.png
Attachment 19: Screenshot_from_2024-06-17_06-45-49.png
Screenshot_from_2024-06-17_06-45-49.png
Attachment 20: Screenshot_from_2024-06-17_09-53-53.png
Screenshot_from_2024-06-17_09-53-53.png
Attachment 21: Screenshot_from_2024-06-17_15-17-20.png
Screenshot_from_2024-06-17_15-17-20.png
Attachment 22: Screenshot_from_2024-06-17_15-16-47.png
Screenshot_from_2024-06-17_15-16-47.png
Attachment 23: Screenshot_from_2024-06-17_15-17-46.png
Screenshot_from_2024-06-17_15-17-46.png
Attachment 24: Screenshot_from_2024-06-17_15-18-09.png
Screenshot_from_2024-06-17_15-18-09.png
Attachment 25: Screenshot_from_2024-06-17_15-18-54.png
Screenshot_from_2024-06-17_15-18-54.png
Attachment 26: Screenshot_from_2024-06-17_15-20-50.png
Screenshot_from_2024-06-17_15-20-50.png
Attachment 27: Screenshot_from_2024-06-17_16-30-26.png
Screenshot_from_2024-06-17_16-30-26.png
Attachment 28: Screenshot_from_2024-06-17_16-30-55.png
Screenshot_from_2024-06-17_16-30-55.png
Attachment 29: Screenshot_from_2024-06-17_16-31-26.png
Screenshot_from_2024-06-17_16-31-26.png
Attachment 30: Screenshot_from_2024-06-17_16-31-48.png
Screenshot_from_2024-06-17_16-31-48.png
Attachment 31: Screenshot_from_2024-06-17_16-33-21.png
Screenshot_from_2024-06-17_16-33-21.png
Attachment 32: Screenshot_from_2024-06-17_16-34-32.png
Screenshot_from_2024-06-17_16-34-32.png
Attachment 33: Screenshot_from_2024-06-17_16-36-35.png
Screenshot_from_2024-06-17_16-36-35.png
Attachment 34: Screenshot_from_2024-06-17_16-37-33.png
Screenshot_from_2024-06-17_16-37-33.png
Attachment 35: Screenshot_from_2024-06-17_16-38-14.png
Screenshot_from_2024-06-17_16-38-14.png
Attachment 36: Screenshot_from_2024-06-17_16-40-21.png
Screenshot_from_2024-06-17_16-40-21.png
Attachment 37: Screenshot_from_2024-06-17_16-41-18.png
Screenshot_from_2024-06-17_16-41-18.png
Attachment 38: Screenshot_from_2024-06-17_16-43-19.png
Screenshot_from_2024-06-17_16-43-19.png
Attachment 39: Screenshot_from_2024-06-17_16-44-24.png
Screenshot_from_2024-06-17_16-44-24.png
Attachment 40: Screenshot_from_2024-06-17_16-45-19.png
Screenshot_from_2024-06-17_16-45-19.png
Attachment 41: Screenshot_from_2024-06-17_16-46-32.png
Screenshot_from_2024-06-17_16-46-32.png
Attachment 42: R12_7
 *** TDR format 3.3.0 analyser - TD - May 2021
 *** ERROR: READ I/O error:       5002
                   blocks:      32000
          ADC data format:  258100864 ( 1330189.9 Hz)
        Other data format:    3819136 (   19682.9 Hz)
 Sample trace data format:          0 (       0.0 Hz)
         Undefined format:          0 (       0.0 Hz)
   Other data format type:      PAUSE:        522 (       2.7 Hz)
                               RESUME:        522 (       2.7 Hz)
                              SYNC100:      32723 (     168.6 Hz)
                              WR48-63:      32723 (     168.6 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:    3752646 (   19340.2 Hz)
                           Other info:          0 (       0.0 Hz)

   ADC data range bit set:     409704 (    2111.5 Hz)

                Timewarps:        ADC:          0 (       0.0 Hz)
                                PAUSE:          0 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:          0 (       0.0 Hz)
                              WR48-63:          0 (       0.0 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:          0 (       0.0 Hz)
                            Undefined:          0 (       0.0 Hz)
                         Sample trace:          0 (       0.0 Hz)

 *** Timestamp elapsed time:      194.033 s
 FEE  elapsed dead time(s) elapsed idle time(s)
  0                0.121                0.000
  1                0.000                0.000
  2                0.050                0.000
  3                0.191                0.000
  4                0.010                0.000
  5                0.209                0.000
  6                0.000                0.000
  7                4.225                0.000
  8                0.000                0.000
  9                2.072                0.000
 10                0.065                0.000
 11                0.000                0.000
 12                0.000                0.000
 13                0.145                0.000
 14                0.002                0.000
 15                1.159                0.000
 16                0.000                0.000
 17                0.000                0.000
 18                0.000                0.000
 19                0.000                0.000
 20                0.000                0.000
 21                0.000                0.000
 22                0.000                0.000
 23                0.000                0.000
 24                0.000                0.000
 25                0.000                0.000
 26                0.000                0.000
 27                0.000                0.000
 28                0.000                0.000
 29                0.000                0.000
 30                0.000                0.000
 31                0.000                0.000
 32                0.000                0.000

 *** Statistics
 FEE  ADC Data Other Data     Sample  Undefined      Pause     Resume    SYNC100    WR48-63       Disc        MBS      Other   HEC Data
  0   13428500       9208          0          0         38         38       1659       1659          0       5814          0          0
  1    4595656       1164          0          0          0          0        582        582          0          0          0          0
  2   13843408    1881338          0          0         11         11       1937       1937          0    1877442          0          0
  3   19769270       5066          0          0         33         33       2500       2500          0          0          0          0
  4    5034345       1228          0          0          4          4        610        610          0          0          0          0
  5   19195180       4990          0          0         40         40       2455       2455          0          0          0          0
  6    5179175      12949          0          0          0          0        653        653          0      11643          0          0
  7   53830866      13806          0          0        173        173       6730       6730          0          0          0     409704
  8    6018644       1564          0          0          0          0        782        782          0          0          0          0
  9   41936601    1868943          0          0        126        126       5472       5472          0    1857747          0          0
 10   16378984       4032          0          0         16         16       2000       2000          0          0          0          0
 11    2633344        646          0          0          0          0        323        323          0          0          0          0
 12    1918841        428          0          0          0          0        214        214          0          0          0          0
 13   12464437       2916          0          0         14         14       1444       1444          0          0          0          0
 14    3882711       1026          0          0          1          1        512        512          0          0          0          0
 15   37990902       9832          0          0         66         66       4850       4850          0          0          0          0
 16          0          0          0          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0          0          0          0

 *** Timewarps
 FEE       ADC      Pause     Resume    SYNC100    WR48-63       Disc        MBS  Undefined    Samples
  0          0          0          0          0          0          0          0          0          0
  1          0          0          0          0          0          0          0          0          0
  2          0          0          0          0          0          0          0          0          0
  3          0          0          0          0          0          0          0          0          0
  4          0          0          0          0          0          0          0          0          0
  5          0          0          0          0          0          0          0          0          0
  6          0          0          0          0          0          0          0          0          0
  7          0          0          0          0          0          0          0          0          0
  8          0          0          0          0          0          0          0          0          0
  9          0          0          0          0          0          0          0          0          0
 10          0          0          0          0          0          0          0          0          0
 11          0          0          0          0          0          0          0          0          0
 12          0          0          0          0          0          0          0          0          0
 13          0          0          0          0          0          0          0          0          0
 14          0          0          0          0          0          0          0          0          0
 15          0          0          0          0          0          0          0          0          0
 16          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0

 *** Program elapsed time:   60.555s (  528.448 blocks/s,  33.028 Mb/s)
Attachment 43: R12_12
 *** TDR format 3.3.0 analyser - TD - May 2021
 *** ERROR: READ I/O error:       5002
                   blocks:       7946
          ADC data format:   63712775 (  945168.0 Hz)
        Other data format:    1325235 (   19659.6 Hz)
 Sample trace data format:          0 (       0.0 Hz)
         Undefined format:          0 (       0.0 Hz)
   Other data format type:      PAUSE:        128 (       1.9 Hz)
                               RESUME:        128 (       1.9 Hz)
                              SYNC100:       8197 (     121.6 Hz)
                              WR48-63:       8197 (     121.6 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:    1308585 (   19412.6 Hz)
                           Other info:          0 (       0.0 Hz)

   ADC data range bit set:     119370 (    1770.8 Hz)

                Timewarps:        ADC:          0 (       0.0 Hz)
                                PAUSE:          0 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:          0 (       0.0 Hz)
                              WR48-63:          0 (       0.0 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:          0 (       0.0 Hz)
                            Undefined:          0 (       0.0 Hz)
                         Sample trace:          0 (       0.0 Hz)

 *** Timestamp elapsed time:       67.409 s
 FEE  elapsed dead time(s) elapsed idle time(s)
  0                0.000                0.000
  1                0.000                0.000
  2                0.000                0.000
  3                0.043                0.000
  4                0.000                0.000
  5                0.026                0.000
  6                0.000                0.000
  7                1.174                0.000
  8                0.000                0.000
  9                0.504                0.000
 10                0.000                0.000
 11                0.000                0.000
 12                0.000                0.000
 13                0.000                0.000
 14                0.000                0.000
 15                0.224                0.000
 16                0.000                0.000
 17                0.000                0.000
 18                0.000                0.000
 19                0.000                0.000
 20                0.000                0.000
 21                0.000                0.000
 22                0.000                0.000
 23                0.000                0.000
 24                0.000                0.000
 25                0.000                0.000
 26                0.000                0.000
 27                0.000                0.000
 28                0.000                0.000
 29                0.000                0.000
 30                0.000                0.000
 31                0.000                0.000
 32                0.000                0.000

 *** Statistics
 FEE  ADC Data Other Data     Sample  Undefined      Pause     Resume    SYNC100    WR48-63       Disc        MBS      Other   HEC Data
  0     896977       2260          0          0          0          0        119        119          0       2022          0          0
  1    1331618        334          0          0          0          0        167        167          0          0          0          0
  2    2033117     654263          0          0          0          0        346        346          0     653571          0          0
  3    6066546       1620          0          0         14         14        796        796          0          0          0          0
  4     632291        132          0          0          0          0         66         66          0          0          0          0
  5    5664311       1468          0          0         11         11        723        723          0          0          0          0
  6    1249421       4352          0          0          0          0        154        154          0       4044          0          0
  7   16999500       4406          0          0         59         59       2144       2144          0          0          0     119370
  8      43320         14          0          0          0          0          7          7          0          0          0          0
  9   12410393     652234          0          0         26         26       1617       1617          0     648948          0          0
 10    3213140        792          0          0          0          0        396        396          0          0          0          0
 11     278998         64          0          0          0          0         32         32          0          0          0          0
 12     433582        116          0          0          0          0         58         58          0          0          0          0
 13    1869315        430          0          0          0          0        215        215          0          0          0          0
 14     612732        128          0          0          0          0         64         64          0          0          0          0
 15    9977514       2622          0          0         18         18       1293       1293          0          0          0          0
 16          0          0          0          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0          0          0          0

 *** Timewarps
 FEE       ADC      Pause     Resume    SYNC100    WR48-63       Disc        MBS  Undefined    Samples
  0          0          0          0          0          0          0          0          0          0
  1          0          0          0          0          0          0          0          0          0
  2          0          0          0          0          0          0          0          0          0
  3          0          0          0          0          0          0          0          0          0
  4          0          0          0          0          0          0          0          0          0
  5          0          0          0          0          0          0          0          0          0
  6          0          0          0          0          0          0          0          0          0
  7          0          0          0          0          0          0          0          0          0
  8          0          0          0          0          0          0          0          0          0
  9          0          0          0          0          0          0          0          0          0
 10          0          0          0          0          0          0          0          0          0
 11          0          0          0          0          0          0          0          0          0
 12          0          0          0          0          0          0          0          0          0
 13          0          0          0          0          0          0          0          0          0
 14          0          0          0          0          0          0          0          0          0
 15          0          0          0          0          0          0          0          0          0
 16          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0

 *** Program elapsed time:   14.254s (  557.461 blocks/s,  34.841 Mb/s)
  461   Mon May 16 15:12:50 2022 RDP, MAMonday 16th May 16:00-00:00

16:00 Took over the shift from Magda.

17:30 Updates: Attached (1-3)

19:30 Updates: Attached (4-6)

21:30 Updates: Attached (7-9)

23:00 Updates: Attached (10-12)

 

 

 

Attachment 1: Statistics_2022-05-16_17-32-40.png
Statistics_2022-05-16_17-32-40.png
Attachment 2: Temperature_2022-05-16_17-34-15.png
Temperature_2022-05-16_17-34-15.png
Attachment 3: LeakageCurrent_2022-05-16_17-35-05.png
LeakageCurrent_2022-05-16_17-35-05.png
Attachment 4: Statistics2022-05-16_19-31-28.png
Statistics2022-05-16_19-31-28.png
Attachment 5: Temperature2022-05-16_19-32-39.png
Temperature2022-05-16_19-32-39.png
Attachment 6: LeakageCurrent_2022-05-16_19-30-41.png
LeakageCurrent_2022-05-16_19-30-41.png
Attachment 7: Statistics2022-05-16_21-27-17.png
Statistics2022-05-16_21-27-17.png
Attachment 8: Temperature_2022-05-16_21-28-21.png
Temperature_2022-05-16_21-28-21.png
Attachment 9: LeakageCurrent2022-05-16_21-26-15.png
LeakageCurrent2022-05-16_21-26-15.png
Attachment 10: Statistics2022-05-16_22-59-14.png
Statistics2022-05-16_22-59-14.png
Attachment 11: Temperature_2022-05-16_23-00-18.png
Temperature_2022-05-16_23-00-18.png
Attachment 12: LeakageCurrent_2022-05-16_22-58-06.png
LeakageCurrent_2022-05-16_22-58-06.png
  459   Mon May 16 07:03:16 2022 MS, OHMonday 16th May 08:00-16:00

08:00 Took over the night shift from Tom (Attachments 1-3)

10:00: Attachement 4-6

around 10:30: Beam went off (Attachment 7)

11:00: Beam is back

12:00 Attachments 8-10

14:00 Attachements 11-13

15:50 Attachments 14-17

Attachment 1: Screenshot_2022-05-16_at_07-59-51_Statistics_aidas-gsi.png
Screenshot_2022-05-16_at_07-59-51_Statistics_aidas-gsi.png
Attachment 2: Screenshot_2022-05-16_at_08-00-15_Temperature_and_status_scan_aidas-gsi.png
Screenshot_2022-05-16_at_08-00-15_Temperature_and_status_scan_aidas-gsi.png
Attachment 3: Screenshot_from_2022-05-16_08-01-20.png
Screenshot_from_2022-05-16_08-01-20.png
Attachment 4: Screenshot_2022-05-16_at_09-59-15_Statistics_aidas-gsi.png
Screenshot_2022-05-16_at_09-59-15_Statistics_aidas-gsi.png
Attachment 5: Screenshot_2022-05-16_at_09-59-45_Temperature_and_status_scan_aidas-gsi.png
Screenshot_2022-05-16_at_09-59-45_Temperature_and_status_scan_aidas-gsi.png
Attachment 6: Screenshot_from_2022-05-16_10-01-47.png
Screenshot_from_2022-05-16_10-01-47.png
Attachment 7: Screenshot_2022-05-16_at_10-47-01_ucesb.png
Screenshot_2022-05-16_at_10-47-01_ucesb.png
Attachment 8: Screenshot_2022-05-16_at_11-59-53_Statistics_aidas-gsi.png
Screenshot_2022-05-16_at_11-59-53_Statistics_aidas-gsi.png
Attachment 9: Screenshot_2022-05-16_at_12-00-19_Temperature_and_status_scan_aidas-gsi.png
Screenshot_2022-05-16_at_12-00-19_Temperature_and_status_scan_aidas-gsi.png
Attachment 10: Screenshot_from_2022-05-16_12-00-57.png
Screenshot_from_2022-05-16_12-00-57.png
Attachment 11: Screenshot_2022-05-16_at_14-00-56_Statistics_aidas-gsi.png
Screenshot_2022-05-16_at_14-00-56_Statistics_aidas-gsi.png
Attachment 12: Screenshot_2022-05-16_at_14-01-19_Temperature_and_status_scan_aidas-gsi.png
Screenshot_2022-05-16_at_14-01-19_Temperature_and_status_scan_aidas-gsi.png
Attachment 13: Screenshot_from_2022-05-16_14-01-56.png
Screenshot_from_2022-05-16_14-01-56.png
Attachment 14: Screenshot_2022-05-16_at_15-47-58_Statistics_aidas-gsi.png
Screenshot_2022-05-16_at_15-47-58_Statistics_aidas-gsi.png
Attachment 15: Screenshot_2022-05-16_at_15-48-15_Temperature_and_status_scan_aidas-gsi.png
Screenshot_2022-05-16_at_15-48-15_Temperature_and_status_scan_aidas-gsi.png
Attachment 16: Screenshot_2022-05-16_at_15-48-34_Lost_activity_monitor_aidas-gsi.png
Screenshot_2022-05-16_at_15-48-34_Lost_activity_monitor_aidas-gsi.png
Attachment 17: Screenshot_from_2022-05-16_15-49-34.png
Screenshot_from_2022-05-16_15-49-34.png
  458   Sun May 15 23:00:08 2022 & TDMonday 16 May 00:00-08:00
00.01 DAQ continues
      file S450/R4_1319

      ADC control register 0xff

      all disc outputs disabled

      ASIC settings 2021Apr29-13-16-00
       slow comparator aida02 & aida04 0x16, aida06 & aida08 0x23, aida12 0xd, all others 0xc

      BNC PB-5 settings
      amplitude 1.000V
      attenuation x1
      tau_d 1ms
      frequency 2Hz
      polarity +

Disk space OK - /media/SecondDrive

[npg@aidas-gsi S450]$ df -h
Filesystem               Size  Used Avail Use% Mounted on
devtmpfs                 7.8G     0  7.8G   0% /dev
tmpfs                    7.8G  389M  7.4G   5% /dev/shm
tmpfs                    7.8G   19M  7.7G   1% /run
tmpfs                    7.8G     0  7.8G   0% /sys/fs/cgroup
/dev/mapper/centos-root   50G   16G   35G  31% /
/dev/sda2               1014M  226M  789M  23% /boot
/dev/sda1                200M   12M  189M   6% /boot/efi
/dev/sde1                7.2T  4.1T  2.8T  61% /media/SecondDrive
/dev/mapper/centos-home  407G   91G  316G  23% /home
tmpfs                    1.6G   52K  1.6G   1% /run/user/1000
/dev/sdd1                7.2T  6.5T  310G  96% /run/media/npg/ThirdDrive

00.03 all histograms zero'd
      system wide checks counter baseline


00.08 check ASIC control - all FEE64s all ASICs

Attachments 1 & 2 - DSSSD bias & Leakage current - OK
                    grafana DSSSD bias, leakage current & temp - OK

Attachment 3 - FEE64 temps OK

Attachments 4-9 - adc, pause, resume & correlation scaler data items, push, flush stats

Attachments 10-16 - TapeServer, NewMerger, NewMerger stats

Attachment 17 - ucesb

04.08

Attachment 18 - DSSSD bias & Leakage current - OK

Attachment 19 - FEE64 temps OK

Attachment 20 - adc data item stats

Attachments 21 - ucesb

system wide checks - all OK *except* aida09 clock fail status 6

06.36

Attachment 22 - DSSSD bias & Leakage current - OK

Attachment 23 - FEE64 temps OK

Attachment 24 - adc data item stats

Attachments 25 - ucesb

system wide checks - all OK *except* aida09 clock fail status 6
Attachment 1: Screenshot_from_2022-05-16_00-10-13.png
Screenshot_from_2022-05-16_00-10-13.png
Attachment 2: Screenshot_from_2022-05-16_00-10-43.png
Screenshot_from_2022-05-16_00-10-43.png
Attachment 3: Screenshot_from_2022-05-16_00-11-18.png
Screenshot_from_2022-05-16_00-11-18.png
Attachment 4: Screenshot_from_2022-05-16_00-11-50.png
Screenshot_from_2022-05-16_00-11-50.png
Attachment 5: Screenshot_from_2022-05-16_00-12-27.png
Screenshot_from_2022-05-16_00-12-27.png
Attachment 6: Screenshot_from_2022-05-16_00-12-54.png
Screenshot_from_2022-05-16_00-12-54.png
Attachment 7: Screenshot_from_2022-05-16_00-13-25.png
Screenshot_from_2022-05-16_00-13-25.png
Attachment 8: Screenshot_from_2022-05-16_00-13-55.png
Screenshot_from_2022-05-16_00-13-55.png
Attachment 9: Screenshot_from_2022-05-16_00-14-22.png
Screenshot_from_2022-05-16_00-14-22.png
Attachment 10: Screenshot_from_2022-05-16_00-15-46.png
Screenshot_from_2022-05-16_00-15-46.png
Attachment 11: Screenshot_from_2022-05-16_00-16-08.png
Screenshot_from_2022-05-16_00-16-08.png
Attachment 12: Screenshot_from_2022-05-16_00-16-36.png
Screenshot_from_2022-05-16_00-16-36.png
Attachment 13: Screenshot_from_2022-05-16_00-16-58.png
Screenshot_from_2022-05-16_00-16-58.png
Attachment 14: Screenshot_from_2022-05-16_00-16-58.png
Screenshot_from_2022-05-16_00-16-58.png
Attachment 15: Screenshot_from_2022-05-16_00-17-32.png
Screenshot_from_2022-05-16_00-17-32.png
Attachment 16: Screenshot_from_2022-05-16_00-17-55.png
Screenshot_from_2022-05-16_00-17-55.png
Attachment 17: Screenshot_from_2022-05-16_00-18-45.png
Screenshot_from_2022-05-16_00-18-45.png
Attachment 18: Screenshot_from_2022-05-16_04-08-34.png
Screenshot_from_2022-05-16_04-08-34.png
Attachment 19: Screenshot_from_2022-05-16_04-09-07.png
Screenshot_from_2022-05-16_04-09-07.png
Attachment 20: Screenshot_from_2022-05-16_04-09-57.png
Screenshot_from_2022-05-16_04-09-57.png
Attachment 21: Screenshot_from_2022-05-16_04-10-27.png
Screenshot_from_2022-05-16_04-10-27.png
Attachment 22: Screenshot_from_2022-05-16_06-35-32.png
Screenshot_from_2022-05-16_06-35-32.png
Attachment 23: Screenshot_from_2022-05-16_06-36-37.png
Screenshot_from_2022-05-16_06-36-37.png
Attachment 24: Screenshot_from_2022-05-16_06-37-04.png
Screenshot_from_2022-05-16_06-37-04.png
Attachment 25: Screenshot_from_2022-05-16_06-37-58.png
Screenshot_from_2022-05-16_06-37-58.png
  201   Mon Mar 15 11:47:51 2021 TDMonday 15 March
12.48 DAQ continues OK - no storage

      ASIC settings 2019Dec19-16.19.51
      DSSSD#1 slow comparator 0xa
      DSSSD#2 slow comparator 0xa
      DSSSD#3 slow comparator 0xd

      BNC PB-5 Pulser 
      Amplitude1.0V
      Attenuation x1
      Frequency 2Hz
      tau_d 1ms
      - polarity
      Delay 250ns, tail pulse

12.50 System wide checks

	
FEE64 module aida09 global clocks failed, 6
 Clock status test result: Passed 11, Failed 1

Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1' 
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable


FEE64 module aida07 failed
FEE64 module aida09 failed
Calibration test result: Passed 10, Failed 2

If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module



		 Base 		Current 	Difference
aida05 fault 	 0x6acb : 	 0x6acc : 	 1  
aida06 fault 	 0x8394 : 	 0x8395 : 	 1  
aida07 fault 	 0x8c77 : 	 0x8c7a : 	 3  
aida08 fault 	 0x1692 : 	 0x1693 : 	 1  
White Rabbit error counter test result: Passed 8, Failed 4

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR



FPGA Timestamp error counter test result: Passed 12, Failed 0
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last


Returned 0 0 0 0 0 0 0 0 0 0 0 0  
Mem(KB) :	4	8	16	32	64	128	256	512	1k	2k	4k
aida01 :     20     18      5      4      2      2      2      2      3      4     10   : 54576
aida02 :     26     18     16     10      6      4      2      4      2      3      6   : 37048
aida03 :     18     10      4      3      2      3      2      4      2      3      6   : 36152
aida04 :     13      9      4      2      2      2      2      4      1      4     15   : 73852
aida05 :     46     72     70     54     52     48     21     19      7      4      7   : 72216
aida06 :     32     10      4     10      0      2      2      4      3      2      5   : 31056
aida07 :     19      7      5      3      2      3      3      3      3      3      6   : 36916
aida08 :     20     14      2      3      1      3      1      4      3      3      6   : 36864
aida09 :      3      6      3      3      2      4      1      2      3      3      6   : 35916
aida10 :     23     11      3      3      1      3      3      2      3      3      6   : 36356
aida11 :      2      5      6      2      2      4      2      3      2      3      6   : 35664
aida12 :     25     10      5      2      2      2      2      3      3      3      6   : 36548


      FEE64 Temperatures OK - attachment 1
      Good event statistics OK - attachment 2
      Detector bias & leakage currents OK - attachment 3
      Merger OK - 4.6M data items/s
      TapeServer OK - 14Mb/s

12.55 Rate spectra - attachments 4 & 5

      Merger server error messages since most recent check

MERGE Data Link (20503): bad timestamp  3 3 0xc0d582ee 0x04f02352 0x000064aa97efd660 0x166c64aa94f02352 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0xc0e680cf 0x04f032f2 0x000064aa94f032f2 0x166c64aa94f032f2 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0xc0c68215 0x04f03ac2 0x000064aa94f03ac2 0x166c64aa94f03ac2 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0xc0fd81f1 0x04f061d2 0x000064aa94f061d2 0x166c64aa94f061d2 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0xc0eb812a 0x04f07942 0x000064aa94f07942 0x166c64aa94f07942 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0xc0f68041 0x04f07942 0x000064aa94f07942 0x166c64aa94f07942 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0x83806b78 0x04f07aa0 0x000064aa94f07aa0 0x166c64aa94f07aa0 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0x83811ea0 0x04f07aa0 0x000064aa94f07aa0 0x166c64aa94f07aa0 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0x83820000 0x04f07aa0 0x000064aa94f07aa0 0x166c64aa94f07aa0 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0xc0c183d5 0x04f0aff2 0x000064aa94f0aff2 0x166c64aa94f0aff2 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0xc0d58220 0x04f0b7c2 0x000064aa94f0b7c2 0x166c64aa94f0b7c2 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0xc0e68097 0x04f0bf92 0x000064aa94f0bf92 0x166c64aa94f0bf92 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0xc0c682e7 0x04f0c762 0x000064aa94f0c762 0x166c64aa94f0c762 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0xc0cf7ff7 0x04f0cf32 0x000064aa94f0cf32 0x166c64aa94f0cf32 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0xc0fd817e 0x04f0ee72 0x000064aa94f0ee72 0x166c64aa94f0ee72 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0xc0e180e5 0x04f105e2 0x000064aa94f105e2 0x166c64aa94f105e2 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0xc0d181fd 0x04f10db2 0x000064aa94f10db2 0x166c64aa94f10db2 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0xc0d78072 0x04f11582 0x000064aa94f11582 0x166c64aa94f11582 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0xc0c17fff 0x04f14462 0x000064aa94f14462 0x166c64aa94f14462 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0xc0c68234 0x04f15bd2 0x000064aa94f15bd2 0x166c64aa94f15bd2 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0xc0c780f9 0x04f17342 0x000064aa94f17342 0x166c64aa94f17342 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0xc0c98277 0x04f17b12 0x000064aa94f17b12 0x166c64aa94f17b12 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0xc0d28174 0x04f17b12 0x000064aa94f17b12 0x166c64aa94f17b12 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0xc0d5812d 0x04f182e2 0x000064aa94f182e2 0x166c64aa94f182e2 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0xc0fd8112 0x04f182e2 0x000064aa94f182e2 0x166c64aa94f182e2 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0x83806b78 0x04f186a2 0x000064aa94f186a2 0x166c64aa94f186a2 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0x83811ea0 0x04f186a2 0x000064aa94f186a2 0x166c64aa94f186a2 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0x83820000 0x04f186a2 0x000064aa94f186a2 0x166c64aa94f186a2 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0xc0d98224 0x04f18ab2 0x000064aa94f18ab2 0x166c64aa94f18ab2 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0xc0e68096 0x04f19a52 0x000064aa94f19a52 0x166c64aa94f19a52 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0xc0c080e1 0x04f1a222 0x000064aa94f1a222 0x166c64aa94f1a222 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0xc0d18282 0x04f1a222 0x000064aa94f1a222 0x166c64aa94f1a222 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0xc0c38198 0x04f1a9f2 0x000064aa94f1a9f2 0x166c64aa94f1a9f2 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0xc0cb8187 0x04f1b1c2 0x000064aa94f1b1c2 0x166c64aa94f1b1c2 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0xc0cc8178 0x04f1b992 0x000064aa94f1b992 0x166c64aa94f1b992 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0xc0c78063 0x04f1ffe2 0x000064aa94f1ffe2 0x166c64aa94f1ffe2 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0xc0e980e1 0x04f1ffe2 0x000064aa94f1ffe2 0x166c64aa94f1ffe2 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0xc0c98129 0x04f207b2 0x000064aa94f207b2 0x166c64aa94f207b2 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0xc0d28107 0x04f207b2 0x000064aa94f207b2 0x166c64aa94f207b2 0x166c64aa97efd660
MERGE Data Link (20503): bad timestamp  3 3 0xc0eb80e2 0x04f207b2 0x000064aa94f207b2 0x166c64aa94f207b2 0x166c64aa97efd660

18.47 FEE64 Temperatures OK - attachment 1
      Good event statistics OK - attachment 2
      Detector bias & leakage currents OK - attachment 3
      Merger OK - 4.6M data items/s
      TapeServer OK - 14Mb/s

      No new merger error messages since 12.55 today

          
Attachment 1: Screenshot_2021-03-15_Temperature_and_status_scan_aidas-gsi.png
Screenshot_2021-03-15_Temperature_and_status_scan_aidas-gsi.png
Attachment 2: Screenshot_2021-03-15_Statistics_aidas-gsi.png
Screenshot_2021-03-15_Statistics_aidas-gsi.png
Attachment 3: 60.png
60.png
Attachment 4: Screenshot_2021-03-15_Spectrum_Browser_aidas-gsi.png
Screenshot_2021-03-15_Spectrum_Browser_aidas-gsi.png
Attachment 5: Screenshot_2021-03-15_Spectrum_Browser_aidas-gsi(1).png
Screenshot_2021-03-15_Spectrum_Browser_aidas-gsi(1).png
Attachment 6: Screenshot_2021-03-15_Temperature_and_status_scan_aidas-gsi(1).png
Screenshot_2021-03-15_Temperature_and_status_scan_aidas-gsi(1).png
Attachment 7: Screenshot_2021-03-15_Statistics_aidas-gsi(1).png
Screenshot_2021-03-15_Statistics_aidas-gsi(1).png
Attachment 8: 61.png
61.png
  368   Tue Jun 15 12:33:46 2021 TDMonday 14 June
  303   Mon May 10 08:49:37 2021 TDMonday 10 May
09.51 All system wide checks OK *except*


		 Base 		Current 	Difference
aida05 fault 	 0x8441 : 	 0x8442 : 	 1  
aida06 fault 	 0xc5c5 : 	 0xc5c6 : 	 1  
aida07 fault 	 0x6f07 : 	 0x6f08 : 	 1  
aida08 fault 	 0x4814 : 	 0x4815 : 	 1  
White Rabbit error counter test result: Passed 12, Failed 4

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

	
Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0  
Mem(KB) :	4	8	16	32	64	128	256	512	1k	2k	4k
aida01 :      1      4      6      3      5      5      2      3      2      3      6   : 36004
aida02 :      4      1      3      6      3      4      3      3      2      3      6   : 36040
aida03 :     12      8      5      2      3      3      3      4      2      3      6   : 36416
aida04 :      1      2      5      4      1      4      3      4      2      3      6   : 36388
aida05 :     22      8      4      2      4      5      1      4      1      4      6   : 37272
aida06 :      5      4      0      6      3      6      3      3      2      3      6   : 36276
aida07 :     38     16      7      7      1      6      1      3      3      3      6   : 37032
aida08 :     16      4      2      3      1      4      1      3      3      3      6   : 36384
aida09 :      2      3      7      6      5      5      2      3      2      3      6   : 36112
aida10 :     42     15      4      6      2      6      2      4      2      3      6   : 36768
aida11 :      1      5     13      7      3      3      3      3      2      3      6   : 36124
aida12 :     23      9      3      4      7      3      2      3      2      3      6   : 35988
aida13 :      2      6      7      3      4      4      3      2      2      3      6   : 35592
aida14 :     17     11      3      4      3      5      2      3      2      3      6   : 35980
aida15 :      4      7      5      2      1      6      2      3      2      3      6   : 35864
aida16 :      4      1      3      3      4      2      3      4      2      3      6   : 36264


 Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same

 FEE : aida01 =>   Options file size is 1026 	 Last changed Sun May 09 15:10:52 CEST 2021
 FEE : aida02 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:43:46 CEST 2021
 FEE : aida03 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:43:50 CEST 2021
 FEE : aida04 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:43:53 CEST 2021
 FEE : aida05 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:43:55 CEST 2021
 FEE : aida06 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:43:59 CEST 2021
 FEE : aida07 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:02 CEST 2021
 FEE : aida08 =>   Options file size is 1025 	 Last changed Wed May 05 12:15:54 CEST 2021
 FEE : aida09 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:08 CEST 2021
 FEE : aida10 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida11 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida12 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida13 =>   Options file size is 1025 	 Last changed Fri May 07 19:40:34 CEST 2021
 FEE : aida14 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida15 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida16 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021

09.52 All histograms, stats and merger stats zero'd

      FEE64 temps OK - attachment 1
      ADC data, disc/WR 28-27/pause/correlation stats OK - attachments 2-6
      Rate spectra - attachment 7
       ASIC check load
      Rate spectra - attachment 8
      1.8.W spectra 20us FSR - attachments 9 & 10
      Grafana - most recent 7 days DSSSD bias and leakage current - attachment 11
      Merger stats/tape server/merger - attachment 12
      DSSSD bias and leakage current OK - attachment 13

10.08 1.8.W spectra 20us, 200us, 2ms & 20ms FSR - attachments 14-21
       we are not missing extrinsic noise features at longer time ranges cf. 20us FSR

12.18 Cannot connect to port 8015, MIDAS terminal says "Killed"
      Restart MIDAS 8015 
      Collect FPGA baseline to avoid TCL error
      FEEs unaffected and running as before

13.25 FEEs power cycled to see if any change in weird noise/rates situation
      No change 
      ASICs synchronised and back to normal

14.34 Start a pulser walkthrough in May21/R10
      Attenuation 5x, Rep Rate 25 Hz
      Amplitudes 10..9...1

14.45 Back to 2 Hz @ 2V Pulser, walkthrough complete. No Storage again.

 PULSER SETTINGS
---------------------------
 Pulse is ON
 Positive Tail Pulse
 Trigger Source is Internal Clock
 Trigger Threshold is 3.5
 Amplitude   : 2.0 Volts
 Rep Rate    : 2.0 hZ
 Delay       : 250.0 ns
 Fall Time   : 1 ms
 Attenuation : 1
 Display is  : Volts
 Equivalent keV is : 200.0 (NB. actually it is 12 MeV)
 Ramp Start at 0.01 Volts
 Ramp Stop  at 9.99 Volts
 Ramp Start at 1.0 keV
 Ramp Stop  at 999.0 keV
 Ramp Time  is 60 seconds
 # Ramp Cycles is 1

OK
Attachment 1: Screenshot_2021-05-10_Temperature_and_status_scan_aidas-gsi.png
Screenshot_2021-05-10_Temperature_and_status_scan_aidas-gsi.png
Attachment 2: Screenshot_2021-05-10_Statistics_aidas-gsi.png
Screenshot_2021-05-10_Statistics_aidas-gsi.png
Attachment 3: Screenshot_2021-05-10_Statistics_aidas-gsi(1).png
Screenshot_2021-05-10_Statistics_aidas-gsi(1).png
Attachment 4: Screenshot_2021-05-10_Statistics_aidas-gsi(2).png
Screenshot_2021-05-10_Statistics_aidas-gsi(2).png
Attachment 5: Screenshot_2021-05-10_Statistics_aidas-gsi(3).png
Screenshot_2021-05-10_Statistics_aidas-gsi(3).png
Attachment 6: Screenshot_2021-05-10_Statistics_aidas-gsi(4).png
Screenshot_2021-05-10_Statistics_aidas-gsi(4).png
Attachment 7: Screenshot_2021-05-10_Spectrum_Browser_aidas-gsi.png
Screenshot_2021-05-10_Spectrum_Browser_aidas-gsi.png
Attachment 8: Screenshot_2021-05-10_Spectrum_Browser_aidas-gsi(1).png
Screenshot_2021-05-10_Spectrum_Browser_aidas-gsi(1).png
Attachment 9: Screenshot_2021-05-10_Spectrum_Browser_aidas-gsi(2).png
Screenshot_2021-05-10_Spectrum_Browser_aidas-gsi(2).png
Attachment 10: Screenshot_2021-05-10_Spectrum_Browser_aidas-gsi(3).png
Screenshot_2021-05-10_Spectrum_Browser_aidas-gsi(3).png
Attachment 11: Screenshot_2021-05-10_AIDA_Alerting_-_Grafana.png
Screenshot_2021-05-10_AIDA_Alerting_-_Grafana.png
Attachment 12: 510.png
510.png
Attachment 13: 511.png
511.png
Attachment 14: Screenshot_2021-05-10_Spectrum_Browser_aidas-gsi(4).png
Screenshot_2021-05-10_Spectrum_Browser_aidas-gsi(4).png
Attachment 15: Screenshot_2021-05-10_Spectrum_Browser_aidas-gsi(11).png
Screenshot_2021-05-10_Spectrum_Browser_aidas-gsi(11).png
Attachment 16: Screenshot_2021-05-10_Spectrum_Browser_aidas-gsi(5).png
Screenshot_2021-05-10_Spectrum_Browser_aidas-gsi(5).png
Attachment 17: Screenshot_2021-05-10_Spectrum_Browser_aidas-gsi(10).png
Screenshot_2021-05-10_Spectrum_Browser_aidas-gsi(10).png
Attachment 18: Screenshot_2021-05-10_Spectrum_Browser_aidas-gsi(6).png
Screenshot_2021-05-10_Spectrum_Browser_aidas-gsi(6).png
Attachment 19: Screenshot_2021-05-10_Spectrum_Browser_aidas-gsi(9).png
Screenshot_2021-05-10_Spectrum_Browser_aidas-gsi(9).png
Attachment 20: Screenshot_2021-05-10_Spectrum_Browser_aidas-gsi(7).png
Screenshot_2021-05-10_Spectrum_Browser_aidas-gsi(7).png
Attachment 21: Screenshot_2021-05-10_Spectrum_Browser_aidas-gsi(8).png
Screenshot_2021-05-10_Spectrum_Browser_aidas-gsi(8).png
  701   Tue Mar 11 11:53:18 2025 TDMonday 10 March
18.30 AIDA powered up for brief test

      DSSSD bias & leakage current OK - attachment 1

      FEE64 temperatures OK - attachment 2

      ADC data item stats - attachment 3
       ASIC settings at power up - slow comparator 0x64

      per p+n FEE64 1.8.L spectra - attachment 4
       pulser peak width aida9 55 ch FWHM (39keV FWHm), aida15 53 ch FWHM (37keV FWHM)

      p+n FEE64 aida15 1.8.W spectra - 20us FSR - attachments 5-6

      saved *.L, *.H, *.W spectra for aida15 - attachments 7-11
       *.W spectra - 20us, 200us, 2ms & 20ms FSR 


19.27 DAQ STOP
      DSSSD bias OFF
      FEE64 power OFF
Attachment 1: Screenshot_from_2025-03-10_18-41-32.png
Screenshot_from_2025-03-10_18-41-32.png
Attachment 2: Screenshot_from_2025-03-10_18-42-02.png
Screenshot_from_2025-03-10_18-42-02.png
Attachment 3: Screenshot_from_2025-03-10_18-42-33.png
Screenshot_from_2025-03-10_18-42-33.png
Attachment 4: Screenshot_from_2025-03-10_18-43-07.png
Screenshot_from_2025-03-10_18-43-07.png
Attachment 5: Screenshot_from_2025-03-10_18-48-03.png
Screenshot_from_2025-03-10_18-48-03.png
Attachment 6: Screenshot_from_2025-03-10_18-48-39.png
Screenshot_from_2025-03-10_18-48-39.png
Attachment 7: 2025Mar10-17.59.58.tar.gz
Attachment 8: 2025Mar10-18.05.01.tar.gz
Attachment 9: 2025Mar10-18.06.29.tar.gz
Attachment 10: 2025Mar10-18.12.03.tar.gz
Attachment 11: 2025Mar10-18.17.55.tar.gz
  480   Mon Jun 13 16:38:37 2022 NHMon 13 Jun
Remove Caen HV Supply

Open up, C23 was grounded (HV grounded in NIM)
Remove C23 (HV floating at NIM) and reinstall

Rebias HV... No Change

Reboot AIDA... No change
  652   Tue Jun 11 18:45:37 2024 PPMid-shift checks, 19:45

All looks good.

Sceenshots attached.

Attachment 1: Screenshot_from_2024-06-11_19-46-41.png
Screenshot_from_2024-06-11_19-46-41.png
Attachment 2: Screenshot_from_2024-06-11_19-47-09.png
Screenshot_from_2024-06-11_19-47-09.png
Attachment 3: Screenshot_from_2024-06-11_19-47-38.png
Screenshot_from_2024-06-11_19-47-38.png
Attachment 4: Screenshot_from_2024-06-11_19-48-12.png
Screenshot_from_2024-06-11_19-48-12.png
Attachment 5: Screenshot_from_2024-06-11_19-48-58.png
Screenshot_from_2024-06-11_19-48-58.png
Attachment 6: Screenshot_from_2024-06-11_19-49-22.png
Screenshot_from_2024-06-11_19-49-22.png
Attachment 7: Screenshot_from_2024-06-11_19-49-43.png
Screenshot_from_2024-06-11_19-49-43.png
Attachment 8: Screenshot_from_2024-06-11_19-50-19.png
Screenshot_from_2024-06-11_19-50-19.png
  656   Wed Jun 12 11:06:55 2024 PPMid-shift checks, 12:00

All seems normal.

Screenshots attached.

Attachment 1: Screenshot_from_2024-06-12_12-07-32.png
Screenshot_from_2024-06-12_12-07-32.png
Attachment 2: Screenshot_from_2024-06-12_12-07-59.png
Screenshot_from_2024-06-12_12-07-59.png
Attachment 3: Screenshot_from_2024-06-12_12-08-24.png
Screenshot_from_2024-06-12_12-08-24.png
Attachment 4: Screenshot_from_2024-06-12_12-09-09.png
Screenshot_from_2024-06-12_12-09-09.png
Attachment 5: Screenshot_from_2024-06-12_12-10-02.png
Screenshot_from_2024-06-12_12-10-02.png
Attachment 6: Screenshot_from_2024-06-12_12-10-30.png
Screenshot_from_2024-06-12_12-10-30.png
Attachment 7: Screenshot_from_2024-06-12_12-10-46.png
Screenshot_from_2024-06-12_12-10-46.png
Attachment 8: Screenshot_from_2024-06-12_12-11-41.png
Screenshot_from_2024-06-12_12-11-41.png
  353   Wed Jun 2 14:53:05 2021 OH, TDMerger/Tapeserver issues after reboot of aida-3
On 01/06/21 aida-3 was rebooted as the graphical user interface had frozen.
We were also unable to connected to MIDAS via ssh port forwarding.

Since the restart we have been unable to write to file.

The tapeserver will allocate/mount/open a file but nothing will be written to it.
The merger will be receiving data but never shows as having any "Current links with data".

/MIDAS currently points to lrwxrwxrwx. 1 root root 45 Jan 23  2019 /MIDAS -> /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119

The current disk structure is:
[npg@aidas-gsi npg]$ lsblk
NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sr0     11:0    1  1024M  0 rom  
sdb      8:16   0   7.3T  0 disk 
&#9500;&#9472;sdb1   8:17   0   200M  0 part 
&#9500;&#9472;sdb2   8:18   0   500M  0 part 
&#9492;&#9472;sdb3   8:19   0   7.3T  0 part 
  &#9500;&#9472;vg_aidas2-lv_root (dm-2)
       253:2    0    50G  0 lvm  
  &#9500;&#9472;vg_aidas2-lv_home (dm-3)
       253:3    0   7.2T  0 lvm  /media/1e121361-83d3-4825-b6ae-8700b07e0ca7
  &#9492;&#9472;vg_aidas2-lv_swap (dm-4)
       253:4    0   7.8G  0 lvm  
sdc      8:32   0   7.3T  0 disk 
&#9492;&#9472;sdc1   8:33   0   7.3T  0 part /media/ThirdDrive
sdd      8:48   0   7.3T  0 disk 
&#9492;&#9472;sdd1   8:49   0   7.3T  0 part /media/SecondDrive
sda      8:0    0 465.8G  0 disk 
&#9500;&#9472;sda1   8:1    0   200M  0 part /boot/efi
&#9500;&#9472;sda2   8:2    0   500M  0 part /boot
&#9492;&#9472;sda3   8:3    0 465.1G  0 part 
  &#9500;&#9472;vg_aidasgsi-lv_root (dm-0)
       253:0    0    50G  0 lvm  /
  &#9500;&#9472;vg_aidasgsi-lv_swap (dm-1)
       253:1    0   7.8G  0 lvm  [SWAP]
  &#9492;&#9472;vg_aidasgsi-lv_home (dm-5)
       253:5    0 407.3G  0 lvm  /home


Terminal outputs for the HTTPd, Merger and TapeServer terminals - attachments 1-3

TapeServer config file - attachment 4
Merger options - attachment 5

Note in the merger window the top bar is different to what we have seen previously - Attachment 6 and https://elog.ph.ed.ac.uk/DESPEC/210418_124830/51.png
Attachment 1: 210602_MergerTape_HTTPd_Startup
tidy up
tclsh8.5_copy2: no process killed

System identified is CPU x86_64; Platform is unix; OS is Linux and Version is 2.6.32-696.el6.x86_64
Environment selected is CPU x64_64; Platform unix; OS Linux64 and Operating System Linux64
MIDASBASE = /MIDAS and MIDAS_LIBRARY = /MIDAS/TclHttpd/Linux64
PATH = /MIDAS/bin_Linux64:/MIDAS/TclHttpd/Linux64:/MIDAS/Linux/bin64:/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/npg/bin
Computer Name = aidas-gsi; Temp Directory = /tmp/tcl25624
package limit is not available: can't find package limit
Running with default file descriptor limit
package setuid is not available: can't find package setuid
Could not change to user 50 group 50: not owner
/debug user "debug" password "z5y30nlwy2e4"
httpd started on port 8115

Custom startup from /MIDAS/config/TclHttpd/aidas-gsi@8115/startup.tcl
/DataBaseAccessServer
/NetVarService
/SigTaskService
Loaded MemSasAccess
/SpectrumService
/TapeServer
loading tcl/NewMergerControl.tcl for namespace ::
/DataAcquisitionControlServer
DefineMessage unknown
Run Control Server Implementation for MERGE
RunControlServer loaded
loading Html/RunControl/implementation.tcl
/MIDAS/TclHttpd/Html/RunControl/common.tcl returned z=1 and couldn't read file "/MIDAS/TclHttpd/Html/RunControl/common.tcl": no such file or directory
ReadRegister failed: Name=NetVar.EXEC.ID; Code= 0x10004; Info= Register name does not exist
Created UI registers
RunControl loaded
loading Html/NewMerger/RunControl/implementation.tcl for namespace ::
ReadRegister failed: Name=NetVar.MERGE.ID; Code= 0x10004; Info= Register name does not exist
Created UI registers
NewMerge Control loaded
Completed custom startup from /MIDAS/TclHttpd/Html/NewMerger/RunControl/stats.defn.tcl
Shared memory area located at 0x7f34d3028000
Tape Server comms table located at 0x7f34d3028000
MERGE command  30
setup MERGER
Stop Merge
Halt (1)
Action has completed: MergeCommand 2 Halt
Setup Merge
Setup (1)
Action has completed: MergeCommand 1 Setup
MERGE command  32
go MERGER
Go Merge
Go (1)
Action has completed: MergeCommand 3 Go
link 0 GO (1)
Action has completed: MergeCommand 22 {link 0 GO} 0 2
link 1 GO (1)
Action has completed: MergeCommand 22 {link 1 GO} 1 2
link 2 GO (1)
Action has completed: MergeCommand 22 {link 2 GO} 2 2
link 3 GO (1)
Action has completed: MergeCommand 22 {link 3 GO} 3 2
link 4 GO (1)
Action has completed: MergeCommand 22 {link 4 GO} 4 2
link 5 GO (1)
Action has completed: MergeCommand 22 {link 5 GO} 5 2
link 6 GO (1)
Action has completed: MergeCommand 22 {link 6 GO} 6 2
link 7 GO (1)
Action has completed: MergeCommand 22 {link 7 GO} 7 2
link 8 GO (1)
Action has completed: MergeCommand 22 {link 8 GO} 8 2
link 9 GO (1)
Action has completed: MergeCommand 22 {link 9 GO} 9 2
link 10 GO (1)
Action has completed: MergeCommand 22 {link 10 GO} 10 2
link 11 GO (1)
Action has completed: MergeCommand 22 {link 11 GO} 11 2
link 12 GO (1)
Action has completed: MergeCommand 22 {link 12 GO} 12 2
link 13 GO (1)
Action has completed: MergeCommand 22 {link 13 GO} 13 2
link 14 GO (1)
Action has completed: MergeCommand 22 {link 14 GO} 14 2
link 15 GO (1)
Action has completed: MergeCommand 22 {link 15 GO} 15 2
Resume MERGER

Attachment 2: 210602_NewMerger_Startup
Tidy up
MERGE: no process killed
merge64.AD: no process killed
Starting New Merger

[1] 25659
MERGE Master (25659): Message logger not contacted.
MERGE Master (25659): MIDAS MERGE Server 64 bit Build May 11 2021  20:32:01
Creating NetVars
Creating NetVars complete
MERGE Master (25659): MIDAS MERGE Server 64 bit Build May 11 2021  20:32:01
MERGE Master (25659): Configuration: TCP port base=11001 ; SHM key=11000; ids=16; links=16; Hardware=0
MERGE Master (25659): File mapped object /SHM_11000 of size 536628 created
MERGE Master (25659): Memory mapping 536628 bytes
MERGE Master (25659): Shared memory segment located at address 0x7fc0a99b8000.
MERGE Master (25659): File mapped object /SHM_11001 of size 100669480 created
MERGE Master (25659): Memory mapping 100669480 bytes
MERGE Master (25659): Shared memory segment located at address 0x7fc0a39b6000.
MERGE Master (25659): File mapped object /SHM_11002 of size 100669480 created
MERGE Master (25659): Memory mapping 100669480 bytes
MERGE Master (25659): Shared memory segment located at address 0x7fc09d9b4000.
MERGE Master (25659): File mapped object /SHM_11003 of size 100669480 created
MERGE Master (25659): Memory mapping 100669480 bytes
MERGE Master (25659): Shared memory segment located at address 0x7fc0979b2000.
MERGE Master (25659): File mapped object /SHM_11004 of size 100669480 created
MERGE Master (25659): Memory mapping 100669480 bytes
MERGE Master (25659): Shared memory segment located at address 0x7fc0919b0000.
MERGE Master (25659): File mapped object /SHM_11005 of size 100669480 created
MERGE Master (25659): Memory mapping 100669480 bytes
MERGE Master (25659): Shared memory segment located at address 0x7fc08b9ae000.
MERGE Master (25659): File mapped object /SHM_11006 of size 100669480 created
MERGE Master (25659): Memory mapping 100669480 bytes
MERGE Master (25659): Shared memory segment located at address 0x7fc0859ac000.
MERGE Master (25659): File mapped object /SHM_11007 of size 100669480 created
MERGE Master (25659): Memory mapping 100669480 bytes
MERGE Master (25659): Shared memory segment located at address 0x7fc07f9aa000.
MERGE Master (25659): File mapped object /SHM_11008 of size 100669480 created
MERGE Master (25659): Memory mapping 100669480 bytes
MERGE Master (25659): Shared memory segment located at address 0x7fc0799a8000.
MERGE Master (25659): File mapped object /SHM_11009 of size 100669480 created
MERGE Master (25659): Memory mapping 100669480 bytes
MERGE Master (25659): Shared memory segment located at address 0x7fc0739a6000.
MERGE Master (25659): File mapped object /SHM_11010 of size 100669480 created
MERGE Master (25659): Memory mapping 100669480 bytes
MERGE Master (25659): Shared memory segment located at address 0x7fc06d9a4000.
MERGE Master (25659): File mapped object /SHM_11011 of size 100669480 created
MERGE Master (25659): Memory mapping 100669480 bytes
MERGE Master (25659): Shared memory segment located at address 0x7fc0679a2000.
MERGE Master (25659): File mapped object /SHM_11012 of size 100669480 created
MERGE Master (25659): Memory mapping 100669480 bytes
MERGE Master (25659): Shared memory segment located at address 0x7fc0619a0000.
MERGE Master (25659): File mapped object /SHM_11013 of size 100669480 created
MERGE Master (25659): Memory mapping 100669480 bytes
MERGE Master (25659): Shared memory segment located at address 0x7fc05b99e000.
MERGE Master (25659): File mapped object /SHM_11014 of size 100669480 created
MERGE Master (25659): Memory mapping 100669480 bytes
MERGE Master (25659): Shared memory segment located at address 0x7fc05599c000.
MERGE Master (25659): File mapped object /SHM_11015 of size 100669480 created
MERGE Master (25659): Memory mapping 100669480 bytes
MERGE Master (25659): Shared memory segment located at address 0x7fc04f99a000.
MERGE Master (25659): File mapped object /SHM_11016 of size 100669480 created
MERGE Master (25659): Memory mapping 100669480 bytes
MERGE Master (25659): Shared memory segment located at address 0x7fc049998000.
MERGE Master (25659): Shared memory segment located at address 0x7fc049998000.
MERGE Master (25659): Starting stats/rates task
MERGE Master (25659): Stats/Rates task has pid 25661
MERGE Master (25659): Starting 16 link tasks
MERGE Master (25659): Link task has pid 25662
MERGE Master (25659): Link task has pid 25663
MERGE Statistics/Rates (25661): Message logger not contacted.
MERGE Statistics/Rates (25661): MIDAS MERGE Statistics 64 bit Build May 11 2021  20:32:03
Creating NetVars
Creating NetVars complete
MERGE Master (25659): Link task has pid 25664
MERGE Data Link (25662): Message logger not contacted.
MERGE Data Link (25662): MIDAS MERGE Data Link 64 bit Build May 11 2021  20:32:02
Creating NetVars
Creating NetVars complete
MERGE Data Link (25662): Configuration: index = 0, SHM key=11000, TCP port base = 11001, module = -1, hardware = 0
MERGE Data Link (25662): File mapped object /SHM_11000 accessed
MERGE Data Link (25662): Memory mapping 536628 bytes
MERGE Data Link (25662): Shared memory segment located at address 0x7f198ee7a000.
MERGE Data Link (25662): Allocated 71720 bytes for local input buffer
MERGE Data Link (25662): File mapped object /SHM_11001 accessed
MERGE Data Link (25662): Memory mapping 100669480 bytes
MERGE Data Link (25662): Shared memory segment located at address 0x7f1988e78000.
MERGE Master (25659): Link task has pid 25665
MERGE Data Link (25663): Message logger not contacted.
MERGE Data Link (25663): MIDAS MERGE Data Link 64 bit Build May 11 2021  20:32:02
Creating NetVars
Creating NetVars complete
MERGE Data Link (25663): Configuration: index = 1, SHM key=11000, TCP port base = 11002, module = 0, hardware = 0
MERGE Data Link (25663): File mapped object /SHM_11000 accessed
MERGE Data Link (25663): Memory mapping 536628 bytes
MERGE Data Link (25663): Shared memory segment located at address 0x7f3bee603000.
MERGE Data Link (25663): Allocated 71720 bytes for local input buffer
MERGE Data Link (25663): File mapped object /SHM_11002 accessed
MERGE Data Link (25663): Memory mapping 100669480 bytes
MERGE Data Link (25663): Shared memory segment located at address 0x7f3be8601000.
MERGE Master (25659): Link task has pid 25666
MERGE Data Link (25664): Message logger not contacted.
MERGE Data Link (25665): Message logger not contacted.
MERGE Data Link (25664): MIDAS MERGE Data Link 64 bit Build May 11 2021  20:32:02
MERGE Data Link (25665): MIDAS MERGE Data Link 64 bit Build May 11 2021  20:32:02
Creating NetVars
Creating NetVars
Creating NetVars complete
MERGE Data Link (25665): Configuration: index = 3, SHM key=11000, TCP port base = 11004, module = 2, hardware = 0
MERGE Data Link (25665): File mapped object /SHM_11000 accessed
MERGE Data Link (25665): Memory mapping 536628 bytes
MERGE Data Link (25665): Shared memory segment located at address 0x7f7418ae6000.
MERGE Data Link (25665): Allocated 71720 bytes for local input buffer
MERGE Data Link (25665): File mapped object /SHM_11004 accessed
MERGE Data Link (25665): Memory mapping 100669480 bytes
MERGE Data Link (25665): Shared memory segment located at address 0x7f7412ae4000.
Creating NetVars complete
MERGE Data Link (25664): Configuration: index = 2, SHM key=11000, TCP port base = 11003, module = 1, hardware = 0
MERGE Data Link (25664): File mapped object /SHM_11000 accessed
MERGE Data Link (25664): Memory mapping 536628 bytes
MERGE Data Link (25664): Shared memory segment located at address 0x7f0da72c8000.
MERGE Data Link (25664): Allocated 71720 bytes for local input buffer
MERGE Data Link (25664): File mapped object /SHM_11003 accessed
MERGE Data Link (25664): Memory mapping 100669480 bytes
MERGE Data Link (25664): Shared memory segment located at address 0x7f0da12c6000.
MERGE Master (25659): Link task has pid 25667
MERGE Master (25659): Link task has pid 25668
MERGE Data Link (25666): Message logger not contacted.
MERGE Data Link (25666): MIDAS MERGE Data Link 64 bit Build May 11 2021  20:32:02
Creating NetVars
Creating NetVars complete
MERGE Data Link (25666): Configuration: index = 4, SHM key=11000, TCP port base = 11005, module = 3, hardware = 0
MERGE Data Link (25666): File mapped object /SHM_11000 accessed
MERGE Data Link (25666): Memory mapping 536628 bytes
MERGE Data Link (25666): Shared memory segment located at address 0x7fb7f4ec2000.
MERGE Data Link (25666): Allocated 71720 bytes for local input buffer
MERGE Data Link (25666): File mapped object /SHM_11005 accessed
MERGE Data Link (25666): Memory mapping 100669480 bytes
MERGE Data Link (25666): Shared memory segment located at address 0x7fb7eeec0000.
MERGE Master (25659): Link task has pid 25669
MERGE Data Link (25667): Message logger not contacted.
MERGE Data Link (25667): MIDAS MERGE Data Link 64 bit Build May 11 2021  20:32:02
Creating NetVars
Creating NetVars complete
MERGE Data Link (25667): Configuration: index = 5, SHM key=11000, TCP port base = 11006, module = 4, hardware = 0
MERGE Data Link (25667): File mapped object /SHM_11000 accessed
MERGE Data Link (25667): Memory mapping 536628 bytes
MERGE Data Link (25667): Shared memory segment located at address 0x7f8b878c9000.
MERGE Data Link (25667): Allocated 71720 bytes for local input buffer
MERGE Data Link (25667): File mapped object /SHM_11006 accessed
MERGE Data Link (25667): Memory mapping 100669480 bytes
MERGE Data Link (25667): Shared memory segment located at address 0x7f8b818c7000.
MERGE Master (25659): Link task has pid 25670
MERGE Data Link (25668): Message logger not contacted.
MERGE Data Link (25668): MIDAS MERGE Data Link 64 bit Build May 11 2021  20:32:02
Creating NetVars
MERGE Master (25659): Link task has pid 25671
Creating NetVars complete
MERGE Data Link (25668): Configuration: index = 6, SHM key=11000, TCP port base = 11007, module = 5, hardware = 0
MERGE Data Link (25668): File mapped object /SHM_11000 accessed
MERGE Data Link (25668): Memory mapping 536628 bytes
MERGE Data Link (25668): Shared memory segment located at address 0x7f88c7c9a000.
MERGE Data Link (25668): Allocated 71720 bytes for local input buffer
MERGE Data Link (25668): File mapped object /SHM_11007 accessed
MERGE Data Link (25668): Memory mapping 100669480 bytes
MERGE Data Link (25668): Shared memory segment located at address 0x7f88c1c98000.
MERGE Data Link (25669): Message logger not contacted.
MERGE Data Link (25669): MIDAS MERGE Data Link 64 bit Build May 11 2021  20:32:02
Creating NetVars
MERGE Master (25659): Link task has pid 25672
Creating NetVars complete
MERGE Data Link (25669): Configuration: index = 7, SHM key=11000, TCP port base = 11008, module = 6, hardware = 0
MERGE Data Link (25669): File mapped object /SHM_11000 accessed
MERGE Data Link (25669): Memory mapping 536628 bytes
MERGE Data Link (25669): Shared memory segment located at address 0x7f87e7704000.
MERGE Data Link (25669): Allocated 71720 bytes for local input buffer
MERGE Data Link (25669): File mapped object /SHM_11008 accessed
MERGE Data Link (25669): Memory mapping 100669480 bytes
MERGE Data Link (25669): Shared memory segment located at address 0x7f87e1702000.
MERGE Data Link (25671): Message logger not contacted.
MERGE Data Link (25671): MIDAS MERGE Data Link 64 bit Build May 11 2021  20:32:02
Creating NetVars
Creating NetVars complete
MERGE Data Link (25671): Configuration: index = 9, SHM key=11000, TCP port base = 11010, module = 8, hardware = 0
MERGE Data Link (25671): File mapped object /SHM_11000 accessed
MERGE Data Link (25671): Memory mapping 536628 bytes
MERGE Data Link (25671): Shared memory segment located at address 0x7f38b4f59000.
MERGE Data Link (25671): Allocated 71720 bytes for local input buffer
MERGE Data Link (25671): File mapped object /SHM_11010 accessed
MERGE Data Link (25671): Memory mapping 100669480 bytes
MERGE Data Link (25671): Shared memory segment located at address 0x7f38aef57000.
MERGE Data Link (25670): Message logger not contacted.
MERGE Data Link (25670): MIDAS MERGE Data Link 64 bit Build May 11 2021  20:32:02
Creating NetVars
MERGE Master (25659): Link task has pid 25673
Creating NetVars complete
MERGE Data Link (25670): Configuration: index = 8, SHM key=11000, TCP port base = 11009, module = 7, hardware = 0
MERGE Data Link (25670): File mapped object /SHM_11000 accessed
MERGE Data Link (25670): Memory mapping 536628 bytes
MERGE Data Link (25672): Message logger not contacted.
MERGE Data Link (25670): Shared memory segment located at address 0x7ff7744ce000.
MERGE Data Link (25672): MIDAS MERGE Data Link 64 bit Build May 11 2021  20:32:02
Creating NetVars
MERGE Data Link (25670): Allocated 71720 bytes for local input buffer
MERGE Data Link (25670): File mapped object /SHM_11009 accessed
MERGE Data Link (25670): Memory mapping 100669480 bytes
MERGE Data Link (25670): Shared memory segment located at address 0x7ff76e4cc000.
Creating NetVars complete
MERGE Data Link (25672): Configuration: index = 10, SHM key=11000, TCP port base = 11011, module = 9, hardware = 0
MERGE Data Link (25672): File mapped object /SHM_11000 accessed
MERGE Data Link (25672): Memory mapping 536628 bytes
MERGE Data Link (25672): Shared memory segment located at address 0x7f4bb287e000.
MERGE Data Link (25672): Allocated 71720 bytes for local input buffer
MERGE Data Link (25672): File mapped object /SHM_11011 accessed
MERGE Data Link (25672): Memory mapping 100669480 bytes
MERGE Data Link (25672): Shared memory segment located at address 0x7f4bac87c000.
MERGE Master (25659): Link task has pid 25674
MERGE Data Link (25673): Message logger not contacted.
MERGE Data Link (25673): MIDAS MERGE Data Link 64 bit Build May 11 2021  20:32:02
Creating NetVars
Creating NetVars complete
MERGE Data Link (25673): Configuration: index = 11, SHM key=11000, TCP port base = 11012, module = 10, hardware = 0
MERGE Data Link (25673): File mapped object /SHM_11000 accessed
MERGE Data Link (25673): Memory mapping 536628 bytes
MERGE Data Link (25673): Shared memory segment located at address 0x7fe3cd54a000.
MERGE Data Link (25673): Allocated 71720 bytes for local input buffer
MERGE Data Link (25673): File mapped object /SHM_11012 accessed
MERGE Data Link (25673): Memory mapping 100669480 bytes
MERGE Data Link (25673): Shared memory segment located at address 0x7fe3c7548000.
MERGE Master (25659): Link task has pid 25675
MERGE Data Link (25674): Message logger not contacted.
MERGE Data Link (25674): MIDAS MERGE Data Link 64 bit Build May 11 2021  20:32:02
Creating NetVars
Creating NetVars complete
MERGE Data Link (25674): Configuration: index = 12, SHM key=11000, TCP port base = 11013, module = 11, hardware = 0
MERGE Data Link (25674): File mapped object /SHM_11000 accessed
MERGE Data Link (25674): Memory mapping 536628 bytes
MERGE Data Link (25674): Shared memory segment located at address 0x7faec5dbd000.
MERGE Data Link (25674): Allocated 71720 bytes for local input buffer
MERGE Data Link (25674): File mapped object /SHM_11013 accessed
MERGE Data Link (25674): Memory mapping 100669480 bytes
MERGE Data Link (25674): Shared memory segment located at address 0x7faebfdbb000.
MERGE Master (25659): Link task has pid 25676
MERGE Master (25659): Link task has pid 25677
MERGE Data Link (25675): Message logger not contacted.
MERGE Data Link (25675): MIDAS MERGE Data Link 64 bit Build May 11 2021  20:32:02
Creating NetVars
Creating NetVars complete
MERGE Data Link (25675): Configuration: index = 13, SHM key=11000, TCP port base = 11014, module = 12, hardware = 0
MERGE Data Link (25675): File mapped object /SHM_11000 accessed
MERGE Data Link (25675): Memory mapping 536628 bytes
MERGE Data Link (25675): Shared memory segment located at address 0x7fef43120000.
MERGE Data Link (25675): Allocated 71720 bytes for local input buffer
MERGE Data Link (25675): File mapped object /SHM_11014 accessed
MERGE Data Link (25675): Memory mapping 100669480 bytes
MERGE Data Link (25675): Shared memory segment located at address 0x7fef3d11e000.
MERGE Data Link (25676): Message logger not contacted.
MERGE Data Link (25676): MIDAS MERGE Data Link 64 bit Build May 11 2021  20:32:02
Creating NetVars
Creating NetVars complete
MERGE Data Link (25676): Configuration: index = 14, SHM key=11000, TCP port base = 11015, module = 13, hardware = 0
MERGE Data Link (25676): File mapped object /SHM_11000 accessed
MERGE Data Link (25676): Memory mapping 536628 bytes
MERGE Data Link (25676): Shared memory segment located at address 0x7f6aea271000.
MERGE Data Link (25676): Allocated 71720 bytes for local input buffer
MERGE Data Link (25676): File mapped object /SHM_11015 accessed
MERGE Data Link (25676): Memory mapping 100669480 bytes
MERGE Data Link (25676): Shared memory segment located at address 0x7f6ae426f000.
MERGE Data Link (25677): Message logger not contacted.
MERGE Data Link (25677): MIDAS MERGE Data Link 64 bit Build May 11 2021  20:32:02
Creating NetVars
Creating NetVars complete
MERGE Data Link (25677): Configuration: index = 15, SHM key=11000, TCP port base = 11016, module = 14, hardware = 0
MERGE Data Link (25677): File mapped object /SHM_11000 accessed
MERGE Data Link (25677): Memory mapping 536628 bytes
MERGE Data Link (25677): Shared memory segment located at address 0x7fc4281e9000.
MERGE Data Link (25677): Allocated 71720 bytes for local input buffer
MERGE Data Link (25677): File mapped object /SHM_11016 accessed
MERGE Data Link (25677): Memory mapping 100669480 bytes
MERGE Data Link (25677): Shared memory segment located at address 0x7fc4221e7000.
MERGE Statistics/Rates (25661): MIDAS MERGE Statistics 64 bit Build May 11 2021  20:32:03
MERGE Statistics thread starting
MERGE Data Link (25665): Shared memory segment located at address 0x7f7412ae4000.
MERGE Data Link (25665): Shared memory segment located at address 0x7f7412ae4000.
MERGE Data Link (25663): Shared memory segment located at address 0x7f3be8601000.
MERGE Data Link (25663): Shared memory segment located at address 0x7f3be8601000.
MERGE Data Link (25662): Shared memory segment located at address 0x7f1988e78000.
MERGE Data Link (25662): Shared memory segment located at address 0x7f1988e78000.
MERGE Data Link (25667): Shared memory segment located at address 0x7f8b818c7000.
MERGE Data Link (25667): Shared memory segment located at address 0x7f8b818c7000.
MERGE Data Link (25668): Shared memory segment located at address 0x7f88c1c98000.
MERGE Data Link (25668): Shared memory segment located at address 0x7f88c1c98000.
MERGE Data Link (25664): Shared memory segment located at address 0x7f0da12c6000.
MERGE Data Link (25664): Shared memory segment located at address 0x7f0da12c6000.
MERGE Data Link (25666): Shared memory segment located at address 0x7fb7eeec0000.
MERGE Data Link (25666): Shared memory segment located at address 0x7fb7eeec0000.
MERGE Data Link (25670): Shared memory segment located at address 0x7ff76e4cc000.
MERGE Data Link (25670): Shared memory segment located at address 0x7ff76e4cc000.
MERGE Data Link (25673): Shared memory segment located at address 0x7fe3c7548000.
MERGE Data Link (25673): Shared memory segment located at address 0x7fe3c7548000.
MERGE Data Link (25672): Shared memory segment located at address 0x7f4bac87c000.
... 201 more lines ...
Attachment 3: 210602_TapeServerStartup
Tidy up
master(4596): Operation not permitted
Starting Tape Server

[1] 25639
MIDAS Tape Server: Message logger not contacted.
MIDAS Tape Server: MIDAS Tape Server Build Mar 16 2021  12:22:14
MIDAS Tape Server: Unable to change scheduling priority - Permission denied
MIDAS Tape Server: Using default startup
MIDAS Tape Server: Configuration: UDP port = 10205, SHM key=10205.
MIDAS Tape Server: File mapped object /SHM_10205 of size 1331104 created
MIDAS Tape Server: Shared memory ID is 3
MIDAS Tape Server: Shared memory segment located at address 7fdf19721000.
MIDAS Tape Server: Configuration file used - /MIDAS/config/TS_10205/TS_configuration
MIDAS Tape Server: Stats task /MIDAS/TapeServer/Linux64/stats
MIDAS Tape Server: Using device file /dev/file/0 /MIDAS/TapeServer/Linux64/driver
MIDAS Tape Server: Using device file /dev/file/1 /MIDAS/TapeServer/Linux64/driver
MIDAS Tape Server: Using device sink /dev/null/0 /MIDAS/TapeServer/Linux64/driver
MIDAS Tape Server: Data link /MIDAS/TapeServer/Linux64/linkTCP 10305                 
MIDAS Tape Server: Data link /MIDAS/TapeServer/Linux64/linkTCP 21001 21002                
MIDAS Tape Server: Message reporting level = 0x180fff8
MIDAS Tape Server: Message logging level = 0xfff8
MIDAS Tape Server: Tape Server Options = 0x0
MIDAS Tape Server: File device path base = /TapeData
MIDAS Tape Server: Data buffer size = 65536
MIDAS Tape Server: Tape block size = 65536
MIDAS Tape Server: File mapped object /SHM_110205 of size 4195880 created
MIDAS Tape Server: Shared memory ID is 3
MIDAS Tape Server: Shared memory segment located at address 7fdf19320000.
MIDAS Tape Server: File mapped object /SHM_210205 of size 3100 created
MIDAS Tape Server: Shared memory ID is 3
MIDAS Tape Server: Shared memory segment located at address 7fdf19881000.
MIDAS Tape Server: Capabilities restored.
MIDAS Tape Server: Master global area initialised.
MIDAS Tape Server: Stats task has pid 25641
MIDAS Tape Server: Driver process for /dev/file/0 has pid 25642
MIDAS Tape Server: Driver process for /dev/file/1 has pid 25643
MIDAS Tape Server: Driver process for /dev/null/0 has pid 25644
MIDAS Tape Server: Link task 0 has pid 25645
MIDAS Tape Server: Link task 1 has pid 25646
MIDAS Tape Server: Starting the RPC interface
MIDAS Tape Statistics: Message logger not contacted.
MIDAS Tape Statistics: MIDAS Tape Statistics Build Mar 16 2021  12:22:20
MIDAS Tape Statistics: Started with args 10205
MIDAS Tape Statistics: Configuration: SHM key=10205
MIDAS Tape Statistics: File mapped object /SHM_10205 of size 1331104 created
MIDAS Tape Statistics: Shared memory ID is 3
MIDAS Tape Statistics: Shared memory segment located at address 7f9c551f7000.
MIDAS Data Link (25645): Message logger not contacted.
MIDAS Data Link (25645): MIDAS Tape Data Link Build Mar 16 2021  12:22:19
MIDAS Data Link (25645): Started with args 10205 10305                 
MIDAS Data Link (25645): Configuration: SHM key=10205, TCP port = 10305, Data Stream = 1
MIDAS Tape Driver (25644): Message logger not contacted.
MIDAS Tape Driver (25644): Message logger not contacted.
MIDAS Data Link (25646): Message logger not contacted.
MIDAS Tape Driver (25644): Started with args 2 10205
MIDAS Data Link (25645): File mapped object /SHM_10205 of size 1331104 created
MIDAS Data Link (25645): Shared memory ID is 3
MIDAS Data Link (25645): Shared memory segment located at address 7f6ba30ef000.
MIDAS Data Link (25646): MIDAS Tape Data Link Build Mar 16 2021  12:22:19
MIDAS Tape Driver (25644): Configuration: driver=2, key=10205.
MIDAS Data Link (25646): Started with args 10205 21001 21002                
MIDAS Data Link (25645): Starting the network interface
MIDAS Tape Driver (25644): File mapped object /SHM_10205 of size 1331104 created
MIDAS Data Link (25646): Configuration: SHM key=10205, TCP port = 21001, Data Stream = 2
MIDAS Tape Driver (25644): Shared memory ID is 3
MIDAS Tape Driver (25644): Shared memory segment located at address 7f09d718d000.
MIDAS Tape Driver (25644): Using device /dev/null/0 of type sink.
MIDAS Data Link (25646): File mapped object /SHM_10205 of size 1331104 created
MIDAS Data Link (25646): Shared memory ID is 3
MIDAS Data Link (25646): Shared memory segment located at address 7fa75ae3c000.
MIDAS Data Link (25646): Starting the network interface
MIDAS Tape Driver (25642): Message logger not contacted.
MIDAS Data Link (25645): TCP socket receive buffer was 87380 - now 249856
MIDAS Tape Driver (25642): Message logger not contacted.
MIDAS Tape Driver (25642): Started with args 0 10205
MIDAS Tape Driver (25642): Configuration: driver=0, key=10205.
MIDAS Tape Driver (25643): Message logger not contacted.
MIDAS Tape Driver (25642): File mapped object /SHM_10205 of size 1331104 created
MIDAS Tape Driver (25643): Message logger not contacted.
MIDAS Tape Driver (25642): Shared memory ID is 3
MIDAS Data Link (25646): TCP socket receive buffer was 87380 - now 249856
MIDAS Data Link (25645): TCP socket send buffer was 16384 - now 249856
MIDAS Tape Driver (25643): Started with args 1 10205
MIDAS Tape Driver (25642): Shared memory segment located at address 7f48b1ea4000.
MIDAS Tape Driver (25642): Using device /dev/file/0 of type file.
MIDAS Data Link (25645): MIDAS Data Link thread 0 using TCP port 10305.
MIDAS Data Link (25645): Entering server loop
MIDAS Tape Driver (25643): Configuration: driver=1, key=10205.
MIDAS Data Link (25645): thread 0 listening on port 10305
MIDAS Tape Driver (25643): File mapped object /SHM_10205 of size 1331104 created
MIDAS Tape Driver (25643): Shared memory ID is 3
MIDAS Tape Driver (25643): Shared memory segment located at address 7f6044052000.
MIDAS Data Link (25646): TCP socket send buffer was 16384 - now 249856
MIDAS Tape Driver (25643): Using device /dev/file/1 of type file.
MIDAS Data Link (25646): MIDAS Data Link thread 0 using TCP port 21001.
MIDAS Data Link (25646): TCP socket receive buffer was 87380 - now 249856
MIDAS Data Link (25646): TCP socket send buffer was 16384 - now 249856
MIDAS Data Link (25646): MIDAS Data Link thread 1 using TCP port 21002.
MIDAS Data Link (25646): Entering server loop
MIDAS Data Link (25646): thread 0 listening on port 21001
MIDAS Data Link (25646): thread 1 listening on port 21002
MIDAS Tape Server: Created RPC Program 28000205 Version 4 on UDP port 10205.
MIDAS Tape Server: Entering server loop
MIDAS Tape Server: MIDAS Tape Server now available on UDP port 10205.
MIDAS Data Link (25645): thread 0 accepted connection from 127.0.0.1, port 37330
MIDAS Data Link (25645): buffer size changed to 65536
MIDAS Tape Server: device /dev/file/0 allocated.
MIDAS Tape Server: Mounting volume S496 on device /dev/file/0.
MIDAS Tape Server: Opening file R31 on device /dev/file/0.
2222 64 0 31056336
3333 31056336 2048000 1 15 

Attachment 4: TS_configuration
#device configuration information
#    any line starting with a # is a comment and is ignored
stats /MIDAS/TapeServer/Linux64/stats
#    devices available   -   list ends with a null line
#    format       class_name     device_name    driver_task_path_name
#       class_name MUST be one of dlt, exabyte, scsitape, sink
file          /dev/file/0    /MIDAS/TapeServer/Linux64/driver
file          /dev/file/1    /MIDAS/TapeServer/Linux64/driver
sink          /dev/null/0     /MIDAS/TapeServer/Linux64/driver

#data link configuration information  - list end with a null line
#    format   link_task_path_name  
/MIDAS/TapeServer/Linux64/linkTCP 10305
/MIDAS/TapeServer/Linux64/linkTCP 21001 21002

#program options - list end with a null line
#tapeserver_options 0x2
tapeserver_options 0x0
msg_reporting_level 0x0180fff8
#   use 0x0080fff8 to enable msg logging
msg_logging_level 0xfff8
#    default for following is 16Kbytes - both MUST be the same at present
data_buffer_size 64
tape_block_size 64
disc_file_size 2000
#
file_path_base  /TapeData

#end of information


Attachment 5: CONTENTS
Index string MERGE.LinksAvailable&&RunNumber&&MERGE.LinksInUse&&MERGE.RunOptions
Stat.shift string 0
0x0006dead string 0x0000
MERGE.LinksAvailable string 16
Stat.offset string 0
RunNumber string 0
Rate.channels string 1024
Stat.channels string 1024
MERGE.RunOptions string 3
MERGE.LinksInUse string 1%1%1%1%1%1%1%1%1%1%1%1%1%1%1%1%
Merge.LinksInUse string 1%1%1%1%1%1%1%1%1%1%1%1%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%0%
Attachment 6: 210602_merger.png
210602_merger.png
  35   Mon Mar 25 14:44:33 2019 NHMerger issues

Plastic electronics have now been installed.

Water is not leaking still.

FEE64s powered on at 15:00 to test MBS connection (fig1: temperatures)

MBS relay connects to MBS server OK...

AIDA Merger does not accept any data... (links with data reads 0) and lots of timestamp errors in log? (Figs 2-4)


Startup procedure incorrect?

Otherwise things still OK.

Attachment 1: Temps.png
Temps.png
Attachment 2: AIDARUNv2.png
AIDARUNv2.png
Attachment 3: Mergerv2.png
Mergerv2.png
Attachment 4: Tapev2.png
Tapev2.png
Attachment 5: MergerConsolev2.png
MergerConsolev2.png
  559   Wed Apr 3 13:02:19 2024 NHMerger for 16 FEEs
Changed /MIDAS/Linux/startup/NewMerger

Change parameters -i and -l in master64 to 16 for 16 FEEs
Update NewMerger Options LinksAvailable to 16, LinksInUse to 1%1%1%1%1%1%1%1%1%1%1%1%1%1%1%1%
Fix NetVar RunOptions 1 (was 0)

Restart Merger HTTPd, Tape, Merger, MBS Spy

Reset/Setup/Go 

16 Links green and status going, all good?

Bias DSSSDs and turn data transfer on

Merger connected, shows rate and updates... no rate in Tape Server?

.. Oops forget to turn on Output to data storage in merger!

Rate in tape server and to MBS: 7 MB/s

Merger, Tape and MBS working with 16 FEEs
Attachment 1: Screenshot_from_2024-04-03_14-15-02.png
Screenshot_from_2024-04-03_14-15-02.png
Attachment 2: Screenshot_from_2024-04-03_14-15-25.png
Screenshot_from_2024-04-03_14-15-25.png
Attachment 3: Screenshot_from_2024-04-03_14-15-36.png
Screenshot_from_2024-04-03_14-15-36.png
Attachment 4: Screenshot_from_2024-04-03_14-15-45.png
Screenshot_from_2024-04-03_14-15-45.png
Attachment 5: Screenshot_from_2024-04-03_14-15-52.png
Screenshot_from_2024-04-03_14-15-52.png
  76   Thu Oct 31 16:43:03 2019 NH CA TDMerger & MBS Performance
Testing merger with 3 DSSDs connected, with thresholds of 10 (slow comparator), 2 (HEC) and 255 (fast discriminator)

Merger handling rate of 1.6 million events per second comfortably. Forwarding to MBS fine.

Network usage: 130 Mbps (AIDA network) and 20 Mbps (MBS network)
Attachment 1: MergerStats.png
MergerStats.png
Attachment 2: MBSRates.png
MBSRates.png
Attachment 3: Mergernetworking.png
Mergernetworking.png
Attachment 4: MergerStatistics.png
MergerStatistics.png
ELOG V3.1.3-7933898