13.28 System wide checks

Clock status test result: Passed 12, Failed 0

Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1' 
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable

Calibration test result: Passed 12, Failed 0

If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module

		 Base 		Current 	Difference
aida01 fault 	 0x44c2 : 	 0x44c5 : 	 3  
aida02 fault 	 0xc81f : 	 0xc822 : 	 3  
aida03 fault 	 0x9342 : 	 0x9344 : 	 2  
aida04 fault 	 0xdb3f : 	 0xdb41 : 	 2  
aida05 fault 	 0x4baf : 	 0x4bb3 : 	 4  
aida06 fault 	 0xd2a4 : 	 0xd2a8 : 	 4  
aida07 fault 	 0xc51c : 	 0xc51f : 	 3  
aida08 fault 	 0xe853 : 	 0xe856 : 	 3  
aida09 fault 	 0x5212 : 	 0x5216 : 	 4  
aida10 fault 	 0xf81d : 	 0xf81f : 	 2  
aida11 fault 	 0x810 : 	 0x812 : 	 2  
aida12 fault 	 0xa279 : 	 0xa27a : 	 1  
White Rabbit error counter test result: Passed 0, Failed 12

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR


			 Base 		Current 		Difference
aida09 fault 	 0x0 : 	 0x5 : 	 5  
aida12 fault 	 0x0 : 	 0xc : 	 12  
FPGA Timestamp error counter test result: Passed 10, Failed 2
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last


Returned 0 0 0 0 0 0 0 0 0 0 0 0  
Mem(KB) :	4	8	16	32	64	128	256	512	1k	2k	4k
aida01 :     57     33     12     15      3      3      3      2      3      2      6   : 35276
aida02 :     15      3      5      2      1      4      1      3      3      3      6   : 36388
aida03 :     13      4      2      1      4      3      1      3      3      3      6   : 36372
aida04 :     21      5      2      1      2      4      4      2      3      3      6   : 36668
aida05 :     10      5      6      3      2      3      3      3      1      4      6   : 36880
aida06 :     22     12      1      0      3      3      1      3      3      3      6   : 36360
aida07 :     19      3      3      0      3      3      2      4      1      4      6   : 37076
aida08 :     21      6      2      4      2      3      1      3      3      3      6   : 36388
aida09 :      2      7      5      3      2      4      2      3      2      3      6   : 35696
aida10 :      7      3      1      3      1      3      3      3      2      3      6   : 35684
aida11 :      1      2      1      3      2      5      2      2      3      3      6   : 36228
aida12 :     17     10      4      2      3      2      2      4      2      3      6   : 36052



 Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same

 FEE : aida01 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:04 CEST 2021
 FEE : aida02 =>   Options file size is 2114 	 Last changed Wed Apr 14 21:52:05 CEST 2021
 FEE : aida03 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:04 CEST 2021
 FEE : aida04 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:04 CEST 2021
 FEE : aida05 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:05 CEST 2021
 FEE : aida06 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:04 CEST 2021
 FEE : aida07 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:04 CEST 2021
 FEE : aida08 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:04 CEST 2021
 FEE : aida09 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:05 CEST 2021
 FEE : aida10 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:06 CEST 2021
 FEE : aida11 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:05 CEST 2021
 FEE : aida12 =>   Options file size is 1025 	 Last changed Wed Apr 14 21:58:54 CEST 2021

13.30 Output of ~/oh/OptionsCheck.py

02:25:00 PM
FAULTS!

Unknown option in FEE2 Options database
Unknown entry starts: 
et&&Aida_Hist_W_Enable&&TS_SYNC_PHASE&&ExtClk&&Aida.shift&&MACB_TRIG_MODE&&Aida.offset&&RunNumber&&Aida_GroupBase&&Rate.channels&&Stat.channels&&Aida_Hist_D_Enable&&WAVE_DMA_HWM&&ASIC.se
ttings&&Aida.Vchannels&&Aida.Wchannels&&Stat.shift&&Aida.channels&&Include.Aida&&Aida_Hist_V_Enable&&DataFormat&&Aida_Hist_H_Enable&&DataAcqPgm&&ASIC_DMA_HWM&&Aida_Hist_L_Enable

Unknown option in FEE2 Options database
Unknown entry starts: 
da_Hist_W_Enable&&TS_SYNC_PHASE&&ExtClk&&Aida.shift&&MACB_TRIG_MODE&&Aida.offset&&RunNumber&&Aida_GroupBase&&Rate.channels&&Stat.channels&&Aida.Vchannels&&ASIC.settings&&WAVE_DMA_HWM&&Ai
da_Hist_D_Enable&&Stat.shift&&Aida.Wchannels&&Aida_Hist_V_Enable&&Include.Aida&&Aida.channels&&DataAcqPgm&&Aida_Hist_H_Enable&&DataFormat&&ASIC_DMA_HWM&&Aida_Hist_L_Enable

Unknown option in FEE2 Options database
Unknown entry starts: 
et&&Aida_Hist_W_Enable&&TS_SYNC_PHASE&&ExtClk&&Aida.shift&&MACB_TRIG_MODE&&Aida.offset&&RunNumber&&Aida_GroupBase&&Rate.channels&&Stat.channels&&Aida.Vchannels&&ASIC.settings&&WAVE_DMA_H
WM&&Aida_Hist_D_Enable&&Stat.shift&&Aida.Wchannels&&Aida_Hist_V_Enable&&Include.Aida&&Aida.channels&&DataAcqPgm&&Aida_Hist_H_Enable&&DataFormat&&ASIC_DMA_HWM&&Aida_Hist_L_Enable

13.37 Detector bias & leakage currents OK - attachment 1
      FEE64 temps OK - attachments 2
      Statistics - good events, ADC data, disc data, info code 4 & 5, pause & resume, correlation - attachments 3-10
      per FEE64 rate spectra - attachments 11 & 12
      per FEE64 1.8.L spectra - attachments 13 & 19
       aida5 peak width 70 ch FWHM, aida06 peak width 89ch FWHM
      per FEE64 1.8.W spectra 20us FSR - attachments 14 & 15
      meger & tape server - attachments 16-18