11.30 DAQ continues - file S496/R18_472
      alpha background

      All system wide checks OK *except*

FEE64 module aida06 global clocks failed, 6
 Clock status test result: Passed 15, Failed 1

Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1' 
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable

		 Base 		Current 	Difference
aida01 fault 	 0x82b0 : 	 0x82b2 : 	 2  
aida02 fault 	 0xfe72 : 	 0xfe74 : 	 2  
aida03 fault 	 0x97f4 : 	 0x97f6 : 	 2  
aida04 fault 	 0x950 : 	 0x952 : 	 2  
aida05 fault 	 0x8e93 : 	 0x8e95 : 	 2  
aida06 fault 	 0x545a : 	 0x545c : 	 2  
aida07 fault 	 0xc04d : 	 0xc04f : 	 2  
aida08 fault 	 0xa0f7 : 	 0xa0f9 : 	 2  
White Rabbit error counter test result: Passed 8, Failed 8

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0  
Mem(KB) :	4	8	16	32	64	128	256	512	1k	2k	4k
aida01 :      1      4      7      1      3      2      2      2      2      4      6   : 36980
aida02 :      3      1      5      1      2      4      3      3      2      4      6   : 37892
aida03 :      8      8      6      6      3      3      2      3      3      3      6   : 36800
aida04 :     37     12      5      6      3      4      2      3      3      3      6   : 37060
aida05 :     11      5      6      4      2      2      2      2      2      4      6   : 37044
aida06 :      4      3      3      3      2      2      2      3      3      4      6   : 38456
aida07 :      1     12      3      2      0      3      3      2      2      4      6   : 37204
aida08 :     13      8      1      2      3      4      3      3      2      4      6   : 38020
aida09 :      7      7      5      1      1      3      2      2      2      4      6   : 36996
aida10 :     12      7      9      2      1      3      3      3      1      4      6   : 36856
aida11 :      2      9      3      1      2      3      3      3      1      4      6   : 36768
aida12 :     11      4      3      3      2      4      2      3      2      4      6   : 37724
aida13 :     10      8      6      2      2      3      2      2      2      4      6   : 37128
aida14 :     16      7      7      1      0      4      2      2      2      4      6   : 37128
aida15 :     12      6      2      2      1      3      3      4      2      4      6   : 38272
aida16 :      9      4     10      1      2      3      2      2      2      4      6   : 37124

Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same

 FEE : aida01 =>   Options file size is 1026 	 Last changed Sun May 23 12:04:25 CEST 2021
 FEE : aida02 =>   Options file size is 1025 	 Last changed Sun May 23 00:19:21 CEST 2021
 FEE : aida03 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:43:50 CEST 2021
 FEE : aida04 =>   Options file size is 1025 	 Last changed Fri May 14 16:54:56 CEST 2021
 FEE : aida05 =>   Options file size is 1025 	 Last changed Mon May 17 06:25:41 CEST 2021
 FEE : aida06 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:43:59 CEST 2021
 FEE : aida07 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:02 CEST 2021
 FEE : aida08 =>   Options file size is 1025 	 Last changed Sun May 23 00:16:54 CEST 2021
 FEE : aida09 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:08 CEST 2021
 FEE : aida10 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida11 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida12 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida13 =>   Options file size is 1025 	 Last changed Fri May 07 19:40:34 CEST 2021
 FEE : aida14 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida15 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida16 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021


      Grafana - DSSSD bias & leakage current - most recent 7 days - attachment 1
      Lost activity monitor - attachment 2
      1.8.W spectra - attachments 3 & 4
      ADC data item stats - attachment 5
      FEE64 temps OK - attachment 6
      DSSSD bias & leakage current OK - attachment 7
      Merger/Tape Server/Merger stats - attachment 8
       no merger errors reported since previous restart

14.30 analysis of file S496/R18_471 - attachment 9
       max deadtime 0.18% aida04