AIDA GELINA BRIKEN nToF CRIB ISOLDE CIRCE nTOFCapture DESPEC DTAS EDI_PSA 179Ta CARME StellarModelling DCF K40
  DESPEC  ELOG logo
Message ID: 367     Entry time: Sun Jun 13 17:29:04 2021
Author: TD 
Subject: Sunday 13 June 
18.15 DAQ continues file S496/R32_946
      Hot HEC channels multiple FEE64s from c. 06.00 this morning
      ASIC check fixes

      Out of disk space on /TapeData filesystem - switch to no storage mode

      System wide checks OK *except*

		 Base 		Current 	Difference
aida01 fault 	 0x8944 : 	 0x8949 : 	 5  
aida02 fault 	 0x9b36 : 	 0x9b3b : 	 5  
aida03 fault 	 0xf26b : 	 0xf270 : 	 5  
aida04 fault 	 0x57a0 : 	 0x57a5 : 	 5  
aida05 fault 	 0x4d3c : 	 0x4d3e : 	 2  
aida05 : WR status 0x10
 aida06 fault 	 0x640c : 	 0x640e : 	 2  
aida07 fault 	 0x255e : 	 0x2560 : 	 2  
aida08 fault 	 0xffbd : 	 0xffbf : 	 2  
aida15 fault 	 0x4e4 : 	 0x1267 : 	 3459  
White Rabbit error counter test result: Passed 7, Failed 9

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

			 Base 		Current 		Difference
aida12 fault 	 0x0 : 	 0x3e01 : 	 15873  
aida13 fault 	 0x0 : 	 0x32d2 : 	 13010  
FPGA Timestamp error counter test result: Passed 14, Failed 2
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last

Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0  
Mem(KB) :	4	8	16	32	64	128	256	512	1k	2k	4k
aida01 :     26     16      4      6      4      4      2      2      2      4      6   : 37608
aida02 :     29     11      9      3      3      4      3      2      2      4      6   : 37756
aida03 :     32      9      6      8      5      3      1      3      3      3      6   : 36840
aida04 :     32      8      3      4      3      4      2      3      3      3      6   : 36912
aida05 :     36      8      6      4      5      4      2      2      2      4      6   : 37616
aida06 :     25     13      4      3      2      3      2      3      4      3      6   : 37740
aida07 :     28     13      4      5      5      2      3      2      2      4      6   : 37624
aida08 :     29      5      9      4      3      4      3      2      2      4      6   : 37740
aida09 :     41      8      4      7      1      3      3      2      2      4      6   : 37572
aida10 :     45     11      7      5      5      3      3      3      3      3      6   : 37340
aida11 :     33     13      5      6      3      4      3      3      1      4      6   : 37308
aida12 :     39      8      6      6      4      4      3      3      1      4      6   : 37372
aida13 :     28     14      3      6      4      4      3      3      1      4      6   : 37328
aida14 :     30     11      3      6      4      4      3      3      3      3      6   : 37312
aida15 :      3      3      4      3      0      3      3      2      1      4      7   : 40260
aida16 :     28      9      5      6      5      3      3      3      1      4      6   : 37256

Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same

 FEE : aida01 =>   Options file size is 1026 	 Last changed Sat Jun 12 15:46:33 CEST 2021
 FEE : aida02 =>   Options file size is 1025 	 Last changed Sun May 23 00:19:21 CEST 2021
 FEE : aida03 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:43:50 CEST 2021
 FEE : aida04 =>   Options file size is 1025 	 Last changed Fri May 14 16:54:56 CEST 2021
 FEE : aida05 =>   Options file size is 1025 	 Last changed Mon May 17 06:25:41 CEST 2021
 FEE : aida06 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:43:59 CEST 2021
 FEE : aida07 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:02 CEST 2021
 FEE : aida08 =>   Options file size is 1025 	 Last changed Sun May 23 00:16:54 CEST 2021
 FEE : aida09 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:08 CEST 2021
 FEE : aida10 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida11 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida12 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida13 =>   Options file size is 1025 	 Last changed Fri May 07 19:40:34 CEST 2021
 FEE : aida14 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida15 =>   Options file size is 1025 	 Last changed Sat Jun 12 15:48:28 CEST 2021
 FEE : aida16 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021

      Grafana - DSSSD bias & leakage current - most recent 7 days - attachment 1
      Lost activity monitor - attachment 2
      1.8.W spectra - 20us FSR - attachments 3 & 4
      ADC data items - attachment 5
      FEE64 temperatures OK - attachment 6
      DSSSD bias & leakage currents OK - attachment 7
      Merger/Tape Server/Merger statistics - attachment 8
       no merger errors reported since previous restart
Attachment 1: Screenshot_2021-06-13_Spectrum_Browser_aidas-gsi(1).png  281 kB  Uploaded Sun Jun 13 19:02:53 2021  | Hide | Hide all
Screenshot_2021-06-13_Spectrum_Browser_aidas-gsi(1).png
Attachment 2: Screenshot_2021-06-13_Spectrum_Browser_aidas-gsi.png  365 kB  Uploaded Sun Jun 13 19:02:53 2021  | Hide | Hide all
Screenshot_2021-06-13_Spectrum_Browser_aidas-gsi.png
Attachment 3: Screenshot_2021-06-13_AIDA_Alerting_-_Grafana.png  312 kB  Uploaded Sun Jun 13 19:02:53 2021  | Hide | Hide all
Screenshot_2021-06-13_AIDA_Alerting_-_Grafana.png
Attachment 4: Screenshot_2021-06-13_Lost_activity_monitor_aidas-gsi.png  157 kB  Uploaded Sun Jun 13 19:02:53 2021  | Hide | Hide all
Screenshot_2021-06-13_Lost_activity_monitor_aidas-gsi.png
Attachment 5: Screenshot_2021-06-13_Statistics_aidas-gsi.png  100 kB  Uploaded Sun Jun 13 19:02:53 2021  | Hide | Hide all
Screenshot_2021-06-13_Statistics_aidas-gsi.png
Attachment 6: Screenshot_2021-06-13_Temperature_and_status_scan_aidas-gsi.png  146 kB  Uploaded Sun Jun 13 19:02:53 2021  | Hide | Hide all
Screenshot_2021-06-13_Temperature_and_status_scan_aidas-gsi.png
Attachment 7: 1000.png  11 kB  Uploaded Sun Jun 13 19:03:48 2021  | Hide | Hide all
1000.png
Attachment 8: 1001.png  558 kB  Uploaded Sun Jun 13 19:03:48 2021  | Hide | Hide all
1001.png
ELOG V3.1.4-unknown