AIDA GELINA BRIKEN nToF CRIB ISOLDE CIRCE nTOFCapture DESPEC DTAS EDI_PSA 179Ta CARME StellarModelling DCF K40
  DESPEC  ELOG logo
Message ID: 343     Entry time: Fri May 21 23:19:23 2021
Author: TD 
Subject: Saturday 22 May 00:00-08:00 
00.16 413Gb remaining /TapeData filesystem
      estimate 413/2 x 177s = 10.15h to filesystem full - OK

00.18 All histograms, statistics and merger statistics zero'd

00.57 All system wide checks OK *except*

	 Base 		Current 	Difference
aida05 fault 	 0x500 : 	 0x572 : 	 114  
White Rabbit error counter test result: Passed 15, Failed 1

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR


Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0  
Mem(KB) :	4	8	16	32	64	128	256	512	1k	2k	4k
aida01 :     19      9      3      4      1      5      2      2      3      3      6   : 36356
aida02 :      4      8      7      2      4      3      3      3      2      3      6   : 35968
aida03 :     12      9      6      2      5      3      2      3      3      3      6   : 36824
aida04 :     11      5      9      2      2      2      2      4      3      3      6   : 37028
aida05 :     24      9      2      1      2      3      2      3      4      3      6   : 37608
aida06 :     25      6      3      4      2      4      3      3      3      3      6   : 37060
aida07 :      3      3      3      1      3      3      2      2      2      2      7   : 37044
aida08 :     20      4      2      3      1      3      2      2      3      3      6   : 36016
aida09 :     15      8      2      5      0      3      3      2      3      3      6   : 36284
aida10 :     16      6      4      1      2      3      3      2      2      4      6   : 37328
aida11 :     20      2      2      1      1      3      2      2      3      3      6   : 35936
aida12 :      1      2      1      2      3      4      1      2      4      3      6   : 36900
aida13 :     25     10      4      1      0      3      2      3      3      3      6   : 36500
aida14 :     20      6      9      2      2      5      1      2      2      4      6   : 37200
aida15 :     31      9      5      2      3      3      1      3      3      3      6   : 36500
aida16 :     24      4      2      3      1      4      2      3      3      3      6   : 36672

Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same

 FEE : aida01 =>   Options file size is 1026 	 Last changed Mon May 17 11:43:04 CEST 2021
 FEE : aida02 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:43:46 CEST 2021
 FEE : aida03 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:43:50 CEST 2021
 FEE : aida04 =>   Options file size is 1025 	 Last changed Fri May 14 16:54:56 CEST 2021
 FEE : aida05 =>   Options file size is 1025 	 Last changed Mon May 17 06:25:41 CEST 2021
 FEE : aida06 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:43:59 CEST 2021
 FEE : aida07 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:02 CEST 2021
 FEE : aida08 =>   Options file size is 1025 	 Last changed Wed May 05 12:15:54 CEST 2021
 FEE : aida09 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:08 CEST 2021
 FEE : aida10 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida11 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida12 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida13 =>   Options file size is 1025 	 Last changed Fri May 07 19:40:34 CEST 2021
 FEE : aida14 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida15 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida16 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021

01.12 Lost activity monitor - attachment 1
      1.8.H spectra - attachments 2 & 3
      1.8.L spectra - attachments 4-6
       aida01 pulser peak width 120 ch FWHM
      Rate specttra - attachments 7 & 8
01.06 Lost activity monitor - attachment 9
      ucesb - attachment 10
      Grafana - DSSSD bias & leakage currents past 7 days - attachment 11
      statistics - adc data, disc data, WR 28-47, pause, correlation scaler, good wave events - attachments 12-17
      FEE64 temperatures OK - attachment 18
      DSSSD bias & leakage currents OK - attachment 19
      Merger/Tape Server/Merger stats - attachment 20
       no merger warning/error messages since last restart
      
01.26 Analysis file S496/R14_1798 - attachment 21
       aida08 5.5% deadtime

01.17 Disk space remaining /TapeData filesystem

[npg@aidas-gsi S496]$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_aidasgsi-lv_root
                       50G   18G   30G  37% /
tmpfs                 7.9G  706M  7.2G   9% /dev/shm
/dev/sda2             477M   42M  410M  10% /boot
/dev/sda1             200M  264K  200M   1% /boot/efi
/dev/mapper/vg_aidasgsi-lv_home
                      401G   90G  292G  24% /home
/dev/mapper/vg_aidas2-lv_home
                      7.2T  6.4T  395G  95% /media/1e121361-83d3-4825-b6ae-8700b07e0ca7
/dev/sdd1             7.2T  5.6T  1.3T  82% /media/SecondDrive
/dev/sdc1             7.2T  6.6T  259G  97% /media/ThirdDrive

estimate 395/2 x 177s = 9.7h to filesystem full - OK

01.30 despec shift reports no data reported by ucesb
      aida stats & tape server ok - no data being forwarded to MBS by data relay

01.39 despec shift restart FATIMA DAQ - data forwarding to MBS by data relay resumes - AIDA file S496/R14_1811

04.35 System wide checks all OK *except*

Base 		Current 	Difference
aida05 fault 	 0x500 : 	 0x573 : 	 115  
White Rabbit error counter test result: Passed 15, Failed 1

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

[npg@aidas-gsi S496]$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_aidasgsi-lv_root
                       50G   18G   30G  37% /
tmpfs                 7.9G  661M  7.2G   9% /dev/shm
/dev/sda2             477M   42M  410M  10% /boot
/dev/sda1             200M  264K  200M   1% /boot/efi
/dev/mapper/vg_aidasgsi-lv_home
                      401G   90G  292G  24% /home
/dev/mapper/vg_aidas2-lv_home
                      7.2T  6.5T  337G  96% /media/1e121361-83d3-4825-b6ae-8700b07e0ca7
/dev/sdd1             7.2T  5.6T  1.3T  82% /media/SecondDrive
/dev/sdc1             7.2T  6.6T  259G  97% /media/ThirdDrive

estimate 337/2 x 177s = 8.3h to filesystem full - OK

FEE64 temperatures OK - attachment 22
ADC data item statistics OK - attachment 24
DSSSD bias & leakage current OK - attachment 26
analysis of file S496/R14_1920 - attachment 28
 aida04 deadtime 5.0%

07.10 Ssystem wide checks all OK *except*

	 Base 		Current 	Difference
aida05 fault 	 0x500 : 	 0x573 : 	 115  
White Rabbit error counter test result: Passed 15, Failed 1

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

[npg@aidas-gsi S496]$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_aidasgsi-lv_root
                       50G   18G   30G  37% /
tmpfs                 7.9G  662M  7.2G   9% /dev/shm
/dev/sda2             477M   42M  410M  10% /boot
/dev/sda1             200M  264K  200M   1% /boot/efi
/dev/mapper/vg_aidasgsi-lv_home
                      401G   90G  292G  24% /home
/dev/mapper/vg_aidas2-lv_home
                      7.2T  6.5T  287G  96% /media/1e121361-83d3-4825-b6ae-8700b07e0ca7
/dev/sdd1             7.2T  5.6T  1.3T  82% /media/SecondDrive
/dev/sdc1             7.2T  6.6T  259G  97% /media/ThirdDrive

estimate 287/2 x 178s = 7.1h to filesystem full - OK

FEE64 temperatures OK - attachment 23
ADC data item statistics OK - attachment 25
DSSSD bias & leakage current OK - attachment 27
analysis of file S496/R14_1920 - attachment 29
 aida04 deadtime 4.4%

07.35 Found MBS data monitor kill'd
      Restarted MBS data relay and data monitor OK

08.00 beam off - end of S496

08.28 Tape Server -> no storage mode
      DAQ continues

09.58 All slow comparators -> 0x64
      BNC PB-5 OFF

      file S496/R15

      alpha background

      Tape server 200kb/s, merger 260k data items/s
Attachment 1: Screenshot_2021-05-22_Lost_activity_monitor_aidas-gsi(1).png  177 kB  Uploaded Sat May 22 01:15:01 2021  | Hide | Hide all | Show all
Screenshot_2021-05-22_Lost_activity_monitor_aidas-gsi(1).png
Attachment 2: Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(7).png  173 kB  Uploaded Sat May 22 01:15:01 2021  | Hide | Hide all | Show all
Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(7).png
Attachment 3: Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(6).png  187 kB  Uploaded Sat May 22 01:15:01 2021  | Hide | Hide all | Show all
Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(6).png
Attachment 4: Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(5).png  173 kB  Uploaded Sat May 22 01:15:01 2021  | Hide | Hide all | Show all
Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(5).png
Attachment 5: Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(3).png  204 kB  Uploaded Sat May 22 01:15:01 2021  | Hide | Hide all | Show all
Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(3).png
Attachment 6: Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(2).png  222 kB  Uploaded Sat May 22 01:15:01 2021  | Hide | Hide all | Show all
Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(2).png
Attachment 7: Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(1).png  216 kB  Uploaded Sat May 22 01:15:01 2021  | Hide | Hide all | Show all
Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(1).png
Attachment 8: Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi.png  214 kB  Uploaded Sat May 22 01:15:01 2021  | Hide | Hide all | Show all
Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi.png
Attachment 9: Screenshot_2021-05-22_Lost_activity_monitor_aidas-gsi.png  158 kB  Uploaded Sat May 22 01:15:01 2021  | Hide | Hide all | Show all
Screenshot_2021-05-22_Lost_activity_monitor_aidas-gsi.png
Attachment 10: Screenshot_2021-05-22_ucesb.png  248 kB  Uploaded Sat May 22 01:15:01 2021  | Hide | Hide all | Show all
Screenshot_2021-05-22_ucesb.png
Attachment 11: Screenshot_2021-05-22_AIDA_Alerting_-_Grafana.png  352 kB  Uploaded Sat May 22 01:15:01 2021  | Hide | Hide all | Show all
Screenshot_2021-05-22_AIDA_Alerting_-_Grafana.png
Attachment 12: Screenshot_2021-05-22_Statistics_aidas-gsi(5).png  78 kB  Uploaded Sat May 22 01:15:01 2021  | Hide | Hide all | Show all
Screenshot_2021-05-22_Statistics_aidas-gsi(5).png
Attachment 13: Screenshot_2021-05-22_Statistics_aidas-gsi(4).png  83 kB  Uploaded Sat May 22 01:15:01 2021  | Hide | Hide all | Show all
Screenshot_2021-05-22_Statistics_aidas-gsi(4).png
Attachment 14: Screenshot_2021-05-22_Statistics_aidas-gsi(3).png  79 kB  Uploaded Sat May 22 01:15:01 2021  | Hide | Hide all | Show all
Screenshot_2021-05-22_Statistics_aidas-gsi(3).png
Attachment 15: Screenshot_2021-05-22_Statistics_aidas-gsi(2).png  89 kB  Uploaded Sat May 22 01:15:01 2021  | Hide | Hide all | Show all
Screenshot_2021-05-22_Statistics_aidas-gsi(2).png
Attachment 16: Screenshot_2021-05-22_Statistics_aidas-gsi(1).png  77 kB  Uploaded Sat May 22 01:15:01 2021  | Hide | Hide all | Show all
Screenshot_2021-05-22_Statistics_aidas-gsi(1).png
Attachment 17: Screenshot_2021-05-22_Statistics_aidas-gsi.png  93 kB  Uploaded Sat May 22 01:15:01 2021  | Hide | Hide all | Show all
Screenshot_2021-05-22_Statistics_aidas-gsi.png
Attachment 18: Screenshot_2021-05-22_Temperature_and_status_scan_aidas-gsi.png  130 kB  Uploaded Sat May 22 01:15:01 2021  | Hide | Hide all | Show all
Screenshot_2021-05-22_Temperature_and_status_scan_aidas-gsi.png
Attachment 19: 800.png  11 kB  Uploaded Sat May 22 01:15:37 2021  | Hide | Hide all | Show all
800.png
Attachment 20: 801.png  546 kB  Uploaded Sat May 22 01:15:37 2021  | Hide | Hide all | Show all
801.png
Attachment 21: R14_1798  11 kB  Uploaded Sat May 22 01:16:27 2021  | Show | Hide all | Show all
Attachment 22: Screenshot_2021-05-22_Statistics_aidas-gsi(7).png  94 kB  Uploaded Sat May 22 07:15:31 2021  | Hide | Hide all | Show all
Screenshot_2021-05-22_Statistics_aidas-gsi(7).png
Attachment 23: Screenshot_2021-05-22_Temperature_and_status_scan_aidas-gsi(2).png  131 kB  Uploaded Sat May 22 07:15:31 2021  | Hide | Hide all | Show all
Screenshot_2021-05-22_Temperature_and_status_scan_aidas-gsi(2).png
Attachment 24: Screenshot_2021-05-22_Statistics_aidas-gsi(6).png  93 kB  Uploaded Sat May 22 07:15:31 2021  | Hide | Hide all | Show all
Screenshot_2021-05-22_Statistics_aidas-gsi(6).png
Attachment 25: Screenshot_2021-05-22_Temperature_and_status_scan_aidas-gsi(1).png  130 kB  Uploaded Sat May 22 07:15:31 2021  | Hide | Hide all | Show all
Screenshot_2021-05-22_Temperature_and_status_scan_aidas-gsi(1).png
Attachment 26: 802.png  11 kB  Uploaded Sat May 22 07:16:05 2021  | Hide | Hide all | Show all
802.png
Attachment 27: 804.png  11 kB  Uploaded Sat May 22 07:16:05 2021  | Hide | Hide all | Show all
804.png
Attachment 28: R14_1867  11 kB  Uploaded Sat May 22 07:17:10 2021  | Show | Hide all | Show all
Attachment 29: R14_1920  11 kB  Uploaded Sat May 22 07:17:10 2021  | Show | Hide all | Show all
ELOG V3.1.4-unknown