ELOG DESPEC

AIDA GELINA BRIKEN nToF CRIB ISOLDE CIRCE nTOFCapture DESPEC DTAS EDI_PSA 179Ta CARME StellarModelling DCF K40

DESPEC

Fields marked with * are required

Entry time:

Sat May 22 13:40:56 2021

Author*:

Subject*:

14.37 DAQ continues file S496/R15_14

Grafana - DSSSD bias & leakage current - attachment 1
      1.8.L spectra - attachments 2 & 3
      Stat spectra - attachments 4 & 5
      FEE64 temperatures OK - attachment 6
      stats - ADC data items, pause items OK - attachments 7 & 8
      DSSSD bias & leakage current OK - attachment 9
      Merger/Tape Server/Merger stats - attachment 10
      analysis of file S496/R14_1920 - attachment 11
       all FEE64 deadtime 0% as expected

17.30 aida01, 3, 5, 9 & 11 have hot HEC channels
       ASIC check fixes
      All histogram, statistics & merger statistics zero'd

DAQ continues
      Note gamma-ray source tests have started with FATIMA and DEGAS

20.20 Analysis file R15_19 - attachment 12

[npg@aidas-gsi S496]$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_aidasgsi-lv_root
                       50G   18G   30G  37% /
tmpfs                 7.9G  665M  7.2G   9% /dev/shm
/dev/sda2             477M   42M  410M  10% /boot
/dev/sda1             200M  264K  200M   1% /boot/efi
/dev/mapper/vg_aidasgsi-lv_home
                      401G   90G  292G  24% /home
/dev/mapper/vg_aidas2-lv_home
                      7.2T  6.3T  530G  93% /media/1e121361-83d3-4825-b6ae-8700b07e0ca7
/dev/sdd1             7.2T  5.6T  1.3T  82% /media/SecondDrive
/dev/sdc1             7.2T  6.6T  259G  97% /media/ThirdDrive

estimate 90/2 x 3168s = 40h to filesystem full - OK

20.22 All system wide checks OK *except*

Base 		Current 	Difference
aida01 fault 	 0xf82c : 	 0xf82f : 	 3  
aida02 fault 	 0xdda6 : 	 0xdda9 : 	 3  
aida03 fault 	 0xb13d : 	 0xb140 : 	 3  
aida04 fault 	 0xf4cd : 	 0xf4d0 : 	 3  
aida05 fault 	 0x500 : 	 0x575 : 	 117  
aida06 fault 	 0xc6e7 : 	 0xc6e9 : 	 2  
aida07 fault 	 0x4920 : 	 0x4922 : 	 2  
aida08 fault 	 0xbd9a : 	 0xbd9c : 	 2  
White Rabbit error counter test result: Passed 8, Failed 8

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0  
Mem(KB) :	4	8	16	32	64	128	256	512	1k	2k	4k
aida01 :      8      4      6      6      1      4      3      3      4      3      6   : 38048
aida02 :     11      8     10      3      4      4      3      4      3      3      6   : 37740
aida03 :      2      8      3      3      2      2      1      4      2      4      6   : 37720
aida04 :     26     15     17      4      4      2      1      4      2      4      6   : 38256
aida05 :     15      6      4      1      1      2      1      3      3      4      6   : 38156
aida06 :      7      7     16      6      3      4      2      2      3      4      6   : 38612
aida07 :     11      5     12      3      3      3      2      3      1      3      7   : 38836
aida08 :     13      6      7      2      1      2      2      2      3      4      6   : 37972
aida09 :     23      3      8      5      0      5      3      3      4      3      6   : 38164
aida10 :     12      5      7      1      2      2      4      3      2      4      6   : 37992
aida11 :     19      2      7      1      2      2      1      2      3      4      6   : 37740
aida12 :     21      5      3      3      3      3      1      2      3      4      6   : 37964
aida13 :     18      7      7      1      2      4      2      3      2      4      6   : 37776
aida14 :      8      3     13      7      2      4      2      3      2      4      6   : 37992
aida15 :     11      5     14      1      3      4      2      3      2      4      6   : 37908
aida16 :     17      6      6      2      2      3      1      2      3      4      6   : 37908

Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same

FEE : aida01 =>   Options file size is 1026 	 Last changed Mon May 17 11:43:04 CEST 2021
 FEE : aida02 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:43:46 CEST 2021
 FEE : aida03 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:43:50 CEST 2021
 FEE : aida04 =>   Options file size is 1025 	 Last changed Fri May 14 16:54:56 CEST 2021
 FEE : aida05 =>   Options file size is 1025 	 Last changed Mon May 17 06:25:41 CEST 2021
 FEE : aida06 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:43:59 CEST 2021
 FEE : aida07 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:02 CEST 2021
 FEE : aida08 =>   Options file size is 1025 	 Last changed Wed May 05 12:15:54 CEST 2021
 FEE : aida09 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:08 CEST 2021
 FEE : aida10 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida11 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida12 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida13 =>   Options file size is 1025 	 Last changed Fri May 07 19:40:34 CEST 2021
 FEE : aida14 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida15 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida16 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021

19.30 Lost activity monitors - attachments 13-15

20.54 1.8.W spectra 20us FSR attachments 16 & 23
      1.8.W spectra 200us FSR attachments 17 & 22
      1.8.W spectra 2ms FSR attachments 18 & 21
      1.8.W spectra 20ms FSR attachments 19 & 20

Encoding: HTML ELCode plain
Suppress Email notification Resubmit as new entry

Attachment 1:

Screenshot_2021-05-22_AIDA_Alerting_-_Grafana(1).png Original size: 2108x1310

Attachment 2:

Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(11).png Original size: 1908x1118

Attachment 3:

Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(10).png Original size: 1908x1122

Attachment 4:

Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(9).png Original size: 1908x1122

Attachment 5:

Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(8).png Original size: 1908x1122

Attachment 6:

Screenshot_2021-05-22_Statistics_aidas-gsi(9).png Original size: 1920x927

Attachment 7:

Screenshot_2021-05-22_Statistics_aidas-gsi(8).png Original size: 1920x927

Attachment 8:

Screenshot_2021-05-22_Temperature_and_status_scan_aidas-gsi(3).png Original size: 1920x927

Attachment 9:

805.png Original size: 502x344

Attachment 10:

806.png Original size: 3840x1080

Attachment 11:

R15_13

 *** TDR format 3.3.0 analyser - TD - May 2021
 *** ERROR: READ I/O error:       5002
                   blocks:      32000
          ADC data format:  102550190 (   10129.0 Hz)
        Other data format:  159369822 (   15741.1 Hz)
 Sample trace data format:          0 (       0.0 Hz)
         Undefined format:          0 (       0.0 Hz)
   Other data format type:      PAUSE:          0 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:      69714 (       6.9 Hz)
                              WR48-63:      69714 (       6.9 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:  159230394 (   15727.3 Hz)
                           Other info:          0 (       0.0 Hz)

   ADC data range bit set:          0 (       0.0 Hz)

                Timewarps:        ADC:          0 (       0.0 Hz)
                                PAUSE:          0 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:          0 (       0.0 Hz)
                              WR48-63:          0 (       0.0 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:          0 (       0.0 Hz)
                            Undefined:          0 (       0.0 Hz)
                         Sample trace:          0 (       0.0 Hz)

 *** Timestamp elapsed time:    10124.450 s
 FEE  elapsed dead time(s) elapsed idle time(s)
  0                0.000                0.000
  1                0.000                0.000
  2                0.000               67.951
  3                0.000                0.000
  4                0.000                0.000
  5                0.000             9971.491
  6                0.000                0.000
  7                0.000            10069.837
  8                0.000                0.000
  9                0.000                0.000
 10                0.000                0.000
 11                0.000                0.000
 12                0.000                0.000
 13                0.000                0.000
 14                0.000                0.000
 15                0.000                0.000
 16                0.000                0.000
 17                0.000                0.000
 18                0.000                0.000
 19                0.000                0.000
 20                0.000                0.000
 21                0.000                0.000
 22                0.000                0.000
 23                0.000                0.000
 24                0.000                0.000
 25                0.000                0.000
 26                0.000                0.000
 27                0.000                0.000
 28                0.000                0.000
 29                0.000                0.000
 30                0.000                0.000
 31                0.000                0.000
 32                0.000                0.000

 *** Statistics
 FEE  ADC Data Other Data     Sample  Undefined      Pause     Resume    SYNC100    WR48-63       Disc        MBS      Other   HEC Data
  0        213          0          0          0          0          0          0          0          0          0          0          0
  1       1159          2          0          0          0          0          1          1          0          0          0          0
  2        279   79660137          0          0          0          0      22470      22470          0   79615197          0          0
  3  101219504   79708933          0          0          0          0      46868      46868          0   79615197          0          0
  4        246          0          0          0          0          0          0          0          0          0          0          0
  5     588847        366          0          0          0          0        183        183          0          0          0          0
  6        280          0          0          0          0          0          0          0          0          0          0          0
  7     737827        382          0          0          0          0        191        191          0          0          0          0
  8        262          0          0          0          0          0          0          0          0          0          0          0
  9        221          0          0          0          0          0          0          0          0          0          0          0
 10        242          2          0          0          0          0          1          1          0          0          0          0
 11        210          0          0          0          0          0          0          0          0          0          0          0
 12        273          0          0          0          0          0          0          0          0          0          0          0
 13        193          0          0          0          0          0          0          0          0          0          0          0
 14        250          0          0          0          0          0          0          0          0          0          0          0
 15        184          0          0          0          0          0          0          0          0          0          0          0
 16          0          0          0          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0          0          0          0

 *** Timewarps
 FEE       ADC      Pause     Resume    SYNC100    WR48-63       Disc        MBS  Undefined    Samples
  0          0          0          0          0          0          0          0          0          0
  1          0          0          0          0          0          0          0          0          0
  2          0          0          0          0          0          0          0          0          0
  3          0          0          0          0          0          0          0          0          0
  4          0          0          0          0          0          0          0          0          0
  5          0          0          0          0          0          0          0          0          0
  6          0          0          0          0          0          0          0          0          0
  7          0          0          0          0          0          0          0          0          0
  8          0          0          0          0          0          0          0          0          0
  9          0          0          0          0          0          0          0          0          0
 10          0          0          0          0          0          0          0          0          0
 11          0          0          0          0          0          0          0          0          0
 12          0          0          0          0          0          0          0          0          0
 13          0          0          0          0          0          0          0          0          0
 14          0          0          0          0          0          0          0          0          0
 15          0          0          0          0          0          0          0          0          0
 16          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0

 *** Program elapsed time:   45.816s (  698.440 blocks/s,  43.652 Mb/s)

Attachment 12:

R15_19

 *** TDR format 3.3.0 analyser - TD - May 2021
 *** ERROR: READ I/O error:       5002
                   blocks:      32000
          ADC data format:   55871232 (   17633.8 Hz)
        Other data format:  206048770 (   65032.0 Hz)
 Sample trace data format:          0 (       0.0 Hz)
         Undefined format:          0 (       0.0 Hz)
   Other data format type:      PAUSE:         56 (       0.0 Hz)
                               RESUME:         56 (       0.0 Hz)
                              SYNC100:      43803 (      13.8 Hz)
                              WR48-63:      43803 (      13.8 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:  205961052 (   65004.3 Hz)
                           Other info:          0 (       0.0 Hz)

   ADC data range bit set:          0 (       0.0 Hz)

                Timewarps:        ADC:          0 (       0.0 Hz)
                                PAUSE:          0 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:          0 (       0.0 Hz)
                              WR48-63:          0 (       0.0 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:          0 (       0.0 Hz)
                            Undefined:          0 (       0.0 Hz)
                         Sample trace:          0 (       0.0 Hz)

 *** Timestamp elapsed time:     3168.420 s
 FEE  elapsed dead time(s) elapsed idle time(s)
  0                0.000                0.000
  1                0.000             2865.072
  2                0.000                0.000
  3                0.419                0.000
  4                0.000                0.000
  5                0.000             2166.184
  6                0.000                0.000
  7                0.000             1949.169
  8                0.000                0.000
  9                0.000                0.000
 10                0.000                0.000
 11                0.000                0.000
 12                0.000                0.000
 13                0.000                0.000
 14                0.000                0.000
 15                0.000                0.000
 16                0.000                0.000
 17                0.000                0.000
 18                0.000                0.000
 19                0.000                0.000
 20                0.000                0.000
 21                0.000                0.000
 22                0.000                0.000
 23                0.000                0.000
 24                0.000                0.000
 25                0.000                0.000
 26                0.000                0.000
 27                0.000                0.000
 28                0.000                0.000
 29                0.000                0.000
 30                0.000                0.000
 31                0.000                0.000
 32                0.000                0.000

 *** Statistics
 FEE  ADC Data Other Data     Sample  Undefined      Pause     Resume    SYNC100    WR48-63       Disc        MBS      Other   HEC Data
  0         55          0          0          0          0          0          0          0          0          0          0          0
  1     133932         56          0          0          0          0         28         28          0          0          0          0
  2         84  103025372          0          0          0          0      18982      18982          0  102987408          0          0
  3   42389344  103017790          0          0         56         56      22017      22017          0  102973644          0          0
  4         56          0          0          0          0          0          0          0          0          0          0          0
  5    6040914       2574          0          0          0          0       1287       1287          0          0          0          0
  6        103          0          0          0          0          0          0          0          0          0          0          0
  7    7306081       2976          0          0          0          0       1488       1488          0          0          0          0
  8         78          0          0          0          0          0          0          0          0          0          0          0
  9         86          0          0          0          0          0          0          0          0          0          0          0
 10         68          0          0          0          0          0          0          0          0          0          0          0
 11         75          0          0          0          0          0          0          0          0          0          0          0
 12        116          2          0          0          0          0          1          1          0          0          0          0
 13         80          0          0          0          0          0          0          0          0          0          0          0
 14         80          0          0          0          0          0          0          0          0          0          0          0
 15         80          0          0          0          0          0          0          0          0          0          0          0
 16          0          0          0          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0          0          0          0

 *** Timewarps
 FEE       ADC      Pause     Resume    SYNC100    WR48-63       Disc        MBS  Undefined    Samples
  0          0          0          0          0          0          0          0          0          0
  1          0          0          0          0          0          0          0          0          0
  2          0          0          0          0          0          0          0          0          0
  3          0          0          0          0          0          0          0          0          0
  4          0          0          0          0          0          0          0          0          0
  5          0          0          0          0          0          0          0          0          0
  6          0          0          0          0          0          0          0          0          0
  7          0          0          0          0          0          0          0          0          0
  8          0          0          0          0          0          0          0          0          0
  9          0          0          0          0          0          0          0          0          0
 10          0          0          0          0          0          0          0          0          0
 11          0          0          0          0          0          0          0          0          0
 12          0          0          0          0          0          0          0          0          0
 13          0          0          0          0          0          0          0          0          0
 14          0          0          0          0          0          0          0          0          0
 15          0          0          0          0          0          0          0          0          0
 16          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0

 *** Program elapsed time:   43.406s (  737.221 blocks/s,  46.076 Mb/s)

Attachment 13:

Screenshot_2021-05-22_Lost_activity_monitor_aidas-gsi(4).png Original size: 1920x927

Attachment 14:

Screenshot_2021-05-22_Lost_activity_monitor_aidas-gsi(3).png Original size: 1920x927

Attachment 15:

Screenshot_2021-05-22_Lost_activity_monitor_aidas-gsi(2).png Original size: 1920x927

Attachment 16:

Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(19).png Original size: 1908x1120

Attachment 17:

Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(18).png Original size: 1908x1120

Attachment 18:

Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(17).png Original size: 1908x1120

Attachment 19:

Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(16).png Original size: 1908x1120

Attachment 20:

Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(15).png Original size: 1908x1122

Attachment 21:

Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(14).png Original size: 1908x1122

Attachment 22:

Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(13).png Original size: 1908x1122

Attachment 23:

Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(12).png Original size: 1908x1122

Attachment 24:

Drop attachments here...

ELOG V3.1.4-unknown