AIDA GELINA BRIKEN nToF CRIB ISOLDE CIRCE nTOFCapture DESPEC DTAS EDI_PSA 179Ta CARME StellarModelling DCF K40
  DESPEC, Page 33 of 35  ELOG logo
ID Date Authordown Subject
  180   Mon Mar 8 09:45:20 2021 CA, TDAnalysis R7_20 (new version of NewMerger with min info code 4 & 5 data items)
An analysis of file R6_70 can be found at https://elog.ph.ed.ac.uk/DESPEC/177 attachment 19
We observe c. 80M ADC data items per 2Gb file and c. 160M info code 4 & 5 data items.

This am NH switched NewMerger for a new version which minimises the number of info code 4 & 5
data items. The analysis of one of these data files (R7_20) is appended - attachment 1. We
observe c. 2M ADC data items per 2Gb file. The number of info code 4 data items is significantly
reduced as expected but info code 5 data items are not observed.

Info code 4 & 5 data items will increment every (2^28 * 1e-9) 0.268s and (2^48 * 1e-9) 3.26d.

If we examine the verbose output (attachment 2) we observe that each data block only contains
c. 60 ADC data items - a small fraction of the capacity of each data block.  
Attachment 1: R7_20
 *** TDR format 3.3.0 analyser - TD - January 2019
 *** ERROR: READ I/O error:       5002
                   blocks:      32000
          ADC data format:    1822115 ( 2111176.3 Hz)
        Other data format:       1885 (    2184.0 Hz)
 Sample trace data format:          0 (       0.0 Hz)
         Undefined format:          0 (       0.0 Hz)
   Other data format type:      PAUSE:          8 (       9.3 Hz)
                               RESUME:         10 (      11.6 Hz)
                              SYNC100:          4 (       4.6 Hz)
                              WR48-63:          0 (       0.0 Hz)
                           FEE64 disc:       1857 (    2151.6 Hz)
                             MBS info:          6 (       7.0 Hz)
                           Other info:          0 (       0.0 Hz)

   ADC data range bit set:          0 (       0.0 Hz)

                Timewarps:        ADC:          0 (       0.0 Hz)
                                PAUSE:          0 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:          0 (       0.0 Hz)
                              WR48-63:          0 (       0.0 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:          0 (       0.0 Hz)
                            Undefined:          0 (       0.0 Hz)
                         Sample trace:          0 (       0.0 Hz)

 *** Timestamp elapsed time:        0.863 s
 FEE  elapsed dead time(s) elapsed idle time(s)
  1                0.000                0.000
  2                0.206                0.000
  3                0.000                0.000
  4                0.000                0.000
  5                0.060                0.000
  6                0.000                0.000
  7                0.000                0.000
  8                0.201                0.000
  9                0.166                0.805
 10                0.000                0.000
 11                0.592                0.000
 12                0.000                0.000
 13                0.000                0.000
 14                0.000                0.000
 15                0.000                0.000
 16                0.000                0.000
 17                0.000                0.000
 18                0.000                0.000
 19                0.000                0.000
 20                0.000                0.000
 21                0.000                0.000
 22                0.000                0.000
 23                0.000                0.000
 24                0.000                0.000
 25                0.000                0.000
 26                0.000                0.000
 27                0.000                0.000
 28                0.000                0.000
 29                0.000                0.000
 30                0.000                0.000
 31                0.000                0.000
 32                0.000                0.000

 *** Statistics
 FEE  ADC Data Other Data     Sample  Undefined      Pause     Resume    SYNC100    WR48-63       Disc        MBS      Other   HEC Data
  0     150071         10          0          0          2          2          0          0          0          6          0          0
  1     194013          1          0          0          0          0          1          0          0          0          0          0
  2     150373       1859          0          0          1          1          0          0       1857          0          0          0
  3     230996          0          0          0          0          0          0          0          0          0          0          0
  4      53452          0          0          0          0          0          0          0          0          0          0          0
  5     142131          5          0          0          2          3          0          0          0          0          0          0
  6     178856          0          0          0          0          0          0          0          0          0          0          0
  7      74144          0          0          0          0          0          0          0          0          0          0          0
  8     183174          3          0          0          1          2          0          0          0          0          0          0
  9     218137          5          0          0          1          1          3          0          0          0          0          0
 10      83482          0          0          0          0          0          0          0          0          0          0          0
 11     163286          2          0          0          1          1          0          0          0          0          0          0
 12          0          0          0          0          0          0          0          0          0          0          0          0
 13          0          0          0          0          0          0          0          0          0          0          0          0
 14          0          0          0          0          0          0          0          0          0          0          0          0
 15          0          0          0          0          0          0          0          0          0          0          0          0
 16          0          0          0          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0          0          0          0

 *** Timewarps
 FEE       ADC      Pause     Resume    SYNC100    WR48-63       Disc        MBS  Undefined    Samples
  0          0          0          0          0          0          0          0          0          0
  1          0          0          0          0          0          0          0          0          0
  2          0          0          0          0          0          0          0          0          0
  3          0          0          0          0          0          0          0          0          0
  4          0          0          0          0          0          0          0          0          0
  5          0          0          0          0          0          0          0          0          0
  6          0          0          0          0          0          0          0          0          0
  7          0          0          0          0          0          0          0          0          0
  8          0          0          0          0          0          0          0          0          0
  9          0          0          0          0          0          0          0          0          0
 10          0          0          0          0          0          0          0          0          0
 11          0          0          0          0          0          0          0          0          0
 12          0          0          0          0          0          0          0          0          0
 13          0          0          0          0          0          0          0          0          0
 14          0          0          0          0          0          0          0          0          0
 15          0          0          0          0          0          0          0          0          0
 16          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0

 *** Program elapsed time:   23.008s ( 1390.832 blocks/s,  86.927 Mb/s)
Attachment 2: R7_20_verbose
[npg@aidas-gsi analyser]$ ./analyser v  /TapeData/S452/R7_20 | more
 *** TDR format 3.3.0 analyser - TD - January 2019
  verbose
 ***            ADC data: block:     1 ptr:     7 data: 0xC1177F65 module:  4 fail: 0 range: 0 id:   279 channel: 23 adc: 32613  ts: 0x00000000078DEB0E
 ***            ADC data: block:     1 ptr:     9 data: 0xC0478143 module:  1 fail: 0 range: 0 id:    71 channel:  7 adc: 33091  ts: 0x00000000078DEB4A
 ***            ADC data: block:     1 ptr:    11 data: 0xC0D182FC module:  3 fail: 0 range: 0 id:   209 channel: 17 adc: 33532  ts: 0x00000000078DEB68
 ***            ADC data: block:     1 ptr:    13 data: 0xC2C081B9 module: 11 fail: 0 range: 0 id:   704 channel:  0 adc: 33209  ts: 0x00000000078DF068
 ***            ADC data: block:     1 ptr:    15 data: 0xC2D7820D module: 11 fail: 0 range: 0 id:   727 channel: 23 adc: 33293  ts: 0x00000000078DF068
 ***            ADC data: block:     1 ptr:    17 data: 0xC2E4835E module: 11 fail: 0 range: 0 id:   740 channel: 36 adc: 33630  ts: 0x00000000078DF068
 ***            ADC data: block:     1 ptr:    19 data: 0xC2F681F4 module: 11 fail: 0 range: 0 id:   758 channel: 54 adc: 33268  ts: 0x00000000078DF068
 ***            ADC data: block:     1 ptr:    21 data: 0xC00B7FDF module:  0 fail: 0 range: 0 id:    11 channel: 11 adc: 32735  ts: 0x00000000078DF1B2
 ***            ADC data: block:     1 ptr:    23 data: 0xC0147DDE module:  0 fail: 0 range: 0 id:    20 channel: 20 adc: 32222  ts: 0x00000000078DF1B2
 ***            ADC data: block:     1 ptr:    25 data: 0xC034803A module:  0 fail: 0 range: 0 id:    52 channel: 52 adc: 32826  ts: 0x00000000078DF1B2
 ***            ADC data: block:     1 ptr:    27 data: 0xC272814E module:  9 fail: 0 range: 0 id:   626 channel: 50 adc: 33102  ts: 0x00000000078DF2A2
 ***            ADC data: block:     1 ptr:    29 data: 0xC11B7E13 module:  4 fail: 0 range: 0 id:   283 channel: 27 adc: 32275  ts: 0x00000000078DF2DE
 ***            ADC data: block:     1 ptr:    31 data: 0xC1327D8E module:  4 fail: 0 range: 0 id:   306 channel: 50 adc: 32142  ts: 0x00000000078DF2DE
 ***            ADC data: block:     1 ptr:    33 data: 0xC05F7FB9 module:  1 fail: 0 range: 0 id:    95 channel: 31 adc: 32697  ts: 0x00000000078DF31A
 ***            ADC data: block:     1 ptr:    35 data: 0xC0C28087 module:  3 fail: 0 range: 0 id:   194 channel:  2 adc: 32903  ts: 0x00000000078DF338
 ***            ADC data: block:     1 ptr:    37 data: 0xC0DB8075 module:  3 fail: 0 range: 0 id:   219 channel: 27 adc: 32885  ts: 0x00000000078DF338
 ***            ADC data: block:     1 ptr:    39 data: 0xC1BA7ECE module:  6 fail: 0 range: 0 id:   442 channel: 58 adc: 32462  ts: 0x00000000078DF644
 ***            ADC data: block:     1 ptr:    41 data: 0xC2C381FB module: 11 fail: 0 range: 0 id:   707 channel:  3 adc: 33275  ts: 0x00000000078DF838
 ***            ADC data: block:     1 ptr:    43 data: 0xC2D981FD module: 11 fail: 0 range: 0 id:   729 channel: 25 adc: 33277  ts: 0x00000000078DF838
 ***            ADC data: block:     1 ptr:    45 data: 0xC2E682A5 module: 11 fail: 0 range: 0 id:   742 channel: 38 adc: 33445  ts: 0x00000000078DF838
 ***            ADC data: block:     1 ptr:    47 data: 0xC2F782E0 module: 11 fail: 0 range: 0 id:   759 channel: 55 adc: 33504  ts: 0x00000000078DF838
 ***            ADC data: block:     1 ptr:    49 data: 0xC0177CE4 module:  0 fail: 0 range: 0 id:    23 channel: 23 adc: 31972  ts: 0x00000000078DF982
 ***            ADC data: block:     1 ptr:    51 data: 0xC27A7F8E module:  9 fail: 0 range: 0 id:   634 channel: 58 adc: 32654  ts: 0x00000000078DFA72
 ***            ADC data: block:     1 ptr:    53 data: 0xC0C3816A module:  3 fail: 0 range: 0 id:   195 channel:  3 adc: 33130  ts: 0x00000000078DFB08
 ***            ADC data: block:     1 ptr:    55 data: 0xC0FD81FD module:  3 fail: 0 range: 0 id:   253 channel: 61 adc: 33277  ts: 0x00000000078DFB08
 ***            ADC data: block:     1 ptr:    57 data: 0xC0847F9A module:  2 fail: 0 range: 0 id:   132 channel:  4 adc: 32666  ts: 0x00000000078DFCB6
 ***            ADC data: block:     1 ptr:    59 data: 0xC0B77EED module:  2 fail: 0 range: 0 id:   183 channel: 55 adc: 32493  ts: 0x00000000078DFCB6
 ***            ADC data: block:     1 ptr:    61 data: 0xC1AD8002 module:  6 fail: 0 range: 0 id:   429 channel: 45 adc: 32770  ts: 0x00000000078DFE14
 ***            ADC data: block:     1 ptr:    63 data: 0xC2C4837A module: 11 fail: 0 range: 0 id:   708 channel:  4 adc: 33658  ts: 0x00000000078E0008
 ***            ADC data: block:     1 ptr:    65 data: 0xC2DC8250 module: 11 fail: 0 range: 0 id:   732 channel: 28 adc: 33360  ts: 0x00000000078E0008
 ***            ADC data: block:     1 ptr:    67 data: 0xC2EA8214 module: 11 fail: 0 range: 0 id:   746 channel: 42 adc: 33300  ts: 0x00000000078E0008
 ***            ADC data: block:     1 ptr:    69 data: 0xC27B812C module:  9 fail: 0 range: 0 id:   635 channel: 59 adc: 33068  ts: 0x00000000078E0242
 ***            ADC data: block:     1 ptr:    71 data: 0xC0C5811F module:  3 fail: 0 range: 0 id:   197 channel:  5 adc: 33055  ts: 0x00000000078E02D8
 ***            ADC data: block:     1 ptr:    73 data: 0xC0877F4D module:  2 fail: 0 range: 0 id:   135 channel:  7 adc: 32589  ts: 0x00000000078E0486
 ***            ADC data: block:     1 ptr:    75 data: 0xC2C782D6 module: 11 fail: 0 range: 0 id:   711 channel:  7 adc: 33494  ts: 0x00000000078E07D8
 ***            ADC data: block:     1 ptr:    77 data: 0xC2DE8261 module: 11 fail: 0 range: 0 id:   734 channel: 30 adc: 33377  ts: 0x00000000078E07D8
 ***            ADC data: block:     1 ptr:    79 data: 0xC2EB8115 module: 11 fail: 0 range: 0 id:   747 channel: 43 adc: 33045  ts: 0x00000000078E07D8
 ***            ADC data: block:     1 ptr:    81 data: 0xC0027C33 module:  0 fail: 0 range: 0 id:     2 channel:  2 adc: 31795  ts: 0x00000000078E0922
 ***            ADC data: block:     1 ptr:    83 data: 0xC0C68218 module:  3 fail: 0 range: 0 id:   198 channel:  6 adc: 33304  ts: 0x00000000078E0AA8
 ***            ADC data: block:     1 ptr:    85 data: 0xC0D58287 module:  3 fail: 0 range: 0 id:   213 channel: 21 adc: 33415  ts: 0x00000000078E0AA8
 ***            ADC data: block:     1 ptr:    87 data: 0xC08C4EFE module:  2 fail: 0 range: 0 id:   140 channel: 12 adc: 20222  ts: 0x00000000078E0C56
 ***            ADC data: block:     1 ptr:    89 data: 0xC2F48203 module: 11 fail: 0 range: 0 id:   756 channel: 52 adc: 33283  ts: 0x00000000078E0FA8
 ***            ADC data: block:     1 ptr:    91 data: 0xC267809D module:  9 fail: 0 range: 0 id:   615 channel: 39 adc: 32925  ts: 0x00000000078E11E2
 ***            ADC data: block:     1 ptr:    93 data: 0xC044810C module:  1 fail: 0 range: 0 id:    68 channel:  4 adc: 33036  ts: 0x00000000078E125A
 ***            ADC data: block:     1 ptr:    95 data: 0xC0C9815B module:  3 fail: 0 range: 0 id:   201 channel:  9 adc: 33115  ts: 0x00000000078E1278
 ***            ADC data: block:     1 ptr:    97 data: 0xC0DA8174 module:  3 fail: 0 range: 0 id:   218 channel: 26 adc: 33140  ts: 0x00000000078E1278
 ***            ADC data: block:     1 ptr:    99 data: 0xC1827F8D module:  6 fail: 0 range: 0 id:   386 channel:  2 adc: 32653  ts: 0x00000000078E1584
 ***            ADC data: block:     1 ptr:   101 data: 0xC1A77E56 module:  6 fail: 0 range: 0 id:   423 channel: 39 adc: 32342  ts: 0x00000000078E1584
 ***            ADC data: block:     1 ptr:   103 data: 0xC2FD846E module: 11 fail: 0 range: 0 id:   765 channel: 61 adc: 33902  ts: 0x00000000078E1778
 ***            ADC data: block:     1 ptr:   105 data: 0xC27081A7 module:  9 fail: 0 range: 0 id:   624 channel: 48 adc: 33191  ts: 0x00000000078E19B2
 ***            ADC data: block:     1 ptr:   107 data: 0xC0CA8171 module:  3 fail: 0 range: 0 id:   202 channel: 10 adc: 33137  ts: 0x00000000078E1A48
 ***            ADC data: block:     1 ptr:   109 data: 0xC0DF80F0 module:  3 fail: 0 range: 0 id:   223 channel: 31 adc: 33008  ts: 0x00000000078E1A48
 ***            ADC data: block:     1 ptr:   111 data: 0xC1F28069 module:  7 fail: 0 range: 0 id:   498 channel: 50 adc: 32873  ts: 0x00000000078E1A7A
 ***            ADC data: block:     1 ptr:   113 data: 0xC1837E89 module:  6 fail: 0 range: 0 id:   387 channel:  3 adc: 32393  ts: 0x00000000078E1D54
 ***            ADC data: block:     1 ptr:   115 data: 0xC1987EC6 module:  6 fail: 0 range: 0 id:   408 channel: 24 adc: 32454  ts: 0x00000000078E1D54
 ***            ADC data: block:     1 ptr:   117 data: 0xC1B37F93 module:  6 fail: 0 range: 0 id:   435 channel: 51 adc: 32659  ts: 0x00000000078E1D54
 ***            ADC data: block:     1 ptr:   119 data: 0xC2C28169 module: 11 fail: 0 range: 0 id:   706 channel:  2 adc: 33129  ts: 0x00000000078E1F48
 ***            ADC data: block:     2 ptr:     7 data: 0xC2D5830E module: 11 fail: 0 range: 0 id:   725 channel: 21 adc: 33550  ts: 0x00000000078E1F48
 ***            ADC data: block:     2 ptr:     9 data: 0xC2E18150 module: 11 fail: 0 range: 0 id:   737 channel: 33 adc: 33104  ts: 0x00000000078E1F48
 ***            ADC data: block:     2 ptr:    11 data: 0xC0117F5D module:  0 fail: 0 range: 0 id:    17 channel: 17 adc: 32605  ts: 0x00000000078E2092
 ***            ADC data: block:     2 ptr:    13 data: 0xC27E81BD module:  9 fail: 0 range: 0 id:   638 channel: 62 adc: 33213  ts: 0x00000000078E2182
 ***            ADC data: block:     2 ptr:    15 data: 0xC07D817F module:  1 fail: 0 range: 0 id:   125 channel: 61 adc: 33151  ts: 0x00000000078E21FA
 ***            ADC data: block:     2 ptr:    17 data: 0xC0CB8125 module:  3 fail: 0 range: 0 id:   203 channel: 11 adc: 33061  ts: 0x00000000078E2218
 ***            ADC data: block:     2 ptr:    19 data: 0xC1C28219 module:  7 fail: 0 range: 0 id:   450 channel:  2 adc: 33305  ts: 0x00000000078E224A
 ***            ADC data: block:     2 ptr:    21 data: 0xC1F77FD8 module:  7 fail: 0 range: 0 id:   503 channel: 55 adc: 32728  ts: 0x00000000078E224A
 ***            ADC data: block:     2 ptr:    23 data: 0xC08B7FBA module:  2 fail: 0 range: 0 id:   139 channel: 11 adc: 32698  ts: 0x00000000078E23C6
 ***            ADC data: block:     2 ptr:    25 data: 0xC09F7E74 module:  2 fail: 0 range: 0 id:   159 channel: 31 adc: 32372  ts: 0x00000000078E23C6
 ***            ADC data: block:     2 ptr:    27 data: 0xC1867FE8 module:  6 fail: 0 range: 0 id:   390 channel:  6 adc: 32744  ts: 0x00000000078E2524
 ***            ADC data: block:     2 ptr:    29 data: 0xC19A7EC3 module:  6 fail: 0 range: 0 id:   410 channel: 26 adc: 32451  ts: 0x00000000078E2524
 ***            ADC data: block:     2 ptr:    31 data: 0xC2C98098 module: 11 fail: 0 range: 0 id:   713 channel:  9 adc: 32920  ts: 0x00000000078E2718
 ***            ADC data: block:     2 ptr:    33 data: 0xC2DA837D module: 11 fail: 0 range: 0 id:   730 channel: 26 adc: 33661  ts: 0x00000000078E2718
 ***            ADC data: block:     2 ptr:    35 data: 0xC2E582C0 module: 11 fail: 0 range: 0 id:   741 channel: 37 adc: 33472  ts: 0x00000000078E2718
 ***            ADC data: block:     2 ptr:    37 data: 0xC0CC81D9 module:  3 fail: 0 range: 0 id:   204 channel: 12 adc: 33241  ts: 0x00000000078E29E8
 ***            ADC data: block:     2 ptr:    39 data: 0xC187804E module:  6 fail: 0 range: 0 id:   391 channel:  7 adc: 32846  ts: 0x00000000078E2CF4
 ***            ADC data: block:     2 ptr:    41 data: 0xC2DB81B9 module: 11 fail: 0 range: 0 id:   731 channel: 27 adc: 33209  ts: 0x00000000078E2EE8
 ***            ADC data: block:     2 ptr:    43 data: 0xC2E782E0 module: 11 fail: 0 range: 0 id:   743 channel: 39 adc: 33504  ts: 0x00000000078E2EE8
 ***            ADC data: block:     2 ptr:    45 data: 0xC0CD80DF module:  3 fail: 0 range: 0 id:   205 channel: 13 adc: 32991  ts: 0x00000000078E31B8
 ***            ADC data: block:     2 ptr:    47 data: 0xC0D380C6 module:  3 fail: 0 range: 0 id:   211 channel: 19 adc: 32966  ts: 0x00000000078E31B8
 ***            ADC data: block:     2 ptr:    49 data: 0xC1897EEF module:  6 fail: 0 range: 0 id:   393 channel:  9 adc: 32495  ts: 0x00000000078E34C4
 ***            ADC data: block:     2 ptr:    51 data: 0xC2E98401 module: 11 fail: 0 range: 0 id:   745 channel: 41 adc: 33793  ts: 0x00000000078E36B8
 ***            ADC data: block:     2 ptr:    53 data: 0xC0E08147 module:  3 fail: 0 range: 0 id:   224 channel: 32 adc: 33095  ts: 0x00000000078E3988
 ***            ADC data: block:     2 ptr:    55 data: 0xC2C881AC module: 11 fail: 0 range: 0 id:   712 channel:  8 adc: 33196  ts: 0x00000000078E3E88
 ***            ADC data: block:     2 ptr:    57 data: 0xC2ED8349 module: 11 fail: 0 range: 0 id:   749 channel: 45 adc: 33609  ts: 0x00000000078E3E88
 ***            ADC data: block:     2 ptr:    59 data: 0xC2F380C2 module: 11 fail: 0 range: 0 id:   755 channel: 51 adc: 32962  ts: 0x00000000078E3E88
 ***            ADC data: block:     2 ptr:    61 data: 0xC1BD7C77 module:  6 fail: 0 range: 0 id:   445 channel: 61 adc: 31863  ts: 0x00000000078E4464
 ***            ADC data: block:     2 ptr:    63 data: 0xC2D3837F module: 11 fail: 0 range: 0 id:   723 channel: 19 adc: 33663  ts: 0x00000000078E4658
 ***            ADC data: block:     2 ptr:    65 data: 0xC24F80FC module:  9 fail: 0 range: 0 id:   591 channel: 15 adc: 33020  ts: 0x00000000078E4892
 ***            ADC data: block:     2 ptr:    67 data: 0xC26C80FA module:  9 fail: 0 range: 0 id:   620 channel: 44 adc: 33018  ts: 0x00000000078E4892
 ***            ADC data: block:     2 ptr:    69 data: 0xC27782CE module:  9 fail: 0 range: 0 id:   631 channel: 55 adc: 33486  ts: 0x00000000078E4892
 ***            ADC data: block:     2 ptr:    71 data: 0xC0C08052 module:  3 fail: 0 range: 0 id:   192 channel:  0 adc: 32850  ts: 0x00000000078E4928
 ***            ADC data: block:     2 ptr:    73 data: 0xC08E4C1D module:  2 fail: 0 range: 0 id:   142 channel: 14 adc: 19485  ts: 0x00000000078E4AD6
 ***            ADC data: block:     2 ptr:    75 data: 0xC1817D36 module:  6 fail: 0 range: 0 id:   385 channel:  1 adc: 32054  ts: 0x00000000078E4C34
 ***            ADC data: block:     2 ptr:    77 data: 0xC1BF7FAF module:  6 fail: 0 range: 0 id:   447 channel: 63 adc: 32687  ts: 0x00000000078E4C34
 ***            ADC data: block:     2 ptr:    79 data: 0xC2E2826E module: 11 fail: 0 range: 0 id:   738 channel: 34 adc: 33390  ts: 0x00000000078E55F8
 ***            ADC data: block:     2 ptr:    81 data: 0xC1C380E1 module:  7 fail: 0 range: 0 id:   451 channel:  3 adc: 32993  ts: 0x00000000078E58FA
 ***            ADC data: block:     2 ptr:    83 data: 0xC0F78257 module:  3 fail: 0 range: 0 id:   247 channel: 55 adc: 33367  ts: 0x00000000078E6098
 ***            ADC data: block:     2 ptr:    85 data: 0xC2C681D1 module: 11 fail: 0 range: 0 id:   710 channel:  6 adc: 33233  ts: 0x00000000078E6598
 ***            ADC data: block:     2 ptr:    87 data: 0xC04E8051 module:  1 fail: 0 range: 0 id:    78 channel: 14 adc: 32849  ts: 0x00000000078E684A
 ***            ADC data: block:     2 ptr:    89 data: 0xC0758189 module:  1 fail: 0 range: 0 id:   117 channel: 53 adc: 33161  ts: 0x00000000078E684A
 ***            ADC data: block:     2 ptr:    91 data: 0xC0948066 module:  2 fail: 0 range: 0 id:   148 channel: 20 adc: 32870  ts: 0x00000000078E6A16
 ***            ADC data: block:     2 ptr:    93 data: 0xC0057DFD module:  0 fail: 0 range: 0 id:     5 channel:  5 adc: 32253  ts: 0x00000000078E6EB2
 ***            ADC data: block:     2 ptr:    95 data: 0xC2418102 module:  9 fail: 0 range: 0 id:   577 channel:  1 adc: 33026  ts: 0x00000000078E6FA2
 ***            ADC data: block:     2 ptr:    97 data: 0xC26B8030 module:  9 fail: 0 range: 0 id:   619 channel: 43 adc: 32816  ts: 0x00000000078E6FA2
 ***            ADC data: block:     2 ptr:    99 data: 0xC0077C68 module:  0 fail: 0 range: 0 id:     7 channel:  7 adc: 31848  ts: 0x00000000078E7682
 ***            ADC data: block:     2 ptr:   101 data: 0xC0107DBC module:  0 fail: 0 range: 0 id:    16 channel: 16 adc: 32188  ts: 0x00000000078E7682
 ***            ADC data: block:     2 ptr:   103 data: 0xC26F8145 module:  9 fail: 0 range: 0 id:   623 channel: 47 adc: 33093  ts: 0x00000000078E7772
 ***            ADC data: block:     2 ptr:   105 data: 0xC0D18120 module:  3 fail: 0 range: 0 id:   209 channel: 17 adc: 33056  ts: 0x00000000078E7808
 ***            ADC data: block:     2 ptr:   107 data: 0xC2C081F4 module: 11 fail: 0 range: 0 id:   704 channel:  0 adc: 33268  ts: 0x00000000078E7D08
 ***            ADC data: block:     2 ptr:   109 data: 0xC2D78240 module: 11 fail: 0 range: 0 id:   727 channel: 23 adc: 33344  ts: 0x00000000078E7D08
 ***            ADC data: block:     2 ptr:   111 data: 0xC2E48241 module: 11 fail: 0 range: 0 id:   740 channel: 36 adc: 33345  ts: 0x00000000078E7D08
 ***            ADC data: block:     2 ptr:   113 data: 0xC0137D35 module:  0 fail: 0 range: 0 id:    19 channel: 19 adc: 32053  ts: 0x00000000078E7E52
 ***            ADC data: block:     2 ptr:   115 data: 0xC2728164 module:  9 fail: 0 range: 0 id:   626 channel: 50 adc: 33124  ts: 0x00000000078E7F42
 ***            ADC data: block:     2 ptr:   117 data: 0xC0C18052 module:  3 fail: 0 range: 0 id:   193 channel:  1 adc: 32850  ts: 0x00000000078E7FD8
 ***            ADC data: block:     2 ptr:   119 data: 0xC2C38244 module: 11 fail: 0 range: 0 id:   707 channel:  3 adc: 33348  ts: 0x00000000078E84D8
 ***            ADC data: block:     3 ptr:     7 data: 0xC2D982DC module: 11 fail: 0 range: 0 id:   729 channel: 25 adc: 33500  ts: 0x00000000078E84D8
 ***            ADC data: block:     3 ptr:     9 data: 0xC2E683C3 module: 11 fail: 0 range: 0 id:   742 channel: 38 adc: 33731  ts: 0x00000000078E84D8
 ***            ADC data: block:     3 ptr:    11 data: 0xC2F68233 module: 11 fail: 0 range: 0 id:   758 channel: 54 adc: 33331  ts: 0x00000000078E84D8
 ***            ADC data: block:     3 ptr:    13 data: 0xC0177D5C module:  0 fail: 0 range: 0 id:    23 channel: 23 adc: 32092  ts: 0x00000000078E8622
 ***            ADC data: block:     3 ptr:    15 data: 0xC0C2803E module:  3 fail: 0 range: 0 id:   194 channel:  2 adc: 32830  ts: 0x00000000078E87A8
 ***            ADC data: block:     3 ptr:    17 data: 0xC0FD81FC module:  3 fail: 0 range: 0 id:   253 channel: 61 adc: 33276  ts: 0x00000000078E87A8
 ***            ADC data: block:     3 ptr:    19 data: 0xC1AD7F36 module:  6 fail: 0 range: 0 id:   429 channel: 45 adc: 32566  ts: 0x00000000078E8AB4
 ***            ADC data: block:     3 ptr:    21 data: 0xC2F78642 module: 11 fail: 0 range: 0 id:   759 channel: 55 adc: 34370  ts: 0x00000000078E8CA8
 ***            ADC data: block:     3 ptr:    23 data: 0xC0C3815B module:  3 fail: 0 range: 0 id:   195 channel:  3 adc: 33115  ts: 0x00000000078E8F78
 ***            ADC data: block:     3 ptr:    25 data: 0xC0847EE9 module:  2 fail: 0 range: 0 id:   132 channel:  4 adc: 32489  ts: 0x00000000078E9126
 ***            ADC data: block:     3 ptr:    27 data: 0xC0B77F35 module:  2 fail: 0 range: 0 id:   183 channel: 55 adc: 32565  ts: 0x00000000078E9126
 ***            ADC data: block:     3 ptr:    29 data: 0xC27B8125 module:  9 fail: 0 range: 0 id:   635 channel: 59 adc: 33061  ts: 0x00000000078E96B2
 ***            ADC data: block:     3 ptr:    31 data: 0xC08C3881 module:  2 fail: 0 range: 0 id:   140 channel: 12 adc: 14465  ts: 0x00000000078E98F6
 ***            ADC data: block:     3 ptr:    33 data: 0xC2C181D1 module: 11 fail: 0 range: 0 id:   705 channel:  1 adc: 33233  ts: 0x00000000078E9C48
 ***            ADC data: block:     3 ptr:    35 data: 0xC2D480BF module: 11 fail: 0 range: 0 id:   724 channel: 20 adc: 32959  ts: 0x00000000078E9C48
 ***            ADC data: block:     3 ptr:    37 data: 0xC2648191 module:  9 fail: 0 range: 0 id:   612 channel: 36 adc: 33169  ts: 0x00000000078E9E82
 ***            ADC data: block:     3 ptr:    39 data: 0xC04780C3 module:  1 fail: 0 range: 0 id:    71 channel:  7 adc: 32963  ts: 0x00000000078E9EFA
 ***            ADC data: block:     3 ptr:    41 data: 0xC0E6809B module:  3 fail: 0 range: 0 id:   230 channel: 38 adc: 32923  ts: 0x00000000078E9F18
 ***            ADC data: block:     3 ptr:    43 data: 0xC2C4834F module: 11 fail: 0 range: 0 id:   708 channel:  4 adc: 33615  ts: 0x00000000078EA418
 ***            ADC data: block:     3 ptr:    45 data: 0xC2DC8132 module: 11 fail: 0 range: 0 id:   732 channel: 28 adc: 33074  ts: 0x00000000078EA418
 ***            ADC data: block:     3 ptr:    47 data: 0xC2FD8325 module: 11 fail: 0 range: 0 id:   765 channel: 61 adc: 33573  ts: 0x00000000078EA418
 ***            ADC data: block:     3 ptr:    49 data: 0xC0C581D9 module:  3 fail: 0 range: 0 id:   197 channel:  5 adc: 33241  ts: 0x00000000078EA6E8
 ***            ADC data: block:     3 ptr:    51 data: 0xC2E183D2 module: 11 fail: 0 range: 0 id:   737 channel: 33 adc: 33746  ts: 0x00000000078EABE8
 ***            ADC data: block:     3 ptr:    53 data: 0xC0C9806C module:  3 fail: 0 range: 0 id:   201 channel:  9 adc: 32876  ts: 0x00000000078EAEB8
 ***            ADC data: block:     3 ptr:    55 data: 0xC0DA814C module:  3 fail: 0 range: 0 id:   218 channel: 26 adc: 33100  ts: 0x00000000078EAEB8
 ***            ADC data: block:     3 ptr:    57 data: 0xC1C28260 module:  7 fail: 0 range: 0 id:   450 channel:  2 adc: 33376  ts: 0x00000000078EAEEA
 ***            ADC data: block:     3 ptr:    59 data: 0xC08B7FEC module:  2 fail: 0 range: 0 id:   139 channel: 11 adc: 32748  ts: 0x00000000078EB066
 ***            ADC data: block:     3 ptr:    61 data: 0xC09F7EF1 module:  2 fail: 0 range: 0 id:   159 channel: 31 adc: 32497  ts: 0x00000000078EB066
 ***            ADC data: block:     3 ptr:    63 data: 0xC2668171 module:  9 fail: 0 range: 0 id:   614 channel: 38 adc: 33137  ts: 0x00000000078EB5F2
 ***            ADC data: block:     3 ptr:    65 data: 0xC0CA8094 module:  3 fail: 0 range: 0 id:   202 channel: 10 adc: 32916  ts: 0x00000000078EB688
 ***            ADC data: block:     3 ptr:    67 data: 0xC0DF806C module:  3 fail: 0 range: 0 id:   223 channel: 31 adc: 32876  ts: 0x00000000078EB688
 ***            ADC data: block:     3 ptr:    69 data: 0xC1F782AB module:  7 fail: 0 range: 0 id:   503 channel: 55 adc: 33451  ts: 0x00000000078EB6BA
 ***            ADC data: block:     3 ptr:    71 data: 0xC2C282CB module: 11 fail: 0 range: 0 id:   706 channel:  2 adc: 33483  ts: 0x00000000078EBB88
 ***            ADC data: block:     3 ptr:    73 data: 0xC07D8111 module:  1 fail: 0 range: 0 id:   125 channel: 61 adc: 33041  ts: 0x00000000078EBE3A
 ***            ADC data: block:     3 ptr:    75 data: 0xC0CB81B8 module:  3 fail: 0 range: 0 id:   203 channel: 11 adc: 33208  ts: 0x00000000078EBE58
 ***            ADC data: block:     3 ptr:    77 data: 0xC2E98262 module: 11 fail: 0 range: 0 id:   745 channel: 41 adc: 33378  ts: 0x00000000078EC358
 ***            ADC data: block:     3 ptr:    79 data: 0xC044807F module:  1 fail: 0 range: 0 id:    68 channel:  4 adc: 32895  ts: 0x00000000078EC60A
 ***            ADC data: block:     3 ptr:    81 data: 0xC19A7DA3 module:  6 fail: 0 range: 0 id:   410 channel: 26 adc: 32163  ts: 0x00000000078EC934
 ***            ADC data: block:     3 ptr:    83 data: 0xC2ED8207 module: 11 fail: 0 range: 0 id:   749 channel: 45 adc: 33287  ts: 0x00000000078ECB28
 ***            ADC data: block:     3 ptr:    85 data: 0xC2F380FC module: 11 fail: 0 range: 0 id:   755 channel: 51 adc: 33020  ts: 0x00000000078ECB28
 ***            ADC data: block:     3 ptr:    87 data: 0xC2708152 module:  9 fail: 0 range: 0 id:   624 channel: 48 adc: 33106  ts: 0x00000000078ECD62
 ***            ADC data: block:     3 ptr:    89 data: 0xC1B47F47 module:  6 fail: 0 range: 0 id:   436 channel: 52 adc: 32583  ts: 0x00000000078ED104
 ***            ADC data: block:     3 ptr:    91 data: 0xC2C88150 module: 11 fail: 0 range: 0 id:   712 channel:  8 adc: 33104  ts: 0x00000000078ED2F8
 ***            ADC data: block:     3 ptr:    93 data: 0xC27781B4 module:  9 fail: 0 range: 0 id:   631 channel: 55 adc: 33204  ts: 0x00000000078ED532
 ***            ADC data: block:     3 ptr:    95 data: 0xC0C081FF module:  3 fail: 0 range: 0 id:   192 channel:  0 adc: 33279  ts: 0x00000000078ED5C8
 ***            ADC data: block:     3 ptr:    97 data: 0xC1C780DD module:  7 fail: 0 range: 0 id:   455 channel:  7 adc: 32989  ts: 0x00000000078ED5FA
 ***            ADC data: block:     3 ptr:    99 data: 0xC1BD7D53 module:  6 fail: 0 range: 0 id:   445 channel: 61 adc: 32083  ts: 0x00000000078ED8D4
 ***            ADC data: block:     3 ptr:   101 data: 0xC2D38300 module: 11 fail: 0 range: 0 id:   723 channel: 19 adc: 33536  ts: 0x00000000078EDAC8
 ***            ADC data: block:     3 ptr:   103 data: 0xC0027F2E module:  0 fail: 0 range: 0 id:     2 channel:  2 adc: 32558  ts: 0x00000000078EDC12
 ***            ADC data: block:     3 ptr:   105 data: 0xC0117FD8 module:  0 fail: 0 range: 0 id:    17 channel: 17 adc: 32728  ts: 0x00000000078EDC12
 ***            ADC data: block:     3 ptr:   107 data: 0xC2588009 module:  9 fail: 0 range: 0 id:   600 channel: 24 adc: 32777  ts: 0x00000000078EDD02
 ***            ADC data: block:     3 ptr:   109 data: 0xC1CD810E module:  7 fail: 0 range: 0 id:   461 channel: 13 adc: 33038  ts: 0x00000000078EDDCA
 ***            ADC data: block:     3 ptr:   111 data: 0xC1927F0C module:  6 fail: 0 range: 0 id:   402 channel: 18 adc: 32524  ts: 0x00000000078EE0A4
 ***            ADC data: block:     3 ptr:   113 data: 0xC2DB8197 module: 11 fail: 0 range: 0 id:   731 channel: 27 adc: 33175  ts: 0x00000000078EE298
 ***            ADC data: block:     3 ptr:   115 data: 0xC2E7821D module: 11 fail: 0 range: 0 id:   743 channel: 39 adc: 33309  ts: 0x00000000078EE298
 ***            ADC data: block:     3 ptr:   117 data: 0xC0097F87 module:  0 fail: 0 range: 0 id:     9 channel:  9 adc: 32647  ts: 0x00000000078EE3E2
 ***            ADC data: block:     3 ptr:   119 data: 0xC2598015 module:  9 fail: 0 range: 0 id:   601 channel: 25 adc: 32789  ts: 0x00000000078EE4D2
 ***            ADC data: block:     4 ptr:     7 data: 0xC0E080A6 module:  3 fail: 0 range: 0 id:   224 channel: 32 adc: 32934  ts: 0x00000000078EE568
 ***            ADC data: block:     4 ptr:     9 data: 0xC08E4C7E module:  2 fail: 0 range: 0 id:   142 channel: 14 adc: 19582  ts: 0x00000000078EE716
 ***            ADC data: block:     4 ptr:    11 data: 0xC2E88140 module: 11 fail: 0 range: 0 id:   744 channel: 40 adc: 33088  ts: 0x00000000078EEA68
 ***            ADC data: block:     4 ptr:    13 data: 0xC279812E module:  9 fail: 0 range: 0 id:   633 channel: 57 adc: 33070  ts: 0x00000000078EECA2
 ***            ADC data: block:     4 ptr:    15 data: 0xC0438270 module:  1 fail: 0 range: 0 id:    67 channel:  3 adc: 33392  ts: 0x00000000078EED1A
 ***            ADC data: block:     4 ptr:    17 data: 0xC0D280BB module:  3 fail: 0 range: 0 id:   210 channel: 18 adc: 32955  ts: 0x00000000078EED38
 ***            ADC data: block:     4 ptr:    19 data: 0xC0F78176 module:  3 fail: 0 range: 0 id:   247 channel: 55 adc: 33142  ts: 0x00000000078EED38
 ***            ADC data: block:     4 ptr:    21 data: 0xC1817BAB module:  6 fail: 0 range: 0 id:   385 channel:  1 adc: 31659  ts: 0x00000000078EF044
 ***            ADC data: block:     4 ptr:    23 data: 0xC2F481FD module: 11 fail: 0 range: 0 id:   756 channel: 52 adc: 33277  ts: 0x00000000078EF238
 ***            ADC data: block:     4 ptr:    25 data: 0xC27A7E9E module:  9 fail: 0 range: 0 id:   634 channel: 58 adc: 32414  ts: 0x00000000078EF472
 ***            ADC data: block:     4 ptr:    27 data: 0xC04E80E8 module:  1 fail: 0 range: 0 id:    78 channel: 14 adc: 33000  ts: 0x00000000078EF4EA
 ***            ADC data: block:     4 ptr:    29 data: 0xC0D3809C module:  3 fail: 0 range: 0 id:   211 channel: 19 adc: 32924  ts: 0x00000000078EF508
 ***            ADC data: block:     4 ptr:    31 data: 0xC2C6810C module: 11 fail: 0 range: 0 id:   710 channel:  6 adc: 33036  ts: 0x00000000078EFA08
 ***            ADC data: block:     4 ptr:    33 data: 0xC257808A module:  9 fail: 0 range: 0 id:   599 channel: 23 adc: 32906  ts: 0x00000000078EFC42
 ***            ADC data: block:     4 ptr:    35 data: 0xC0107E1C module:  0 fail: 0 range: 0 id:    16 channel: 16 adc: 32284  ts: 0x00000000078F0322
 ***            ADC data: block:     4 ptr:    37 data: 0xC25A7FA5 module:  9 fail: 0 range: 0 id:   602 channel: 26 adc: 32677  ts: 0x00000000078F0412

[npg@aidas-gsi analyser]$ 

  311   Thu May 13 07:10:17 2021 CA, TDMay 13th 08:00 - 16:00 shift
08:10 all system wide checks ok *except*

      all FEE64 modules fail ADC calibration

      WR decoder status:

      Base 		Current 	Difference
aida01 fault 	 0xf932 : 	 0xf933 : 	 1  
aida02 fault 	 0x62ec : 	 0x62ed : 	 1  
aida03 fault 	 0x8679 : 	 0x867a : 	 1  
aida04 fault 	 0xf0e4 : 	 0xf0e5 : 	 1  
aida05 fault 	 0x9db8 : 	 0x9db9 : 	 1  
aida06 fault 	 0x7f18 : 	 0x7f19 : 	 1  
aida07 fault 	 0xdd2c : 	 0xdd2d : 	 1  
aida08 fault 	 0x1557 : 	 0x1558 : 	 1  
White Rabbit error counter test result: Passed 8, Failed 8

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

08:16 attempt to recalibrate ADCs using FADC align and control - unsuccessful on all

08:18 FEE64 temps ok - attachment 1

      bias/leakage currents ok - attachment 2

      statistics - attachments 3-8

10:13 all system wide checks ok *except*

      all FEE64 modules fail ADC calibration

      WR decoder status:

		 Base 		Current 	Difference
aida01 fault 	 0xf932 : 	 0xf933 : 	 1  
aida02 fault 	 0x62ec : 	 0x62ed : 	 1  
aida03 fault 	 0x8679 : 	 0x867a : 	 1  
aida04 fault 	 0xf0e4 : 	 0xf0e5 : 	 1  
aida05 fault 	 0x9db8 : 	 0x9db9 : 	 1  
aida06 fault 	 0x7f18 : 	 0x7f19 : 	 1  
aida07 fault 	 0xdd2c : 	 0xdd2d : 	 1  
aida08 fault 	 0x1557 : 	 0x1558 : 	 1  
White Rabbit error counter test result: Passed 8, Failed 8

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

10:16 FEE64 temps ok - attachment 9

      bias/leakage currents ok - attachment 10

      statistics - attachments 11-16

12:16 all system wide checks ok *except*

      all FEE64 modules fail ADC calibration

      WR decoder status:

		 Base 		Current 	Difference
aida01 fault 	 0xf932 : 	 0xf933 : 	 1  
aida02 fault 	 0x62ec : 	 0x62ed : 	 1  
aida03 fault 	 0x8679 : 	 0x867a : 	 1  
aida04 fault 	 0xf0e4 : 	 0xf0e5 : 	 1  
aida05 fault 	 0x9db8 : 	 0x9dba : 	 2  
aida06 fault 	 0x7f18 : 	 0x7f19 : 	 1  
aida07 fault 	 0xdd2c : 	 0xdd2d : 	 1  
aida08 fault 	 0x1557 : 	 0x1558 : 	 1  
White Rabbit error counter test result: Passed 8, Failed 8

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

12:16 FEE64 temps ok - attachment 17

      bias/leakage currents ok - attachment 18

      statistics - attachments 19-25

14:08 all system wide checks ok *except*

      all FEE64 modules fail ADC calibration

      WR decoder status:

		 Base 		Current 	Difference
aida01 fault 	 0xf932 : 	 0xf933 : 	 1  
aida02 fault 	 0x62ec : 	 0x62ed : 	 1  
aida03 fault 	 0x8679 : 	 0x867a : 	 1  
aida04 fault 	 0xf0e4 : 	 0xf0e5 : 	 1  
aida05 fault 	 0x9db8 : 	 0x9dba : 	 2  
aida06 fault 	 0x7f18 : 	 0x7f19 : 	 1  
aida07 fault 	 0xdd2c : 	 0xdd2d : 	 1  
aida08 fault 	 0x1557 : 	 0x1558 : 	 1  
White Rabbit error counter test result: Passed 8, Failed 8

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

FEE64 temps ok - attachment 16

      bias/leakage currents ok - attachment 27

      statistics - attachments 28-35
Attachment 1: Screenshot_from_2021-05-13_07-17-47.png
Screenshot_from_2021-05-13_07-17-47.png
Attachment 2: Screenshot_from_2021-05-13_07-18-57.png
Screenshot_from_2021-05-13_07-18-57.png
Attachment 3: Screenshot_from_2021-05-13_07-19-42.png
Screenshot_from_2021-05-13_07-19-42.png
Attachment 4: Screenshot_from_2021-05-13_07-20-45.png
Screenshot_from_2021-05-13_07-20-45.png
Attachment 5: Screenshot_from_2021-05-13_07-23-01.png
Screenshot_from_2021-05-13_07-23-01.png
Attachment 6: Screenshot_from_2021-05-13_07-23-35.png
Screenshot_from_2021-05-13_07-23-35.png
Attachment 7: Screenshot_from_2021-05-13_07-24-02.png
Screenshot_from_2021-05-13_07-24-02.png
Attachment 8: Screenshot_from_2021-05-13_07-25-15.png
Screenshot_from_2021-05-13_07-25-15.png
Attachment 9: Screenshot_from_2021-05-13_09-17-57.png
Screenshot_from_2021-05-13_09-17-57.png
Attachment 10: Screenshot_from_2021-05-13_09-18-21.png
Screenshot_from_2021-05-13_09-18-21.png
Attachment 11: Screenshot_from_2021-05-13_09-18-58.png
Screenshot_from_2021-05-13_09-18-58.png
Attachment 12: Screenshot_from_2021-05-13_09-19-32.png
Screenshot_from_2021-05-13_09-19-32.png
Attachment 13: Screenshot_from_2021-05-13_09-20-23.png
Screenshot_from_2021-05-13_09-20-23.png
Attachment 14: Screenshot_from_2021-05-13_09-21-05.png
Screenshot_from_2021-05-13_09-21-05.png
Attachment 15: Screenshot_from_2021-05-13_09-21-53.png
Screenshot_from_2021-05-13_09-21-53.png
Attachment 16: Screenshot_from_2021-05-13_09-22-18.png
Screenshot_from_2021-05-13_09-22-18.png
Attachment 17: Screenshot_from_2021-05-13_11-19-16.png
Screenshot_from_2021-05-13_11-19-16.png
Attachment 18: Screenshot_from_2021-05-13_11-19-30.png
Screenshot_from_2021-05-13_11-19-30.png
Attachment 19: Screenshot_from_2021-05-13_11-20-09.png
Screenshot_from_2021-05-13_11-20-09.png
Attachment 20: Screenshot_from_2021-05-13_11-20-33.png
Screenshot_from_2021-05-13_11-20-33.png
Attachment 21: Screenshot_from_2021-05-13_11-20-59.png
Screenshot_from_2021-05-13_11-20-59.png
Attachment 22: Screenshot_from_2021-05-13_11-21-29.png
Screenshot_from_2021-05-13_11-21-29.png
Attachment 23: Screenshot_from_2021-05-13_11-21-53.png
Screenshot_from_2021-05-13_11-21-53.png
Attachment 24: Screenshot_from_2021-05-13_11-22-30.png
Screenshot_from_2021-05-13_11-22-30.png
Attachment 25: Screenshot_from_2021-05-13_11-24-36.png
Screenshot_from_2021-05-13_11-24-36.png
Attachment 26: Screenshot_from_2021-05-13_13-09-43.png
Screenshot_from_2021-05-13_13-09-43.png
Attachment 27: Screenshot_from_2021-05-13_13-10-03.png
Screenshot_from_2021-05-13_13-10-03.png
Attachment 28: Screenshot_from_2021-05-13_13-10-28.png
Screenshot_from_2021-05-13_13-10-28.png
Attachment 29: Screenshot_from_2021-05-13_13-10-52.png
Screenshot_from_2021-05-13_13-10-52.png
Attachment 30: Screenshot_from_2021-05-13_13-11-22.png
Screenshot_from_2021-05-13_13-11-22.png
Attachment 31: Screenshot_from_2021-05-13_13-11-51.png
Screenshot_from_2021-05-13_13-11-51.png
Attachment 32: Screenshot_from_2021-05-13_13-12-24.png
Screenshot_from_2021-05-13_13-12-24.png
Attachment 33: Screenshot_from_2021-05-13_13-13-34.png
Screenshot_from_2021-05-13_13-13-34.png
Attachment 34: Screenshot_from_2021-05-13_13-14-16.png
Screenshot_from_2021-05-13_13-14-16.png
  49   Thu Apr 11 15:28:10 2019 CA, OH, CB, TDOffline analysis of 290319 files R13 and R16
X projection of high energy channel ADC hits from R13 (attachment 1)

XY hit pattern from R13 (attachment 2)
    -discontinuity observed
    -possible timing issues or issues with Event Clustering

X projection of high energy channel ADC hits from R16 (attachment 3)

XY hit pattern from R16 (attachment 4)
    -Beam was being diffused at this point, and tpcs were being calibrated.
    -Gap between Y strips ~ 23 -> 30

High Energy Ey vs Ex plots from R13 and R16 (attachments 5 and 6 respectively)
Attachment 1: R13highEnergyhitpattern.pdf
R13highEnergyhitpattern.pdf
Attachment 2: R13highEnergyXYhitpattern.pdf
R13highEnergyXYhitpattern.pdf
Attachment 3: R16highEnergyhitpattern.pdf
R16highEnergyhitpattern.pdf
Attachment 4: R16highEnergyXYhitpattern.pdf
R16highEnergyXYhitpattern.pdf
Attachment 5: R13HighEnergyExEy.pdf
R13HighEnergyExEy.pdf
Attachment 6: R16HighEnergyExEy.pdf
R16HighEnergyExEy.pdf
  150   Fri Mar 13 23:06:39 2020 CA, OHSaturday 14th March 00:00 - 08:00
ASIC settings 2019Oct31-13.24.23
     slow comparator 0xa

BNC PB-5 pulser
     amplitude 1.0V , attenuator x1
     frequency 2Hz
     decay time 1ms

00.00 DAQ momentarily set to NoStorage
      OH changes symbolic link such that AIDA is writing directly to mounted hard-drives ("SecondDrive" and "ThirdDrive")

00.02 DAQ now writing to file again - /media/SecondDrive/TapeData/S480/R12_0

00.30 all system wide checks ok

      FEE64 temperatures ok - attachment 1

      good event statistics ok - attachment 2

      detector bias and leakage currents ok - attachment 3
 
00:43 merger ok - writing at ~ 3 million items/s

      TapeServer ok - writing at ~ 27 Mb/s

      Data forwarding to MBS ok

02:14 all system wide checks ok

      FEE64 temperatures ok - attachment 4

      good event statistics ok - attachment 5

      detector bias and leakage currents ok - attachment 6

02:17 Merger ok  ~ 2.7 million items/s

      TapeServer ok  ~ 27 Mb/s

      Data forwarding to MBS ok

02:18 currently writing to file /media/SecondDrive/TapeData/S480/R12_110

02:57 attachments 7 & 8 - high energy channel spectra

      attachment 9 - hit rate spectra

04:21  FEE64 temperatures ok - attachment 10

      good event statistics ok - attachment 11

      detector bias and leakage currents ok - attachment 12

      system wide checks ok - except WR decoder status (see attachment 13)

04:26 check ASIC control

04:30 system functions -> select required function -> collect all WhiteRabbit status error counters for baseline
      WR decoder status check now ok

04:50 Merger ok  ~ 3 million items/s

      TapeServer ok  ~ 25 Mb/s

      Data forwarding to MBS ok

04:52 currently writing to file /media/SecondDrive/TapeData/S480/R12_231
 
06:11 all system wide checks ok

      FEE64 temperatures ok - attachment 14

      good event statistics ok - attachment 15

      detector bias and leakage currents ok - attachment 16


06:41 Merger ok  ~ 3.3 million items/s

      TapeServer ok  ~ 25 Mb/s

      Data forwarding to MBS ok

06:42 currently writing to file /media/SecondDrive/TapeData/S480/R12_315

07:25 hit-rate spectra - attachment 17

07:36 low energy channel spectra (1.8.L) - attachments 18 & 19



 


      
Attachment 1: 200314_0020_temp.png
200314_0020_temp.png
Attachment 2: 200314_0030_stats.png
200314_0030_stats.png
Attachment 3: 200314_0031_bias.png
200314_0031_bias.png
Attachment 4: 200314_0212_temp.png
200314_0212_temp.png
Attachment 5: 200314_0213_stats.png
200314_0213_stats.png
Attachment 6: 200314_0214_bias.png
200314_0214_bias.png
Attachment 7: even_he.png
even_he.png
Attachment 8: odd_he.png
odd_he.png
Attachment 9: 200314_0254_rates.png
200314_0254_rates.png
Attachment 10: 200314_0416_temp.png
200314_0416_temp.png
Attachment 11: 200314_0417_stats.png
200314_0417_stats.png
Attachment 12: 200314_0418_bias.png
200314_0418_bias.png
Attachment 13: 200314_WRerror.png
200314_WRerror.png
Attachment 14: 200314_0608_temp.png
200314_0608_temp.png
Attachment 15: 200314_0609_stats.png
200314_0609_stats.png
Attachment 16: 200314_0610_bias.png
200314_0610_bias.png
Attachment 17: 200314_0724_rates.png
200314_0724_rates.png
Attachment 18: 200314_0733_18l_odd.png
200314_0733_18l_odd.png
Attachment 19: 200314_0735_18l_even.png
200314_0735_18l_even.png
  173   Fri Mar 5 23:21:59 2021 CA, OHMarch 6th 2021 00:00 - 08:00 shift
00:23 206Hg setting - No beam in AIDA yet

      AIDA not writing to file

      Data being forwarded to MBS

      Fragments expected tonight (?)

00:45 GSI DAQ restart

      FEE64 Temps ok - attachment 1

      All system wide checks ok

01:00 Good event statistics - attachment 2

01:48 detector biases/ leakage currents - ok - attachment 3

02:10 beam on, plug removed

      beam delivered to AIDA

      rates spectra - attachment 4

      peaks at ~ 10 counts/channel in DSSD1 c.2000Hz

02:20 beam stop

02:26 beam restart - rate spectra - attachment 5

02:43 high energy channel spectra - attachment 6

      saved the high energy spectra to layout 2 in spectrum browser

03:00 beam stop

03:08 temperatures ok - attachment 7

      statistics ok - attachment 8

      bias/leakage currents ok - attachment 9

      NewMerger ok - ~4.5M data items merge rate

      Tape service ok - ~46MB/s, NoStorage

      Data forwarding to MBS ok

      all system wide checks ok

04:24 beam is back

      rate spectra - attachment 10

04:51 206Hg implanting in DSSD2

      Rate spectra - attachment 11

      HEC spectra - attachment 12

05:08 temperatures ok - attachment 13

      statistics ok - attachment 14

      detector bias/leakage currents ok - attachment 15

      All system wide checks ok

      NewMerger ok - ~4.5M data items merge rate

      Tape service ok - ~46MB/s, NoStorage

      Data forwarding to MBS ok 

05:45 beam stopped

05:50 beam back

06:45 beam running, but no implants in AIDA

      possible issues with beam delivery to S4

      NewMerger ok - ~4.5M data items merge rate

      Tape service ok - ~46MB/s, NoStorage

      Data forwarding to MBS ok

07:10 temperatures ok - attachment 16

      statistics ok - attachment 17

      detector bias/leakage currents ok - attachment 18

      all system wide checks OK














      

      

      
Attachment 1: Screenshot_from_2021-03-06_00-00-27.png
Screenshot_from_2021-03-06_00-00-27.png
Attachment 2: Screenshot_from_2021-03-06_00-05-18.png
Screenshot_from_2021-03-06_00-05-18.png
Attachment 3: Screenshot_from_2021-03-06_00-47-41.png
Screenshot_from_2021-03-06_00-47-41.png
Attachment 4: Screenshot_from_2021-03-06_01-17-12.png
Screenshot_from_2021-03-06_01-17-12.png
Attachment 5: Screenshot_from_2021-03-06_01-25-51.png
Screenshot_from_2021-03-06_01-25-51.png
Attachment 6: Screenshot_from_2021-03-06_01-42-58.png
Screenshot_from_2021-03-06_01-42-58.png
Attachment 7: Screenshot_from_2021-03-06_02-05-46.png
Screenshot_from_2021-03-06_02-05-46.png
Attachment 8: Screenshot_from_2021-03-06_02-06-32.png
Screenshot_from_2021-03-06_02-06-32.png
Attachment 9: Screenshot_from_2021-03-06_02-07-47.png
Screenshot_from_2021-03-06_02-07-47.png
Attachment 10: Screenshot_from_2021-03-06_03-24-28.png
Screenshot_from_2021-03-06_03-24-28.png
Attachment 11: Screenshot_from_2021-03-06_03-53-35.png
Screenshot_from_2021-03-06_03-53-35.png
Attachment 12: Screenshot_from_2021-03-06_03-54-17_-_1.png
Screenshot_from_2021-03-06_03-54-17_-_1.png
Attachment 13: Screenshot_from_2021-03-06_04-09-25.png
Screenshot_from_2021-03-06_04-09-25.png
Attachment 14: Screenshot_from_2021-03-06_04-10-07.png
Screenshot_from_2021-03-06_04-10-07.png
Attachment 15: Screenshot_from_2021-03-06_04-10-32.png
Screenshot_from_2021-03-06_04-10-32.png
Attachment 16: Screenshot_from_2021-03-06_06-07-19.png
Screenshot_from_2021-03-06_06-07-19.png
Attachment 17: Screenshot_from_2021-03-06_06-08-16.png
Screenshot_from_2021-03-06_06-08-16.png
Attachment 18: Screenshot_from_2021-03-06_06-08-44.png
Screenshot_from_2021-03-06_06-08-44.png
  39   Mon Apr 1 10:14:09 2019 CA, NH, TDFriday 29th March 2019
16:30 good event statistics ok, with 9,10,11,12 running faster (detector FEE64s) (attachment 1)

      FEE temperatures ok (attachment 2)

      detector bias/leakage currents ok (attachment 3)

      system wide checks ok *except* FEE64 Linux memory information check causes script error,
                                     WR decoder status returns errors

      DAQ & Merger running ok

      slow comparator threshold raised to 0x64

   
Attachment 1: 290319_stats.png
290319_stats.png
Attachment 2: 290319_temp.png
290319_temp.png
Attachment 3: 290913_bias.png
290913_bias.png
  40   Mon Apr 1 10:20:20 2019 CA, NH, TDMBS Relay startup
To start the MBS Data Relay, click on the top rightmost of the MIDAS icons (MBS relay, Data Relay => MBS) in
Desktop 3. (attachment 1)

This will bring up the Data Relay MBS terminal.

Ensure the path defined in the datarelaymbs is set to the correct directory (shown in attachment 2) 
Attachment 1: mbsrealaypic.png
mbsrealaypic.png
Attachment 2: mbscommand.png
mbscommand.png
  96   Wed Nov 13 09:10:37 2019 CA, NH, OH13th November 2019
10:10 moved HV cables from 11 and 12 to 10 and 9 - all detectors biased in same configuration

      lowered slow comparator threshold to 0xa
   
      DAQ start

      pulser on

      zero'd histograms

pulser peak widths:
                   FEE      width (ch)

                   aida01 - 130

                   aida02 - 168
             
                   aida03 - 83

                   aida04 - 83

                   aida05 - 200

                   aida06 - 148

                   aida07 - 250

                   aida08 - 81

                   aida09 - 134

                   aida10 - 190

                   aida11 - 333

                   aida12 - 149

For comparison, peak widths from yesterday's measurement:

                   pulser peak widths -   FEE         width(ch)
                                           1             128
                                           2             145
                                           3             84
                                           4             74
                                           5             207
                                           6             125
                                           7             172
                                           8             80
                                           9             193
                                           10            183
                                           11            238
                                           12            138

Attachments 1 & 2 : pulser peak spectra

Attachment 3 - good event statistics

Attachment 4 - detector bias/ leakage currents

                   
Attachment 1: 191113_pulserpeaksOdd.png
191113_pulserpeaksOdd.png
Attachment 2: 191113_pulserpeaksEven.png
191113_pulserpeaksEven.png
Attachment 3: 191113_goodevents.png
191113_goodevents.png
Attachment 4: 191113_bias.png
191113_bias.png
  232   Fri Apr 16 23:13:57 2021 CA, MA, BASaturday 16th 00:00 - 08:00
00:05 DESPEC tuning beam, changing spill length

00:21 DAQ continues ok
      Merger/TapeServer ok

00:25 still no beam in AIDA

00:30 statistics ok, all FEE64 showing rates

      no error messages in database check

      AIDA scalars on ucesb ok - attachment 1

00:52 beam in AIDA - rates spectra - attachment 2

01:05 statistics ok, all FEE64 showing rates

      no error messages in database check

      AIDA scalars on ucesb ok - attachment 3

      beam is off again

01:30 beam back

01:32 statistics ok, all FEE64 showing rates

      no error messages in database check

      AIDA scalars on ucesb ok - attachment 4

01:37 beam off - FRS increasing spill length to 5s

01:48 beam back with new spill setting - 5s on spill, 2s off spill

02:07 system wide checks ok except;

WR decoder status

      		 Base 		Current 	Difference
aida05 fault 	 0xc879 : 	 0xc87b : 	 2  
aida06 fault 	 0x323c : 	 0x323e : 	 2  
aida07 fault 	 0xfb3a : 	 0xfb42 : 	 8  
aida08 fault 	 0xd3d6 : 	 0xd3d8 : 	 2  
White Rabbit error counter test result: Passed 8, Failed 4

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

FPGA check :

			 Base 		Current 		Difference
aida07 fault 	 0xf : 	 0x10 : 	 1  
FPGA Timestamp error counter test result: Passed 11, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last

02:10 FEE64 Temps ok - attachment 5

02:20 good event statistics ok - attachment 6

02:27 detector bias / leakage currents ok - attachment 7

02:30 statistics ok, all FEE64 showing rates

      no error messages in database check

      AIDA scalars on ucesb ok

      beam off

03:00 stats/ucesb/database checks ok

03:30 stats/ucesb/database checks ok

      still no beam in AIDA

05:11 system is checked, beam is back with low intensity 

05:13 FEE64 Temps ok - attachment 8

05:13 good event statistics ok - attachment 9

05:13 detector bias / leakage currents ok - attachment 10

05:33 stats/ucesb/database checks ok

06:00 AIDA crashes, lose connection to aida09

06:20 aida08 does not respond on powercycle and reset

06:30 Called TD 

06:50 successfully reset AIDA. Writing to file R30

      synchronise ASIC clocks:
	
FEE64 module aida01  => 	7
FEE64 module aida02  => 	7
FEE64 module aida03  => 	7
FEE64 module aida04  => 	7
FEE64 module aida05  => 	7
FEE64 module aida06  => 	7
FEE64 module aida07  => 	7
FEE64 module aida08  => 	7
FEE64 module aida09  => 	7
FEE64 module aida10  => 	7
FEE64 module aida11  => 	7
FEE64 module aida12  => 	7


ASIC Clock Timestamp check test result: Passed 12, Failed 0

Database options file size check:


If any of the above values are different from the rest then check the clocks and the White Rabbit decoder status

 Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same

 FEE : aida01 =>   Options file size is 1025 	 Last changed Sat Apr 17 06:14:30 CEST 2021
 FEE : aida02 =>   Options file size is 1014 	 Last changed Fri Apr 16 00:56:20 CEST 2021
 FEE : aida03 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:04 CEST 2021
 FEE : aida04 =>   Options file size is 1025 	 Last changed Sat Apr 17 06:07:36 CEST 2021
 FEE : aida05 =>   Options file size is 1025 	 Last changed Fri Apr 16 00:53:25 CEST 2021
 FEE : aida06 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:04 CEST 2021
 FEE : aida07 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:04 CEST 2021
 FEE : aida08 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:04 CEST 2021
 FEE : aida09 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:05 CEST 2021
 FEE : aida10 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:06 CEST 2021
 FEE : aida11 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:05 CEST 2021
 FEE : aida12 =>   Options file size is 1025 	 Last changed Wed Apr 14 21:58:54 CEST 2021

System wide checks ok *except*

FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida10 failed
Calibration test result: Passed 9, Failed 3

If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module

re-calibration of these FEE modules unsuccessful in FADC align and control.

07:05 FEE64 Temps ok - attachment 12

      good event statistics ok - attachment 13

      detector bias / leakage currents ok - attachment 14

07:10 stats/ucesb/database checks ok

07:30 stats/ucesb/database checks ok


      
      
Attachment 1: Screenshot_from_2021-04-16_23-35-20.png
Screenshot_from_2021-04-16_23-35-20.png
Attachment 2: Screenshot_from_2021-04-16_23-52-09.png
Screenshot_from_2021-04-16_23-52-09.png
Attachment 3: Screenshot_from_2021-04-17_00-05-06.png
Screenshot_from_2021-04-17_00-05-06.png
Attachment 4: Screenshot_from_2021-04-17_00-31-46.png
Screenshot_from_2021-04-17_00-31-46.png
Attachment 5: Screenshot_from_2021-04-17_01-11-18.png
Screenshot_from_2021-04-17_01-11-18.png
Attachment 6: Screenshot_from_2021-04-17_01-12-26.png
Screenshot_from_2021-04-17_01-12-26.png
Attachment 7: Screenshot_from_2021-04-17_01-13-22.png
Screenshot_from_2021-04-17_01-13-22.png
Attachment 8: 20210417_0506.png
20210417_0506.png
Attachment 9: 20210417_0602_tem.png
20210417_0602_tem.png
Attachment 10: 20210417_0601.png
20210417_0601.png
Attachment 11: 20210417_0603_Bias.png
20210417_0603_Bias.png
Attachment 12: Screenshot_from_2021-04-17_06-03-52.png
Screenshot_from_2021-04-17_06-03-52.png
Attachment 13: Screenshot_from_2021-04-17_06-04-14.png
Screenshot_from_2021-04-17_06-04-14.png
Attachment 14: Screenshot_from_2021-04-17_06-04-37.png
Screenshot_from_2021-04-17_06-04-37.png
  141   Wed Mar 11 16:29:17 2020 CA, LS, TDslow comparator threshold comparison
17.29 good event statistics for slow comparator threshold at 100 keV (0xa) - attachment 1

17.36 good event statistics for slow comparator threshold at 110 keV (0xb) - attachment 2

17.39 good event statistics for slow comparator threshold at 120 keV (0xc) - attachment 3

17.53 good event statistics for slow comparator threshold at 80 keV (0x8) for DSSD2, and 100keV (0xa) for DSSD 1&3 - attachment 4
Attachment 1: 100keV_stats.png
100keV_stats.png
Attachment 2: 110keVStats.png
110keVStats.png
Attachment 3: 120kevstats.png
120kevstats.png
Attachment 4: 80kevDSSD2.png
80kevDSSD2.png
  153   Sun Mar 15 06:45:00 2020 CA, LS, CGSunday 15th March 08:00 - 24:00
ASIC settings 2019Oct31-13.24.23
     slow comparator 0xa

BNC PB-5 pulser
     amplitude 1.0V , attenuator x1
     frequency 2Hz
     decay time 1ms

07:46 all system wide checks ok

      good event statistics ok - attachment 1
  
      FEE64 temperatures ok - attachment 2

      detector bias/leakage currents ok - attachment 3

08:48 merger ok ~ 3 million data items/s

      TapeServer ok ~ 27 MB/s

      data forwarding to MBS ok

08:50 AIDA writing to file SecondDrive/TapeData/S480/R12_1524

09:56 all system wide checks ok

      good event statistics ok - attachment 4

      detector bias/leakage currents ok - attachment 5
  
      FEE64 temperatures ok - attachment 6

12:09 all system wide checks ok


12:13 all system wide checks ok

      good event statistics ok - attachment 7
  
      FEE64 temperatures ok - attachment 8

      detector bias/leakage currents ok, but ramping up - attachment 9


13:32 merger ok ~ 3 million data items/s

      TapeServer ok ~ 23 MB/s

      data forwarding to MBS ok

      Beam seems to be off

13:33 AIDA writing to file R12_1743

*update* checked DESPEC S480 Elog, issues with UNILAC, beam lost at 13:29

13:55 beam back

13:57 AIDA writing to file R12_1759

14:35 all system wide checks ok

      FEE64 temperatures ok - attachment 10

      good event statistics ok - attachment 11
  
      detector bias/leakage currents ok, but ramping up - attachment 12

15:18 merger ok ~ 2.7 million data items/s

      TapeServer ok ~ 27 MB/s

      data forwarding to MBS ok

      AIDA writing to file R12_1283

16:00 1.8.H high energy spectra - attachments 13 & 14

      hit-rate spectra - attachment 15
 
16.22 all system wide checks okay 
      FEE temperatures okay (attachment 16)
      leakage currents okay and recorded to spreadsheet (attachment 17)
      good event stats okay (attachment 18)
      merger running at 3M items/sec
      tape service running at 26MB/sec

18.00 began compressing R7 files with command: nice -n 10 gzip -v R7_* &
      
      all system wide checks okay 
      FEE temperatures okay (attachment 19)
      leakage currents okay and recorded to spreadsheet (attachment 20)
      good event stats okay (attachment 21)
      merger running at 3M items/sec
      tape service running at 27MB/sec    

20.10 all system wide checks okay 
      FEE temperatures okay (attachment 22)
      leakage currents okay and recorded to spreadsheet, detectors 1 and 2 still rising (attachment 23)
      good event stats okay (attachment 24)
      merger running at 3M items/sec
      tape service running at 25MB/sec

      rates histogram (attachment 25)
      low energy histograms (attachment 26 and 27)
      high energy histograms (attachments 28 and 29)

      
22.07 all system wide checks okay 
      FEE temperatures okay (attachment 30)
      leakage currents okay and recorded to spreadsheet (attachment 31)
      good event stats okay (attachment 32)
      merger running at 3M items/sec
      tape service running at 25MB/sec
      currently on R12_2097

23.56 all system wide checks okay 
      FEE temperatures okay (attachment 33)
      leakage currents okay and recorded to spreadsheet (attachment 34)
      good event stats okay (attachment 35)
      merger running at 3M items/sec
      tape service running at 26MB/sec
      
finishing up now. night shift group have been informed how to check up on AIDA, there is a sheet on the table with relevant information
      
Attachment 1: 200315_0749_stats.png
200315_0749_stats.png
Attachment 2: 200315_0749_temp.png
200315_0749_temp.png
Attachment 3: 200315_0750_bias.png
200315_0750_bias.png
Attachment 4: 200315_0956_stats.png
200315_0956_stats.png
Attachment 5: 200315_0957_bias.png
200315_0957_bias.png
Attachment 6: 200315_0957_temp.png
200315_0957_temp.png
Attachment 7: 200315_1210_stats.png
200315_1210_stats.png
Attachment 8: 200315_1211_temp.png
200315_1211_temp.png
Attachment 9: 200315_1212_bias.png
200315_1212_bias.png
Attachment 10: corrigan_temperature.png
corrigan_temperature.png
Attachment 11: corrigan_stats.png
corrigan_stats.png
Attachment 12: corrigan_bias.png
corrigan_bias.png
Attachment 13: corriganhe1.png
corriganhe1.png
Attachment 14: corriganhe2.png
corriganhe2.png
Attachment 15: 200315_1553_rates.png
200315_1553_rates.png
Attachment 16: 200315_1619_feetemps.png
200315_1619_feetemps.png
Attachment 17: 200315_1620_leakagecurrents.png
200315_1620_leakagecurrents.png
Attachment 18: 200315_1620_goodevents.png
200315_1620_goodevents.png
Attachment 19: 200315_1758_feetemps.png
200315_1758_feetemps.png
Attachment 20: 200315_1759_leakagecurrents.png
200315_1759_leakagecurrents.png
Attachment 21: 200315_1758_goodevents.png
200315_1758_goodevents.png
Attachment 22: 200315_2004_feetemps.png
200315_2004_feetemps.png
Attachment 23: 200315_2007_leakagecurrents#.png
200315_2007_leakagecurrents#.png
Attachment 24: 200315_2006_goodevents.png
200315_2006_goodevents.png
Attachment 25: 200315_2009_rates.png
200315_2009_rates.png
Attachment 26: 200315_2010_lowodd.png
200315_2010_lowodd.png
Attachment 27: 200315_2011_loweven.png
200315_2011_loweven.png
Attachment 28: 200315_2012_highodd.png
200315_2012_highodd.png
Attachment 29: 200315_2014_higheven.png
200315_2014_higheven.png
Attachment 30: 200315_2202_feetemps.png
200315_2202_feetemps.png
Attachment 31: 200315_2203_leakagecurrents.png
200315_2203_leakagecurrents.png
Attachment 32: 200315_2204_goodevents.png
200315_2204_goodevents.png
Attachment 33: 200315_2358_FEETEMPS.png
200315_2358_FEETEMPS.png
Attachment 34: 200315_2358LEAKAGECURRENTS.png
200315_2358LEAKAGECURRENTS.png
Attachment 35: 200315_2359_goodevents.png
200315_2359_goodevents.png
  177   Sun Mar 7 07:23:26 2021 CA, LSMarch 7th
08:00 ASIC settings 2019Dec19-16.19.51
      DSSSD#1 slow comparator 0xa
      DSSSD#2 slow comparator 0xa
      DSSSD#3 slow comparator 0xa

      BNC PB-5 Pulser 
      Amplitude1.0V
      Attenuation x1
      Frequency 2Hz
      tau_d 1ms
      - polarity
      Delay 250ns, tail pulse


08:30 - all system wide checks ok

      - Temperatures ok - attachment 1

      - good event statistics ok - attachment 2

      - detector bias/leakage currents ok - attachment 3

      DAQ continues to run - R1_1148

      Merger ok - 4.6M data items/s

      TapeServer ok - 43MB/s

08:55 Giovanna Benzoni - confirms contribution of light ions - attachment 4 see AOQ plots

09:19 zeroed all histograms

      writing to file R1_1204

      FRS on file 72

      DESPEC to add AIDA XY hit patterns to online analysis histograms

09:35 beam stop

09:56 beam back 

      writing to file R1_1249

      rate spectra - attachment 5

09:30 AIDA DAQ crashed - lost connection to aida05

      power cycle and reset

09:40 AIDA ok now, writing to file R2

09:50 LS takes over

11.10 (German time)
      System wide checks produces some fails: 
      FEE64 module aida06 global clocks failed, 6 (screenshot6)
      This error is seen in RIKEN too, tried a RESYNC from master timestamp tab but error still there

      FEE64 module aida06 failed
      FEE64 module aida07 failed
      Calibration test result: Passed 10, Failed 2 (screenshot7)
      Tried a recalibration for the two FEEs in FADC align & control tab but error still there

      Rest of the checks pass
      Statistics (screenshot8)
      FEE temps okay (screenshot9)
      Leakage currents written to sheets (screenshot10)
      Merger ~4.5M data items/s
      TapeServer ~ 45MB/s

13.00 System wide checks produces same fails as last time: 
      FEE64 module aida06 global clocks failed, 6
      FEE64 module aida06 failed
      FEE64 module aida07 failed
      Calibration test result: Passed 10, Failed 2
      Rest of the checks are okay

      Statistics (screenshot11)
      Spectra rate (screenshot12)
      FEE temps okay (screenshot13)
      Leakage currents written to sheets (screenshot14)
      Merger ~4.5M data items/s
      TapeServer ~ 45MB/s
      There is 1.5TB left on current drive

14.00 beam stopped to look at bPlast, have use the time to change slow comparator threshold
      Slow comparators in FEE9-12 raised to 0xd
      

      

      
      
      



        
Attachment 1: Screenshot_from_2021-03-07_07-30-28.png
Screenshot_from_2021-03-07_07-30-28.png
Attachment 2: Screenshot_from_2021-03-07_07-31-27.png
Screenshot_from_2021-03-07_07-31-27.png
Attachment 3: Screenshot_from_2021-03-07_07-31-55.png
Screenshot_from_2021-03-07_07-31-55.png
Attachment 4: Screenshot_from_2021-03-07_07-53-05.png
Screenshot_from_2021-03-07_07-53-05.png
Attachment 5: Screenshot_from_2021-03-07_08-59-23.png
Screenshot_from_2021-03-07_08-59-23.png
Attachment 6: 1110_Systemchecks.PNG
1110_Systemchecks.PNG
Attachment 7: 1110_Systemchecks2.PNG
1110_Systemchecks2.PNG
Attachment 8: 1110_Statistics.PNG
1110_Statistics.PNG
Attachment 9: 1110_FEEtemps.PNG
1110_FEEtemps.PNG
Attachment 10: 1110_Leakagecurrents.PNG
1110_Leakagecurrents.PNG
Attachment 11: 1300_Statistics.PNG
1300_Statistics.PNG
Attachment 12: 1300_Spectrumrate.PNG
1300_Spectrumrate.PNG
Attachment 13: 1300_FEEtemps.PNG
1300_FEEtemps.PNG
Attachment 14: 1300_Leakagecurrents.PNG
1300_Leakagecurrents.PNG
  179   Mon Mar 8 07:02:17 2021 CA, LSMonday 8th March 08:00 - 17:00
08:00 ASIC settings 2019Dec19-16.19.51
      DSSSD#1 slow comparator 0xa
      DSSSD#2 slow comparator 0xa
      DSSSD#3 slow comparator 0xd

      BNC PB-5 Pulser 
      Amplitude 1.0V
      Attenuation x1
      Frequency 2Hz
      tau_d 1ms
      - polarity
      Delay 250ns, tail pulse

      All system wide checks ok *except aida06 fails clock status 6*
      Statistics - attachment 1
      FEE Temperatures - attachment 2
      Bias and leakage currents ok - attachment 3

08:20 Merger ~4.5M data items/s
      TapeServer ~ 45MB/s

      Writing to file R6_1148

09:00 Beam off for optimisation - writing to file R6_1212

      Expected back at 09:30

09:30 Beam will be off for a bit longer - an hour or two

09:34 All system wide checks ok *except aida06 fails clock status 6*
      Statistics - attachment 4
      FEE Temperatures - attachment 5
      Bias and leakage currents ok - attachment 6

      Merger ~4.5M data items/s
      TapeServer ~ 45MB/s

10:10 (NH) Stopped DAQ to try the new merger from VP, new run R7, R8 for these tests
      Tape rate was 110 MB/s (!) and MBS relay seemed nothing... also it didn't stop when DAQ was stopped but took 10+ seconds 
to finish
      Something confusing, will look at data and report to VP
      Reverted to original merger and back... 45 MB/s
      R9

12.00 (LS)
      System wide checks okay except 'FEE64 module aida06 global clocks failed, 6'
      Statistics (attachment7)
      FEE temps (attatchment8)
      Rate spectra (attachment9), beam still down
      Leakage Currents written to sheets (attachment10)
      Merger ~45M items/s
      Tapeserver ~43MB/s

14.00 Still no beam but is expected to return soon
      System wide checks okay except 'FEE64 module aida06 global clocks failed, 6'
      Statistics (attachment11)
      FEE temps (attatchment12)
      Leakage Currents written to sheets (attachment13)
      Merger ~45M items/s
      Tapeserver ~44MB/s

14.40 Leakage currents still rising slowly compared to previous days, so will keep an eye on them (attachment14)

15.00 Beam is back (attachment15)
      Still writing to R9 (roughly around R9_386)

15.28 No beam (attachment16)

15.41 Beam back (attachment17)
      Roughly R9 (roughly around R9_439)

16.00 Analysis of R9_469, from FEE0 and FEE2 the hit rate for one side is around 48 HEC data items/second (attachment18)
 
16.20 System wide checks okay except 'FEE64 module aida06 global clocks failed, 6'
      Statistics (attachment19)
      Spectra Rate (attachment20)
      FEE temps (attatchment21)
      Leakage Currents written to sheets (attachment22), 
      still rising but looking on Grafana 7 day period should start to lower around 17.00
      Merger ~45M items/s
      Tapeserver ~43MB/s

16.55 Beam down (attachment23), back a couple minutes later

15.51 beam down (attachment24)
      
Attachment 1: Screenshot_from_2021-03-08_07-04-21.png
Screenshot_from_2021-03-08_07-04-21.png
Attachment 2: Screenshot_from_2021-03-08_07-07-24.png
Screenshot_from_2021-03-08_07-07-24.png
Attachment 3: Screenshot_from_2021-03-08_07-07-49.png
Screenshot_from_2021-03-08_07-07-49.png
Attachment 4: Screenshot_from_2021-03-08_08-31-37.png
Screenshot_from_2021-03-08_08-31-37.png
Attachment 5: Screenshot_from_2021-03-08_08-32-58.png
Screenshot_from_2021-03-08_08-32-58.png
Attachment 6: Screenshot_from_2021-03-08_08-34-25.png
Screenshot_from_2021-03-08_08-34-25.png
Attachment 7: 1200_Statistics.PNG
1200_Statistics.PNG
Attachment 8: 1200_FEETemps.PNG
1200_FEETemps.PNG
Attachment 9: 1200_SpectraRate.PNG
1200_SpectraRate.PNG
Attachment 10: 1200_LeakageCurrents.PNG
1200_LeakageCurrents.PNG
Attachment 11: 1400_Statistics.PNG
1400_Statistics.PNG
Attachment 12: 1400_FEETemps.PNG
1400_FEETemps.PNG
Attachment 13: 1400_LeakageCurrents.PNG
1400_LeakageCurrents.PNG
Attachment 14: 1500_Grafana_LeakageCurrents.PNG
1500_Grafana_LeakageCurrents.PNG
Attachment 15: 1500_SpectraRate.PNG
1500_SpectraRate.PNG
Attachment 16: 1528_SpectraRate.PNG
1528_SpectraRate.PNG
Attachment 17: 1541_SpectraRate.PNG
1541_SpectraRate.PNG
Attachment 18: 1600_R9_469_hit_rates.PNG
1600_R9_469_hit_rates.PNG
Attachment 19: 1620_Statistics.PNG
1620_Statistics.PNG
Attachment 20: 1620_SpectraRate.PNG
1620_SpectraRate.PNG
Attachment 21: 1620_FEETemps.PNG
1620_FEETemps.PNG
Attachment 22: 1620_LeakageCurrents.PNG
1620_LeakageCurrents.PNG
Attachment 23: 1655_SpectraRate.PNG
1655_SpectraRate.PNG
Attachment 24: 1749_SpectraRate.PNG
1749_SpectraRate.PNG
  183   Tue Mar 9 06:58:14 2021 CA, LSMarch 9th 08:00-11:00 shift
08:00 DAQ continues ok - writing to file R13_120

      Merger ok - 4.6M data items/s

      TapeServer ok - 43 MB/s

08:03 - all system wide checks ok

      - Temperatures ok - attachment 1

      - good event statistics ok - attachment 2

      - detector bias/leakage currents ok - attachment 3

08:08 ASIC settings 2019Dec19-16.19.51
      DSSSD#1 slow comparator 0xa
      DSSSD#2 slow comparator 0xa
      DSSSD#3 slow comparator 0xd

      BNC PB-5 Pulser 
      Amplitude1.0V
      Attenuation x1
      Frequency 2Hz
      tau_d 1ms
      - polarity
      Delay 250ns, tail pulse

08:32 CA briefly lost network connection - back now

08:42 DESPEC on file 102

09:24 Rate spectra - attachment 4

09:32 HEC spectra - attachment 5/6

      beam off - writing to file R13_243

09:42 merger bad timestamp errors - attachment 7

      all system wide checks ok

10:30 NH - Beam was down, I try the new merger from Vic again (R14)
           I made a mistake and had to powercycle fees as they got stuck! aida10 seemed to reboot itslef
           After fixing my mistakes, all running with new merger and rate 15 MB/s (R14)
           It all looked OK even going to MBS/Go4
           I reverted back to original hot merger, R15 and look at the data.
           Beam is now back and I return AIDA to shifters :)

11.30  System wide checks okay except ADC calibration:
       
       FEE64 module aida07 failed
       FEE64 module aida10 failed
       Calibration test result: Passed 10, Failed 2
       If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun 
       calibration for that module

       Reran calibration for both FEEs  but still fail
       Statistics (attachment8)
       Rate spectra (attachment9)
       FEE temps (attachment10)
       Leakage currents written to sheet (attachment11)
       Tapeserver ~33MB/s
       Merger ~ 3.4M items/s
       AIDA file R15_92

11.55 Beam off for a couple minutes but back now

12.00 New Merge terminal disappeared so stopped daq and restarted, and also restarted MBS back working
      Now writing to R16
      Merger ~ 4.5M items/s
      Tapeserver ~44MB/s

12.41 no beam

13.00 DAQ crashed (aida03 no responding), AIDA recovered from power cycle while beam is down
      Beam is back now, forwarding to MBS
      Merger ~ 45M items/s
      Tapeserver ~44MB/s
      Writing to R17

13.20 System wide checks okay except ADC calibration:
       
       FEE64 module aida07 failed
       FEE64 module aida10 failed
       Calibration test result: Passed 10, Failed 2
       If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun 
       calibration for that module

       Statistics (attachment12)
       Rate spectra (attachment13)
       FEE temps (attachment14)
       Leakage currents written to sheet (attachment15)
       Tapeserver ~45MB/s
       Merger ~ 4.4M items/s

14.27 beam off
      file R17_115
      back a couple minutes later

14.40 no beam again back at 14.46

15.40 System wide checks okay except ADC calibration:
       
       FEE64 module aida07 failed
       FEE64 module aida10 failed
       Calibration test result: Passed 10, Failed 2
       If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun 
       calibration for that module

       Statistics (attachment16)
       Rate spectra (attachment17)
       FEE temps (attachment18)
       Leakage currents written to sheet (attachment19)
       Tapeserver ~44MB/s
       Merger ~ 4.4M items/s
       
15.52 beam down R17_221, back at 15.58 R17_229

16.07 Beam down
       
16:20 Removed S480 files from /media/1e... and /media/SecondDrive
      Set merger to no output and stopped tapedata.
      Changed symbolic link of /TapeData to /media/1e.....
      Restarted tapedata and the merger with outputting to file R20
      Local DESPEC was made aware of this before carrying out and stopped the run.
      Beam back

17.12 beam down for a couple of minutes R20_90

17.40 System wide checks okay except ADC calibration:
       
       FEE64 module aida07 failed
       FEE64 module aida10 failed
       Calibration test result: Passed 10, Failed 2
       If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun 
       calibration for that module

       Statistics (attachment20)
       Rate spectra (attachment21)
       FEE temps (attachment22)
       Leakage currents written to sheet (attachment23)
       Tapeserver ~45MB/s
       Merger ~ 4.5M items/s

      
Attachment 1: Screenshot_from_2021-03-09_07-05-35.png
Screenshot_from_2021-03-09_07-05-35.png
Attachment 2: Screenshot_from_2021-03-09_07-06-17.png
Screenshot_from_2021-03-09_07-06-17.png
Attachment 3: Screenshot_from_2021-03-09_07-06-53.png
Screenshot_from_2021-03-09_07-06-53.png
Attachment 4: Screenshot_from_2021-03-09_08-08-45_-_1.png
Screenshot_from_2021-03-09_08-08-45_-_1.png
Attachment 5: Screenshot_from_2021-03-09_08-24-06.png
Screenshot_from_2021-03-09_08-24-06.png
Attachment 6: Screenshot_from_2021-03-09_08-26-47.png
Screenshot_from_2021-03-09_08-26-47.png
Attachment 7: Screenshot_from_2021-03-09_08-44-17.png
Screenshot_from_2021-03-09_08-44-17.png
Attachment 8: 1130_Statistics.PNG
1130_Statistics.PNG
Attachment 9: 1130_SpectraRates.PNG
1130_SpectraRates.PNG
Attachment 10: 1130_FEETemps.PNG
1130_FEETemps.PNG
Attachment 11: 1130_LeakageCurrents.PNG
1130_LeakageCurrents.PNG
Attachment 12: 1330_Statistics.PNG
1330_Statistics.PNG
Attachment 13: 1330_SpectraRates.PNG
1330_SpectraRates.PNG
Attachment 14: 1330_FEETemps.PNG
1330_FEETemps.PNG
Attachment 15: 1330_LeakageCurrents.PNG
1330_LeakageCurrents.PNG
Attachment 16: 1540_Statistics.PNG
1540_Statistics.PNG
Attachment 17: 1540_SpectraRates.PNG
1540_SpectraRates.PNG
Attachment 18: 1540_FEETemps.PNG
1540_FEETemps.PNG
Attachment 19: 1540_LeakageCurrents.PNG
1540_LeakageCurrents.PNG
Attachment 20: 1730_Statistics.PNG
1730_Statistics.PNG
Attachment 21: 1730_SpectraRates.PNG
1730_SpectraRates.PNG
Attachment 22: 1730_FEETemps.PNG
1730_FEETemps.PNG
Attachment 23: 1730_LeakageCurrents.PNG
1730_LeakageCurrents.PNG
  103   Thu Dec 5 22:00:11 2019 CA, DK, NHDec 5 Night Shift
15:30
  TD found the merger was running at ~43 Mb/s
  The odd-numbered FEE64s all had significant (>200k) disk rates
  ALL LEC/MEC fast discriminators and increased slow comparator from 0xa to 0x14
  Merger rate dropped to a more nominal value like 2.3 Mb/s
  Settings not saved

40Ar beam 300 MeV/u expected, but SIS problem at the moment.

23:00
  Leak currents are stable as previously (20-21 Nov) and entered to Google spreadsheet
  S4 vault is closed.
  Presently AIDA is running without tape storage
  System wide checks: 
    Master clock / aida01 sync reports error as expected; 
    All white rabbit decoders fail, but they are just off by 1 bit (no problem)
    Everything else okay.
  Stats look okay, but keep in mind above discriminator threshold -- see attachment 1
  Temperatures fine -- see attachment 2
  MBS looks okay -- we see a possible hint for the 1 bit difference with an "unexpected data word" -- see attachment 3 
  Data rate is 1.7 Mb/s, merger okay -- see attachment 4

23:30 
  Now we wait for the beam

00:05
  Attachments 5-11: results of system wide checks after collecting sync errors and WR errors from baseline - all ok

00:25
  Histograms zeroed at 00:25.

00:35

  1.8.L pulser peak widths

FEE64       width(ch)
  1            92
  2            131
  3            75
  4            97
  5            89
  6            117
  7            90
  8            87
  9            223
 10            58
 11            112
 12            126 

no peak observed on aida09

For comparison, pulser peaks measured on 13/11/19:

pulser peak widths:
                   FEE      width (ch)

                   aida01 - 130

                   aida02 - 168
             
                   aida03 - 83

                   aida04 - 83

                   aida05 - 200

                   aida06 - 148

                   aida07 - 250

                   aida08 - 81

                   aida09 - 134

                   aida10 - 190

                   aida11 - 333

                   aida12 - 149

2:00
  40Ar beam tuning is finished.  Intensity at S4 is about 3 kHz per spill.  1 spill per / 10 seconds
  MBS system cannot be used for online merging tonight, because NeuLAND group left their connection open
  We can merge offline using the White Rabbit timestamps
  Remove the beam blocking steel flange from in front of AIDA et al.
  Writing to file Dec19/R1

03.20 Rate, Stat, 1.8.L, 1.8.H, 1.8.W spectra - attachments 13-20
      Observe
       light ions in 1.8.L spectra
       low energy (<1.5GeV) heavy ions in 1.8.H spectra with most heavy ions stopping in DSSSDs 1 & 2
       Stat hit patterns indicate broad (c. 5cm FWHM) centred heavy ion peak

03.24 Detector biases & leakage currents OK - attachment 21
      FEE64 temperatures OK - attachment 22
      Good event statistics (12 of 12 < 50k) - attachment 23

      System wide checks
      List All Sync Error Counters - attachment 24
      List all Sync Received Counters - attachment 25

      Baseline invoked earlier c. 00.05
      Collect all SYNC Error Counters for Baseline
      Collect All White Rabbit status Error Counters for Baseline 

04:07 2.6 mm Al degrader. 300Hz beam. 300 MeV/u 40Ar beam.

04.14 pulser peak seen in aida09 after performing ASIC check-load earlier in shift - width = 219 ch

04.45 1.8.H energy spectra (attachment 26/27)

06:08 attachments 28-30: detector bias/leakage currents ok
                         temperatures ok
                         good event statistics ok
                         all system wide checks ok

08:49 DAQ stop/start. TapeServer writing to No Storage. End of beam.


      
Attachment 1: 05-Dec-Stats.png
05-Dec-Stats.png
Attachment 2: 05-Dec-Temps.png
05-Dec-Temps.png
Attachment 3: 05-Dec-MBS.png
05-Dec-MBS.png
Attachment 4: 05-Dec-MergerRates.png
05-Dec-MergerRates.png
Attachment 5: adccalib.png
adccalib.png
Attachment 6: clockstatus.png
clockstatus.png
Attachment 7: meminfo.png
meminfo.png
Attachment 8: syncerrors.png
syncerrors.png
Attachment 9: syncpulses.png
syncpulses.png
Attachment 10: wrdecoder.png
wrdecoder.png
Attachment 11: 1.png
1.png
Attachment 12: 2.png
2.png
Attachment 13: 3.png
3.png
Attachment 14: 4.png
4.png
Attachment 15: 5.png
5.png
Attachment 16: 6.png
6.png
Attachment 17: 7.png
7.png
Attachment 18: 8.png
8.png
Attachment 19: 9.png
9.png
Attachment 20: 10.png
10.png
Attachment 21: 11.png
11.png
Attachment 22: 12.png
12.png
Attachment 23: 13.png
13.png
Attachment 24: 18hOdd.png
18hOdd.png
Attachment 25: even18h.png
even18h.png
Attachment 26: cja_bias.png
cja_bias.png
Attachment 27: cja_stats.png
cja_stats.png
Attachment 28: cja_temp.png
cja_temp.png
Attachment 29: pulser_peaks1.png
pulser_peaks1.png
Attachment 30: pulser_peaks2.png
pulser_peaks2.png
  104   Fri Dec 6 20:34:56 2019 CA, DK, NHDec 6 Night Shift
21.35 AIDA DAQ currently running, TapeService on, not writing to storage

      System wide checks ok

      Attachments 1-3: temperatures, good event stats, detector bias/leakage currents ok

22.04 waiting for beam. Possibility of fragments, further news to come

22.43 expecting fragments with 300 MeV/u 40Ar primary beam

      histograms zeroed

00.31 still waiting for beam

      detector bias/leakage currents ok (see google spreadsheet)

      FEE64 temps and good event statistics ok

02:02 still awaiting beam - issues with interlock

03:17 beam tuning - centering primary beam on target

03:43 beam reached AIDA, fragmentation beam (see elog entry by DK for PID)

      primary beam 300 Mev/u 40Ar, fragments in region of 34Si

      started writing to file Dec19/R2 at rate ~ 4000 kb/s

      histograms zeroed

03.49 system wide checks all ok

      attachments 4-6:

      temperatures ok

      bias/leakage currents ok

      good event statistics ok, except aida09 running slightly fast

04:03 attachments 7-8 - 1.8.H energy spectra

                      - no high energy events seen in aida10 -> checked ASIC control

                      - still no high energy events in aida10

04:09 attachment 9 - rate spectra for all FEE modules

04.16 attachment 10 - 1D hits for all FEE modules

04.19 beam stop - file R2_3

04:52 beam back on - file R2_6 - rate of ~ 5000 ions per spill

06:13 temperatures ok

      good event statistics ok (aida09 still running fast) - attachment 11

      detector bias/ leakage currents ok - see google spreadsheet

      all system wide checks ok

      1.8.H spectra - attachments 12/13

06:20 attachment 14 - rate spectra for all FEE modules

      attachment 15 - 1D hits for all FEE modules

      still no high energy events in aida10


      
Attachment 1: 191206_2139_temp.png
191206_2139_temp.png
Attachment 2: 191206_2140_bias.png
191206_2140_bias.png
Attachment 3: 191206_2140_stats.png
191206_2140_stats.png
Attachment 4: 191107_0351_temp.png
191107_0351_temp.png
Attachment 5: 191207_0352_bias.png
191207_0352_bias.png
Attachment 6: 191207_0352_stats.png
191207_0352_stats.png
Attachment 7: 191207_18h_2.png
191207_18h_2.png
Attachment 8: 191207_18H_1.png
191207_18H_1.png
Attachment 9: 191207_0408_rates.png
191207_0408_rates.png
Attachment 10: 191207_0415_statspec.png
191207_0415_statspec.png
Attachment 11: 191207_0614_stats.png
191207_0614_stats.png
Attachment 12: 191207_0606_18H_1.png
191207_0606_18H_1.png
Attachment 13: 191207_0609_18h_2.png
191207_0609_18h_2.png
Attachment 14: 191207_0618_rate.png
191207_0618_rate.png
Attachment 15: 191207_0619_hits.png
191207_0619_hits.png
  43   Wed Apr 3 09:28:07 2019 CA, CB, TDWednesday April 3rd 2018
Arrived at 09:54 - sequenced all FEE64 on

FEE64 not mounted properly - something happened to network overnight, see attachment 1,2 for AIDA system log

10.23 - FEE64 still not mounted properly, power cycling FEE64

10.30 - sequenced all FEE64 on

10.43 - FEE64 mounted successfully

      - DAQ reset and setup - looks ok

      - histogram enabled on aida12

      - clock ReSYNC

      - system wide checks - all FEE64 pass clock status check
           
                           - aida2 and aida12 fail ADC calibration

                           - SYNC pulse and SYNC error checks ok

                           - WR decoder check returns errors (attachment 3)


10.53 - FEE64 temperatures ok (attachment 4)

      - detector bias/leakage current ok (attachment 5)

11.04 - TapeServer start - not writing to storage

      - ASIC check load performed

      - DAQ start

      - ASIC control tab (attachment 6)

      - Run control keeps reloading - has returned to 'reset' screen with aida11 showing "undefined" (attachment 7)

      - good event statistics - all FEE64 showing 0 *except* aida09 (attachment 8)

11.20 - stopped tape server

      - reset daq - returns error (attachment 9)

11.41 DAQ off, detector bias off, FEE64 off

13.30 detector bias on, FEE64 sequenced on

      all browser tabs reopened

      merger set back up - ok

      mbs data relay set back up - ok

      DAQ setup and started, initially ok

      good event statistics (attachment 10)

16.25 FEE64 restarted, along with Merger, TapeServer, DAQ (not writing to file)

      thresholds (slow comparator, fast comparator HEC, fast comparator LEC) raised to 0xff for testing.

      aida10 stats appear reasonable, however wave events still reported despite being deactivated (see report entry
on DAQ stall) (attachment 11)

17.00 DAQ, TapeServer, Merger all stopped

      Detector bias off

      FEE64s switched off

      coolant water still running (cannot be disabled due to lack of access)

      AIDA in safe state
Attachment 1: 030419_aidasystemlog.txt
Apr  3 04:06:56 aidas-gsi NetworkManager[3917]: <info>   nameserver '140.181.96.11'
Apr  3 04:06:56 aidas-gsi NetworkManager[3917]: <info>   nameserver '140.181.96.29'
Apr  3 04:06:56 aidas-gsi NetworkManager[3917]: <info>   domain name 'gsi.de'
Apr  3 04:08:00 aidas-gsi dhcpd: Dynamic and static leases present for 192.168.11.117.
Apr  3 04:08:00 aidas-gsi dhcpd: Remove host declaration nnrpi2 or remove 192.168.11.117
Apr  3 04:08:00 aidas-gsi dhcpd: from the dynamic address pool for 192.168.11.0/24
Apr  3 04:08:00 aidas-gsi dhcpd: uid lease 192.168.11.104 for client b8:27:eb:40:53:e8 is duplicate on 192.168.11.0/24
Apr  3 04:08:00 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.117 from b8:27:eb:40:53:e8 via p4p1
Apr  3 04:08:00 aidas-gsi dhcpd: DHCPACK on 192.168.11.117 to b8:27:eb:40:53:e8 via p4p1
Apr  3 04:36:05 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.251 from b8:27:eb:bb:46:7b via p4p1
Apr  3 04:36:05 aidas-gsi dhcpd: DHCPACK on 192.168.11.251 to b8:27:eb:bb:46:7b via p4p1
Apr  3 04:38:00 aidas-gsi dhcpd: Dynamic and static leases present for 192.168.11.117.
Apr  3 04:38:00 aidas-gsi dhcpd: Remove host declaration nnrpi2 or remove 192.168.11.117
Apr  3 04:38:00 aidas-gsi dhcpd: from the dynamic address pool for 192.168.11.0/24
Apr  3 04:38:00 aidas-gsi dhcpd: uid lease 192.168.11.104 for client b8:27:eb:40:53:e8 is duplicate on 192.168.11.0/24
Apr  3 04:38:00 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.117 from b8:27:eb:40:53:e8 via p4p1
Apr  3 04:38:00 aidas-gsi dhcpd: DHCPACK on 192.168.11.117 to b8:27:eb:40:53:e8 via p4p1
Apr  3 04:54:49 aidas-gsi dhclient[3982]: DHCPREQUEST on eth0 to 10.99.0.13 port 67 (xid=0x2d1bd666)
Apr  3 04:54:49 aidas-gsi dhclient[3982]: DHCPACK from 10.99.0.13 (xid=0x2d1bd666)
Apr  3 04:54:49 aidas-gsi dhclient[3982]: bound to 10.99.2.56 -- renewal in 2761 seconds.
Apr  3 04:54:49 aidas-gsi NetworkManager[3917]: <info> (eth0): DHCPv4 state changed renew -> renew
Apr  3 04:54:49 aidas-gsi NetworkManager[3917]: <info>   address 10.99.2.56
Apr  3 04:54:49 aidas-gsi NetworkManager[3917]: <info>   prefix 17 (255.255.128.0)
Apr  3 04:54:49 aidas-gsi NetworkManager[3917]: <info>   gateway 10.99.0.1
Apr  3 04:54:49 aidas-gsi NetworkManager[3917]: <info>   hostname 'aida-3'
Apr  3 04:54:49 aidas-gsi NetworkManager[3917]: <info>   nameserver '140.181.96.11'
Apr  3 04:54:49 aidas-gsi NetworkManager[3917]: <info>   nameserver '140.181.96.29'
Apr  3 04:54:49 aidas-gsi NetworkManager[3917]: <info>   domain name 'gsi.de'
Apr  3 05:06:05 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.251 from b8:27:eb:bb:46:7b via p4p1
Apr  3 05:06:05 aidas-gsi dhcpd: DHCPACK on 192.168.11.251 to b8:27:eb:bb:46:7b via p4p1
Apr  3 05:07:22 aidas-gsi yum[25016]: Updated: tzdata-2019a-1.el6.noarch
Apr  3 05:07:23 aidas-gsi yum[25016]: Updated: tzdata-java-2019a-1.el6.noarch
Apr  3 05:08:00 aidas-gsi dhcpd: Dynamic and static leases present for 192.168.11.117.
Apr  3 05:08:00 aidas-gsi dhcpd: Remove host declaration nnrpi2 or remove 192.168.11.117
Apr  3 05:08:00 aidas-gsi dhcpd: from the dynamic address pool for 192.168.11.0/24
Apr  3 05:08:00 aidas-gsi dhcpd: uid lease 192.168.11.104 for client b8:27:eb:40:53:e8 is duplicate on 192.168.11.0/24
Apr  3 05:08:00 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.117 from b8:27:eb:40:53:e8 via p4p1
Apr  3 05:08:00 aidas-gsi dhcpd: DHCPACK on 192.168.11.117 to b8:27:eb:40:53:e8 via p4p1
Apr  3 05:36:05 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.251 from b8:27:eb:bb:46:7b via p4p1
Apr  3 05:36:05 aidas-gsi dhcpd: DHCPACK on 192.168.11.251 to b8:27:eb:bb:46:7b via p4p1
Apr  3 05:38:00 aidas-gsi dhcpd: Dynamic and static leases present for 192.168.11.117.
Apr  3 05:38:00 aidas-gsi dhcpd: Remove host declaration nnrpi2 or remove 192.168.11.117
Apr  3 05:38:00 aidas-gsi dhcpd: from the dynamic address pool for 192.168.11.0/24
Apr  3 05:38:00 aidas-gsi dhcpd: uid lease 192.168.11.104 for client b8:27:eb:40:53:e8 is duplicate on 192.168.11.0/24
Apr  3 05:38:00 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.117 from b8:27:eb:40:53:e8 via p4p1
Apr  3 05:38:00 aidas-gsi dhcpd: DHCPACK on 192.168.11.117 to b8:27:eb:40:53:e8 via p4p1
Apr  3 05:40:50 aidas-gsi dhclient[3982]: DHCPREQUEST on eth0 to 10.99.0.13 port 67 (xid=0x2d1bd666)
Apr  3 05:40:50 aidas-gsi dhclient[3982]: DHCPACK from 10.99.0.13 (xid=0x2d1bd666)
Apr  3 05:40:50 aidas-gsi dhclient[3982]: bound to 10.99.2.56 -- renewal in 3003 seconds.
Apr  3 05:40:50 aidas-gsi NetworkManager[3917]: <info> (eth0): DHCPv4 state changed renew -> renew
Apr  3 05:40:50 aidas-gsi NetworkManager[3917]: <info>   address 10.99.2.56
Apr  3 05:40:50 aidas-gsi NetworkManager[3917]: <info>   prefix 17 (255.255.128.0)
Apr  3 05:40:50 aidas-gsi NetworkManager[3917]: <info>   gateway 10.99.0.1
Apr  3 05:40:50 aidas-gsi NetworkManager[3917]: <info>   hostname 'aida-3'
Apr  3 05:40:50 aidas-gsi NetworkManager[3917]: <info>   nameserver '140.181.96.11'
Apr  3 05:40:50 aidas-gsi NetworkManager[3917]: <info>   nameserver '140.181.96.29'
Apr  3 05:40:50 aidas-gsi NetworkManager[3917]: <info>   domain name 'gsi.de'
Apr  3 06:06:05 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.251 from b8:27:eb:bb:46:7b via p4p1
Apr  3 06:06:05 aidas-gsi dhcpd: DHCPACK on 192.168.11.251 to b8:27:eb:bb:46:7b via p4p1
Apr  3 06:08:00 aidas-gsi dhcpd: Dynamic and static leases present for 192.168.11.117.
Apr  3 06:08:00 aidas-gsi dhcpd: Remove host declaration nnrpi2 or remove 192.168.11.117
Apr  3 06:08:00 aidas-gsi dhcpd: from the dynamic address pool for 192.168.11.0/24
Apr  3 06:08:00 aidas-gsi dhcpd: uid lease 192.168.11.104 for client b8:27:eb:40:53:e8 is duplicate on 192.168.11.0/24
Apr  3 06:08:00 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.117 from b8:27:eb:40:53:e8 via p4p1
Apr  3 06:08:00 aidas-gsi dhcpd: DHCPACK on 192.168.11.117 to b8:27:eb:40:53:e8 via p4p1
Apr  3 06:30:53 aidas-gsi dhclient[3982]: DHCPREQUEST on eth0 to 10.99.0.13 port 67 (xid=0x2d1bd666)
Apr  3 06:30:53 aidas-gsi dhclient[3982]: DHCPACK from 10.99.0.13 (xid=0x2d1bd666)
Apr  3 06:30:53 aidas-gsi dhclient[3982]: bound to 10.99.2.56 -- renewal in 3269 seconds.
Apr  3 06:30:53 aidas-gsi NetworkManager[3917]: <info> (eth0): DHCPv4 state changed renew -> renew
Apr  3 06:30:53 aidas-gsi NetworkManager[3917]: <info>   address 10.99.2.56
Apr  3 06:30:53 aidas-gsi NetworkManager[3917]: <info>   prefix 17 (255.255.128.0)
Apr  3 06:30:53 aidas-gsi NetworkManager[3917]: <info>   gateway 10.99.0.1
Apr  3 06:30:53 aidas-gsi NetworkManager[3917]: <info>   hostname 'aida-3'
Apr  3 06:30:53 aidas-gsi NetworkManager[3917]: <info>   nameserver '140.181.96.11'
Apr  3 06:30:53 aidas-gsi NetworkManager[3917]: <info>   nameserver '140.181.96.29'
Apr  3 06:30:53 aidas-gsi NetworkManager[3917]: <info>   domain name 'gsi.de'
Apr  3 06:36:05 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.251 from b8:27:eb:bb:46:7b via p4p1
Apr  3 06:36:05 aidas-gsi dhcpd: DHCPACK on 192.168.11.251 to b8:27:eb:bb:46:7b via p4p1
Apr  3 06:38:00 aidas-gsi dhcpd: Dynamic and static leases present for 192.168.11.117.
Apr  3 06:38:00 aidas-gsi dhcpd: Remove host declaration nnrpi2 or remove 192.168.11.117
Apr  3 06:38:00 aidas-gsi dhcpd: from the dynamic address pool for 192.168.11.0/24
Apr  3 06:38:00 aidas-gsi dhcpd: uid lease 192.168.11.104 for client b8:27:eb:40:53:e8 is duplicate on 192.168.11.0/24
Apr  3 06:38:00 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.117 from b8:27:eb:40:53:e8 via p4p1
Apr  3 06:38:00 aidas-gsi dhcpd: DHCPACK on 192.168.11.117 to b8:27:eb:40:53:e8 via p4p1
Apr  3 07:06:05 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.251 from b8:27:eb:bb:46:7b via p4p1
Apr  3 07:06:05 aidas-gsi dhcpd: DHCPACK on 192.168.11.251 to b8:27:eb:bb:46:7b via p4p1
Apr  3 07:08:00 aidas-gsi dhcpd: Dynamic and static leases present for 192.168.11.117.
Apr  3 07:08:00 aidas-gsi dhcpd: Remove host declaration nnrpi2 or remove 192.168.11.117
Apr  3 07:08:00 aidas-gsi dhcpd: from the dynamic address pool for 192.168.11.0/24
Apr  3 07:08:00 aidas-gsi dhcpd: uid lease 192.168.11.104 for client b8:27:eb:40:53:e8 is duplicate on 192.168.11.0/24
Apr  3 07:08:00 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.117 from b8:27:eb:40:53:e8 via p4p1
Apr  3 07:08:00 aidas-gsi dhcpd: DHCPACK on 192.168.11.117 to b8:27:eb:40:53:e8 via p4p1
Apr  3 07:25:22 aidas-gsi dhclient[3982]: DHCPREQUEST on eth0 to 10.99.0.13 port 67 (xid=0x2d1bd666)
Apr  3 07:25:22 aidas-gsi dhclient[3982]: DHCPACK from 10.99.0.13 (xid=0x2d1bd666)
Apr  3 07:25:22 aidas-gsi dhclient[3982]: bound to 10.99.2.56 -- renewal in 2773 seconds.
Apr  3 07:25:22 aidas-gsi NetworkManager[3917]: <info> (eth0): DHCPv4 state changed renew -> renew
Apr  3 07:25:22 aidas-gsi NetworkManager[3917]: <info>   address 10.99.2.56
Apr  3 07:25:22 aidas-gsi NetworkManager[3917]: <info>   prefix 17 (255.255.128.0)
Apr  3 07:25:22 aidas-gsi NetworkManager[3917]: <info>   gateway 10.99.0.1
Apr  3 07:25:22 aidas-gsi NetworkManager[3917]: <info>   hostname 'aida-3'
Apr  3 07:25:22 aidas-gsi NetworkManager[3917]: <info>   nameserver '140.181.96.11'
Apr  3 07:25:22 aidas-gsi NetworkManager[3917]: <info>   nameserver '140.181.96.29'
Apr  3 07:25:22 aidas-gsi NetworkManager[3917]: <info>   domain name 'gsi.de'
Apr  3 07:36:05 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.251 from b8:27:eb:bb:46:7b via p4p1
Apr  3 07:36:05 aidas-gsi dhcpd: DHCPACK on 192.168.11.251 to b8:27:eb:bb:46:7b via p4p1
Apr  3 07:38:00 aidas-gsi dhcpd: Dynamic and static leases present for 192.168.11.117.
Apr  3 07:38:00 aidas-gsi dhcpd: Remove host declaration nnrpi2 or remove 192.168.11.117
Apr  3 07:38:00 aidas-gsi dhcpd: from the dynamic address pool for 192.168.11.0/24
Apr  3 07:38:00 aidas-gsi dhcpd: uid lease 192.168.11.104 for client b8:27:eb:40:53:e8 is duplicate on 192.168.11.0/24
Apr  3 07:38:00 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.117 from b8:27:eb:40:53:e8 via p4p1
Apr  3 07:38:00 aidas-gsi dhcpd: DHCPACK on 192.168.11.117 to b8:27:eb:40:53:e8 via p4p1
Apr  3 08:06:05 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.251 from b8:27:eb:bb:46:7b via p4p1
Apr  3 08:06:05 aidas-gsi dhcpd: DHCPACK on 192.168.11.251 to b8:27:eb:bb:46:7b via p4p1
Apr  3 08:08:00 aidas-gsi dhcpd: Dynamic and static leases present for 192.168.11.117.
Apr  3 08:08:00 aidas-gsi dhcpd: Remove host declaration nnrpi2 or remove 192.168.11.117
Apr  3 08:08:00 aidas-gsi dhcpd: from the dynamic address pool for 192.168.11.0/24
Apr  3 08:08:00 aidas-gsi dhcpd: uid lease 192.168.11.104 for client b8:27:eb:40:53:e8 is duplicate on 192.168.11.0/24
Apr  3 08:08:00 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.117 from b8:27:eb:40:53:e8 via p4p1
Apr  3 08:08:00 aidas-gsi dhcpd: DHCPACK on 192.168.11.117 to b8:27:eb:40:53:e8 via p4p1
Apr  3 08:11:35 aidas-gsi dhclient[3982]: DHCPREQUEST on eth0 to 10.99.0.13 port 67 (xid=0x2d1bd666)
Apr  3 08:11:35 aidas-gsi dhclient[3982]: DHCPACK from 10.99.0.13 (xid=0x2d1bd666)
Apr  3 08:11:35 aidas-gsi dhclient[3982]: bound to 10.99.2.56 -- renewal in 3455 seconds.
Apr  3 08:11:35 aidas-gsi NetworkManager[3917]: <info> (eth0): DHCPv4 state changed renew -> renew
Apr  3 08:11:35 aidas-gsi NetworkManager[3917]: <info>   address 10.99.2.56
Apr  3 08:11:35 aidas-gsi NetworkManager[3917]: <info>   prefix 17 (255.255.128.0)
Apr  3 08:11:35 aidas-gsi NetworkManager[3917]: <info>   gateway 10.99.0.1
Apr  3 08:11:35 aidas-gsi NetworkManager[3917]: <info>   hostname 'aida-3'
Apr  3 08:11:35 aidas-gsi NetworkManager[3917]: <info>   nameserver '140.181.96.11'
Apr  3 08:11:35 aidas-gsi NetworkManager[3917]: <info>   nameserver '140.181.96.29'
Apr  3 08:11:35 aidas-gsi NetworkManager[3917]: <info>   domain name 'gsi.de'
Apr  3 08:36:05 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.251 from b8:27:eb:bb:46:7b via p4p1
Apr  3 08:36:05 aidas-gsi dhcpd: DHCPACK on 192.168.11.251 to b8:27:eb:bb:46:7b via p4p1
Apr  3 08:38:00 aidas-gsi dhcpd: Dynamic and static leases present for 192.168.11.117.
Apr  3 08:38:00 aidas-gsi dhcpd: Remove host declaration nnrpi2 or remove 192.168.11.117
Apr  3 08:38:00 aidas-gsi dhcpd: from the dynamic address pool for 192.168.11.0/24
Apr  3 08:38:00 aidas-gsi dhcpd: uid lease 192.168.11.104 for client b8:27:eb:40:53:e8 is duplicate on 192.168.11.0/24
Apr  3 08:38:00 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.117 from b8:27:eb:40:53:e8 via p4p1
Apr  3 08:38:00 aidas-gsi dhcpd: DHCPACK on 192.168.11.117 to b8:27:eb:40:53:e8 via p4p1
Apr  3 09:06:05 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.251 from b8:27:eb:bb:46:7b via p4p1
Apr  3 09:06:05 aidas-gsi dhcpd: DHCPACK on 192.168.11.251 to b8:27:eb:bb:46:7b via p4p1
Apr  3 09:08:00 aidas-gsi dhcpd: Dynamic and static leases present for 192.168.11.117.
Apr  3 09:08:00 aidas-gsi dhcpd: Remove host declaration nnrpi2 or remove 192.168.11.117
Apr  3 09:08:00 aidas-gsi dhcpd: from the dynamic address pool for 192.168.11.0/24
Apr  3 09:08:00 aidas-gsi dhcpd: uid lease 192.168.11.104 for client b8:27:eb:40:53:e8 is duplicate on 192.168.11.0/24
Apr  3 09:08:00 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.117 from b8:27:eb:40:53:e8 via p4p1
Apr  3 09:08:00 aidas-gsi dhcpd: DHCPACK on 192.168.11.117 to b8:27:eb:40:53:e8 via p4p1
Apr  3 09:09:10 aidas-gsi dhclient[3982]: DHCPREQUEST on eth0 to 10.99.0.13 port 67 (xid=0x2d1bd666)
Apr  3 09:09:10 aidas-gsi dhclient[3982]: DHCPACK from 10.99.0.13 (xid=0x2d1bd666)
Apr  3 09:09:10 aidas-gsi dhclient[3982]: bound to 10.99.2.56 -- renewal in 2865 seconds.
Apr  3 09:09:10 aidas-gsi NetworkManager[3917]: <info> (eth0): DHCPv4 state changed renew -> renew
Apr  3 09:09:10 aidas-gsi NetworkManager[3917]: <info>   address 10.99.2.56
Apr  3 09:09:10 aidas-gsi NetworkManager[3917]: <info>   prefix 17 (255.255.128.0)
Apr  3 09:09:10 aidas-gsi NetworkManager[3917]: <info>   gateway 10.99.0.1
Apr  3 09:09:10 aidas-gsi NetworkManager[3917]: <info>   hostname 'aida-3'
Apr  3 09:09:10 aidas-gsi NetworkManager[3917]: <info>   nameserver '140.181.96.11'
Apr  3 09:09:10 aidas-gsi NetworkManager[3917]: <info>   nameserver '140.181.96.29'
Apr  3 09:09:10 aidas-gsi NetworkManager[3917]: <info>   domain name 'gsi.de'
Apr  3 09:36:05 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.251 from b8:27:eb:bb:46:7b via p4p1
Apr  3 09:36:05 aidas-gsi dhcpd: DHCPACK on 192.168.11.251 to b8:27:eb:bb:46:7b via p4p1
Apr  3 09:38:00 aidas-gsi dhcpd: Dynamic and static leases present for 192.168.11.117.
Apr  3 09:38:00 aidas-gsi dhcpd: Remove host declaration nnrpi2 or remove 192.168.11.117
Apr  3 09:38:00 aidas-gsi dhcpd: from the dynamic address pool for 192.168.11.0/24
Apr  3 09:38:00 aidas-gsi dhcpd: uid lease 192.168.11.104 for client b8:27:eb:40:53:e8 is duplicate on 192.168.11.0/24
Apr  3 09:38:00 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.117 from b8:27:eb:40:53:e8 via p4p1
Apr  3 09:38:00 aidas-gsi dhcpd: DHCPACK on 192.168.11.117 to b8:27:eb:40:53:e8 via p4p1
Apr  3 09:53:59 aidas-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:f6:b7 via p4p1
Apr  3 09:53:59 aidas-gsi dhcpd: DHCPOFFER on 192.168.11.4 to d8:80:39:41:f6:b7 via p4p1
Apr  3 09:53:59 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.4 (192.168.11.99) from d8:80:39:41:f6:b7 via p4p1
Apr  3 09:53:59 aidas-gsi dhcpd: DHCPACK on 192.168.11.4 to d8:80:39:41:f6:b7 via p4p1
Apr  3 09:53:59 aidas-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:d8:21 via p4p1
Apr  3 09:53:59 aidas-gsi dhcpd: DHCPOFFER on 192.168.11.3 to d8:80:39:41:d8:21 via p4p1
Apr  3 09:53:59 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.3 (192.168.11.99) from d8:80:39:41:d8:21 via p4p1
Apr  3 09:53:59 aidas-gsi dhcpd: DHCPACK on 192.168.11.3 to d8:80:39:41:d8:21 via p4p1
Apr  3 09:53:59 aidas-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:d7:cc via p4p1
Apr  3 09:53:59 aidas-gsi dhcpd: DHCPOFFER on 192.168.11.5 to d8:80:39:41:d7:cc via p4p1
Apr  3 09:53:59 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.5 (192.168.11.99) from d8:80:39:41:d7:cc via p4p1
Apr  3 09:53:59 aidas-gsi dhcpd: DHCPACK on 192.168.11.5 to d8:80:39:41:d7:cc via p4p1
Apr  3 09:53:59 aidas-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:ba:2b via p4p1
Apr  3 09:53:59 aidas-gsi dhcpd: DHCPOFFER on 192.168.11.8 to d8:80:39:41:ba:2b via p4p1
Apr  3 09:53:59 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.8 (192.168.11.99) from d8:80:39:41:ba:2b via p4p1
Apr  3 09:53:59 aidas-gsi dhcpd: DHCPACK on 192.168.11.8 to d8:80:39:41:ba:2b via p4p1
Apr  3 09:53:59 aidas-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:ba:8a via p4p1
Apr  3 09:53:59 aidas-gsi dhcpd: DHCPOFFER on 192.168.11.1 to d8:80:39:41:ba:8a via p4p1
Apr  3 09:53:59 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.1 (192.168.11.99) from d8:80:39:41:ba:8a via p4p1
Apr  3 09:53:59 aidas-gsi dhcpd: DHCPACK on 192.168.11.1 to d8:80:39:41:ba:8a via p4p1
Apr  3 09:53:59 aidas-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:ba:22 via p4p1
Apr  3 09:53:59 aidas-gsi dhcpd: DHCPOFFER on 192.168.11.2 to d8:80:39:41:ba:22 via p4p1
Apr  3 09:53:59 aidas-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:ee:72 via p4p1
Apr  3 09:53:59 aidas-gsi dhcpd: DHCPOFFER on 192.168.11.6 to d8:80:39:41:ee:72 via p4p1
Apr  3 09:53:59 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.6 (192.168.11.99) from d8:80:39:41:ee:72 via p4p1
Apr  3 09:53:59 aidas-gsi dhcpd: DHCPACK on 192.168.11.6 to d8:80:39:41:ee:72 via p4p1
Apr  3 09:53:59 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.2 (192.168.11.99) from d8:80:39:41:ba:22 via p4p1
Apr  3 09:53:59 aidas-gsi dhcpd: DHCPACK on 192.168.11.2 to d8:80:39:41:ba:22 via p4p1
Apr  3 09:54:00 aidas-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:b4:0c via p4p1
Apr  3 09:54:00 aidas-gsi dhcpd: DHCPOFFER on 192.168.11.7 to d8:80:39:41:b4:0c via p4p1
Apr  3 09:54:00 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.7 (192.168.11.99) from d8:80:39:41:b4:0c via p4p1
Apr  3 09:54:00 aidas-gsi dhcpd: DHCPACK on 192.168.11.7 to d8:80:39:41:b4:0c via p4p1
Apr  3 09:54:00 aidas-gsi rpc.mountd[4324]: authenticated mount request from 192.168.11.4:666 for /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida04 (/home/Embedded/XilinxLinux/ppc_4xx/rfs)
Apr  3 09:54:00 aidas-gsi rpc.mountd[4324]: authenticated mount request from 192.168.11.3:1013 for /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida03 (/home/Embedded/XilinxLinux/ppc_4xx/rfs)
Apr  3 09:54:00 aidas-gsi rpc.mountd[4324]: authenticated mount request from 192.168.11.5:904 for /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida05 (/home/Embedded/XilinxLinux/ppc_4xx/rfs)
Apr  3 09:54:00 aidas-gsi rpc.mountd[4324]: authenticated mount request from 192.168.11.8:890 for /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida08 (/home/Embedded/XilinxLinux/ppc_4xx/rfs)
Apr  3 09:54:00 aidas-gsi rpc.mountd[4324]: authenticated mount request from 192.168.11.1:1007 for /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida01 (/home/Embedded/XilinxLinux/ppc_4xx/rfs)
Apr  3 09:54:00 aidas-gsi rpc.mountd[4324]: authenticated mount request from 192.168.11.6:710 for /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida06 (/home/Embedded/XilinxLinux/ppc_4xx/rfs)
Apr  3 09:54:00 aidas-gsi rpc.mountd[4324]: authenticated mount request from 192.168.11.2:727 for /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida02 (/home/Embedded/XilinxLinux/ppc_4xx/rfs)
Apr  3 09:54:00 aidas-gsi rpc.mountd[4324]: authenticated mount request from 192.168.11.7:698 for /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida07 (/home/Embedded/XilinxLinux/ppc_4xx/rfs)
Apr  3 09:54:00 aidas-gsi dhcpd: DHCPDISCOVER from d8:80:39:42:0d:0c via p4p1
Apr  3 09:54:00 aidas-gsi dhcpd: DHCPOFFER on 192.168.11.9 to d8:80:39:42:0d:0c via p4p1
Apr  3 09:54:00 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.9 (192.168.11.99) from d8:80:39:42:0d:0c via p4p1
Apr  3 09:54:00 aidas-gsi dhcpd: DHCPACK on 192.168.11.9 to d8:80:39:42:0d:0c via p4p1
Apr  3 09:54:00 aidas-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:cf:ac via p4p1
Apr  3 09:54:00 aidas-gsi dhcpd: DHCPOFFER on 192.168.11.12 to d8:80:39:41:cf:ac via p4p1
Apr  3 09:54:00 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.12 (192.168.11.99) from d8:80:39:41:cf:ac via p4p1
Apr  3 09:54:00 aidas-gsi dhcpd: DHCPACK on 192.168.11.12 to d8:80:39:41:cf:ac via p4p1
Apr  3 09:54:00 aidas-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:f6:5a via p4p1
Apr  3 09:54:00 aidas-gsi dhcpd: DHCPOFFER on 192.168.11.11 to d8:80:39:41:f6:5a via p4p1
Apr  3 09:54:00 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.11 (192.168.11.99) from d8:80:39:41:f6:5a via p4p1
Apr  3 09:54:00 aidas-gsi dhcpd: DHCPACK on 192.168.11.11 to d8:80:39:41:f6:5a via p4p1
Apr  3 09:54:00 aidas-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:ee:10 via p4p1
Apr  3 09:54:00 aidas-gsi dhcpd: DHCPOFFER on 192.168.11.10 to d8:80:39:41:ee:10 via p4p1
Apr  3 09:54:00 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.10 (192.168.11.99) from d8:80:39:41:ee:10 via p4p1
Apr  3 09:54:00 aidas-gsi dhcpd: DHCPACK on 192.168.11.10 to d8:80:39:41:ee:10 via p4p1
Apr  3 09:54:01 aidas-gsi rpc.mountd[4324]: authenticated mount request from 192.168.11.9:907 for /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida09 (/home/Embedded/XilinxLinux/ppc_4xx/rfs)














































































... 190 more lines ...
Attachment 2: 030419_usb.png
030419_usb.png
Attachment 3: WRerrorcheck.png
WRerrorcheck.png
Attachment 4: 030419_1051_temp.png
030419_1051_temp.png
Attachment 5: 030419_1052)bias.png
030419_1052)bias.png
Attachment 6: 030519_asiccontrol.png
030519_asiccontrol.png
Attachment 7: 030419_runcontrol.png
030419_runcontrol.png
Attachment 8: 030419_1116_stats.png
030419_1116_stats.png
Attachment 9: 030419_runcontrol_1120.png
030419_runcontrol_1120.png
Attachment 10: 030419_1343_stats.png
030419_1343_stats.png
Attachment 11: 030419_1652_stats10.png
030419_1652_stats10.png
  44   Wed Apr 3 13:29:20 2019 CA, CB, TDreport - high DAQ stall
aida10 runs very fast for a period, then stops.

Have to restart Merger and TapeServer to get it running again.

As shown in attachment 1, aida10 records very high number of bad events.

Waveform for all FEE64s are disabled (see for example attachment 4) *BUT* note that 'wave events' are reported
in aida10 stats (attachment 1)
Attachment 1: 030419_1416_aida10stats.png
030419_1416_aida10stats.png
Attachment 2: 030419_1417_aida09stats.png
030419_1417_aida09stats.png
Attachment 3: 030419_1417_aida11stats.png
030419_1417_aida11stats.png
Attachment 4: 030419_1418_aida12stats.png
030419_1418_aida12stats.png
Attachment 5: 030419_1419_aida10_led.png
030419_1419_aida10_led.png
  98   Tue Nov 19 21:22:06 2019 CA, CB, DK, NH19/20 November 2019 - overnight shift
19 November

Waiting for 40Ar beam to be delivered for tests.
Objective today is to assess degrader thickness suitable to stop 40Ar in 2nd DSSD layer.
Energy expected at 300 MeV/A, around 1E3-4 beam intensity - to be determined.

Tomorrow fragments could be expected.

DSSD / FEE64 pairing as follows

DSSD3 (most downstream): FEEs 9,10,11,12
DSSD2 (middle): FEEs 5,6,7,8
DSSD1 (most upstream): FEEs 1,2,3,4
+plastic scintillator furthest upstream

New AIDA Google spreadsheet created with leakage current, to monitor them during beamtime. First entry added. Bias attached.
https://docs.google.com/spreadsheets/d/12hgbrywB10hGsKt5uc2HnLfZymh8_uvM6cEASbbqnBE/edit#gid=813167023

DESPEC platform has already been moved. No effect on rates - see attached. FEE1 is missing one channel. FEE9, 10 noisy as expected. See attached.


System wide checks *all OK* except
Master clock failed (normal)
ADC calibration fails on 6 and 12. Recalibrated via FADC Align. All OK now.

MBS transfer OK. See attached.

Good event statistics OK. See attached.

FEE64 temperatures OK. See attached.

Merger and data rate OK. See attached.

Waiting for beam.



20 November

01:07 - Still no beam. 

System wide checks OK. 
Stats OK. 
Temps OK. 
MBS OK. 
Merger and data rate OK.


02:35  - Beam is expected soon
Tried enabling writing data to disk. Merger crashed. FEE64 stopped responding to commands.
Reset Merger. FEE64 now respond to commands.
Tried enabling writing to disk again. Merger crashed once more. Reset merger.
Reset Tape Server interface. Now writing to disk.

No data currently being sent to MBS, but operators say they "are adding a new detector". May be normal.

DAQ & Merger going OK. Writing to Nov19/R1

No beam yet.

Temperatures OK
System wide checks OK
Stats OK. See attached.
MBS not working.
Merger and data rate OK. See attached.

~02:48 Beam on. 40Ar. 300 MeV/A, 500 pps. Degrader settings unclear.

03:54 Degrader settings have been changing between 4.5, 1 and 6 ug/cm2
Currently on 6 g/cm2.
AIDA seen through MBS shows no HE on detector 3, and Det 1 and 2 show no counts on positive y (FEE6 and 1). That is not the case from MIDAS.

04:30 The aim of the tests above was to have no 40Ar at all in DSSD3. It does not appear that 6 g/cm2 is sufficient, as we still see events. The reduction between 1 & 2 vs. 3 is about a factor of 2-10 depending on ADC channel. See attached.

~04:30 Moved to 6.2 g/cm2 degrader. See attached.

~04:45 Moved to 5.8 g/cm2. See attached.

~05:09 Moved to 6.4 g/cm2. See attached.

05:31 High energy channel spectra (1.8.H) for all FEE64 w/ 6.4 g/cm2 degrader - See attachments 13 & 14

05:38: Moved to 1.0 g/cm2. See attached. Difference is not huge which may indicate these spectra are misleading, as partly expected. A more detailed analysis will be carried out off line.

06:00: Beam stopped. ~R77. AIDA keeps running background over the day. Beam should restart around 11 pm later today.
Attachment 1: Bias2235.png
Bias2235.png
Attachment 2: Rates2236.png
Rates2236.png
Attachment 3: 19112241MBS.png
19112241MBS.png
Attachment 4: 19112242Stats.png
19112242Stats.png
Attachment 5: 19112244Temperatures.png
19112244Temperatures.png
Attachment 6: 19112246Merger.png
19112246Merger.png
Attachment 7: 20110239.png
20110239.png
Attachment 8: 20110240DataTransfer.png
20110240DataTransfer.png
Attachment 9: 6gdegrader.png
6gdegrader.png
Attachment 10: 6.2degrader.png
6.2degrader.png
Attachment 11: 5.8degrader.png
5.8degrader.png
Attachment 12: 6.4degrader.png
6.4degrader.png
Attachment 13: 18h_0530.png
18h_0530.png
Attachment 14: 18H_0528.png
18H_0528.png
Attachment 15: 1gcm2degrader.png
1gcm2degrader.png
  99   Wed Nov 20 20:36:14 2019 CA, CB, DK, NH20/21 November Night Shift
21:35 (NH) - A few merger timestamp errors were appearing - Decided to powercycle all FEE64s, all checks OK.
 No Screenshot due to forgetting before I closed merger 
 FEEs were often not stopping properly and needing merger reset as well.

Beam Plan:

40Ar 300 MeV/u Primary Beam
and/or
34Si Fragment from Ar

22:45

FEES were restart near these timestamps in the TapeData, evidently
-rw-rw-r--.  1 npg  npg  2.0G Nov 20 21:17 R1_361
-rw-rw-r--.  1 npg  npg  2.0G Nov 20 21:36 R1_362

Run number was kept the same (Tape Server was not stopped @ FEE restart?)

All system wide checks are okay (except of course the master clock fails)
Temps look okay (attachment 1, dated 21/Nov by mistake)
Leak currents all around 0.9 uA as before (attachment 2)
Stats are behaved. We don't have beam yet (attachment 3)
Merger and tape server happy (see attachment 4)
Data rate histos while beam is still off (see attachment 5)

==Nov 21==

0:00
TD noted that the data rates have gone up at some point (about double).  
Now we increased the slow comparator thresholds from 0xa to 0xb (data rate drops from 24 Mb/s to 16 Mb/s).
We increased it again from 0xb to 0xc, but near that time the beam was also turned on, and the data rate was about 25 Mb/s.

Beam was turned on near R1_472 or a bit earlier.

TD requested waveforms, but at the moment they are not appearing.  Will enable WFs after a condition change.

Beam seen in all three DSSSDs, see attachment 6.  Beam was turned off before I could get more screencaps.

Degrader change ... near R1_481

00:32 waveforms now enabled under Run Control.  They also appear to be enabled on ASIC1 undre LED and Waveform controls.  However, we are unable to see the waveforms in the relevant histograms.

00:44 3 g/cm^2 inserted R1_488

00:45 3.2 g/cm^2 degrader in R1_489.

1:00 Several degraders and combinations are being tested since last note.

1:10 It seems 1 g/cm^2 of energy loss is not accounted for by LISE++.  Trying to understand the reason.

1:28 Satisfied with a total degrader thickness of 2.6 g/cm^2 -- seems to calculate that fragments should implant mainly in DSSSD2?  Issue seems to be resolved updating the LISE++ file.
  Near R1_507

Can see the implant samples attached as #7 and #8

1:43 According to the stats, most of the beam energy is being deposited in DSSSD1.  See #9.
  Present beam burst repetition is around 0.1 Hz (each ~10 sec)

2:00 better example of stats histos (see #10)

2:45 tuning the FRS degraders (ones for RIB separation rather than AIDA implantation depth).

3:17 Just started fragments.  Disregard previous filenames with "34Si".  Finally no degrader in the FRS was needed to get Z separation.
  R1_559
  As the DAQs are not synchronized, a quick call to the BNC PB-5 pulser shifiting from 2 -> 50 -> 2 Hz was used to make a signal burst.
  S4 degrader is 5.2 g/cm^2

3:22 Updated biases in Google Sheets.

Having trouble seeing the fragments in the AIDA histograms.  Since we just started fragments, zero the histos.

3:35 Attachment 11, with 34Si fragments as Stats.  Difficult to click Update on the Rates at the right time.  
   From the beam monitors, fragments should be at around 3000 cps.

3:45 (Up to now, some optics tuning with slits, etc)

3:46 Optics settings accepted, AIDA at R1_572, accept this condition for data runs on ~34Si fragments.

3:47 Sent a pulser burst 2 -> 50 -> 2 Hz

3:50 zero histograms again.
   Temperatures attachment 12
   Biases attachment 13
   Statistics attachment 14
   Merger / Tape Server attachment 15

3:56   
   StatHistos attachment 16

4:09 FRS starts new run #56, AIDA R1_583.  Pulser 2-> 50 -> 2 Hz

4:56 Stat histograms in attachment 17.  Not zeroed since noted at 3:50
Attachment 1: 21-11-temps.png
21-11-temps.png
Attachment 2: 20-11-biases.png
20-11-biases.png
Attachment 3: 20-11-stats.png
20-11-stats.png
Attachment 4: 20-11-merger-ts.png
20-11-merger-ts.png
Attachment 5: 20-11-statshistos.png
20-11-statshistos.png
Attachment 6: 21-11-layout5histos.png
21-11-layout5histos.png
Attachment 7: 34si-implants-130-1.png
34si-implants-130-1.png
Attachment 8: 34si-implants-130-2.png
34si-implants-130-2.png
Attachment 9: 34si-stats-histos.png
34si-stats-histos.png
Attachment 10: stats-histos-2am.png
stats-histos-2am.png
Attachment 11: 34si-fragments-stats.png
34si-fragments-stats.png
Attachment 12: temperatures-fragments.png
temperatures-fragments.png
Attachment 13: biases-fragments.png
biases-fragments.png
Attachment 14: stats-fragments.png
stats-fragments.png
Attachment 15: merger-tapeserver-fragments.png
merger-tapeserver-fragments.png
Attachment 16: stathistos-fragments.png
stathistos-fragments.png
Attachment 17: stathistos-5am.png
stathistos-5am.png
ELOG V3.1.4-unknown