AIDA GELINA BRIKEN nToF CRIB ISOLDE CIRCE nTOFCapture DESPEC DTAS EDI_PSA 179Ta CARME StellarModelling DCF K40
  DESPEC, Page 37 of 37  ELOG logo
ID Date Author Subjectdown
  614   Sat Apr 27 23:04:42 2024 Marc00:00-08:00 Sunday 28 April

Encore un matin, un matin pour ..... a shift !!!ramping up again ! S4 temperature not really going down either: 25C

0:40 AIDA04 is down. I informed the DAQ shift crew and attempting to restart AIDA04.

After rebooting AIDA04 all seemed going until I reached the merger check. No data was being merged.

I realise then that AIDA01 data transfer was not active. I did not spot it when I enabled the data transfer for all.

I stopped the daq and rebooted AIDA01 following all the instructions in the ELOG and this worked.

We are now back in business with AIDA back in main DAQ at  01:40

01:45 full checks:

           Bias: AIDA-Graphana - Attachment 1

               Leakage current ramping up again ! S4 temperaturenot really going down either: 25C

      FEE64 temperatures OK - attachment 2

      ADC data item stats - attachment 3

      per FEE64 Rate spectra - attachment 4

      Merger etc - attachment 5

      ucesb - attachment 6

03:00 full checks

          No change to be highlighted. All pretty much the same as during previous full checks.

05:00 full checks

Expt still running very smoothly.

           Bias: AIDA-Graphana - Attachment 7-8

               Leakage current going down slightly

      FEE64 temperatures OK - attachment 9

      ADC data item stats - attachment 10
ramping up again ! S4 temperaturenot really going down either: 25C
      per FEE64 Rate spectra - attachment 11

      Merger etc - attachment 12

      ucesb - attachment 13

07:15 all running smoothly.

07:20 Spoke too fast AIDA DAQ is down. DAQ shift crew informed.

07:40: AIDA10 was rebooted successfully and AIDA is now back in the main DAQ.

 

Attachment 1: Screenshot_from_2024-04-28_01-51-37.png
Screenshot_from_2024-04-28_01-51-37.png
Attachment 2: Screenshot_from_2024-04-28_01-53-29.png
Screenshot_from_2024-04-28_01-53-29.png
Attachment 3: Screenshot_from_2024-04-28_01-53-52.png
Screenshot_from_2024-04-28_01-53-52.png
Attachment 4: Screenshot_from_2024-04-28_01-54-16.png
Screenshot_from_2024-04-28_01-54-16.png
Attachment 5: Screenshot_from_2024-04-28_01-56-14.png
Screenshot_from_2024-04-28_01-56-14.png
Attachment 6: Screenshot_from_2024-04-28_04-59-46.png
Screenshot_from_2024-04-28_04-59-46.png
Attachment 7: Screenshot_from_2024-04-28_04-59-51.png
Screenshot_from_2024-04-28_04-59-51.png
Attachment 8: Screenshot_from_2024-04-28_05-00-33.png
Screenshot_from_2024-04-28_05-00-33.png
Attachment 9: Screenshot_from_2024-04-28_05-00-25.png
Screenshot_from_2024-04-28_05-00-25.png
Attachment 10: Screenshot_from_2024-04-28_05-00-38.png
Screenshot_from_2024-04-28_05-00-38.png
Attachment 11: Screenshot_from_2024-04-28_05-02-44.png
Screenshot_from_2024-04-28_05-02-44.png
Attachment 12: Screenshot_from_2024-04-28_05-00-03.png
Screenshot_from_2024-04-28_05-00-03.png
  611   Fri Apr 26 23:04:51 2024 Marc00:00-08:00 Saturday 27 April

It's a new day, it's a new shift and I'm feeeeeliiiiing goooood !!!!!

0.15 Checks:
      DSSSD bias & leakage current - Grafana - attachments 1-2
       leakage current dropped a bit since the evening shift ~22:00.

      FEE64 temperatures OK - attachment 3

      ADC data item stats - attachment 4

      per FEE64 Rate spectra - attachment 5

      Merger etc - attachment 6

      ucesb - attachment 7

2:20 Nothing new to report. All good. Leakage currents continue to go down slowly. See attachement 8 to 12

3:30 Still running smoothly. 

5:20 AIDA 4 stop counting in the last 20 min. I informed the main DAQ and they started a new run without AIDA.

        DAQ restarted OK. Main DAQ informed.

 

Attachment 1: Screenshot_from_2024-04-27_00-17-00.png
Screenshot_from_2024-04-27_00-17-00.png
Attachment 2: Screenshot_from_2024-04-27_00-16-16.png
Screenshot_from_2024-04-27_00-16-16.png
Attachment 3: Screenshot_from_2024-04-27_00-18-49.png
Screenshot_from_2024-04-27_00-18-49.png
Attachment 4: Screenshot_from_2024-04-27_00-19-05.png
Screenshot_from_2024-04-27_00-19-05.png
Attachment 5: Screenshot_from_2024-04-27_00-19-19.png
Screenshot_from_2024-04-27_00-19-19.png
Attachment 6: Screenshot_from_2024-04-27_00-21-44.png
Screenshot_from_2024-04-27_00-21-44.png
Attachment 7: Screenshot_from_2024-04-27_00-21-00.png
Screenshot_from_2024-04-27_00-21-00.png
Attachment 8: Screenshot_from_2024-04-27_02-32-22.png
Screenshot_from_2024-04-27_02-32-22.png
Attachment 9: Screenshot_from_2024-04-27_02-37-20.png
Screenshot_from_2024-04-27_02-37-20.png
Attachment 10: Screenshot_from_2024-04-27_02-37-31.png
Screenshot_from_2024-04-27_02-37-31.png
Attachment 11: Screenshot_from_2024-04-27_02-40-39.png
Screenshot_from_2024-04-27_02-40-39.png
Attachment 12: Screenshot_from_2024-04-27_02-41-13.png
Screenshot_from_2024-04-27_02-41-13.png
  649   Mon Jun 10 23:36:14 2024 TD0000-08:00 Tuesday 11 June
00.09 DSSSD bias & leakage current OK - attachment 1

      FEE64 temperatures OK - attachment 2

      ADC data items stats OK - attachment 3
   
      per FEE64 Rate spectra - attachment 4

      per p+n FEE64 1.8.L spectra - attachments 5-6
       aida09 pulser peak width 53 ch FWHM 

      per FEE64 1.8.H spectra - attachments 7-8

      per n+n FEE64 1.8.W spectra - 20us FSR - attachment 9
       p+n 1.8.W not running currently

      Merger, TapeServer etc - attachments 10-11

      analysis data file S181/R4_427 - attachment 12
       max FEE64 deadtime 1.7%
       HEC data rate 3.1kHz

00.28 all histograms zero'd


02.39 DAQ continues data file S181/R4_460
      Beam off since 02.08

      DSSSD bias & leakage current OK - attachment 13

      FEE64 temperatures OK - attachment 14

      ADC data items stats OK - attachment 15
       9x < 20k, max 239k aida08
   
      per FEE64 Rate spectra - attachment 16


07.15 DAQ continues data file S181/R4_517
      Beam off since 02.08
      HKR report c 3-4 h until UNILAC fixed

      DSSSD bias & leakage current OK - attachment 17

      FEE64 temperatures OK - attachment 18

      ADC data items stats OK - attachment 19
       8x < 20k, max 238k aida08
   
      per FEE64 Rate spectra - attachment 20
Attachment 1: Screenshot_from_2024-06-11_00-09-43.png
Screenshot_from_2024-06-11_00-09-43.png
Attachment 2: Screenshot_from_2024-06-11_00-10-36.png
Screenshot_from_2024-06-11_00-10-36.png
Attachment 3: Screenshot_from_2024-06-11_00-11-13.png
Screenshot_from_2024-06-11_00-11-13.png
Attachment 4: Screenshot_from_2024-06-11_00-11-45.png
Screenshot_from_2024-06-11_00-11-45.png
Attachment 5: Screenshot_from_2024-06-11_00-13-25.png
Screenshot_from_2024-06-11_00-13-25.png
Attachment 6: Screenshot_from_2024-06-11_00-16-14.png
Screenshot_from_2024-06-11_00-16-14.png
Attachment 7: Screenshot_from_2024-06-11_00-24-51.png
Screenshot_from_2024-06-11_00-24-51.png
Attachment 8: Screenshot_from_2024-06-11_00-25-44.png
Screenshot_from_2024-06-11_00-25-44.png
Attachment 9: Screenshot_from_2024-06-11_00-27-29.png
Screenshot_from_2024-06-11_00-27-29.png
Attachment 10: Screenshot_from_2024-06-11_00-29-21.png
Screenshot_from_2024-06-11_00-29-21.png
Attachment 11: Screenshot_from_2024-06-11_00-29-41.png
Screenshot_from_2024-06-11_00-29-41.png
Attachment 12: R4_427
 *** TDR format 3.3.0 analyser - TD - May 2021
 *** ERROR: READ I/O error:       5002
                   blocks:      32000
          ADC data format:  256031206 ( 1090943.7 Hz)
        Other data format:    5888794 (   25092.0 Hz)
 Sample trace data format:          0 (       0.0 Hz)
         Undefined format:          0 (       0.0 Hz)
   Other data format type:      PAUSE:        689 (       2.9 Hz)
                               RESUME:        689 (       2.9 Hz)
                              SYNC100:      32874 (     140.1 Hz)
                              WR48-63:      32874 (     140.1 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:    5821668 (   24806.0 Hz)
                           Other info:          0 (       0.0 Hz)

   ADC data range bit set:     725584 (    3091.7 Hz)

                Timewarps:        ADC:          0 (       0.0 Hz)
                                PAUSE:          0 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:          0 (       0.0 Hz)
                              WR48-63:          0 (       0.0 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:          0 (       0.0 Hz)
                            Undefined:          0 (       0.0 Hz)
                         Sample trace:          0 (       0.0 Hz)

 *** Timestamp elapsed time:      234.688 s
 FEE  elapsed dead time(s) elapsed idle time(s)
  0                0.082                0.000
  1                0.081                0.000
  2                0.099                0.000
  3                1.580                0.000
  4                0.860                0.000
  5                0.325                0.000
  6                0.000                0.000
  7                4.102                0.000
  8                0.020                0.000
  9                2.330                0.000
 10                0.170                0.000
 11                0.000                0.000
 12                0.000                0.000
 13                0.000                0.000
 14                0.000                0.000
 15                0.981                0.000
 16                0.000                0.000
 17                0.000                0.000
 18                0.000                0.000
 19                0.000                0.000
 20                0.000                0.000
 21                0.000                0.000
 22                0.000                0.000
 23                0.000                0.000
 24                0.000                0.000
 25                0.000                0.000
 26                0.000                0.000
 27                0.000                0.000
 28                0.000                0.000
 29                0.000                0.000
 30                0.000                0.000
 31                0.000                0.000
 32                0.000                0.000

 *** Statistics
 FEE  ADC Data Other Data     Sample  Undefined      Pause     Resume    SYNC100    WR48-63       Disc        MBS      Other   HEC Data
  0    3851535       8075          0          0          3          3        517        517          0       7035          0      14311
  1   10547580       2760          0          0          9          9       1371       1371          0          0          0      35298
  2    7097044    2015426          0          0          1          1       1095       1095          0    2013234          0      19431
  3   24895003       6544          0          0        192        192       3080       3080          0          0          0      27887
  4    3219755     585586          0          0          4          4        493        493          0     584592          0      17915
  5   19996310       5104          0          0         46         46       2506       2506          0          0          0      32222
  6    5825374     446331          0          0          0          0        741        741          0     444849          0      18892
  7   61314555      15906          0          0        191        191       7762       7762          0          0          0     429462
  8    8374860       2050          0          0          1          1       1024       1024          0          0          0      21845
  9   43291179    2005473          0          0        168        168       5805       5805          0    1993527          0      21309
 10   13902520       3510          0          0          6          6       1749       1749          0          0          0      12563
 11    2813051     779317          0          0          0          0        443        443          0     778431          0      21093
 12    5093380       1266          0          0          0          0        633        633          0          0          0      11468
 13    7142579       1774          0          0          0          0        887        887          0          0          0      18724
 14    3712134        902          0          0          0          0        451        451          0          0          0      12892
 15   34954347       8770          0          0         68         68       4317       4317          0          0          0      10272
 16          0          0          0          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0          0          0          0

 *** Timewarps
 FEE       ADC      Pause     Resume    SYNC100    WR48-63       Disc        MBS  Undefined    Samples
  0          0          0          0          0          0          0          0          0          0
  1          0          0          0          0          0          0          0          0          0
  2          0          0          0          0          0          0          0          0          0
  3          0          0          0          0          0          0          0          0          0
  4          0          0          0          0          0          0          0          0          0
  5          0          0          0          0          0          0          0          0          0
  6          0          0          0          0          0          0          0          0          0
  7          0          0          0          0          0          0          0          0          0
  8          0          0          0          0          0          0          0          0          0
  9          0          0          0          0          0          0          0          0          0
 10          0          0          0          0          0          0          0          0          0
 11          0          0          0          0          0          0          0          0          0
 12          0          0          0          0          0          0          0          0          0
 13          0          0          0          0          0          0          0          0          0
 14          0          0          0          0          0          0          0          0          0
 15          0          0          0          0          0          0          0          0          0
 16          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0

 *** Program elapsed time:   34.627s (  924.132 blocks/s,  57.758 Mb/s)
Attachment 13: Screenshot_from_2024-06-11_02-36-59.png
Screenshot_from_2024-06-11_02-36-59.png
Attachment 14: Screenshot_from_2024-06-11_02-41-01.png
Screenshot_from_2024-06-11_02-41-01.png
Attachment 15: Screenshot_from_2024-06-11_02-41-50.png
Screenshot_from_2024-06-11_02-41-50.png
Attachment 16: Screenshot_from_2024-06-11_02-42-40.png
Screenshot_from_2024-06-11_02-42-40.png
Attachment 17: Screenshot_from_2024-06-11_07-13-28.png
Screenshot_from_2024-06-11_07-13-28.png
Attachment 18: Screenshot_from_2024-06-11_07-15-41.png
Screenshot_from_2024-06-11_07-15-41.png
Attachment 19: Screenshot_from_2024-06-11_07-16-08.png
Screenshot_from_2024-06-11_07-16-08.png
Attachment 20: Screenshot_from_2024-06-11_07-16-35.png
Screenshot_from_2024-06-11_07-16-35.png
  589   Mon Apr 22 22:35:59 2024 TD00.00-08.00 Tuesday 22 April
23.36 Analysis of data file R17_34 - attachment 1
      max deadtime c. 1% (aida02)
      no timewarps

23.38 DSSSD bias & leakage current OK - attachment 2

      FEE64 temperatures OK - attachment 3
       aida02 ASIC temp - known fault

      ADC data item stats - attachment 4

      per FEE64 Rate spectra - attachment 5

      Merger etc - attachment 6

01.03 DSSSD bias & leakage current OK - attachment 7

      FEE64 temperatures OK - attachment 8
       aida02 ASIC temp - known fault

      ADC data item stats - attachment 9

      per FEE64 Rate spectra - attachment 10

      Merger etc - attachment 11

03.06 DSSSD bias & leakage current OK - attachment 12

      FEE64 temperatures OK - attachment 13
       aida02 ASIC temp - known fault

      ADC data item stats - attachment 14

      per FEE64 Rate spectra - attachment 15

      Merger etc - attachment 16

05.04 DSSSD bias & leakage current OK - attachment 17

      FEE64 temperatures OK - attachment 18
       aida02 ASIC temp - known fault

      ADC data item stats - attachment 19

      per FEE64 Rate spectra - attachment 20

      Merger etc - attachment 21

07.29 DSSSD bias & leakage current OK - attachment 22

      FEE64 temperatures OK - attachment 23
       aida02 ASIC temp - known fault

      ADC data item stats - attachment 24

      per FEE64 Rate spectra - attachment 25

      Merger etc - attachment 26
Attachment 1: R17_34
 *** TDR format 3.3.0 analyser - TD - May 2021
 *** ERROR: READ I/O error:       5002
                   blocks:      32000
          ADC data format:  242086746 (  241481.1 Hz)
        Other data format:   19833254 (   19783.6 Hz)
 Sample trace data format:          0 (       0.0 Hz)
         Undefined format:          0 (       0.0 Hz)
   Other data format type:      PAUSE:        347 (       0.3 Hz)
                               RESUME:        347 (       0.3 Hz)
                              SYNC100:      35735 (      35.6 Hz)
                              WR48-63:      35735 (      35.6 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:   19761090 (   19711.7 Hz)
                           Other info:          0 (       0.0 Hz)

   ADC data range bit set:     676877 (     675.2 Hz)

                Timewarps:        ADC:          0 (       0.0 Hz)
                                PAUSE:          0 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:          0 (       0.0 Hz)
                              WR48-63:          0 (       0.0 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:          0 (       0.0 Hz)
                            Undefined:          0 (       0.0 Hz)
                         Sample trace:          0 (       0.0 Hz)

 *** Timestamp elapsed time:     1002.508 s
 FEE  elapsed dead time(s) elapsed idle time(s)
  0                0.000                0.000
  1                7.276                0.000
  2                0.000                0.000
  3                3.384                0.000
  4                0.000                0.000
  5                0.005                0.000
  6                0.000                0.000
  7                0.131                0.000
  8                0.000                0.000
  9                0.000                0.000
 10                0.000                0.000
 11                0.000                0.000
 12                0.000                0.000
 13                0.000                0.000
 14                0.000                0.000
 15                0.020                0.000
 16                0.000                0.000
 17                0.000                0.000
 18                0.000                0.000
 19                0.000                0.000
 20                0.000                0.000
 21                0.000                0.000
 22                0.000                0.000
 23                0.000                0.000
 24                0.000                0.000
 25                0.000                0.000
 26                0.000                0.000
 27                0.000                0.000
 28                0.000                0.000
 29                0.000                0.000
 30                0.000                0.000
 31                0.000                0.000
 32                0.000                0.000

 *** Statistics
 FEE  ADC Data Other Data     Sample  Undefined      Pause     Resume    SYNC100    WR48-63       Disc        MBS      Other   HEC Data
  0    2611636      30762          0          0          0          0        342        342          0      30078          0      64523
  1   15105593     133663          0          0        117        117       1949       1949          0     129531          0      31773
  2    4737354    9053995          0          0          0          0       1811       1811          0    9050373          0      45719
  3   93081574    8973436          0          0        187        187      14170      14170          0    8944722          0      56135
  4    2149658     687339          0          0          0          0        339        339          0     686661          0      47171
  5   16270251     924233          0          0          3          3       2251       2251          0     919725          0      42612
  6    4771409       1270          0          0          0          0        635        635          0          0          0      12858
  7   64665607      18074          0          0         39         39       8998       8998          0          0          0     197755
  8    1902284        428          0          0          0          0        214        214          0          0          0      25268
  9    4387089       1362          0          0          0          0        681        681          0          0          0      10256
 10    8523582       2388          0          0          0          0       1194       1194          0          0          0       4627
 11    2287700        576          0          0          0          0        288        288          0          0          0      83305
 12    4391198       1112          0          0          0          0        556        556          0          0          0      12633
 13    3103598        860          0          0          0          0        430        430          0          0          0      15208
 14    1965500        506          0          0          0          0        253        253          0          0          0      10217
 15   12132713       3250          0          0          1          1       1624       1624          0          0          0      16817
 16          0          0          0          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0          0          0          0

 *** Timewarps
 FEE       ADC      Pause     Resume    SYNC100    WR48-63       Disc        MBS  Undefined    Samples
  0          0          0          0          0          0          0          0          0          0
  1          0          0          0          0          0          0          0          0          0
  2          0          0          0          0          0          0          0          0          0
  3          0          0          0          0          0          0          0          0          0
  4          0          0          0          0          0          0          0          0          0
  5          0          0          0          0          0          0          0          0          0
  6          0          0          0          0          0          0          0          0          0
  7          0          0          0          0          0          0          0          0          0
  8          0          0          0          0          0          0          0          0          0
  9          0          0          0          0          0          0          0          0          0
 10          0          0          0          0          0          0          0          0          0
 11          0          0          0          0          0          0          0          0          0
 12          0          0          0          0          0          0          0          0          0
 13          0          0          0          0          0          0          0          0          0
 14          0          0          0          0          0          0          0          0          0
 15          0          0          0          0          0          0          0          0          0
 16          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0

 *** Program elapsed time:   30.539s ( 1047.838 blocks/s,  65.490 Mb/s)
Attachment 2: Screenshot_from_2024-04-22_23-38-08.png
Screenshot_from_2024-04-22_23-38-08.png
Attachment 3: Screenshot_from_2024-04-22_23-39-17.png
Screenshot_from_2024-04-22_23-39-17.png
Attachment 4: Screenshot_from_2024-04-22_23-39-44.png
Screenshot_from_2024-04-22_23-39-44.png
Attachment 5: Screenshot_from_2024-04-22_23-40-27.png
Screenshot_from_2024-04-22_23-40-27.png
Attachment 6: Screenshot_from_2024-04-22_23-40-54.png
Screenshot_from_2024-04-22_23-40-54.png
Attachment 7: Screenshot_from_2024-04-23_01-04-32.png
Screenshot_from_2024-04-23_01-04-32.png
Attachment 8: Screenshot_from_2024-04-23_01-05-40.png
Screenshot_from_2024-04-23_01-05-40.png
Attachment 9: Screenshot_from_2024-04-23_01-06-12.png
Screenshot_from_2024-04-23_01-06-12.png
Attachment 10: Screenshot_from_2024-04-23_01-06-40.png
Screenshot_from_2024-04-23_01-06-40.png
Attachment 11: Screenshot_from_2024-04-23_01-07-18.png
Screenshot_from_2024-04-23_01-07-18.png
Attachment 12: Screenshot_from_2024-04-23_03-06-28.png
Screenshot_from_2024-04-23_03-06-28.png
Attachment 13: Screenshot_from_2024-04-23_03-07-02.png
Screenshot_from_2024-04-23_03-07-02.png
Attachment 14: Screenshot_from_2024-04-23_03-07-35.png
Screenshot_from_2024-04-23_03-07-35.png
Attachment 15: Screenshot_from_2024-04-23_03-08-09.png
Screenshot_from_2024-04-23_03-08-09.png
Attachment 16: Screenshot_from_2024-04-23_03-08-43.png
Screenshot_from_2024-04-23_03-08-43.png
Attachment 17: Screenshot_from_2024-04-23_05-02-11.png
Screenshot_from_2024-04-23_05-02-11.png
Attachment 18: Screenshot_from_2024-04-23_05-03-10.png
Screenshot_from_2024-04-23_05-03-10.png
Attachment 19: Screenshot_from_2024-04-23_05-03-57.png
Screenshot_from_2024-04-23_05-03-57.png
Attachment 20: Screenshot_from_2024-04-23_05-04-37.png
Screenshot_from_2024-04-23_05-04-37.png
Attachment 21: Screenshot_from_2024-04-23_05-05-13.png
Screenshot_from_2024-04-23_05-05-13.png
Attachment 22: Screenshot_from_2024-04-23_07-29-15.png
Screenshot_from_2024-04-23_07-29-15.png
Attachment 23: Screenshot_from_2024-04-23_07-29-59.png
Screenshot_from_2024-04-23_07-29-59.png
Attachment 24: Screenshot_from_2024-04-23_07-30-27.png
Screenshot_from_2024-04-23_07-30-27.png
Attachment 25: Screenshot_from_2024-04-23_07-30-54.png
Screenshot_from_2024-04-23_07-30-54.png
Attachment 26: Screenshot_from_2024-04-23_07-31-16.png
Screenshot_from_2024-04-23_07-31-16.png
  661   Thu Jun 13 23:04:01 2024 Dan Judson, TD0.00-8.00 14/6/24

0.00 checks - all looks ok

Voltage/currents - attachment 1

Rates attachment 2 + 3

Temps - attachment 4

Merger - attachment 5

ucesb - attachment 6

Grafana - attachment 7

 

1.30 -aida1 in merger Links, aida02 in rate spectra, has stopped - resetting DAQ.

Datafile R7_961 being written when stopped

1.40 - seems ok

 

1.50 checks - seems ok

Voltage/currents - attachment 8

Rates attachment 9 + 10

Temps - attachment 11

Merger - attachment 12

ucesb - attachment 13

Grafana - attachment 14

 

3.30 -aida1 in merger Links, aida02 in rate spectra, has stopped again - resetting DAQ.

Datafile R7_1010 being written when stopped

3.38 - seems ok

 

3.40 checks - all looks ok

Voltage/currents - attachment 15

Rates attachment 16 + 17

Temps - attachment 18

Merger - attachment 19

ucesb - attachment 20

Grafana - attachment 21

 

Zeroed spectra - attachments 22-28

 

 

5.39 -aida1 in merger Links, aida02 in rate spectra, has stopped again - resetting DAQ.

Datafile R7_1062 being written when stopped

AIDA out of the main data

Could not get it to restart folowing the notes on the elog - SOAP errors as shown in attachment 29. called Tom who''s logged in 

6.10 - seems to be working again. Not sure what happened. Tom thinks multiple restarts fixed it

Started data file R8

AIDA back in the main data

 

FRS shifters report a problem with the AIDA white rabbit time stamp - attachment 30

The following are timestamp values from each of the FEEs taken in sequence
If time does not increase in a reasonable manner run the system wide checks

aida01 : White Rabbit=>  17D8C4ED 8F46CF7B , WR/10=>   2627A17C18714BF, Readout Time =>   2627A17C2A74000
aida02 : White Rabbit=>  17D8C4ED A0A1B623 , WR/10=>   2627A17C3435F03, Readout Time =>   2627A17C4410000
aida03 : White Rabbit=>  17D8C4ED AFDACE68 , WR/10=>   2627A17C4C914A4, Readout Time =>   2627A17C5D3C000
aida04 : White Rabbit=>  17D8C4ED C0516FC2 , WR/10=>   2627A17C66E8B2D, Readout Time =>   2627A17C7AC4000
aida05 : White Rabbit=>  17D8C4ED D3B93DAB , WR/10=>   2627A17C85F52F7, Readout Time =>   2627A17C9AEC000
aida06 : White Rabbit=>  17D8C4ED E8C9B243 , WR/10=>   2627A17CA7A91D3, Readout Time =>   2627A17CB704000
aida07 : White Rabbit=>  17D8C4ED FFE475FD , WR/10=>   2627A17CCCA0BCC, Readout Time =>   2627A17CDD94000
aida08 : White Rabbit=>  17D8C4EE 103232A6 , WR/10=>   2627A17CE6B6B77, Readout Time =>   2627A17CF7BC000
aida09 : White Rabbit=>  17D8C4EE 20A1F612 , WR/10=>   2627A17D0103235, Readout Time =>   2627A17D112C000
aida10 : White Rabbit=>  17D8C4EE 307D8FE7 , WR/10=>   2627A17D1A627FD, Readout Time =>   2627A17D2958000
aida11 : White Rabbit=>  17D8C4EE 4402829D , WR/10=>   2627A17D399D9DC, Readout Time =>   2627A17D4570000
aida12 : White Rabbit=>  17D8C4EE 6288B04D , WR/10=>   2627A17D6A744D4, Readout Time =>   2627A17D7D94000
aida13 : White Rabbit=>  17D8C4EE 751E1306 , WR/10=>   2627A17D88301E7, Readout Time =>   2627A17D9C34000
aida14 : White Rabbit=>  17D8C4EE 894E8806 , WR/10=>   2627A17DA87DA67, Readout Time =>   2627A17DBFF4000
aida15 : White Rabbit=>  17D8C4EE 9EEEB997 , WR/10=>   2627A17DCB178F5, Readout Time =>   2627A17DDC18000
aida16 : White Rabbit=>  17D8C4EE AF0BC923 , WR/10=>   2627A17DE4DFA83, Readout Time =>   2627A17DF5C8000

Tom thinks everything appears ok with AIDA

Have converted WR timestamp to data/time - see attachment 40

At the ~second timescale AIDA WR timestamp looks OK, i.e. no gross errors. Will need to check correlations with other detector sub-systems to confirm WR on the ~us time timescale.. 

However, ~10us offset reported by DESPEC online crew is correct (AIDA timestamps data at a later point in the signal processing cycle than other detector sub-systems). Have suggested online crew contact Nic or Calum to check what the AIDA WR timediff spectra should look like.

 

 

7.00 checks - all looks ok

Voltage/currents - attachment 31

Rates attachment 32 + 33

Temps - attachment 36

Merger - attachment 37

ucesb - attachment 39

Grafana - attachment 38

 

Handed over to Tom

 

Attachment 1: Screenshot_from_2024-06-13_23-58-44.png
Screenshot_from_2024-06-13_23-58-44.png
Attachment 2: Screenshot_from_2024-06-13_23-59-34.png
Screenshot_from_2024-06-13_23-59-34.png
Attachment 3: Screenshot_from_2024-06-14_00-00-09.png
Screenshot_from_2024-06-14_00-00-09.png
Attachment 4: Screenshot_from_2024-06-14_00-00-43.png
Screenshot_from_2024-06-14_00-00-43.png
Attachment 5: Screenshot_from_2024-06-14_00-01-36.png
Screenshot_from_2024-06-14_00-01-36.png
Attachment 6: Screenshot_from_2024-06-14_00-02-45.png
Screenshot_from_2024-06-14_00-02-45.png
Attachment 7: Screenshot_from_2024-06-14_00-03-26.png
Screenshot_from_2024-06-14_00-03-26.png
Attachment 8: Screenshot_from_2024-06-14_01-47-38.png
Screenshot_from_2024-06-14_01-47-38.png
Attachment 9: Screenshot_from_2024-06-14_01-48-12.png
Screenshot_from_2024-06-14_01-48-12.png
Attachment 10: Screenshot_from_2024-06-14_01-48-45.png
Screenshot_from_2024-06-14_01-48-45.png
Attachment 11: Screenshot_from_2024-06-14_01-49-16.png
Screenshot_from_2024-06-14_01-49-16.png
Attachment 12: Screenshot_from_2024-06-14_01-49-48.png
Screenshot_from_2024-06-14_01-49-48.png
Attachment 13: Screenshot_from_2024-06-14_01-50-46.png
Screenshot_from_2024-06-14_01-50-46.png
Attachment 14: Screenshot_from_2024-06-14_01-51-18.png
Screenshot_from_2024-06-14_01-51-18.png
Attachment 15: Screenshot_from_2024-06-14_03-40-33.png
Screenshot_from_2024-06-14_03-40-33.png
Attachment 16: Screenshot_from_2024-06-14_03-40-56.png
Screenshot_from_2024-06-14_03-40-56.png
Attachment 17: Screenshot_from_2024-06-14_03-41-35.png
Screenshot_from_2024-06-14_03-41-35.png
Attachment 18: Screenshot_from_2024-06-14_03-42-15.png
Screenshot_from_2024-06-14_03-42-15.png
Attachment 19: Screenshot_from_2024-06-14_03-42-42.png
Screenshot_from_2024-06-14_03-42-42.png
Attachment 20: Screenshot_from_2024-06-14_03-43-39.png
Screenshot_from_2024-06-14_03-43-39.png
Attachment 21: Screenshot_from_2024-06-14_03-44-11.png
Screenshot_from_2024-06-14_03-44-11.png
Attachment 22: Screenshot_from_2024-06-14_03-53-08.png
Screenshot_from_2024-06-14_03-53-08.png
Attachment 23: Screenshot_from_2024-06-14_03-53-51.png
Screenshot_from_2024-06-14_03-53-51.png
Attachment 24: Screenshot_from_2024-06-14_03-54-36.png
Screenshot_from_2024-06-14_03-54-36.png
Attachment 25: Screenshot_from_2024-06-14_03-55-19.png
Screenshot_from_2024-06-14_03-55-19.png
Attachment 26: Screenshot_from_2024-06-14_03-55-50.png
Screenshot_from_2024-06-14_03-55-50.png
Attachment 27: Screenshot_from_2024-06-14_03-56-56.png
Screenshot_from_2024-06-14_03-56-56.png
Attachment 28: Screenshot_from_2024-06-14_03-58-10.png
Screenshot_from_2024-06-14_03-58-10.png
Attachment 29: Screenshot_from_2024-06-14_05-52-47.png
Screenshot_from_2024-06-14_05-52-47.png
Attachment 30: Screenshot_2024-06-14_at_05.31.21.png
Screenshot_2024-06-14_at_05.31.21.png
Attachment 31: Screenshot_from_2024-06-14_07-02-14.png
Screenshot_from_2024-06-14_07-02-14.png
Attachment 32: Screenshot_from_2024-06-14_07-00-53.png
Screenshot_from_2024-06-14_07-00-53.png
Attachment 33: Screenshot_from_2024-06-14_07-01-20.png
Screenshot_from_2024-06-14_07-01-20.png
Attachment 34: Screenshot_from_2024-06-14_07-01-55.png
Screenshot_from_2024-06-14_07-01-55.png
Attachment 35: Screenshot_from_2024-06-14_07-02-47.png
Screenshot_from_2024-06-14_07-02-47.png
Attachment 36: Screenshot_from_2024-06-14_07-01-55.png
Screenshot_from_2024-06-14_07-01-55.png
Attachment 37: Screenshot_from_2024-06-14_07-02-47.png
Screenshot_from_2024-06-14_07-02-47.png
Attachment 38: Screenshot_from_2024-06-14_07-03-23.png
Screenshot_from_2024-06-14_07-03-23.png
Attachment 39: Screenshot_from_2024-06-14_07-04-01.png
Screenshot_from_2024-06-14_07-04-01.png
Attachment 40: Capture.PNG
Capture.PNG
  543   Wed Mar 20 17:02:53 2024 NH/dev/sdd
The following messages are in the system log very often:

Mar 20 18:00:56 aidas-gsi smartd[1076]: Device: /dev/sdd [SAT], 2224 Currently unreadable (pending) sectors
Mar 20 18:00:56 aidas-gsi smartd[1076]: Device: /dev/sdd [SAT], 257 Offline uncorrectable sectors

This (to me) suggests /dev/sdd may be failing. It should be backed up and later replaced (it is not used at the moment for /TapeData, older disk)
  129   Fri Feb 21 09:21:10 2020 NH(Early) Alpha Analysis
Overnight alpha data

======================================
AIDA Unpacker Analysis
--------------------------------------
First event: Thu Feb 20 17:46:47 2020
Last event : Fri Feb 21 10:07:57 2020
Duration   : 58869.9 seconds
--------------------------------------
MBS Events   (M):     1248648 (21 Hz)
AIDA Words   (W): 10222680133 (27 kHz)
ADC Words    (A):  3407431088 (-15076 Hz)
-- Decays    (D):  3407017078 (-15083 Hz)
-- Implants  (I):      414010 (7 Hz)
Info Words   (F):  6815249045 (-30146 Hz)
-- SYNC48    (S):  3407559635 (-15074 Hz)
-- SYNC63    (s):  3407559635 (-15074 Hz)
-- PAUSE     (P):         614 (0 Hz)
-- RESUME    (R):         614 (0 Hz)
-- Discrim   (D):      128547 (2 Hz)
-- Unknown   (U):           0 (0 Hz)
Timewarps    (T):          79 (0 Hz)
Dead Time    (X): 26.8658 s (0.04563588%)
======================================
======================================
Individual FEE Overview
======================================
 FEE |           W           A           D           I           F           S           s     P     R       D     U     T  X
   1 |        5025        1246        1093         153        3779        1675        1675     0     0     429     0     0  0
   2 |        6180         991         851         140        5189        2060        2060     0     0    1069     0     0  0
   3 |  7692461404  2564028610  2564028286         324  5128432794  2564153392  2564153392   614   614 	124782     0    58 26.8658
   4 |        2277         642         581          61        1635         759         759     0     0     117     0     0  0
   5 |        6174        1249        1010         239        4925        2058        2058     0     0     809     0     0  0
   6 |     2467062      822244      822191          53     1644818      822354      822354     0     0     110     0     0  0
   7 |        2484         740         701          39        1744         828         828     0     0      88     0     0  0
   8 |   358445004   119481621   119069026      412595   238963383   119481668   119481668     0     0      47     0     1  0
   9 |        7134        1530        1243         287        5604        2378        2378     0     0     848     0     0  0
  10 |  2168031885   722677295   722677295           0  1445354590   722677295   722677295     0     0       0     0    20  0
  11 |        3981        1252        1190          62        2729        1327        1327     0     0      75     0     0  0
  12 |     1241523      413668      413611          57      827855      413841      413841     0     0     173     0     0  0
**** DESPECAnalysis: PostLoop

A few FEEs running "hot", maybe check. 3 had some alignment issues may need rechecking.
Timewarps somehow in aida03 and aida10 but may be code mistake - must check.
Only aida03 ran fast enough to have dead time.

aida10 shows decay data now - DSSD seems ok

Fig 1: DSSD 1D hit pattern 

Fig 2: DSSD Energy

Fig 3: Single channel in aida03 that is hot

Fig 4: Channels 48-64 in aida08 are hot (wiring?)

Fig 5: Channels 17, 19 (crosstalk?) and 27, 33, 38, 39

All otehr channels quiet and OK. 
Attachment 1: DSSD_1d_HP.png
DSSD_1d_HP.png
Attachment 2: DSSD_E.png
DSSD_E.png
Attachment 3: FEE3_43_Hot.png
FEE3_43_Hot.png
Attachment 4: fee8_hot.png
fee8_hot.png
Attachment 5: Fee10_hot.png
Fee10_hot.png
  663   Fri Jun 14 15:00:47 2024 Norah , Muneerah 16:00-20:00 Friday 14 June

Everything appears to be running smoothly. Attachments 1-4 were taken around 16:00

DSSSD Bias & Leakage Current - Check: OK (see attachment 1)
FEE64 Temperature - Check: OK (see attachment 2)
ADC Data Item Stats and Spectrum Browser - Check: OK (see attachments 3 and 4)

Another set of screenshots was taken around 18:00, and everything looks good.

DSSSD bias & leakage current  ok - attachment # 5-6
FEE64 temperatures  ok - attachment # 7
ADC data item stats - attachment # 8
Merger ok - Attachement #9
Tape service - attachement # 10
ucesb - attachment #11

At 19:17, AIDA03 temperatures gave no response (see attachment 12), but after two minutes, they returned to normal.

I noted that the  bplse in ucesb has no data (see attachment 13). It may need to change the disc thresholds again
 it does not affect AIDA

At 19:30 most aida temperature are high and I spoke to people on site via zoom one of them she is working with AIDA so she was able to help. She called Nick and it is found that there is a problem in the cooling system so she stopped AIDA

At 22 the cooling system fixed and Nic and the other person on site restarted AIDA

 

 

 

 

 

Attachment 1: Screenshot_from_2024-06-14_16-06-09.png
Screenshot_from_2024-06-14_16-06-09.png
Attachment 2: Screenshot_from_2024-06-14_16-04-58.png
Screenshot_from_2024-06-14_16-04-58.png
Attachment 3: Screenshot_from_2024-06-14_16-04-23.png
Screenshot_from_2024-06-14_16-04-23.png
Attachment 4: Screenshot_from_2024-06-14_16-05-33.png
Screenshot_from_2024-06-14_16-05-33.png
Attachment 5: Screenshot_from_2024-06-14_18-02-36.png
Screenshot_from_2024-06-14_18-02-36.png
Attachment 6: Screenshot_from_2024-06-14_18-02-08.png
Screenshot_from_2024-06-14_18-02-08.png
Attachment 7: Screenshot_from_2024-06-14_18-04-40.png
Screenshot_from_2024-06-14_18-04-40.png
Attachment 8: Screenshot_from_2024-06-14_18-05-44.png
Screenshot_from_2024-06-14_18-05-44.png
Attachment 9: Screenshot_from_2024-06-14_18-08-50.png
Screenshot_from_2024-06-14_18-08-50.png
Attachment 10: Screenshot_from_2024-06-14_18-11-04.png
Screenshot_from_2024-06-14_18-11-04.png
Attachment 11: Screenshot_from_2024-06-14_18-17-15.png
Screenshot_from_2024-06-14_18-17-15.png
Attachment 12: Screenshot_from_2024-06-14_19-23-32.png
Screenshot_from_2024-06-14_19-23-32.png
Attachment 13: Screenshot_from_2024-06-14_19-35-11.png
Screenshot_from_2024-06-14_19-35-11.png
ELOG V3.1.3-7933898