AIDA GELINA BRIKEN nToF CRIB ISOLDE CIRCE nTOFCapture DESPEC DTAS EDI_PSA 179Ta CARME StellarModelling DCF K40
  DESPEC, Page 14 of 35  ELOG logo
  ID Date Author Subject
  437   Fri May 13 02:28:36 2022 MLbeam dropped

We see less counts in AIDA implant DSSD1 (~1Hz) than at the beginning of the night shift (~10Hz).

This seems to be beam related as the count rates in DESPEC & FRS Scalers seems to have dropped too.

  436   Fri May 13 00:50:16 2022 ML2h-Shift Checks

All seems ok  (apart for my virgin network connection usually quite reliable but not so tonight !?!).

Statistics seems ok.

Temperatures appears stable

Leakage current ok.

System Wide Checks:

Clock check: Passed= 14: - Failed=: 0

ADC calibration check:

FEE64 module aida01 failed
FEE64 module aida04 failed
FEE64 module aida06 failed
FEE64 module aida13 failed
Calibration test result: Passed=10 ; Failed= 4

If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module -> DID THAT BUT NO CHANGES

White rabbit decode status: Passed=14 ; Failed=0

FPGA timestamp check: Passed=14 ; Failed=0

Memory information: Page loading failed with the error message below:

Got the error Server Internal Error
while trying to obtain /AIDA/Check/Check.tml.
could not read "/Embedded/XilinxLinux/ppc_4xx/rfs/aida01/tmp/mybuddy.txt": no such file or directory while executing "file size $mybuddy " (procedure "do_collect_info" line 16) invoked from within "do_collect_info " ("COLLECTINFO" arm line 1) invoked from within "switch -glob $w { RESET {set started 0} CLEAR {EmptyLog} ELOG {PrintLog} ..." (procedure "do_click" line 23) invoked from within "do_click" invoked from within "if {$started != 0} { variable JS "" variable LogFlag if {$LogFlag == 1} {InsertLog "Last Updated: [clock format [clock seconds] -form..." (file "Check.tcl" line 8) invoked from within "source Check.tcl" (in namespace eval "::Check" script line 6) invoked from within "namespace eval Check { global env source [file join $env(MIDASBASE) TclHttpd tcl Common common.tcl] source [file join $env(MIDASB..." invoked from within "subst { [Doc_Dynamic] <! [global Httpd; upvar #0 Httpd[set Httpd(currentSocket)] data; set ClientIPAddress $data(ipaddr); set MyInfo $data(self)] >..." ("uplevel" body line 1) invoked from within "uplevel #0 [list subst $script]" (procedure "SubstFile" line 12) invoked from within "SubstFile /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119/TclHttpd/Html/AIDA/Check/Check.tml {}" ("uplevel" body line 1) invoked from within "uplevel 1 [list SubstFile $path $interp]" (procedure "Subst_File" line 7) invoked from within "Subst_File $template $interp" invoked from within "TemplateInstantiate $sock $path {} $suffix {} $Template(templateInterp)" (procedure "Doc_application/x-tcl-template" line 9) invoked from within "$cmd $path $suffix $sock" (procedure "Doc_Handle" line 20) invoked from within "Doc_Handle $prefix $path $suffix $sock" (procedure "DocDomain" line 44) invoked from within "DocDomain / /MIDAS/TclHttpd/Html sock8 AIDA/Check/Check.tml" ("eval" body line 1) invoked from within "eval $Url(command,$prefix) [list $sock $suffix]"

Attachment 1: Stats_Screenshot_from_2022-05-13_01-45-56.png
Stats_Screenshot_from_2022-05-13_01-45-56.png
Attachment 2: Temp_Screenshot_from_2022-05-13_01-47-31.png
Temp_Screenshot_from_2022-05-13_01-47-31.png
Attachment 3: HV_LC_Screenshot_from_2022-05-13_01-48-21.png
HV_LC_Screenshot_from_2022-05-13_01-48-21.png
  435   Thu May 12 22:49:39 2022 BA, MALow Rates

The rate is very low in all aida, not sure .

Attachment 1: RatesLow2022-05-12_23-48-01.png
RatesLow2022-05-12_23-48-01.png
  434   Thu May 12 21:43:49 2022 BA, MARates

 

Statistic check (screenshot attached).

Temperatures OK (screenshot attached).

Bias and leakage currents OK (same screenshot as temps).

 

Attachment 1: Stati2022-05-12_22-47-26.png
Stati2022-05-12_22-47-26.png
Attachment 2: Temp2022-05-12_22-48-41.png
Temp2022-05-12_22-48-41.png
Attachment 3: LeakageCurrent2022-05-12_22-45-44.png
LeakageCurrent2022-05-12_22-45-44.png
  433   Thu May 12 16:30:47 2022 NH, OH, TDNotes for mounting a second switch
Update DHCPD as expected
Make sure new ethernet device is trusted in firewall
Make sure hosts is updated
make sure /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida08/etc/hosts "myserver" points to correct ip address
make sure /MIDAS/linux-ppc_4xx/startup/aidaXX has netvar "TS_Server" pointing to correct ip address
Update the IP in /MIDAS/linux-ppc_4xx/startup/aida*
  432   Thu May 12 10:10:51 2022 TD, OHAIDA data throughput
Attachment 1 - top aida06

Attachment 2 - top aida01

Attachment 3 - top aida-gsi (AIDA server)

Attachment 4 - TapeServer

Attachment 5 - NewMerger stats block 6 (aida06)

Attachment 6 - NewMerger stats block 1 (aida01)

Attachment 7 - NewMerger stats block 0 (all FEE64s)

Attachment 8 - NewMerger options

Attachment 9 - NewMerger link stats

Attachment 10 - NewMerger

Attachments 11-14 - ADC, pause, resume & correlation scaler data item stats

Attacgment 15 - Network traffic
Attachment 1: Screenshot_from_2022-05-12_11-08-34.png
Screenshot_from_2022-05-12_11-08-34.png
Attachment 2: Screenshot_from_2022-05-12_11-07-44.png
Screenshot_from_2022-05-12_11-07-44.png
Attachment 3: Screenshot_from_2022-05-12_11-07-14.png
Screenshot_from_2022-05-12_11-07-14.png
Attachment 4: Screenshot_from_2022-05-12_11-05-53.png
Screenshot_from_2022-05-12_11-05-53.png
Attachment 5: Screenshot_from_2022-05-12_11-05-40.png
Screenshot_from_2022-05-12_11-05-40.png
Attachment 6: Screenshot_from_2022-05-12_11-05-18.png
Screenshot_from_2022-05-12_11-05-18.png
Attachment 7: Screenshot_from_2022-05-12_11-05-09.png
Screenshot_from_2022-05-12_11-05-09.png
Attachment 8: Screenshot_from_2022-05-12_11-04-46.png
Screenshot_from_2022-05-12_11-04-46.png
Attachment 9: Screenshot_from_2022-05-12_11-04-22.png
Screenshot_from_2022-05-12_11-04-22.png
Attachment 10: Screenshot_from_2022-05-12_11-03-49.png
Screenshot_from_2022-05-12_11-03-49.png
Attachment 11: Screenshot_from_2022-05-12_11-03-37.png
Screenshot_from_2022-05-12_11-03-37.png
Attachment 12: Screenshot_from_2022-05-12_11-03-28.png
Screenshot_from_2022-05-12_11-03-28.png
Attachment 13: Screenshot_from_2022-05-12_11-03-13.png
Screenshot_from_2022-05-12_11-03-13.png
Attachment 14: Screenshot_from_2022-05-12_11-02-58.png
Screenshot_from_2022-05-12_11-02-58.png
Attachment 15: Screenshot_from_2022-05-12_11-15-56.png
Screenshot_from_2022-05-12_11-15-56.png
  431   Thu May 12 07:09:43 2022 TDThursday 12 May 08:00-24:00
08.10 AIDA DAQ status
    
      14x FEE64s configured - aida05, aida14 power OFF - dhcpd.conf changed - aida15 -> aida05, aida16 -> aida14
2
      AD9222 OFF - waveforms disabed
      disc - all disabled

      ASIC settings 2021Apr29-13-16-00
       slow comparator 0xc *except* aida02 & aida04 0x16, aida06 & aida08 0x23

      BNC PB-5
      amplitude 1.000V
      attenuation x1
      tau_d 1ms (tail pulse)
      polarity +
      frequency 22Hz

08.14 all histograms zero'd
2
08.15 system wide check counter baseline

08.17 check ASIC control - all FEE64s all ASICs

Attachment 1 - grafana DSSSD bias & leakage current - OK

Attachment 2 - per FEE64 stat spectra

Attachments 3 & 4 - per FEE64 1.8.H spectra - few events c. 5GeV p+n strips

Attachments 5 & 6 - per FEE64 1.8.L spectra
 pulser peak width aida01 124ch FWHM, aida04 293ch FWHM

Attachment 7 - per FEE64 rate spectra

Attachments 8-12 - system wide checks - all OK

Attachments 13-16 - statistics - adc, pause, resume & correlation data

Attachment 17 - FEE64 temps OK

Attachment 18 - DSSSD bias & leakage current OK - start new section of googledoc spreadsheet to monitor long term trends

Attachments 19 & 20 - NewMerger & TapeServer - OK - no warning messages

08.44 DAQ continues - file NOTAPE/R18 - no storage mode

08.47 switch to storage mode to generate data file to evaluate FEE64 dead times etc

08.53 no storage mode enabled

Attachment 21 - analysis of file NOTAPE/R18_20

Attachment 22- analysis of file NOTAPE/R19_19
               AIDA12 slow comp up to 0xd improved deadtime drastically for FEE

10:54 Beam on detector. Take another run to test deadtime - attachment 23
      Deadtimes ranging from 10-40% in n+n FEEs

Attachment 23 & 24 aida06 aida02 1.*.H spectra


14:07 - Currently no beam so had acces. OH checked the cables on AIDA12 were seated correctly. No improvement in the rate for 12 but better for remaining p+n strips.
        Tested running with dataspy off for PCS but observed no change in the deadtimes (Other than a decrease because no beamtime)
Attachment 1: Screenshot_from_2022-05-12_08-25-44.png
Screenshot_from_2022-05-12_08-25-44.png
Attachment 2: Screenshot_from_2022-05-12_08-25-20.png
Screenshot_from_2022-05-12_08-25-20.png
Attachment 3: Screenshot_from_2022-05-12_08-24-01.png
Screenshot_from_2022-05-12_08-24-01.png
Attachment 4: Screenshot_from_2022-05-12_08-23-35.png
Screenshot_from_2022-05-12_08-23-35.png
Attachment 5: Screenshot_from_2022-05-12_08-22-24.png
Screenshot_from_2022-05-12_08-22-24.png
Attachment 6: Screenshot_from_2022-05-12_08-21-18.png
Screenshot_from_2022-05-12_08-21-18.png
Attachment 7: Screenshot_from_2022-05-12_08-19-15.png
Screenshot_from_2022-05-12_08-19-15.png
Attachment 8: Screenshot_from_2022-05-12_08-16-40.png
Screenshot_from_2022-05-12_08-16-40.png
Attachment 9: Screenshot_from_2022-05-12_08-16-30.png
Screenshot_from_2022-05-12_08-16-30.png
Attachment 10: Screenshot_from_2022-05-12_08-16-25.png
Screenshot_from_2022-05-12_08-16-25.png
Attachment 11: Screenshot_from_2022-05-12_08-16-17.png
Screenshot_from_2022-05-12_08-16-17.png
Attachment 12: Screenshot_from_2022-05-12_08-16-08.png
Screenshot_from_2022-05-12_08-16-08.png
Attachment 13: Screenshot_from_2022-05-12_08-10-02.png
Screenshot_from_2022-05-12_08-10-02.png
Attachment 14: Screenshot_from_2022-05-12_08-09-44.png
Screenshot_from_2022-05-12_08-09-44.png
Attachment 15: Screenshot_from_2022-05-12_08-09-20.png
Screenshot_from_2022-05-12_08-09-20.png
Attachment 16: Screenshot_from_2022-05-12_08-09-02.png
Screenshot_from_2022-05-12_08-09-02.png
Attachment 17: Screenshot_from_2022-05-12_08-08-39.png
Screenshot_from_2022-05-12_08-08-39.png
Attachment 18: Screenshot_from_2022-05-12_08-08-26.png
Screenshot_from_2022-05-12_08-08-26.png
Attachment 19: Screenshot_from_2022-05-12_08-39-14.png
Screenshot_from_2022-05-12_08-39-14.png
Attachment 20: Screenshot_from_2022-05-12_08-39-05.png
Screenshot_from_2022-05-12_08-39-05.png
Attachment 21: R18_20
 *** TDR format 3.3.0 analyser - TD - May 2021
 *** ERROR: READ I/O error:       5002
                   blocks:      32000
          ADC data format:  259331519 ( 1693112.6 Hz)
        Other data format:    2588481 (   16899.6 Hz)
 Sample trace data format:          0 (       0.0 Hz)
         Undefined format:          0 (       0.0 Hz)
   Other data format type:      PAUSE:       1016 (       6.6 Hz)
                               RESUME:       1017 (       6.6 Hz)
                              SYNC100:      32570 (     212.6 Hz)
                              WR48-63:      32570 (     212.6 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:    2521308 (   16461.0 Hz)
                           Other info:          0 (       0.0 Hz)

   ADC data range bit set:          0 (       0.0 Hz)

                Timewarps:        ADC:          0 (       0.0 Hz)
                                PAUSE:          0 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:          0 (       0.0 Hz)
                              WR48-63:          0 (       0.0 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:          0 (       0.0 Hz)
                            Undefined:          0 (       0.0 Hz)
                         Sample trace:          0 (       0.0 Hz)

 *** Timestamp elapsed time:      153.169 s
 FEE  elapsed dead time(s) elapsed idle time(s)
  0                0.171                0.000
  1                9.417                0.000
  2                0.061                0.000
  3               24.348                0.000
  4                0.008                0.000
  5               72.336                0.000
  6                0.231                0.000
  7               25.570                0.000
  8                0.125                0.000
  9                0.007                0.000
 10                0.000                7.342
 11               41.574                0.000
 12                0.027                0.000
 13                0.000                0.000
 14                0.000                0.000
 15                0.000                0.000
 16                0.000                0.000
 17                0.000                0.000
 18                0.000                0.000
 19                0.000                0.000
 20                0.000                0.000
 21                0.000                0.000
 22                0.000                0.000
 23                0.000                0.000
 24                0.000                0.000
 25                0.000                0.000
 26                0.000                0.000
 27                0.000                0.000
 28                0.000                0.000
 29                0.000                0.000
 30                0.000                0.000
 31                0.000                0.000
 32                0.000                0.000

 *** Statistics
 FEE  ADC Data Other Data     Sample  Undefined      Pause     Resume    SYNC100    WR48-63       Disc        MBS      Other   HEC Data
  0    7431923      12514          0          0          8          8        933        933          0      10632          0          0
  1   37380104       9380          0          0         51         51       4639       4639          0          0          0          0
  2    7217279    1365495          0          0          3          3       1056       1056          0    1363377          0          0
  3   38015689    1157497          0          0        179        179       4920       4920          0    1147299          0          0
  4    4440445       1078          0          0          2          2        537        537          0          0          0          0
  5   40380917      10697          0          0        319        320       5029       5029          0          0          0          0
  6   14913071       3788          0          0          5          5       1889       1889          0          0          0          0
  7   41232478      10490          0          0        181        181       5064       5064          0          0          0          0
  8   11952709       2932          0          0          4          4       1462       1462          0          0          0          0
  9    4910914       1250          0          0          1          1        624        624          0          0          0          0
 10    2469929        580          0          0          0          0        290        290          0          0          0          0
 11   39767127      10486          0          0        257        257       4986       4986          0          0          0          0
 12    5557988       1406          0          0          6          6        697        697          0          0          0          0
 13    3660946        888          0          0          0          0        444        444          0          0          0          0
 14          0          0          0          0          0          0          0          0          0          0          0          0
 15          0          0          0          0          0          0          0          0          0          0          0          0
 16          0          0          0          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0          0          0          0

 *** Timewarps
 FEE       ADC      Pause     Resume    SYNC100    WR48-63       Disc        MBS  Undefined    Samples
  0          0          0          0          0          0          0          0          0          0
  1          0          0          0          0          0          0          0          0          0
  2          0          0          0          0          0          0          0          0          0
  3          0          0          0          0          0          0          0          0          0
  4          0          0          0          0          0          0          0          0          0
  5          0          0          0          0          0          0          0          0          0
  6          0          0          0          0          0          0          0          0          0
  7          0          0          0          0          0          0          0          0          0
  8          0          0          0          0          0          0          0          0          0
  9          0          0          0          0          0          0          0          0          0
 10          0          0          0          0          0          0          0          0          0
 11          0          0          0          0          0          0          0          0          0
 12          0          0          0          0          0          0          0          0          0
 13          0          0          0          0          0          0          0          0          0
 14          0          0          0          0          0          0          0          0          0
 15          0          0          0          0          0          0          0          0          0
 16          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0

 *** Program elapsed time:   36.578s (  874.840 blocks/s,  54.677 Mb/s)
Attachment 22: R19_19.txt
 *** TDR format 3.3.0 analyser - TD - May 2021
 *** ERROR: READ I/O error:       5002
                   blocks:      17139
          ADC data format:  138519076 ( 1563611.9 Hz)
        Other data format:    1763639 (   19908.1 Hz)
 Sample trace data format:          0 (       0.0 Hz)
         Undefined format:          0 (       0.0 Hz)
   Other data format type:      PAUSE:        358 (       4.0 Hz)
                               RESUME:        358 (       4.0 Hz)
                              SYNC100:      17469 (     197.2 Hz)
                              WR48-63:      17469 (     197.2 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:    1727985 (   19505.6 Hz)
                           Other info:          0 (       0.0 Hz)

   ADC data range bit set:          0 (       0.0 Hz)

                Timewarps:        ADC:          0 (       0.0 Hz)
                                PAUSE:          0 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:          0 (       0.0 Hz)
                              WR48-63:          0 (       0.0 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:          0 (       0.0 Hz)
                            Undefined:          0 (       0.0 Hz)
                         Sample trace:          0 (       0.0 Hz)

 *** Timestamp elapsed time:       88.589 s
 FEE  elapsed dead time(s) elapsed idle time(s)
  0                0.084                0.000
  1                0.528                0.000
  2                0.008                0.000
  3               10.973                0.000
  4                0.000                0.000
  5               30.421                0.000
  6                0.000                0.000
  7                7.252                0.000
  8                0.055                0.000
  9                0.000                0.000
 10                0.000               12.541
 11                3.332                0.000
 12                0.009                0.000
 13                0.000                0.000
 14                0.000                0.000
 15                0.000                0.000
 16                0.000                0.000
 17                0.000                0.000
 18                0.000                0.000
 19                0.000                0.000
 20                0.000                0.000
 21                0.000                0.000
 22                0.000                0.000
 23                0.000                0.000
 24                0.000                0.000
 25                0.000                0.000
 26                0.000                0.000
 27                0.000                0.000
 28                0.000                0.000
 29                0.000                0.000
 30                0.000                0.000
 31                0.000                0.000
 32                0.000                0.000

 *** Statistics
 FEE  ADC Data Other Data     Sample  Undefined      Pause     Resume    SYNC100    WR48-63       Disc        MBS      Other   HEC Data
  0    4390551       7288          0          0          4          4        565        565          0       6150          0          0
  1   21021919       5186          0          0          9          9       2584       2584          0          0          0          0
  2    3174721     918030          0          0          1          1        518        518          0     916992          0          0
  3   22097261     810837          0          0         77         77       2920       2920          0     804843          0          0
  4    2596723        620          0          0          0          0        310        310          0          0          0          0
  5   23543560       6240          0          0        174        174       2946       2946          0          0          0          0
  6    4211356       1020          0          0          0          0        510        510          0          0          0          0
  7   24080430       6296          0          0         63         63       3085       3085          0          0          0          0
  8    4823696       1068          0          0          3          3        531        531          0          0          0          0
  9    1770702        408          0          0          0          0        204        204          0          0          0          0
 10     933213        222          0          0          0          0        111        111          0          0          0          0
 11   21419475       5280          0          0         21         21       2619       2619          0          0          0          0
 12    2929744        698          0          0          6          6        343        343          0          0          0          0
 13    1525725        446          0          0          0          0        223        223          0          0          0          0
 14          0          0          0          0          0          0          0          0          0          0          0          0
 15          0          0          0          0          0          0          0          0          0          0          0          0
 16          0          0          0          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0          0          0          0

 *** Timewarps
 FEE       ADC      Pause     Resume    SYNC100    WR48-63       Disc        MBS  Undefined    Samples
  0          0          0          0          0          0          0          0          0          0
  1          0          0          0          0          0          0          0          0          0
  2          0          0          0          0          0          0          0          0          0
  3          0          0          0          0          0          0          0          0          0
  4          0          0          0          0          0          0          0          0          0
  5          0          0          0          0          0          0          0          0          0
  6          0          0          0          0          0          0          0          0          0
  7          0          0          0          0          0          0          0          0          0
  8          0          0          0          0          0          0          0          0          0
  9          0          0          0          0          0          0          0          0          0
 10          0          0          0          0          0          0          0          0          0
 11          0          0          0          0          0          0          0          0          0
 12          0          0          0          0          0          0          0          0          0
 13          0          0          0          0          0          0          0          0          0
 14          0          0          0          0          0          0          0          0          0
 15          0          0          0          0          0          0          0          0          0
 16          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0

 *** Program elapsed time:   17.484s (  980.247 blocks/s,  61.265 Mb/s)
Attachment 23: R20_20.txt
 *** TDR format 3.3.0 analyser - TD - May 2021
 *** ERROR: READ I/O error:       5002
                   blocks:       4909
          ADC data format:   39765494 ( 1644562.3 Hz)
        Other data format:     414671 (   17149.3 Hz)
 Sample trace data format:          0 (       0.0 Hz)
         Undefined format:          0 (       0.0 Hz)
   Other data format type:      PAUSE:        151 (       6.2 Hz)
                               RESUME:        152 (       6.3 Hz)
                              SYNC100:       4999 (     206.7 Hz)
                              WR48-63:       4999 (     206.7 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:     404370 (   16723.3 Hz)
                           Other info:          0 (       0.0 Hz)

   ADC data range bit set:      23252 (     961.6 Hz)

                Timewarps:        ADC:          0 (       0.0 Hz)
                                PAUSE:          0 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:          0 (       0.0 Hz)
                              WR48-63:          0 (       0.0 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:          0 (       0.0 Hz)
                            Undefined:          0 (       0.0 Hz)
                         Sample trace:          0 (       0.0 Hz)

 *** Timestamp elapsed time:       24.180 s
 FEE  elapsed dead time(s) elapsed idle time(s)
  0                0.000                0.000
  1                1.954                0.000
  2                0.000                0.000
  3                4.569                0.000
  4                0.000                0.000
  5               10.824                0.000
  6                0.000                0.000
  7                5.869                0.000
  8                0.000                0.000
  9                0.008                0.000
 10                0.000                0.000
 11                2.942                0.000
 12                0.007                0.000
 13                0.000                0.000
 14                0.000                0.000
 15                0.000                0.000
 16                0.000                0.000
 17                0.000                0.000
 18                0.000                0.000
 19                0.000                0.000
 20                0.000                0.000
 21                0.000                0.000
 22                0.000                0.000
 23                0.000                0.000
 24                0.000                0.000
 25                0.000                0.000
 26                0.000                0.000
 27                0.000                0.000
 28                0.000                0.000
 29                0.000                0.000
 30                0.000                0.000
 31                0.000                0.000
 32                0.000                0.000

 *** Statistics
 FEE  ADC Data Other Data     Sample  Undefined      Pause     Resume    SYNC100    WR48-63       Disc        MBS      Other   HEC Data
  0    1157062       1976          0          0          0          0        148        148          0       1680          0        159
  1    5834981       1480          0          0         12         12        728        728          0          0          0      22412
  2    1125294     220409          0          0          0          0        178        178          0     220053          0         83
  3    5924210     180469          0          0         32         32        741        741          0     178923          0         86
  4     669291        180          0          0          0          0         90         90          0          0          0         10
  5    6069826       1577          0          0         47         48        741        741          0          0          0         20
  6    1749760       2346          0          0          0          0        222        222          0       1902          0         32
  7    6604292       3590          0          0         38         38        851        851          0       1812          0         32
  8    1794890        416          0          0          0          0        208        208          0          0          0         42
  9     736012        164          0          0          1          1         81         81          0          0          0        124
 10     410664         98          0          0          0          0         49         49          0          0          0         15
 11    6315369       1576          0          0         19         19        769        769          0          0          0        153
 12     852659        248          0          0          2          2        122        122          0          0          0         23
 13     521184        142          0          0          0          0         71         71          0          0          0         61
 14          0          0          0          0          0          0          0          0          0          0          0          0
 15          0          0          0          0          0          0          0          0          0          0          0          0
 16          0          0          0          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0          0          0          0

 *** Timewarps
 FEE       ADC      Pause     Resume    SYNC100    WR48-63       Disc        MBS  Undefined    Samples
  0          0          0          0          0          0          0          0          0          0
  1          0          0          0          0          0          0          0          0          0
  2          0          0          0          0          0          0          0          0          0
  3          0          0          0          0          0          0          0          0          0
  4          0          0          0          0          0          0          0          0          0
  5          0          0          0          0          0          0          0          0          0
  6          0          0          0          0          0          0          0          0          0
  7          0          0          0          0          0          0          0          0          0
  8          0          0          0          0          0          0          0          0          0
  9          0          0          0          0          0          0          0          0          0
 10          0          0          0          0          0          0          0          0          0
 11          0          0          0          0          0          0          0          0          0
 12          0          0          0          0          0          0          0          0          0
 13          0          0          0          0          0          0          0          0          0
 14          0          0          0          0          0          0          0          0          0
 15          0          0          0          0          0          0          0          0          0
 16          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0

 *** Program elapsed time:    5.102s (  962.254 blocks/s,  60.141 Mb/s)
Attachment 24: Screenshot_from_2022-05-12_11-57-44.png
Screenshot_from_2022-05-12_11-57-44.png
Attachment 25: Screenshot_from_2022-05-12_11-56-42.png
Screenshot_from_2022-05-12_11-56-42.png
Attachment 26: Screenshot_from_2022-05-12_22-32-53.png
Screenshot_from_2022-05-12_22-32-53.png
Attachment 27: Screenshot_from_2022-05-12_22-32-22.png
Screenshot_from_2022-05-12_22-32-22.png
  430   Thu May 12 00:37:52 2022 OH, AMThursday 12 May
01:37 Over the past hour the degrader settings have been checked by impinging 209Pb onto the detectors.
      That is now complete.
      Will find optimum thresholds for n+n side and then leave DAQ running with n+n at optimum and p+n at 0xc

      R16 slow comp 0x19 (250keV) 2 and 4 low deadtime 6 and 8 high deadtime. - attachment1
      Will lower 2 and 4 to 200 and raise 6 and 8 to 300

      R17 2 good, 4 shakey 6 and 8 poor - attachment 2

      R18 2 and 4 at 220 and 6 and 7 at 350
      2 and 8 low deadtime, 4 and 6 could be better. - attachment 3


      Will run with p+n at 0xc 2+4 at 0x16 and 6 and 8 at 0x23
      Statistics at this rate - attachment 4




02:18 Statistics - attachment 7
      Temperatures - attachment 6
      Bias & leakage current - attachment 5

      Clock Status - OK
      White Rabbit - OK
      The other 3 checks currently aren't working




04:11 Statistics - attachment 10
      Temperatures - attachment 9
      Bias & leakage current - attachment 8

      Clock Status - OK
      White Rabbit - OK




06:05 Statistics - attachment 13
      Temperatures - attachment 12
      Bias & leakage current - attachment 11

      Clock Status - OK
      White Rabbit - OK
Attachment 1: R16_19_analyse.txt
 *** TDR format 3.3.0 analyser - TD - May 2021
 *** ERROR: READ I/O error:       5002
                   blocks:      18197
          ADC data format:  144240941 ( 1037416.0 Hz)
        Other data format:    4701506 (   33814.4 Hz)
 Sample trace data format:          0 (       0.0 Hz)
         Undefined format:          0 (       0.0 Hz)
   Other data format type:      PAUSE:        651 (       4.7 Hz)
                               RESUME:        651 (       4.7 Hz)
                              SYNC100:      18715 (     134.6 Hz)
                              WR48-63:      18715 (     134.6 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:    4662774 (   33535.8 Hz)
                           Other info:          0 (       0.0 Hz)

   ADC data range bit set:       9906 (      71.2 Hz)

                Timewarps:        ADC:          0 (       0.0 Hz)
                                PAUSE:          0 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:          0 (       0.0 Hz)
                              WR48-63:          0 (       0.0 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:          0 (       0.0 Hz)
                            Undefined:          0 (       0.0 Hz)
                         Sample trace:          0 (       0.0 Hz)

 *** Timestamp elapsed time:      139.039 s
 FEE  elapsed dead time(s) elapsed idle time(s)
  0                0.000               96.976
  1                1.224                0.000
  2                0.000                2.784
  3                4.445                0.000
  4                0.000              108.587
  5               78.372                0.000
  6                0.000               48.067
  7               67.687                0.000
  8                0.000              100.043
  9                0.000              102.969
 10                0.000              122.226
 11                0.000               21.170
 12                0.000                2.713
 13                0.000              116.657
 14                0.000                0.000
 15                0.000                0.000
 16                0.000                0.000
 17                0.000                0.000
 18                0.000                0.000
 19                0.000                0.000
 20                0.000                0.000
 21                0.000                0.000
 22                0.000                0.000
 23                0.000                0.000
 24                0.000                0.000
 25                0.000                0.000
 26                0.000                0.000
 27                0.000                0.000
 28                0.000                0.000
 29                0.000                0.000
 30                0.000                0.000
 31                0.000                0.000
 32                0.000                0.000

 *** Statistics
 FEE  ADC Data Other Data     Sample  Undefined      Pause     Resume    SYNC100    WR48-63       Disc        MBS      Other   HEC Data
  0     346857       9766          0          0          0          0         47         47          0       9672          0       2994
  1   28465912       7268          0          0         10         10       3624       3624          0          0          0       3890
  2     235746    2352664          0          0          0          0        299        299          0    2352066          0        528
  3   33196911    2283279          0          0         20         20       4471       4471          0    2274297          0        644
  4     346134         72          0          0          0          0         36         36          0          0          0          4
  5   37750759       9970          0          0        309        309       4676       4676          0          0          0        836
  6     855292      17907          0          0          0          0        114        114          0      17679          0        522
  7   38520306      19374          0          0        312        312       4845       4845          0       9060          0        313
  8     312768         68          0          0          0          0         34         34          0          0          0         23
  9     342945         90          0          0          0          0         45         45          0          0          0         14
 10     176833         46          0          0          0          0         23         23          0          0          0          7
 11    1559416        390          0          0          0          0        195        195          0          0          0         62
 12    1848830        538          0          0          0          0        269        269          0          0          0         19
 13     282232         74          0          0          0          0         37         37          0          0          0         50
 14          0          0          0          0          0          0          0          0          0          0          0          0
 15          0          0          0          0          0          0          0          0          0          0          0          0
 16          0          0          0          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0          0          0          0

 *** Timewarps
 FEE       ADC      Pause     Resume    SYNC100    WR48-63       Disc        MBS  Undefined    Samples
  0          0          0          0          0          0          0          0          0          0
  1          0          0          0          0          0          0          0          0          0
  2          0          0          0          0          0          0          0          0          0
  3          0          0          0          0          0          0          0          0          0
  4          0          0          0          0          0          0          0          0          0
  5          0          0          0          0          0          0          0          0          0
  6          0          0          0          0          0          0          0          0          0
  7          0          0          0          0          0          0          0          0          0
  8          0          0          0          0          0          0          0          0          0
  9          0          0          0          0          0          0          0          0          0
 10          0          0          0          0          0          0          0          0          0
 11          0          0          0          0          0          0          0          0          0
 12          0          0          0          0          0          0          0          0          0
 13          0          0          0          0          0          0          0          0          0
 14          0          0          0          0          0          0          0          0          0
 15          0          0          0          0          0          0          0          0          0
 16          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0

 *** Program elapsed time:   18.576s (  979.614 blocks/s,  61.226 Mb/s)
Attachment 2: R17_19_analyse.txt
 *** TDR format 3.3.0 analyser - TD - May 2021
 *** ERROR: READ I/O error:       5002
                   blocks:       9844
          ADC data format:   78473161 ( 1119546.3 Hz)
        Other data format:    2099979 (   29959.6 Hz)
 Sample trace data format:          0 (       0.0 Hz)
         Undefined format:          0 (       0.0 Hz)
   Other data format type:      PAUSE:        470 (       6.7 Hz)
                               RESUME:        470 (       6.7 Hz)
                              SYNC100:      10105 (     144.2 Hz)
                              WR48-63:      10105 (     144.2 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:    2078829 (   29657.8 Hz)
                           Other info:          0 (       0.0 Hz)

   ADC data range bit set:          0 (       0.0 Hz)

                Timewarps:        ADC:          0 (       0.0 Hz)
                                PAUSE:          0 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:          0 (       0.0 Hz)
                              WR48-63:          0 (       0.0 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:          0 (       0.0 Hz)
                            Undefined:          0 (       0.0 Hz)
                         Sample trace:          0 (       0.0 Hz)

 *** Timestamp elapsed time:       70.094 s
 FEE  elapsed dead time(s) elapsed idle time(s)
  0                0.000               61.805
  1                4.886                0.000
  2                0.000                0.000
  3               17.776                0.000
  4                0.000               55.325
  5               34.711                0.000
  6                0.000               20.226
  7               27.353                0.000
  8                0.000               57.388
  9                0.000               53.302
 10                0.000               59.261
 11                0.000                7.690
 12                0.000               11.252
 13                0.000               58.364
 14                0.000                0.000
 15                0.000                0.000
 16                0.000                0.000
 17                0.000                0.000
 18                0.000                0.000
 19                0.000                0.000
 20                0.000                0.000
 21                0.000                0.000
 22                0.000                0.000
 23                0.000                0.000
 24                0.000                0.000
 25                0.000                0.000
 26                0.000                0.000
 27                0.000                0.000
 28                0.000                0.000
 29                0.000                0.000
 30                0.000                0.000
 31                0.000                0.000
 32                0.000                0.000

 *** Statistics
 FEE  ADC Data Other Data     Sample  Undefined      Pause     Resume    SYNC100    WR48-63       Disc        MBS      Other   HEC Data
  0     137192       4907          0          0          0          0         16         16          0       4875          0          0
  1   18540829       4668          0          0         40         40       2294       2294          0          0          0          0
  2     107308    1183296          0          0          0          0        177        177          0    1182942          0          0
  3   18083643     896022          0          0        126        126       2379       2379          0     891012          0          0
  4     171162         40          0          0          0          0         20         20          0          0          0          0
  5   19343787       5154          0          0        155        155       2422       2422          0          0          0          0
  6     395908        126          0          0          0          0         63         63          0          0          0          0
  7   19401210       5206          0          0        149        149       2454       2454          0          0          0          0
  8     159836         40          0          0          0          0         20         20          0          0          0          0
  9     172050         38          0          0          0          0         19         19          0          0          0          0
 10      94357         20          0          0          0          0         10         10          0          0          0          0
 11     792883        200          0          0          0          0        100        100          0          0          0          0
 12     933311        234          0          0          0          0        117        117          0          0          0          0
 13     139685         28          0          0          0          0         14         14          0          0          0          0
 14          0          0          0          0          0          0          0          0          0          0          0          0
 15          0          0          0          0          0          0          0          0          0          0          0          0
 16          0          0          0          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0          0          0          0

 *** Timewarps
 FEE       ADC      Pause     Resume    SYNC100    WR48-63       Disc        MBS  Undefined    Samples
  0          0          0          0          0          0          0          0          0          0
  1          0          0          0          0          0          0          0          0          0
  2          0          0          0          0          0          0          0          0          0
  3          0          0          0          0          0          0          0          0          0
  4          0          0          0          0          0          0          0          0          0
  5          0          0          0          0          0          0          0          0          0
  6          0          0          0          0          0          0          0          0          0
  7          0          0          0          0          0          0          0          0          0
  8          0          0          0          0          0          0          0          0          0
  9          0          0          0          0          0          0          0          0          0
 10          0          0          0          0          0          0          0          0          0
 11          0          0          0          0          0          0          0          0          0
 12          0          0          0          0          0          0          0          0          0
 13          0          0          0          0          0          0          0          0          0
 14          0          0          0          0          0          0          0          0          0
 15          0          0          0          0          0          0          0          0          0
 16          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0

 *** Program elapsed time:   10.115s (  973.186 blocks/s,  60.824 Mb/s)
Attachment 3: R18_19_analyse.txt
 *** TDR format 3.3.0 analyser - TD - May 2021
 *** ERROR: READ I/O error:       5002
                   blocks:       9700
          ADC data format:   70825654 ( 1030660.0 Hz)
        Other data format:    8568846 (  124694.5 Hz)
 Sample trace data format:          0 (       0.0 Hz)
         Undefined format:          0 (       0.0 Hz)
   Other data format type:      PAUSE:        268 (       3.9 Hz)
                               RESUME:        268 (       3.9 Hz)
                              SYNC100:       9956 (     144.9 Hz)
                              WR48-63:       9956 (     144.9 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:    8548398 (  124396.9 Hz)
                           Other info:          0 (       0.0 Hz)

   ADC data range bit set:          2 (       0.0 Hz)

                Timewarps:        ADC:          0 (       0.0 Hz)
                                PAUSE:          0 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:          0 (       0.0 Hz)
                              WR48-63:          0 (       0.0 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:          0 (       0.0 Hz)
                            Undefined:          0 (       0.0 Hz)
                         Sample trace:          0 (       0.0 Hz)

 *** Timestamp elapsed time:       68.719 s
 FEE  elapsed dead time(s) elapsed idle time(s)
  0                0.000               40.039
  1                0.526                0.000
  2                0.044                0.000
  3               16.252                0.000
  4                0.000               41.343
  5               16.542                0.000
  6                0.000               37.115
  7                4.958                0.000
  8                0.000               62.160
  9                0.000               48.394
 10                0.000               57.328
 11                0.000               39.469
 12                0.000                0.000
 13                0.000               54.952
 14                0.000                0.000
 15                0.000                0.000
 16                0.000                0.000
 17                0.000                0.000
 18                0.000                0.000
 19                0.000                0.000
 20                0.000                0.000
 21                0.000                0.000
 22                0.000                0.000
 23                0.000                0.000
 24                0.000                0.000
 25                0.000                0.000
 26                0.000                0.000
 27                0.000                0.000
 28                0.000                0.000
 29                0.000                0.000
 30                0.000                0.000
 31                0.000                0.000
 32                0.000                0.000

 *** Statistics
 FEE  ADC Data Other Data     Sample  Undefined      Pause     Resume    SYNC100    WR48-63       Disc        MBS      Other   HEC Data
  0     123341       4797          0          0          0          0          9          9          0       4779          0          0
  1   16458296       4022          0          0          5          5       2006       2006          0          0          0          0
  2      67098    4844172          0          0          2          2        649        649          0    4842870          0          2
  3   14877427    3705657          0          0        108        108       2346       2346          0    3700749          0          0
  4     201730         54          0          0          0          0         27         27          0          0          0          0
  5   18757776       4968          0          0        118        118       2366       2366          0          0          0          0
  6     275379         62          0          0          0          0         31         31          0          0          0          0
  7   18591077       4758          0          0         35         35       2344       2344          0          0          0          0
  8      87837         12          0          0          0          0          6          6          0          0          0          0
  9     156274         28          0          0          0          0         14         14          0          0          0          0
 10      61114         18          0          0          0          0          9          9          0          0          0          0
 11     292464         64          0          0          0          0         32         32          0          0          0          0
 12     736144        204          0          0          0          0        102        102          0          0          0          0
 13     139697         30          0          0          0          0         15         15          0          0          0          0
 14          0          0          0          0          0          0          0          0          0          0          0          0
 15          0          0          0          0          0          0          0          0          0          0          0          0
 16          0          0          0          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0          0          0          0

 *** Timewarps
 FEE       ADC      Pause     Resume    SYNC100    WR48-63       Disc        MBS  Undefined    Samples
  0          0          0          0          0          0          0          0          0          0
  1          0          0          0          0          0          0          0          0          0
  2          0          0          0          0          0          0          0          0          0
  3          0          0          0          0          0          0          0          0          0
  4          0          0          0          0          0          0          0          0          0
  5          0          0          0          0          0          0          0          0          0
  6          0          0          0          0          0          0          0          0          0
  7          0          0          0          0          0          0          0          0          0
  8          0          0          0          0          0          0          0          0          0
  9          0          0          0          0          0          0          0          0          0
 10          0          0          0          0          0          0          0          0          0
 11          0          0          0          0          0          0          0          0          0
 12          0          0          0          0          0          0          0          0          0
 13          0          0          0          0          0          0          0          0          0
 14          0          0          0          0          0          0          0          0          0
 15          0          0          0          0          0          0          0          0          0
 16          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0

 *** Program elapsed time:   10.125s (  958.025 blocks/s,  59.877 Mb/s)
Attachment 4: 220512_0155_stats.png
220512_0155_stats.png
Attachment 5: 220512_0217_biasleakage.png
220512_0217_biasleakage.png
Attachment 6: 220512_0215_temps.png
220512_0215_temps.png
Attachment 7: 220512_0214_stats.png
220512_0214_stats.png
Attachment 8: 220512_0410_biasLeak.png
220512_0410_biasLeak.png
Attachment 9: 220512_0410_temps.png
220512_0410_temps.png
Attachment 10: 220512_0408_stats.png
220512_0408_stats.png
Attachment 11: 220512_0604_biasLeakage.png
220512_0604_biasLeakage.png
Attachment 12: 220512_0603_temps.png
220512_0603_temps.png
Attachment 13: 220512_0601_stats.png
220512_0601_stats.png
  429   Wed May 11 19:08:41 2022 CB, PPWednesday 11th May - evening shift

20:19 no beam yet.

Statistic check (screenshot attached).

Temperatures OK (screenshot attached).

Bias and leakage currents OK (same screenshot as temps).

All ADC failed calibration. Clock and white rabbit all passed.

 

22:12 no beam yet.

Statistic check (screenshot attached).

Temperatures OK (screenshot attached).

Bias and leakage currents OK (same screenshot as temps).

All ADC failed calibration. Clock and white rabbit all passed.

Attachment 1: 16.png
16.png
Attachment 2: 22.png
22.png
Attachment 3: 13.png
13.png
Attachment 4: 14.png
14.png
  428   Wed May 11 07:04:13 2022 OH, NH, TDWednesday 11th May
08:00 Overnight at some point it appears that aida14 has stopped producing any ASIC data
      It is currently sending data pause requests and data push requests but nothing else - attachment 1
      telnet into aida14 and run top to see what the current state of the FEE is. AIDAExec currently using 0% CPU usage.
      FEE appears to have hung. - attachment 2
      Still passes all system wide checks.
      When performing a check load can see the tclsh process take up CPU usage

09:02 AIDA out of time sorter. Going to perform a power cycle to see if it recovers FEE14
      AIDA14 recovered but ASIC2 still missing
    
      Current working hypothesis is ASIC2 on AIDA14 is dead and that the problem with AIDA05 is either DSSD, cable or adaptor board.
      Given access would change the adapter board on AIDA05.
      Could consider installing the old AIDA05 FEE above AIDA14 and moving the cables up 1.
      Would retain the option of a speedybacktrack

      Alternatively the HEC events in AIDA05 are just above threshold. Can raise the HEC threshold
      A threshold of 0x4 reduced the rate in the HEC channels considerably
      A threshold of 0x6 removes entirely

      Note when the rate of HEC channels decreases can see the pulser in all channels of AIDA05 again. Showing that the adapter board is properly aligned

09:59 Had access so replaced the adapter card on AIDA05
      Did note that while replacing the card the right hand edge of the right hand ribbon cable was lifted slightly.
      This could also have been the cause


10:09 Restarted DAQ and there is no change to the behavior of AIDA05
      Still see events in HEC
      Again raising threshold to 0x5 in ASIC1 largely removes them. 

13.10 Attachments 3-5
      DAQ histogramming only enabled, waveforms/data transfer disabled, sampling ADCs OFF
      ASIC check load
      ASIC slow comparator all FEE64s all ASICS 0xa
      ADC data item stats

13.17 Attachments 6-9
      DAQ histogramming & data transfer enabled, waveforms disabled, sampling ADCs OFF
      ADC data item stats
      NewMerger/Tape Server

18:00 As part of tests in the afternoon it was investigated powering only one DSSD worth of FEEs at a time
      It was found that reasonable rates could be found for DSSD1 but poor rates only for DSSD2

      To try and help isolate the two detectors the jumpers 2 and 4 were removed from FEEs 13, 5 and 14 (Top, DSSD2)
      After powering on it was found there was no improvement and possible worse noise than before

      We were able to recover the rates in the 1DSSD setup so instead have removed FEE power from FEE5 and FEE14
Attachment 1: 220511_0806_aida14.png
220511_0806_aida14.png
Attachment 2: 220511_0817_aida14_top.png
220511_0817_aida14_top.png
Attachment 3: Screenshot_from_2022-05-11_13-11-09.png
Screenshot_from_2022-05-11_13-11-09.png
Attachment 4: Screenshot_from_2022-05-11_13-10-29.png
Screenshot_from_2022-05-11_13-10-29.png
Attachment 5: Screenshot_from_2022-05-11_13-09-12.png
Screenshot_from_2022-05-11_13-09-12.png
Attachment 6: Screenshot_from_2022-05-11_13-16-32.png
Screenshot_from_2022-05-11_13-16-32.png
Attachment 7: Screenshot_from_2022-05-11_13-16-24.png
Screenshot_from_2022-05-11_13-16-24.png
Attachment 8: Screenshot_from_2022-05-11_13-16-01.png
Screenshot_from_2022-05-11_13-16-01.png
Attachment 9: Screenshot_from_2022-05-11_13-15-45.png
Screenshot_from_2022-05-11_13-15-45.png
  427   Tue May 10 07:41:28 2022 NH TDTuesday 10 May
8:15am S4 CONTROLLED ACCESS to remove beam plug and check bPlas/AIDA response to beam

Reseat aida05 ribbon cable on adapter board while in...

No difference to rates, aida05 still shows hot asic01
aida14 seems seated OK, still asic02 basically empty

Noise level still high, threshold set to 0x20 (320 keV) which has decent rate

Start forwarding to MBS with 0x20 thresholds

10.12 slow comparator 0x20 -> 0xa

Attachments 1-5

Attachment 6 ADC data items 
             c. 150-180k per FEE64, c. 21Mb to disk

Attachment 7 FEE64 temps OK

Attachment 8 DSSSD bias & leakage currents OK


13.30 system checks

      N.B. aida05 ASIC temp low and pause/resume data rate high

Attachment 9 grafana monitor of DSSSD bias & leakage current & temperature OK

Attachments 10-15 spectra

Attacahments 16-19 system wide checks OK *except* WR error counter

Attachment 20 ADC data items 
             c. 150-180k per FEE64, c. 21Mb to disk

Attachment 21 FEE64 temps OK

Attachment 22 DSSSD bias & leakage currents OK

Attachments 23-29 newMerger stats

20:37 - Begin implantation test, raise threshold to 0x64 just to ensure system is stable

22:26 - No implantation test although we had some brief beam through AIDA.
      - FRS team having difficulty aligning beam still, will work overnight and try to implant tomorrow
        Will keep DAQ on 0x64 to eliminate deadtime issues for implantation
Attachment 1: Screenshot_from_2022-05-10_10-11-54.png
Screenshot_from_2022-05-10_10-11-54.png
Attachment 2: Screenshot_from_2022-05-10_10-10-09.png
Screenshot_from_2022-05-10_10-10-09.png
Attachment 3: Screenshot_from_2022-05-10_10-08-55.png
Screenshot_from_2022-05-10_10-08-55.png
Attachment 4: Screenshot_from_2022-05-10_10-08-02.png
Screenshot_from_2022-05-10_10-08-02.png
Attachment 5: Screenshot_from_2022-05-10_10-07-01.png
Screenshot_from_2022-05-10_10-07-01.png
Attachment 6: Screenshot_from_2022-05-10_10-06-30.png
Screenshot_from_2022-05-10_10-06-30.png
Attachment 7: Screenshot_from_2022-05-10_10-06-20.png
Screenshot_from_2022-05-10_10-06-20.png
Attachment 8: Screenshot_from_2022-05-10_10-06-15.png
Screenshot_from_2022-05-10_10-06-15.png
Attachment 9: Screenshot_from_2022-05-10_13-40-43.png
Screenshot_from_2022-05-10_13-40-43.png
Attachment 10: Screenshot_from_2022-05-10_13-40-01.png
Screenshot_from_2022-05-10_13-40-01.png
Attachment 11: Screenshot_from_2022-05-10_13-37-40.png
Screenshot_from_2022-05-10_13-37-40.png
Attachment 12: Screenshot_from_2022-05-10_13-37-33.png
Screenshot_from_2022-05-10_13-37-33.png
Attachment 13: Screenshot_from_2022-05-10_13-36-39.png
Screenshot_from_2022-05-10_13-36-39.png
Attachment 14: Screenshot_from_2022-05-10_13-35-06.png
Screenshot_from_2022-05-10_13-35-06.png
Attachment 15: Screenshot_from_2022-05-10_13-34-44.png
Screenshot_from_2022-05-10_13-34-44.png
Attachment 16: Screenshot_from_2022-05-10_13-31-05.png
Screenshot_from_2022-05-10_13-31-05.png
Attachment 17: Screenshot_from_2022-05-10_13-30-59.png
Screenshot_from_2022-05-10_13-30-59.png
Attachment 18: Screenshot_from_2022-05-10_13-30-52.png
Screenshot_from_2022-05-10_13-30-52.png
Attachment 19: Screenshot_from_2022-05-10_13-29-58.png
Screenshot_from_2022-05-10_13-29-58.png
Attachment 20: Screenshot_from_2022-05-10_13-29-42.png
Screenshot_from_2022-05-10_13-29-42.png
Attachment 21: Screenshot_from_2022-05-10_13-29-20.png
Screenshot_from_2022-05-10_13-29-20.png
Attachment 22: Screenshot_from_2022-05-10_13-28-48.png
Screenshot_from_2022-05-10_13-28-48.png
Attachment 23: Screenshot_from_2022-05-10_13-51-23.png
Screenshot_from_2022-05-10_13-51-23.png
Attachment 24: Screenshot_from_2022-05-10_13-51-05.png
Screenshot_from_2022-05-10_13-51-05.png
Attachment 25: Screenshot_from_2022-05-10_13-49-53.png
Screenshot_from_2022-05-10_13-49-53.png
Attachment 26: Screenshot_from_2022-05-10_13-49-41.png
Screenshot_from_2022-05-10_13-49-41.png
Attachment 27: Screenshot_from_2022-05-10_13-49-32.png
Screenshot_from_2022-05-10_13-49-32.png
Attachment 28: Screenshot_from_2022-05-10_13-49-12.png
Screenshot_from_2022-05-10_13-49-12.png
Attachment 29: Screenshot_from_2022-05-10_13-48-56.png
Screenshot_from_2022-05-10_13-48-56.png
  426   Mon May 9 08:36:23 2022 TD, NHMonday 9 May
09.32 Re-start AIDA

Attachments 1 & 2 1.8.W spectra 20us FSR

Attachments 3 & 4 1.8.L spectra
 aida10 98ch FWHM
 aida06 384 ch FWHM

Attachment 5 per FEE64 rate spectra

Attachments 6-13 system wide checks - all OK

Attachment 14 ADC data item stats
 n+n FEE64s c. 250k -> c. 370k
 aida01, 7 & 12 c. 200-400k 

Attachment 15 FEE64 temps OK

Attachment 16 DSSD bias & leakage current OK

Attachment 17 per FEE64 sys console status summary



10.15 Move triPlast Lemo-00 cabling away from aida02 DSSSD-adaptor PCB ribbon cable
      ASIC check load

Attachments 18 ADC data item stats
 no change to n+n FEE64s, very high p+n FEE64s now at lower rates

Attachments 19 & 20 1.8.W spectra 20us FSR


18.27 all ADC data rates very high
      aida05, asic #1 is again producing HEC data
      aida14, asic #2 producing very little data

Attachments 23 & 24 1.8.W spectra 20us FSR

Attachments 25 & 26 1.8.L spectra
 aida10 165ch FWHM

Attachment 27 per FEE64 rate spectra

Attachment 28 ADC data item stats

Attachment 29 FEE64 temps OK

Attachment 30 DSSD bias & leakage current OK

Attachments 21 & 22 1.8.L spectra
 aida10 65ch FWHM
 aida06 394ch FWHM

Attachment 31 Rate with threesholds at 0x64

14.00 Check 3x IC RF signals are not source of extrinsic source (c. 2MHz & c. 100kHz) observed. NT switches off all 3x RF signals - no change. 


14.30 Replace FEE64 aida05
      old d8:80:39:41:d7:cc -> d8:80:39:41:b4:0b
      old dhcpd.conf file backed up
Attachment 1: Screenshot_from_2022-05-09_09-58-10.png
Screenshot_from_2022-05-09_09-58-10.png
Attachment 2: Screenshot_from_2022-05-09_09-56-53.png
Screenshot_from_2022-05-09_09-56-53.png
Attachment 3: Screenshot_from_2022-05-09_09-54-56.png
Screenshot_from_2022-05-09_09-54-56.png
Attachment 4: Screenshot_from_2022-05-09_09-53-43.png
Screenshot_from_2022-05-09_09-53-43.png
Attachment 5: Screenshot_from_2022-05-09_09-51-41.png
Screenshot_from_2022-05-09_09-51-41.png
Attachment 6: Screenshot_from_2022-05-09_09-50-11.png
Screenshot_from_2022-05-09_09-50-11.png
Attachment 7: Screenshot_from_2022-05-09_09-50-07.png
Screenshot_from_2022-05-09_09-50-07.png
Attachment 8: Screenshot_from_2022-05-09_09-50-02.png
Screenshot_from_2022-05-09_09-50-02.png
Attachment 9: Screenshot_from_2022-05-09_09-48-13.png
Screenshot_from_2022-05-09_09-48-13.png
Attachment 10: Screenshot_from_2022-05-09_09-48-06.png
Screenshot_from_2022-05-09_09-48-06.png
Attachment 11: Screenshot_from_2022-05-09_09-48-00.png
Screenshot_from_2022-05-09_09-48-00.png
Attachment 12: Screenshot_from_2022-05-09_09-47-50.png
Screenshot_from_2022-05-09_09-47-50.png
Attachment 13: Screenshot_from_2022-05-09_09-47-27.png
Screenshot_from_2022-05-09_09-47-27.png
Attachment 14: Screenshot_from_2022-05-09_09-47-21.png
Screenshot_from_2022-05-09_09-47-21.png
Attachment 15: Screenshot_from_2022-05-09_09-47-12.png
Screenshot_from_2022-05-09_09-47-12.png
Attachment 16: Screenshot_from_2022-05-09_09-47-00.png
Screenshot_from_2022-05-09_09-47-00.png
Attachment 17: Screenshot_from_2022-05-09_09-36-29.png
Screenshot_from_2022-05-09_09-36-29.png
Attachment 18: Screenshot_from_2022-05-09_10-16-16.png
Screenshot_from_2022-05-09_10-16-16.png
Attachment 19: Screenshot_from_2022-05-09_10-17-40.png
Screenshot_from_2022-05-09_10-17-40.png
Attachment 20: Screenshot_from_2022-05-09_10-17-09.png
Screenshot_from_2022-05-09_10-17-09.png
Attachment 21: Screenshot_from_2022-05-09_10-32-52.png
Screenshot_from_2022-05-09_10-32-52.png
Attachment 22: Screenshot_from_2022-05-09_10-31-48.png
Screenshot_from_2022-05-09_10-31-48.png
Attachment 23: Screenshot_from_2022-05-09_18-16-39.png
Screenshot_from_2022-05-09_18-16-39.png
Attachment 24: Screenshot_from_2022-05-09_18-16-01.png
Screenshot_from_2022-05-09_18-16-01.png
Attachment 25: Screenshot_from_2022-05-09_18-15-32.png
Screenshot_from_2022-05-09_18-15-32.png
Attachment 26: Screenshot_from_2022-05-09_18-14-56.png
Screenshot_from_2022-05-09_18-14-56.png
Attachment 27: Screenshot_from_2022-05-09_18-13-43.png
Screenshot_from_2022-05-09_18-13-43.png
Attachment 28: Screenshot_from_2022-05-09_18-13-26.png
Screenshot_from_2022-05-09_18-13-26.png
Attachment 29: Screenshot_from_2022-05-09_18-13-16.png
Screenshot_from_2022-05-09_18-13-16.png
Attachment 30: Screenshot_from_2022-05-09_18-13-03.png
Screenshot_from_2022-05-09_18-13-03.png
Attachment 31: AnyDeskMSI_2022-05-09_21-45-12.png
AnyDeskMSI_2022-05-09_21-45-12.png
  425   Sun May 8 17:43:57 2022 NHAIDA FEE64 Layout
AIDA FEE64 layout with respect to the beam.
Attachment 1: AIDA_Adapter_Configuration(1).png
AIDA_Adapter_Configuration(1).png
  424   Sun May 8 15:10:45 2022 TDSunday 8 May
DSSSD bias and FEE64 power OFF during Ge install



Photographs of AIDA (triple) + DEGAS setup as of Sunday pm.

Attachments 1-4 side/front/oblique views - note AIDA snout droops and twists

Attachments 5 & 6 - position of the AIDA stand vertical adjustment threads (brass) 

The asymmetry upstream-downstream pre-dates this setup. To lift the end of the snout clear of the of lower DEGAS detectors all 3x 1/4 turns clockwise (viewed from above) were applied to each of the four threads.
Attachment 1: 20220508_160103.jpg
20220508_160103.jpg
Attachment 2: 20220508_160115.jpg
20220508_160115.jpg
Attachment 3: 20220508_160128.jpg
20220508_160128.jpg
Attachment 4: 20220508_160140.jpg
20220508_160140.jpg
Attachment 5: 20220508_160720.jpg
20220508_160720.jpg
Attachment 6: 20220508_160804.jpg
20220508_160804.jpg
  423   Sat May 7 13:27:39 2022 TDAIDA configuration
Adaptor PCB link settings

n+n FEE64s

aida02, 4, 6 & 8 - LK1 fitted, HV Lemo-00 not connected - left/right wafer n+n side bias ring -> gnd

p+n FEE64s

   top - aida09, 1, 10 & aida13, 5, 14 - LK2 & LK4 fitted - front/back FP -> gnd
bottom - aida11, 3, 12 & aida15, 7, 16 - LK2 & LK4 fitted - front/back FP -> gnd

   top - aida01, 5 - LK3 fitted - middle wafer n+n side bias ring -> gnd
bottom - aida03, 7 - LK3 fitted - DSSSD PCB gnd -> gnd

AIDA PSU

   top 1-2, 3-4, 9-10, 11-12
bottom 5-6, 7-8, 13-14, 15-16


ASIC settings 2021Apr29-13-16-00
     slow comparator 0x64 -> 0xa

BNC PB-5 settings
    amplitude 1.000V
    attenuation x1
    tail pulse
    tau_d 1ms
    polarity +
    frequency 25Hz
  422   Sat May 7 09:55:53 2022 TDSaturday 7 May
Attachments 1 & 2 per FEE64 1.8.L spectra 
 pulser peak widths aida01 121 ch FWHM, aida02 517 ch FWHM

Attachments 3 & 4 per FEE64 1.8.W spectra 20us FSR

Attachment 5 per FEE64 rate spectra

Attachment 6 - grafana DSSSD bias/leakage current for previous 7 days showing effect of day/night temp variations

Attachments 7 & 8 - merger & tape server c. 14Mb/s to disk - no errors reported by servers

Attachments 9-12 - system wide checks - all OK

Attachment 13 ADC data item stats

Attachment 14 FEE64 temps OK

Attachment 15 DSSSD bias & leakage currents OK

16.15 aida05 zero ADC data items stats
      merger restart, DAQ STOP/Go cycle to recover


      activity is S4 area near AIDA DSSSDs as install of DEGAS detectors commences

19.50 DEGAS install continues

      DSSSD bias and FEE64 power OFF
 
Attachment 1: Screenshot_from_2022-05-07_10-53-56.png
Screenshot_from_2022-05-07_10-53-56.png
Attachment 2: Screenshot_from_2022-05-07_10-52-54.png
Screenshot_from_2022-05-07_10-52-54.png
Attachment 3: Screenshot_from_2022-05-07_10-51-33.png
Screenshot_from_2022-05-07_10-51-33.png
Attachment 4: Screenshot_from_2022-05-07_10-51-06.png
Screenshot_from_2022-05-07_10-51-06.png
Attachment 5: Screenshot_from_2022-05-07_10-47-13.png
Screenshot_from_2022-05-07_10-47-13.png
Attachment 6: Screenshot_from_2022-05-07_10-46-40.png
Screenshot_from_2022-05-07_10-46-40.png
Attachment 7: Screenshot_from_2022-05-07_10-46-26.png
Screenshot_from_2022-05-07_10-46-26.png
Attachment 8: Screenshot_from_2022-05-07_10-46-16.png
Screenshot_from_2022-05-07_10-46-16.png
Attachment 9: Screenshot_from_2022-05-07_10-45-34.png
Screenshot_from_2022-05-07_10-45-34.png
Attachment 10: Screenshot_from_2022-05-07_10-45-28.png
Screenshot_from_2022-05-07_10-45-28.png
Attachment 11: Screenshot_from_2022-05-07_10-45-23.png
Screenshot_from_2022-05-07_10-45-23.png
Attachment 12: Screenshot_from_2022-05-07_10-45-17.png
Screenshot_from_2022-05-07_10-45-17.png
Attachment 13: Screenshot_from_2022-05-07_10-45-08.png
Screenshot_from_2022-05-07_10-45-08.png
Attachment 14: Screenshot_from_2022-05-07_10-43-29.png
Screenshot_from_2022-05-07_10-43-29.png
Attachment 15: Screenshot_from_2022-05-07_10-43-14.png
Screenshot_from_2022-05-07_10-43-14.png
  421   Fri May 6 09:31:41 2022 TD NHFriday 6 May
10:30 DAQ appears to have stopped

      Attachment 2 - DSSSD bias and leakage current OK

      Attachments 1, 3-5 - zero rate adc data items all FEE64s - note aida08 adc data item counter zero

      RESET/SETUP/GO aida08 *only*
      DAQ STOP *all* FEE64s


      Attachment 6 - FEE64 temperatures OK

      Attachments 7-23 - system wide checks - all OK

10:48 DAQ GO *all* FEE64s
      all spectra zero'd

      attachment 29 - ADC data items
       n+n FEE64s aida02, 4, 6 & 8 c. 250k
       p+n FEE64s aida07, 9 & 12 c. 100-150k, all other FEE64s < 100k

      attachments 26 & 27 1.8.L spectra
       aida01 peak width 112ch FWHM
       aida02 peak width 485 ch FWHM

      attachments 24 & 25 1.8.W spectra 20us FSR
      
      attachment 30 - aida05 1*H spectra - aida05 asic#1 is producing HEC data

      attachment 31 - aida05 also producing disc data

      attachments 32 & 33 merger and tape server OK - 14MB/s to disk - no data to MBS currently? 


18.36 Disconnect ribbon cabling from adaptor PCB - continue to observe HEC events from aida05 asic#1
      Disconnect test inputs from adaptor PCB - continue to observe HEC events from aida05 asic#1

      Conclude aida05 asic#1 has probably failed in operation and should be replaced. Other FEE64s are currently in CRYRING hall and measurement is in progress - ML says we should be able to access Monday.
Attachment 1: Screenshot_from_2022-05-06_10-27-27.png
Screenshot_from_2022-05-06_10-27-27.png
Attachment 2: Screenshot_from_2022-05-06_10-26-59.png
Screenshot_from_2022-05-06_10-26-59.png
Attachment 3: Screenshot_from_2022-05-06_10-26-33.png
Screenshot_from_2022-05-06_10-26-33.png
Attachment 4: Screenshot_from_2022-05-06_10-25-44.png
Screenshot_from_2022-05-06_10-25-44.png
Attachment 5: Screenshot_from_2022-05-06_10-24-54.png
Screenshot_from_2022-05-06_10-24-54.png
Attachment 6: Screenshot_from_2022-05-06_10-42-18.png
Screenshot_from_2022-05-06_10-42-18.png
Attachment 7: Screenshot_from_2022-05-06_10-41-06.png
Screenshot_from_2022-05-06_10-41-06.png
Attachment 8: Screenshot_from_2022-05-06_10-41-00.png
Screenshot_from_2022-05-06_10-41-00.png
Attachment 9: Screenshot_from_2022-05-06_10-40-54.png
Screenshot_from_2022-05-06_10-40-54.png
Attachment 10: Screenshot_from_2022-05-06_10-40-09.png
Screenshot_from_2022-05-06_10-40-09.png
Attachment 11: Screenshot_from_2022-05-06_10-39-07.png
Screenshot_from_2022-05-06_10-39-07.png
Attachment 12: Screenshot_from_2022-05-06_10-38-58.png
Screenshot_from_2022-05-06_10-38-58.png
Attachment 13: Screenshot_from_2022-05-06_10-38-35.png
Screenshot_from_2022-05-06_10-38-35.png
Attachment 14: Screenshot_from_2022-05-06_10-38-21.png
Screenshot_from_2022-05-06_10-38-21.png
Attachment 15: Screenshot_from_2022-05-06_10-38-16.png
Screenshot_from_2022-05-06_10-38-16.png
Attachment 16: Screenshot_from_2022-05-06_10-38-10.png
Screenshot_from_2022-05-06_10-38-10.png
Attachment 17: Screenshot_from_2022-05-06_10-38-02.png
Screenshot_from_2022-05-06_10-38-02.png
Attachment 18: Screenshot_from_2022-05-06_10-37-52.png
Screenshot_from_2022-05-06_10-37-52.png
Attachment 19: Screenshot_from_2022-05-06_10-37-46.png
Screenshot_from_2022-05-06_10-37-46.png
Attachment 20: Screenshot_from_2022-05-06_10-37-39.png
Screenshot_from_2022-05-06_10-37-39.png
Attachment 21: Screenshot_from_2022-05-06_10-37-31.png
Screenshot_from_2022-05-06_10-37-31.png
Attachment 22: Screenshot_from_2022-05-06_10-36-24.png
Screenshot_from_2022-05-06_10-36-24.png
Attachment 23: Screenshot_from_2022-05-06_10-35-52.png
Screenshot_from_2022-05-06_10-35-52.png
Attachment 24: Screenshot_from_2022-05-06_10-56-31.png
Screenshot_from_2022-05-06_10-56-31.png
Attachment 25: Screenshot_from_2022-05-06_10-55-56.png
Screenshot_from_2022-05-06_10-55-56.png
Attachment 26: Screenshot_from_2022-05-06_10-55-13.png
Screenshot_from_2022-05-06_10-55-13.png
Attachment 27: Screenshot_from_2022-05-06_10-53-53.png
Screenshot_from_2022-05-06_10-53-53.png
Attachment 28: Screenshot_from_2022-05-06_10-51-00.png
Screenshot_from_2022-05-06_10-51-00.png
Attachment 29: Screenshot_from_2022-05-06_10-50-25.png
Screenshot_from_2022-05-06_10-50-25.png
Attachment 30: Screenshot_from_2022-05-06_11-09-15.png
Screenshot_from_2022-05-06_11-09-15.png
Attachment 31: Screenshot_from_2022-05-06_11-02-33.png
Screenshot_from_2022-05-06_11-02-33.png
Attachment 32: Screenshot_from_2022-05-06_11-19-48.png
Screenshot_from_2022-05-06_11-19-48.png
Attachment 33: Screenshot_from_2022-05-06_11-19-27.png
Screenshot_from_2022-05-06_11-19-27.png
  420   Wed May 4 16:50:04 2022 NHAIDA 4th May
bPlas has been mostly cabled in now

Reseat LAN cable (AIDA Switch -> AIDA-3) and reachieve 1 gbit/s speed as needed.

CHeck DAQ with MBS, working well (16 MB/s). Performance seems better than before on MBS side too

Note implant counts in aida05, observable in rate spectra (fig 1)
Also a few early channels in aida10 seem empty

ASIC check-loads didn't fix
Try powercycle... also didn't fix

Tomorrow will inspect adapter board and ribbon cable, perhaps it is loose.

Absolute rates otherwise seem OK (fig 2) aida01 and aida07 are both very hot.
aida01 is usually quieter but aida07 has always been observed running fast
Attachment 1: Screenshot_20220504_174929.png
Screenshot_20220504_174929.png
Attachment 2: Screenshot_20220504_175201.png
Screenshot_20220504_175201.png
  419   Tue May 3 14:15:17 2022 NHAIDA Updates 2 May
PAUSE/RESUME stats are dominated by 2x FEE64s - aida07 and aida16

These FEE64s also have the highest rates of discriminator data?

> The snout endcap has been sealed with new copper tape over the red tape
> 
> Observe significant drop in leakage current on DSSD #2 down to 7 uA (about 0.5 uA above DSSD #1) as it looked when the black cloth was on
> It is possible more sealing needs to be done when bPlas is biased as bPlas is more sensitive
> 
> 
> 
> -
> 
> the AIDA LAN network interface is only running at 100 MBit/s - this will be problematic during the run
> When 2A is free will try to reconnect cable to see if it returens to 1 Gbit/s
> 
> -
> 
> Preliminary alpha background analysis (over weekend, 90 hours statistics)
> 
> Cause of sporadic PAUSE/RESUME is not clear as rate is nearly always 0 outside of aida02/aida04
> 
> ======================================
> AIDA Unpacker Analysis
> --------------------------------------
> First event: Thu Apr 28 19:50:20 2022
> Last event : Mon May  2 14:04:03 2022
> Duration   : 324823 seconds
> --------------------------------------
> MBS Events   (M):     912899 (2 Hz)
> AIDA Words   (W): 7472078760 (-3441 Hz)
> ADC Words    (A): 2742871865 (-4778 Hz)
> -- Decays    (D): 2728717223 (-4821 Hz)
> -- Implants  (I):   14154642 (43 Hz)
> Info Words   (F): 4729206895 (1 kHz)
> -- SYNC48    (S):    2093328 (6 Hz)
> -- SYNC63    (s):    2093328 (6 Hz)
> -- PAUSE     (P):    3823161 (11 Hz)
> -- RESUME    (R):    3823161 (11 Hz)
> -- Discrim   (D): 1390520565 (4 kHz)
> -- Scaler    (C): 3326853352 (3 kHz)
> -- Unknown   (U):          0 (0 Hz)
> Timewarps    (T):         20 (0 Hz)
> Dead Time    (X): 5024.43 s (1.54682%)
> ======================================
> ======================================
> Individual FEE Overview
> ======================================
>  FEE |          W          A          D          I          F          S          s     P     R          D     C     U     T X
>    1 |       9593       6985       6628        357       2608          2          2     0     0       2604     0     0     0 0
>    2 |   31134221   31115668   31115166        502      18553       8183       8183     0     0       2187     0     0     0 0
>    3 |  717765720      17634      17218        416  717748086     216426     216426     0     0       1653 717313581     0     0 0
>    4 | 3467468322 2684820687 2684820157        530  782647635    1293089    1293089     0     0   62747129 717314328     0    20 0
>    5 |     177575     108631     107352       1279      68944         21         21    20    20      68862     0     0     0 0.00102142
>    6 |    8371118    8361524    8358540       2984       9594       1350       1350    23    23       6848     0     0     0 10.6775
>    7 | 1396478942    4860615      57719    4802896 1391618327     346048     346048 1286994 1286994  446838393 941513850     0     0 3071.3
>    8 |  954933629    3978543    3975110       3433  950955086     118195     118195     5     5       7093 950711593     0     0 5.47318
>    9 |       9877       7923       7429        494       1954          4          4     0     0       1946     0     0     0 0
>   10 |       8076       6236       5917        319       1840          4          4     0     0       1832     0     0     0 0
>   11 |      26792      24399      24029        370       2393          3          3     0     0       2387     0     0     0 0
>   12 |      18271      16025      15382        643       2246          6          6     0     0       2234     0     0     0 0
>   13 |    1094646      51444      48305       3139    1043202        195        195     1     1    1042810     0     0     0 0.179207
>   14 |    7314376     152823      84786      68037    7161553        906        906 18158 18158    7123425     0     0     0 32.882
>   15 |     438092      22117      20290       1827     415975         56         56   251   251     415361     0     0     0 4.91581
>   16 |  886829510    9320611      53195    9267416  877508899     108840     108840 2517709 2517709  872255801     0     0     0 4970.9
  418   Mon May 2 13:08:49 2022 NHAIDA Updates 2 May
The snout endcap has been sealed with new copper tape over the red tape

Observe significant drop in leakage current on DSSD #2 down to 7 uA (about 0.5 uA above DSSD #1) as it looked when the black cloth was on
It is possible more sealing needs to be done when bPlas is biased as bPlas is more sensitive



-

the AIDA LAN network interface is only running at 100 MBit/s - this will be problematic during the run
When 2A is free will try to reconnect cable to see if it returens to 1 Gbit/s

-

Preliminary alpha background analysis (over weekend, 90 hours statistics)

Cause of sporadic PAUSE/RESUME is not clear as rate is nearly always 0 outside of aida02/aida04

======================================
AIDA Unpacker Analysis
--------------------------------------
First event: Thu Apr 28 19:50:20 2022
Last event : Mon May  2 14:04:03 2022
Duration   : 324823 seconds
--------------------------------------
MBS Events   (M):     912899 (2 Hz)
AIDA Words   (W): 7472078760 (-3441 Hz)
ADC Words    (A): 2742871865 (-4778 Hz)
-- Decays    (D): 2728717223 (-4821 Hz)
-- Implants  (I):   14154642 (43 Hz)
Info Words   (F): 4729206895 (1 kHz)
-- SYNC48    (S):    2093328 (6 Hz)
-- SYNC63    (s):    2093328 (6 Hz)
-- PAUSE     (P):    3823161 (11 Hz)
-- RESUME    (R):    3823161 (11 Hz)
-- Discrim   (D): 1390520565 (4 kHz)
-- Scaler    (C): 3326853352 (3 kHz)
-- Unknown   (U):          0 (0 Hz)
Timewarps    (T):         20 (0 Hz)
Dead Time    (X): 5024.43 s (1.54682%)
======================================
======================================
Individual FEE Overview
======================================
 FEE |          W          A          D          I          F          S          s     P     R          D     C     U     T X
   1 |       9593       6985       6628        357       2608          2          2     0     0       2604     0     0     0 0
   2 |   31134221   31115668   31115166        502      18553       8183       8183     0     0       2187     0     0     0 0
   3 |  717765720      17634      17218        416  717748086     216426     216426     0     0       1653 717313581     0     0 0
   4 | 3467468322 2684820687 2684820157        530  782647635    1293089    1293089     0     0   62747129 717314328     0    20 0
   5 |     177575     108631     107352       1279      68944         21         21    20    20      68862     0     0     0 0.00102142
   6 |    8371118    8361524    8358540       2984       9594       1350       1350    23    23       6848     0     0     0 10.6775
   7 | 1396478942    4860615      57719    4802896 1391618327     346048     346048 1286994 1286994  446838393 941513850     0     0 3071.3
   8 |  954933629    3978543    3975110       3433  950955086     118195     118195     5     5       7093 950711593     0     0 5.47318
   9 |       9877       7923       7429        494       1954          4          4     0     0       1946     0     0     0 0
  10 |       8076       6236       5917        319       1840          4          4     0     0       1832     0     0     0 0
  11 |      26792      24399      24029        370       2393          3          3     0     0       2387     0     0     0 0
  12 |      18271      16025      15382        643       2246          6          6     0     0       2234     0     0     0 0
  13 |    1094646      51444      48305       3139    1043202        195        195     1     1    1042810     0     0     0 0.179207
  14 |    7314376     152823      84786      68037    7161553        906        906 18158 18158    7123425     0     0     0 32.882
  15 |     438092      22117      20290       1827     415975         56         56   251   251     415361     0     0     0 4.91581
  16 |  886829510    9320611      53195    9267416  877508899     108840     108840 2517709 2517709  872255801     0     0     0 4970.9
Attachment 1: elog_temps1.png
elog_temps1.png
Attachment 2: alpha_1d.png
alpha_1d.png
ELOG V3.1.4-unknown