14.12 DAQ stalled as expected
      Most recent file R86_245 last accessed 06.26

      DSSSD bias & leakage current OK - see attachments 1 & 2
      FEE64 temperatures OK - see attachment 3
     
      Some merger errors messages - all link 4 - see attachment 4
  
      All system wide checks OK *except SYNC errors & SYNC pulses - see attachments 5-8

      Restart merger
      DAQ STOP OK
      DAQ RESET & SETUP OK
      ReSYNC OK
      All system-wide checks OK
      Disable H & data transfer #1
      Restart merger & tapeserver
      Enable H & data transfer #1

14.34 DAQ started (R87)

19.50 DAQ continues (R87)

      System-wide checks OK *except* SYNC errors & SYNC pulses - see attachments 9-12

10:08 Normal system restart ok
      System currently in 8DSSD configuration as was left in June 2018

      Bias and leakage currents ok - attachment 1
      Temp +24.0 deg C, H 46.8%, D.P +11.7 deg C
      FEE Temperatures ok - attachment 2
      System wide checks all ok (aida7 fails ADC)
      
      DAQ Started with pulser test input
      Statistics ok - attachment 3
      Hit rate laout for first 30 FEEs (Note second ASIC in FEE29 still missing) - attachment 4
      Waveforms for Layout 7 (Front side) ok - attachment 5
      Waveforms for Layout 8 (Back side) good (nnaida 7 missing due to failed ADC check) - attachment 6

MSL type BB18 serial # 2977-20 bias +100V I_L +12.0uA T=23 deg C
(biased by nnaida # 3 & 5 )

DSSSD - AIDA adaptor PCB cabling
 
4 off LH Coupler (Kapton PCB, 5cm), 2x 34-way Samtec ribbon cable (62cm p+n, 53cm n+n),
RH coupler(Kapton PCB, 10cm)
 + 3M 1245 1.4mil copper foil screen ribbon cables (i.e. not RH & LH couplers)
 + drain wires -> gold-plated Lemo-00 test input connectors

to FEE64s in 4x FEE module crates

Snout drain wire to FEE module Cu block

nnaida 3 & 8 AIDA adaptor PCB rev C
nnaida 5 & 9 AIDA adaptor PCB rev C (LK1 fitted) 
ground links LK3 & LK7 fitted to nnaida3, 5, 8, & 9 AIDA adaptor PCBs

*** Note added 11.58 sat 30 Apr 2016
Detector bias cable to nnaida3 & 5 adaptor PCBs *but* LK1 fitted to nnaida9 *not* nnaida5.
This has been set this way since start of current series of tests Wed 27 April. Connecting
via a different adaptor PCB to that connected to the CAEN N1419 bias supply causes significant
noise issues presumably due to high resistance path via 2x adaptor PCB to DSSSD cables
and the DSSSD bias bus line on the DSSSD.

Heavy duty copper cable connects copper front end frames of FEE modules
Nitto 5011N conductive gasket between FEE module and front end frames
 
Standard ASIC settings
 nnaida 3 & 8 - negative input
 nnaida 5 & 9 - positive input
 
Pulser BNC PB-4 
Fall time 1.0ms
Rate c.100Hz
Delay min
Amplitude 50,000
Polarity -
Pulse top Tail
Atten 10x
+ polarity via EG&G Ortec 433A Sum & Invert Amplifier

Pulser peak widths broad, structured, non gaussian c. 200ch FWHM

attachments 1-4 ASIC # 1 waveforms for each acqserver with all acqservers connected to same PSU

attachments 5-6 for comparison, from yesterday, with acqservers from two different PSUs - note
absence of HF noise in attachments 1-4 cf. attachments 5-6. 

10:27 Bias and leakage currents ok - attachment 1
      Fee temperatures ok - attachment 2
      Temperature 24.7C, Humdidity 44.5%, D.P +11.8C
      System wide checks all ok
      Currently on file R9_69

      Vandle guys are about to start building. They have been made aware to avoid the power cables for the
chiller and AIDA
      Also of the network cables and HDMI cables.

14:46 Bias and leakage currents ok - attachment 3
      System wide checks all ok
      FEE temperatures all ok
      Currently on R9_70

19:55 Bias and leakage currents ok - attachment 4
      Fee temperatures ok
      System wide checks ok
      

11.00 Detector biases and leakage currents - see attachments 1-2
  
      FEE64 temperatures - see attachment 3

      DAQ ran overnight without problem - statistics - attachments 4-8

      Plan for today

      1) Develop AIDA external trigger output for fast LEC/MEC discriminator threshold 0x4, or lower

      2) Replace non-rev B MACB with rev B MACB in master timestamp chain (NIM module #2) 

16.15 Non-rev B MACB in master timestamp chain replaced with rev B MACB module

      FEE64s power cycled and rebooted.

      System ReSYNC'd successfully 

      System-wide tests 
            clock states 24/24 pass
            adc calibration 24/24 pass
            ASIC clock timestamp check 23/24 pass, nnaida19 fails
            SYNC errors not counting 23/24 pass, nnaida19 fails

      N.B. We obtained the same nnaida19 errors before rpelacing non-rev B MACB *and* with FEE64
           previously installed as nnaida19

      Module timestamps - see attachment 9

      Master timestamp module nnaida6 - all clocks locked


      Fitted ground jumpers to front and rear field plate connections on all four adaptor PCBs
      for each of DSSSD#2 - #5, i.e. DSSSD#1 - #6 now have field plate ground jumpers fitted.

      See attachments 10 & 11 showing discriminator rates before and after fitting these additional
      ground jumpers (LEC/MEC fast discriminator threshold 0x4 or 0x5)

18.25 NP1306 R58
      
      207Bi + 60Co LEC/MEC disc threshold test
      ASIC settings 2016Jun03-21.03.42

21.05 End R58

21.30 Moved extant data acquisition server options files to /MIDAS/DB/EXPERIMENT/AIDA/nnaida?/CONTENTS.BAK-030616
      Saved options for all data acquisition servers with
        TS_SYNC_PHASE = 0x3
        MACB_TRIG_MODE = 0x4
        ASIC Settings DB key 2016Jun03-21.03.42
        (all LEC/MEC fast disc settings 0x5 *except* nnaida5 & nnaida6 = 0x20)

       

10.56 DAQ running OK - data transfer not enabled

      BNC PB-4
      Amplitude 90,000
      x5 attenuator IN
      t_r 100ns tau_d 50us
      - polarity
      frequency range 25Hz
      delay MIN, tail pulse, INT ref

      ASIC settings 2017Oct23-14.38.03
      *all* FEE64s slow comparators 0xa *except* nnaida19-22 0xc

      Detector biases & leakage currents OK - see attachments 1
      FEE64 Temperatures OK - see attachment 2
      DAQ statistics OK - see attachment 3
      *all* system wide checks OK *except* nnaida21 fails ADC calibration

11.37 Attempt to use external 50MHz clock for Master MACB (w/ 50MHz front panel label)

      NIM 50MHz -> NIM-TTL translator -> Master MACB (rear panel)

      Master MACB mode 0 -> 5

      Fails to ReSync, clock errors etc - see attachments 1-5

      Change NIM 50MHz -> 25MHz (to improve signal quality) and attempt ReSync (no power FEE64 power cycle) - no
change 

14.30 Master MACB mode -> 0

      NIM 25MHz -> Master MACB correlation clock input (front panel)

      BNC PB_4 frequency c. 26Hz
      Trigger output -> SCA -> LeCroy 429A -> Master MACB accepted trigger input

./analyser2 v /TapeData/Sept2016/R5_0 | grep MBS

 :
 :

 ***     MBS information: block:   471 ptr:  7203 data: 0x86807CAB module:  6 information type:  8 information
index:  0 MBS data:7CAB ts: 0x00000017AC2C4582
 ***     MBS information: block:   471 ptr:  7205 data: 0x8681575E module:  6 information type:  8 information
index:  1 MBS data:575E ts: 0x00000017AC2C4582
 ***     MBS information: block:   471 ptr:  7207 data: 0x86820000 module:  6 information type:  8 information
index:  2 MBS data:0000 ts: 0x00000017AC2C4582
 ***     MBS information: block:   471 ptr: 15767 data: 0x86800FA8 module:  6 information type:  8 information
index:  0 MBS data:0FA8 ts: 0x00000017AC669176
 ***     MBS information: block:   471 ptr: 15769 data: 0x8681576D module:  6 information type:  8 information
index:  1 MBS data:576D ts: 0x00000017AC669176
 ***     MBS information: block:   471 ptr: 15771 data: 0x86820000 module:  6 information type:  8 information
index:  2 MBS data:0000 ts: 0x00000017AC669176

0x576D0FA8 - 0x575E7CAB = 0xE92FD = 955133 * 40ns = 38205320ns = 38.2ms => 26Hz
0x00000017AC669176 - 0x00000017AC2C4582 = 0x3a4bf4 = 3820532 * 10ns = 38205320ns QED


15.45 Master MACB mode 0 -> 5
      
      NIM 50MHz -> Phillips 726 NIM-TTL-NIM -> Master MACB external clock (rear panel)
      Phillips 726 provides better quality TTL signal with c. 3ns rise/falltimes and 3.5V amplitude into 50 Ohm
cf. NIM-TTL-NIM module used previously
      See http://www.phillipsscientific.com/pdf/726ds.pdf

      RESET/SETUP DAQ OK 

      System-wide checks OK *except* nnaida19 fails ASIC timestamp check (no change)

      Start test run R6_0 to disk but no AIDA issue correlation triggers are recorded

      Can manually issue correlation triggers and resets via Correlation Status & Control tab

      Observe statistics count 3x for each trigger and reset - see https://elog.ph.ed.ac.uk/AIDA/362

aidas1> ./analyser2 v /TapeData/Sept2016/R6_0 | grep MBS

 :
 :
 Test #1 - manual correlation trigger

 ***     MBS information: block:  9626 ptr:  9991 data: 0x86807774 module:  6 information type:  8 information
index:  0 MBS data:7774 ts: 0x00000077517D2816
 ***     MBS information: block:  9626 ptr:  9993 data: 0x86817213 module:  6 information type:  8 information
index:  1 MBS data:7213 ts: 0x00000077517D2816
 ***     MBS information: block:  9626 ptr:  9995 data: 0x86820003 module:  6 information type:  8 information
index:  2 MBS data:0003 ts: 0x00000077517D2816

 :
 :
 Test #2 - manual correlation trigger x2 separated by ~10s

 ***     MBS information: block: 16751 ptr: 10787 data: 0x86807285 module:  6 information type:  8 information
index:  0 MBS data:7285 ts: 0x000000833335145A
 ***     MBS information: block: 16751 ptr: 10789 data: 0x86816A81 module:  6 information type:  8 information
index:  1 MBS data:6A81 ts: 0x000000833335145A
 ***     MBS information: block: 16751 ptr: 10791 data: 0x86820006 module:  6 information type:  8 information
index:  2 MBS data:0006 ts: 0x000000833335145A

 ***     MBS information: block: 16929 ptr:  8209 data: 0x86807BC5 module:  6 information type:  8 information
index:  0 MBS data:7BC5 ts: 0x000000837F39395A
 ***     MBS information: block: 16929 ptr:  8211 data: 0x86817D82 module:  6 information type:  8 information
index:  1 MBS data:7D82 ts: 0x000000837F39395A
 ***     MBS information: block: 16929 ptr:  8213 data: 0x86820006 module:  6 information type:  8 information
index:  2 MBS data:0006 ts: 0x000000837F39395A

0x67D827BC5 − 0x66A817285 = 0x13010940 = 318835008 × 20ns = 6376700160ns = 6.4s

0x837F39395A - 0x833335145A = 0x4C042500 = 1275340032 * 10ns = 12753400320ns = 12.8s => correlation clock
frequency = 25MHz *not* 50MHz

 :
 :
 Test #3 - manual correlation reset, followed by correlation trigger ~2s later 

 ***     MBS information: block: 17065 ptr: 15987 data: 0x86804D22 module:  6 information type:  8 information
index:  0 MBS data:4D22 ts: 0x00000083B96F05A6
 ***     MBS information: block: 17065 ptr: 15989 data: 0x86810344 module:  6 information type:  8 information
index:  1 MBS data:0344 ts: 0x00000083B96F05A6
 ***     MBS information: block: 17065 ptr: 15991 data: 0x86820000 module:  6 information type:  8 information
index:  2 MBS data:0000 ts: 0x00000083B96F05A6

0x3444d22 = 54807842 * 40ns = 2192313680ns = 2.2s

*re-check*

Correlation trigger input OK, pulse width 500ns
External clock is 50MHz
MACB master mode 6

So ... correlation clock frequency is 25MHz and Correlation DAQ Accepted Trigger not working

18.30 Per PCS request

      Master MACB clock output  25Mhz
                  reset output  not observed at DAQ GO as expected

11.31 DAQ continues OK

      ASIC settings 2019May19-17.54.03
      slow comparator 0xa

      BNC PB-4 Pulser 
      Amplitude 90,000
      x5 attenuator IN
      Frequency 25Hz range
      t_r 100ns, tau_d 50us
      - polarity
      Delay MIN, tail pulse

      all system wide checks OK *except* nnaida7 fails ADC calibration

      detector biases & leakage currents OK - see attachments 1 & 2

      FEE64 temperatures OK - see attachment 3
      good event statistics OK - see attachments 4
      12x < 20k, 21x < 50k, 24x < 100k - cf. yesterday 10x < 20k, 21x < 50k

      1.8.L spectra - see attachments 5 & 6
      nnaida1 74 ch FWHM, nnaida3 78 ch FWHM - same as yesterday
      p+n FEE64s 1.8.W spectra 20us FSR - attachment 7
      n+n FEE64s 1.8.W spectra 20us FSR - attachment 8
      no indication of light ions with beam to ZDS & F11

      FEE64 Rate spectra - attachment 9

      Check ASIC Control *all* FEE64s & ASICs
      zero all spectra

17.21 detector biases & leakage currents OK - see attachments 10 & 11

      FEE64 temperatures OK - see attachment 12
      good event statistics OK - see attachments 13
      11x < 20k, 19x < 50k, 21x < 100k - cf. 11.30 today 12x < 20k, 21x < 50k, 24x <100k
      increased rates observed for nnaida17, 20 & 23 - all DSSSD#1 - nnaida22 ~15k -> ~30k

      Check ASIC Control *all* FEE64s & ASICs
      zero all spectra

      cycled DSSSD#1 bias - no change
      
      nnaida2 - no good events
      unable to DAQ stop nnaida20 & nnaida23
      unable to DAQ reset nnaida20 & nnaida23
      obtain timeout errors from nnaida20 and DAQ error status for nnaida20 & nnaida23
      reboot nnaida20 & nnaida23
      DAQ reset
      DAQ setup

      ASIC settings 2019May19-17.54.03
      slow comparator 0x64 -> 0xa

      Check ASIC Control *all* FEE64s & ASICs
      zero all spectra

      still observe increased rates for nnaida17, 20 & 23 - all DSSSD#1 - nnaida22 ~32k

18.05 DAQ statistics return to ~normal - attachment 14
      10x < 20k, 21x < 50k, 24x < 100k
              11.31 17.21 18.05
      nnaida17  13k  100k  43k
      nnaida20  59k  136k  70k
      nnaida22  16k   32k  26k
      nnaida23  46k  111k  53k

07.00 DAQ stopped
      Overnight run for background alphas

10.07 FEE64 temperatures OK - attachment 1
      DSSSD bias +100V leakage current +8.210uA - attachment 2

      Stats - attachments 3-12


     per FEE64 1.8.W spectra 200us FSR - attachments 21-22
     per FEE64 rate spectra - attachment 23

10.45 Power cycle and reboot

     DSSSD bias +100V leakage current +8.365uA

     All system wide checks OK

     Slow comparator 0x64

     per FEE64 rate spectra - attachment 1
     per FEE764 1.8.W spectra - attachments 16 & 17 ( ignore attachments 2 & 3 - bias off/low )
     adc data item stats - attachment 4


     slow comparator 0xf

     per FEE64 rate spectra - attachment 5
     adc data item stats - attachment 6


     slow comparator 0xe

     per FEE64 rate spectra
     adc data item stats - attachment 7


     slow comparator 0xd

     per FEE64 rate spectra - attachment 8
     adc data item stats - attachment 9


     slow comparator 0xc

     per FEE64 rate spectra - attachment 10
     adc data item stats - attachment 11

     slow comparator 0xb

     per FEE64 rate spectra - attachment 12
     adc data item stats - attachment 13


     slow comparator 0xa

     per FEE64 rate spectra - attachment 14
     adc data item stats - attachment 15

00.40-02.30 Discussion with PJCS
 
      Confirm both NIM bin voltages OK

      Move master MACB to lower NIM bin (with BNC PB_4, Ortec 433, 2x CAEN N1419B, 1x CAEN N1419)
       successful ReSYNC
       system-wide checks clock test fails from zero to all of the following FEE64s - nnaida7, 8, 15 & 16
       SYNC count test OK *except* nnaida19 which has greater number of SYNC pulses
       online analysis indicates nnaida19 ADC data not synchronised with other FEE64s by ~200ms

      Replace master MACB rev B mode 0 with MACB module #13 mode 5 (external clock 50MHz TTL)
      Move MACB module 2 from upper NIM bin to lower NIM bin
      Swap nnaida19/port1 <-> nnaida20/port2 

      Power cycle FEE64s and restart
      System wide checks fail - see attachments 1-4

      Re-seat HDMI cabling master MACB -> MACB modules 3-6
      System wide checks fail - see attachments 7-10

      Swap ports of 2x HDMI cables from master MACB
      Power cycle FEE64s
      Re_SYNC & system wide checks fail - see attachments 11-15

10.20 ReSYNC fails for all TS_SYNC_PHASE settings 0-3

      DK checks MACB mode settings
      master -> 5
      module #2 2
      module #3 2
      modules #4-11 3

      Check HDMI cable seating nnaida6 to master MACB

      No change - ReSYNC fails, system wide checks fail as before

      Per PJCS e-mail check MACB settings per https://elog.ph.ed.ac.uk/AIDA/258
      and change MACB module # 7 from mode 3 to mode 2. MACB settings are now

      MACB #  1  2  3  4  5  6  7  8  9 10 11      spare 1   spare 2
      Mode    5  2  3  3  3  3  2  3  3  3  3
      Rev B      y  y                              50MHz?    y

      *BUT* remember the system has been in the previous state since Elog 258
      abd has been working OK

      No change - ReSYNC fails, system wide checks fail as before

      Check 50MHz TTL external clock signal and observe transients and width changes
      in clock signal - see attachment 16
      This may not be *the* explanation of current problems but I doubt the the hardware
      will be 'happy' with it either

      JAR cycles NIM signal through NIM-TTL signal converter several times to produce
      better (no transients or width changes) clock signal with c. 3.2V amplitude

      Power cycle and restart FEE64s
      No change - ReSYNC and system wide checks fail - see attachments 17-21

12.40 Re-checked HDMI cabling 
      found master MACB port 1 -> MACB #7 next, port 2 -> MACB #3 next
      change to master MACB port 1 -> MACB #3 next, port 2 -> MACB #7 next
      Power cycle FEE64s

      ReSYNC partial failure (left side OK, right side of tree fails), system wide checks fail - see attachments
22-26

13.05 Swap HDMI cables from master MACB to parents, i.e. HDMI cable master port 1 to MACB #3 next
      changes to master port 2 to MACB #7 next 

      ReSYNC and system wide checks fail - see attachments 30-34
      

13.35 Swapped HDMI cables from master MACB to MACB #3 & #7
      for HDMI cables from MACB #7 ports 1 & 2 to MACB modules #8 & #9

      ReSYNC and stsrem wide test failures - see attachments 1-5

15.00 Swap HDMI cable from MACB #2 port 1 to MACB #3 next

      ReSYNC and system wide test fails - see attachments 6-10

      *Conclusion* all of HDMI cables between HDMI cable from nnaida6 and master MACB
      have now been swapped for other HDMI cables between MACB # 7 and MACB # 8, 9, 10
      but we still observe ReSYNC and clock issues 

16.05 CG points out that I have misinterpreted Elog 258 - switch back MAC
      mode settings to the following

      MACB #  1  2  3  4  5  6  7  8  9 10 11      spare 1   spare 2
      Mode    5  2  2  3  3  3  3  3  3  3  3
      Rev B      y  y                              50MHz?    y

      FEE64 power cycle

      ReSYNC and system wide fails - see attachments 11-16


16,50 Re-considering options
      Install MACB #11 (rev B) mode 0
      Will attempt to use 25MHz correlation scaler clock NIM output from master MACB to BRIKEN+RIBFDAQ

      Power cycle FEE64s

      ReSYNC OK
      System wide checks OK but some issues remain - see attachments 17-25

17.10 DAQ starts
      Writing to disk file R115, merge rate 0.9M data items/, TapeServer ~7Mb/s

17.20 Analysis of R115_1 confirms ADC data synchronised - see attachment 26

17.25 MERGE stats look OK - see attachments 27-29

17.35 System wide tests SYNC errors & SYNC pulses OK

17.38 DAQ stalled - manual start of ORNL clover LN2 fill cycle by Nishimura-san
      Most recent file R115_5 last accessed 17.38

      DAQ stop OK

      Disable histogram and data transfer #1 
      DAQ RESET
      DAQ SETUP
      Restart merger and TapeServer
      Enable histogram and data transfer #1 

17.47 DAQ start (R116)

17.58 DAQ stopped again at end of fill cycle.
      1. Disable histogram and data transfer #1
      2. DAQ RESET
      3. DAQ SETUP
      4. Restart merger and TapeServer
      5. Enable histogram and data transfer #1 
      Had to repeat 1-5 multiple times to get all links working with the merger. R117 created during this (empty).

18.08 Run R118 started.

18.45 TD changing things for runs R119-R120.

18.50 Run R121 started.
      First few minutes are pulser run 30-10,000 @ 250Hz on coarse dial.

18.54 Beam ON.
      Continuing R121. Pb shield immediately after final MUSIC detector.

21.35 DAQ continues OK
      Most recent file R121_30

      All system-wide tests OK *but* note nnaida5 now fails clock status test in addition to combinations of
nnaida7/8/15/16
      Merge/Tapeserver desktop, Merge statistics 
      - see attachments 30-36

21.45 R121_32 ADC data synchronised

Overnight

Phong Vi completed the pulser walkthrough interrupted earlier this evening.
BNC PB-4 amplitude settings 30,000 - 20,000 @ 10,000 step, 8,000-2,000 @ 2,000 step  
The data is in files R121_62, _63 and _64 (which have been copied to d05 for further analysis).

14.05 Julabo FL11006 chiller set point +20 deg C actual +20.0 deg C
      Water level c. 70% M-H markers
      Temperature +25.6 deg C dew point +10.3 deg C RH 37.7%

13.30 DAQ continues OK

      ASIC settings 2019May19-17.54.03
      slow comparator 0xa

      BNC PB-4 Pulser 
      Amplitude 90,000
      x5 attenuator IN
      Frequency 25Hz range
      t_r 100ns, tau_d 50us
      - polarity
      Delay MIN, tail pulse

      all system wide checks OK *except* nnaida7 fails ADC calibration

      detector biases & leakage currents OK - see attachments 1 & 2

      FEE64 temperatures OK - see attachment 3

      good event statistics OK - see attachments 4
      13x <20k 21x < 50k 24x <100k cf. same yesterday afternoon
      Merger OK - see attachment 5
      TapeServer OK - see attachment 6

      p+n FEE64s 1.8.W spectra 20us FSR - attachment 7
      n+n FEE64s 1.8.W spectra 20us FSR - attachment 8
      1.8.L spectra - see attachments 9 & 10
      nnaida1 80 ch FWHM, nnaida3 87 ch FWHM - cf. 96 & 103 ch FWHM respectively yesterday aftenoon

      FEE64 Rate spectra - attachment 11
      Check ASIC Control *all* FEE64s & ASICs x2
      FEE64 Rate spectra - attachment 12 - some channels recovered

14.20 BNC PB-4 pulser
      Frequency changed from c. 4Hz to c. 2.5Hz (minimum)
      25Hz range, variable fully anti-clockwise

14.33 zero *all* spectra
      zero DAQ stats

      ADC data synchronised - attachment 13

16.50 DAQ start file May19/R18

      AIDA+DTAS merger test - 12kBq 60Co positioned on top of AIDA snout

      DAQ sync issued by DTAS during R18_0

      ASIC settings 2019May19-17.54.03
      slow comparator 0xa

      BNC PB-4 Pulser 
      Amplitude 90,000
      x5 attenuator IN
      Frequency 25Hz range
      t_r 100ns, tau_d 50us
      - polarity
      Delay MIN, tail pulse

17.28 DAQ stop file May19/R18_4

17.40 DAQ start file May19/R19

      AIDA+DTAS merger test - c. 30kBq? 137Cs positioned on top of AIDA snout

      DAQ sync issued by DTAS during R18_0

      ASIC settings 2019May19-17.54.03
      slow comparator 0xa

      BNC PB-4 Pulser 
      Amplitude 90,000
      x5 attenuator IN
      Frequency 25Hz range
      t_r 100ns, tau_d 50us
      - polarity
      Delay MIN, tail pulse

18.18 DAQ stop file May19/R19_5

19.36 all system wide checks OK *except* nnaida7 fails ADC calibration

      detector biases & leakage currents OK - see attachments 14 & 15

      FEE64 temperatures OK - see attachment 16

      good event statistics OK - see attachments 17
      12x <20k 21x < 50k 24x <100k cf. 13x <20k 21x < 50k 24x <100k earlier this afternoon

19.45 Julabo FL11006 chiller set point +20 deg C actual +20.0 deg C
      Water level c. 70% M-H markers
      Temperature +25.1 deg C dew point +7.9 deg C RH 33.6%
      

10.29 DAQ left running overnight
      Histograms only - no data transfer to Merger & TapeServer

      DSSSD biases & leakage currents OK - see attachments 1 & 2
      FEE64 temperatures OK - see attachment 3

      DAQ run control OK - see attachment 4

      Statistics - see attachments 5-16
      
      N.B. some FEE64s are producing SYNCs, some not, in particular nnaida6 is not

      System-wide tests - see attachments 17-22

11.31 DAQ recovered with STOP-GO (no data transfer)
      AIDA SYNCs OK for all FEE64s

22.46 DAQ still running OK

      Power down in preparation for tomorrow's planned site-wide power outage 

Change n+n FEE64 aida04 d8:80:39:41:f6:b7
 asic #1 faulty

aida04 MAC from d8:80:39:41:f6:b7 to d8:80:39:41:ba:2b

restart dhcpd ( systemctl restart dhcpd )


DSSSD bias -150V -5.17uA - attachment 1

FEE64 temperatures OK - attachment 2

All system wide checks OK

ADC data item stats - attachment 3
 High rates observed in all FEE64s *except* aida01 & aida04 => aida04 asics #1-4 OK
 CC/VP have commenced cabling bPlast - will await completion of install to review noise
 

n+n 1.8.W spectra - 20 us FSR - attachments 5
 
per FEE64 rate spectra - attachment 4



18:20 AIDA noise status test

      bPlast - cabling complete, power on - all sides 
      BB7 Mesystec preamps connected, power off, no detector HV

DSSSD bias -150V -5.355uA - attachment 6

FEE64 temperatures OK - attachment 7

All system wide checks OK

ADC data item stats - attachment 8
 Rates c. 10-20k mostly dominated by a few 'hot' channels

per FEE64 rate spectra - attachments 9 & 10

p+n 1.8.W spectra - 20us FSR - attachment 11
 bPlast using 152Eu source for test - signal observed aida01 3.8.W may be recoil electron from Compton event
n+n 1.8.W spectra - 20 us FSR - attachment 12

Some photos of bPlast & BB& preamp/cabling attached

re-bPlast note that
cables screened 3M 1245 + Kpaton + drain wire -> drain wires *not* connected to ground
PSU ch 4 not ground referenced
 

08.50 FEE64 temperatures (attachment 1)

09.03 R70 ends OK
      Beamtime ends

      R71
      background/residual activity

12.12 R71 ends

      R72
      Pulser walkthough, amplitude 90,000-10,000 @ 10,000 step, freq 25Hz 

12.33 R72 ends 
      External clock was removed by mistake - DAQ stopped
      Pulser data for amplitudes 90,000-50,000 only

      Ignore R73, R74

12.59 R75
      Conclusion of pulser walkthrough for amplitudes 40,000-10,000

00:06 System wide checks all ok
      Bias and leakage currents ok - attachment 1
      Good event statistics - attachment 2
      FEE temperatures - attachment 3

00:29 BRIKEN and BigRIPS stop their DAQs
      AIDA on file R7_494

00:31 BigRIPS starts run 3080
      BRIKEN starts run 77
      AIDA on file R7_494

01:31 BigRIPS and BRIKEN stop DAQs
      AIDA on file R7_508

01:33 BRIKEN starts run 78
      BigRIPS starts run 3081
      AIDA on file R7_508

02:10 System wide checks all ok
      Bias and leakage ok - attachment 4
      Good event statistics - attachment 5
      FEE temperatures - attachment 6

02:30 BRIKEN and BigRIPS stop their DAQs
      AIDA on file R7_211

02:32 BigRIPs started Run 3082
      BigRIPS stopped

02:33 Beam stopped to put and extra 1mm of degrader in to shift beam
      Current degrader settings:
      F7 = 2mm Al
      F11 = 4.5mm Al

02:39 biGRIPS starts run 3083
      BRIKEN starts run 79
      AIDA on file R7_523

      DSSD rates from R7_235
 *** DSSSD # 1 count:    119355 old count:     78874 dt:      1.05 s  LEC rate:  38696.78 Hz
 *** DSSSD # 2 count:    188044 old count:    124828 dt:      1.05 s  LEC rate:  60429.72 Hz
 *** DSSSD # 3 count:     53151 old count:     34736 dt:      1.05 s  LEC rate:  17603.35 Hz
 *** DSSSD # 4 count:     61647 old count:     40919 dt:      1.05 s  LEC rate:  19814.40 Hz
 *** DSSSD # 5 count:     14623 old count:      9739 dt:      1.05 s  LEC rate:   4668.74 Hz
 *** DSSSD # 6 count:     41905 old count:     27687 dt:      1.05 s  LEC rate:  13591.33 Hz
 *** DSSSD # 1 count:         1 old count:         1 dt:      1.05 s  HEC rate:      0.00 Hz
 *** DSSSD # 2 count:        11 old count:         7 dt:      1.05 s  HEC rate:      3.82 Hz
 *** DSSSD # 3 count:        19 old count:        10 dt:      1.05 s  HEC rate:      8.60 Hz
 *** DSSSD # 4 count:        15 old count:        10 dt:      1.05 s  HEC rate:      4.78 Hz
 *** DSSSD # 5 count:         0 old count:         0 dt:      1.05 s  HEC rate:      0.00 Hz
 *** DSSSD # 6 count:         3 old count:         2 dt:      1.05 s  HEC rate:      0.96 Hz

03:37 bigRIPS and BRIKEN stop their daqs
      AIDA on file R7_537

03:39 BigRIPS starts run 3084
      BRIKEN starts run 80
      AIDA on file R7_537

04:16 System wide checks ok
      Leakage and bias ok - attachment 7
      Good events - attachment 8
      FEE temperatures - attachment 9

04:33 BRIKEN AND BigRIPS stop daqs
      AIDA on file R7_549

04:34 BRIKEN starts run 81
      BigRIPS starts run 3085
      AIDA on file R7_550

05:30 BigRIPS and BRIKEN stop their DAQs
      AIDA on file R7_563

05:40 bigRIPS starts run 3086
      BRIKEN starts run 82
      AIDA on R7_565

06:14 System wide checks all ok
      Bias and leakages ok - attachment 10
      Good events - attachment 11
      FEE temperatures - attachment 12

06:33 BigRIPS and BRIKEN stop their DAQs
      AIDA on file R7_577

06:34 BigRIPS starts run 3087
      BRIKEN starts run 83
      AIDA on file R7_577

07:30 BRIKEN and BigRIPS stop daqs
      AIDA on file R7_590

07:31 BigRIPS starts run 3088
      BRIKEN starts run 84
      AIDA on file R7_591

08:36 DAQ still running
      Not writing data to disk

      Bias and leakage currents ok - attachment 1
      System wide checks all ok
      FEE Temperatures ok - attachment 2
      Statistics - attachment 3
      Check ASIC Control done

08:56 Pulser peaks are ok and similar to what has been observed over the past two weeks
      Layout 4 and 5 - attachments 4 and 5

11.55 Lost connection to FEE64 1-16 - fuse blown
      stopped DAQ
      switched all FEE64 off
      Replaced 20A fuse
      switched FEE64 back on individually
      system wide checks ok, except nnaida7 failure in ADC calibration
      DAQ start (not writing to file)
      FEE64 temperatures ok (attachment 6)
      Bias/Leakage currents ok (attachment 7)
      good event statistics ok (attachment 8)

17.08 all system wide checks OK *except* nnaida7 fails ADC calibration
      detector biases & leakage currents OK - see attachments 9 & 10

      FEE64 temperatures OK - see attachment 11
      good event statistics OK - see attachment 12

      per FEE64 rate spectra - see attcahment 13

      1.8.L spectra - see attachments 14 & 15
       nnaida1 pulser peak width 72 ch FWHM

      1.8.W spectra 20us FSR - see attachments 16 & 20
      1.8.W spectra 200us FSR - see attachments 17 & 21
      1.8.W spectra 2ms FSR - see attachments 18 & 22
      1.8.W spectra 20ms FSR - see attachments 19 & 23

11 am: Changed configuration of power source for AIDA PSU (see paper logbook). Now both cables with power for
USB relay are connected to the 'C' unit of the F11 power supply. They use different socket pads, and share them
with power cables for raspberry pi and NIM crates. Situation intermediate to what we have on Tuesday.

Resolution of pulser peaks improved slightly w.r.t. yesterday, but still we have the high frequency noise
(500kHz) present (attachments).

12 pm: We found the FEEs crash (loose connection/synchronization) when we enable the fast discriminators (see
attach. 1). This could be in the case when some of the discriminator channels are triggering at a fast rate
(noise above threshold). We observed the same pattern a few times.

5 pm: During all afternoon we've not been able to put AIDA in an operational state. There is always one or more
FEEs that do not synchronize or loose connection. Also the fuse of the PSU broke and we had to replace them.
Source for state of the system could be related to current configuration of PSU power, or the presence of noise
we observed in last couple of days.


6 pm: Back to configuration of PSU from yesterday:

F11 power supply 'C' ---->----> PSU relays 1-4
                          |
                          |---> NIM crates, RPI, etc


F11 power supply 'B' ----> PSU relays 5-8



Started by checking the options. Loaded Options 2016May03-20.44.33. It looks like settings of ASICs are not the
standard ones (e.g. threshold 47). Tried restore settings in Save/Restore option. It worked for all but NNAIDA25
(fri22.png), but other have proper settings (but values of #25 also look ok!).

Performed 'Check ASIC clock timestime' in systemwide checks: all failed (fri23.png)
Performed 'Check ADC calibration' in systemwide checks: all passed, except for nnaida26 (fri24.png)
Master Timestamp Control -> Perform ReSYNC: worked for all modules except nnaida26


we lost connection to 26..... (fri26.png)

Attempted ADC calibration of NNAIDA26 (FADC align and control) failed (fri27.png)

Disabled NNAIDA from Run Control, and Reset all modules. But now after reset the setup button does not appear,
just Reset one as if the action had no effect.

Tryed restore settings again. Again nnaida25 returned problem, but yet again nnaida26 is the one that actually
doesn't have proper ASIC settings. Confirmed this with 'Check ASIC control' (fri29.png). Is there an error in the
error report message of 'Restore Settings'?

We checked cable connections (HDMI, ethernet), and afterwards NNAIDA26 keeps giving problems. We give up on it,
remove it from DAQ configuration (https://elog.ph.ed.ac.uk/AIDA/189), and system nos seems to be runing ok
without it.


8 pm: disconnected all remaining cables from MACB->other crates (only daq correlation in macb1), and went back to
yesterdays configuration of PSU power with all power from 'C'

F11 power supply 'C' ---->----> PSU relays 1-4
                     |    |
                     |    |---> PSU relays 5-8
                     |
                     |--->---> NIM crates, RPI, etc

Relatively painless start-up. Had an issue with firefox where all bookmarks had been forgotten.
All drop down menues in the DAQ did not work.
Kill all firefox processes to restore.

Had to restart DHCP to get the rasp pi working again.

At first no pulser was visible in all p-side FEEs. Cable and BNC adapter on output of inverter were switched out
and the inverter output
was switched from B to A.

After this pulser signals were visible in all ASICS bar NNAIDA 14 ASIC 1. Power cycle was needed to regain this.
No ADC data and no fast discriminator data. Waveform showed baseline at 16K compared to 8K expected.

Layouts 1,2,3,4,7 and 8 attached (Zoomed in waveforms are included for 7 and 8 around baseline) attachments 1-8

Temperatures included as attachment 9

Leakages and bias all ok see attachment 10
Ambient T 24.3C Humidity 34.5% Dew point 7.6C (Measurements taken AM)

system wide checks all ok.

Pulser settings

Amplitude 90000
x5 Attenuator in
Rise time tr 50ns fall time tau d 50us
Negative polarity
freq 25Hz range
Delay min
Internal ref, tail pulse

ASIC settings DB key 2017Mar27-16.12.39

16:26 Slow comparator threshold moved to 100
      Fast comparator theshold LEC/MEC to 255
      Run started to disk May2017/R1