AIDA GELINA BRIKEN nToF CRIB ISOLDE CIRCE nTOFCapture DESPEC DTAS EDI_PSA 179Ta CARME StellarModelling DCF K40
  AIDA, Page 18 of 46  ELOG logo
Entry  Sun Jun 2 04:00:50 2019, TD, Sunday 2 June 21x
12.00 DAQ continues file May19/R15_169

      ASIC settings 2019May19-17.54.03
      slow comparator 0xa

      BNC PB-4 Pulser 
      Amplitude 90,000
      x5 attenuator IN
      Frequency 25Hz range
      t_r 100ns, tau_d 50us
      - polarity
      Delay MIN, tail pulse

      all system wide checks OK *except* nnaida7 fails ADC calibration

      detector biases & leakage currents OK - see attachments 1 & 2
      FEE64 temperatures OK - see attachment 3
      good event statistics OK - see attachments 4
      10x <20k, 19x <50k, 24x <100k - same as yesterday evening

      Merger OK - see attachment 5
      TapeServer OK - see attachment 6

      nnaida13 1.*.L spectra - attachment 7 - some evidence of light ions (5 < E < 20MeV) but low rate
      FEE64 Rate spectra - attachment 8

      1.8.L spectra - see attachments 9 & 10
      nnaida1 75 ch FWHM, nnaida3 76 ch FWHM - cf. 77 & 82 ch FWHM respectively yesterday afternoon
      p+n FEE64s 1.8.W spectra 20us FSR - attachment 11
      n+n FEE64s 1.8.W spectra 20us FSR - attachment 12

      Check ASIC Control *all* FEE64s & ASICs

12.20 Offline analysis from R15_0 to R15_167 ADC & disc data synchronisedfor file R15_0 - attachments 13, 14 & 15

18.27 DAQ stop file May19/R15_245

      DAQ start - no storage

      DAQ stall - reboot all FEE64s *except* nnaida4, 7, 8, 14, 15 & 16 - N.B. no power cycle required

      all system wide checks OK *except* nnaida7 fails ADC calibration

      detector biases & leakage currents OK - see attachments 19 & 20
      FEE64 temperatures OK - see attachment 21
      good event statistics OK - see attachments 16
      8x <20k, 15x <50k, 21x <100k - cf. earlier today 10x <20k, 19x <50k, 24x <100k

      p+n FEE64s 1.8.W spectra 20us FSR - attachment 17
      n+n FEE64s 1.8.W spectra 20us FSR - attachment 18
Entry  Sat Apr 1 16:16:05 2017, OH, CG, Sunday 2 April bias_020417_0146.pngbias_020417_0742.pngBigRIPSlogbias_020417_0922.png
00:12 BigRips started run 0223 (Started at 00:12)
      AIDA file R20_344
      System wide checks all ok
      BRIKEN start run 67

01.38 Beam stopped to F11. ~1/2 way through R20_364.

01.43 Beam returned to F11. 1GB through R20_366.
      BigRIPS file 0224.
      BRIKEN run 68.
      System-wise checks all ok, except nnaida1 global clocks failed 6.
      Temps ok. Leakage currents ok (see attachment 1).

02.43 BigRIPS started run 0225. Start of R20_381
      
02.48 BRIKEN run 69 started. Start of R20_382.

03.28 System-wide checks same as previous. (R20_391)

03.43 BigRIPS file 0226. ~1/2 way through R20_394

03.48 BRIKEN run 70 started.

04.46 BigRIPS file 0227 started. End of R20_410.

04.48 BRIKEN run 71 started. Start of R20_411.

05.31 36Ne setting is over. End of R20_421.
      BigRIPS team is changing setting for calibration. No beam will go through Zerodegree
      
05.51 R20 stopped (I was getting food).

06.30 20mm Al degrader + 3mm plastic removed from F8. 23mm Al degrader placed at F11 after IC.

06.35 BigRIPS file 0237 started - 38Mg setting.

06.37 R21 started on 38Mg setting.

06.57 Beam stopped to F11 - accelerator problem.
      ~700MB into R21_5.
      Returned shortly after.

07.23 BigRIPS file 0245 started. ~3/4 way through R21_10.

07.26 BigRIPS file 0246 started. ~1/2 way through R21_11.

07.32 BigRIPS file 0247 started. Start of R21_13.

07.34 BigRIPS file 0248 started. ~1/2 way through R21_13.

07.38 BigRIPS file 0249 started. ~1GB into R21_15.

07.40 BigRIPS file 0250 started. Start of R21_16.

07.41 BigRIPS file 0251 started. Little way through R21_16.

08.06 38Mg setting finished. R21 stopped.

08.08 R22 started - got a bit trigger happy with the DAQ. No beam to F11.

08.10 Degrader set to 10mm Al.

08.16 BigRIPS file 0252 started on 39Al setting.

08.17 R23 started on 39Al setting.

08.20 R23 ended.

08.26 40Si setting. Too high rate for AIDA so beam won't be sent down ZDS.

08.40 F11 degrader set to 15mm Al fixed + 4mm Al variable (all in).

08.43 2nd run at 39Al setting.
      BigRIPS file 0254 started.
      R24 started.

09.00 R24_4 closed. May have been issues with degrader - not sure we were implanting in AIDA...
      End of experiment!

09.07 AIDA background run started - R25.
      Pulser is on in the run though.

09.20 R25 ended.
      BigRIPS run summary attached.
Entry  Sun Jul 1 02:26:50 2018, OH, Sunday 1st July 180701_1025_bias.png
10:25 Bias and leakage currents ok - attachment 1
      FEE Temperatures ok
      System wide checks ok
      Alpha run R9 stopped on file R9_74
Entry  Sun Nov 19 01:46:10 2017, CB, TD, Sunday 19 November 14x
10.45 Detector biases & leakage currents OK - see attachments 1
      FEE64 Temperatures OK - see attachment 2
      DAQ statistics OK - note *all* slow comparators 0xa - see attachment 3
      7x FEE64s <10k
      4x FEE64s 10-20k
      4x FEE64s 20-30k
      4x FEE64s 30-40k
      1x FEE64s 40-50k
      4x FEE64 >50k
      *all* system wide checks OK *except* nnaida7 fails ADC calibration

14.05 *all spectra zero'd

14.24 Detector biases & leakage currents OK - see attachments 4 & 5
      FEE64 Temperatures OK - see attachment 6
      DAQ statistics OK - see attachment 7
      7x FEE64s <10k
      3x FEE64s 10-20k
      4x FEE64s 20-30k
      5x FEE64s 30-40k
      1x FEE64s 40-50k
      4x FEE64 >50k
      *all* system wide checks OK *except* nnaida7 fails ADC calibration

      per FEE64 rate, 1.8.L and 1.8.W spectra (20ms FSR) - see attachments 8-13

14.48 MONLOCK monitor - see attachment 14
Entry  Sun May 19 02:28:07 2019, OH, TD, CA, Sunday 19 May 11x
10:28 DAQ Continues ok
      Alpha run continues - Currently on file R7_5

      System wide checks all ok
      leakage currents are up from yesterday - attachment 1
      Temperature is up 2 deg C from yesterday which explains leakage current increase
      Temp +26.6 deg C, Humidity 28.8% AND D.P +7.2 deg C
      
      FEE Temperatures ok - attachment 2

      Good event statistics ok - attachment 3

17.40 Julabo FL11006 chiller set point +20 deg C actual +20.0 deg C - water level 75% H-L markers
      Temperature +25.4 deg C dew point +7.2 deg C RH 30.9%

17.47 DAQ continues OK to file May19/R7_6

      alpha background

      ASIC settings 2018Oct03-13.26.58
      *except* slow comparator 0xa -> 0x64

      BNC PB-4 Pulser OFF

      all system wide checks OK *except*
      nnaida7 fails ADC calibration
      Check SYNC errors not counted since Baseline - see attachment 4 - also failed test following Collect All
SYNC Count Errors for Baseline
       N.B. Check SYNC pulses received OK

      detector biases & leakage currents OK - see attachments 5 & 6

      FEE64 temperatures OK - see attachment 7
      good event statistics OK - see attachments 8

      Merger OK - see attachment 9
      TapeServer OK - see attachment 10

17.53 DAQ stopped file May19/R7_6

      DAQ RESET
      DAQ SETUP
      all system wide checks OK *except* nnaida7 fails ADC calibration   
      Sync all ASIC clocks using Re-Sync   
      Check ASIC Control *all* ASICs & FEE64s

17.57 DAQ start file May19/R8

      alpha background

      ASIC settings 2019May19-17.54.03
      slow comparator 0x64

13.13 analysis of May19/R7_6 - see attachment 11
      deadtime OK
      ADC data for FEE64s nnaida5 & nnaida6 hot channels (Stat spectrum channel # 104)
      Also observed at a much lower rate for nnaida1 & nnaida2
Entry  Sun Jun 18 02:56:43 2017, TD, Sunday 18 June 11x
10.56 Detector biases & leakage currents OK - see attachments 1 & 2

      USB-controlled ac mains relay input #1 fuse blown - replaced

11.12 *all* system wide checks OK *except* nnaida21 fails ADC calibration

      BNC PB-4
      Amplitude 90,000
      x5 attenuator IN
      t_r 100ns tau_d 50us
      - polarity
      frequency 25Hz range 2.6Hz
      delay MIN, tail pulse, INT ref

      ASIC settings 2017Mar27-16.12.39

11.36 nnaida13 & nnaida11 1*W spectra - see attachments 3 & 4 
      per FEE64 1.8.W spectra - see attachments 5 & 6
      good events statistics - see attachment 7
  
      Note c. 3kHz modulated HF noise has disappeared overnight

12.00 2x unused (possibly faulty) MACBs removed from AIDA NIM bins preparatory to UK return
      ambient temperature +24.0 deg C, d.p. +8.9 deg C, RH 38.8%
      Julabo FL11006 set point +20.0 deg C, actual +20.0 deg C, water level c. 75%

14.03 nnaida13 1*W spectra
      Ibias LF feedback 0x8 (default) - see attachments 8 & 10
      Ibias LF feedback 0xf - see attachments 9 & 11

14.30 BRIKEN 3He and clovers off
      no obvious change in preamp waveform noise

14.33 DSSSD bias off

14.37 FEE64 power off

14.43 2x NIM bins off
      BNC PB-4 pulser off
      USB-controlled ac mains relay off, interlock enabled, USB cable removed
      ac mains circuit off
      Tektronix AFG3201C off
      
      Julabo FL11006 off
      3ph supply off 
Entry  Sun Oct 16 03:14:41 2016, TD, Sunday 16 October 16x
11.12 DAQ start (R47)

      Detector bias and leakage currents OK - see attachments 1 & 2
      FEE64 temperatures OK - see attachment 3

      Statistics - see attachments 4-14

11.56 Temperature +25.0 deg C, d.p. +8.4 deg C, RH 34.5%

20.33 DAQ found stopped
      Most recent file R47_120 (last access 18.26)

      R47_120 ADC data synchronised

      Restart merger and request DAQ STOP - DAQ stops OK

      System-wide tests SYNC errors and SYNC pulses - see attachments 15 & 16 - all other tests OK

      DAQ RESET & SETUP OK

      Normal restart which failed shortly after enabling ASIC readout - restarting merger did not
      recover DAQ control - power cycle required

21.05 DAQ starts (R49)
      
Entry  Sun Jun 16 04:09:40 2019, CA, CB, ML, TD, CG, Sunday 16 June noon-midnight 16x
12.10 DAQ continues OK file RIBF147/R2_145

      100Sn setting, no degrader

      ASIC settings 2019Jun13-08.56.35

      BNC PB-4 Pulser 
      Amplitude 90,000
      x5 attenuator IN
      Frequency 25Hz range, variable min. c. 2.5Hz
      t_r 100ns, tau_d 50us
      - polarity
      Delay MIN, tail pulse

      good event statistics - DSSSD#1 noisy - see attachment 3
      10x <20k, 19x <50k, 20x <100k

      p+n FEE64s 1.8.W spectra 20us, 200us, 2ms, 20ms FSR - DSSSD#1 noisy - attachments 4, 6, 7 & 8
      n+n FEE64s 1.8.W spectra 20us FSR - DSSSD#1 noisy - attachment 5

      FEE64 Rate spectra - attachment 1
      Check ASIC Control *all* FEE64s & ASICs x1
      FEE64 Rate spectra - attachment 2

      Observe that nnaida17 is the really noisy FEE64 - nnaida20, 22 & 23 show increased noise but
      not at same level as nnaida17. Is nnaida17 actually the problem? LH kapton PCB is the only
      one we did not replace for example.

12.21 DSSSD#1 stats OK

12.36 AIDA R2_151
      DTAS run 027
      BigRIPS still 2012

13:43 DAQ continues running OK file RIBF147/R2_167
      https://www.reddit.com/r/politics/
      Settings as before

      Good events stats OK - DSSD#1 no longer noisy - see attach 9
      Leakage current OK - see attach 10
      Systems wide checks OK except nnaida7 fails calibration

      Merger ~550k events/second, TapeServer 4435 Kbytes/s

      Will increase thresholds on DSSD#1 and decrease on DSSD#3 on next DTAS stop.

14:20 DSSD#1 was found noisy again (Rate>100kHz & Tapeserver 10Mbytes/s)) but then soon after was back to a
      normal counting rate (Rate back to ~40kHz & Tapeserver back to 5 Mbytes/s)

14:39 DTAS & BigRIPS stops - changing threshold during AIDA R2_174
      DSSD#1 (nnaida17, 20, 22, 23) slow comparator threshold -> 1d (300 keV)
      DSSD#3 (nnaida9, 12, 14, 15) slow comparator threshold -> 8 (80 keV)

      Check ASIC control x1 on all FEE64

      Restarting
      AIDA R2_175
      DTAS 30
      BigRIPS 2013

      New stats look OK - zero'd Counters - see attach  11
      nnaida 17,20,22,23 rates are down
      nnaida 9,12,14,15 rates are up
      It appears the data rate balances out. Data items & write to disk similar to before the changes. 
      See attach 12 & 13

      New sample waveforms in attach 14 & 15. nnaida17 looks significantly better than before.
      Rates spectra also attached - see 16
      
17:12 Data rate still ~600k items/s
      File rolled over to R2_196

17:30 Current F11 rate ~190 cps
      Estimation of implant rate in AIDA of ~270 implants/s (number of HEC events in DSSD1 y strips over duration
of file R2_197)
         > increased AIDA rate likely due to double counting of multi-strip events
      Temps all ok


18.11 DAQ stats zero'd
      *all* histograms zero'd

20:13 DAQ continues to run OK
      Data rate ~620k items/s. FEE64 data rates remain approx. same
      Temps OK
      Just rolled over to file R2_222

20:44 DTAS run 39 started
      BigRIPS run 2015
      AIDA at 1.2GB into R2_226

22:44 DTAS run 42 started
      BigRIPS run 2017
      AIDA at 1.3GB into R2_243

      Carried out ASIC check successfully
      Temps and data rate remain unchanged.

23:16 Been almost 1 day running. 
      DTAS running out of range in some of their timing histograms so reset the correlation scaler
      Reset successfully at start of R2_249

23:19 Reset signal sent again. Still in file R2_249

23:23 DTAS run 43 started
      BigRIPS run 2018
      AIDA run R2_250

23:54 MIDASSORT spectra saved to /homes/npg/spectra/specs-2019Jun16-23.52.55
      MIDASORT online started.
Entry  Sat Jun 15 16:53:22 2019, CB, OH, DK, ML, Sunday 16 June 37x
00.30 AIDA file R2_26
      DTAS run 007
      BigRIPS run 2006

00:55
      Merger status - attachment 1
      TapeSever status - attachment 2

01:40
      Before zeroing stats - attachment 3
      Histos up to now saved in /homes/npg/spectra/specs-2019Jun16-01.46.50
      Before zeroing histos - XY attachment 4

02:22  Up to AIDA R2_41
      Stats - attachment 5
      XY implants - attachment 6

02:29 AIDA R2_42 
      DTAS run 011
      BigRIPS still 2006

04:08 Up to AIDA R2_55
      Temps - attachment 7
      Biases - attachment 8
      Stats - attachment 9
      XY implants - attachment 10

04:50 Did some major UI updates to merger /data02/2019data/dtas/scripts/parallelscript/runall_xe19_dev.sh
         If no parameters are passed, it asks to select each option one-by-one, showing available options
             For DTAS, if there is only one run for a given header, it automatically selects it
         After that, it checks that all the raw files actually exist
         Finally, it composes the command, asks the user if it looks OK, and then launches the script (or quits)
         (For experts, it can still take the original case where all 7 parameters are passed at the command line)
         There may still be some minor bugs, so for now we don't replace the original one until Alvaro can check

06:09 Stats - attachment 11
      XY implant profiles -- attachment 12

06:30 DTAS 018; AIDA R2_80; BigRIPS 2009

Things getting a bit noisier?
      Merger - attachment 13
      TapeServer - attachment 14
      Stats - attachment 15

   The FEEs corresponding to DSSSD1 are all showing factors 2 to 10 higher than just 30 min. ago

   F11 rates still near 180 cps, primary beam still around 130 pnA.

06:43 Before I can diagnose the high noise problem in DSSSD1, it goes back to normal
      Merger - attachment 16
      TapeServer - attachment 17
      Stats - attachment 18
   Maybe the issue was from looking at the details too often :)

07:46 Shift change.

      DAQ continues running OK.
      Statistics show  noisy DSSD #1 (100-200kHz) - see attach 19
      Tapeserver also shows higher than normal write to disk rate - see attach 20
      After a few seconds write to disk rate goes back to normal - see attach 21
      Data items rate looks OK (500k). This was checked after the rates in DSSD#1 went back to normal - see attach 22
      FEE64 temperatures OK - see attach 23
      Implants look OK - see attach 24

      System-wide checks OK except nnaida7 fails ADC calibration as usual.

      Reset Statistics
      Stats now OK - see attach 25

      Reset HEC implants

      Timewarp events appear to be very frequent, consistently every 3-5 seconds.
      This seems to have been happening ever since beam was put on target.
      Analysed R2_94 using analyser in Desktop 3. No timewarps reported. See attach 26.
      
08:32 AIDA: R2_100,
      DTAS: 21  ,
      BigRips: 2010   

08:41 Another noise spike from DSSD#1
      Observed up to 1,600k data items per second - see attach 27
      Write to disk ~10 MB/s as before.
      See LEC xy plot - attach 28
      HEC xy plot shows no hints of issues - see attach 29
      
      Rate seems manageable - see attach 30 for analysis of noisy run R2_103. 
      Note few seconds deadtime in some FEE64 and unequal number of Pause/Resume in nnaida 22      

09:45 Rate in DSSD#1 back to normal. Data rate 500k, Write to disk 5 Mb/s.

10:00 Started compressing R2_* files to save disk space.
      Used nice gzip to compress to avoid conflicts with the DAQ.
      Status of compression can be checked in ELOG desktop in the terminal window running ls -alth

10:32 DTAS and BigRIPS stop runs
      AIDA on file R2_128

      DTAS:24
      BigRIPS 2011

10:56 Lost beam. BigRIPS and DTAS DAQs continue running. Accelerator operators advise beam should be back in 10 minutes.

11:12 Beam is back. BigRIPS & DTAS continue running and did not stop. AIDA file R2_133.

11:25 System-wide checks OK except nnaida7 fails ADC calibration as usual.
      Stats OK, but nnaida19 is a bit more noisy than befor (80k) - see attach 31
      Temperatures OK - see attach 32
      TapeServer & Merger OK - see attach 33, 34 - rates have been low since 10 am
      HEC xy plot OK - see attach 35

      Reset online spectra

      Leakage current OK - attach 36
     
Entry  Sun Jul 16 14:20:31 2023, TD, Sunday 16 July contd. 27x
15.20 All ERNI pins of aida01 adaptor PCB OK => FEE64 #1 ASIC #2 channel #5 issue faulty test C? faulty ASIC channel?

      Fitted drain wire from aida06 & aida08 adaptor PCBs to FEE64 cooling plates

      n+n FEE64 1.8.W spectra 20us FSR - attachments 1-2

      p+n FEE64 1.8.W spectra 20us FSR - attachments 3

      p+n FEE64 1.8.L spectra - attachment 4
       aida01 1.8.L pulser peak width 15 ch FWHM

      n+n FEE64 1.8.L spectra - attachment 5
       double peak spectra

      Significant improvement in noise for aida06 & aida08 but still not as good as aida02 & aida04. Some additional extrinsic noise observed for aida02 & aida04.

      No change in 1.8.L spectra

      n+n FEE64 1.8.W spectra aida02 200us FSR, other FEE64s 20us FSR - attachments 6
       baseline sinusoid observed - 4T = 320 samples = 64us = 15.6kHz

15:50 BNC PB-5
      switch polarity + to -
      switch BNC cables from BNC PB-5 and EG&G Ortec 534 outputs for test +/-

      Attachments 7-8
       baseline sinusoid observed for p+n *not* n+n FEE64 1.8.W spectra
       suggests issue is EG&G Ortec 534

      Restore original cabling



16.55 Swap aida09 HDMI cable from MACB #4 port 1 to MACB #5 port 1 - power cycle and restart - no change - aida09 clock not locked

      Replace aida09 HDMI with *new* HDMI cable to MACB # 4 port 1 - power cycle and restart - aida09 clock locked


      FEE64 temperatures OK - attachment 9

      System wide checks - attachments 10-15
      OK *except* cannot calibrate aida12 ADCs

      per FEE64 rate spectra - attachment 16

      p+n FEE64 1.8.L spectra - attachments 17
       aida01 1.8.L pulser peak width 15 ch FWHM

      n+n FEE64 1.8.L spectra - attachments 18
       double peak spectra

      p+n FEE64 1.8.W spectra 20us FSR - attachments 19-20

      n+n FEE64 1.8.W spectra 20us FSR - attachments 21-22


17.10 n+n FEE64 1.8.W spectra 200us FSR
       BNC PB-5 decay time 50us - attachment 23
       BNC PB-5 decay time 1ms - attachment 24


18.05 ADC data item stats

      slow comparator 0x64 -> 0xa
      BNC PB-5 pulser frequency 100Hz so expect rate of ADC data items per FEE64 to be c. 6.4kHz
  
      attachment 25 - aida06 & aida08 ground from FEE64 cooling plate to adaptor PCB fitted, test - connected to aida02, aida04, aida06 & aida08
      attachment 26 - aida06 & aida08 ground from FEE64 cooling plate to adaptor PCB fitted, test - disconnected from aida02, aida04, aida06 & aida08
      attachment 27 - aida06 & aida08 ground from FEE64 cooling plate to adaptor PCB not fitted, test - disconnected from aida02, aida04, aida06 & aida08

      Rev A adaptors require grounding (as expected). Output of EG&G Ortec 534 produces c. 15kHz sinsusoid at test input - cable length/capacitative load?
      
      
Entry  Sun May 14 01:52:49 2017, TD, OH, Sunday 14 May 16x
09.54 DAQ continues OK 
      Current file May2017/R7_292

      ASIC settings 2017Mar27-16.12.39 *except*
      slow comparator 0x0a
      LEC/MEC fast comparator 0xff

      Detector biases & leakage currents OK - see attachments 1 & 2
      FEE64 Temperatures OK - see attachment 3
      'good events' statistics OK - see attachment 4
      No warnings/errors reported by merger since DAQ GO - see attachment 5

      *all* system wide checks OK

10.02 Rate/HitRate spectra - see attachments 6 & 7
      1.8.L ADC spectra - see attachments 8 & 9
      1.8.W waveforms - see attachments 10-13

10.07 Analysis of data file R7_294 - see attachment 14
      dead-time/idle-time < 0.5%
      AIDA SYNC100 variations ~330/102000

10.17 E17/F11 ambient temperature +25.1 deg C, RH 37.2%, d.p. +9.4 deg C
      Julabo FL11006 set point +20 deg C, actual +20.0 deg C, water level OK (c. 65%)

10.19 DAQ stops OK
      Current file May2017/R7_298

      Files May2017/R7* *deleted* 

      ASIC settings 2017Mar27-16.12.39 *except*
      slow comparator 0x0a
      LEC/MEC fast comparator 0xff

      BNC PB-4 settings 
      Amplitude 90,000
      x5 attenuator IN
      t_r 50ns, tau_d 50us
      polarity -
      rate 250Hz range
      delay MIN, tail pulse

      Check ASIC Control x2
      Zero all spectra

10.22 DAQ starts
      File May2017/R8

      Analysis of May2017/R8_3 confirms ADC data synchronisation - see attachments 15 & 16
Entry  Mon May 13 03:43:08 2019, CA, TD, OH, Sunday 12 May 14x
19.12 all system wide checks OK *except* nnaida7 fails ADC calibration

      FEE64 temperatures OK - see attachment 9
      detector biases & leakage currents OK - see attachments 10 & 11
      good event statistics OK - see attachment 12
      all FEE64s <100k, 10x < 20k

      1.8.W spectra 20us FSR - see attachments 13 & 14
Entry  Sun Jun 12 03:05:37 2016, TD, AE, CG, MA, Sunday 12 June 2016 part 2 11x
BigRIPS switched from 60Ti to 64V

AIDA48 (AIDA)2034 (EURICA) 2153 (BigRIPS) 

start 08.44
stop 09.08 (FEE64 synchronisation lost -> power cycle & reboot FEE64s)

AIDA49 (AIDA) 2036 (EURICA) 2154 (BigRIPS) 

Power cycle and reboot FEE64s
FADC power down (per PJCS recommendation)
All FEE64s pass all system wide checks *except* nnaida16 which failed the ADC calibration (not significant
as waveforms are disabled and FADC powered down anyway)

start 09.30
stop 09.58

ASIC settings 2016Jun12-09.09.37


AIDA50 (AIDA) 2037(EURICA) 2155 (BigRIPS) 

start 10.02
stop 10.09 (beam lost)

ASIC settings 2016Jun12-10.10.08
  changes to nnaida15 discriminator settings to reduce deadtime (pause-resume rate)

AIDA51 (AIDA) 2038 (EURICA) 2156 (BigRIPS) 

start 10.45
stop 11.55

ASIC settings 2016Jun12-10.10.08

aidas1> ~/td/GREAT/analyser2 /TapeData/NP1512/AIDA51_0
 *** GREAT format 3.2.0 analyser - TD - May 2014
 *** ERROR: READ I/O error:       5002
                   blocks:      32000
          ADC data format:  163625493 (  974305.9 Hz)
        Other data format:   97494507 (  580529.8 Hz)
 Sample trace data format:          0 (       0.0 Hz)
         Undefined format:          0 (       0.0 Hz)
   Other data format type:      PAUSE:       1227 (       7.3 Hz)
                               RESUME:       1227 (       7.3 Hz)
                              SYNC100:      64065 (     381.5 Hz)
                           FEE64 disc:   96631002 (  575388.1 Hz)
                             MBS info:     796986 (    4745.6 Hz)
                           Other info:          0 (       0.0 Hz)

   ADC data range bit set:     213982 (    1274.2 Hz)

                Timewarps:        ADC:          0 (       0.0 Hz)
                                PAUSE:          0 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:          0 (       0.0 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:          0 (       0.0 Hz)
                            Undefined:          0 (       0.0 Hz)
                         Sample trace:          0 (       0.0 Hz)

 Timestamp elapsed time:      167.941 s
 FEE module #:   1 elapsed dead time       0.142 s
 FEE module #:   2 elapsed dead time       1.125 s
 FEE module #:   3 elapsed dead time       0.000 s
 FEE module #:   4 elapsed dead time       0.000 s
 FEE module #:   5 elapsed dead time       0.307 s
 FEE module #:   6 elapsed dead time       0.579 s
 FEE module #:   7 elapsed dead time       0.000 s
 FEE module #:   8 elapsed dead time       0.000 s
 FEE module #:   9 elapsed dead time       0.082 s
 FEE module #:  10 elapsed dead time       0.112 s
 FEE module #:  11 elapsed dead time       3.204 s
 FEE module #:  12 elapsed dead time       0.005 s
 FEE module #:  13 elapsed dead time       0.212 s
 FEE module #:  14 elapsed dead time       0.426 s
 FEE module #:  15 elapsed dead time       0.000 s
 FEE module #:  16 elapsed dead time       0.000 s
 FEE module #:  17 elapsed dead time       0.000 s
 FEE module #:  18 elapsed dead time       0.046 s
 FEE module #:  19 elapsed dead time       0.000 s
 FEE module #:  20 elapsed dead time       0.064 s
 FEE module #:  21 elapsed dead time       0.319 s
 FEE module #:  22 elapsed dead time       0.248 s
 FEE module #:  23 elapsed dead time       0.010 s
 FEE module #:  24 elapsed dead time       0.190 s
 FEE module #:  25 elapsed dead time       0.000 s
 FEE module #:  26 elapsed dead time       0.000 s
 FEE module #:  27 elapsed dead time       0.000 s
 FEE module #:  28 elapsed dead time       0.000 s
 FEE module #:  29 elapsed dead time       0.000 s
 FEE module #:  30 elapsed dead time       0.000 s
 FEE module #:  31 elapsed dead time       0.000 s
 FEE module #:  32 elapsed dead time       0.000 s

 *** Program elapsed time:   60.988s (  524.691 blocks/s,  32.793 Mb/s)

See attachments 1-7 for AIDA51 rates etc

See attachment 8 for FEE64 synchronisation check (file AIDA51_0) - OK

AIDA52 (AIDA) 2039 (EURICA) 2158 (BigRIPS)  

Start 11.56
Stop 12.54 (AIDA FEE lose sync)

Settings file 2016Jun12-10.10.08
Checked correlation of FEEs in file AIDA52_0 at 12.06 and everything seems ok.

aidas1> ~/td/GREAT/analyser2 /TapeData/NP1512/AIDA52_9
 *** GREAT format 3.2.0 analyser - TD - May 2014
 *** ERROR: READ I/O error:       5002
                   blocks:      32000
          ADC data format:  163879044 (  985540.7 Hz)
        Other data format:   97240956 (  584790.6 Hz)
 Sample trace data format:          0 (       0.0 Hz)
         Undefined format:          0 (       0.0 Hz)
   Other data format type:      PAUSE:       1315 (       7.9 Hz)
                               RESUME:       1315 (       7.9 Hz)
                              SYNC100:      63433 (     381.5 Hz)
                           FEE64 disc:   96378426 (  579603.4 Hz)
                             MBS info:     796467 (    4789.8 Hz)
                           Other info:          0 (       0.0 Hz)

   ADC data range bit set:     215707 (    1297.2 Hz)

                Timewarps:        ADC:          0 (       0.0 Hz)
                                PAUSE:          0 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:          1 (       0.0 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:          0 (       0.0 Hz)
                            Undefined:          0 (       0.0 Hz)
                         Sample trace:          0 (       0.0 Hz)

 Timestamp elapsed time:      166.283 s
 FEE module #:   1 elapsed dead time       0.088 s
 FEE module #:   2 elapsed dead time       0.724 s
 FEE module #:   3 elapsed dead time       0.000 s
 FEE module #:   4 elapsed dead time       0.000 s
 FEE module #:   5 elapsed dead time       0.510 s
 FEE module #:   6 elapsed dead time       1.337 s
 FEE module #:   7 elapsed dead time       0.000 s
 FEE module #:   8 elapsed dead time       0.000 s
 FEE module #:   9 elapsed dead time       0.087 s
 FEE module #:  10 elapsed dead time       0.121 s
 FEE module #:  11 elapsed dead time       2.706 s
 FEE module #:  12 elapsed dead time       0.000 s
 FEE module #:  13 elapsed dead time       0.127 s
 FEE module #:  14 elapsed dead time       0.191 s
 FEE module #:  15 elapsed dead time       0.000 s
 FEE module #:  16 elapsed dead time       0.000 s
 FEE module #:  17 elapsed dead time       0.000 s
 FEE module #:  18 elapsed dead time       0.058 s
 FEE module #:  19 elapsed dead time       0.000 s
 FEE module #:  20 elapsed dead time       0.071 s
 FEE module #:  21 elapsed dead time       0.399 s
 FEE module #:  22 elapsed dead time       0.222 s
 FEE module #:  23 elapsed dead time       0.015 s
 FEE module #:  24 elapsed dead time       0.061 s
 FEE module #:  25 elapsed dead time       0.000 s
 FEE module #:  26 elapsed dead time       0.000 s
 FEE module #:  27 elapsed dead time       0.000 s
 FEE module #:  28 elapsed dead time       0.000 s
 FEE module #:  29 elapsed dead time       0.000 s
 FEE module #:  30 elapsed dead time       0.000 s
 FEE module #:  31 elapsed dead time       0.000 s
 FEE module #:  32 elapsed dead time       0.000 s

 *** Program elapsed time:   47.215s (  677.753 blocks/s,  42.360 Mb/s)

12.20 FEE64 temperatures - see attachment 9
      Note with all FADC powered off all temperatures now green cf. attachment 8 https://elog.ph.ed.ac.uk/AIDA/305
      Julabo FL11006 set point +17 deg C (B3F ambient temperature +25.1 deg C, d.p. +11.2 deg C, RH 40.3%)

13.11 AIDA52_13 per FEE64 ADC timestamp activity OK

12.54 AIDA FEE64 lose synchronisation
      No change in system-wide checks 

AIDA53 (AIDA) 2040 (EURICA) 2159 (BigRIPS)

Power cycled and rebooted FEE64s
System-wide checks all OK except 6 failures for ADC calibration - not retried

Start 13.09
Stop 13.44 (stop to reset EURICA VME crate)

ASIC Settings file 2016Jun12-10.10.08

13.17 AIDA53_0 FEE64 sync check - OK

aidas1> ~/td/GREAT/analyser2 /TapeData/NP1512/AIDA53_0
 *** GREAT format 3.2.0 analyser - TD - May 2014
 *** ERROR: READ I/O error:       5002
                   blocks:      32000
          ADC data format:  167386044 ( 1100597.1 Hz)
        Other data format:   93733956 (  616319.7 Hz)
 Sample trace data format:          0 (       0.0 Hz)
         Undefined format:          0 (       0.0 Hz)
   Other data format type:      PAUSE:        714 (       4.7 Hz)
                               RESUME:        714 (       4.7 Hz)
                              SYNC100:      58017 (     381.5 Hz)
                           FEE64 disc:   92936706 (  611077.6 Hz)
                             MBS info:     737805 (    4851.2 Hz)
                           Other info:          0 (       0.0 Hz)

   ADC data range bit set:     166106 (    1092.2 Hz)

                Timewarps:        ADC:          0 (       0.0 Hz)
                                PAUSE:          0 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:          1 (       0.0 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:          0 (       0.0 Hz)
                            Undefined:          0 (       0.0 Hz)
                         Sample trace:          0 (       0.0 Hz)

 Timestamp elapsed time:      152.087 s
 FEE module #:   1 elapsed dead time       0.020 s
 FEE module #:   2 elapsed dead time       0.209 s
 FEE module #:   3 elapsed dead time       0.000 s
 FEE module #:   4 elapsed dead time       0.000 s
 FEE module #:   5 elapsed dead time       0.107 s
 FEE module #:   6 elapsed dead time       0.824 s
 FEE module #:   7 elapsed dead time       0.000 s
 FEE module #:   8 elapsed dead time       0.008 s
 FEE module #:   9 elapsed dead time       0.003 s
 FEE module #:  10 elapsed dead time       0.000 s
 FEE module #:  11 elapsed dead time       0.378 s
 FEE module #:  12 elapsed dead time       0.017 s
 FEE module #:  13 elapsed dead time       0.000 s
 FEE module #:  14 elapsed dead time       0.059 s
 FEE module #:  15 elapsed dead time       0.172 s
 FEE module #:  16 elapsed dead time       1.101 s
 FEE module #:  17 elapsed dead time       0.000 s
 FEE module #:  18 elapsed dead time       0.018 s
 FEE module #:  19 elapsed dead time       0.000 s
 FEE module #:  20 elapsed dead time       0.093 s
 FEE module #:  21 elapsed dead time       0.216 s
 FEE module #:  22 elapsed dead time       0.161 s
 FEE module #:  23 elapsed dead time       0.665 s
 FEE module #:  24 elapsed dead time       0.201 s
 FEE module #:  25 elapsed dead time       0.000 s
 FEE module #:  26 elapsed dead time       0.000 s
 FEE module #:  27 elapsed dead time       0.000 s
 FEE module #:  28 elapsed dead time       0.000 s
 FEE module #:  29 elapsed dead time       0.000 s
 FEE module #:  30 elapsed dead time       0.000 s
 FEE module #:  31 elapsed dead time       0.000 s
 FEE module #:  32 elapsed dead time       0.000 s

 *** Program elapsed time:   25.180s ( 1270.866 blocks/s,  79.429 Mb/s)

AIDA54 (AIDA) 2041 (EURICA) 2160 (BigRIPS) 

Start 13.46
Stop 13.53 (loss of sync AIDA54_0)

ASIC Settings file 2016Jun12-10.10.08

System-wide checks OK following loss of sync

19:25
Stable operation most of the afternoon through multiple DAQ start/stops.
FEEs remained synchronised throughout.
SYNCs lost in run AIDA57_37 leading to run stop/start.
-> in AIDA57_37 noticed relatively large chunks of data lost in nnaida6. (see attachments 10+11

19.40 Cumulative pause-resume time for AIDA57_37 (see above) and most recent
complete data file. The pause-resume times are relatively large compared to
other FEE64s but low relative to the measurement time. But we should remember
that we have observed periods of inactivity not bounded by pause-resume tags.

aidas1> ./analyser2 /TapeData/NP1512/AIDA57_37
 *** GREAT format 3.2.0 analyser - TD - May 2014
 *** ERROR: READ I/O error:       5002
                   blocks:      32000
          ADC data format:  164401851 (  929891.0 Hz)
        Other data format:   96718149 (  547057.9 Hz)
 Sample trace data format:          0 (       0.0 Hz)
         Undefined format:          0 (       0.0 Hz)
   Other data format type:      PAUSE:       1081 (       6.1 Hz)
                               RESUME:       1081 (       6.1 Hz)
                              SYNC100:      67440 (     381.5 Hz)
                           FEE64 disc:   95842999 (  542107.9 Hz)
                             MBS info:     805548 (    4556.3 Hz)
                           Other info:          0 (       0.0 Hz)

   ADC data range bit set:     194112 (    1097.9 Hz)

                Timewarps:        ADC:          0 (       0.0 Hz)
                                PAUSE:          0 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:          0 (       0.0 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:          0 (       0.0 Hz)
                            Undefined:          0 (       0.0 Hz)
                         Sample trace:          0 (       0.0 Hz)

 Timestamp elapsed time:      176.797 s
 FEE module #:   1 elapsed dead time       0.149 s
 FEE module #:   2 elapsed dead time       0.741 s
 FEE module #:   3 elapsed dead time       0.004 s
 FEE module #:   4 elapsed dead time       0.000 s
 FEE module #:   5 elapsed dead time       0.425 s
 FEE module #:   6 elapsed dead time       1.697 s
 FEE module #:   7 elapsed dead time       0.000 s
 FEE module #:   8 elapsed dead time       0.000 s
 FEE module #:   9 elapsed dead time       0.118 s
 FEE module #:  10 elapsed dead time       0.084 s
 FEE module #:  11 elapsed dead time       1.064 s
 FEE module #:  12 elapsed dead time       0.014 s
 FEE module #:  13 elapsed dead time       0.063 s
 FEE module #:  14 elapsed dead time       0.267 s
 FEE module #:  15 elapsed dead time       0.000 s
 FEE module #:  16 elapsed dead time       0.000 s
 FEE module #:  17 elapsed dead time       0.000 s
 FEE module #:  18 elapsed dead time       0.010 s
 FEE module #:  19 elapsed dead time       0.000 s
 FEE module #:  20 elapsed dead time       0.059 s
 FEE module #:  21 elapsed dead time       0.125 s
 FEE module #:  22 elapsed dead time       0.225 s
 FEE module #:  23 elapsed dead time       0.000 s
 FEE module #:  24 elapsed dead time       0.205 s
 FEE module #:  25 elapsed dead time       0.000 s
 FEE module #:  26 elapsed dead time       0.000 s
 FEE module #:  27 elapsed dead time       0.000 s
 FEE module #:  28 elapsed dead time       0.000 s
 FEE module #:  29 elapsed dead time       0.000 s
 FEE module #:  30 elapsed dead time       0.000 s
 FEE module #:  31 elapsed dead time       0.000 s
 FEE module #:  32 elapsed dead time       0.000 s

 *** Program elapsed time:   23.047s ( 1388.475 blocks/s,  86.780 Mb/s)

aidas1> ./analyser2 /TapeData/NP1512/AIDA58_6
 *** GREAT format 3.2.0 analyser - TD - May 2014
 ***            ADC data: block: 14849 ptr: 14685 data: 0xC40181C9 module: 16 fail: 0 range: 0 id:  1025 channel:  
1 adc: 33225  ts: 0x01AB000000A0 OLD: ts: 0x01AB1000003E tag: 3 type: 0
 *** ERROR: READ I/O error:       5002
                   blocks:      32000
          ADC data format:  160685457 (  937303.1 Hz)
        Other data format:  100434543 (  585850.2 Hz)
 Sample trace data format:          0 (       0.0 Hz)
         Undefined format:          0 (       0.0 Hz)
   Other data format type:      PAUSE:       1282 (       7.5 Hz)
                               RESUME:       1282 (       7.5 Hz)
                              SYNC100:      65397 (     381.5 Hz)
                           FEE64 disc:   99536560 (  580612.1 Hz)
                             MBS info:     830022 (    4841.6 Hz)
                           Other info:          0 (       0.0 Hz)

   ADC data range bit set:     213213 (    1243.7 Hz)

                Timewarps:        ADC:          1 (       0.0 Hz)
                                PAUSE:          0 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:          0 (       0.0 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:          0 (       0.0 Hz)
                            Undefined:          0 (       0.0 Hz)
                         Sample trace:          0 (       0.0 Hz)

 Timestamp elapsed time:      171.434 s
 FEE module #:   1 elapsed dead time       0.091 s
 FEE module #:   2 elapsed dead time       0.870 s
 FEE module #:   3 elapsed dead time       0.000 s
 FEE module #:   4 elapsed dead time       0.000 s
 FEE module #:   5 elapsed dead time       0.107 s
 FEE module #:   6 elapsed dead time       0.518 s
 FEE module #:   7 elapsed dead time       0.000 s
 FEE module #:   8 elapsed dead time       0.000 s
 FEE module #:   9 elapsed dead time       0.058 s
 FEE module #:  10 elapsed dead time       0.095 s
 FEE module #:  11 elapsed dead time       2.397 s
 FEE module #:  12 elapsed dead time       0.055 s
 FEE module #:  13 elapsed dead time       0.157 s
 FEE module #:  14 elapsed dead time       0.238 s
 FEE module #:  15 elapsed dead time       0.000 s
 FEE module #:  16 elapsed dead time       0.000 s
 FEE module #:  17 elapsed dead time       0.000 s
 FEE module #:  18 elapsed dead time       0.013 s
 FEE module #:  19 elapsed dead time       0.000 s
 FEE module #:  20 elapsed dead time       0.174 s
 FEE module #:  21 elapsed dead time       0.298 s
 FEE module #:  22 elapsed dead time       0.280 s
 FEE module #:  23 elapsed dead time       0.004 s
 FEE module #:  24 elapsed dead time       0.071 s
 FEE module #:  25 elapsed dead time       0.000 s
 FEE module #:  26 elapsed dead time       0.000 s
 FEE module #:  27 elapsed dead time       0.000 s
 FEE module #:  28 elapsed dead time       0.000 s
 FEE module #:  29 elapsed dead time       0.000 s
 FEE module #:  30 elapsed dead time       0.000 s
 FEE module #:  31 elapsed dead time       0.000 s
 FEE module #:  32 elapsed dead time       0.000 s

 *** Program elapsed time:   18.234s ( 1754.927 blocks/s, 109.683 Mb/s)
Entry  Sat Jun 11 17:24:11 2016, DK, TD, Sunday 12 June 2016 14x

Runs 41 and 42 are quite short.  Run 41 was interrupted by the beam dropping (operators called), and run 42 we are just trying to get EURICA to synchronize with AIDA DAQ.

1:08 Started AIDA Run 43 for 60Ti beam (EURICA run 2027; BigRIPS run 2132; CAEN Run 3018)

          Leak currents less than 5 uA.  See attachments 1 & 2

          Scalers are shown as attachment 3

          FEEE synchronization with pulser signal is shown as attachment 4

          1:21 operator calls to note that the beam is stopped.

 

1:25 started AIDA run 44 for 60Ti beam (EURICA run 2028; BigRIPS #2133; CAEN #3019)

         1:31 stopped because CAEN DAQ goes down.

 

1:34 started AIDA run 45 for 60Ti beam (EURICA run 2029; BigRIPS #2134; CAEN #3020)

          1:37 data stops writing to disk for AIDA for unknown reason.

 

1:50 nnaida11 showed no SYNCs.  Stop / GO on Run Control resolves it.

          but 18 is off for the merger link.  was this the status quo?  try to reboot the merger but everything gets upset.

          my first power cycle alone! (daid)

          load the settings file 2016Jun10-20.30.18 which seems to be the most recent one

          it takes some time, but finally I can get things up and running it seems.

          2:40 still 18 link is down.  strange.  hopefully it's an acceptable loss, because I think it has been a long day for everyone.

          OH DUH!  nnaida19 hasn't been working for the entire experiment!!  Just I forgot on the merge links it is offset 0 and not offset 1 for counting.

          I could get some practice restoring AIDA at least?

          Anyway, there had been some problems from run 45, and now all is well.

 

2:40 AIDA run 46.  It's just a test that the system is working.  But I forgot to put the TapeServer Volume Name so who knows where it ended up (discarded to /dev/null ?)

 

2:42 AIDA run 47, started with EURICA up and ready.  (BigRIPS #2137; EUIRCA #2032; CAEN #3023)

          Basically everything looks reasonable I think

         SSD XY pattern looks basically okay.  See attachment

          Pulser shows the FEEs are synchronized nicely.  See attachment 6

          Biases and leak currents are shown in attachments 7 and 8

          Scalers are shown as attachment 9

 

Check the pause/resumes.  From the html Statistics page, nnaida20 shows some pauses, while nnaida24 seems to pause regularly.

I use Tom's program as outlined in a previous elog.  He said we shall want the elapsed dead time to be much less than the timestamp elapsed

~/td/GREAT/analyser2 /TapeData/NP1512/AIDA47_26

*** GREAT format 3.2.0 analyser - TD - May 2014
 *** FEE64 discriminator: block: 23312 ptr:  2071 data: 0x98638000 module: 24 information type:  6 information field: 0x00038000 ts: 0x0083500000A0 OLD: ts: 0x00836000009F tag: 2 type: 6
 *** ERROR: READ I/O error:       5002
                   blocks:      32000
          ADC data format:  140888182 (  963280.2 Hz)
        Other data format:  120231818 (  822048.6 Hz)
 Sample trace data format:          0 (       0.0 Hz)
         Undefined format:          0 (       0.0 Hz)
   Other data format type:      PAUSE:       4747 (      32.5 Hz)
                               RESUME:       4747 (      32.5 Hz)
                              SYNC100:      55794 (     381.5 Hz)
                           FEE64 disc:  119436810 (  816612.9 Hz)
                             MBS info:     729720 (    4989.2 Hz)
                           Other info:          0 (       0.0 Hz)

   ADC data range bit set:     286072 (    1955.9 Hz)

                Timewarps:        ADC:          0 (       0.0 Hz)
                                PAUSE:          0 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:          1 (       0.0 Hz)
                           FEE64 disc:          1 (       0.0 Hz)
                             MBS info:          0 (       0.0 Hz)
                            Undefined:          0 (       0.0 Hz)
                         Sample trace:          0 (       0.0 Hz)

 Timestamp elapsed time:      146.259 s
 FEE module #:   1 elapsed dead time       0.019 s
 FEE module #:   2 elapsed dead time       0.232 s
 FEE module #:   3 elapsed dead time       0.002 s
 FEE module #:   4 elapsed dead time       0.000 s
 FEE module #:   5 elapsed dead time       0.255 s
 FEE module #:   6 elapsed dead time       0.215 s
 FEE module #:   7 elapsed dead time       0.001 s
 FEE module #:   8 elapsed dead time       0.000 s
 FEE module #:   9 elapsed dead time       0.028 s
 FEE module #:  10 elapsed dead time       0.000 s
 FEE module #:  11 elapsed dead time       0.036 s
 FEE module #:  12 elapsed dead time       0.089 s
 FEE module #:  13 elapsed dead time       0.001 s
 FEE module #:  14 elapsed dead time       0.034 s
 FEE module #:  15 elapsed dead time       0.000 s
 FEE module #:  16 elapsed dead time       0.478 s
 FEE module #:  17 elapsed dead time       0.000 s
 FEE module #:  18 elapsed dead time       0.077 s
 FEE module #:  19 elapsed dead time       0.000 s
 FEE module #:  20 elapsed dead time       0.166 s
 FEE module #:  21 elapsed dead time       0.760 s
 FEE module #:  22 elapsed dead time       0.029 s
 FEE module #:  23 elapsed dead time       0.755 s
 FEE module #:  24 elapsed dead time       0.448 s
 FEE module #:  25 elapsed dead time       0.000 s
 FEE module #:  26 elapsed dead time       0.000 s
 FEE module #:  27 elapsed dead time       0.000 s
 FEE module #:  28 elapsed dead time       0.000 s
 FEE module #:  29 elapsed dead time       0.000 s
 FEE module #:  30 elapsed dead time       0.000 s
 FEE module #:  31 elapsed dead time       0.000 s
 FEE module #:  32 elapsed dead time       0.000 s

It could be around 5% total, but I do not think it is so terrible.

 

4:56 AIDA Run 47_53 begins 4:56.  We do not restart the runs.  EURICA also automatically had the new run 2033 at 4:56.  BigRIPS manually stop / start is now at 2138 (4:57).  CAEN run is still 3023.

          5:02: At start of run, the beam dropped

          5:05 PPAC F3 1A trips.

          Leak current shown as attachment 10 and 11

          Scalers shown as attachment 12

          FEE sync looks good, as well as SSD XY.  About 370 MB of ROOT tree (compared to typical check of 50 MB) Attachments 13 and 14, respectively.

 

Here are Tom's pause information

 

aidas1> ~/td/GREAT/analyser2 /TapeData/NP1512/AIDA47_53
 *** GREAT format 3.2.0 analyser - TD - May 2014
 *** ERROR: READ I/O error:       5002
                   blocks:      32000
          ADC data format:  146454667 (  944617.0 Hz)
        Other data format:  114665333 (  739579.2 Hz)
 Sample trace data format:          0 (       0.0 Hz)
         Undefined format:          0 (       0.0 Hz)
   Other data format type:      PAUSE:       4216 (      27.2 Hz)
                               RESUME:       4216 (      27.2 Hz)
                              SYNC100:      59144 (     381.5 Hz)
                           FEE64 disc:  113857390 (  734368.1 Hz)
                             MBS info:     740367 (    4775.3 Hz)
                           Other info:          0 (       0.0 Hz)

   ADC data range bit set:     272434 (    1757.2 Hz)

                Timewarps:        ADC:          0 (       0.0 Hz)
                                PAUSE:          1 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:          0 (       0.0 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:          0 (       0.0 Hz)
                            Undefined:          0 (       0.0 Hz)
                         Sample trace:          0 (       0.0 Hz)

 Timestamp elapsed time:      155.041 s
 FEE module #:   1 elapsed dead time       0.014 s
 FEE module #:   2 elapsed dead time       0.231 s
 FEE module #:   3 elapsed dead time       0.005 s
 FEE module #:   4 elapsed dead time       0.000 s
 FEE module #:   5 elapsed dead time       0.220 s
 FEE module #:   6 elapsed dead time       0.040 s
 FEE module #:   7 elapsed dead time       0.000 s
 FEE module #:   8 elapsed dead time       0.009 s
 FEE module #:   9 elapsed dead time       0.043 s
 FEE module #:  10 elapsed dead time       0.005 s
 FEE module #:  11 elapsed dead time       0.064 s
 FEE module #:  12 elapsed dead time       0.006 s
 FEE module #:  13 elapsed dead time       0.008 s
 FEE module #:  14 elapsed dead time       0.039 s
 FEE module #:  15 elapsed dead time       0.065 s
 FEE module #:  16 elapsed dead time       0.457 s
 FEE module #:  17 elapsed dead time       0.000 s
 FEE module #:  18 elapsed dead time       0.015 s
 FEE module #:  19 elapsed dead time       0.000 s
 FEE module #:  20 elapsed dead time       0.059 s
 FEE module #:  21 elapsed dead time       0.330 s
 FEE module #:  22 elapsed dead time       0.043 s
 FEE module #:  23 elapsed dead time       0.989 s
 FEE module #:  24 elapsed dead time       0.462 s
 FEE module #:  25 elapsed dead time       0.000 s
 FEE module #:  26 elapsed dead time       0.000 s
 FEE module #:  27 elapsed dead time       0.000 s
 FEE module #:  28 elapsed dead time       0.000 s
 FEE module #:  29 elapsed dead time       0.000 s
 FEE module #:  30 elapsed dead time       0.000 s
 FEE module #:  31 elapsed dead time       0.000 s
 FEE module #:  32 elapsed dead time       0.000 s

 *** Program elapsed time:   21.295s ( 1502.706 blocks/s,  93.919 Mb/s)
 

Entry  Sat Nov 10 21:53:29 2018, CA, CB, TD, OH, Sunday 11 November 21x
06.53 DAQ continues OK - data transfer disabled

06.55 all system wide checks OK *except* nnaida7 fails ADC calibration
      detector biases & leakage currents OK - see attachments 1 & 2
      Julabo FL11006 chiller set point +20 deg C actual +20 deg C, water level c. 75% (L-H)
      ambient tempature + 26.9 deg C, RH 21.6%, d.p. +2.9 deg C

      FEE64 temperatures OK - see attachment 3
      good event statistics OK - see attachment 4

      per FEE64 rate spectra - see attachment 5
      ASIC check load
      per FEE64 rate spectra - see attachment 6
      good event statistics OK - see attachment 7

      1.8.L spectra - see attachments 8 & 9
       nnaida1 pulser peak width 72 ch FWHM

      1.8.W spectra 20us FSR - see attachments 10 & 14
      1.8.W spectra 200us FSR - see attachments 11 & 15
      1.8.W spectra 2ms FSR - see attachments 12 & 16
      1.8.W spectra 20ms FSR - see attachments 13 & 17
      Note - LED threshold 9000 for *all* FEE64s

07.27 statistics zero'd
      some 32bit counters (e.g. nnaida19 & nnaida20) had overflowed/wrapped around

18:19 Computer appears to be running slow though cannot see a reason why.
      No programs using to much ram or cpu
      FEE temp ok - attachment 18
      bias and leakage currents ok - attachment 19
      System wide checks ok

21:02 System wide checks ok
      DAQ Continues ok - data transfer disabled.
      Bias and leakage currents ok - attachment 20
      FEE Temperatures ok - attachment 21
Entry  Sun Jun 11 08:23:23 2017, PV, Sunday 11 June 16:00-00:00  21x
16:40 beam lost - stripper foil problem
      AIDA file RIBF128/R9_55

16:45 beam is back
      AIDA file R9_57


17:19 BRIKEN and BigRIPS stop DAQs
      AIDA on file R9_66

17:21 BigRIPS starts run 4024
      BRIKEN starts run 134
      AIDA on file R9_67

17:24 beam lost
      AIDA on file R9_68

17:50 beam is still off
      BRIKEN and BigRIPS stop DAQs
      AIDA on file R9_74

17.52 Detector biases & leakage currents OK - see attachments 1
      FEE64 Temperatures OK - see attachment 2
      good events statistics stable (nnaida13 still ~140k) - see attachment 3
      Merger and Tape server OK - see attachment 4,5

      *all* system wide checks OK except ADC calibration for nnaida13


18:00 beam is back
      BRIKEN starts run 135
      BigRIPS starts run 4025
      AIDA on file R9_76

18:05 analysis of file RIBF128/R9_77 no timewarps, 6 MBS timewarps  ~1% dead time on nnaida13 - attachment 6

18:15 beam lost. RF is down.
      AIDA on file R9_80

18:19 beam is back
      AIDA on file R9_82

19:04 BRIKEN and BigRIPS stop DAQs
      AIDA on file R9_93

19:05 BigRIPS starts run 4026
      BRIKEN starts run 136
      AIDA on file R9_94

19:40 analysis of file RIBF128/R9_102 no timewarps, 1 MBS timewarps - attachment 7 

19:44 online high energy hit pattern - attachment 8
      AIDA on file R9_104
      zeroed Midas sort online histogram

19.50 Detector biases & leakage currents OK - see attachments 9
      FEE64 Temperatures OK - see attachment 10
      good events statistics stable - see attachment 11
      Merger and Tape server OK - see attachment 12, 13

      *all* system wide checks OK except ADC calibration for nnaida13

20:05 BRIKEN and BigRIPS stop DAQs
      AIDA on file R9_110

20:07 BigRIPS starts run 4027
      BRIKEN starts run 137
      AIDA on file R9_111

20:11 Accelerator people stop the beam to fix the stripper system for about 30 min.
      BRIKEN and BigRIPS stop DAQs
      AIDA on file R9_112

21:12 Beam is back
      AIDA on file R9_127

21:12 BigRIPS starts run 4028
      BRIKEN starts run 138
      AIDA on file R9_127

21:21 offline analysis RIBF128/R9_128 rates
 *** DSSSD # 1 count:    469757 old count:    409726 dt:      1.01 s  LEC rate:  59182.53 Hz
 *** DSSSD # 2 count:    688773 old count:    600379 dt:      1.01 s  LEC rate:  87144.65 Hz
 *** DSSSD # 3 count:    226922 old count:    196673 dt:      1.01 s  LEC rate:  29821.47 Hz
 *** DSSSD # 4 count:    197996 old count:    173086 dt:      1.01 s  LEC rate:  24557.93 Hz
 *** DSSSD # 5 count:     75519 old count:     66217 dt:      1.01 s  LEC rate:   9170.53 Hz
 *** DSSSD # 6 count:    161720 old count:    141631 dt:      1.01 s  LEC rate:  19805.07 Hz
 *** DSSSD # 1 count:         6 old count:         6 dt:      1.01 s  HEC rate:      0.00 Hz
 *** DSSSD # 2 count:       212 old count:       179 dt:      1.01 s  HEC rate:     32.53 Hz
 *** DSSSD # 3 count:        54 old count:        49 dt:      1.01 s  HEC rate:      4.93 Hz
 *** DSSSD # 4 count:        20 old count:        16 dt:      1.01 s  HEC rate:      3.94 Hz
 *** DSSSD # 5 count:        43 old count:        37 dt:      1.01 s  HEC rate:      5.92 Hz
 *** DSSSD # 6 count:       117 old count:       105 dt:      1.01 s  HEC rate:     11.83 Hz

zeroed Midas sort histograms

21:38 beam lost due to stripper problem
      AIDA on file R9_134

21:41 analysis of file RIBF128/R9_134 no timewarps, no MBS timewarps - attachment 14

20:05 BRIKEN and BigRIPS stop DAQs
      AIDA on file R9_138
     
22.00 Detector biases & leakage currents OK - see attachments 15
      FEE64 Temperatures OK - see attachment 16
      good events statistics OK (no beam now) - see attachment 17
      Merger and Tape server OK - see attachment 18, 19

      *all* system wide checks OK except ADC calibration for nnaida13    

22:09 beam is back
      BigRIPS starts run 4029
      BRIKEN starts run 139
      AIDA on file R9_142

22:42 online implantation profile - attachment 20
      AIDA on file R9_151
      zeroed Midas sort online histograms

23:10 BRIKEN and BigRIPS stop DAQs
      AIDA on file R9_159

23:11 BigRIPS starts run 4030
      BRIKEN starts run 140
      AIDA on file R9_159

23:37 analysis of file RIBF128/R9_165 no timewarps, 2 MBS timewarps - attachment 21
Entry  Sun Jun 11 00:07:24 2017, OH, Sunday 11 June 08:00-16:00 25x
08:10 BRIKEN and BigRIPS stops DAQs
      AIDA on file R7_1287

08:11 Beam going for half an hour. Problem with vacuum around RRC

08:24 Beam is back

08:25 BigRIPS starts run 4016
      BRIKEN starts run 126
      AIDA on file R7_1291

08:32 RF down should be 5 minutes

08:44 Beam back

09:25 RF Down

09:26 BigRIPS and BRIKEN stop DAQs
      AIDA on file R7_1306

09:35 Beam back
      BigRIPS starts run 4017
      BRIKEN starts run 127
      AIDA on file R7_1308

09:53 System wide checks all ok
      Bias and leakage currents ok - attachment 1
      Good events - attachment 2
      FEE temperatures - attachment 3

10:29 BRIKEN and BigRIPS stop DAQs
      AIDA on file R7_1322

10:31 BigRIPS starts run 4018
      BRIKEN starts run 128
      AIDA on file R7_1323

11:31 BigRIPS and BRIKEN stop their DAQs
      AIDA on file R7_1338

11:32 BigRIPS starts run 4019
      BRIKEN starts run 129
      AIDA on file R7_1338

11.49 Detector biases & leakage currents OK - see attachments 4 & 5
      FEE64 Temperatures OK - see attachment 6
      good events statistics stable & OK - see attachment 7
      No warnings/errors reported by merger since DAQ GO - see attachment 8

      *all* system wide checks OK (zero fails)

11.53 online analysis
      per DSSSD x versus y (HEC) - see attachment 9
      per DSSSD E_p versus E_n (HEC) - see attachment 10
      ADC & disc data synchronised - see attachment 11

11.55 analysis of file RIBF128/R7_1344 - dead time < 0.5%, no timewarps, 1 MBS timewarps - see attachment 12

12.03 offline analysis RIBF128/R7_1346 rates

 *** DSSSD # 1 count:    942048 old count:    885423 dt:      1.04 s  LEC rate:  54296.31 Hz
 *** DSSSD # 2 count:   1402896 old count:   1319759 dt:      1.04 s  LEC rate:  79718.00 Hz
 *** DSSSD # 3 count:    422965 old count:    397383 dt:      1.04 s  LEC rate:  24529.94 Hz
 *** DSSSD # 4 count:    356558 old count:    336037 dt:      1.04 s  LEC rate:  19677.08 Hz
 *** DSSSD # 5 count:    124978 old count:    117233 dt:      1.04 s  LEC rate:   7426.49 Hz
 *** DSSSD # 6 count:    277397 old count:    260863 dt:      1.04 s  LEC rate:  15854.04 Hz
 *** DSSSD # 1 count:         6 old count:         6 dt:      1.04 s  HEC rate:      0.00 Hz
 *** DSSSD # 2 count:       150 old count:       139 dt:      1.04 s  HEC rate:     10.55 Hz
 *** DSSSD # 3 count:        98 old count:        96 dt:      1.04 s  HEC rate:      1.92 Hz
 *** DSSSD # 4 count:        12 old count:        10 dt:      1.04 s  HEC rate:      1.92 Hz
 *** DSSSD # 5 count:        36 old count:        33 dt:      1.04 s  HEC rate:      2.88 Hz
 *** DSSSD # 6 count:       155 old count:       145 dt:      1.04 s  HEC rate:      9.59 Hz

      all SortSas spectra & histograms zero'd, waveforms set x-range = 1000ch = 2ms

12.04 BNC PB-4
      Amplitude 90,000
      x5 attenuator IN
      t_r 100ns tau_d 50us
      - polarity
      frequency 25Hz range 2.6Hz
      delay MIN, tail pulse, INT ref

      ASIC settings 2017Mar27-16.12.39

      BigRIPS 149Xe setting, F1 slits 51mm,  degrader 2mm Al (at F7), 1.3mm Al fixed + 2mm Al variable (at F11)

12.15 layout spectra - see attachments 13-20

12.20 FEE64# 1-16 power fail
      AIDA on file R7_1351

12.30 requested beam off, replaced 250V/16A/fast fuse

      nnaida13 good events rate ~250k, cycled DSSSD bias reduced to ~140k

13:06 BRIKEN starts run 130
      BigRIPS starts run 4020
      AIDA on file R9_0

13:07 Ambient Temperature +23.9 - Humidity 29.6% - Dew point +5.1C
      Leakage currents and bias ok - attachment 21
      Water level in chiller ok

14:06 BigRIPS and BRIKEN stop their daqs
      AIDA on file R9_16

14:08 BigRIPS starts run 4021
      BRIKEN starts run 131
      AIDA on file R9_16

14:31 System wide checks ok
      Good events - attachment 22
      FEE temperatures - attachment 23
      Analysis output for R9_16 - attachment 24

15:10 BRIKEN and BigRIPS stop daqs
      AIDA on file R9_32

15:11 BRIKEN starts run 132
      BigRIPS starts run 4022
      AIDA on R9_32

15:13 Leakage and bias ok - attachment 25

15:30 DSSD Rates from R9_36
 *** DSSSD # 1 count:     95157 old count:     45985 dt:      1.03 s  LEC rate:  47603.61 Hz
 *** DSSSD # 2 count:    153477 old count:     76756 dt:      1.03 s  LEC rate:  74273.90 Hz
 *** DSSSD # 3 count:     40379 old count:     20036 dt:      1.03 s  LEC rate:  19694.14 Hz
 *** DSSSD # 4 count:     46339 old count:     23014 dt:      1.03 s  LEC rate:  22581.02 Hz
 *** DSSSD # 5 count:     14241 old count:      7012 dt:      1.03 s  LEC rate:   6998.42 Hz
 *** DSSSD # 6 count:     36061 old count:     18008 dt:      1.03 s  LEC rate:  17477.18 Hz
 *** DSSSD # 1 count:         0 old count:         0 dt:      1.03 s  HEC rate:      0.00 Hz
 *** DSSSD # 2 count:        16 old count:         8 dt:      1.03 s  HEC rate:      7.74 Hz
 *** DSSSD # 3 count:        14 old count:         8 dt:      1.03 s  HEC rate:      5.81 Hz
 *** DSSSD # 4 count:         1 old count:         0 dt:      1.03 s  HEC rate:      0.97 Hz
 *** DSSSD # 5 count:         4 old count:         1 dt:      1.03 s  HEC rate:      2.90 Hz
 *** DSSSD # 6 count:        26 old count:        13 dt:      1.03 s  HEC rate:     12.59 Hz

16:10 BigRIPS and BRIKEN stop DAQs
      AIDA on file R9_47

16:12 BRIKEN starts run 133
      BigRIPS starts run 4023
      AIDA on file R9_47

16:18 BigRIPS slit is closing
     DSSD rates for R9_49

 *** DSSSD # 1 count:    121641 old count:     64167 dt:      1.00 s  LEC rate:  57287.24 Hz
 *** DSSSD # 2 count:    179018 old count:     93277 dt:      1.00 s  LEC rate:  85462.38 Hz
 *** DSSSD # 3 count:     52138 old count:     28005 dt:      1.00 s  LEC rate:  24054.58 Hz
 *** DSSSD # 4 count:     53927 old count:     27419 dt:      1.00 s  LEC rate:  26421.86 Hz
 *** DSSSD # 5 count:     16617 old count:      9291 dt:      1.00 s  LEC rate:   7302.19 Hz
 *** DSSSD # 6 count:     38751 old count:     19986 dt:      1.00 s  LEC rate:  18704.02 Hz
 *** DSSSD # 1 count:         1 old count:         0 dt:      1.00 s  HEC rate:      1.00 Hz
 *** DSSSD # 2 count:        52 old count:        30 dt:      1.00 s  HEC rate:     21.93 Hz
 *** DSSSD # 3 count:        12 old count:         7 dt:      1.00 s  HEC rate:      4.98 Hz
 *** DSSSD # 4 count:         3 old count:         3 dt:      1.00 s  HEC rate:      0.00 Hz
 *** DSSSD # 5 count:         9 old count:         5 dt:      1.00 s  HEC rate:      3.99 Hz
 *** DSSSD # 6 count:        37 old count:        19 dt:      1.00 s  HEC rate:     17.94 Hz
Entry  Sun Dec 10 13:06:27 2023, TD, Sunday 10 December 14x
14.05 S4 closed

Water flow and temperature ( gauges outside S4 entrance ) OK

BNC PB-5 pulser connected to p+n FEE64s by diasy chain of test + inputs

ASIC settings 2023Nov16-18.55.20
 p+n FEE64s slow comparator 0xa, n+n FEE64s 0xf

All system wide checkks OK *except* adia08 ADC calibration failed

DSSSD bias -150V leakage current -5.28uA - OK - attachment 1

FEE64 temperatures OK - attachment 2

ADC data item stats - attachment 3
 rates dominated by relatively few 'hot' channels
 aida06 - no cables connected to FEE64 adaptor PCB

per FEE64 Rate spectra - attachment 4
 aida06 - no cables connected to FEE64 adaptor PCB 
 system relatively quiet but has been quieter - no grounding of bPlast cabling and PSUs?

p+n and n+n FEE64 1.8.W spectra - 20us FSR - attachments 5-8
 aida06 - no cables connected to FEE64 adaptor PCB
 aida08 - no ADC calibration 

18.25 

SSSD bias -150V leakage current -5.100uA - OK - attachment 9

FEE64 temperatures OK - attachment 10

ADC data item stats - no significant change - attachment 11
 rates dominated by relatively few 'hot' channels
 aida06 - no cables connected to FEE64 adaptor PCB

per FEE64 Rate spectra - no significant change - attachment 12
 aida06 - no cables connected to FEE64 adaptor PCB 
 system relatively quiet but has been quieter - no grounding of bPlast cabling and PSUs?

p+n and n+n FEE64 1.8.W spectra - 20us FSR - attachments 13-14
 aida06 - no cables connected to FEE64 adaptor PCB
 aida08 - no ADC calibration 
Entry  Sun Oct 1 12:38:56 2023, TD, Sunday 1 October 13x
12.50 Power up

      All system wide checks OK

      per FEE64 p+n 1.8.W spectra 20us FSR - attachments 3-4
      per FEE64 p+n 1,8.L spectra - attachment 6
       aida01 pulser peak width 15 ch FWHM
      per FEE64 Rate spectra - attachment 5


      BNC PB-5 pulser
      Amplitude 10.0V
      Attenuation x10
      Decay 1ms
      Frequency 25Hz
      Polarity + ( n+n Ohmic test input from EG&G Ortec 534 Sum & Invert Amplifier not connected )
      Clamp ON

      ASIC settings 2021Apr29-13-16-00
      slow comparator 0x64


      FEE64 temperatures OK - see attachment 1
      ADC data item stats OK - see attachment 2
Entry  Sun Nov 6 01:34:15 2016, TD, OH, CG, Sun 6th November 18x
10:34 Detector Bias a leakage currents all ok. (Attachments 1 and 2)
      Temperatures all ok (Attachment 3)
      All system wide checks ok other than clock status see attachment 4
      DAQ continues to run ok (R124) (No stop at 6:30am)
      DAQ running at about 14MB/s to disc which is faster than before.
      Most likely due to the increased rate in detector 1 which is represented at FEEs 17, 20, 22 and 23 in
      attachment 5.

      Also attached are the spectra for layouts 1-4 & 7-8 as attachments 6-11
      When looking at the pulser spectrum in layouts 3 and 4 (attachments 9 and 10) we see that the pulser peaks
relating to this detector is broader shown by
      the decreased peak height.

      Comparing the waveforms in attachments 11 and 12 to those taken on Monday 31st Oct we see that the noise
in nnaida 5 and 6 has been greatly reduced.
      On the others however there appears to be an increase in the amplitude of the noise.

      11:25 The second putty window for leakage currents was found to have been frozen for the past 24 hours.
Was reloaded and updated leakage currents are in attachment 13 and 14

12.15 Analysis of R124_291 shows ADC & disc data synchronised - see attachment 15
      Analysis of data file - see attachment 16

18:26 DAQ stopped, presumably during LN2 fill cycle.
      Restarted the merger and tried "Stop" on the DAQ control page. ~50% of nnaida now show "error."
      R124 closed 18.29
      Could not regain control of the system. Jorge switched BRIKEN to its internal clock and FEEs power cycled @
19.20.

23.35 DAQ started (R126 - no storage mode)

      DSSSD bias & leakage currents OK - see attachment 
ELOG V3.1.3-7933898