> > > 
> > > 14.00 Both AIDA NIM bins U/S 
> > >       Upper NIM bin - BNC PB-4, Ortec 433, 2x CAEN N1419, 1x CAEN N1419B - ac mains fuse 100V/8A/slow? blowing
> > >       Lower NIM bin - 11x MACB - not powering up
> > > 
> > >       Replaced both NIM bins.
> > >       Upper NIM bin - 11x MACB
> > >       Lower NIM bin - BNC PB-4, Ortec 433, 2x CAEN N1419, 1x CAEN N1419B
> > > 
> > > 15.50 Julabo FL11006 chiller ON - set point +20 deg C
> > >       B2F/F11 ambient temp +24.0 deg C, RH 54%, DP +13.8 deg C
> > > 
> > >       Bias 6x MSL type BB18 DSSSDs - see attachments 1-2
> > > 
> > > 16.10 Startup DAQ nnaida1-24 *except* nnaida19
> > > 
> > >       System-wide checks
> > > 
> > >       Check clock status OK
> > >       Check ADC calibration - nnaida7 failed
> > >       Check ASIC clock timestamp - multiple failures - see attachment 3
> > >       Check SYNC errors not counted Since Baseline - see attachment 4
> > >       Check SYNC pulses received - see attachment 5
> > > 
> > >       BNC PB-4 
> > > 
> > >       Amplitude 90,000, x5 attenuator IN
> > >       t_r 50ns, tau_d 50us
> > >       Polarity -
> > >       INT ref, delay MIN, tail pulse
> > > 
> > >       FEE64 temperatures - see attachment 6
> > > 
> > > 16.34 R1 starts (directory Aug2016)
> > > 
> > >       slow comparator 0x10, LEC/MEC fast comparator 0x10
> > > 
> > >       merge 555k data items/s, tapeserver 4430 kb/s
> > Looks like a problem with the inter-MACB wiring.
> > The screen shots show groups of 4 with the problems.
> > I suggest re-seating the short HDMI cables and reset the system.
> 
> Short HDMI cables re-seated. See attachment 1

ok, the discrepancy is odd. Not sure if it is due to an uneven reset or more noise in the wiring.
In system wide checks redo the baseline for the SYNC error counters and keep an eye on it.

Maybe a power cycle will bring all the SYNC received counters into line.

Certainly they should be exactly the same.

Is there any correlation between the ones that are wrong and the MACB layout/cabling ?

AIDA ASIC settuings 2023Nov16-18.55.20
     low comparator p+n FEE64s 0xa, n+n 0xf

     BNC PB-5 settings
     amplitide 10.0V
     attenuation x10
     decay time 1ms
     polarity + ( connected to FEE64 aida01 only )
     frequency 22Hz
     tail pulse
     remote control

16.46 per p+n FEE64 1.8.W spectra - 20us, 200us, 2ms & 20ms FSR - attachments 1-4
      per n+n FEE64 1.8.W spectra - 20us, 200us, 2ms & 20ms FSR - attachments 5-8

      DSSSD bias -150V leakage -5.690uA - attachment 9
      RH 32.7% ambient 2+3.2 deg C d.p. +5.9 deg C
      FEE64 temps OK - attachment 10


     bPlast configuration
     1x long and 1x short side connected to 3x bPlast booster PCBs
     Rohde & Schawrtz HMP4040 PSU ch 4 +4V ( ref'd to PSU chassis ground ) 
     Mesytec SiPM bias
     Screens of flat cables from bPlast to booster cards electrically connected together and to bPlast NIM bin c( below AIDA stand ) chassis
     bPlast NIM bin and TAMEX electrically isolated from AIDA stand/DESPEC platform

     Photos - attachments 11-16

17.00 Connect SC21 L disc o/p fan-out -> ESN LA8000 TTl output -> BNC PB-5 pulser ext trig
      Should enable triggering ( and delay ) of pulser from heavy-ions in scintillator SC21


17.17 LEC/MEC selection 0x -> 0x1
      ADC data item stats - attachment 20
      per FEE64 1.8.W spectra - 20us FSR - attachments 18-19
      per FEE64 Rate spectra - attachment 17
      BNC PB-5 pulser to aida01 - some channels missing asic # 2, 3 & 4
      aida04 asic# 1 rates still high but aida06 ( was aida10 ) high rates  in only two channels of asic #4
      Otherwise system very quiet as expected for MEC ( 1GeV FSR ).

17.56 LEC/MEC selection 0x1 -> 0x0

22.10 NH starts data transfer to Merger and TapeServer ( no storage mode ). Date Relay to MBS also started - see attachments  21-25

22.18 aida04 asic #1 slow comparator 0xf -> 0xff
      aida06 asic #4 slow comparator 0xa -> 0xff
      Merger data transfer c. 500k data items/s



     

19.03 DAQ started OK - data transfer not enabled

      BNC PB-4
      Amplitude 90,000
      x5 attenuator IN
      t_r 100ns tau_d 50us
      - polarity
      frequency range 25Hz
      delay MIN, tail pulse, INT ref

      ASIC settings 2017Oct23-14.38.03
      *all* FEE64s slow comparators 0xa

      Detector biases & leakage currents OK - see attachments 1
      FEE64 Temperatures OK - see attachment 2
      DAQ statistics OK - see attachment 3
      *all* system wide checks OK *except* nnaida7 fails ADC calibration
      Rate look OK. A few dropped channels. See attach 4
      Waveforms look OK. Noise at reasonable levels. See attach 5 & 6

19:15 Started run Nov17/R1
      Kbyte/sec written 2611
      Merger data rate 346210 items/sec

19:26 Observed increase in rate in asics nnaida13, 19 (>50 000)
      Noise profile looks typical. See waveforms - attach 7,8
      Increased FEE64 slow comparators for nnaida 13, 19 from 0xa to 0xc
      Rate decreases significantly. See attach 9

19.40 Beam on
      BigRIPS 115Nb setting
      WAS3ABI + YSO

      Good Event, merger and data rates OK - see attachments 10-12
      8x FEE64s <10k
      8x FEE64s 10-20k
      6x FEE64s 20-30k
      2x FEE64s 30-40k

19.57 per FEE64 1.8.W spectra (2ms FSR) - see attachments 14-15

20.06 per FEE64 1.8.L spectra - see attachments 16-17
      nnaida17 67 ch FWHM
      nnaida8  73 ch FWHM

20.50 DAQ starts

RIKEN run: 005
Wasabi Run: 4048
BigRIPS run: 2057
AIDA run R2_0
Listmode DLT file:171117_2050_115Nb_005.dlt

BRIKEN Rate (h:12): ~ 42 cps
F11 Rate (h:18): ~ 37 cps
CLOVERS Rate (h:15): ~ 33 cps
AIDA Veto Plastic Rate (h:19): ~ 23

      WAS3ABi in
      YSO removed
      AIDA inserted to HPGe clover cryostat position

      BigRIPS 115Nb setting
      Degrader 5mm Al

      BNC PB-4
      Amplitude 90,000
      x5 attenuator IN
      t_r 100ns tau_d 50us
      - polarity
      frequency range 25Hz
      delay MIN, tail pulse, INT ref

      ASIC settings 2017Oct23-14.38.03
      *all* FEE64s slow comparators 0xa *except* nnaida13 0xf, nnaida19 0xc, nnaida20-21 0xb

20.57 all spectra zero'd

      Good Event stats - see attachment 18
      7x FEE64s <10k
      5x FEE64s 10-20k
      4x FEE64s 20-30k
      4x FEE64s 30-40k
      3x FEE64s 40-50k
      1x FEE64 >50k (nnaida13 73k)



21.22 Not all ASICs settings restored from 2017Oct23-14.38.03
      Some p+n FEE64s Ibias LF feedback 0x8 *not* 0xf as expected - now changed manually
      AIDA data file Nov17/R2_5

      ASIC settings saved to 2017Nov17-21.24.47
      Updated options

21.33 New BigRIPS, BRIKEN, WAS3ABi files started
      AIDA continues from R2_6

Date: 17/11/017
Start: 2133
RIKEN run: 006
Wasabi Run: 4061
BigRIPS run: 2058
AIDA run R2_06
Listmode DLT file:171117_2132_115Nb_006.dlt

BRIKEN Rate (h:12): ~ 42 cps
F11 Rate (h:18): ~ 37 cps
CLOVERS Rate (h:15): ~ 33 cps
AIDA Veto Plastic Rate (h:19): ~ 23


21.40 Detector biases & leakage currents OK - see attachments 19 & 20
      FEE64 Temperatures OK - see attachment 21
      DAQ statistics OK - see attachment 22
      *all* system wide checks OK *except* nnaida7 fails ADC calibration

      per FEE64 1.8.* spectra (20ms FSR) see attachments 23 & 24
      p+n spectra now look OK with Ibias LF feedback changed from 0x8 to 0xf
      note 3kHz appears in nnaida19, 21 & 22

22.41 DAQ stop
      AIDA data file Nov17/R2_15

      *use R2_8 - R2_15* for offline analysis

22.50 AIDA withdrawn from BRIKEN
      YSO re-installed

23.10 Detector biases & leakage currents OK - see attachments 25 & 26
      FEE64 Temperatures OK - see attachment 27
      DAQ statistics OK - see attachment 28
      7x FEE64s <10k
      4x FEE64s 10-20k
      8x FEE64s 20-30k
      2x FEE64s 30-40k
      2x FEE64s 40-50k
      1x FEE64 >50k (nnaida13 64k)
      *all* system wide checks OK *except* nnaida7 fails ADC calibration

      per FEE64 1.8.W spectra (20ms FSR) - see attachments 29 & 30
      per FEE64 rate spectra linear and log scales - see attachments 31 & 32

23.30 all spectra zero'd

10.18 DAQ continues OK to file May19/R5_1

      alpha background

      ASIC settings 2018Oct03-13.26.58
      *except* slow comparator 0xa -> 0x64

      BNC PB-4 Pulser OFF

      all system wide checks OK *except* nnaida7 fails ADC calibration

      detector biases & leakage currents OK - see attachments 1 & 2

      FEE64 temperatures OK - see attachment 3
      good event statistics OK - see attachments 4

      Merger OK - see attachment 5
      TapeServer OK - see attachment 6

      1.8.W spectra 20us FSR - attachments 7 & 8
      1.8.W spectra 20ms FSR - attachments 9 & 10

      Julabo FL11006 chiller set point +20 deg C actual +20.0 deg C - water topped up
      Temperature +24.8 deg C dew point +8.3 deg C RH 35.3%

10.42 DAQ stop May19/R5_2

      slow comparator 0x64 -> 0xa
      Check ASIC Control *all* ASICs & FEE64s

10:52 AIDA inserted ~4cm into DTAS
      60Co (~40kBq) + 137Cs (~8kBq) sources located approximately 4cm in front of the front face of the snout

10.54 DAQ start May19/R6

      good event statistics OK - see attachments 11
      *all* FEE64s <100k, 9x < 20k
      data rate 4.9Mb/s 0.64M data items/s

11.04 nnaida17 good events c. 350k returns to 13k following Check ASIC Control (nnaida1 OK - see
https://elog.ph.ed.ac.uk/AIDA/801 )
      AIDA disc statistics indicate that fast discriminators were source of higher rate - next time this 
      happens we should check HitRate to see whether this is a specific channel or channels and possibly 
      mask

11.17 DTAS DAQ issues correlation scaler reset 
      AIDA data file May19/R6_3

      use files *from* R6_4

11.39 DAQ stop file May19/R6_8

      nnrpi1 is offline - we suspect wrench or power tool placed too close to RPi when we moved AIDA
      into DTAS and we jostled one or other of the cables connected to nnrpi1

      nnrpi1 power cycled to reboot
      *all* FEE64s OK
      USB ac mains relay controls recovered OK
      CAEN N1419 contrs recovered OK

11.48 AIDA withdrawn from DTAS for installation of upstream (CH2)n shield and additional Pb wall

11.52 DAQ starts to file May19/R7

      alpha background resumes

      ASIC settings 2018Oct03-13.26.58
      *except* slow comparator 0xa -> 0x64

      BNC PB-4 Pulser OFF

18.06 DAQ continues ok, on file May19/R7_0

      all system wide checks OK *except* nnaida7 fails ADC calibration

      detector biases & leakage currents OK - see attachment 13

      FEE64 temperatures OK - see attachment 14
      good event statistics OK - see attachment 15

      Merger and TapeServer ok

      Temperature +25.0 deg C dew point +4.4 deg C RH 27.0%



  

07.56 DAQ continues - AIDA file RIBF148/R5_822
      165Pm BigRIPS setting + 2.8mm Al degrader

08.13 zeroed xy hit pattern on online analysis and MIDASsort

08.30 system wide checks ok, except nnaida7 ADC calibration failure
      FEE64 temperatures ok (attachment 1)
      good event statistics ok (attachment 2)
      detector bias/leakage currents ok (attachment 3)

08.34 Merger and TapeServer ok, no error messages in terminal (attachments 4/5)

08.40 zeroed all spectra on online analysis and MIDASsort

08.44 BRIKEN and BigRIPS stop
      AIDA file RIBF148/R5_830

08.45 BigRIPS starts run 1087
      BRIKEN starts run 102
      AIDA on file RIBF148/R5_830

09.13 n vs p multiplicity (attachment 6)
      xy hit pattern (*deleted, took wrong screenshot, sorry! -CA@12.16*)
      Energy X vs Energy Y plot (*deleted, took wrong screenshot, sorry! -CA@12.16*)

12.17 see elog entry for 15th November 2018 00.00-08.00 (attachments 9 and 10) for screenshots of the above before
      degrader change
      
09.17 BRIKEN and BigRIPS stop
      AIDA on file RIBF148/R5_836
      beam stop to change Al degrader thickness to 3.3mm

09.23 chiller refilled while access available
      humidity, temperature, dew point measurements obtained

09.25 detector bias/ leakage currents ok (attachment 7)

09.30 zeroed all spectra in online analysis and MIDASsort

09.31 beam back on
      BRIKEN and BigRIPS start
      AIDA file RIBF148/R5_838
      BRIKEN run 103
      BigRIPS run 1088

10.32 BRIKEN and BigRIPS stop
      AIDA file RIBF148/R5_848

10.33 BRIKEN and BigRIPS start
      AIDA file RIBF148/R5_848
      BRIKEN run 104
      BigRIPS run 1089

10.45 system wide checks ok, except nnaida7 ADC calibration failure
      FEE64 temperatures ok (attachment 8)
      good event statistics ok (attachment 10)
      detector bias/leakage currents ok (attachment 9)

11.20 ran analyser on file R5_847 - attachment 11
      no timewarps observed
      dead time in nnaida19 at ~1s out of 355.283s (~0.3%)
      rest at zero, except nnaida20 at ~0.6s and nnaida21 at ~0.1s

11.33 BRIKEN and BigRIPS stop
      AIDA file RIBF148/R5_859

11.34 BRIKEN and BigRIPS start
      AIDA file RIBF148/R5_859
      BRIKEN run 105
      BigRIPS run 1090

12.05 n vs p multiplicity (attachment 12)
      xy hit pattern (attachment 13)
      Energy X vs Energy Y plot (attachment 14)
      this is after change of Al degrader thickness to 3.3mm

12.36 BRIKEN and BigRIPS stop
      AIDA file RIBF148/R5_869

12.36 BRIKEN and BigRIPS start
      AIDA file RIBF148/R5_869
      BRIKEN run 106
      BigRIPS run 1091

12.40 online histograms
      per FEE64 Rate - attachment 15
      per FEE64 HitRate - attachment 16
      1.8.L spectra - attachments 17-20
       nnaida1 pulser peak width 70ch FWHM
      1.8.H spectra - attachments 21-22
      1.8.W spectra - attachments 23-24

12.47 beam off
      AIDA file RIBF148/R5_870

12.54 BRIKEN and BigRIPS stop
      AIDA file RIBF148/R5_871

13.10 MIDASsort analysis RIBF148/R5_871 - all ADC data synchronised - attachments 25-26

13.16 correlation analysis decay plots for each detector with the Al degrader thickness at 3.3mm (attachment 23)
      see elog entry November 15th 2018 00:00-08:00 (attachments 16-18) for same plots with previous degrader
      thickness.

13.26 system wide checks ok, except nnaida7 ADC calibration failure
      FEE64 temperatures ok (attachment 24)
      good event statistics ok (attachment 25)
      detector bias/leakage currents ok (attachment 26)

15.04 see other elog entries for correlation analysis and high energy Ex vs Ey plots compared between different Al
degrader thicknesses.

15.38 system wide checks ok, except nnaida7 ADC calibration failure
      FEE64 temperatures ok (attachment 28)
      good event statistics ok (attachment 27)
      detector bias/leakage currents ok (attachment 29)

15.55 BRIKEN start background measurement
      BRIKEN not writing to file
      AIDA file RIBF148/R5_896
      Online histograms available

16.00 BRIKEN start writing to file
      AIDA file RIBF/R5_897
      BRIKEN run 107

17.18 DAQ stop AIDA file RIBF148/R5_907

      End of run RIBF148 due to accelerator problems requiring ~3-4 days to fix
      'low' mass BigRIPS setting postponed 

      ASIC check load

      all onine histograms zero'd

17.22 DAQ start AIDA file RIBF148/R6
      Pulser walkthrough
      BNC PB-4 amplitudes 90,000 - 10,000 @ 10,000 step
      x5 attenuator IN
      Frequency 25Hz range
      Delay MIN
      Polarity -
      Tail pulse

17.37 DAQ trop AIDA file RIBF148/R6_2
      1.8.L spectra - see attachments 30 & 31
 

17.40 DAQ start AIDA file RIBF148/R7
      Pulser walkthrough
      BNC PB-4 amplitudes 1,200 - 1,000 @ 1,000 step
      x5 attenuator IN
      Frequency 25Hz range
      Delay MIN
      Polarity -
      Tail pulse

17.50 DAQ stop AIDA file RIBF148/R7_1
      1.8.L spectra - see attachments 32 & 33

17.54 DAQ start AIDA file RIBF148/R8
      background
      BNC PB-4 pulser OFF

18.00 detector biases & leakage currents OK - see attachments 34 & 35

      FEE64 temperatures OK - see attachment 36
      good event statistics OK - see attachments 37

      Merger OK - see attachment 38
      TapeServer OK - see attachment 39
      no error or warning messages terminal server sesssions reported

      per FEE64 Rate spectra - attachment 40

      1.8.W spectra 20ms FSR - attachments 41 & 42

18.10 DAQ stop AIDA file RIBF148/R8_2

      Detector bias OFF
      FEE64 power OFF

18.12 Ambient temperature +25.5 deg C, d.p. +2.2 deg C, RH 21.5%

18.15 Withdraw AIDA from BRIKEN - done

      2x NIM bin OFF
      1x NIM bin fan tray OFF
      1x USB ac mains relay OFF
      Julabo FL11006 chiller OFF front panel/rear panel/wall-mounted 3ph outlet
      2x network cables disconnected and coiled/cable-tied to stairwell railings
      disconnect coorelation scaler trigger & reset inputs to root MACB front panel
      disconnect 50MHz TTL clock root MACB rear panel
      disconnect AIDA plastic veto
      discopnnect 2x ac mains cabnles from ac mains distribution box

      remove AIDA support assembly nose support
19.00 raise AIDA support assembly feet/lower AIDA support assembly wheels from/to F11 floor


       











 








      
      












      



      



      


  
     
      






 

00.25 System wide checks ok * except nnaida 7 fails adc*
      FEE Temps ok - attachment 1
      Stats ok - attachment 2 
      Leakage currents ok - attachment 3
      tape server and merger ok

00:57 BigRIPS and BRIKEN stop
      AIDA on file R5_753

00.59 BigRIPS starts run 1079
      BRIKEN starts run 94
      AIDA on file R5_753

01:56 BigRIPS and BRIKEN stop
      AIDA on file R5_762

01.57 BigRIPS starts run 1080
      BRIKEN starts run 95
      AIDA on file R5_762

02:47 BigRIPS and BRIKEN stop
      AIDA on file R5_770

02.48 BigRIPS starts run 1081
      BRIKEN starts run 96
      AIDA on file R5_770

03:45 BigRIPS and BRIKEN stop
      AIDA on file R5_780

03.47 BigRIPS starts run 1082
      BRIKEN starts run 97
      AIDA on file R5_780

04:45 BigRIPS and BRIKEN stop
      AIDA on file R5_790

04.47 BigRIPS starts run 1083
      BRIKEN starts run 98
      AIDA on file R5_790

04.50 System wide checks ok * except nnaida 7 fails adc*
      FEE Temps ok - attachment 4
      Stats ok - attachment 5
      Leakage currents ok - attachment 6
      tape server and merger ok

05.46 BigRIPS and BRIKEN stop
      AIDA on file R5_800

05.47 BigRIPS starts run 1084
      BRIKEN starts run 99
      AIDA on file R5_800

06.45 BigRIPS and BRIKEN stop
      AIDA on file R5_810

06.46 BigRIPS starts run 1085
      BRIKEN starts run 100
      AIDA on file R5_810

07.25 System wide checks ok * except nnaida 7 fails adc*
      FEE Temps ok - attachment 7
      Stats ok - attachment 8
      Leakage currents ok - attachment 9
      tape server and merger ok

07.44 BigRIPS and BRIKEN stop
      AIDA on file R5_820

07.45 BigRIPS starts run 1086
      BRIKEN starts run 101
      AIDA on file R5_820

10.35 DAQ continues
      AIDA file RIBF148/R10_14

      BNC PB-4
      Amplitude 90,000
      x5 attenuator IN
      t_r 100ns tau_d 50us
      - polarity
      frequency 25Hz range 2.6Hz
      delay MIN, tail pulse, INT ref

      ASIC settings 2017Mar27-16.12.39
      *except* 
      slow comparator 0x64
      LEC/MEC fast comparator 0x14

10.35 Detector biases & leakage currents OK - see attachments 1 & 2
      FEE64 Temperatures OK - see attachment 3
      good events statistics stable & OK - see attachment 4 
      No warnings/errors reported by merger since start of RIBF148/R10  - see attachment 5
      No changes to PLL and DCM clock Lock monitor - see attachment 6
      *all* system wide checks OK (zero fails)

10.41 analysis file RIBF148/R8_39 - zero deadtime, 2x MBS and zero ts timewarps - see attachment 7

10.47 online analysis 
      ADC & disc data synchronised - see attachment 7
      per DSSSD m_p versus m_n (LEC) - see attachments 8
      per DSSSD x versus y (LEC) - see attachment 9
      per DSSSD E_p versus E_n (LEC) - 512x512 spectra - nominal 20keV per channel - see attachment 10
      per DSSSD side ADC ts - event ts - see attachment 11

11.02 Layout per FEE64 Rate, HitRate and 1.8.* spectra - see attachments 11-21
      nnaida11 & nnaida13 1.*.W spectra - see attachments 22-24

      3kHz modulated HF noise positive input FEE64s
      BRIKEN and Clovers on
       
11.30 Ambient temperature +24.0 deg C, d.p. +9.6 deg C, RH 39.9%
      Julabo FL11006 set point +20.0 deg C actual +20.0 deg C - water level c. 75%

16.19 Detector biases & leakage currents OK - see attachments 23 & 24

19.48 Detector biases & leakage currents OK - see attachments 25 & 26

Friday 14th June 

Authors: TD, DK, CB

08:40

  elog appears to be down.  Relevant screenshots are in /tmp/june14/ so please upload when we can access

  Julabo chiller 75% M-H markers
  Room Temp +24.5 C deg, dew point +8.1 deg C, Room humidity 35.2%

  Pulser, ASIC settings as yesterday (copy/paste)

      ASIC settings 2019May19-17.54.03
      slow comparator 0xa

      BNC PB-4 Pulser 
      Amplitude 90,000
      x5 attenuator IN
      Frequency 25Hz range
      t_r 100ns, tau_d 50us
      - polarity
      Delay MIN, tail pulse

  DAQ continues OK
  Statistics as before.  nnaida19 counter appears to have overflowed and wrapped around
    See stats.png
  
  System-wide checks OK, except nnaida7 fails ADC calibration as before

  Biases and leak currents reasonable -- see biases.png

  FEE64 temperatures okay, see fee-temps.png

  Merger looks okay, ~600k items / sec; see merger.png

  TapeServer as before, around 4.5 Mbytes/sec; see tapeserver.png


13:52

   Pulser, ASIC settings same as this morning

   DAQ continues OK
   Statistics as before. See attach 190614_1354_stats. Counts RESET.

   System-wide checks OK, except nnaida7 fails ADC calibration as before

   Bias and leak currents OK. See attach 190614_1356_bias

   FEE64 temperatures OK, see attach 190614_1357_temps

   Merger keeps running at ~560k items /s; see 190614_1359_merger
   TapeServer as before. See 190614_tape

17:24 AIDA DAQ crashed by DTAS touching the clock function generator
      Function generator set at 50MHz square high +5V low 0V

18:38 problem loading ASIC settings - mismatch in slow comparator threshold and shaping time
      performed save/restore with 2019Jun13-08.56.35 - all looks ok now
      
      System wide checks ok except nnaida7 fails adc calibration

      FEE64 temperatures ok - attachment 12

      good event statistics ok - attachment 13

      bias/leakage currents ok - attachment 14

     
18:38 for reals

      DAQ Stopped
      R30_0 opened writing to disk
      
      Was to test DTAS sync whilst writing to file
18:43 DAQ stopped on file R30_0

18:45 DAQ restarted on file R31 with no writing to disk

21:50 DAQ found crashed (Valencia group believe it crashed around 20:00)

      Trying to reset FEEs resulted in an error message attachment 15 and 16

      Each FEE was rebooted using telnet to avoid a power cycle
      System reset after reboot

      System wide checks ok
      Check ASIC control 

      FEE temps ok

      DAQ Started again
      On startup FEEs 17, 20, 22 and 23 were all about 150k
      All relate to DSSD0
      Pressed on kapton and noise went down.

      Statistics now good

22.45 Review /var/log/messages and observe nnaida15 rebooted at 19.27 - reason unknown
      This probably explains why DAQ stopped.

Jun 14 19:27:32 aidas1 dhcpd: DHCPDISCOVER from 00:04:a3:2a:f3:ac via p2p2
Jun 14 19:27:32 aidas1 dhcpd: DHCPOFFER on 10.1.2.15 to 00:04:a3:2a:f3:ac via p2p2
Jun 14 19:27:32 aidas1 dhcpd: DHCPREQUEST for 10.1.2.15 (10.1.2.250) from 00:04:a3:2a:f3:ac via p2p2
Jun 14 19:27:32 aidas1 dhcpd: DHCPACK on 10.1.2.15 to 00:04:a3:2a:f3:ac via p2p2
Jun 14 19:27:32 aidas1 rpc.mountd[2868]: authenticated mount request from 10.1.2.15:784 for
/data10/XilinxLinux/ppc_4xx/rfs/nnaida15 (/data10/XilinxLinux/ppc_4xx/rfs)
Jun 14 19:27:59 aidas1 rpc.mountd[2868]: authenticated mount request from 10.1.2.15:864 for /MIDAS@aidas
(/MIDAS@aidas)


22.53 ADC data synchronised - attachment 17

23.05 zero DAQ statistics
      zero all histograms
      

23.20 Check ASIC Control x1 
      FEE64 Rate spectra - attachment 18

      1.8.L spectra - see attachments 19 & 20
      nnaida1 81 ch FWHM, nnaida3 77 ch FWHM

      p+n FEE64s 1.8.W spectra 20us FSR - attachment 21
      n+n FEE64s 1.8.W spectra 20us FSR - attachment 22

23.36 DAQ start file May19/R34_0

      DTAS+AIDA correlation test using gamma ray sources

      Correlation scaler reset issued during R34_0

      ASIC settings 2019Jun13-08.56.35

      BNC PB-4 Pulser 
      Amplitude 90,000
      x5 attenuator IN
      Frequency 25Hz range
      t_r 100ns, tau_d 50us
      - polarity
      Delay MIN, tail pulse

23.53 DAQ stop file May19/R34_2

      nnaida15 did not stop - state unknown - reboot nnaida15

      This did not work - unable to reset nnaida15

      Closed all FireFox broweser tabs, stopped all servers and restarted

00.14 DAQ start

      ASIC settings 2019Jun13-08.56.35

      BNC PB-4 Pulser 
      Amplitude 90,000
      x5 attenuator IN
      Frequency 25Hz range
      t_r 100ns, tau_d 50us
      - polarity
      Delay MIN, tail pulse

DAQ continues (R33)

aidas1> ./analyser2 /TapeData/Sept2016/R33_0
 *** GREAT format 3.2.0 analyser - TD - May 2014
 *** ERROR: READ I/O error:       5002
                   blocks:      32000
          ADC data format:  258854184 ( 1075951.6 Hz)
        Other data format:    2233816 (    9285.1 Hz)
 Sample trace data format:          0 (       0.0 Hz)
         Undefined format:          0 (       0.0 Hz)
   Other data format type:      PAUSE:         95 (       0.4 Hz)
                               RESUME:         95 (       0.4 Hz)
                              SYNC100:    2200965 (    9148.5 Hz)
                           FEE64 disc:      26965 (     112.1 Hz)
                             MBS info:       5696 (      23.7 Hz)
                           Other info:          0 (       0.0 Hz)

   ADC data range bit set:        273 (       1.1 Hz)

                Timewarps:        ADC:          0 (       0.0 Hz)
                                PAUSE:          0 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:          0 (       0.0 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:          0 (       0.0 Hz)
                            Undefined:          0 (       0.0 Hz)
                         Sample trace:          0 (       0.0 Hz)

 Timestamp elapsed time:      240.582 s
 FEE module #:   1 elapsed dead time       0.000 s
 FEE module #:   2 elapsed dead time       0.026 s
 FEE module #:   3 elapsed dead time       0.000 s
 FEE module #:   4 elapsed dead time       0.000 s
 FEE module #:   5 elapsed dead time       0.278 s
 FEE module #:   6 elapsed dead time       1.219 s
 FEE module #:   7 elapsed dead time       0.000 s
 FEE module #:   8 elapsed dead time       0.000 s
 FEE module #:   9 elapsed dead time       0.000 s
 FEE module #:  10 elapsed dead time       0.000 s
 FEE module #:  11 elapsed dead time       0.000 s
 FEE module #:  12 elapsed dead time       0.000 s
 FEE module #:  13 elapsed dead time       0.000 s
 FEE module #:  14 elapsed dead time       0.001 s
 FEE module #:  15 elapsed dead time       0.014 s
 FEE module #:  16 elapsed dead time       0.015 s
 FEE module #:  17 elapsed dead time       0.703 s
 FEE module #:  18 elapsed dead time       0.196 s
 FEE module #:  19 elapsed dead time       1.037 s
 FEE module #:  20 elapsed dead time       0.063 s
 FEE module #:  21 elapsed dead time       0.000 s
 FEE module #:  22 elapsed dead time       0.006 s
 FEE module #:  23 elapsed dead time       0.073 s
 FEE module #:  24 elapsed dead time       0.000 s
 FEE module #:  25 elapsed dead time       0.000 s
 FEE module #:  26 elapsed dead time       0.000 s
 FEE module #:  27 elapsed dead time       0.000 s
 FEE module #:  28 elapsed dead time       0.000 s
 FEE module #:  29 elapsed dead time       0.000 s
 FEE module #:  30 elapsed dead time       0.000 s
 FEE module #:  31 elapsed dead time       0.000 s
 FEE module #:  32 elapsed dead time       0.000 s

 *** Program elapsed time:   22.278s ( 1436.394 blocks/s,  89.775 Mb/s)

00.16 DSSSD bias and leakage currents OK - see attachments 1 & 2
      FEE64 temperatures OK - see attachment 3
      Temperature +25.0 deg C, d.p. +8.2 deg C, RH 33.8%

00.25 DAQ stopped
      Unable to restart merger - see attachment 4 - DAQ STOP OK
      System wide tests fail - see attachments 5 & 6
      DAQ RESET & SETUP, ReSYNC OK
      System wide checks fail - as above

      Several FEE64 power cycles and eventually an aidas1 reboot due to
      1) timeouts from random FEE64s following power cycle + 10 mins
      2) data transfer links (X) dropping at DAQ GO 

01.49 DAQ starts (R40)

08.31 DAQ found stopped
      Most recently accessed file R40_91 @ 06.26

      Restart merger & request DAQ STOP
      Transient timeout from nnaida8 but STOP completed OK

11.10 DSSSD bias and leakage currents OK - see attachments 7 & 8
      FEE64 temperatures OK - see attachment 9

      System-wide checks - system clock, SYNC errors and SUNC count fail - see attachments 10-13

      ReSYNC fails - see attachment 14
      
11.30 Analysis of R40 data

      R40_91 ADC data synchronised

aidas1> ./analyser2 /TapeData/Sept2016/R40_91
 *** GREAT format 3.2.0 analyser - TD - May 2014
 *** ERROR: READ I/O error:       5002
                   blocks:      30880
          ADC data format:  249997809 ( 1194334.5 Hz)
        Other data format:    1952111 (    9326.0 Hz)
 Sample trace data format:          0 (       0.0 Hz)
         Undefined format:          0 (       0.0 Hz)
   Other data format type:      PAUSE:         55 (       0.3 Hz)
                               RESUME:         55 (       0.3 Hz)
                              SYNC100:    1915685 (    9152.0 Hz)
                           FEE64 disc:      30044 (     143.5 Hz)
                             MBS info:       6272 (      30.0 Hz)
                           Other info:          0 (       0.0 Hz)

   ADC data range bit set:        303 (       1.4 Hz)

                Timewarps:        ADC:          0 (       0.0 Hz)
                                PAUSE:          0 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:          0 (       0.0 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:          0 (       0.0 Hz)
                            Undefined:          0 (       0.0 Hz)
                         Sample trace:          0 (       0.0 Hz)

 Timestamp elapsed time:      209.320 s
 FEE module #:   1 elapsed dead time       0.000 s
 FEE module #:   2 elapsed dead time       0.000 s
 FEE module #:   3 elapsed dead time       0.000 s
 FEE module #:   4 elapsed dead time       0.000 s
 FEE module #:   5 elapsed dead time       0.167 s
 FEE module #:   6 elapsed dead time       0.529 s
 FEE module #:   7 elapsed dead time       0.000 s
 FEE module #:   8 elapsed dead time       0.000 s
 FEE module #:   9 elapsed dead time       0.000 s
 FEE module #:  10 elapsed dead time       0.000 s
 FEE module #:  11 elapsed dead time       0.000 s
 FEE module #:  12 elapsed dead time       0.000 s
 FEE module #:  13 elapsed dead time       0.000 s
 FEE module #:  14 elapsed dead time       0.002 s
 FEE module #:  15 elapsed dead time       0.018 s
 FEE module #:  16 elapsed dead time       0.043 s
 FEE module #:  17 elapsed dead time       0.098 s
 FEE module #:  18 elapsed dead time       0.035 s
 FEE module #:  19 elapsed dead time       0.378 s
 FEE module #:  20 elapsed dead time       0.010 s
 FEE module #:  21 elapsed dead time       0.000 s
 FEE module #:  22 elapsed dead time       0.000 s
 FEE module #:  23 elapsed dead time       0.046 s
 FEE module #:  24 elapsed dead time       0.000 s
 FEE module #:  25 elapsed dead time       0.000 s
 FEE module #:  26 elapsed dead time       0.000 s
 FEE module #:  27 elapsed dead time       0.000 s
 FEE module #:  28 elapsed dead time       0.000 s
 FEE module #:  29 elapsed dead time       0.000 s
 FEE module #:  30 elapsed dead time       0.000 s
 FEE module #:  31 elapsed dead time       0.000 s
 FEE module #:  32 elapsed dead time       0.000 s

 *** Program elapsed time:    6.613s ( 4669.392 blocks/s, 291.837 Mb/s)


21.52 DAQ starts (Sept2016/R41)  BRIKEN filename 252CfwithAida75mm

      Correlation scaler reset requested by BRIKEN DAQ following start of run to disk

      End of AIDA snout positioned at edge of (CH2)n moderator aperture for
      ORNL HPGe clover detectors

      RIKEN 252Cf source ~25k neutrons/s

      FEE64 temperatures OK
      *All* system-wide checks OK

      B2F/F11 temperature + deg C, d.p. + deg C,RH %

      ASIC settings 2016Sep02-18.17.01
      slow comparator 0x10 -> 0x0a
      LEC/MEC fast comparators # 1, 55 & 58 disabled for all FEE64s

      Master MACB serial #13 (see https://elog.ph.ed.ac.uk/AIDA/367)
      Master MACB mode 5
      TS_SYNC_PHASE 0

22.17 DAQ stops OK
      BRIKEN DAQ stopped at about same time

22.45 DAQ starts (R42)

      AIDA withdrawn from (CH2)n moderator, 252Cf source removed

10.55 DAQ continues OK 
      Current file May2017/R6_181

      ASIC settings 2017Mar27-16.12.39 *except*
      slow comparator 0x20
      LEC/MEC fast comparator 0xff

      Detector biases & leakage currents OK - see attachments 1 & 2
      FEE64 Temperatures OK - see attachment 3
      'good events' statistics OK - see attachment 4
      No warnings/errors reported by merger since DAQ GO - see attachment 5

      *all* system wide checks OK

      All histograms zero'd

11.04 Analysis of data file R6_180 - see attachment 6
      Zero AIDA pause/resume
      Zero idle-time
      AIDA SYNC100 variations ~22/499760

11.20 E17/F11 ambient temperature +24.6 deg C, RH 39.6%, d.p. +9.9 deg C
      Julabo FL11006 set point +20 deg C, actual +20.0 deg C, water level OK (c. 65%)

10.00 DAQ continues (R33) in no storage mode

      DSSSD bias & leakage currents OK - see attachments 1 & 2
      FEE64 temperatures OK - see attachment 3
      System wide checks OK *except* clock status - see attachments 4 & 5
      Merge statistics - see attachments 6-11

10.21 Ambient temperature +24.9 deg C, d.p. +1.4 deg C, RH 21.5% 

      

11.00 - chiller refilled and switched on
        set to 20 degrees celsius

        AIDA stand moved and secured in F11
        clock connected -> 50 MHz square wave at 5.0 V

        AIDA connected to power box
        network cables attached
        NIM modules switched on

        Raspberry Pis switched on
        Detector bias switched on for detectors 1-6
     
        
        
13.58 - AIDA power relay 1,2,5,6 sequenced on
       bias/leakage currents ok (attachment 1)
       DAQ setup
       System wide checks ok, except naida07 fails ADC calibration, and clock status check returns error in
nnaida06, 5.

       Above issue with clock no longer present

       FEE temperatures ok (attachment 2)
       good event statistics (attachment 3)

16.30 1.8.W spectra n+n strips
      FSR 20us, 200us, 2ms, 20ms attachments 4-7

      1.8.W spectra p+n strips
      FSR 20us, 200us, 2ms, 20ms attachments 8-11
       
      compare with https://elog.ph.ed.ac.uk/AIDA/746

      naida19, naida20, naida23 show low frequency noise structure, worse than in above elog entry

17.22 1.8.L spectra p+n strips

      peaks:

      aida01 72
      aida02 112
      aida03 80
      aida04 78
      aida05 74
      aida06 95
      aida07 78
      aida08 71
      aida09 82
      aida10 85
      aida11 93
      aida12 96
      aida13 101
      aida14 70
      aida15 87
      aida16 87
      aida17 69
      aida18 83
      aida19 127
      aida20 116
      aida21 84
      aida22 74
      aida23 108
      aida24 117
      

      

10.50 SyncCheck ... EURICA server refusing connection

[aida@d05 euricatssender]$ ./euricatssender -t
euricatssender
Setting Transfer Block Size 65536
TCP transfer library version 4.0T
0: TCP socket send buffer was 16384 - now 425984
0: TCP socket receive buffer was 87380 - now 425984
0: TCP socket created OK - now connecting to 10.32.0.12 port 10305
0: Connected to 10.32.0.12 port 10305
E-f_evcli: Error connecting node:e01, port:6003. Msg:
-I- f_stc connection refused.
Can't connect Event Server = e01

11.08 Run AIDA9
      Stat/rate hit patterns - see attachments 1-3 

11.12 B3F ambient temp +25.0 deg C, d.p. +12.2 deg C, RH 44.2%

12.10 Attachments 4-9 - Right HEC 1*H spectra for DSSSD #1 - #6

12.17 Attachments 10-13 - nnaida24 (DSSSD #2) ASIC # 1-4 *H spectra

12.49 System wide checks OK 
      Some failures of FADC calibration - OK - readout is currently disabled
      nnaida19 fails SYNC errors not counting & ASIC clock timestamp

13.40 Run AIDA10

14.46 AIDA10_14 deadtime check

 *** GREAT format 3.2.0 analyser - TD - May 2014
 ***            ADC data: block: 30030 ptr: 12945 data: 0xC4A87DD1 module: 18 fail: 0 range: 0 id:  1192 
channel: 40 adc: 32209  ts: 0x0F0AC00000A0 OLD: ts: 0x0F0AD0000039 tag: 3 type: 0
 *** ERROR: READ I/O error:       5002
                   blocks:      32000
          ADC data format:  204745037 (  910555.1 Hz)
        Other data format:   56374963 (  250714.3 Hz)
 Sample trace data format:          0 (       0.0 Hz)
         Undefined format:          0 (       0.0 Hz)
   Other data format type:      PAUSE:        332 (       1.5 Hz)
                               RESUME:        332 (       1.5 Hz)
                              SYNC100:      85777 (     381.5 Hz)
                           FEE64 disc:   56286977 (  250323.0 Hz)
                             MBS info:       1545 (       6.9 Hz)
                           Other info:          0 (       0.0 Hz)

   ADC data range bit set:     177071 (     787.5 Hz)

                Timewarps:        ADC:          1 (       0.0 Hz)
                                PAUSE:          0 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:          0 (       0.0 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:          0 (       0.0 Hz)
                            Undefined:          0 (       0.0 Hz)
                         Sample trace:          0 (       0.0 Hz)

 Timestamp elapsed time:      224.857 s
 FEE module #:   1 elapsed dead time       0.010 s
 FEE module #:   2 elapsed dead time       0.209 s
 FEE module #:   3 elapsed dead time       0.001 s
 FEE module #:   4 elapsed dead time       0.000 s
 FEE module #:   5 elapsed dead time       0.120 s
 FEE module #:   6 elapsed dead time       0.362 s
 FEE module #:   7 elapsed dead time       0.000 s
 FEE module #:   8 elapsed dead time       0.000 s
 FEE module #:   9 elapsed dead time       0.019 s
 FEE module #:  10 elapsed dead time       0.011 s
 FEE module #:  11 elapsed dead time       0.000 s
 FEE module #:  12 elapsed dead time       0.005 s
 FEE module #:  13 elapsed dead time       0.000 s
 FEE module #:  14 elapsed dead time       0.000 s
 FEE module #:  15 elapsed dead time       0.059 s
 FEE module #:  16 elapsed dead time       0.304 s
 FEE module #:  17 elapsed dead time       0.000 s
 FEE module #:  18 elapsed dead time       0.010 s
 FEE module #:  19 elapsed dead time       0.000 s
 FEE module #:  20 elapsed dead time       0.029 s
 FEE module #:  21 elapsed dead time       0.035 s
 FEE module #:  22 elapsed dead time       0.006 s
 FEE module #:  23 elapsed dead time       0.052 s
 FEE module #:  24 elapsed dead time       0.044 s
 FEE module #:  25 elapsed dead time       0.000 s
 FEE module #:  26 elapsed dead time       0.000 s
 FEE module #:  27 elapsed dead time       0.000 s
 FEE module #:  28 elapsed dead time       0.000 s
 FEE module #:  29 elapsed dead time       0.000 s
 FEE module #:  30 elapsed dead time       0.000 s
 FEE module #:  31 elapsed dead time       0.000 s
 FEE module #:  32 elapsed dead time       0.000 s

 *** Program elapsed time:   68.563s (  466.727 blocks/s,  29.170 Mb/s)

21.08 Detector bias and leakage current screenshots - attachments 14-15

21.36 FEE64 temperatures - attachment 16
      Julabo FL11006 set point +17 deg C (B3F temperature +25.0 deg C, d.p. +11.5 deg C, RH 42.4%)
         

      diode link threshold 0xca -> 0xb6
                           0x11 -> 0x2c 

00.20 R31
      1mm + 1.4mm Al degrader
00.42 R31 ends

      R32
      -1mm Al degrader
00.54 R32 ends

Data indicates changes to diode link threshold is 'switching off' decay p+n strip events,
n+n strips appear to be less affected

01.00 R33
      diode link threshold 0xca -> 0xbf
                           0x11 -> 0x23 

      ASIC parameters saved to /MIDAS/DB/EXPERIMENTS/AIDA/2015Apr30-diode-mod2

Ignore R34 & R35

09.40 Fast LEC/MEC discriminators disabled:
     Mask bits 31->0, 63->32 *not* 0->31 and 32->63 as implied by Disc Controls window

         Chan                 Mask
                             31->0      63->32
nnaida1                        0x0   0x4000000
nnaida2  ch 22, 26       0x4400001   0x8000000
nnaida6  ch  1                 0x2         0x0
nnaida9  ch 45 "               0x0      0x2000
nnaida10 ch 31 "        0x82000000  0x80000000
nnaida11 ch 55 "               0x0    0x800000
nnaida13                       0x0   0x4000000
nnaida16 ch 55 "               0x0    0x800000

Local Controls -> Timing O/P Control Register 0xc -> 0xe (OR64) https://elog.ph.ed.ac.uk/AIDA/79

09.48 R36
      BigRIPS 110Zr setting
      Degrader ?
10.43 R36 ends
      Beam off for ion source re-tune

10.50 nnaida6 good events c. 250k, c. 20Hz PAUSE/RESUME
      change slow comparator 0x14 -> 0x20
      rate/channel ~5kHz -> ~1kHz

      ASIC parameters saved to /MIDAS/DB/EXPERIMENTS/AIDA/2015Apr30-diode-mod2

12.00 VFEP updates 
      DataAcquisitionControl.tcl and sys.tcl

      Fast comparator masks are now saved
      MACB trigger mode default 0xe (OR64) and can be saved as a setting

      Stop/Go logic altered so that electronics data generation is the last to be GOED and first to be STOPPED 

12.10 Beam back

12.13 R38

20/02/15

1400: Measured leakage current as function of bias V for the Caen 1419B high voltage supply and detector BB18 2977-20. (see attachment #1)

1500: Modified MACB installed and connected to BELEN and RIBF DAQs (see attachment #2 for signal flow diagram).

          Increased noise in several nnaida, possibly due to grounding issues introduced by inclusion of modules for other DAQs.

The problem :-

The DAQ stops working and can only be fixed by a reboot of the FEE64s.

The Candidate:-

The DAQ program running in the FEE64s which handles the transfer of the data items from the FPGA to the Merger, AidaExecV8, uses a device driver, aidamem, to copy the data from DMA memory into Linux memory.

Sometimes AidaExecV8 can be killed by the Kernal due to a "page allocation failure" which occurs when the Kernal memory space has become fragmented. So when a block of contiguous memory is requested to receive the copy of the data from the FPGA DMA memory the Kernal can't allocate memory and kills the process.

The effect of this would be seen at the Merger where it would be waiting for data from the FEE.

Since the AidaExecV8 has been killed then there will be no response to status requests.

A recent change to the operation of the Aida system , flushing buffers regularly, is the most likely cuplrit. The flush of the buffers on a slow FEE will use small memory blocks while faster FEE data will always require full size buffers and mean that the Kernal memory will have large contiguous areas of memory in constant use. This will explain why this failure type is relatively recent .

A solution :- Change the request for memory copy to always use the maximum size. This way there shouldn't be the fragmentation of the memory. Also the fragmentation of memory will be investigated to see if there is a away to monitor it.

Some corroboration:-

The error messages from the Kernal that indicate this has fault has occured can be read from the FEE64 root file system. At the text file /var/log/messages. A grep of these in RIKEN using the phrase "page allocation failure" shows they have occured. It remains to see if the date and times of the failures align with the date and times of the system failure.

All comments and ideas gratefuly received.

###### # ##  # # # # #   Tested this solution on T9 system and it doesn't work ! The failure still occurs when the DAQ is set to write all input to disc.

Logged into aidas1 and opened the nnrpi1 relay control window ... powered up the FEE64s.

seperatly connected to ribfdaq and opened a firefox to aidas1:8015.

By the time the firefox had appeared on my desktop the FEE64s had started up.

Ran through the standard startup. Set thresholds to 255 for all 3 sets of comparators. Disabled all discriminators.

Enable histograms.

Started DAQ.

Using ELOG 74 as a guide logged into nnrpi1 and ran two putty sessions ( USB2 and USB3 ).

Based on the instructions in the HV unit manual attached to ELOG75 powered ion all 6 detectors ( current when settled at 100v = 3 to 5uA )

Using spectrum broswer at nnaida6 and ASIC4 to control the slow comparator threshold ..... needed to reach 32 before the pulser became visible.

Using nnaida6:1.3.L found the pulser peak at channel 32136 with a peak width of 67.73 channels.

Then poweroff the HV on all 6 detectors.

Power off the FEE64s.

Logout of all RIKEN machines and consider the best way to take advantage of the situation which allows the system to be operated completely remotely.

nnaida2 running Version 8 Firmware and data acquisition software outputting both and ASIC data stream and a Waveform data stream to the Merge process.

Merged data is written to file ... attached.

The waveform is running at 100Hz and ASIC data is running at 10Hz.

 S O R T    S T A R T I N G .....



S O R T    S T A R T E D ..... Wed Feb 24 16:11:54 2016

  *** AIDA @ RIKEN - May 2015
  *** Entry init commences
  *** READ I/O error:        5010
   gain(   0-   7) 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
   gain(   8-  15) 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
   gain(  16-  23) 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
   gain(  24-  31) 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
   gain(  32-  39) 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
   gain(  40-  47) 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
   gain(  48-  55) 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
   gain(  56-  63) 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
   gain(  64-  71) 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
   gain(  72-  79) 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
   gain(  80-  87) 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
   gain(  88-  95) 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
   gain(  96- 103) 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
   gain( 104- 111) 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
   gain( 112- 119) 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
   gain( 120- 127) 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000
 offset(   0-   7)    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00
 offset(   8-  15)    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00
 offset(  16-  23)    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00
 offset(  24-  31)    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00
 offset(  32-  39)    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00
 offset(  40-  47)    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00
 offset(  48-  55)    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00
 offset(  56-  63)    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00
 offset(  64-  71)    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00
 offset(  72-  79)    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00
 offset(  80-  87)    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00
 offset(  88-  95)    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00
 offset(  96- 103)    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00
 offset( 104- 111)    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00
 offset( 112- 119)    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00
 offset( 120- 127)    0.00    0.00    0.00    0.00    0.00    0.00    0.00    0.00
      c(   0-   7)   1.000   1.000   1.000   1.000   1.000   1.000   1.000   0.000
      w(   0-   7)       0       0       0       0       0       0       0       0
      w(   8-  15)       0       0       0       0       0       0       0       0
      w(  16-  23)       0       0       0       0       0       0       0       0
      w(  24-  31)       0       0       0       0       0       0       0       0
  *** Entry init ends
plugin_loader: loaded /MIDAS_local/MIDASsort/Linux/lib64/great_fmt.so at 0x7fb5105de8cc

plugin_loader: loaded /MIDAS_local/MIDASsort/Linux/lib64/disc.so at 0x7fb5103a3dc7


Get_Block: Opened file -> /homes/npg/Downloads/R1_0.gz for read

 008F 0000 0000 0001 FFFF 0A04 11AE
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C0A0411AE ch: 2127
 008F 0000 0000 0001 FFFF 0A9C 875E
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C0A9C875E ch: 2127
 008F 0000 0000 0001 FFFF 0D96 D496
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C0D96D496 ch: 2127
 008F 0000 0000 0001 FFFF 0E2F 4A46
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C0E2F4A46 ch: 2127
 008F 0000 0000 0001 FFFF 0F60 366E
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C0F60366E ch: 2127
 008F 0000 0000 0001 FFFF 0091 2296
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C10912296 ch: 2127
 008F 0000 0000 0001 FFFF 01C2 0EBE
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C11C20EBE ch: 2127
 008F 0000 0000 0001 FFFF 025A 8536
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C125A8536 ch: 2127
 008F 0000 0000 0001 FFFF 02F2 FBAE
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C12F2FBAE ch: 2127
 008F 0000 0000 0001 FFFF 084F 25E6
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C184F25E6 ch: 2127
 008F 0000 0000 0001 FFFF 0980 139E
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C1980139E ch: 2127
 008F 0000 0000 0001 FFFF 0AB1 0156
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C1AB10156 ch: 2127
 008F 0000 0000 0001 FFFF 0EDC 4416
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C1EDC4416 ch: 2127
 008F 0000 0000 0001 FFFF 0F74 BB56
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C1F74BB56 ch: 2127
 008F 0000 0000 0001 FFFF 00A5 AA9E
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C20A5AA9E ch: 2127
 008F 0000 0000 0001 FFFF 01D6 99E6
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C21D699E6 ch: 2127
 008F 0000 0000 0001 FFFF 0307 892E
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C2307892E ch: 2127
 008F 0000 0000 0001 FFFF 0438 7876
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C24387876 ch: 2127
 008F 0000 0000 0001 FFFF 0569 67BE
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C256967BE ch: 2127
 008F 0000 0000 0001 FFFF 069A 5706
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C269A5706 ch: 2127
 008F 0000 0000 0001 FFFF 0732 CF0E
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C2732CF0E ch: 2127
 008F 0000 0000 0001 FFFF 0863 BE56
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C2863BE56 ch: 2127
 008F 0000 0000 0001 FFFF 08FC 365E
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C28FC365E ch: 2127
 008F 0000 0000 0001 FFFF 0B5E 15B6
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C2B5E15B6 ch: 2127
 008F 0000 0000 0001 FFFF 0C8F 05C6
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C2C8F05C6 ch: 2127
 008F 0000 0000 0001 FFFF 0D27 7DCE
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C2D277DCE ch: 2127
 008F 0000 0000 0001 FFFF 0DBF F5D6
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C2DBFF5D6 ch: 2127
 008F 0000 0000 0001 FFFF 0E58 6DDE
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C2E586DDE ch: 2127
 008F 0000 0000 0001 FFFF 0EF0 E5E6
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C2EF0E5E6 ch: 2127
 008F 0000 0000 0001 FFFF 0F89 5DEE
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C2F895DEE ch: 2127
 008F 0000 0000 0001 FFFF 0021 D5F6
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C3021D5F6 ch: 2127
 008F 0000 0000 0001 FFFF 00BA 4DFE
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C30BA4DFE ch: 2127
 008F 0000 0000 0001 FFFF 0152 C606
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C3152C606 ch: 2127
 008F 0000 0000 0001 FFFF 01EB 3E0E
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C31EB3E0E ch: 2127
 008F 0000 0000 0001 FFFF 0616 857E
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C3616857E ch: 2127
 008F 0000 0000 0001 FFFF 06AE FD86
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C36AEFD86 ch: 2127
 008F 0000 0000 0001 FFFF 0747 758E
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C3747758E ch: 2127
 008F 0000 0000 0001 FFFF 09A9 54E6
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C39A954E6 ch: 2127
 008F 0000 0000 0001 FFFF 0A41 CCEE
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C3A41CCEE ch: 2127
 008F 0000 0000 0001 FFFF 0B72 BC36
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C3B72BC36 ch: 2127
 008F 0000 0000 0001 FFFF 0D3C 2386
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C3D3C2386 ch: 2127
 008F 0000 0000 0001 FFFF 0E6D 12CE
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C3E6D12CE ch: 2127
 008F 0000 0000 0001 FFFF 0F9E 0216
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C3F9E0216 ch: 2127
 008F 0000 0000 0001 FFFF 00CE F15E
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C40CEF15E ch: 2127
 008F 0000 0000 0001 FFFF 0298 57E6
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C429857E6 ch: 2127
 008F 0000 0000 0001 FFFF 0330 CF26
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C4330CF26 ch: 2127
 008F 0000 0000 0001 FFFF 04FA 35AE
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C44FA35AE ch: 2127
 008F 0000 0000 0001 FFFF 062B 242E
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C462B242E ch: 2127
 008F 0000 0000 0001 FFFF 06C3 9B6E
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C46C39B6E ch: 2127
 008F 0000 0000 0001 FFFF 09BD EEE6
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C49BDEEE6 ch: 2127
 008F 0000 0000 0001 FFFF 0A56 6626
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C4A566626 ch: 2127
 008F 0000 0000 0001 FFFF 0B87 53DE
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C4B8753DE ch: 2127
 008F 0000 0000 0001 FFFF 0C1F CB1E
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C4C1FCB1E ch: 2127
 008F 0000 0000 0001 FFFF 0D50 B8D6
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C4D50B8D6 ch: 2127
 008F 0000 0000 0001 FFFF 0E81 A68E
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C4E81A68E ch: 2127
 008F 0000 0000 0001 FFFF 0F1A 1D06
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C4F1A1D06 ch: 2127
 008F 0000 0000 0001 FFFF 004B 0ABE
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C504B0ABE ch: 2127
 008F 0000 0000 0001 FFFF 00E3 8136
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C50E38136 ch: 2127
 008F 0000 0000 0001 FFFF 017B F7AE
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C517BF7AE ch: 2127
 008F 0000 0000 0001 FFFF 050E BE7E
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C550EBE7E ch: 2127
 008F 0000 0000 0001 FFFF 05A7 34F6
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C55A734F6 ch: 2127
 008F 0000 0000 0001 FFFF 06D8 211E
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C56D8211E ch: 2127
 008F 0000 0000 0001 FFFF 0770 9796
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C57709796 ch: 2127
 008F 0000 0000 0001 FFFF 0939 F96E
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C5939F96E ch: 2127
 008F 0000 0000 0001 FFFF 0B03 5B46
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C5B035B46 ch: 2127
 008F 0000 0000 0001 FFFF 0B9B D0F6
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C5B9BD0F6 ch: 2127
 008F 0000 0000 0001 FFFF 0C34 46A6
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C5C3446A6 ch: 2127
 008F 0000 0000 0001 FFFF 0CCC BC56
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C5CCCBC56 ch: 2127
 008F 0000 0000 0001 FFFF 0D65 3206
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C5D653206 ch: 2127
 008F 0000 0000 0001 FFFF 0DFD A7B6
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C5DFDA7B6 ch: 2127
 008F 0000 0000 0001 FFFF 005F 7DAE
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C605F7DAE ch: 2127
 008F 0000 0000 0001 FFFF 0190 6846
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C61906846 ch: 2127
 008F 0000 0000 0001 FFFF 0228 DD2E
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C6228DD2E ch: 2127
 008F 0000 0000 0001 FFFF 0523 267E
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C6523267E ch: 2127
 008F 0000 0000 0001 FFFF 05BB 9B66
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C65BB9B66 ch: 2127
 008F 0000 0000 0001 FFFF 081D 6E3E
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C681D6E3E ch: 2127
 008F 0000 0000 0001 FFFF 08B5 E326
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C68B5E326 ch: 2127
 008F 0000 0000 0001 FFFF 094E 580E
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C694E580E ch: 2127
 008F 0000 0000 0001 FFFF 0A7F 4116
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C6A7F4116 ch: 2127
 008F 0000 0000 0001 FFFF 0BB0 2A1E
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C6BB02A1E ch: 2127
 008F 0000 0000 0001 FFFF 0D79 8746
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C6D798746 ch: 2127
 008F 0000 0000 0001 FFFF 0E11 FB66
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C6E11FB66 ch: 2127
 008F 0000 0000 0001 FFFF 0EAA 6F86
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C6EAA6F86 ch: 2127
 008F 0000 0000 0001 FFFF 036E 1086
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C736E1086 ch: 2127
 008F 0000 0000 0001 FFFF 0537 6C1E
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C75376C1E ch: 2127
 008F 0000 0000 0001 FFFF 05CF E03E
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C75CFE03E ch: 2127
 008F 0000 0000 0001 FFFF 0700 C7B6
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C7700C7B6 ch: 2127
 008F 0000 0000 0001 FFFF 08CA 234E
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C78CA234E ch: 2127
 008F 0000 0000 0001 FFFF 09FB 0AC6
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C79FB0AC6 ch: 2127
 008F 0000 0000 0001 FFFF 0B2B F23E
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C7B2BF23E ch: 2127
 008F 0000 0000 0001 FFFF 0C5C D9B6
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C7C5CD9B6 ch: 2127
 008F 0000 0000 0001 FFFF 0D8D C12E
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C7D8DC12E ch: 2127
 008F 0000 0000 0001 FFFF 0EBE A8A6
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C7EBEA8A6 ch: 2127
 008F 0000 0000 0001 FFFF 0FEF 901E
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C7FEF901E ch: 2127
 008F 0000 0000 0001 FFFF 0120 7796
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C81207796 ch: 2127
 008F 0000 0000 0001 FFFF 02E9 D32E
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C82E9D32E ch: 2127
 008F 0000 0000 0001 FFFF 0382 474E
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C8382474E ch: 2127
 008F 0000 0000 0001 FFFF 041A BB6E
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C841ABB6E ch: 2127
 008F 0000 0000 0001 FFFF 05E4 17CE
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C85E417CE ch: 2127
 008F 0000 0000 0001 FFFF 067C 8BEE
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C867C8BEE ch: 2127
 008F 0000 0000 0001 FFFF 0715 000E
 module:   2 channel:  15 range: 1 adc_data:FFFF ts:000C8715000E ch: 2127
  *** items:      114240  events:       10590  count:       21180
  *** time warp:           0  count2:           0
  *** elapsed time:  9.76562500E-02  s items:   1169817.6      events:   108441.60      count:   216883.20    
  *** Entry finish



S O R T    C O M P L E T E ..... Wed Feb 24 16:11:54 2016

  *** items:      114240  events:       10590  count:       21180
  *** time warp:           0  count2:           0
  *** elapsed time:  9.76562500E-02  s items:   1169817.6      events:   108441.60      count:   216883.20    
  *** Entry finish

S O R T    S T O P P E D ..... Wed Feb 24 16:11:54 2016

My GREAT data analyser program reports that the 'fail' bit is set ?

$ ./a.exe v R1_0 | more
 *** GREAT format 3.2.0 analyser - TD - May 2014
  verbose
 ***   SYNC100 timestamp: block:     1 ptr:     7 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C088000A0
 ***   SYNC100 timestamp: block:     1 ptr:     9 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C088400A0
 ***   SYNC100 timestamp: block:     1 ptr:    11 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C088800A0
 ***   SYNC100 timestamp: block:     1 ptr:    13 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C088C00A0
 ***   SYNC100 timestamp: block:     1 ptr:    15 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C089000A0
 ***   SYNC100 timestamp: block:     1 ptr:    17 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C089400A0
 ***   SYNC100 timestamp: block:     1 ptr:    19 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C089800A0
 ***   SYNC100 timestamp: block:     1 ptr:    21 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C089C00A0
 ***   SYNC100 timestamp: block:     1 ptr:    23 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C08A000A0
 ***   SYNC100 timestamp: block:     1 ptr:    25 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C08A400A0
 ***   SYNC100 timestamp: block:     1 ptr:    27 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C08A800A0
 ***   SYNC100 timestamp: block:     1 ptr:    29 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C08AC00A0
 ***   SYNC100 timestamp: block:     1 ptr:    31 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C08B000A0
 ***   SYNC100 timestamp: block:     1 ptr:    33 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C08B400A0
 ***   SYNC100 timestamp: block:     1 ptr:    35 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C08B800A0
 ***   SYNC100 timestamp: block:     1 ptr:    37 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C08BC00A0
 ***   SYNC100 timestamp: block:     1 ptr:    39 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C08C000A0
 ***   SYNC100 timestamp: block:     1 ptr:    41 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C08C400A0
 ***   SYNC100 timestamp: block:     1 ptr:    43 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C08C800A0
 ***   SYNC100 timestamp: block:     1 ptr:    45 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C08CC00A0
 ***   SYNC100 timestamp: block:     1 ptr:    47 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C08D000A0
 ***   SYNC100 timestamp: block:     1 ptr:    49 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C08D400A0
 ***   SYNC100 timestamp: block:     1 ptr:    51 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C08D800A0
 ***   SYNC100 timestamp: block:     1 ptr:    53 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C08DC00A0
 ***   SYNC100 timestamp: block:     1 ptr:    55 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C08E000A0
 ***   SYNC100 timestamp: block:     1 ptr:    57 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C08E400A0
 ***   SYNC100 timestamp: block:     1 ptr:    59 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C08E800A0
 ***   SYNC100 timestamp: block:     1 ptr:    61 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C08EC00A0
 ***   SYNC100 timestamp: block:     1 ptr:    63 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C08F000A0
 ***   SYNC100 timestamp: block:     1 ptr:    65 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C08F400A0
 ***   SYNC100 timestamp: block:     1 ptr:    67 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C08F800A0
 ***   SYNC100 timestamp: block:     1 ptr:    69 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C08FC00A0
 ***   SYNC100 timestamp: block:     1 ptr:    71 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C090000A0
 ***   SYNC100 timestamp: block:     1 ptr:    73 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C090400A0
 ***   SYNC100 timestamp: block:     1 ptr:    75 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C090800A0
 ***   SYNC100 timestamp: block:     1 ptr:    77 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C090C00A0
 ***   SYNC100 timestamp: block:     1 ptr:    79 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C091000A0
 ***   SYNC100 timestamp: block:     1 ptr:    81 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C091400A0
 ***   SYNC100 timestamp: block:     1 ptr:    83 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C091800A0
 ***   SYNC100 timestamp: block:     1 ptr:    85 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C091C00A0
 ***   SYNC100 timestamp: block:     1 ptr:    87 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C092000A0
 ***   SYNC100 timestamp: block:     1 ptr:    89 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C092400A0
 ***   SYNC100 timestamp: block:     1 ptr:    91 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C092800A0
 ***   SYNC100 timestamp: block:     1 ptr:    93 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C092C00A0
 ***   SYNC100 timestamp: block:     1 ptr:    95 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C093000A0
 ***   SYNC100 timestamp: block:     1 ptr:    97 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C093400A0
 ***   SYNC100 timestamp: block:     1 ptr:    99 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C093800A0
 ***   SYNC100 timestamp: block:     1 ptr:   101 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C093C00A0
 ***   SYNC100 timestamp: block:     1 ptr:   103 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C094000A0
 ***   SYNC100 timestamp: block:     1 ptr:   105 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C094400A0
 ***   SYNC100 timestamp: block:     1 ptr:   107 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C094800A0
 ***   SYNC100 timestamp: block:     1 ptr:   109 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C094C00A0
 ***   SYNC100 timestamp: block:     1 ptr:   111 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C095000A0
 ***   SYNC100 timestamp: block:     1 ptr:   113 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C095400A0
 ***   SYNC100 timestamp: block:     1 ptr:   115 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C095800A0
 ***   SYNC100 timestamp: block:     1 ptr:   117 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C095C00A0
 ***   SYNC100 timestamp: block:     1 ptr:   119 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C096000A0
 ***   SYNC100 timestamp: block:     1 ptr:   121 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C096400A0
 ***   SYNC100 timestamp: block:     1 ptr:   123 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C096800A0
 ***   SYNC100 timestamp: block:     1 ptr:   125 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C096C00A0
 ***   SYNC100 timestamp: block:     1 ptr:   127 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C097000A0
 ***   SYNC100 timestamp: block:     1 ptr:   129 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C097400A0
 ***   SYNC100 timestamp: block:     1 ptr:   131 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C097800A0
 ***   SYNC100 timestamp: block:     1 ptr:   133 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C097C00A0
 ***   SYNC100 timestamp: block:     1 ptr:   135 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C098000A0
 ***   SYNC100 timestamp: block:     1 ptr:   137 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C098400A0
 ***   SYNC100 timestamp: block:     1 ptr:   139 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C098800A0
 ***   SYNC100 timestamp: block:     1 ptr:   141 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C098C00A0
 ***   SYNC100 timestamp: block:     1 ptr:   143 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C099000A0
 ***   SYNC100 timestamp: block:     1 ptr:   145 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C099400A0
 ***   SYNC100 timestamp: block:     1 ptr:   147 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C099800A0
 ***   SYNC100 timestamp: block:     1 ptr:   149 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C099C00A0
 ***   SYNC100 timestamp: block:     1 ptr:   151 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C09A000A0
 ***   SYNC100 timestamp: block:     1 ptr:   153 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C09A400A0
 ***   SYNC100 timestamp: block:     1 ptr:   155 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C09A800A0
 ***   SYNC100 timestamp: block:     1 ptr:   157 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C09AC00A0
 ***   SYNC100 timestamp: block:     1 ptr:   159 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C09B000A0
 ***   SYNC100 timestamp: block:     1 ptr:   161 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C09B400A0
 ***   SYNC100 timestamp: block:     1 ptr:   163 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C09B800A0
 ***   SYNC100 timestamp: block:     1 ptr:   165 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C09BC00A0
 ***   SYNC100 timestamp: block:     1 ptr:   167 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C09C000A0
 ***   SYNC100 timestamp: block:     1 ptr:   169 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C09C400A0
 ***   SYNC100 timestamp: block:     1 ptr:   171 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C09C800A0
 ***   SYNC100 timestamp: block:     1 ptr:   173 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C09CC00A0
 ***   SYNC100 timestamp: block:     1 ptr:   175 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C09D000A0
 ***   SYNC100 timestamp: block:     1 ptr:   177 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C09D400A0
 ***   SYNC100 timestamp: block:     1 ptr:   179 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C09D800A0
 ***   SYNC100 timestamp: block:     1 ptr:   181 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C09DC00A0
 ***   SYNC100 timestamp: block:     1 ptr:   183 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C09E000A0
 ***   SYNC100 timestamp: block:     1 ptr:   185 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C09E400A0
 ***   SYNC100 timestamp: block:     1 ptr:   187 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C09E800A0
 ***   SYNC100 timestamp: block:     1 ptr:   189 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C09EC00A0
 ***   SYNC100 timestamp: block:     1 ptr:   191 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C09F000A0
 ***   SYNC100 timestamp: block:     1 ptr:   193 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C09F400A0
 ***   SYNC100 timestamp: block:     1 ptr:   195 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C09F800A0
 ***   SYNC100 timestamp: block:     1 ptr:   197 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C09FC00A0
 ***   SYNC100 timestamp: block:     1 ptr:   199 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C0A0000A0
 ***   SYNC100 timestamp: block:     1 ptr:   201 data: 0x824000C0 module:  2 information type:  4 information 
field: 0x000000C0 ts: 0x000C0A0400A0
 ***            ADC data: block:     1 ptr:   203 data: 0xD08FFFFF module:  2 fail: 0 range: 1 id:   143 
channel: 15 adc: 65535  ts: 0x000C0A0411AE

 ***   SYNC100 timestamp: block:     3 ptr:  1071 data: 0x824000D0 module:  2 information type:  4 information 
field: 0x000000D0 ts: 0x000D0C7000A0
 *** ERROR: fail bit set: block:     3 ptr:  1073 data:E09F0730
 ***            ADC data: block:     3 ptr:  1073 data: 0xE09F0730 module:  2 fail: 1 range: 0 id:   159 
channel: 31 adc:  1840  ts: 0x000D0C71DC7E
 ***   SYNC100 timestamp: block:     3 ptr:  1075 data: 0x824000D0 module:  2 information type:  4 information 
field: 0x000000D0 ts: 0x000D0C7400A0
 ***   SYNC100 timestamp: block:     3 ptr:  1077 data: 0x824000D0 module:  2 information type:  4 information 
field: 0x000000D0 ts: 0x000D0C7800A0
 ***   SYNC100 timestamp: block:     3 ptr:  1079 data: 0x824000D0 module:  2 information type:  4 information 
field: 0x000000D0 ts: 0x000D0C7C00A0
 ***   SYNC100 timestamp: block:     3 ptr:  1081 data: 0x824000D0 module:  2 information type:  4 information 
field: 0x000000D0 ts: 0x000D0C8000A0
 *** ERROR: fail bit set: block:     3 ptr:  1083 data:E09F0678
 ***            ADC data: block:     3 ptr:  1083 data: 0xE09F0678 module:  2 fail: 1 range: 0 id:   159 
channel: 31 adc:  1656  ts: 0x000D0C811EB8
 ***   SYNC100 timestamp: block:     3 ptr:  1085 data: 0x824000D0 module:  2 information type:  4 information 
field: 0x000000D0 ts: 0x000D0C8400A0
 ***   SYNC100 timestamp: block:     3 ptr:  1087 data: 0x824000D0 module:  2 information type:  4 information 
field: 0x000000D0 ts: 0x000D0C8800A0
 ***   SYNC100 timestamp: block:     3 ptr:  1089 data: 0x824000D0 module:  2 information type:  4 information 
field: 0x000000D0 ts: 0x000D0C8C00A0
 ***   SYNC100 timestamp: block:     3 ptr:  1091 data: 0x824000D0 module:  2 information type:  4 information 
field: 0x000000D0 ts: 0x000D0C9000A0
 *** ERROR: fail bit set: block:     3 ptr:  1093 data:E09F06BC
 ***            ADC data: block:     3 ptr:  1093 data: 0xE09F06BC module:  2 fail: 1 range: 0 id:   159 
channel: 31 adc:  1724  ts: 0x000D0C9060F0
 ***   SYNC100 timestamp: block:     3 ptr:  1095 data: 0x824000D0 module:  2 information type:  4 information 
field: 0x000000D0 ts: 0x000D0C9400A0
 ***   SYNC100 timestamp: block:     3 ptr:  1097 data: 0x824000D0 module:  2 information type:  4 information 
field: 0x000000D0 ts: 0x000D0C9800A0
 ***   SYNC100 timestamp: block:     3 ptr:  1099 data: 0x824000D0 module:  2 information type:  4 information 
field: 0x000000D0 ts: 0x000D0C9C00A0
 *** ERROR: fail bit set: block:     3 ptr:  1101 data:E09F0703
 ***            ADC data: block:     3 ptr:  1101 data: 0xE09F0703 module:  2 fail: 1 range: 0 id:   159 
channel: 31 adc:  1795  ts: 0x000D0C9FA328
 ***   SYNC100 timestamp: block:     3 ptr:  1103 data: 0x824000D0 module:  2 information type:  4 information 
field: 0x000000D0 ts: 0x000D0CA000A0
 ***   SYNC100 timestamp: block:     3 ptr:  1105 data: 0x824000D0 module:  2 information type:  4 information 
field: 0x000000D0 ts: 0x000D0CA400A0
 ***   SYNC100 timestamp: block:     3 ptr:  1107 data: 0x824000D0 module:  2 information type:  4 information 
field: 0x000000D0 ts: 0x000D0CA800A0
 ***   SYNC100 timestamp: block:     3 ptr:  1109 data: 0x824000D0 module:  2 information type:  4 information 
field: 0x000000D0 ts: 0x000D0CAC00A0
 *** ERROR: fail bit set: block:     3 ptr:  1111 data:E09F0749


                   blocks:         14
          ADC data format:      21180 (      86.8 Hz)
        Other data format:      93060 (     381.5 Hz)
 Sample trace data format:          0 (       0.0 Hz)
         Undefined format:          0 (       0.0 Hz)
   Other data format type:      PAUSE:          0 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:      93060 (     381.5 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:          0 (       0.0 Hz)
                           Other info:          0 (       0.0 Hz)

   ADC data range bit set:       1239 (       5.1 Hz)

                Timewarps:        ADC:          0 (       0.0 Hz)
                                PAUSE:          0 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:          0 (       0.0 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:          0 (       0.0 Hz)
                            Undefined:          0 (       0.0 Hz)
                         Sample trace:          0 (       0.0 Hz)

 Timestamp elapsed time:      243.950 s

 *** Program elapsed time:    0.719s (   19.478 blocks/s,   1.217 Mb/s)