AIDA GELINA BRIKEN nToF CRIB ISOLDE CIRCE nTOFCapture DESPEC DTAS EDI_PSA 179Ta CARME StellarModelling DCF K40
  AIDA, Page 32 of 46  ELOG logo
 New |  Find |  Select |  Import |  Config |  Last day |  Help 
 Full | Summary | Threaded | Show attachments  Show only new entries    920 Entries 
Goto page Previous  1, 2, 3 ... 31, 32, 33 ... 44, 45, 46   Next  
ID Date Author Subjectdown
  454   Mon Nov 14 05:51:36 2016 TDMonday 14 November
14.51 DSSSD bias & leakage currents OK - see attachments 1 & 2
      Ambient temperature +24.8 deg C, d.p. +7.2 deg C, RH 32.6%
      FEE64 temperatures OK - see attachment 3
  795   Mon May 13 03:23:58 2019 CA, TD, OHMonday 13 May
09.55 all system wide checks OK *except* nnaida7 fails ADC calibration

      FEE64 temperatures OK - see attachment 1

      good event statistics OK - see attachment 2
      all FEE64s <100k, 11x < 20k

      1.8.L spectra - see attachments 3 & 4
      nnaida1 70ch FWHM, nnaida3 80 ch FWHM

      detector biases & leakage currents OK - see attachments 5 & 6

      1.8.W spectra 20us FSR - see attachments 7 & 8

      nnaida1 Options, Local Controls, Data Redaout, Discriminator Controls - see attachment 9-13

      Lost Activity Monitor -see attachment 14

18.01 DAQ found stopped

      This was caused by a chnage to the AIDA external clock frequency - see https://elog.ph.ed.ac.uk/AIDA/798

      Following change of AIDA external clock frequency to 50MHz

      DAQ RESET
      DAQ SETUP

      Sync all ASIC clocks using Resync
      
      all system wide checks OK *except* nnaida7 fails ADC calibration

      zero all histograms

      DAQ GO

18.31 FEE64 temperatures OK - see attachment 15

      good event statistics OK - see attachment 16
      all FEE64s <100k, 10x < 20k

      detector biases & leakage currents OK - see attachments 17 & 18
  313   Mon Jun 13 03:45:15 2016 TDMonday 13 June 2016
10.20 AIDA62 found stalled - no SYNCs, no data transfer
      All FEE64s pass all system-wide tests *except* nnaida3 & nnaida8 which fail ADC calibration
      Data files indicate DAQ stopped c. 06.56
      DAQ stop and file closed

10.30 Detector bias & leakage currents, trigger rates and FEE64 temperatures - see attcahments 1-4
      Julabo FL11006 set point +17 deg C (B2F/F11 ambient temperature +25.3 deg C, d.p. +11.6 deg C, RH 42.7%)

10.48 AIDA63
      Pulser walkthrough

      BNC PB4 settings

      Amplitudes 10,000-90,000 @ 10,000 step
      x5 attenuator IN
      Frequency 250Hz range 
      t_r 50ns tau_d 50us
      - polarity (+ polarity via EG&G Ortec 433A Sum & Invert Amp)
      INT ref, tail pulse, MIN delay

      ASIC settings 2016Jun12-09.09.37

      See attachment 5 showing 9x pulser peaks, noise/beta/background and low amplitude pulser-related peak

      Spectra saved to /MIDAS/SpecData c. 11.00 13 June 2016
      Note directory names are derived from internal FEE64 date/time settings which be wrong or unset

      All spectra cleared

11.06 AIDA64
     
      Per AIDA63 *except*
  
      BNC PB-4 amplitudes

      10,000-2,000 @ 2,000 step

      Spectra saved to /MIDAS/SpecData c. 11.13 13 June 2016

      Note that not all pulse peak amplitudes will be above the slow comparator threshold 

      Attachments 6-10 hit/stat spectra and example energy spectra

EURICA standalone gamma energy and efficiency measurements using 152Eu and 133Ba.
The gamma-ray sources were placed immediately upstream of DSSSD#1 inside the AIDA 
snout.

16.00 Obtained 'unable to connect' to nnaida1 through multiple power cycles/reboots and restarts
      of MERGE unable to clear problem. Finally reosrted to a reboot of aidas1 which did 'fix' the
      problem.

      AIDA65
      Brief test run  to check DAQ/merger/TapeServer - OK

16.06 AIDA66 (AIDA) 1 (EURICA)

      ~11kBq (current activity) 60Co source immediately upstream of DSSSD#1 inside the AIDA snout.

      ASIC settings 2016Jun12-09.09.37

      FADC disabled

      BNC PB-4 pulser
      Amplitude 90,000
      Frequency 25Hz range (probably ~1-few Hz)

      Attachments 11-12 Detector biases & leakage currents -OK

16.37 Attachment 13 - SyncCheck plots

16.50 FR adds ~40kBq 60Co source on top of the AIDA snout immediately above the DSSSDs
      DAQ continues

aidas1> ~/td/GREAT/analyser2 AIDA66_17
 *** GREAT format 3.2.0 analyser - TD - May 2014
 ***   SYNC100: block:     1 ptr:  5567 data: 0x82400595 module:  2 information type:  4 information field:
0x00000595 ts: 0x000000595DD400A0 OLD: ts: 0x07158F300DD4009D tag: 3 type: 0
 *** FEE64 discriminator: block: 15627 ptr: 14341 data: 0x97620008 module: 23 information type:  6 information
field: 0x00020008 ts: 0x005B300000A0 OLD: ts: 0x005B40000050 tag: 2 type: 6
 *** ERROR: READ I/O error:       5002
                   blocks:      32000
          ADC data format:  161285942 (  973887.8 Hz)
        Other data format:   99834058 (  602824.8 Hz)
 Sample trace data format:          0 (       0.0 Hz)
         Undefined format:          0 (       0.0 Hz)
   Other data format type:      PAUSE:        118 (       0.7 Hz)
                               RESUME:        118 (       0.7 Hz)
                              SYNC100:      63176 (     381.5 Hz)
                           FEE64 disc:   98983953 (  597691.6 Hz)
                             MBS info:     786693 (    4750.3 Hz)
                           Other info:          0 (       0.0 Hz)

   ADC data range bit set:      26988 (     163.0 Hz)

                Timewarps:        ADC:          0 (       0.0 Hz)
                                PAUSE:          0 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:          1 (       0.0 Hz)
                           FEE64 disc:          1 (       0.0 Hz)
                             MBS info:          0 (       0.0 Hz)
                            Undefined:          0 (       0.0 Hz)
                         Sample trace:          0 (       0.0 Hz)

 Timestamp elapsed time:      165.610 s
 FEE module #:   1 elapsed dead time       0.378 s
 FEE module #:   2 elapsed dead time       0.464 s
 FEE module #:   3 elapsed dead time       0.002 s
 FEE module #:   4 elapsed dead time       0.000 s
 FEE module #:   5 elapsed dead time       4.701 s
 FEE module #:   6 elapsed dead time       0.228 s
 FEE module #:   7 elapsed dead time       0.000 s
 FEE module #:   8 elapsed dead time       0.000 s
 FEE module #:   9 elapsed dead time       0.304 s
 FEE module #:  10 elapsed dead time       0.154 s
 FEE module #:  11 elapsed dead time       0.901 s
 FEE module #:  12 elapsed dead time       0.000 s
 FEE module #:  13 elapsed dead time       0.203 s
 FEE module #:  14 elapsed dead time       0.164 s
 FEE module #:  15 elapsed dead time       0.000 s
 FEE module #:  16 elapsed dead time       0.000 s
 FEE module #:  17 elapsed dead time       0.000 s
 FEE module #:  18 elapsed dead time       0.000 s
 FEE module #:  19 elapsed dead time       0.000 s
 FEE module #:  20 elapsed dead time       0.000 s
 FEE module #:  21 elapsed dead time       0.000 s
 FEE module #:  22 elapsed dead time       0.346 s
 FEE module #:  23 elapsed dead time       0.000 s
 FEE module #:  24 elapsed dead time       0.000 s
 FEE module #:  25 elapsed dead time       0.000 s
 FEE module #:  26 elapsed dead time       0.000 s
 FEE module #:  27 elapsed dead time       0.000 s
 FEE module #:  28 elapsed dead time       0.000 s
 FEE module #:  29 elapsed dead time       0.000 s
 FEE module #:  30 elapsed dead time       0.000 s
 FEE module #:  31 elapsed dead time       0.000 s
 FEE module #:  32 elapsed dead time       0.000 s

 *** Program elapsed time:  123.023s (  260.113 blocks/s,  16.257 Mb/s)

23.00 Found DAQ stalled - zero AIDA SYNCs from all FEE64s, and data trsabfer halted
      Disk files indicate stall at c. 18.58 (AIDA66_58)

      See attachments 14-19

      All FEE64s pass all system-wide tests

      Installed new versions of check.tcl/check.tml - see attachment 20
      Collect All SYNC Error Counters for Baseline selected

23.15 Standard DAQ stop/go sequence restarts DAQ

      AIDA67
   
      No changes from AIDA66

[td@localhost 1E47-5D81]$ ~/GREAT/analyser2 AIDA67_7
 *** GREAT format 3.2.0 analyser - TD - May 2014
 ***   SYNC100 timestamp: block:     1 ptr:  6123 data: 0x8B4028BF module: 11 information type:  4 information 
field: 0x000028BF ts: 0x0000028BF38C00A0 OLD: ts: 0x0DAF8F30038C0093 tag: 2 type: 6
 *** ERROR: READ I/O error:       5002
                   blocks:      32000
          ADC data format:  162789406 (  931310.8 Hz)
        Other data format:   98330594 (  562544.8 Hz)
 Sample trace data format:          0 (       0.0 Hz)
         Undefined format:          0 (       0.0 Hz)
   Other data format type:      PAUSE:         81 (       0.5 Hz)
                               RESUME:         81 (       0.5 Hz)
                              SYNC100:      66680 (     381.5 Hz)
                           FEE64 disc:   97373010 (  557066.5 Hz)
                             MBS info:     890742 (    5095.9 Hz)
                           Other info:          0 (       0.0 Hz)

   ADC data range bit set:      40512 (     231.8 Hz)

                Timewarps:        ADC:          0 (       0.0 Hz)
                                PAUSE:          0 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:          1 (       0.0 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:          0 (       0.0 Hz)
                            Undefined:          0 (       0.0 Hz)
                         Sample trace:          0 (       0.0 Hz)

 Timestamp elapsed time:      174.796 s
 FEE module #:   1 elapsed dead time       0.162 s
 FEE module #:   2 elapsed dead time       0.232 s
 FEE module #:   3 elapsed dead time       0.000 s
 FEE module #:   4 elapsed dead time       0.000 s
 FEE module #:   5 elapsed dead time       0.213 s
 FEE module #:   6 elapsed dead time       0.273 s
 FEE module #:   7 elapsed dead time       0.000 s
 FEE module #:   8 elapsed dead time       0.000 s
 FEE module #:   9 elapsed dead time       0.433 s
 FEE module #:  10 elapsed dead time       0.123 s
 FEE module #:  11 elapsed dead time       0.417 s
 FEE module #:  12 elapsed dead time       0.000 s
 FEE module #:  13 elapsed dead time       0.127 s
 FEE module #:  14 elapsed dead time       0.000 s
 FEE module #:  15 elapsed dead time       0.000 s
 FEE module #:  16 elapsed dead time       0.000 s
 FEE module #:  17 elapsed dead time       0.000 s
 FEE module #:  18 elapsed dead time       0.000 s
 FEE module #:  19 elapsed dead time       0.000 s
 FEE module #:  20 elapsed dead time       0.095 s
 FEE module #:  21 elapsed dead time       0.000 s
 FEE module #:  22 elapsed dead time       0.030 s
 FEE module #:  23 elapsed dead time       0.000 s
 FEE module #:  24 elapsed dead time       0.000 s
 FEE module #:  25 elapsed dead time       0.000 s
 FEE module #:  26 elapsed dead time       0.000 s
 FEE module #:  27 elapsed dead time       0.000 s
 FEE module #:  28 elapsed dead time       0.000 s
 FEE module #:  29 elapsed dead time       0.000 s
 FEE module #:  30 elapsed dead time       0.000 s
 FEE module #:  31 elapsed dead time       0.000 s
 FEE module #:  32 elapsed dead time       0.000 s

 *** Program elapsed time:  338.012s (   94.671 blocks/s,   5.917 Mb/s)

***            ADC data: block:     1 ptr:  6003 data: 0xC63781A0 module: 24 fail: 0 range: 0 id:  1591 
channel: 55 adc: 33184  ts: 0x************
 ***            ADC data: block:     1 ptr:  6005 data: 0xC2837F23 module: 10 fail: 0 range: 0 id:   643 
channel:  3 adc: 32547  ts: 0x************
 ***            ADC data: block:     1 ptr:  6007 data: 0xC27E7DF9 module:  9 fail: 0 range: 0 id:   638 
channel: 62 adc: 32249  ts: 0x************
 ***            ADC data: block:     1 ptr:  6009 data: 0xC1BA7D10 module:  6 fail: 0 range: 0 id:   442 
channel: 58 adc: 32016  ts: 0x************
 ***            ADC data: block:     1 ptr:  6011 data: 0xC37A7DE7 module: 13 fail: 0 range: 0 id:   890 
channel: 58 adc: 32231  ts: 0x************
 ***            ADC data: block:     1 ptr:  6013 data: 0xC62C8141 module: 24 fail: 0 range: 0 id:  1580 
channel: 44 adc: 33089  ts: 0x************
 *** FEE64 discriminator: block:     1 ptr:  6015 data: 0x81641120 module:  1 information type:  6 information 
field: 0x00041120 ts: 0x************
 *** FEE64 discriminator: block:     1 ptr:  6017 data: 0x81640806 module:  1 information type:  6 information 
field: 0x00040806 ts: 0x************
 *** FEE64 discriminator: block:     1 ptr:  6019 data: 0x8B622000 module: 11 information type:  6 information 
field: 0x00022000 ts: 0x************
 *** FEE64 discriminator: block:     1 ptr:  6021 data: 0x81621000 module:  1 information type:  6 information 
field: 0x00021000 ts: 0x************
 *** FEE64 discriminator: block:     1 ptr:  6023 data: 0x86640912 module:  6 information type:  6 information 
field: 0x00040912 ts: 0x************
 *** FEE64 discriminator: block:     1 ptr:  6025 data: 0x81640008 module:  1 information type:  6 information 
field: 0x00040008 ts: 0x************
 *** FEE64 discriminator: block:     1 ptr:  6027 data: 0x86644009 module:  6 information type:  6 information 
field: 0x00044009 ts: 0x************
 *** FEE64 discriminator: block:     1 ptr:  6029 data: 0x81622880 module:  1 information type:  6 information 
field: 0x00022880 ts: 0x************
 *** FEE64 discriminator: block:     1 ptr:  6031 data: 0x8B622000 module: 11 information type:  6 information 
field: 0x00022000 ts: 0x************
 *** FEE64 discriminator: block:     1 ptr:  6033 data: 0x81620510 module:  1 information type:  6 information 
field: 0x00020510 ts: 0x************
 ***            ADC data: block:     1 ptr:  6035 data: 0xC17774D5 module:  5 fail: 0 range: 0 id:   375 
channel: 55 adc: 29909  ts: 0x************
 ***            ADC data: block:     1 ptr:  6037 data: 0xC31480E9 module: 12 fail: 0 range: 0 id:   788 
channel: 20 adc: 33001  ts: 0x************
 ***            ADC data: block:     1 ptr:  6039 data: 0xC1BD7ED2 module:  6 fail: 0 range: 0 id:   445 
channel: 61 adc: 32466  ts: 0x************
 ***            ADC data: block:     1 ptr:  6041 data: 0xC3957E68 module: 14 fail: 0 range: 0 id:   917 
channel: 21 adc: 32360  ts: 0x************
 ***            ADC data: block:     1 ptr:  6043 data: 0xC25E7E3C module:  9 fail: 0 range: 0 id:   606 
channel: 30 adc: 32316  ts: 0x************
 ***            ADC data: block:     1 ptr:  6045 data: 0xC2777F9B module:  9 fail: 0 range: 0 id:   631 
channel: 55 adc: 32667  ts: 0x************
 ***            ADC data: block:     1 ptr:  6047 data: 0xC3627F99 module: 13 fail: 0 range: 0 id:   866 
channel: 34 adc: 32665  ts: 0x************
 ***            ADC data: block:     1 ptr:  6049 data: 0xC0C08485 module:  3 fail: 0 range: 0 id:   192 
channel:  0 adc: 33925  ts: 0x************
 ***            ADC data: block:     1 ptr:  6051 data: 0xC3B77E63 module: 14 fail: 0 range: 0 id:   951 
channel: 55 adc: 32355  ts: 0x************
 ***            ADC data: block:     1 ptr:  6053 data: 0xC28A7F9E module: 10 fail: 0 range: 0 id:   650 
channel: 10 adc: 32670  ts: 0x************
 ***            ADC data: block:     1 ptr:  6055 data: 0xC2B77ED9 module: 10 fail: 0 range: 0 id:   695 
channel: 55 adc: 32473  ts: 0x************
 ***            ADC data: block:     1 ptr:  6057 data: 0xC2657E47 module:  9 fail: 0 range: 0 id:   613 
channel: 37 adc: 32327  ts: 0x************
 ***            ADC data: block:     1 ptr:  6059 data: 0xC5417EB7 module: 21 fail: 0 range: 0 id:  1345 
channel:  1 adc: 32439  ts: 0x************
 ***            ADC data: block:     1 ptr:  6061 data: 0xC5777ECD module: 21 fail: 0 range: 0 id:  1399 
channel: 55 adc: 32461  ts: 0x************
 ***            ADC data: block:     1 ptr:  6063 data: 0xC3777D25 module: 13 fail: 0 range: 0 id:   887 
channel: 55 adc: 32037  ts: 0x************
 ***            ADC data: block:     1 ptr:  6065 data: 0xC2417E9B module:  9 fail: 0 range: 0 id:   577 
channel:  1 adc: 32411  ts: 0x************
 ***            ADC data: block:     1 ptr:  6067 data: 0xC63782CB module: 24 fail: 0 range: 0 id:  1591 
channel: 55 adc: 33483  ts: 0x************
 ***            ADC data: block:     1 ptr:  6069 data: 0xC24A7F3D module:  9 fail: 0 range: 0 id:   586 
channel: 10 adc: 32573  ts: 0x************
 ***            ADC data: block:     1 ptr:  6071 data: 0xC2DC8028 module: 11 fail: 0 range: 0 id:   732 
channel: 28 adc: 32808  ts: 0x************
 ***            ADC data: block:     1 ptr:  6073 data: 0xC281801A module: 10 fail: 0 range: 0 id:   641 
channel:  1 adc: 32794  ts: 0x************
 ***            ADC data: block:     1 ptr:  6075 data: 0xC1BA7E55 module:  6 fail: 0 range: 0 id:   442 
channel: 58 adc: 32341  ts: 0x************
 ***            ADC data: block:     1 ptr:  6077 data: 0xC1817EBA module:  6 fail: 0 range: 0 id:   385 
channel:  1 adc: 32442  ts: 0x************
 ***            ADC data: block:     1 ptr:  6079 data: 0xC4B67EFE module: 18 fail: 0 range: 0 id:  1206 
channel: 54 adc: 32510  ts: 0x************
 ***            ADC data: block:     1 ptr:  6081 data: 0xC3BA7DF5 module: 14 fail: 0 range: 0 id:   954 
channel: 58 adc: 32245  ts: 0x************
 ***            ADC data: block:     1 ptr:  6083 data: 0xC0BD7E2A module:  2 fail: 0 range: 0 id:   189 
channel: 61 adc: 32298  ts: 0x************
 ***            ADC data: block:     1 ptr:  6085 data: 0xC37A7D4F module: 13 fail: 0 range: 0 id:   890 
channel: 58 adc: 32079  ts: 0x************
 ***            ADC data: block:     1 ptr:  6087 data: 0xC2837F32 module: 10 fail: 0 range: 0 id:   643 
channel:  3 adc: 32562  ts: 0x************
 ***            ADC data: block:     1 ptr:  6089 data: 0xC17779E8 module:  5 fail: 0 range: 0 id:   375 
channel: 55 adc: 31208  ts: 0x************
 ***            ADC data: block:     1 ptr:  6091 data: 0xC0407D64 module:  1 fail: 0 range: 0 id:    64 
channel:  0 adc: 32100  ts: 0x************
 ***            ADC data: block:     1 ptr:  6093 data: 0xC27E7EBB module:  9 fail: 0 range: 0 id:   638 
channel: 62 adc: 32443  ts: 0x************
 ***            ADC data: block:     1 ptr:  6095 data: 0xC1BD7EA1 module:  6 fail: 0 range: 0 id:   445 
channel: 61 adc: 32417  ts: 0x************
 *** FEE64 discriminator: block:     1 ptr:  6097 data: 0x8B648040 module: 11 information type:  6 information 
field: 0x00048040 ts: 0x************
 *** FEE64 discriminator: block:     1 ptr:  6099 data: 0x8B632000 module: 11 information type:  6 information 
field: 0x00032000 ts: 0x************
 *** FEE64 discriminator: block:     1 ptr:  6101 data: 0x8B640014 module: 11 information type:  6 information 
field: 0x00040014 ts: 0x************
 *** FEE64 discriminator: block:     1 ptr:  6103 data: 0x8B622000 module: 11 information type:  6 information 
field: 0x00022000 ts: 0x************
 *** FEE64 discriminator: block:     1 ptr:  6105 data: 0x85640100 module:  5 information type:  6 information 
field: 0x00040100 ts: 0x************
 ***            ADC data: block:     1 ptr:  6107 data: 0xC2048185 module:  8 fail: 0 range: 0 id:   516 
channel:  4 adc: 33157  ts: 0x************
 ***            ADC data: block:     1 ptr:  6109 data: 0xC3817CB5 module: 14 fail: 0 range: 0 id:   897 
channel:  1 adc: 31925  ts: 0x************
 ***            ADC data: block:     1 ptr:  6111 data: 0xC0B76DE1 module:  2 fail: 0 range: 0 id:   183 
channel: 55 adc: 28129  ts: 0x************
 ***            ADC data: block:     1 ptr:  6113 data: 0xC5347FB5 module: 20 fail: 0 range: 0 id:  1332 
channel: 52 adc: 32693  ts: 0x************
 ***            ADC data: block:     1 ptr:  6115 data: 0xC2B87EFA module: 10 fail: 0 range: 0 id:   696 
channel: 56 adc: 32506  ts: 0x************
 *** FEE64 discriminator: block:     1 ptr:  6117 data: 0x82640010 module:  2 information type:  6 information 
field: 0x00040010 ts: 0x************
 *** FEE64 discriminator: block:     1 ptr:  6119 data: 0x82640008 module:  2 information type:  6 information 
field: 0x00040008 ts: 0x************
 *** FEE64 discriminator: block:     1 ptr:  6121 data: 0x82640003 module:  2 information type:  6 information 
field: 0x00040003 ts: 0x************
 ***   SYNC100 timestamp: block:     1 ptr:  6123 data: 0x8B4028BF module: 11 information type:  4 information 
field: 0x000028BF ts: 0x0000028BF38C00A0 OLD: 
ts: 0x0DAF8F30038C0093 tag: 2 type: 6
 ***   SYNC100 timestamp: block:     1 ptr:  6123 data: 0x8B4028BF module: 11 information type:  4 information 
field: 0x000028BF ts: 0x028BF38C00A0
 ***            ADC data: block:     1 ptr:  6125 data: 0xC30B80FA module: 12 fail: 0 range: 0 id:   779 
channel: 11 adc: 33018  ts: 0x028BF38C00BD
 ***            ADC data: block:     1 ptr:  6127 data: 0xC0817E77 module:  2 fail: 0 range: 0 id:   129 
channel:  1 adc: 32375  ts: 0x028BF38C00C2
 ***            ADC data: block:     1 ptr:  6129 data: 0xC277758A module:  9 fail: 0 range: 0 id:   631 
channel: 55 adc: 30090  ts: 0x028BF38C00CA
 *** FEE64 discriminator: block:     1 ptr:  6131 data: 0x8B648000 module: 11 information type:  6 information 
field: 0x00048000 ts: 0x028BF38C01D1
 *** FEE64 discriminator: block:     1 ptr:  6133 data: 0x8B640040 module: 11 information type:  6 information 
field: 0x00040040 ts: 0x028BF38C01D2
 *** FEE64 discriminator: block:     1 ptr:  6135 data: 0x8B640004 module: 11 information type:  6 information 
field: 0x00040004 ts: 0x028BF38C01D5
 *** FEE64 discriminator: block:     1 ptr:  6137 data: 0x8B640010 module: 11 information type:  6 information 
field: 0x00040010 ts: 0x028BF38C01D9
 ***            ADC data: block:     1 ptr:  6139 data: 0xC3B77874 module: 14 fail: 0 range: 0 id:   951 
channel: 55 adc: 30836  ts: 0x028BF38C020A
 ***            ADC data: block:     1 ptr:  6141 data: 0xC0C0848E module:  3 fail: 0 range: 0 id:   192 
channel:  0 adc: 33934  ts: 0x028BF38C0281
 ***            ADC data: block:     1 ptr:  6143 data: 0xC63C8026 module: 24 fail: 0 range: 0 id:  1596 
channel: 60 adc: 32806  ts: 0x028BF38C02DC
 *** FEE64 discriminator: block:     1 ptr:  6145 data: 0x81641102 module:  1 information type:  6 information 
field: 0x00041102 ts: 0x028BF38C0395
 *** FEE64 discriminator: block:     1 ptr:  6147 data: 0x81640020 module:  1 information type:  6 information 
field: 0x00040020 ts: 0x028BF38C0396
 *** FEE64 discriminator: block:     1 ptr:  6149 data: 0x8B620001 module: 11 information type:  6 information 
field: 0x00020001 ts: 0x028BF38C0397
 ***            ADC data: block:     1 ptr:  6151 data: 0xC2B77CE0 module: 10 fail: 0 range: 0 id:   695 
channel: 55 adc: 31968  ts: 0x028BF38C03E1
 *** FEE64 discriminator: block:     1 ptr:  6153 data: 0x8E640200 module: 14 information type:  6 information 
field: 0x00040200 ts: 0x028BF38C03E5
 *** FEE64 discriminator: block:     1 ptr:  6155 data: 0x82640010 module:  2 information type:  6 information 
field: 0x00040010 ts: 0x028BF38C046F
 *** FEE64 discriminator: block:     1 ptr:  6157 data: 0x82640009 module:  2 information type:  6 information 
field: 0x00040009 ts: 0x028BF38C0470
 *** FEE64 discriminator: block:     1 ptr:  6159 data: 0x82640002 module:  2 information type:  6 information 
field: 0x00040002 ts: 0x028BF38C0473
 ***            ADC data: block:     1 ptr:  6161 data: 0xC3777F1E module: 13 fail: 0 range: 0 id:   887 
channel: 55 adc: 32542  ts: 0x028BF38C05F3
 ***            ADC data: block:     1 ptr:  6163 data: 0xC0BA7F03 module:  2 fail: 0 range: 0 id:   186 
channel: 58 adc: 32515  ts: 0x028BF38C063A
 ***            ADC data: block:     1 ptr:  6165 data: 0xC63781EA module: 24 fail: 0 range: 0 id:  1591 
channel: 55 adc: 33258  ts: 0x028BF38C06C4
 ***            ADC data: block:     1 ptr:  6167 data: 0xC25E7EC0 module:  9 fail: 0 range: 0 id:   606 
channel: 30 adc: 32448  ts: 0x028BF38C070A
 ***            ADC data: block:     1 ptr:  6169 data: 0xC310814C module: 12 fail: 0 range: 0 id:   784 
channel: 16 adc: 33100  ts: 0x028BF38C07C5
 ***            ADC data: block:     1 ptr:  6171 data: 0xC2817FBE module: 10 fail: 0 range: 0 id:   641 
channel:  1 adc: 32702  ts: 0x028BF38C07C9
 *** FEE64 discriminator: block:     1 ptr:  6173 data: 0x82640010 module:  2 information type:  6 information 
field: 0x00040010 ts: 0x028BF38C0851
 *** FEE64 discriminator: block:     1 ptr:  6175 data: 0x82640008 module:  2 information type:  6 information 
field: 0x00040008 ts: 0x028BF38C0854
 ***            ADC data: block:     1 ptr:  6177 data: 0xC1817E3E module:  6 fail: 0 range: 0 id:   385 
channel:  1 adc: 32318  ts: 0x028BF38C095D
 *** FEE64 discriminator: block:     1 ptr:  6179 data: 0x86631000 module:  6 information type:  6 information 
field: 0x00031000 ts: 0x028BF38C0998
 ***            ADC data: block:     1 ptr:  6181 data: 0xC5807DFE module: 22 fail: 0 range: 0 id:  1408 
channel:  0 adc: 32254  ts: 0x028BF38C09BC
 ***     MBS information: block:     1 ptr:  6183 data: 0x8680FCC8 module:  6 information type:  8 information 
index:  0 MBS data:FCC8 ts: 0x028BF38C09F7
 ***     MBS information: block:     1 ptr:  6185 data: 0x8681680D module:  6 information type:  8 information 
index:  1 MBS data:680D ts: 0x028BF38C09F7
 ***     MBS information: block:     1 ptr:  6187 data: 0x86820007 module:  6 information type:  8 information 
index:  2 MBS data:0007 ts: 0x028BF38C09F7
 ***            ADC data: block:     1 ptr:  6189 data: 0xC0BD7F10 module:  2 fail: 0 range: 0 id:   189 
channel: 61 adc: 32528  ts: 0x028BF38C0A22
 ***            ADC data: block:     1 ptr:  6191 data: 0xC1777A9E module:  5 fail: 0 range: 0 id:   375 
channel: 55 adc: 31390  ts: 0x028BF38C0B47
 ***            ADC data: block:     1 ptr:  6193 data: 0xC3928023 module: 14 fail: 0 range: 0 id:   914 
channel: 18 adc: 32803  ts: 0x028BF38C0CFA
 ***            ADC data: block:     1 ptr:  6195 data: 0xC3427F14 module: 13 fail: 0 range: 0 id:   834 
channel:  2 adc: 32532  ts: 0x028BF38C0CFB
 *** FEE64 discriminator: block:     1 ptr:  6197 data: 0x85640100 module:  5 information type:  6 information 
field: 0x00040100 ts: 0x028BF38C0D7B
 *** FEE64 discriminator: block:     1 ptr:  6199 data: 0x85640028 module:  5 information type:  6 information 
field: 0x00040028 ts: 0x028BF38C0D7C
 *** FEE64 discriminator: block:     1 ptr:  6201 data: 0x85640020 module:  5 information type:  6 information 
field: 0x00040020 ts: 0x028BF38C0D7F
 ***            ADC data: block:     1 ptr:  6203 data: 0xC5817E5C module: 22 fail: 0 range: 0 id:  1409 
channel:  1 adc: 32348  ts: 0x028BF38C0DA4
 ***            ADC data: block:     1 ptr:  6205 data: 0xC3817F2D module: 14 fail: 0 range: 0 id:   897 
channel:  1 adc: 32557  ts: 0x028BF38C0DC2
 ***            ADC data: block:     1 ptr:  6207 data: 0xC0B76C9E module:  2 fail: 0 range: 0 id:   183 
channel: 55 adc: 27806  ts: 0x028BF38C0E0A
 *** FEE64 discriminator: block:     1 ptr:  6209 data: 0x82648000 module:  2 information type:  6 information 
field: 0x00048000 ts: 0x028BF38C0F44
 *** FEE64 discriminator: block:     1 ptr:  6211 data: 0x82640010 module:  2 information type:  6 information 
field: 0x00040010 ts: 0x028BF38C0F45
 *** FEE64 discriminator: block:     1 ptr:  6213 data: 0x82630008 module:  2 information type:  6 information 
field: 0x00030008 ts: 0x028BF38C0F46
 *** FEE64 discriminator: block:     1 ptr:  6215 data: 0x82640008 module:  2 information type:  6 information 
field: 0x00040008 ts: 0x028BF38C0F46
 ***            ADC data: block:     1 ptr:  6217 data: 0xC0817E94 module:  2 fail: 0 range: 0 id:   129 
channel:  1 adc: 32404  ts: 0x028BF38C0F9A
 *** FEE64 discriminator: block:     1 ptr:  6219 data: 0x82640010 module:  2 information type:  6 information 
field: 0x00040010 ts: 0x028BF38C1018
 ***            ADC data: block:     1 ptr:  6221 data: 0xC61980E8 module: 24 fail: 0 range: 0 id:  1561 
channel: 25 adc: 33000  ts: 0x028BF38C1024
 ***            ADC data: block:     1 ptr:  6223 data: 0xC6248081 module: 24 fail: 0 range: 0 id:  1572 
channel: 36 adc: 32897  ts: 0x028BF38C1024
 ***            ADC data: block:     1 ptr:  6225 data: 0xC5398017 module: 20 fail: 0 range: 0 id:  1337 
channel: 57 adc: 32791  ts: 0x028BF38C103B
 ***            ADC data: block:     1 ptr:  6227 data: 0xC4727E47 module: 17 fail: 0 range: 0 id:  1138 
channel: 50 adc: 32327  ts: 0x028BF38C1089
 ***            ADC data: block:     1 ptr:  6229 data: 0xC20080B8 module:  8 fail: 0 range: 0 id:   512 
channel:  0 adc: 32952  ts: 0x028BF38C10C1
 ***            ADC data: block:     1 ptr:  6231 data: 0xC0C08465 module:  3 fail: 0 range: 0 id:   192 
channel:  0 adc: 33893  ts: 0x028BF38C1159
 ***            ADC data: block:     1 ptr:  6233 data: 0xC2657E31 module:  9 fail: 0 range: 0 id:   613 
channel: 37 adc: 32305  ts: 0x028BF38C12C2
 *** FEE64 discriminator: block:     1 ptr:  6235 data: 0x82648010 module:  2 information type:  6 information 
field: 0x00048010 ts: 0x028BF38C12DF
 *** FEE64 discriminator: block:     1 ptr:  6237 data: 0x82640008 module:  2 information type:  6 information 
field: 0x00040008 ts: 0x028BF38C12E3
 ***            ADC data: block:     1 ptr:  6239 data: 0xC36C801D module: 13 fail: 0 range: 0 id:   876 
channel: 44 adc: 32797  ts: 0x028BF38C14CB
 *** FEE64 discriminator: block:     1 ptr:  6241 data: 0x81641000 module:  1 information type:  6 information 
field: 0x00041000 ts: 0x028BF38C153A
 *** FEE64 discriminator: block:     1 ptr:  6243 data: 0x8B632000 module: 11 information type:  6 information 
field: 0x00032000 ts: 0x028BF38C153A
 *** FEE64 discriminator: block:     1 ptr:  6245 data: 0x8B640040 module: 11 information type:  6 information 
field: 0x00040040 ts: 0x028BF38C153A
 ***            ADC data: block:     1 ptr:  6247 data: 0xC5F7816D module: 23 fail: 0 range: 0 id:  1527 
channel: 55 adc: 33133  ts: 0x028BF38C154C
 ***            ADC data: block:     1 ptr:  6249 data: 0xC3BB7E35 module: 14 fail: 0 range: 0 id:   955 
channel: 59 adc: 32309  ts: 0x028BF38C1592
 *** FEE64 discriminator: block:     1 ptr:  6251 data: 0x97630080 module: 23 information type:  6 information 
field: 0x00030080 ts: 0x028BF38C164D
  776   Mon Nov 12 08:59:43 2018 LS, CA, TDMonday 12 November 6pm onwards

17:59 DAQ continues OK
      Stats OK - attachment 1
      FEE temperatures OK - attachment 2
      Bias and leakage currents OK - attachment 3
      System-wide checks OK *except* nnaida7 fails ADC calibration


19:07 DAQ continues OK
      Stats OK - attachment 4
      FEE temperatures OK - attachment 5
      Bias and leakage currents OK - attachment 6
      System-wide checks OK *except* nnaida7 fails ADC calibration

20.19 DAQ starts
      AIDA file RIBF148/R5_4
      BRIKEN file 16
      BigRIPS 1001

      165Pm setting + 3mm Al degrader

20.21 all online & MIDASsort histograms zero'd

20.23 FEE64 good event statistics - see attachment 7
      Merger - see attachment 8
      TapeServer - see attachment 9

      HEC x versus y spectra - see attachment 10
       SN thinks DSSSD #3 are the ions of interest, #4 are the ions of interest at different charge state
       and #5 & #6 are lighter ions which do not stop in DSSSD #1-6

21.21 DAQ stops 

      DAQ starts
      AIDA file RIBF148/R5_14
      BRIKEN file 18
      BigRIPS 1003

21.26 HEC x vs y multiplicity spectra - attachment 11

21.29 HEC Ex vs Ey spectra - attachment 12
      FEE64 rate spectra - attachment 13

21.31 1.8.L front/rear energy spectra (attachments 14/15) LEC

21.33 1.8.H front/rear energy spectra (attachments 16/17) HEC

21.46 system wide checks ok, except nnaida7 ADC calibration failure
      FEE64 good event statistics ok (attachment 18)
      FEE64 temperatures ok (attachment 19)
      Leakage currents/detector bias ok (attachment 20)

21.49 Merger and TapeServer ok, no error messages in terminal
      DAQ still running (RIBF148/R5_18)

22.10 attachment 21 - pulser spectra in all channels fired at same time
                    - FEE64 in sync

22.12 SYNC100 at 9153.8 Hz
      FEE64 discriminator at ~15000 Hz - higher than expected
      ~1 second deadtime in FEE19 - due to nnaida12 running fast
      no timewarps observed in FEE64
      all of above in attachment 22
   
22:32 DAQ continues OK
      Stats OK - attachment 24
      FEE temperatures OK - attachment 23
      Bias and leakage currents OK - attachment 25
      System-wide checks OK *except* nnaida7 fails ADC calibration   

23:30 DAQ continues OK
      Stats OK - attachment 26
      FEE temperatures OK - attachment 27
      Bias and leakage currents OK - attachment 28
      System-wide checks OK *except* nnaida7 fails ADC calibration
  775   Sun Nov 11 21:35:42 2018 CB, OH, CA, LSMonday 12 November
06:35 DAQ Continues ok
      Stats ok - attachment 1
      FEE Temps ok - attachment 2
      Bias and leakage currents ok - attachment 3
      System wide checks all ok
      Chiller refilled

07:57 DAQ continues OK
      Stats OK - attachment 4
      FEE temperatures OK - attachment 5
      Bias and leakage currents OK - attachment 6
      System-wide checks OK *except* nnaida7 fails ADC calibration

10:06 DAQ continues OK
      Stats OK - attachment 7
      FEE temperatures OK - attachment 8
      Bias and leakage currents OK - attachment 9
      System-wide checks OK *except* nnaida7 fails ADC calibration
      Workstation occasionally feels sluggish. See iostat 5 (attach 10) & top -a (attach 11)

12:08 DAQ continues OK
      Stats OK - attachment 12
      FEE temperatures OK - attachment 13
      Bias and leakage currents OK - attachment 14
      System-wide checks OK *except* nnaida7 fails ADC calibration
      

15:40 DAQ continues OK
      Stats OK - attachment 15
      FEE temperatures OK - attachment 16
      Bias and leakage currents OK - attachment 17
      System-wide checks OK *except* nnaida7 fails ADC calibration


16:58 DAQ continues OK
      Stats OK - attachment 18
      FEE temperatures OK - attachment 19
      Bias and leakage currents OK - attachment 20
      System-wide checks OK *except* nnaida7 fails ADC calibration
  658   Mon Jun 12 08:02:01 2017 TDMonday 12 June 16.00-24.00
16.01 DAQ continues

16.01 Detector biases & leakage currents OK - see attachments 1 & 2
      FEE64 Temperatures OK - see attachment 3
      good events statistics stable & OK - see attachment 4 
      No warnings/errors reported by merger since start of RIBF148/R3  - see attachment 5
      *all* system wide checks OK *except* nnaida13 fails ADC calibration

      BNC PB-4
      Amplitude 90,000
      x5 attenuator IN
      t_r 100ns tau_d 50us
      - polarity
      frequency 25Hz range 2.6Hz
      delay MIN, tail pulse, INT ref

      ASIC settings 2017Mar27-16.12.39

      147Sm BigRIPS setting - 0mm Al (at F7) + 2mm variable + 0mm Al fixed (at F11) = 2mm Al

16.08 *all* online histograms zero'd 

16.31 DAQ stop
      AIDA file RIBF148/R3_84

16.31 DAQ start
      AIDA file RIBF148/R3_84
      BigRIPS file 2504
      BRIKEN file 32

17.33 DAQ stop
      AIDA file RIBF148/R3_104

17.34 DAQ start
      AIDA file RIBF148/R3_104
      BigRIPS file 2505
      BRIKEN file 33

18.01 Detector biases & leakage currents OK - see attachments 6 & 7
      FEE64 Temperatures OK - see attachment 8
      good events statistics stable & OK - see attachment 9
      No warnings/errors reported by merger since start of RIBF148/R3  - see attachment 10
      *all* system wide checks OK *except* nnaida13 fails ADC calibration

18.03 online analysis file RIBF148/R3_113 - deadtime < 0.5%, 2 MBS & 0 ts timewarps - see attachment 11

18.04 DAQ stop
      AIDA file RIBF148/R3_114

18.05 DAQ start
      AIDA file RIBF148/R3_114
      BigRIPS file 2506
      BRIKEN file 34

      147Sm BigRIPS setting - 0mm Al (at F7) + 0mm variable + 0mm Al fixed (at F11) = 0mm Al

19.14 DAQ stop
      AIDA file RIBF148/R3_135

19.15 DAQ start
      AIDA file RIBF148/R3_135
      BigRIPS file 2507
      BRIKEN file 35

20.01 Detector biases & leakage currents OK - see attachments 12 & 13
      FEE64 Temperatures OK - see attachment 14
      good events statistics stable & OK - see attachment 15
      No warnings/errors reported by merger since start of RIBF148/R3  - see attachment 16
      *all* system wide checks OK *except* nnaida13 fails ADC calibration

20.03 online analysis file RIBF148/R3_150 - deadtime < 0.5%, 0 MBS & ts timewarps - see attachment 17

20.25 DAQ stop
      AIDA file RIBF148/R3_157

20.26 DAQ start
      AIDA file RIBF148/R3_158
      BigRIPS file 2508
      BRIKEN file 36

21.30 DAQ stop
      AIDA file RIBF148/R3_178

21.31 DAQ start
      AIDA file RIBF148/R3_179
      BigRIPS file 2509
      BRIKEN file 37

22.06 Detector biases & leakage currents OK - see attachments 18 & 19
      FEE64 Temperatures OK - see attachment 20
      good events statistics stable & OK - see attachment 21
      No warnings/errors reported by merger since start of RIBF148/R3  - see attachment 22
      *all* system wide checks OK *except* nnaida13 fails ADC calibration

22.08 online analysis file RIBF148/R3_190 - deadtime < 1%, 1 MBS & 0 ts timewarps - see attachment 23

22.26 DAQ stop
      AIDA file RIBF148/R3_196

22.27 DAQ start
      AIDA file RIBF148/R3_197
      BigRIPS file 2510
      BRIKEN file 38

23.03 online analysis
      per DSSSD x versus y (HEC) - see attachment 24
      per DSSSD E_P versus E_n (HEC) - see attachment 25
      ADC & disc data synchronised - see attachment 26

23.16 beam lost
      AIDA file RIBF148/R3_212

23.16 beam back within 1 min
      AIDA file RIBF148/R3_213

23.31 DAQ stop
      AIDA file RIBF148/R3_217

22.32 DAQ start
      AIDA file RIBF148/R3_218
      BigRIPS file 2511
      BRIKEN file 39
  656   Sun Jun 11 16:04:12 2017 DKMonday 12 June 0:00 ~ 8:00
0:04    Biases and leak currents shown as attachment #1, 12JunBias1.png
        System-wide checks: nnaida13 fails ADC calib, all others okay
        Temperatures shown as attachment #2, 12JunTemp1.png
        Implants are 20 to 40 cps, mostly in SSD2 and SSD6 see attachment #3 12JunImplant1.png
        Statistics shown attachment #4, 12JunStats1.png
        Analysis shown as R9_175.txt attachment #5
        

00:17   No implants in AIDA nor proper evt. numbers in BRIKEN R9_177, expect beam is down
        Operators call, RF dropped of IRC

00:23   Problem of another RF.

00:24   Beam is back R9_179 for AIDA

00:36   AIDA R9_182 at run close.

00:39   AIDA R9_183, BRIKEN 141, run BigRIPS 4031

00:50   We stop the beam to adjust degraders in F11.  AIDA R9_186
        BRIKEN and BigRIPS runs stopped

00:54   Removed 1.0 mm Al, added 0.5 mm (net is subtracting 0.5 mm Al)
        R9_187 AIDA run, BRIKEN #142, BigRIPS #4032

01:34   AIDA R9_198 at changeover

01:35   Same AIDA run R9_198, BRIKEN #143, BigRIPS #4033
        System wide checks okay (except ADC calib on nnaida13)

02:34   R9_214 at changeover.  

2:36    R9_215 with new runs BRIKEN 144 and BigRIPS 4034.
        More implants are now seen in SSD3 rather than SSD2.  Zeroed MIDAS histos, so updated later with stats.
        Biases as attachment #6, 12JunBiases2.png

02:45  Beam stopped R9_217.  Beam stripper has a problem.

02:48  Beam is back, R9_220

03:12  R9_224 operators call, beam is stopped from stripper foil issue

03:18  Beam is back, AIDA run R9_226

03:34  R9_230 at changeover
       System wide checks all okay, except nnaida13 ADC calib as above.

03:35  The end of R9_230 corresponds with BRIKEN 145 and BigRIPS 4034.

03:51  Operators call, beam looks stopped.  R9_235 at AIDA.  Change stripper foil

03:57  R9_236 when beam returns.

04:35  We noticed beam is gone.  R9_246 for AIDA.  

04:36  Beam comes back.  R9_247 BRIKEN 146 and BigRIPS 4035
       Attachment #7 12JunImplant2.png shows implantation profile, confirming SSD 3 is the main one with new
degrader setting
       Statistics shown as attachment #8, 12JunStats2.png
       Analysis shown as attachment #9, R9_244.txt
         No ts timewarps, no MBS timewarps, deadtime on nnaida13,21 nearly 1%, most others small.

05:20  Noticed /data10 only has ~250 GB left, so we should do something to resolve it.
       First thing is, not much compression is done recently.  So I begin to compress /TapeData/RIBF128/R7* with
nice 10
       However, compression at this rate will take much too long so it looks like something should be removed.
       Confirmed that RIBF127R1 data file names and sizes are the same between /TapeData and the new external HDD
at /ntfs

05:33  AIDA R9_262, BRIKEN 147, BigRIPS 4036

06:34  AIDA R9_279 at run change

06:36  BRIKEN run 148, BigRIPS 4037
       System-wide checks passed, except nnaida13 ADC calib still fails.

06:54  R9_284 beam is stopped.

6:59   Beam is back.

07:33  R9_295 at start of BRIKEN 149, BigRIPS 4037

07:36  No syncs are seen at BRIKEN (but they were seen at the start of other runs)
       Implant profiles shown as Attachment #10, 12JunImplants3.png
       Several resets or new runs on BRIKEN / BigRIPS are done to troubleshoot

07:48  AIDA passes all system checks (besides the nnaida13 ADC calib)
       It seems (part of) the issue is triggers from BigRIPS.

08:04  BigRIPS scalers show events are presented, but no triggers are accepted.
       Some part of the DAQ is vetoing the events basically.

08:11  Baba san came on shift.  Issue was ccnet03 crashed.  We reset the Camac crate and reinitialized, DAQ comes back
       AIDA R9_306, BRIKEN #152, BigRIPS # 4041
  657   Mon Jun 12 00:31:29 2017 ZLiu, TD, OHMonday 12 June 08:00-16:00 
09:00   Stop DAQ  to tune the Sm beam, may need 3 hours
        AIDA file RIBF128/R9_319
        
        removed the 1.5 mm degrader before the Pb box
     

09:  Start DAQ
        AIDA file RIBF128/R9


10.35     Detector biases & leakage currents OK - see attachments 1 & 2

             FEE64 Temperatures OK - see attachment 3

      good events statistics stable & OK - see attachment 4
 
      1x error reported by merger since c. 15.00 Sunday 12 June  - see attachment 5

      *all* system wide checks OK *except* nnaida13 fails ADC calibration

11.33 DAQ stop

          AIDA file RIBF128/R9_358

11.39 DAQ start

          AIDA file RIBF148/R3_0

          Correlation scaler reset request sent


11.58 analysis file RIBF148/R3_3 - deadtime < 0.5%, 2 MBS timewarps, zero ts timewarps - see attachment 6

13:33 Leakage currents as beam starts - attachment 7

13:40 BigRIPS starts run 2501
      BRIKEN starts run 29
      AIDA on file R3_31
      No degraders in at F7 and no variable degraders in

13:42 DSSD rates from R3_30
 *** DSSSD # 1 count:    111755 old count:     54792 dt:      1.01 s  LEC rate:  56401.81 Hz
 *** DSSSD # 2 count:    163809 old count:     82230 dt:      1.01 s  LEC rate:  80775.29 Hz
 *** DSSSD # 3 count:     61328 old count:     31972 dt:      1.01 s  LEC rate:  29066.79 Hz
 *** DSSSD # 4 count:     50635 old count:     25404 dt:      1.01 s  LEC rate:  24982.43 Hz
 *** DSSSD # 5 count:     23710 old count:     12126 dt:      1.01 s  LEC rate:  11469.88 Hz
 *** DSSSD # 6 count:     41872 old count:     21419 dt:      1.01 s  LEC rate:  20251.50 Hz
 *** DSSSD # 1 count:         2 old count:         0 dt:      1.01 s  HEC rate:      1.98 Hz
 *** DSSSD # 2 count:         8 old count:         5 dt:      1.01 s  HEC rate:      2.97 Hz
 *** DSSSD # 3 count:         0 old count:         0 dt:      1.01 s  HEC rate:      0.00 Hz
 *** DSSSD # 4 count:         0 old count:         0 dt:      1.01 s  HEC rate:      0.00 Hz
 *** DSSSD # 5 count:        20 old count:         8 dt:      1.01 s  HEC rate:     11.88 Hz
 *** DSSSD # 6 count:       192 old count:        91 dt:      1.01 s  HEC rate:    100.00 Hz

13:44 BRIKEN and BigRIPS stop their DAQs

13:44 BigRIPS setting 147Sm 0mm Al (at F7), 2mm Al (variable) + 0mm Al fixed (at F11)

13:50 DSSD rates from R3_33
 *** DSSSD # 1 count:    173930 old count:    115392 dt:      1.02 s  LEC rate:  57530.02 Hz
 *** DSSSD # 2 count:    247217 old count:    167042 dt:      1.02 s  LEC rate:  78794.45 Hz
 *** DSSSD # 3 count:    106853 old count:     71057 dt:      1.02 s  LEC rate:  35179.62 Hz
 *** DSSSD # 4 count:     87071 old count:     58930 dt:      1.02 s  LEC rate:  27656.43 Hz
 *** DSSSD # 5 count:     31923 old count:     21528 dt:      1.02 s  LEC rate:  10216.01 Hz
 *** DSSSD # 6 count:     61469 old count:     40889 dt:      1.02 s  LEC rate:  20225.63 Hz
 *** DSSSD # 1 count:         1 old count:         1 dt:      1.02 s  HEC rate:      0.00 Hz
 *** DSSSD # 2 count:        18 old count:        11 dt:      1.02 s  HEC rate:      6.88 Hz
 *** DSSSD # 3 count:        23 old count:        13 dt:      1.02 s  HEC rate:      9.83 Hz
 *** DSSSD # 4 count:       140 old count:        90 dt:      1.02 s  HEC rate:     49.14 Hz
 *** DSSSD # 5 count:       117 old count:        80 dt:      1.02 s  HEC rate:     36.36 Hz
 *** DSSSD # 6 count:        38 old count:        28 dt:      1.02 s  HEC rate:      9.83 Hz

13:52 BRIKEN starts run 30
      BigRIPS starts run 2502
      AIDA on file R3_35

13:54 System wide checks ok
      DSSD rates from R3_35
 *** DSSSD # 1 count:    112192 old count:     55424 dt:      1.03 s  LEC rate:  55291.13 Hz
 *** DSSSD # 2 count:    165577 old count:     82475 dt:      1.03 s  LEC rate:  80940.03 Hz
 *** DSSSD # 3 count:     64444 old count:     31995 dt:      1.03 s  LEC rate:  31604.81 Hz
 *** DSSSD # 4 count:     54003 old count:     26596 dt:      1.03 s  LEC rate:  26693.98 Hz
 *** DSSSD # 5 count:     29688 old count:     13945 dt:      1.03 s  LEC rate:  15333.43 Hz
 *** DSSSD # 6 count:     41562 old count:     21408 dt:      1.03 s  LEC rate:  19629.68 Hz
 *** DSSSD # 1 count:         0 old count:         0 dt:      1.03 s  HEC rate:      0.00 Hz
 *** DSSSD # 2 count:         4 old count:         2 dt:      1.03 s  HEC rate:      1.95 Hz
 *** DSSSD # 3 count:         9 old count:         5 dt:      1.03 s  HEC rate:      3.90 Hz
 *** DSSSD # 4 count:         8 old count:         3 dt:      1.03 s  HEC rate:      4.87 Hz
 *** DSSSD # 5 count:        85 old count:        34 dt:      1.03 s  HEC rate:     49.67 Hz
 *** DSSSD # 6 count:       102 old count:        50 dt:      1.03 s  HEC rate:     50.65 Hz

14:20   no beam now

14:22 Beam back

14.50 analysis file RIBF148/R3_50 - deadtime < 0.5%, 7 MBS timewarps, zero ts timewarps - see attachment 8

15.02 DAQ stop
      AIDA file RIBF148/R3_57

      Beam off - scheduled maintenance ~2h

15.18 online analysis
      per DSSSD x versus y (HEC) - see attachment 9
      per DSSSD E_p versus E_n (HEC) 20MeV/ch nominal - see attachment 10
      ADC & disc data synchronised - see attachment 11


15.30 DAQ start
      AIDA file RIBF148/R3_64
      BigRIPS file 2503
      BRIKEN file 31
  937   Mon Dec 11 09:57:54 2023 TDMonday 11 December
Water flow and temperature ( gauges outside S4 entrance ) OK

BNC PB-5 pulser connected to p+n FEE64s by diasy chain of test + inputs

ASIC settings 2023Nov16-18.55.20
 p+n FEE64s slow comparator 0xa, n+n FEE64s 0xf

All system wide checkks OK

DSSSD bias -150V leakage current -5.37uA - OK - attachment 1

FEE64 temperatures OK - attachment 2

ADC data item stats - attachment 3
 rates dominated by relatively few 'hot' channels
 aida06 - no cables connected to FEE64 adaptor PCB

per FEE64 Rate spectra - attachment 4
 aida06 - no cables connected to FEE64 adaptor PCB 
 system relatively quiet but has been quieter - no grounding of bPlast cabling and PSUs?

p+n and n+n FEE64 1.8.W spectra - 20us FSR - attachments 5-8
 aida06 - no cables connected to FEE64 adaptor PCB
 aida08 - no ADC calibration 



14.43 Pulser walkthrough - p+n junction FEE64s

BNC PB-5 settings
amplitude 9.0-1.0V @ 1.0V step
x10 attenuation
tail pulse
polarity +
frequency 22Hz

file DEC23/R5

15.47 R45 ends OK - see attachments 9-10
aida01 1.8.L pulser peak width 53 ch FWHM
aida06 1.8.L pulser peak width 14 ch FWHM ( no input )

15.47 file DEC23/R6 

BNC PB-5 pulser
amplitude 9.0V
frequency 2Hz

c. 19.00
Low intensity 238U fully stripped/H+/He+ - attachments 11-14
Fully stripped close to centre of DSSSD
HEC energies observed from threshold to c. 1-2GeV

19.25 Beam off
      SC41 scaler input to level 1 MACB #3 ( of 4 ) moved to level 1 MACB #4 ( of 4 )


21.00 checks


DSSSD bias -150V leakage current -6.050uA - OK - attachment 15

FEE64 temperatures OK - attachment 16

ADC data item stats - attachment 17
 rates dominated by relatively few 'hot' channels
 aida06 - no cables connected to FEE64 adaptor PCB

BNC PB-5 config - attachment 18

Merger and TapeServer browser tabs and terminal servers - attachments 19-20
 c. 160k data items/s merged
 c. 600kb to disk and forwarded to MBS

21.05.37 *all* stats zero'd

22.18 ADC data, DISC data, PAUSE, RESUME, WR data item stats - attachments 21-28
zero rate PAUSE/RESUME - one observed in c. 75 minutes
  381   Mon Oct 10 03:15:59 2016 CG & TDMonday 10 October
10.30 DAQ found stopped

      All FEE64 AIDA SYNC rates zero, AIDA SYNC count ~ 4.4M => DAQ stopped @ 01.40

      3He detector DAQ off - no 50MHz clock to Master MACB

      System wide checks - see attachments 26-18

      N.B. It would appear that the DAQ stalled (TS_SYNC_PHASE 1) c. 01.40 and at some
      later time (c. 06.00) the 3He detector DAQ was powered off which means that AIDA
      lost the external 50MHz clock to the Master MACB - system-wide screenshots should
      be interpreted with caution.

14.15 3He detector DAQ 50MHz clock restarted
 
      VME and NIM crates ON
      Excecute command ConfClock_50MHz on 3He detector DAQ computer

      Power cycle AIDA FEE64s
      Set TS_SYNC_PHASE 2

      DSSSD bias/leakage current OK
      FEE64 temperatures OK
      *All* system-wide checks OK

      B2F/F11 temperature + deg C, d.p. + deg C,RH %

      ASIC settings 2016Sep02-18.17.01
      slow comparator 0x10 -> 0x0a

      Master MACB serial #13 (see https://elog.ph.ed.ac.uk/AIDA/367)
      Master MACB mode 5

14.20 DAQ starts (no data transfer)

18.10 DAQ found stopped

      All FEE64 AIDA SYNC rates zero, AIDA SYNC count ~ 3.4M => DAQ stopped @ 16.50

      System-wide checks - see attachments 16-9

      Power cycle AIDA FEE64s
      Set TS_SYNC_PHASE 3

      ASIC settings 2016Sep02-18.17.01
      slow comparator 0x10 -> 0x0a

20.10 DAQ found stalled

      System-wide checks - see attachments 8-1

      Power cycle AIDA FEE64s
      Set TS_SYNC_PHASE 0

      ASIC settings 2016Sep02-18.17.01
      slow comparator 0x10 -> 0x0a

20.35 DAQ starts (no data transfer)

      slow comparator 0x10 -> 0x0a

18.35 DAQ starts (no data transfer)

20.16 DAQ found stopped
      All FEE64 AIDA SYNC rates zero, AIDA SYNC count ~ 120k (17/24 FEE64s) or 1.49M (7/24) => FEE64s stopped @ 18.40 & 19.40

20.36 DAQ starts (no data transfer)

23.47 DAQ OK
  835   Mon Jun 10 01:43:16 2019 CA, CB, TD, CG, OH, DKMonday 10 June
09:45 DAQ continues ok 

      all system wide checks OK *except* nnaida7 fails ADC calibration
      
      detector biases & leakage currents OK - see attachment 1

      FEE64 temperatures OK - see attachment 2

      Good event statistics prior to reload - glitch with display? - attachment 3

      Reload done - good event statistics ok - attachment 4

10.00 Merger & TapeServer ok - attachments 5 & 6

15.14 DAQ continues OK

      ASIC settings 2019May19-17.54.03
      slow comparator 0xa

      BNC PB-4 Pulser 
      Amplitude 90,000
      x5 attenuator IN
      Frequency 25Hz range, variable min. c. 2.5Hz
      t_r 100ns, tau_d 50us
      - polarity
      Delay MIN, tail pulse

      all system wide checks OK *except* nnaida7 fails ADC calibration

      detector biases & leakage currents OK - see attachments 7 & 8

      FEE64 temperatures OK - see attachment 9

      good event statistics - DSSSD#1 noisy - see attachments 10
      8x <20k, 23x <50k, 24x <100k cf. same as earlier today
      Merger OK - see attachment 11
      TapeServer OK - see attachment 12

      p+n FEE64s 1.8.W spectra 20us FSR - DSSSD#1 noisy - attachment 13
      n+n FEE64s 1.8.W spectra 20us FSR - DSSSD#1 noisy - attachment 14
      1.8.L spectra - see attachments 9 & 10
      nnaida1 73 ch FWHM, nnaida3 78 ch FWHM - cf. 76 & 83 ch FWHM respectively yesterday morning

      FEE64 Rate spectra - attachment 15
      Check ASIC Control *all* FEE64s & ASICs x2
      FEE64 Rate spectra - attachment 16

15.26 ADC data synchronised - attachment 17

16.55 Julabo FL11006 chiller set point +20 deg C actual +20.0 deg C
      Water level c. 80% M-H markers - topup to 100% with 0.5x jugful water
      Temperature +24.6 deg C dew point +8.1 deg C RH 34.5%
  745   Mon Oct 1 01:10:57 2018 OH, TDMonday 1 October
09.10 DAQ continues OK - current file Sep18/R2_16

09.15 all system wide checks OK *except* nnaida7 fails ADC calibration

      detector biases & leakage currents OK - see attachments 1 & 2

      FEE64 temperatures OK - see attachment 3
      good event statistics OK - see attachment 4
      nnaida2 rate due to one hot channel

      Merger OK - see attachment 5
      TapeServer OK - see attachment 6
      no errors or warnings shown by server terminal sessions 

      1.8.L spectra - see attachments 7 & 8
      1.8.W spectra 20us FSR - see attachments 9 & 10
      
09.24 temperature +25.5 deg C, RH 43.4%, d.p. +11.9 deg C
      Julabo FL11006 set point +20 deg C, actual +20.0 deg C, water level c. 75%

16.25 Julabo FL11006 refill to just below H indicator

      DAQ continues OK - current file Sep18/R2_19
  850   Mon Jun 17 08:03:37 2019 CA, CB, TD, OHMon Jun 17 16:00-24:00
16:04  DAQ continues running OK
       All system-wide checks OK except nnaida7 ADC calibration fails
       FEE64 temperatures OK - attach 1
       Merger and Tape Server OK - see attach 2 & 3

16:30 DTAS and BigRIPS stop
      AIDA on file R2_406

16:35 Beam stop.
      AIDA plastic gain increased by a factor 2
      Moved the lead bricks between AIDA and F11, 1 cm further away to the sides

      Chiller water level OK
      Humidity  34.6%
      Temperature 23.4 C
      Dew point 7.1 C
      

16:40 Beam restart
      AIDA run R2_408
      DTAS 70 (from DTAS ELOG)
      BigRIPS 2030  (from DTAS ELOG)

      No obvious changes to the implantation pattern. May need more stats.   

      Detector leakage current OK - see attach 4    

17.31 nnaida14 3.*.H & 4.*.H spectra - attachments 5 & 6

      Corresponds to RHS DSSSD#3 - decrease in rate observed but no evidence of systematic variation of HEC energy 

      Modified implantdecay4.f to select events with x & y energy > ADC ch 1600 (c. 1MeV) to observe x-y hit pattern
      of light particles - see attachments 7 & 8
      Low stats but light particle distribution appears to be symmetric with no obvious RHS cut


17:37 DTAS & BigRIPS DAQ stop
      Beam stop
      Lead moved back as before
      AIDA plastic gain decreased by a factor 5

17:40 DTAS and BigRIPS stop
      AIDA on file R2_416


17.48 zero *all* SortSas spectra

17:50 Beam restarting
      AIDA R2_418
      DTAS  73  (72 is a mistake - ignore)
      BigRIPS  2031


18:43 DAQ continues OK
      All system wide checks OK - except nnaida7 fails ADC calibration as usual
      FEE64 temperature OK - see attach 9
      Stats OK - see attach 10
      Merger & Tape Server OK - see attach 11 & 12
      Leakage currents OK - see attach 13
      HEC & LEC events plots attached - attach 14 & 15 - note LEC in DSSD#3 are > 3 MeV only (light particle-like)

18:50 AIDA R2_427
      DTAS 74
      BigRIPS 2031

19.08 DTAS & BigRIPS DAQ stop - beam lost
      AIDA file R2_429

21:15 Beam is back - briefly
      AIDA R2_446

21:17 Beam lost - operators advise "this will take a long time"

22:16 Beam is back!
      AIDA R2_454
      DTAS 78
      BigRIPS 2034

      Statistics OK - attach 15
      FEE64 temperatures OK - attach 16
      Merger & Tape Server OK - see attach 17 & 18
      System-wide checks as usual

      DTAS reports rate in F11 down by 20%, and rate in AIDA plastic sees similar drop.
  681   Mon Oct 23 11:50:09 2017 TD, OHMon 23 Oct - further information please.
> 10:18 Leakage and bias ok - Attachment 1
>       FEE temperatures ok - Attachment 2
>       Statistics - Attachment 3
>       DAQ continues ok
> 
> 10:23 DAQ stopped on R7_10
>       Alpha run through night
> 
> 10:28 MIDAS Sort spectra from the alpha run
>       Ep versus En - Attachment 4
>       x versus y 2D hit pattern - Attachment 5
>       Front versus back multiplicity - Attachment 6
> 
>       2D Hit pattern indicates detection of alphas in all 4 regions of each of the 6 detectors
>       indicates cabling is ok
> 
> 11:55 FEEs turned off after being unable to recover after crash
Please could you define this 'crash' to allow a better understanding of problems with the system. 
For example.... what is unresponsive.... does a parse of the FEE64 log files ( before doing anything ) reveal a panic that is new? Does a STOP of the Tape work ? Does a Toggle of the MERGE pause state 
work ? 
Thus it will be possible to say that ..... FEE # is always the one responsible for DAQ failure or by < ???? > or the MERGE has found < ??? > problem.
Thanks in advance ;-)))

> 
> 13:49 FEEs powered on again
>       Pi monitor indicates panic on nnaida22 the FEE that crashed
> 
>       ReSync done on setup except for 23 and 24
>       Master ReSync done succesfully
>       ASIC settings loaded default to 2017Mar27
>       Restoring 2017Oct22
>       
>       System wide checks ok
>       Leakage and bias ok - Attachment 7
>       FEE temperatures crashed - Attachment 8
> 
>       Upon reboot all rates across the FEEs appear lower
>       In particular 13 and 14. Thresholds have been reduced back to 0xa for all ASICs on both FEEs
> 
>       Layout 1 linear common y of 2000 - Attachment 9
>       Layout 1 log common y of 10 - Attachment 10
> 
>       All front side FEE64 1.8.W waveforms - 20us range - Attachment 11
>       All back side FEE64 1.8.W waveforms - 20us range - Attachment 12
> 
>       All front side FEE64 1.8.W waveforms - 2ms range - Attachment 13
>       All back side FEE64 1.8.W waveforms - 2m range - Attachment 14
> 
>       FEE64 nnaida13 1.*.W waveforms - Ibias LF 0xf - 2ms range - Attachment 15
> 
>       Statistics after switch and thresholds lowered - Attachment 16
>       Pulser widths measured. On boards changed 10 out of 12 FEEs show greater than 10% improvements
>       11 out of 24 total show greater than 10%
>       FEE stats post pulser width - Attachment 17
> 
>       New ASIC settings DB key: 2017Oct23-14.38.03
>       Slow comparator 0xa for all FEE64s and all ASICs
>       IBias LF feedback at 0xf for all front side FEEs except nnaida 2 which is at 0xc
>       IBias LF feedback at 0xb for all back side FEEs
>       ASIC settings DB file checked OK
> 
>       Ambient temperature +23.3 deg C, D.P +7.1 deg C,  R.H. 35.2%
>       Julabo FL11006 set point +20.0 deg C, water le
> 
> 15:38 Merger and TapeServer screenshots - Attachments 17-18
>
  135   Mon Dec 14 19:25:35 2015 AE, CG ,PhongMerger/TapeServer trouble report
Sat. 12/12/15

Working with a 8-FEE setup (as in https://elog.ph.ed.ac.uk/AIDA/133). We could open a file with the 
TapeServer. Merger also seems to be working (cursor spinning, and merge statistics ~ok). However the 
TapeServer doesn’t write anything to disk (just empty an file).

A few observations; I only took ‘systematic’ notes after we had reduced the number of FEEs to 8, but it's 
likely we were also facing the same problems before with a full system:

+ When enabling data transfer from NNAIDA modules (in Run Control page), it switches to the right mode for 
all FEEs except for NNAIDA5. NNAIDA5 is the master FEE. The merger also doesn’t show any stats for NNAIDA5 
(but stats look ok for all other FEEs). The spectra for NNAIDA5 look ok, so the FEE is generating data.

+ A few times we observe the following behavior: DAQ running without trouble. We start Merger+TapeServer, 
and when attempting some actions in RunControl (start-stop, or reset), we loose the ability to communicate 
with a number of FEEs. These FEEs can not perform a ReSYNC, and the only way to get them back to life is 
going through a power cycle. It seems like the TaperServer or Merger is somehow upsetting the FEEs or the 
DAQ. We can still ‘ping’ all FEEs, even after the ‘connect failed’ message. 

Attached are a few screenshots, and the text in the terminals for Merger and TapeServer. The following is 
the error returned by one FEE when we can’t ReSync them:


STATE for nnaida3 returned with an error
connect failed 
NONE
connect failed 
    while executing
"::http::geturl http://nnaida3:8015/DataAcquisitionControlServer -headers {} -type text/xml -timeout 20000 
-query {<?xml version="1.0" encoding="UTF-8"..."
    ("eval" body line 1)
    invoked from within
"eval [list ::http::geturl $url] $args"
    (procedure "::http::geturl_followRedirects" line 4)
    invoked from within
"::http::geturl_followRedirects http://nnaida3:8015/DataAcquisitionControlServer -headers {} -type 
text/xml -timeout 20000 -query {<?xml version="1.0" ..."
    ("eval" body line 1)
    invoked from within
"eval ::http::geturl_followRedirects [list $url]  -headers [list $local_headers]  -type text/xml  -timeout 
$timeout  -query [list $request]  $local_pro..."
    (procedure "::SOAP::Transport::http::xfer" line 61)
    invoked from within
"$transport $procVarName $url $req"
    (procedure "::SOAP::invoke" line 18)
    invoked from within
"::SOAP::invoke ::SOAP::_DataAcquisitionControlClient__GetState"
    ("eval" body line 1)
    invoked from within
"eval ::SOAP::invoke ::SOAP::_DataAcquisitionControlClient__GetState $args"
    (procedure "DataAcquisitionControlClient__GetState" line 1)
    invoked from within
"DataAcquisitionControlClient__GetState"
GET OPTION for nnaida3 returned with an error
connect failed 
NONE
connect failed 
    while executing
"::http::geturl http://nnaida3:8015/DataAcquisitionControlServer -headers {} -type text/xml -timeout 60000 
-query {<?xml version="1.0" encoding="UTF-8"..."
    ("eval" body line 1)
    invoked from within
"eval [list ::http::geturl $url] $args"
    (procedure "::http::geturl_followRedirects" line 4)
    invoked from within
"::http::geturl_followRedirects http://nnaida3:8015/DataAcquisitionControlServer -headers {} -type 
text/xml -timeout 60000 -query {<?xml version="1.0" ..."
    ("eval" body line 1)
    invoked from within
"eval ::http::geturl_followRedirects [list $url]  -headers [list $local_headers]  -type text/xml  -timeout 
$timeout  -query [list $request]  $local_pro..."
    (procedure "::SOAP::Transport::http::xfer" line 61)
    invoked from within
"$transport $procVarName $url $req"
    (procedure "::SOAP::invoke" line 18)
    invoked from within
"::SOAP::invoke ::SOAP::_DataAcquisitionControlClient__GetOption NetVar=NetVar.DACQ.Chistenable"
    ("eval" body line 1)
    invoked from within
"eval ::SOAP::invoke ::SOAP::_DataAcquisitionControlClient__GetOption $args"
    (procedure "DataAcquisitionControlClient__GetOption" line 1)
    invoked from within
"DataAcquisitionControlClient__GetOption "$AcqGetOption""
  275   Sun Jun 5 01:20:39 2016 DK, AE, GL, PJWMerger wants first SYNC

Trying to start the AIDA DAQ now that the RI beam is (nearly) ready.  However, we cannot get merging to work.  But the merger status has "Merge State = GOing     :    paused     :    xfer enabled     :    want first SYNC"  but the last part ought to show "output paused" or after merge begins, "ready for SYNC".  We restarted the DAQ according to the procedure several times but this aspect of the messaging does not change, and the merging does not go live.

  610   Mon May 29 07:05:06 2017 TDMerger error messages and SEGV
Overnight and earlier today Obtained the following Merger error messages

Merge: Attempting resynchronisation to timestamp 26c3b4700056 or greater
Merge: Last sync pulse time: 26c3b46c00a0
Merge: Last timestamp seen: 26c3b4700055
Merge: Restarting at timestamp 26c3b47000a0
Merge: Timestamp (26c8062c00a0)[Q 5, Stream 5] is older than previous (26c8162bf81d) [Q 5, Stream 5]
Merge: Attempting resynchronisation to timestamp 26c8162bf81e or greater
Merge: Last sync pulse time: 26c8162800a0
Merge: Last timestamp seen: 26c8162bf81d
Merge: Restarting at timestamp 26c8162c00a0
Merge: Timestamp (26caa2109e35)[Q 5, Stream 5] is older than previous (26cab11e8415) [Q 5, Stream 5]
Merge: Attempting resynchronisation to timestamp 26cab11e8416 or greater
Merge: Last sync pulse time: 26cab11c00a0
Merge: Last timestamp seen: 26cab11e8415
Merge: Restarting at timestamp 26cab12000a0
Merge: Timestamp (26cbfe8400a0)[Q 5, Stream 5] is older than previous (26cc0e83f4e5) [Q 5, Stream 5]
Merge: Attempting resynchronisation to timestamp 26cc0e83f4e6 or greater
Merge: Last sync pulse time: 26cc0e8000a0
Merge: Last timestamp seen: 26cc0e83f4e5
Merge: Restarting at timestamp 26cc0e8400a0
Merge: Timestamp (26cf059000a0)[Q 5, Stream 5] is older than previous (26cf158ffb05) [Q 5, Stream 5]
Merge: Attempting resynchronisation to timestamp 26cf158ffb06 or greater
Merge: Last sync pulse time: 26cf158c00a0
Merge: Last timestamp seen: 26cf158ffb05
Merge: Restarting at timestamp 26cf159000a0
Merge: Timestamp (26cf899c00a0)[Q 5, Stream 5] is older than previous (26cf999bf9ad) [Q 5, Stream 5]
Merge: Attempting resynchronisation to timestamp 26cf999bf9ae or greater
Merge: Last sync pulse time: 26cf999800a0
Merge: Last timestamp seen: 26cf999bf9ad
Merge: Restarting at timestamp 26cf999c00a0
Merge: Timestamp (26dc07cac965)[Q 5, Stream 5] is older than previous (26dc16e6255d) [Q 5, Stream 5]
Merge: Attempting resynchronisation to timestamp 26dc16e6255e or greater
Merge: Last sync pulse time: 26dc16e400a0
Merge: Last timestamp seen: 26dc16e6255d
Merge: Restarting at timestamp 26dc16e800a0
Merge: Timestamp (26e035dac9cd)[Q 5, Stream 5] is older than previous (26e043cef5cd) [Q 5, Stream 5]
Merge: Attempting resynchronisation to timestamp 26e043cef5ce or greater
Merge: Last sync pulse time: 26e043cc00a0
Merge: Last timestamp seen: 26e043cef5cd
Merge: Restarting at timestamp 26e043d000a0
Merge: Timestamp (26e20f9000a0)[Q 5, Stream 5] is older than previous (26e21f8ffbdd) [Q 5, Stream 5]
Merge: Attempting resynchronisation to timestamp 26e21f8ffbde or greater
Merge: Last sync pulse time: 26e21f8c00a0
Merge: Last timestamp seen: 26e21f8ffbdd
Merge: Restarting at timestamp 26e21f9000a0
Merge: Timestamp (26e2817000a0)[Q 5, Stream 5] is older than previous (26e2916ffd25) [Q 5, Stream 5]

 :
 :
 :

This afternoon obtained the following error messages

MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Program (23229): check_for_data: index 6, unexpected header sequence number in pending data, found
787625409, last 27437
MERGE Program (23229): check_for_data: index 7, unexpected header sequence number in pending data, found
1814152641, last 82834
MERGE Data Link (23172): Received SEGV signal; exiting.
MERGE Server: Received signal 17.

Analysis of final two files of R33 - see attachments 1 & 2
Note idle time >> dead time (defined by PAUSE/RESUME) for nnaida5

Offline analysis R33_52 confirms ADC & disc data are synchronised - see attachments 3 & 4
  618   Wed May 31 16:43:52 2017 TDMerger & TapeServer server startup scripts modified
aidas1> cd /MIDAS/Merger/AIDA
aidas1> ls
GS_configuration   GS_configuration.BAK-030516  Startup~
GS_configuration~  Startup                      Startup.BAK-010617
aidas1> diff Startup Startup.BAK-010617
63,64c63
< # 1 June 2017 - increase sleep time from 1000000 (11.6 days) to 150000000 (4.75 years)
< sleep 150000000
---
> sleep 1000000
aidas1> cd /MIDAS/TapeServer/startup
aidas1> ls
AidaTapeServer  AidaTapeServer~  AidaTapeServer.BAK-010617
aidas1> diff AidaTapeServer AidaTapeServer.BAK-010617
24,26c24
< # 1 June 2017 - increase sleep time from 1000000 (11.6 days) to 150000000 (4.75 years)
< sleep 150000000
< 
---
> sleep 1000000
  65   Wed Apr 22 07:28:25 2015 AE (PM)Mechanical setup instructions: AIDA+EURICA at F11
Attached are Paul Morrall's instructions on how to move new AIDA support table into place, for a setup with 
EURICA at F11 (March 2015).
  312   Sun Jun 12 19:19:22 2016 DKMachine Time End

3:13 operators call to tell us the machine time is over.  No more EURICA!

          AIDA62_23 ought to be the last file with legitimate physics data

          We will leave the DAQ running overnight as a b/g run.

Goto page Previous  1, 2, 3 ... 31, 32, 33 ... 44, 45, 46   Next  
ELOG V3.1.3-7933898