AIDA GELINA BRIKEN nToF CRIB ISOLDE CIRCE nTOFCapture DESPEC DTAS EDI_PSA 179Ta CARME StellarModelling DCF K40
  AIDA, Page 10 of 46  ELOG logo
ID Date Author Subject
  765   Mon Nov 5 01:40:10 2018 CA, CB, TD, OH, PJWMonday 5 November
10.40 Lowered slow comparator threshold from 100 to 10
      Good event statistics ok (attachment 1)
      System wide checks ok, except for nnaida7 ADC calibration failure
      FEE64 temperatures ok (attachment 2)
      Bias/Leakage currents ok (attachment 3)
      1.0.L spectra obtained, look ok (attachment 4)
      DAQ stopped (R16)
 
10.49 Slow comparator threshold raised from 10 to 100
      DAQ restarted (R17)
      Merger and TapeServer ok, no error messages in terminals
      Good event statistics ok (attachment 5)
      nnaida2 is now at a good rate, after fixing kapton connection.

15:09 DAQ still running (Oct18/R17_0)
      Merger and TapeServer ok
      System wide checks ok, except for nnaida7 ADC calibration failure
      FEE64 temperatures ok (attachment 6)
      Bias/Leakage currents ok (attachment 7)
      Good event statistics ok (attachment 8)


18.48 good event statistics OK *except* nnaida17 c. 400k
      ASIC Control -> Check ASIC Control
      *all* FEE64 good events statistics now OK
      DAQ stop - current file Oct18/R17_20

18.50 DAQ start - file Oct18/R18
      BNC PB-4 OFF
      LEC/MEC slow comparator 0x64

      alpha background run

18.54 all system wide checks OK *except* nnaida7 fails ADC calibration

      detector biases & leakage currents OK - see attachments 9 & 10
      FEE64 temperatures OK - see attachment 11
      good event statistics OK - see attachment 12
      Merger - Link16 errors reported - see attachment 13
      TapeServer OK - see attachment 14

      1.8.W spectra - attachment 15 & 16


      
  764   Sun Nov 4 04:31:49 2018 TDOffline analysis R4
Offline analysis of Oct18/R4_0 to _2

alpha background and pulser

Attachments 1-24 - per FEE64 ADC spectra (32k->4k)

Attachments 25-26 - expanded view pulser peaks FEE64 #2 & #17 & #24
                    anomalously small pulser peak widths observed -> possible LH Kapton PCB issue

Attachements 27-28 - sort program (pulser.f) and variables (variables.dat)
  762   Sun Nov 4 01:11:41 2018 CA, CB, TD, OH, PJWSunday 4 November
10.03 DAQ continues OK - current file Oct18/R16_9
      FEE64 LEC/MEC slow comparator threshold 0x64
      beam background

10.05 all system wide checks OK *except* nnaida7 fails ADC calibration

      detector biases & leakage currents OK - see attachments 1 & 2

      FEE64 temperatures OK - see attachment 3
      good event statistics OK *except* nnaida2 - see attachment 4
      nnaida2 rate due to one hot channel as before

      Merger OK - see attachment 5
      TapeServer OK - see attachment 6
      no errors or warnings shown by server terminal sessions 

  
      1.8.L spectra - see attachments 7 & 8

       N.B. evidence of minimally ionising particles (presumably protons) from beam to F11 E detector
            ~10-15cm Pb bricks were placed within BRIKEN (CH2)n stack to stop heavy particles

      1.8.W spectra 20us FSR - see attachments 9 & 10

      online analysis with MIDASsort

      attachment 11 - per DSSSD m_p versus m_n
                 12 - per DSSSD x versus y
                 13 - per DSSSD E_p versus E_n
                 14 - channel versus E_p
                 15 - hit pattern (ASIC & DSSSD order)

14.35 DAQ continues OK - current file Oct18/R16_11
      FEE64 LEC/MEC slow comparator threshold 0x64
      beam background

14.35 all system wide checks OK *except* nnaida7 fails ADC calibration

      detector biases & leakage currents OK - see attachments 15 & 16

      FEE64 temperatures OK - see attachment 17
      good event statistics OK *except* nnaida2 - see attachment 18
      nnaida2 rate due to one hot channel as before

14.51 analysis file Oct18/R16_10 - attachment 19
  761   Sat Nov 3 00:49:22 2018 TDOffline analysis R15 - beam background
Offline analysis of Oct18/R15_0 to 7


BNC PB-4 pulser OFF, LEC/MEC slow comparator 0x64 (~1MeV)
background (presumably high energy, low Z particles) from beam stopping in F11 E detector upstream 
of AIDA and BRIKEN - ~10-15cm Pb shielding installed in BRIKEN to stop heavy opns reaching AIDA

 *** ENTRY finish

 *** data items: 2088704000 (  42315.52 Hz)
 *** ADC events: 1236420713 (  25048.92 Hz)
 *** time warps:          0 (      0.00 Hz)
 *** DSSSD # 1 count:     12557 old count:     12557 dt:  63395.57 s  LEC rate:      0.20 Hz
 *** DSSSD # 2 count:     10332 old count:     10332 dt:  63395.57 s  LEC rate:      0.16 Hz
 *** DSSSD # 3 count:      9644 old count:      9644 dt:  63395.57 s  LEC rate:      0.15 Hz
 *** DSSSD # 4 count:     10024 old count:     10024 dt:  63395.57 s  LEC rate:      0.16 Hz
 *** DSSSD # 5 count:     11035 old count:     11035 dt:  63395.57 s  LEC rate:      0.17 Hz
 *** DSSSD # 6 count:     11684 old count:     11684 dt:  63395.57 s  LEC rate:      0.18 Hz
 *** DSSSD # 1 count:         0 old count:         0 dt:  63395.57 s  HEC rate:      0.00 Hz
 *** DSSSD # 2 count:         0 old count:         0 dt:  63395.57 s  HEC rate:      0.00 Hz
 *** DSSSD # 3 count:         0 old count:         0 dt:  63395.57 s  HEC rate:      0.00 Hz
 *** DSSSD # 4 count:         0 old count:         0 dt:  63395.57 s  HEC rate:      0.00 Hz
 *** DSSSD # 5 count:         0 old count:         0 dt:  63395.57 s  HEC rate:      0.00 Hz
 *** DSSSD # 6 count:         0 old count:         0 dt:  63395.57 s  HEC rate:      0.00 Hz

 *** ENTRY finish ends

attachment 1 - per DSSSD m_p versus m_n
           2 - per DSSSD x versus y
           3 - per DSSSD E_p versus E_n
           4 - DSSSD hit pattern (ASIC & DSSSD order) LEC & HEC events (note HEC events from ASIC check loads)
           5 - DSSSD hit pattern (ASIC & DSSSD order) expanded view of LEC events

DSSSD configuration looks OK. With improved statistics (cf alpha background) some missing
strips identifiable.
  760   Sat Nov 3 00:37:29 2018 CA, CB, TD, OHSaturday 3 November
09.37 DAQ continues OK - data transfer disabled
      FEE64 LEC/MEC slow comparator threshold 0xa

09.39 all system wide checks OK *except* nnaida7 fails ADC calibration

      detector biases & leakage currents OK - see attachments 1 & 2

      FEE64 temperatures OK - see attachment 3
      good event statistics OK *except* nnaida17 c. 400k - see attachment 4
      nnaida2 rate due to one hot channel as before
  
      1.8.L spectra - see attachments 5 & 6

       N.B. evidence of minimally ionising particles (presumably protons) from beam to F11 E detector
            ~10-15cm Pb bricks were placed within BRIKEN (CH2)n stack to stop heavy particles

      1.8.W spectra 20us FSR - see attachments 7 & 8


10.10 LEC/MEC slow comparator from 0xa to 0x64
      3x ASIC check load
      DAQ stop
      all histograms zero'd
      enable data transfer
      DAQ start - file Oct18/R16

      beam background

      good event statistics OK *except* nnaida2 20kHz - see attachment 9
  759   Fri Nov 2 01:56:00 2018 TD[How To] Recover from unable to connect to nnaida ...
If you obtain the 'unable to connect' from one, or more, of the FEE64s try the
following procedures *before* power-cycling/rebooting all of the FEE64s:

1) 'unable to connect' whilst DAQ is going using the Merger
   
   https://elog.ph.ed.ac.uk/AIDA/303

2) determine whether multiple FEE64s are unable to connect

aidas1> ping nnaida1
PING nnaida1 (10.1.1.1) 56(84) bytes of data.
64 bytes from nnaida1 (10.1.1.1): icmp_seq=1 ttl=64 time=2.81 ms
64 bytes from nnaida1 (10.1.1.1): icmp_seq=2 ttl=64 time=2.81 ms
64 bytes from nnaida1 (10.1.1.1): icmp_seq=3 ttl=64 time=2.80 ms
^C
--- nnaida1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2029ms
rtt min/avg/max/mdev = 2.807/2.812/2.817/0.061 ms
aidas1> ping nnaida2
PING nnaida2 (10.1.1.2) 56(84) bytes of data.
64 bytes from nnaida2 (10.1.1.2): icmp_seq=1 ttl=64 time=2.87 ms
64 bytes from nnaida2 (10.1.1.2): icmp_seq=2 ttl=64 time=2.81 ms
64 bytes from nnaida2 (10.1.1.2): icmp_seq=3 ttl=64 time=2.83 ms
64 bytes from nnaida2 (10.1.1.2): icmp_seq=4 ttl=64 time=2.81 ms
^C
--- nnaida2 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3350ms
rtt min/avg/max/mdev = 2.810/2.833/2.874/0.069 ms

  :
  :
   
  etc

If you find that you are unable to ping a group (or groups) of 8x FEE64s
it is probable that the issue is a fuse failure(s) in the USB-controlled
ac mains relay. It will be necessary to replace the fuse(s) and perform a
cold start of the FEE64s

 https://elog.ph.ed.ac.uk/AIDA/418

3) If you are able to ping all FEE64s telnet to the FEE64 you are unable to
connect to, login as root, and issue a reboot command

aidas1> telnet nnaida2
Trying 10.1.1.2...
Connected to nnaida2.
Escape character is '^]'.

Linux 2.6.31 (localhost) (08:23 on Thursday, 01 November 2018)

login: root
Password: 
Last login: Mon May 23 16:02:30 from myserver
-bash-3.2# ls
a.out  ld_aidamem.csh  xaida
-bash-3.2# ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  0 Oct28 ?        00:00:07 init [3]  
root         2     0  0 Oct28 ?        00:00:00 [kthreadd]
root         3     2  0 Oct28 ?        00:00:00 [ksoftirqd/0]
root         4     2  0 Oct28 ?        00:00:00 [watchdog/0]
root         5     2  0 Oct28 ?        00:00:00 [events/0]
root         6     2  0 Oct28 ?        00:00:00 [khelper]
root         7     2  0 Oct28 ?        00:00:00 [async/mgr]
root         8     2  0 Oct28 ?        00:00:00 [kblockd/0]
root         9     2  0 Oct28 ?        00:00:00 [kseriod]
root        10     2  0 Oct28 ?        00:00:00 [khungtaskd]
root        11     2  0 Oct28 ?        00:00:00 [pdflush]
root        12     2  0 Oct28 ?        00:00:00 [pdflush]
root        13     2  0 Oct28 ?        00:00:00 [kswapd0]
root        14     2  0 Oct28 ?        00:00:00 [aio/0]
root        15     2  0 Oct28 ?        00:00:00 [nfsiod]
root        20     2  0 Oct28 ?        00:00:00 [81400400.hd-xps]
root        21     2  0 Oct28 ?        00:00:00 [81400000.xps-sp]
root        22     2  0 Oct28 ?        00:00:00 [kpsmoused]
root        25     2  0 Oct28 ?        00:00:00 [rpciod/0]
root        54     1  0 Oct28 ?        00:00:00 /sbin/udevd -d
root       226     1  0 Oct28 ?        00:00:00 syslogd -m 0
root       229     1  0 Oct28 ?        00:00:00 klogd -x
root       259     1  0 Oct28 ?        00:00:00 rpcbind
root       275     1  0 Oct28 ?        00:00:00 xinetd -stayalive -pidfile /var/
root       377     1 98 Oct28 ?        3-22:18:16 ./AidaExecV8
root       392     1  0 Oct28 ttyS0    00:00:00 /sbin/mingetty --noclear console
root       404   275  0 08:23 ?        00:00:00 in.telnetd: myserver
root       405   404  0 08:23 ?        00:00:00 login -- root            
root       406   405  1 08:23 ttyp0    00:00:00 -bash
root       427   406  6 08:23 ttyp0    00:00:00 ps -ef
-bash-3.2# reboot

Broadcast message from root (ttyp0) (Thu Nov  1 08:25:20 2018):

The system is going down for reboot NOW!
-bash-3.2# Connection closed by foreign host.

Wait 5 minutes for the filesystem to be mounted and the FEE64 boot sequence
to complete. 

Switch to Desktop 1 and re-select 'Data Acquisition Run Control' tab and select 
Update. The status of the FEE64 you were unable to connect to should now 'undefined'.

Follow cold start sequence steps 7-10 and 15

 https://elog.ph.ed.ac.uk/AIDA/418
  758   Fri Nov 2 01:19:49 2018 TDOffline analysis R9 - alpha background
 
Offline analysis of Oct18/R9_0 to 17 (c. 46h data)

alpha background, BNC PB-4 pulser OFF, LEC/MEC slow comparator 0x64 (~1MeV)

*** ENTRY finish

 *** data items:  404616704 (  10823.34 Hz)
 *** ADC events: ********** ( -30856.57 Hz)
 *** time warps:          0 (      0.00 Hz)
 *** DSSSD # 1 count:      1004 old count:      1004 dt: 160265.10 s  LEC rate:      0.01 Hz
 *** DSSSD # 2 count:       423 old count:       423 dt: 160265.10 s  LEC rate:      0.00 Hz
 *** DSSSD # 3 count:       364 old count:       364 dt: 160265.10 s  LEC rate:      0.00 Hz
 *** DSSSD # 4 count:       363 old count:       363 dt: 160265.10 s  LEC rate:      0.00 Hz
 *** DSSSD # 5 count:       455 old count:       455 dt: 160265.10 s  LEC rate:      0.00 Hz
 *** DSSSD # 6 count:       590 old count:       590 dt: 160265.10 s  LEC rate:      0.00 Hz
 *** DSSSD # 1 count:         0 old count:         0 dt: 160265.10 s  HEC rate:      0.00 Hz
 *** DSSSD # 2 count:         0 old count:         0 dt: 160265.10 s  HEC rate:      0.00 Hz
 *** DSSSD # 3 count:         0 old count:         0 dt: 160265.10 s  HEC rate:      0.00 Hz
 *** DSSSD # 4 count:         0 old count:         0 dt: 160265.10 s  HEC rate:      0.00 Hz
 *** DSSSD # 5 count:         0 old count:         0 dt: 160265.10 s  HEC rate:      0.00 Hz
 *** DSSSD # 6 count:         0 old count:         0 dt: 160265.10 s  HEC rate:      0.00 Hz

 *** ENTRY finish ends

attachment 1 - per DSSSD m_p versus m_n
           2 - per DSSSD x versus y
           3 - per DSSSD E_p versus E_n
           4 - DSSSD channel versus E_p

           5 - implantdecay4.f
           6 - titles
           7 - variables 
               N.B. ADC offsets not set, 0 by default
                    | E_p - E_n | < 2000


DSSSD configuration looks OK. No obvious issues with missing ASICs  
  757   Fri Nov 2 00:57:24 2018 CA, CB, TD, OHFriday 2 November
09.58 DAQ continues OK - current file Oct18/R15_6
      alpha background run

09.20 all system wide checks OK *except*
        nnaida7 fails ADC calibration
        SYNC errors since baseline - see attachment 1 - baseline not zero'd following reboot of nnaida2 and
setup of all FEE64s yesterday
        SYNC errors OK following new baseline check

      detector biases & leakage currents OK - see attachments 2 & 3

      FEE64 temperatures OK - see attachment 4
      good event statistics OK - see attachment 5
      nnaida2 rate due to one hot channel

      Merger OK - see attachment 6
      TapeServer OK - see attachment 7
      no errors or warnings shown by server terminal sessions 

      1.8.L spectra - see attachments 8 & 9

       N.B. evidence of minimally ionising particles (presumably protons) from beam to F11 E detector
            ~10-15cm Pb bricks were placed within BRIKEN (CH2)n stack to stop heavy particles

      1.8.W spectra - see attachments 10 & 11 - trace updates for 9/12 p+n FEE64s have stopped, n+n FEE64s OK

15.54 DAQ continues - current file Oct18/R15_15
      nnaida17 good event rate c. 400kHz


15.57 FEE64 reset, setup, system wide checks
      all ASIC clocks sync'd using ReSYNC
      all spectra zero'd
      ASIC check load
      
      DAQ go - data transfer disabled

      good events statistics - attachment 12

      1.8.W spectra - see attachments 13 & 14
      changed LED threshold 7700 -> 8500 for 9/12 p+n FEE64s
  756   Thu Nov 1 00:24:23 2018 TD, OH, CAThursday 1 November
09.24 DAQ continues OK - current file Oct18/R12_116
      alpha background run

09.20 all system wide checks OK *except* nnaida7 fails ADC calibration

      detector biases & leakage currents OK - see attachments 1 & 2

      FEE64 temperatures OK - see attachment 3
      good event statistics OK *except* nnaida17 c. 400k - see attachment 4
      nnaida2 rate due to one hot channel as before

      Merger - Link16 errors reported - see attachment 5
      TapeServer OK - see attachment 6

      problem with nnaida17 rate resolved with ASIC check load - see attachment 7
  
      1.8.L spectra - see attachments 7 & 8

       N.B. evidence of minimally ionising particles (presumably protons) from beam to F11 E detector
            ~10-15cm Pb bricks were placed within BRIKEN (CH2)n stack to stop heavy particles

      1.8.W spectra 20us FSR - see attachments 9 & 10

09.33 DAQ stop - file Oct18/R12_117
      All Oct18/R12_* files deleted

09.34 all spectra zero'd
      DAQ start - file Oct18/R13
      alpha background run

      all FEE64 good events statistics OK - see attachment 11

15.11 DAQ continues ok, on file Oct18/R13_2 (alpha run)
      system wide checks ok, except nnaida7 failing ADC calibration
      FEE64 temperatures ok (attachment 13)
      detector bias and leakage currents ok (attachment 14)
      good event statistics ok, except nnaida2 due to one hot channel (attachment 15)
      Merger and TapeServer ok, no errors in terminal

15.20 DAQ stop - file Oct18/R13_2
      
15.24 Lowered slow comparator threshold from 100 to 10
      good event statistics ok (attachment 16)

15.42 took waveforms, triggering ok (attachment 17)
      raised slow comparator threshold to 100 from 10
      start DAQ (R14)
      nnaida22, no statistics showing (see attachment 18)
      subsequent failure to connect to nnaida2

16:44 fixed by rebooting nnaida2 *only*, nnaida2 reset, setup all FEE64s
      slow comparator threshold re-raised from 10 to 100, as default was restored
      DAQ and alpha run restarted (R15)
      Merger and TapeServer ok, no errors in terminal
      good event statistics all ok, except for nnaida2 hot channel (attachment 19)
      
      
  755   Wed Oct 31 00:18:12 2018 TD, OH, CAWednesday 31 October
09.18 DAQ continues OK - current file Oct18/R10_7
      alpha background run

09.20 all system wide checks OK *except* nnaida7 fails ADC calibration

      detector biases & leakage currents OK - see attachments 1 & 2

      FEE64 temperatures OK - see attachment 3
      good event statistics OK - see attachment 4
      nnaida2 rate due to one hot channel

      Merger OK - see attachment 5
      TapeServer OK - see attachment 6
      no errors or warnings shown by server terminal sessions 

      1.8.L spectra - see attachments 7 & 8

       N.B. evidence of minimally ionising particles (presumably protons) from overnight(?) BigRIPs tuning
            to F11 E detector upstream of BRIKEN & AIDA and ongoing beam to F11 E detector
            
            ~10-15cm Pb bricks were placed within BRIKEN (CH2)n stack to stop heavy particles

09.30 JLT reports ~3kHz neutrons in BRIKEN, ~12kHz in F11

      1.8.W spectra 20us FSR - see attachments 9 & 13
      1.8.W spectra 200us FSR - see attachments 10 & 14
      1.8.W spectra 2ms FSR - see attachments 11 & 15
      1.8.W spectra 20ms FSR - see attachments 12 & 16

13.17 Run stopped on file R10_8 (13.16)

13.21 Lowered slow comparator threshold from 100 to 10
      good event statistics ok (attachment 17)

13.36 Took waveforms, all ok (attachment 18)

13.55 Raised slow comparator threshold back to 100 from 10
      Started new alpha run file R12
      TapeServer and Merger ok
      System wide checks ok, except FEE7 fails ADC calibration
      FEE64 temperatures ok (attachment 19)
      Leakage currents ok (attachment 20)
     


      
  754   Mon Oct 29 23:40:41 2018 TD, OH, CATuesday 30 October
08.40 DAQ continues OK - current file Oct18/R9_15

08.42 all system wide checks OK *except* nnaida7 fails ADC calibration

      detector biases & leakage currents OK - see attachments 1 & 2

      FEE64 temperatures OK - see attachment 3
      good event statistics OK - see attachment 4
      nnaida2 rate due to one hot channel

      Merger OK - see attachment 5
      TapeServer OK - see attachment 6
      no errors or warnings shown by server terminal sessions 

      1.8.L spectra - see attachments 7 & 8
      1.8.W spectra 20us FSR - see attachments 9 & 10

14.43 system wide checks ok, except nnaida7 failing adc calibration

      detector bias/leakage currents all ok (attachment 11)

      FEE64 temperatures ok (attachment 12)

      good event statistics ok

      no errors or warnings in terminal for Merger and TapeServer

15.05 Stopped alpha run - file R9_18 at 14.50
      lowered slow comparator threshold from 100 to 10
      good event statistics were ok, but with FEE1 at 400k (attachment 13)
      did check on ASICs, stats were behaving ok (attachment 14)
      took waveforms, were not initially triggering on p+n side. Change threshold from 8000 to 8500 (attachment 15, 16)

15.12 start new file (alpha run, R10)
      changed threshold back to 100 from 10
      Merger and TapeServer ok
      

   
      
      
  753   Sun Oct 28 23:34:28 2018 TD, OH, CAMonday 29 October
08.34 DAQ continues OK - current file Oct18/R9_6

08.37 all system wide checks OK *except* nnaida7 fails ADC calibration

      detector biases & leakage currents OK - see attachments 1 & 2

      FEE64 temperatures OK - see attachment 3
      good event statistics OK - see attachment 4
      nnaida2 rate due to one hot channel

      Merger OK - see attachment 5
      TapeServer OK - see attachment 6
      no errors or warnings shown by server terminal sessions 

      1.8.L spectra - see attachments 7 & 8
      1.8.W spectra 20us FSR - see attachments 9 & 10

08.50 temperature +23.8 deg C, RH 38.0%, d.p. +8.6 deg C
      Julabo FL11006 set point +20 deg C, actual +20.0 deg C, water level c. 75%

11.10 DAQ continues OK - current file Oct18/R9_7

15.18 system wide checks OK, except nnaida7 ADC calibration failure
      leakage currents OK (attachment 11)
      FEE64 temperatures okay (attachment 12)

15.23 good event statistics OK (attachment 13)
      Merger and TapeServer OK (attachments 14 and 15)
      DAQ still running OK - Oct18/R9_8
      
15.54 Chiller refilled
  752   Sun Oct 28 01:33:34 2018 CA, OHSunday 28 October
09:33 DAQ still running, file R4_11. 
      Bias/leakage ok (attachment 1)

      Leakage currents whilst still acceptable are up around 400nA on yesterday
      Have checked all detectors and rates and pulser peak widths are still good across the board
      Will keep an eye on but for now it appears to be ok
      BRIKEN is currently doing a Cf source test so could be a result of charging from the source

      Humidity: 33.1, Temperature: 24.6, Dew point: 7.1
      FEE temperatures ok (attachment 2)

09:38 System wide checks ok

10:25 Pulser peak widths all checked
      Widths comparable to 26th apart from 24 which is much wider from ~70 -> ~120
      Waveforms, layout 7 and 8 - attachments 3 and 4

10:37 Started new run on file R5 at time 10:37
      Threshold changed back to 100 from 10
      Pulser on at 2Hz

16:34 Stopped R5 alpha run
      Lowered thresholds to 0xa to produce higher rate data to test AIDA<->BRIKEN synchronisation
      Started file R6 to do this
      DAQ crashed on start ASIC readout with unable to connect to nnaida19 message appearing.
      Could not clear despite attempts at restarting the merge if that was the cause.

      FEE power cycle

17:03 DAQ recovered, R7 started
      AIDA did not crash
      BRIKEN started Ge fill cycle
      AIDA did not crash during or after

17:17 R7 Stopped sync program is being worked on by Jorge,

17:32 Thresholds raised back to 100 from 10
      Pulser switched off
      R8 started
      leakage current dropping down (attachment 5)
  751   Sat Oct 27 01:22:43 2018 CA, OHSaturday 27 October
09:22 DAQ found crashed. Appears to have crashed at 20:23 last night whilst writing R3_0 to file.
      Run Control is non responsive
      Leakage currents okay - attachment 1

      Unable to ping any of the nnaida FEEs
      Restarted merger in an attempt to clear links still no response.
      Have powered down FEE64
      
      Replaced blown fuse
      Checked temperatures of FEEs, all okay - attachment 2
      

10:19 Restarted alpha run, write to file Oct18/R4.
      System wide checks - all ok.
      Slow comparator threshold changed to 100 from 10

14:12 FEE temperatures ok (attachment 3)
      Bias/leakage currents ok (attachment 4)
      System wide checks ok
      Alpha run on R4_1

17.45 DAQ continues OK, file Oct18/R4_3
  750   Fri Oct 26 03:28:10 2018 OH, CAFriday 26 October
October 25th:
In anticipation of losing access in the evening on arrival at RIKEN the following was done:
- Chiller was refilled
- All power cables reconnected after being removed in October for the RIKEN power down
- All NIM bins, fan trays and RPis were turned on
- 50MHz AIDA clock was set up and connected
- Remote connections to AC Mains relay and RPi for CAEN bias was tested
- Chiller was powered up

October 26th:

11:31 Still have access to the F11 area due to a bigRIPS delay.
      Double checked wiring from yesterday and all still seems good
      Chiller is still running with set point at +20 deg C 

11:36 FEEs powered up. All 24 mounted

11:57 Bias and leakage currents all ok - attachment 1
      FEE temperatures all ok - attachment 2
      System wide checks all ok

13:26 Bias and leakage currents ok - attachment 3
      FEE temperatures all ok - attachment 4
      System wide checks all ok

14:00 Stats are ok - attachment 5
      Appear similar to last time: https://elog.ph.ed.ac.uk/AIDA/747
      However FEE24 and 24 appear much noisier than last time with rates around 100k

19:24 Pulser walkthrough run from 90000 to 10000 in steps of 10000 for R2
      Opened file Oct18/R3, alpha run, threshold at 100kev, pulser on.
      
  749   Wed Oct 3 10:47:06 2018 OHAnalysis of R2_0 -> R2_37 Alpha Run
Data was analysed using offsets calculated based on pulser walkthroughs carried out in June 2018
Offsets appear to still be good. Ex V Ey plots - attachment 1

Hit pattern across the FEEs looks good - attachment 2
x axis is (FEE-1)*64 + Channel

No obvious dead areas in x vs y plots either - attachment 3
  748   Wed Oct 3 07:09:03 2018 OH, TDCurrent FEE Configuration Map October 3rd 2018
             FEE Number
DSSD      A    B    C    D
  1       17   20   22   23
  2       18   19   21   24
  3       9    12   14   15
  4       10   11   13   16
  5       1    4    6    7
  6       2    3    5    8


    DSSD 1 3058-4  Top - Upstream
    DSSD 2 3058-3
    DSSD 3 2998-23
    DSSD 4 3058-5
    DSSD 5 3058-6
    DSSD 6 3058-7  Bottom - Downstream
  747   Wed Oct 3 01:48:21 2018 OH, TDWednesday 3 October
09.42 DAQ continues OK - current file Sep18/R2_36

09.42 all system wide checks OK *except* nnaida7 fails ADC calibration

      detector biases & leakage currents OK - see attachments 1 & 2

      FEE64 temperatures OK - see attachment 3
      good event statistics OK - see attachment 4
      nnaida2 rate due to one hot channel

      Merger OK - see attachment 5
      TapeServer OK - see attachment 6
      no errors or warnings shown by server terminal sessions 

      1.8.L spectra - see attachments 7 & 8
      1.8.W spectra 20us FSR - see attachments 9 & 10

09.46 ASIC check load (file Sep18/R2_36)
      Note for p+n junction strip FEE64s the 1.8.W spectra are only updating for nnaida2, 10 & 17 - these traces
look OK
      Other FEE64s show a noise transient and do not update - ASIC check load did not fix - probably requires
DAQ STOP/GO  

09.57 temperature +25.4 deg C, RH 39.2%, d.p. +10.6 deg C
      Julabo FL11006 set point +20 deg C, actual +20.0 deg C, water level c. 80%

13:17 FEE temperatures and system wide checks all ok
      Alpha run R2 stopped
      R2_0 -> R2_37

13:18 Thresholds set to 0xa from 0x64 for all FEEs and ASICs

13:22 R3_0 started
      Thresholds at 0xa and pulser on at 25Hz setting ~ 2Hz

      ASIC settings saved to EXPERIMENTS/AIDA/2018Oct03-13.26.58
      Merger screenshot - attachment 11
      Stats - attachment 12
      Layouts 7 and 8 - attachments 13 & 14 

13:31 R3_0 Stopped

13:34 R4_0 Started
      ASIC thresholds all at 0x9
      Merger screenshot - attachment 15
      Stats - attachment 16
      Layout 1 - attachment 17

13:38 R4_0 Stopped

13:40 R5_0 Started
      ASIC thresholds all at 0x8
      Merger screenshot - attachment 18
      Stats - attachment 19
      Layout 1 - attachment 20

13:45 R5_0 Stopped

13:49 R6_0 Started
      ASIC slow comp at 0x7
      Merger screenshot - attachment 21
      Stats - attachment 22
      Layout 1 - attachment 23

13:52 R6_1 Stopped

      R7_0 Started
      ASIC Slow comp at 0x6
      Merger screenshot - attachment 24
      Stats - attachment 25
      Layout 1 - attachment 26
      Tapeserver window screenshot - attachment 27
      Top window showing 72% CPU usage - attachment 28

15:06 Pulser measurements carried out with thresholds at 0xa
      Typical pulser peak width was ~70-80 with a couple above 100.
      Double peaking that was observed earlier in the week is no longer observed.
      Both in the FEEs and on a scope

      Backside pulser peaks - attachment 29
      Frontside pulser peaks - attachment 30
  746   Tue Oct 2 01:55:11 2018 OH, TDTuesday 2 October
09.54 DAQ continues OK - current file Sep18/R2_26

09.55 all system wide checks OK *except* nnaida7 fails ADC calibration

      detector biases & leakage currents OK - see attachments 1 & 2

      FEE64 temperatures OK - see attachment 3
      good event statistics OK - see attachment 4
      nnaida2 rate due to one hot channel

      Merger OK - see attachment 5
      TapeServer OK - see attachment 6
      no errors or warnings shown by server terminal sessions 

      1.8.L spectra - see attachments 7 & 8
      1.8.W spectra 20us FSR - see attachments 9 & 10
      
10.00 temperature +25.7 deg C, RH 29.6%, d.p. +6.4 deg C
      Julabo FL11006 set point +20 deg C, actual +20.0 deg C, water level c. 80%

10.03 ASIC check load (file Sep18/R2_26)
  745   Mon Oct 1 01:10:57 2018 OH, TDMonday 1 October
09.10 DAQ continues OK - current file Sep18/R2_16

09.15 all system wide checks OK *except* nnaida7 fails ADC calibration

      detector biases & leakage currents OK - see attachments 1 & 2

      FEE64 temperatures OK - see attachment 3
      good event statistics OK - see attachment 4
      nnaida2 rate due to one hot channel

      Merger OK - see attachment 5
      TapeServer OK - see attachment 6
      no errors or warnings shown by server terminal sessions 

      1.8.L spectra - see attachments 7 & 8
      1.8.W spectra 20us FSR - see attachments 9 & 10
      
09.24 temperature +25.5 deg C, RH 43.4%, d.p. +11.9 deg C
      Julabo FL11006 set point +20 deg C, actual +20.0 deg C, water level c. 75%

16.25 Julabo FL11006 refill to just below H indicator

      DAQ continues OK - current file Sep18/R2_19
ELOG V3.1.4-unknown