Image of interior of closed EURICA array with AIDA inserted.

25/02/15

1200: Swapped out old green power switch for new one from UoE (having previously tested them both yesterday and found them to work fine).

           With first switch connected and PuTTY session opened cannot get RPi to connect to it. See (bodged) screen shots below.

           Tried many permutations of trying to connect using ttyUSB0/1/2 with various permissions set, none yielded any results other than the error messages below.

1500: Swapped 1st UoE power switch out for the second. Plugged it straight in and it works... ???

           Later unplugged it and when reconnected saw the same errors as before. We got rid of these by plugging in the USB cable once the unit was powered on. Does this make any sense, or was it just luck?

           Carried out pulser walkthrough to file R22_0. Use same "best" settings as yesterday.

           Changed polarity of detector bias (from -100V to +100) and observed a similar leakage current as previously (11.5uA) and normal rates/spectra for each detector. No obvious changes from -ve bias polarity. GOOD.

1830: Looked at effect of supplying each FEE64 from various power supplies (as per request PC-S email on Sun 22nd Feb).

           Original configuration :     |  1   2  |  4   3  |  5   7  |  8   6  |    (all in one PSU)

           Changed to:     |  1   -  |  -    -  |  -    -  |  -   -  |       PSU#1

                                    |  3   -  |  -    -  |  -    -  |  -   -  |       PSU#2

                                    |  6   -  |  -    -  |  -    -  |  -   -  |       PSU#3

                                    |  8   -  |  -    -  |  -    -  |  -   -  |       PSU#4

                      with PSUs #3+4 completely unplugged apart from nnaida6+8. In PSUs #1+2, the remainder of the slots were connected as previously (apart from for nnaida3,6,8, in which case they were swapped for another FEE64 at random).

           Initially all PSUs were plugged into one half of the power switch, but the fuse blew. We replaced it with a spare fuse from the unused supply and distributed the PSUs 2 to each half of the switch.

           This configuration yielded larger FWHM than original set up  ---> worsened performance.

08/12/15 - Day 0.5

Test fitted new snout to snout base

     - the holes in the base of the just-shipped section all line up with those in the corresponding part just taken off.

     - however, the raised corners of the rear metal plate (around the rods), which sits in the downstream end of the Al section of the snout now presents a conflict.

     - the old snout sat inside the raised section but, now that the side length of the snout has been increased, there is a conflict between the snout and the raised portion (see attached photos), which means the snout will not seat properly in the base section.

Checked for electrical connection between cooling crate and FEE mezzanines

      - connection present in all FEEs with resistance < 2 Ohms for most, and the maximum at 1 kOhm for one of the FEE modules.

09/12/15

Moved AIDA back into rough experimental position, re-cabled setup and powered on complete system.

Chiller levels ok, no leaks after ~1hr of operation. Temperatures all remained low and stable.

 07/03/16

The new snout doesn't fit due to the screw holes being misaligned. I am going to try modifying the old one (that was in conflict with some material in the base) over the next couple of days. It is too long by 11mm, so if I can remove that it should then fit..

So after some modifications to the supports, the PSUs and NIM crates are now fully isolated from the frame.

I am currently working through changing all the guide pins for the new plastic ones and adding the shielding gasket.

Shunji made an interesting comment when we were discussing isolating the FEEs yesterday, in that, the ground connections for the chiller and the power we provide to our PSUs are connected to the same network.

    - I measure the resistance between the chiller ground and the power ground to be 2.5 Ohms.

    - The resistance between the chiller ground and the main Cu water pipe is ~2.2 kOhms.

    - We have three possible options here (that we have come up with so far):

          * disconnect the chiller ground from the network -> potentially very dangerous

          * put  a large resistance in between the chiller and the ground

          * Shunji has available a noise cut trans with up to 30A capability that could be used for our NIM crates and power supplies (I think we draw < 30A IIRC)  [photos attached]

I will carry out some short pulser tests with a DSSSD connected via the most recent kaptons this afternoon to see what effect the isolation and gasket has had so far.

Have you identified a specific noise issue with the Julabo recirculating chiller?

The STFC DL T9 test system has the recirculating chiller and AIDA PSUs connected to the same ac mains extender.

11.00 Moved detector stack forward several cm. 1st DSSD now ~6cm from end of snout. Middle of stack now ~47cm from
base -> approx centre of BRIKEN.
      
      4 new 79cm ribbon cables (+kaptons) added to p-side of first two DSSDs.
      
      Bias and pulser re-connected to all relevant FEEs. 

      Detector order unchanged, i.e. DSSD1 has same serial number as in June 2016 expt. Same for DSSDs 2-6.
      
      DSSD-to-FEE map also unchanged.

15.45 R7 starts

      ASIC settings 2016Sep02-18.17.01

      BNC PB-4
       Amplitude 90,000, x2 attenuator IN
       Frequency c. 26Hz

      Master MACB
       mode 5
       50MHz external clock (rear panel)
       BNC PB-4 trigger output -> Ortec SCA -> LeCroy 429A -> Master MACB Correlation DAQ Accepted Trigger - not
working - firmware issue?

      DSSSD bias/leakage current OK - see attachments 1-2
      FEE64 temperatures OK - see attachment 3

      System-wide checks OK *except* nnaida19 fails ASIC timestamp check (no change) *and* nnaida7, nnaida21
fail ADC calibration

      AIDA SYNC, PAUSE, RESUME, Correlation L, SYNC time warp, Good Events, AIDA disc, nnaida6, nnaida5,
nnaida19 & nnaida20 statistics - see attachments 4-14

      Analysis of R7_0 - ADC data synchronised

DSSSD bias voltages and leakage currents OK - see attachments 1 & 2
FEE64 temperatures OK - see attachment 3

Install MACB serial #13 (firmware 16.4.15) as Master MACB, mode 5

System-wide checks all OK 
 N.B. nnaida19 ASIC clock OK (cf previous Elog entries) - see attachment 4

Setup AIDA-BELEN DAQ correlation - RIBF still to do - see attachments 5 & 6
  reset request and trigger signals to Master MACB - NIM, width ~200ns

BELEN clock (10Hz) triggers BELEN BNC PB-5 and AIDA Master MACB

 ***     MBS information: block:  9349 ptr:  3725 data: 0x86807FF5 module:  6 information type:  8 information index:  0 MBS data:7FF5 ts: 0x00000026D0FE47A2
 ***     MBS information: block:  9349 ptr:  3727 data: 0x8681AF75 module:  6 information type:  8 information index:  1 MBS data:AF75 ts: 0x00000026D0FE47A2
 ***     MBS information: block:  9349 ptr:  3729 data: 0x86820000 module:  6 information type:  8 information index:  2 MBS data:0000 ts: 0x00000026D0FE47A2
 ***     MBS information: block:  9362 ptr:  7641 data: 0x8680A5AD module:  6 information type:  8 information index:  0 MBS data:A5AD ts: 0x00000026D196DE82
 ***     MBS information: block:  9362 ptr:  7643 data: 0x8681AF9B module:  6 information type:  8 information index:  1 MBS data:AF9B ts: 0x00000026D196DE82
 ***     MBS information: block:  9362 ptr:  7645 data: 0x86820000 module:  6 information type:  8 information index:  2 MBS data:0000 ts: 0x00000026D196DE82

0x0000AF9BA5AD - 0x0000AF757FF5 = 0x2625B8 = 2500024 * 40ns = 100000960ns = 0.1s

0x26D196DE82 - 0x26D0FE47A2 = 0x9896E0 = 10000096 * 10ns = 100000960ns = 0.1s

BELEN DAQ manual reset -> Master MACB reset request

 ***     MBS information: block:  9375 ptr:  3043 data: 0x86808A8A module:  6 information type:  8 information index:  0 MBS data:8A8A ts: 0x00000026D22F755E
 ***     MBS information: block:  9375 ptr:  3045 data: 0x8681001F module:  6 information type:  8 information index:  1 MBS data:001F ts: 0x00000026D22F755E
 ***     MBS information: block:  9375 ptr:  3047 data: 0x86820000 module:  6 information type:  8 information index:  2 MBS data:0000 ts: 0x00000026D22F755E

0x0000001F8A8A = 2067082 * 40ns = 82683280ns = 82.7us (which is < 0.1s as expected)

 ***     MBS information: block:  9388 ptr:  3291 data: 0x8680B041 module:  6 information type:  8 information index:  0 MBS data:B041 ts: 0x00000026D2C80C3A
 ***     MBS information: block:  9388 ptr:  3293 data: 0x86810045 module:  6 information type:  8 information index:  1 MBS data:0045 ts: 0x00000026D2C80C3A
 ***     MBS information: block:  9388 ptr:  3295 data: 0x86820000 module:  6 information type:  8 information index:  2 MBS data:0000 ts: 0x00000026D2C80C3A
 ***     MBS information: block:  9401 ptr:  5841 data: 0x8680D5F9 module:  6 information type:  8 information index:  0 MBS data:D5F9 ts: 0x00000026D360A31A
 ***     MBS information: block:  9401 ptr:  5843 data: 0x8681006B module:  6 information type:  8 information index:  1 MBS data:006B ts: 0x00000026D360A31A
 ***     MBS information: block:  9401 ptr:  5845 data: 0x86820000 module:  6 information type:  8 information index:  2 MBS data:0000 ts: 0x00000026D360A31A

19.05 R13_0 starts

23.19 Returned from dinner to find DAQ had stopped writing to disk at 22.19. Merger paused and Tape run ended successfully.
      No SYNC pulses being issued.
      Carried out system-wide checks. Clock status, ADC calib and ASIC clock passed for all 24 FEEs.
      Attachments 7 & 8 show errors messages issued upon fail of SYNC errors and SYNC pulses system checks.

      From attachment 8

      ( 6016000 SYNC / 9150 SYNC/s ) * 24 ~ 15800s => DAQ stopped 23.07 which is not consistent with observation
      as final write to disk files was c. 22.19 - note 23.21 for R13_47 indicates file closed at 23.21 cf. access
      times for R13_46, _45 etc

-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 19:09 R13_0
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 19:13 R13_1
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 19:50 R13_10
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 19:54 R13_11
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 19:58 R13_12
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 20:02 R13_13
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 20:06 R13_14
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 20:10 R13_15
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 20:14 R13_16
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 20:19 R13_17
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 20:23 R13_18
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 20:27 R13_19
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 19:17 R13_2
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 20:31 R13_20
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 20:35 R13_21
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 20:39 R13_22
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 20:43 R13_23
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 20:47 R13_24
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 20:51 R13_25
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 20:55 R13_26
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 20:59 R13_27
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 21:03 R13_28
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 21:08 R13_29
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 19:21 R13_3
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 21:12 R13_30
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 21:16 R13_31
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 21:20 R13_32
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 21:24 R13_33
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 21:28 R13_34
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 21:32 R13_35
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 21:36 R13_36
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 21:40 R13_37
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 21:44 R13_38
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 21:48 R13_39
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 19:25 R13_4
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 21:52 R13_40
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 21:56 R13_41
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 22:00 R13_42
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 22:04 R13_43
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 22:08 R13_44
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 22:13 R13_45
-rw-rw-rw-. 1 npg npgstaff 2097152000 Oct  6 22:17 R13_46
-rw-rw-rw-. 1 npg npgstaff  777519104 Oct  6 23:21 R13_47

23.21 R13_47 file closed

23.50 On trying to stop DAQ received nnaida1 timeout error (attachment 9).
      DAQ power-cycled, re-started with no data transfer 

10.30 DAQ found stopped

      All FEE64 AIDA SYNC rates zero, AIDA SYNC count ~ 4.4M => DAQ stopped @ 01.40

      3He detector DAQ off - no 50MHz clock to Master MACB

      System wide checks - see attachments 26-18

      N.B. It would appear that the DAQ stalled (TS_SYNC_PHASE 1) c. 01.40 and at some
      later time (c. 06.00) the 3He detector DAQ was powered off which means that AIDA
      lost the external 50MHz clock to the Master MACB - system-wide screenshots should
      be interpreted with caution.

14.15 3He detector DAQ 50MHz clock restarted
 
      VME and NIM crates ON
      Excecute command ConfClock_50MHz on 3He detector DAQ computer

      Power cycle AIDA FEE64s
      Set TS_SYNC_PHASE 2

      DSSSD bias/leakage current OK
      FEE64 temperatures OK
      *All* system-wide checks OK

      B2F/F11 temperature + deg C, d.p. + deg C,RH %

      ASIC settings 2016Sep02-18.17.01
      slow comparator 0x10 -> 0x0a

      Master MACB serial #13 (see https://elog.ph.ed.ac.uk/AIDA/367)
      Master MACB mode 5

14.20 DAQ starts (no data transfer)

18.10 DAQ found stopped

      All FEE64 AIDA SYNC rates zero, AIDA SYNC count ~ 3.4M => DAQ stopped @ 16.50

      System-wide checks - see attachments 16-9

      Power cycle AIDA FEE64s
      Set TS_SYNC_PHASE 3

      ASIC settings 2016Sep02-18.17.01
      slow comparator 0x10 -> 0x0a

20.10 DAQ found stalled

      System-wide checks - see attachments 8-1

      Power cycle AIDA FEE64s
      Set TS_SYNC_PHASE 0

      ASIC settings 2016Sep02-18.17.01
      slow comparator 0x10 -> 0x0a

20.35 DAQ starts (no data transfer)

      slow comparator 0x10 -> 0x0a

18.35 DAQ starts (no data transfer)

20.16 DAQ found stopped
      All FEE64 AIDA SYNC rates zero, AIDA SYNC count ~ 120k (17/24 FEE64s) or 1.49M (7/24) => FEE64s stopped @ 18.40 & 19.40

20.36 DAQ starts (no data transfer)

23.47 DAQ OK

10.40 DAQ found stopped

      All FEE64 AIDA SYNC rates zero, AIDA SYNC count ~ 13.8M => DAQ stopped @ 06.35

      System wide checks - see attachments 1-4 - all other system-wide checks OK

11.15 Replace cable for Master MACB external clock signal
           - change from c. 4m RG174 (Lemo-00 to Lemo-00) to c. 4m RG58 (BNC to BNC)
      Power cycle AIDA FEE64s
      Set TS_SYNC_PHASE 0

      DSSSD bias/leakage current OK - see attacment 5-6
      FEE64 temperatures OK - see attachment 7
      *All* system-wide checks OK (N.B. several ReSYNCs required) 

      B2F/F11 temperature + deg C, d.p. + deg C,RH %

      ASIC settings 2016Sep02-18.17.01
      slow comparator 0x10 -> 0x0a

      Master MACB serial #13 (see https://elog.ph.ed.ac.uk/AIDA/367)
      Master MACB mode 5

11.58 DAQ starts

15.41 DAQ OK

15.44 DAQ stop requested

15.46 DAQ start
      Data transfer to disk enabled to test DAQ correlation real-time monitor

22.50 AIDA SYNCs OK
      # Master SYNCs *not equal* to # FEE64 SYNCs - all FEE64 SYNCs # equal
            - all other system-wide checks OK - see attachments 8-15
      Merger and TapeServer OK

     R17_197 ADC data synchonised

aidas1> ./analyser2 /TapeData/Sept2016/R17_197
 *** GREAT format 3.2.0 analyser - TD - May 2014
 *** ERROR: READ I/O error:       5002
                   blocks:      32000
          ADC data format:  165074646 ( 1182230.4 Hz)
        Other data format:   96013354 (  687627.8 Hz)
 Sample trace data format:          0 (       0.0 Hz)
         Undefined format:          0 (       0.0 Hz)
   Other data format type:      PAUSE:        113 (       0.8 Hz)
                               RESUME:        113 (       0.8 Hz)
                              SYNC100:    1275326 (    9133.6 Hz)
                           FEE64 disc:   94733738 (  678463.4 Hz)
                             MBS info:       4064 (      29.1 Hz)
                           Other info:          0 (       0.0 Hz)

   ADC data range bit set:        191 (       1.4 Hz)

                Timewarps:        ADC:          0 (       0.0 Hz)
                                PAUSE:          0 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:          0 (       0.0 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:          0 (       0.0 Hz)
                            Undefined:          0 (       0.0 Hz)
                         Sample trace:          0 (       0.0 Hz)

 Timestamp elapsed time:      139.630 s
 FEE module #:   1 elapsed dead time       0.000 s
 FEE module #:   2 elapsed dead time       0.206 s
 FEE module #:   3 elapsed dead time       0.000 s
 FEE module #:   4 elapsed dead time       0.000 s
 FEE module #:   5 elapsed dead time       1.084 s
 FEE module #:   6 elapsed dead time       4.015 s
 FEE module #:   7 elapsed dead time       0.000 s
 FEE module #:   8 elapsed dead time       0.000 s
 FEE module #:   9 elapsed dead time       0.201 s
 FEE module #:  10 elapsed dead time       0.000 s
 FEE module #:  11 elapsed dead time       0.000 s
 FEE module #:  12 elapsed dead time       0.000 s
 FEE module #:  13 elapsed dead time       0.003 s
 FEE module #:  14 elapsed dead time       0.193 s
 FEE module #:  15 elapsed dead time       0.024 s
 FEE module #:  16 elapsed dead time       0.000 s
 FEE module #:  17 elapsed dead time       1.198 s
 FEE module #:  18 elapsed dead time       0.040 s
 FEE module #:  19 elapsed dead time       0.124 s
 FEE module #:  20 elapsed dead time       0.000 s
 FEE module #:  21 elapsed dead time       0.000 s
 FEE module #:  22 elapsed dead time       0.187 s
 FEE module #:  23 elapsed dead time       0.258 s
 FEE module #:  24 elapsed dead time       0.000 s
 FEE module #:  25 elapsed dead time       0.000 s
 FEE module #:  26 elapsed dead time       0.000 s
 FEE module #:  27 elapsed dead time       0.000 s
 FEE module #:  28 elapsed dead time       0.000 s
 FEE module #:  29 elapsed dead time       0.000 s
 FEE module #:  30 elapsed dead time       0.000 s
 FEE module #:  31 elapsed dead time       0.000 s
 FEE module #:  32 elapsed dead time       0.000 s

 *** Program elapsed time:   19.555s ( 1636.436 blocks/s, 102.277 Mb/s)

11.10 DSSSD bias & leakage currents OK - see attachments 1 & 2
      FEE64 temperatures OK - see attachment 3

      Rate, HitRate, per FEE64 1.8.L and 1.8.W spectra - see attachments 4-9

16.10 DSSD bias and leakage currents ok (attachments 10+11).
      FEE temps ok.

04/03/15

Please find attached a scan of the log book for the Feb 2015 AIDA tests at RIKEN. 

To do (roughly in order):

•  Move shipped boxes down from Gabor's office to the experimental area. Check contents. Should include:
  • 3x 3Tb HDD
  • NIM floating HV supply
  • 2x Si detector boxes, containing MSL type BB18 detectors 3131-9 + 3131-11 and 3058-9 + 3058-25
  • flex circuit tester
  • misc items including Al plates, grub nuts etc.
• Put everything back into working positions @ F11.
• Put computer table back to normal. Turn on/check aidas1.
• Reconnect power to chiller. Check water levels, fill if needed. Turn on and check for leaks.
• Install PSUs, USB and network switches in new AIDA base.
• Set up Raspberry Pi.
• Check modified PSU voltages.
• Reconnect FEE64s (power/network/HDMI). Label everything.
• Power on. Check
  • Basic pusler test to check function and server. Confirm FWHMs are as expected.

Late Monday or on Tuesday:

• Talk to Shunji re. AIDA/BRIKEN + EURICA setup requirements.
• Install Al/Si accordingly.
• Complete AIDA build with snout, kaptons etc.
• Acquire sources (Am/Bi) for source test ------> Speak to Gabor first thing about this.

A table summarising the run information within the lab book and ELog for Apr-May.

Powered system up from state it was left in last night. Biased to +200V with leakage current 3.9uA @ room temp 23.3oC.

nnaida14 now performing as it was at the beginning of yesterday, with nnaida11 back to having noisy waveforms.

Tried various combinations of grounding to ascertain minimum possible configuration to maintain (or improve) current performance.

       - detector ground jumper required, noise in waveforms increases significantly if removed

       - system seemingly has no preference of jumper combinations for the left-hand pairs of pins, other than all 4 at once increase noise. Any combination of 3 or fewer seems to have no effect

       - fitted additional small braids to kaptons and took them directly to the PCBs (like in the previous ELog entry) which made no difference

Found LEMO connectors on both PCBs to show open circuit with the Cu adapter mount block (not mezzanine block). Wrapped thin braid round connectors and took attached directly to each respective Cu block.

Resistance between any two points between the detector and Cu mezzanine blocks now < 0.5 Ohm.

       - when i left for lunch this had no effect on nnaida11, but nnaida14 showed improvement in fine noise on waveforms but none for the low frequency 'wave' (see attachment 1)

       - after returning from lunch, both were noisy again and I was unable to repeat original improvement over a few attempts at the same or similar grounding. Subsequently removed completely and performance returned to normal...

When we arrived on Wednesday, detector box was floating and performance was OK. Box is now not floating, and attempts to make it so were unsuccessful, although nothing I did changed the noise situation.

Removing the Al plates either side of the detector made had no impact on the noise.

Didn't get chance to try swapping the PCBs over.

Powered on system ok.

Electrical connection between crate and mezzanines still persists, but only to the top cool plate (no connection to the bottom or sides), eith resistance ~5-10 ohms

Seems to have no effect on performance though as without the kaptons connected, still see pulser widths of ~14ch at t_shape = 8us.

Connecting cables reinstates double peaking in n-side.

Without HV or detector connected, nnaida13+14 gibe single Gaussian peaks of width ~45ch, with  almost perfect waveforms.

         - Patrick saw this too, I'm not going crazy, but on zeroing histograms (didn't change a single other thing) it went back to having double peaks again.

Replaced nnaida13 shielded kapton with unshielded one to see if that is causing this effect

         - still see double peaking but width of tallest peak is down by ~25% (likely just due to capacitance)

With shielding on, do see much noisier waveforms however.

         - note: attachments 1 and 2 were taken using exactly the same threshold of 512. When shielded, bumping this up to 1000 still made little difference and 'actual' traces were only caught ~25% of the time

Changed firmware to V8.

No difference in performance.

13/12/15

Current jumper setup:

     - detector 2977-15 connected to nnaida1,3,5,8 with GND kapton to nnaida1

     - bias of +75V to nnaida3 and nnaida5

     - Jumpers:

           - nnaida1 > Lk1. If Lk3 3 connected unable to bias detector

           - nnaida3 > Lk3. if Lk1 connected unable to bias detector as the HV ground reference is connected at both p- and n-sides of detector.

           - nnaida5 > Lk1 + Lk3

           - nnaida8 > Lk3

Powered system up as it was last night -> 2977-15 connected to nnaida1,3,5,8 with only 4 FEE modules powered.

     - n-side widths ~190-200 ch

     - p-side widths ~100ch

     - jumpers as above, ASIC setting as 2015Dec12_smallSetup, wav thresholds all at 500 apart from nnaida5 @ 250.

Spend remainder of day trying to fit mezzanine cables.

14/12/15

Completed installation of mezzanine cables.

Resistance between all mezzanines and cooling crates still ~200 kOhms (up from ~2-5 Ohms on our arrival), apart from nnaida7+8, which is now at ~5 Ohms

However, all see resistance to main vertical copper pipe of ~3 Ohms.

As best as I can see, isolating tape in still in place for all FEEs     =>     unknown point of contact.

       > LEMO cables to test inputs carry ground connection from NIM bin -> NIM bin in electrical contact with support frame -> coolant loop in electrical contact with support frame  => electrical connection still present between FEEs and coolant loop, even in insulating guide pins are introduced

       > this problem is not observed at DL because the NIM bin in on the wooden table top and isolated from the coolant loop and any support structure

       > is it possible to isolate the NIM bins from the support structure?

       > need to come up with some more systematic and fixed way/positions for measuring these resistances, i.e. always measure mezzanine to X.

Resistance between all mezzanines now ~ 0.5 Ohms.

n-side widths ~ 120ch and p-side widths ~ 120ch    =>    peak widths almost halved in n-side, but increased by ~20% in p-side.

Attempted to bias previously tripped detector.

      - same jumper combination as other detector caused failure to bias > leakage current jumps to 50uA at ~2V and trips shortly after, ramping back down from there.

      - removed all jumpers and see same thing.

Will try again in the morning by attaching 3131-11 to cables currently on 2977-15, but it looks like 3131-11 really is defective somehow.

 Tues 15/12/15

Swapped previously tripped detector (3131-11) on to kaptons already in place for (working) 2977-15 and attempted to bias

   - i.e. made no changes (FEEs, or jumpers etc) to the known working setup of 2977-15, just replaced the detector with 3131-11

   > still trips at ~6-7V

   > conclusion: He's dead Jim.

Took some measurements of rough max positions of DSSSDs from the FEEs with the current 90cm kaptons (pictures in attached PowerPoint and PDF)

   - Vertical EURIKA support to centre of EURIKA ~ 60cm

   - AIDA Cu cooling pipe to end of snout ~ 75cm

   - Base of snout to end =  57cm

   - Min distance from end of snout to inner most FEE (horizontal FEEs) ~ 92cm

           - additional 6cm per extra adapter

   - Further DSSSD can go from base of snout ~ 54cm (60cm from Cu pipe)

           - limiting factor is kaptons to nnaida5+6 (bottom stack). Still a few more cm slack in the other kaptons.

   - For kaptons in 4th horizontal connector (furthest slot in 2nd adapter), detector 48cm from base of snout.

   - For kaptons in 8th horizontal connector (furthest slot in 4th adapter), detector 36cm from base of snout.

Note (PCS): In disconnecting things to check ground connections, and when putting AIDA away before I left, I noticed ~20% of the network cables are of the shielded variety. I know this didn't seem to make any difference in the setup at DL, but I don't know how concerned you are about this and if you would suggest changing them in future.

10.30 Pulser walkthrough carried out 90,000 - 10,000 @ 10,000 step

10.40 .msf spectra saved /MIDAS/SpecData/AIDA

13.30 Swapped MACB module #1 - no change (root module)
      Swapped MACB module #2 - no change
      Swapped MACB module #3 - no change (nnaida5 attached to port 1 of this MACB)

      Note cabling not changed during above tests

15.20 Swap nnaida5 & nnaida6 FEE module for nnaida31 & nnaida32 and update startup.tcl and dhcpd.conf
      accordingly - see attached and appended 

      No resyncs at all

16.15 Re-install original nnaida5 & nnaida6 

      No resyncs at all - see attachment 2

#
# DHCP Server Configuration file.
#   see /usr/share/doc/dhcp*/dhcpd.conf.sample  
#

authoritative;

ddns-update-style none; ddns-updates off;

max-lease-time 691200;

option domain-search code 119 = string;

subnet 10.1.1.0 netmask 255.255.255.0 {
   option subnet-mask 255.255.255.0;
   option broadcast-address 10.1.1.255;
   option routers 10.1.1.250;
   pool {
      range 10.1.1.100 10.1.1.249;
   }
}

subnet 10.1.2.0 netmask 255.255.255.0 {
   option subnet-mask 255.255.255.0;
   option broadcast-address 10.1.2.255;
   option routers 10.1.2.250;
   pool {
      range 10.1.2.100 10.1.2.249;
   }
}

subnet 10.1.3.0 netmask 255.255.255.0 {
   option subnet-mask 255.255.255.0;
   option broadcast-address 10.1.3.255;
   option routers 10.1.3.250;
   pool {
      range 10.1.3.100 10.1.3.249;
   }
}

subnet 10.1.4.0 netmask 255.255.255.0 {
   option subnet-mask 255.255.255.0;
   option broadcast-address 10.1.4.255;
   option routers 10.1.4.250;
   pool {
      range 10.1.4.100 10.1.4.249;
   }
}

subnet 10.1.5.0 netmask 255.255.255.0 {
   option subnet-mask 255.255.255.0;
   option broadcast-address 10.1.5.255;
   option routers 10.1.5.250;
   pool {
      range 10.1.5.100 10.1.5.249;
   }
}

group {
   use-host-decl-names true;
   default-lease-time 3600;
   max-lease-time 14400;

   server-name "10.1.1.250";
   next-server 10.1.1.250;

   host nnrpi1 {
     hardware ethernet b8:27:eb:78:ee:44;
     fixed-address 10.1.1.251;
   }
   host nnaida1 {
     hardware ethernet 00:04:a3:2a:d0:1b;
     fixed-address 10.1.1.1;
     option root-path "/MIDAS@aidas/XilinxLinux/ppc_4xx/rfs/nnaida1,tcp";
   }
   host nnaida2 {
     hardware ethernet 00:04:a3:2a:d9:e1;
     fixed-address 10.1.1.2;
     option root-path "/MIDAS@aidas/XilinxLinux/ppc_4xx/rfs/nnaida2,tcp";
   } 
#   host nnaida5 {
#     hardware ethernet 00:04:a3:2a:d0:10;
#     fixed-address 10.1.1.5;
#     option root-path "/MIDAS@aidas/XilinxLinux/ppc_4xx/rfs/nnaida5,tcp";
#   }
#   host nnaida6 {
#     hardware ethernet 00:04:a3:2a:d4:b7;
#     fixed-address 10.1.1.6;
#     option root-path "/MIDAS@aidas/XilinxLinux/ppc_4xx/rfs/nnaida6,tcp";
#   }
   host nnaida5 {
     hardware ethernet 00:04:a3:2b:11:ae;
     fixed-address 10.1.1.5;
     option root-path "/MIDAS@aidas/XilinxLinux/ppc_4xx/rfs/nnaida5,tcp";
   }
   host nnaida6 {
     hardware ethernet 00:04:a3:2a:ed:b9;
     fixed-address 10.1.1.6;
     option root-path "/MIDAS@aidas/XilinxLinux/ppc_4xx/rfs/nnaida6,tcp";
   }
   host nnaida9 {
     hardware ethernet 00:04:a3:2a:d9:ad;
     fixed-address 10.1.1.9;
     option root-path "/MIDAS@aidas/XilinxLinux/ppc_4xx/rfs/nnaida9,tcp";
   }
   host nnaida10 {
     hardware ethernet 00:04:a3:2a:f7:07;
     fixed-address 10.1.1.10;
     option root-path "/MIDAS@aidas/XilinxLinux/ppc_4xx/rfs/nnaida10,tcp";
   }
   host nnaida13 {
     hardware ethernet 00:04:a3:2b:11:b9;
     fixed-address 10.1.1.13;
     option root-path "/MIDAS@aidas/XilinxLinux/ppc_4xx/rfs/nnaida13,tcp";
   }
   host nnaida14 {
     hardware ethernet 00:04:a3:2b:11:b8;
     fixed-address 10.1.1.14;
     option root-path "/MIDAS@aidas/XilinxLinux/ppc_4xx/rfs/nnaida14,tcp";
   }
   host nnaida17 {
     hardware ethernet 00:04:a3:2a:d9:cc;
     fixed-address 10.1.1.17;
     option root-path "/MIDAS@aidas/XilinxLinux/ppc_4xx/rfs/nnaida17,tcp";
   }
   host nnaida18 {
     hardware ethernet 00:04:a3:2a:b2:b2;
     fixed-address 10.1.1.18;
     option root-path "/MIDAS@aidas/XilinxLinux/ppc_4xx/rfs/nnaida18,tcp";
   }
   host nnaida21 {
     hardware ethernet 00:04:a3:2b:27:23;
     fixed-address 10.1.1.21;
     option root-path "/MIDAS@aidas/XilinxLinux/ppc_4xx/rfs/nnaida21,tcp";
   }
   host nnaida22 {
     hardware ethernet 00:04:a3:2a:f6:e9;
     fixed-address 10.1.1.22;
     option root-path "/MIDAS@aidas/XilinxLinux/ppc_4xx/rfs/nnaida22,tcp";
   }
   host nnaida25 {
     hardware ethernet 00:04:a3:2a:d0:11;
     fixed-address 10.1.1.25;
     option root-path "/MIDAS@aidas/XilinxLinux/ppc_4xx/rfs/nnaida25,tcp";
   }
   host nnaida26 {
     hardware ethernet 00:04:a3:2b:27:54;
     fixed-address 10.1.1.26;
     option root-path "/MIDAS@aidas/XilinxLinux/ppc_4xx/rfs/nnaida26,tcp";
   }
   host nnaida29 {
     hardware ethernet 00:04:a3:2a:d0:06;
     fixed-address 10.1.1.29;
     option root-path "/MIDAS@aidas/XilinxLinux/ppc_4xx/rfs/nnaida29,tcp";
   }
   host nnaida30 {
     hardware ethernet 00:04:a3:2a:d9:b6;
     fixed-address 10.1.1.30;
     option root-path "/MIDAS@aidas/XilinxLinux/ppc_4xx/rfs/nnaida30,tcp";
   }


}


group {
   use-host-decl-names true;
   default-lease-time 3600;
   max-lease-time 14400;

   server-name "10.1.2.250";
   next-server 10.1.2.250;

   host nnaida3 {
     hardware ethernet 00:04:a3:2a:f7:06;
     fixed-address 10.1.2.3;
     option root-path "/MIDAS@aidas/XilinxLinux/ppc_4xx/rfs/nnaida3,tcp";
   }
   host nnaida4 {
     hardware ethernet 00:04:a3:2a:ed:b8;
     fixed-address 10.1.2.4;
     option root-path "/MIDAS@aidas/XilinxLinux/ppc_4xx/rfs/nnaida4,tcp";
   }
   host nnaida7 {
     hardware ethernet 00:04:a3:2a:b8:8d;
     fixed-address 10.1.2.7;
     option root-path "/MIDAS@aidas/XilinxLinux/ppc_4xx/rfs/nnaida7,tcp";
   }
   host nnaida8 {
     hardware ethernet 00:04:a3:2a:9c:07;
     fixed-address 10.1.2.8;
     option root-path "/MIDAS@aidas/XilinxLinux/ppc_4xx/rfs/nnaida8,tcp";
   }
   host nnaida11 {
     hardware ethernet 00:04:a3:2b:11:af;
     fixed-address 10.1.2.11;
     option root-path "/MIDAS@aidas/XilinxLinux/ppc_4xx/rfs/nnaida11,tcp";
   }
   host nnaida12 {
     hardware ethernet 00:04:a3:2a:d0:2f;
     fixed-address 10.1.2.12;
     option root-path "/MIDAS@aidas/XilinxLinux/ppc_4xx/rfs/nnaida12,tcp";
   }
   host nnaida15 {
     hardware ethernet 00:04:a3:2a:f3:ac;
     fixed-address 10.1.2.15;
     option root-path "/MIDAS@aidas/XilinxLinux/ppc_4xx/rfs/nnaida15,tcp";
   }
   host nnaida16 {
     hardware ethernet 00:04:a3:2a:9c:0e;
     fixed-address 10.1.2.16;
     option root-path "/MIDAS@aidas/XilinxLinux/ppc_4xx/rfs/nnaida16,tcp";
   }
# 1 of 2 scratch FEE modules assembled by PJCS April 2016
   host nnaida19 {
     hardware ethernet 00:04:a3:2a:ed:9a;
     fixed-address 10.1.2.19;
     option root-path "/MIDAS@aidas/XilinxLinux/ppc_4xx/rfs/nnaida19,tcp";
   }
   host nnaida20 {
     hardware ethernet 00:04:a3:2b:11:c5;
     fixed-address 10.1.2.20;
     option root-path "/MIDAS@aidas/XilinxLinux/ppc_4xx/rfs/nnaida20,tcp";
   }
   host nnaida23 {
     hardware ethernet 00:04:a3:2a:b8:90;
     fixed-address 10.1.2.23;
     option root-path "/MIDAS@aidas/XilinxLinux/ppc_4xx/rfs/nnaida23,tcp";
   }
   host nnaida24 {
     hardware ethernet 00:04:a3:2a:b6:45;
     fixed-address 10.1.2.24;
     option root-path "/MIDAS@aidas/XilinxLinux/ppc_4xx/rfs/nnaida24,tcp";
   }
   host nnaida27 {
     hardware ethernet 00:04:a3:2b:22:55;
     fixed-address 10.1.2.27;
     option root-path "/MIDAS@aidas/XilinxLinux/ppc_4xx/rfs/nnaida27,tcp";
   }
# 2 of 2 scratch FEE modules assembled by PJCS April 2016
   host nnaida28 {
     hardware ethernet 00:04:a3:2b:09:e7;
     fixed-address 10.1.2.28;
     option root-path "/MIDAS@aidas/XilinxLinux/ppc_4xx/rfs/nnaida28,tcp";
   }
#   host nnaida31 {
#     hardware ethernet 00:04:a3:2b:11:ae;
#     fixed-address 10.1.2.31;
#     option root-path "/MIDAS@aidas/XilinxLinux/ppc_4xx/rfs/nnaida31,tcp";
#   }
#   host nnaida32 {
#     hardware ethernet 00:04:a3:2a:ed:b9;
#     fixed-address 10.1.2.32;
#     option root-path "/MIDAS@aidas/XilinxLinux/ppc_4xx/rfs/nnaida32,tcp";
#   }
}


group {
# 8 days
   default-lease-time 691200;

#    option dhcp-client-identifier "\0foo";

   host nndhcp052 {
# Analysis server
     hardware ethernet 00:22:19:50:02:40;
     fixed-address 10.1.1.252;
   }

}

04.45 Lowered slow comp threshold on all FEEs bar nnaida24 to 8 (nnaida24 @ 12).
      Fast comp thresh @ 6 on all nnaida.
      See CPU load and MIDASsort in attachments 1+2.

11.20 Directory /MIDAS/SpecData moved to /ntfs/TapeData/NP1306/ (Edinburgh Seagate USB 8Tb disk Z840DXMA)
      to create disk space on root filesystem (100% usage, 134M remaining space) 

11.36 Saved current *.L spectra to /MIDAS/SpecData
 
11.40 Detector biases & leakage currents - OK - see attachments 3-4
      FEE64 temperatures - see attachment 5

12.15 Julabo FL11006 set point +15 deg C (B2F/F11 temp +25.5 deg C, d.p. +12.9 deg C , RH 45.4% )
      changed set point from +15 to +17 deg C

13.18 FEE64 temperatures - see attachment 6

Original system configuration before my arrival: nnaida11 - 2b:22:6e
                                                 nnaida12 - 2b:33:15
                                                 nnaida13 - 2b:09:ce
All on firmware version 0x1900c1f.

With 8us shaping time and pulser amplitude @ 5V, FWHM [ch] were:

   Ch   | nnaida11 | nnaida12 | nnaida13 | nnaida14
  1.4.L |   38     |    33    |   38     |   34
  1.6.L |   24     |    25    |   28     |   23
  1.8.L |   20     |    17    |   17     |   16
 1.10.L |   21     |    17    |   17     |   16

Slightly higher than expected for a few channels, but FWHM fall within ~16-40ch for all channels in all FEEs.
All system checks passed.

nnaida13/14 kept constant throughout as control.

(1) Swapped nnaida11/12: nnaida11 -> 2b:33:0c     nnaida12 -> 2a:d0:1a
    Initially failed SYNC error test - consequence of old firmware version.
    Updated firmware from 0x10300c12 to 0x1900c1f.
    All system tests passed.
    FWHM on 8us shaping time and same pulser settings.

     Ch   | nnaida11 | nnaida12 | nnaida13 | nnaida14
    1.4.L |   37     |    40    |   42     |   35
    1.6.L |   31     |    28    |   29     |   23
    1.8.L |   22     |    18    |   15     |   15
   1.10.L |   23     |    20    |   16     |   15

(2) Swapped nnaida11/12: nnaida11 -> 2a:b8:90     nnaida12 -> 2a:b6:45
    Updated firmware from 0x10300c12 to 0x1900c1f.
    All system checks passed.
    No ASIC temp in nnaida11.
    FWHM on 8us shaping time and same pulser settings.

     Ch   | nnaida11 | nnaida12 | nnaida13 | nnaida14
    1.4.L |   40     |    32    |   40     |   34
    1.6.L |   29     |    25    |   29     |   23
    1.8.L |   21     |    18    |   17     |   16
   1.10.L |   21     |    18    |   17     |   16

(3) Swapped nnaida11/12: nnaida11 -> 2a:ed:9a     nnaida12 -> 2b:11:c5
    Updated firmware from 0x10300c12 to 0x1900c1f.
    All system checks passed.
    No ASIC temp in nnaida12.
    FWHM on 8us shaping time and same pulser settings.

     Ch   | nnaida11 | nnaida12 | nnaida13 | nnaida14
    1.4.L |   44     |    36    |   40     |   35
    1.6.L |   30     |    27    |   29     |   24
    1.8.L |   22     |    18    |   16     |   17
   1.10.L |   23     |    18    |   17     |   16


The above were all the repaired modules and seem to be operating normally.
Following are the 2 modules for which no fault report was provided. I only had time to complete testing on one, the other remains unknown.

(4) Swapped nnaida11/12: nnaida11 -> 2a:ed:8f     nnaida12 -> 2b:09:da
    Updated firmware from 0x10300c12 to 0x1900c1f.
    nnaida11 failed clock status tests. Tried 'Perform resync' from Master Timestamp window with no effect.
    No SYNC errors reported. All other system tests passed.
    nnaida12 missing lots of channels but all system checks passed.
    FWHM on 8us shaping time and same pulser settings.

     Ch   | nnaida11 | nnaida12 | nnaida13 | nnaida14
    1.4.L |   46     |    46    |   41     |   36
    1.6.L |   31     |    26    |   28     |   24
    1.8.L |   22     |    19    |   17     |   16
   1.10.L |   22     |    20    |   17     |   16


**

Overall, the first three modules which have been repaired seem to be operating normally. The fourth seems not be and the last module remains unknown.
Ch 1.4.L displays greater than expected FWHM throughout in all nnaida11/12, but FWHM were largely constant (to within a few channels) throughout and all FWHM fell in the range ~15-40ch apart from 1.1.L in *all* nnaida.