FEEs have been on this morning, temperatures attached

Leakage current attached

the DSSD is attached to the following AIDA modules:
Beam Left: 10
Beam Right: 12
Bottom: 11
Top: 9

MIDAS Servers on, will be looking at resolution (pulser)

The AIDA snout (with single DSSD and plastic) has been mounted in S4 cave again. DSSD was connected to the outer 4 FEEs again.

-160 V Bias applied and DSSD holds voltage with leakage current of around 1.3 uA (was dropping as only biased briefly)

Bias off and FEEs offline again now as day nearly done. MBS testing will be performed shortly, as well as resolution checks.

All 12 FEEs powered up, and temperatures are OK.

Passed ASIC checks and system wide checks - now checking cabling and to mount snout.

The AIDA water system has been repaired and not leaking.

The interlock did not light immediately, but power cable reconnected to box and now it works.

Currently

- 1 DSSD and Plastic is in the AIDA snout, which is not mounted

- The FEEs are offline due to a water supply issue, this will be repaired early 21/03/2019

- The pis & aida-3 are online

The water interlock also didn't clear, which will be investigated

Ideally installation can be completed tomorrow but this depends on water

Summary statistics for WR test data files R1_0, R2_0, R3_0, R4_0, R7_0 and R10_0 (attachments 1-6)

Latest version of WR TDR data analyser (attachment 7)

If you have a GSI account it should be possible to connect to aida-gsi remotely via the following
SSH into lx-pool.gsi.de with GSI Linux username/password (ssh username@lx-pool.gsi.de)
Then SSH into aida-gsi via ssh npg@aida-3

Two raspberry pis:

nnrpi1 - FEE64 system consoles
nnrpi2 - AC Mains Relay & CAEN HV

Startup:

1. Plug in micro-USB on pis
2. Connect via ssh: ssh pi@nnrpi1  or ssh pi@nnrpi2
3. Start MIDAS:
cd /MIDAS/TclHttpd/Linux-arm
./TclHttpd-server

4. Connect from aida-gsi web browser
http://nnrpi2:8015 - For AC Relay Control
http://nnrpi1:8015 - For Pi Monitoring (Get list of USB terminals, Connect)
-> Parse USB log for details to check if all FEEs have finished booting completely or not

5. For CAEN HV connect via ssh/X (ssh -X pi@nnrpi2) and run
putty &
Opens putty window to connect to the CAEN HV module (Serial /dev/ttyACM0)

09.30: AIDA still running, not writing to file
       Still running ok
       No water leaks

09.45: FEE64 temperatures ok (attachment 1)
       Good event statistics ok (attachment 2)

10.00: Results of System Wide Checks
       ADC Calibration check (attachment 3)
       Clock Status (attachment 4)
       Sync error counter (attachment 5)
       White Rabbit Decoder status (attachment 7)
       Memory Information (attachment 8) 
       
10.30: Clock ReSync performed (attachment 6)
       Start writing to file TapeData/NULL/R7_0

11.50: DAQ stopped, TapeServer stopped
       
       TapeServer restarted, DAQ restarted

       Writing to file TapeData/NULL/R9_0 with Clock ReSync at start.

       Wasn't writing to disk, restarting (DAQ stop, TapeServer stop, TapeServer start TO DISK, DAQ Start)

       Writing to file TapeData/NULL/R10_0

12.04: DAQ stopped, TapeServer stopped, merger stopped
       FEE64 power relay switched off
       Raspberry PIs switched off and disconnected
       water cooler stopped
       Mains IEC cables removed from relay

       
        

AIDA@DESPEC MACB time switch settings before and after WR are as follows.

Attachment 1 - MACB 1-4 from left to right

MACB number:                           1        2        3        4

100MHZ clock setting (before WR):      0        2        2        2

White Rabbit setting (after WR):       3        3        3        3

Attachment 1 - summary analysis R2_0

               increased slow comparator & LEC/MEC fast comparators cf. R1_0 to minimise random noise

Attachment 2 - summary analysis R3_0

               per R2_0 but BNC PB-5 pulser frequency increased from c. 2Hz to c. 1kHz
               pulser to *all* channels 12x FEE64s

Attachment 3 - summary analysis R4_0

               open TapeSever file, DAQ GO, Start ASIC Readout (*correct* DAQ start sequence)

Observations

- some RESUME but *zero* PAUSE data items observed

- information types 4 & 5 not equal for R2_0 and almost, but not quite, equal for R3_0 & R4_0 

- information types 4 & 5 start data file with 'correct' DAQ start sequence

           

Analysis of file GSIJan19/R1_0 (white rabbit data)

attachment 1 - overall statistics

attachment 2 - verbose output listing all data around first six identified timewarps

attachment 3 - analyser.f (old version for 48 bit timestamp)

attachment 4 - analyser.f (new version for 64 bit WR timestamp)

Observations

1) File does not commence with information type 4 & 5 data items

   DAQ/TapeServer start sequence issue??

2) DAQ RESUMEs observed for all FEE64s - DAQ PAUSEs *not* observed

3) Number of information type 4 & 5 data items for each FEE64 *not* the same

4) reported idle & elapsed times incorrect - presumably analyser.f bug

 

1) Click on item "Merger MK2" in top panel
   This kills any existing Merger existence and loads a new copy.

2) Start (if needed) browser conection to localhost:8115.

3) Click on "NewMerger Control"

4) In NewMerger Control, click "Reload"

5) If MergerState = IDLE, click "SETUP".

6) Wait until state = STOPed

7) If links shown are correct, click "GO"

8) Wait until popup about "Resume Merger" appears and click OK

9) when writing to disk, click "Forward Output to Data"

10.30 AIDA setup and DAQ start complete
      writing to file TapeData/NULL/R1

      FEE64 Temperatures and Good event statistics ok (attachments 1 and 2)

11.00 results of analyser - shows significant number of timewarps in aida07, and a few in aida01-04

11.24 compared timestamps between mbs and AIDA, agree to 8 digits (attachment 3)

12.49 DAQ and Merger stopped and restarted

      Raised slow comparator threshold from 0xa to 0x64

      Raised fast comparator threshold (LEC/MEC) from 0xf to 0xff

12.51 Good event statistics: aida08 running at a normal rate

      pulser peak (1.8.L) no longer has noise shoulder (attachment 4)

      writing to file TapeData/NULL/R2


13.58 stopped writing to file. Pulser frequency increased to 1 kHz

      writing to file TapeData/NULL/R3

15.00 file R4 - DAQ startup

17.58 trimmed and re-crimped ground wire on aida08
      lowered pulser freq back to 50Hz
      reduced slow comparator threshold to 0xa, fast comparator threshold (LEC) to 0xf

      good event stats ok, aida08 no longer running hot (attachment 6)
      1.8.L and 1.8.W spectra of aida08 (attachment 5)
      pulser peak width now at 55.71

      FEE64 temperatures ok (attachment 7)

Post White Rabbit update

Attachment 1 - 1.8.W pulser waveform spectra for all FEE64
             - pulser signals observed in all FEE64

Attachment 2 - good event statistics - aida08 has significant noise

Attachment 3 - FEE64 temperatures - all ok, but firmware version not displayed

Attachment 4 - 1.8.L pulser peak spectra

             - widths:

             aida01 - 23.66
             aida02 - 19.76
             aida03 - 20.43
             aida04 - 18.82
             aida05 - 20.05
             aida06 - 41.04
             aida07 - 21.24
             aida08 - 205.02 (noisy FEE64)
             aida09 - 24.81
             aida10 - 22.85
             aida11 - 13.98
             aida12 - 19.90

Attachment 5 - New Merger statistics

Attachment 6 - Raw data stream

Pictures of leaking cooling pipe

MACB switch settings

AIDA mount - needs rotated

Attachment 11 - AIDA@DESPEC MACB settings - 100MHz and WR

Attachment 1 - 1.8.W waveform spectra with pulser signal

Attachment 2 - 1.8.L pulser peak spectra

widths: aida01 - 16.06
       
        aida04 - 20.68

        aida05 - 14.03

        aida08 - 73.02

        aida09 - 669.29

        aida11 - 405.35

        aida12 - 397.6

Attachment 3 - good event statistics
               fast comparator threshold (LEC) at 0xff
               FEEs 8,9,11,12 running hot

Attachment 4 - FEE64 temperatures - all ok

System wide checks ok, except aida02 fails ADC calibration

Leak found in seal on cooling pipe, switched power off

11.12 System wide checks ok (attachment 1)
      FEE temperatures ok (attachment 2)
      bias/leakage currents ok (attachment 3)

11.13 1.8.W spectra for naida 1,9,10,11,12 (attachment 4)
      1.8.W spectra expanded in Y direction (attachment 5)
      pulser peaks (attachment 6)
      
11.20 pulser peak widths:
      
      Aida10 -> 98.82

      Aida09 -> 109.17

      Aida11 -> 71.72

      Aida12 -> 92.82

      Aida01 -> 24.05

11.23 hit rates (attachment 7)

11.41 pictures of setup with scintillator mounted (attachment 8-14)

nnrpi1 - FEE64 system consoles

nnrpi2 - USB-controlled ac mains relay, CAEN N1419ET

username: pi
password: *******

To startup TclHttpd server for web access via port 8015 execute
 
 /MIDAS/TclHttpd/Linux-arm/TclHttpd-server

Update - 9 March 2020 command line should be

/MIDAS/TclHttpd/linux-arm/TclHttpd-server

Analysis of data file R2_0

Attachment 1 - summary output of TDR analyser

Attachment 2 - verbose output of TDR analyser