09:51 Taken over from Tom following the night shift
      Experiment is still running smoothly
      Compression of the files is complete up to the start of R5.
      Have started compression of files in R5 which should run up to R5_499
      Currently on R5_528

      Statistics ok - attachment 1
      Temperatures ok - attachment 2
      Bias and leakage currents ok - attachment 3
      System wide checks:
          		 Base 		Current 	Difference
      aida07 fault 	 0xc53d : 	 0xc594 : 	 87  
      aida08 fault 	 0xf1be : 	 0xf271 : 	 179  
      White Rabbit error counter test result: Passed 6, Failed 2

      			 Base 		Current 	Difference
      aida07 fault 	 0x2a : 	 0x3b : 	 17  
      FPGA Timestamp error counter test result: Passed 7, Failed 1

      Current merger rate is 2E6-4E6 events per second
      Current tapeserver rate is 5800 kB/s
      Free HDD space is 1.1 TB
      At current rates will last 54 hours which should see out the experiment
      Experiment currently scheduled to finish at 6am on Tuesday morning (May get until 8am)
      
      Analysis of R5_528 - attachment 4
      Deadtime of AIDA02 currently around 15%

11:56 Statistics ok - attachment 5
      Temps ok - attachment 6
      Bias and leakage currents ok - attachment 7
      System wide checks:
      Clocks all ok 
      
         		 Base 		Current 	Difference
      aida07 fault 	 0xc53d : 	 0xc59c : 	 95  
      aida08 fault 	 0xf1be : 	 0xf27a : 	 188  
      White Rabbit error counter test result: Passed 6, Failed 2

	
			 Base 		Current 		Difference
      aida07 fault 	 0x2a : 	 0x3b : 	 17  
      FPGA Timestamp error counter test result: Passed 7, Failed 1

      Analysis of file R5_458 - attachment 8
      Deadtime in AIDA02 only 11.5% in this file
      TapeServer rate still 5.5 MB/s

14:36 Has been no beam for the last while or so.
      With no beam AIDA02 has 0.25% deadtime so the deadtime is almost entirely due to the spill on time
      Stats- attachment 9
      Temp - attachment 10
      Bias and leakage currents ok - attachment 11
      System wide checks:
      Clock ok
      	
        		 Base 		Current 	Difference
      aida07 fault 	 0xc53d : 	 0xc59d : 	 96  
      aida08 fault 	 0xf1be : 	 0xf27c : 	 190  
      White Rabbit error counter test result: Passed 6, Failed 2
      	
			 Base 		Current 		Difference
      aida07 fault 	 0x2a : 	 0x3c : 	 18  
      FPGA Timestamp error counter test result: Passed 7, Failed 1

13:30 Beam taken to change an ion source

14:57 Beam back

16:21 Statistics ok - attachment 12
      Temperatures ok - attachment 13
      Bias and leakage currents ok - attachment 14

      System wide checks - ASIC clocks ok 
	
	        	 Base 		Current 	Difference
      aida07 fault 	 0xc53d : 	 0xc5a3 : 	 102  
      aida08 fault 	 0xf1be : 	 0xf285 : 	 199  
      White Rabbit error counter test result: Passed 6, Failed 2

	
			 Base 		Current 		Difference
      aida07 fault 	 0x2a : 	 0x3e : 	 20  
      FPGA Timestamp error counter test result: Passed 7, Failed 1

      Currently on file R5_592
      Analysis of R5_591 - Attachment 15
      Deadtime of AIDA02 sitting at 17%

[NH taking over for OH so he can get home]

18:53 - FRS DAQ problems mean we weren't taking DESPEC data for the past hour or so... now all back
        Statitics ok - attachement 16
        Temps ok - attachement 17
        Bias & leakage ok - attachement 18

        System wide checks - 
        Clocks OK
        ADC Calibration N/A 
	WR Decoder -
        		 Base 		Current 	Difference
        aida07 fault 	 0xc53d : 	 0xc5ac : 	 111  
        aida08 fault 	 0xf1be : 	 0xf28c : 	 206  
        White Rabbit error counter test result: Passed 6, Failed 2
        FPGA -
         Base 		Current 		Difference
        aida07 fault 	 0x2a : 	 0x3f : 	 21  
        PLL OK

        Currently on file R5_621
        Analysis of R5_620 - Attachement 19
        aida02 deadtime only 5%, all others negligible 

20:20 Stats ok - attachment 20
      Temps ok - attachment 21
      Bias and leakage currents ok - attachment 22
      System wide checks:
      Clock ok
    	
	        	 Base 		Current 	Difference
      aida07 fault 	 0xc53d : 	 0xc5ad : 	 112  
      aida08 fault 	 0xf1be : 	 0xf293 : 	 213  
      White Rabbit error counter test result: Passed 6, Failed 2

     	
			 Base 		Current 		Difference
      aida07 fault 	 0x2a : 	 0x3f : 	 21  
      FPGA Timestamp error counter test result: Passed 7, Failed 1

      Analysis of R6_637 - attachment 23

22:13 Statistics ok - attachment 24
      Temperatures ok - attachment 25
      Bias and leakage currents ok - attachment 26
      ASIC clock check ok

	
		         Base 		Current 	Difference
      aida07 fault 	 0xc53d : 	 0xc5ba : 	 125  
      aida08 fault 	 0xf1be : 	 0xf2a2 : 	 228  
      White Rabbit error counter test result: Passed 6, Failed 2

			 Base 		Current 		Difference
      aida07 fault 	 0x2a : 	 0x40 : 	 22  
      FPGA Timestamp error counter test result: Passed 7, Failed 1

      Analysis of R5_658 - attachment 27

23.43 Check ASIC control
      Zero stats & all histograms

ASIC settings 2021Apr29-13-16-00
 slow comparator 0x64 -> 0xa

all waveform AD9252 ADCs disabled
all fast discs disabled

BNC PB-5 settings (to p+n FEE64s only)
 amplitude 1.0V
 attenuator x1
 decay time 1ms
 polarity +
 frequency 22Hz

All system wide checks OK *except* WR & FPGA errors - attachments 1 & 2

DSSSD bias & leakage currents OK - attachment 3

adc data item stats - attachment 4

FEE64 temps OK - attachment 5

DSSSD bias & leakage currents OK - attachment 6

analysis of file S505/R5_415 - attachment 7
 zero timewarps
 max deadtime aida02 c. 17%, all other FEE64s < 1%



02:36

analysis of file S505/R5_449 - attachment 8
 zero timewarps
 max deadtime aida02 c. 15%, all other FEE64s < 1%

per FEE64 rate & stat spectra - attachments 9 & 10

per p+n FEE64 1.8.L spectra - attachment 11
 aida01 pulser peak width 99 ch FWHM

per FEE64 1.8.H spectra - attachments 12 & 13

All system wide checks OK *except* WR & FPGA errors - attachments 14 & 15

adc data item stats - attachment 16

FEE64 temps OK - attachment 17

DSSSD bias & leakage currents OK - attachment 18


05:38

analysis of file S505/R5_483 - attachment 19
 zero timewarps
 max deadtime aida02 c. 16%, all other FEE64s < 2%

All system wide checks OK *except* WR & FPGA errors - attachments 20 & 21

adc data item stats - attachment 22

FEE64 temps OK - attachment 23

DSSSD bias & leakage currents OK - attachment 24



07:27

analysis of file S505/R5_502 - attachment 25
 zero timewarps
 max deadtime aida02 c. 16%, all other FEE64s < 2%

All system wide checks OK *except* WR & FPGA errors - attachments 26 & 27

adc data item stats - attachment 28

FEE64 temps OK - attachment 29

DSSSD bias & leakage currents OK - attachment 30

Took over the night shift from Magda

18:00 : attachments 1-3

20:00 : attachments 4-6

22:00 : attachments 7-9

00:00 : attachments 10-12

Took over the night shift from Tom.

7:00 The beam is back.

8:00 : attachments 1-3

10:00 : attachments 4-6

12:00 : attachments 7-9

14:00 : attachments 10-12

16:00 : attachments 13-15

23:36 Check ASIC contorol
      Zero stats & all histograms

ASIC settings 2021Apr29-13-16-00
 slow comparator 0x64 -> 0xa

all waveform AD9252 ADCs disabled
all fast discs disabled

BNC PB-5 settings (to p+n FEE64s only)
 amplitude 1.0V
 attenuator x1
 decay time 1ms
 polarity +
 frequency 22Hz

analysis of file S505/R5_154 - attachment 1
 zero timewarps
 max deadtime aida02 c. 17%, all other FEE64s < 5%

stats - attachments 2-6

per FEE64 1.8.H spectra - attachments 7 & 8

per p+n FEE64 1.8.L spectra - attachment 9
 aida01 pulser peak width 101 ch FWHM

per FEE64 stat & rate spectra - attachments 10 & 11

adc data item stats - attachment 12

FEE64 temps OK - attachment 13

DSSSD bias & leakage currents OK - attachments 14 & 15

00:20 all system wide checks OK *except* WR and FPGA errors - attachments 16 & 17


03:03

analysis of file S505/R5_189 - attachment 18
 zero timewarps
 max deadtime aida02 c. 17%, all other FEE64s < 5%

all system wide checks OK *except* WR and FPGA errors - attachments 19 & 20

adc data item stats - attachment 21

FEE64 temps OK - attachment 22

DSSSD bias & leakage currents OK - attachment 23

03:30

per FEE64 1.8.H spectra - attcahments 24 & 25


06:09 no beam 

analysis of file S505/R5_221 - attachment 26
 zero timewarps
 all dead times << 1%

all system wide checks OK *except* WR and FPGA errors - attachments 27 & 28

adc data item stats - attachment 29

FEE64 temps OK - attachment 30

DSSSD bias & leakage currents OK - attachment 31

16:05 - Last checked was at 15:30. (see previous entry. All running smoothly.  

Next wide check will be in about an hour.

17:00

Stats -ok - attachment 1

Temperatures ok - attachement 2

Leakage current ok but - attachment 3

ucesb screen-shot - attachment 4

Wide check completed. Nothing different.

WR status decoder status: 
         Base         Current     Difference
aida07 fault      0xc53d :      0xc547 :      10  
aida08 fault      0xf1be :      0xf1e9 :      43  
White Rabbit error counter test result: Passed 6, Failed 2

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

FPGA timestamp::  
             Base         Current         Difference
aida07 fault      0x2a :      0x2c :      2  
FPGA Timestamp error counter test result: Passed 7, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last

Note that the leakage current is ok but has increased since yesterday , See graphana - attachement 5 - Prossibly due to higher beam intensity.

19:20 -

Stats  ok - attachment 6

Temp ok - attachment 7

Leakage current  ok - attachement 8

Wide check completed and same output as above.

22:00

Stats  ok - attachment 9

Temp ok - attachment 10

Leakage current  ok - attachement 11

Wide check completed and same output as above.

07:56 Took over from AM. No issues reported overnight
      Stats ok - attachment 1
      Temperature ok - attachment 2
      Bias and leakage current ok - attachment 3
      
      System wide checks:
      Clocks ok
      WR
      		 Base 		Current 	Difference
      aida07 fault 	 0xc4fe : 	 0xc53b : 	 61  
      aida08 fault 	 0xf0e9 : 	 0xf1b7 : 	 206  
      White Rabbit error counter test result: Passed 6, Failed 2
      FPGA Errors
      			 Base 		Current 		Difference
      aida07 fault 	 0x11 : 	 0x29 : 	 24  
      aida08 fault 	 0x1a : 	 0x2c : 	 18  
      FPGA Timestamp error counter test result: Passed 6, Failed 2

      Analysis of R3_359 - attachment 4
      Dead time still around 15.9%

      Merger item rate around 2E6-4E6
      Tape server rate at 7750 kB/s
      Current HDD free space 1.7 TB
      Time left in HDD 2.59 days
      Will run out of space at some point on Sunday
      Have started compression of uncompressed raw data on the HDD. Using nice +10


08:45 AIDA07 dropped out from the merger at some point in the preceeding 5-10 minutes. - attachment 5
      Am regularly checking the statistics
      Was able to stop the DAQ by relaunching the merger.
      Reset the FEEs and recovered without a powercylce
      Started R4 following this stop

08:58 They have taken the beam to change the ion source for maybe 2 hours
      Will stop writing data but continue forwarding to MBS
      No storage ticked. Following break will be on R5


11:10 Beam is starting to come back R5 started
      Stats ok - attachment 6
      Temps ok - attachment 7
      Bias and leakage currents ok - attachment 8

13:05 Statistics ok - attachment 9
      Temps ok - attachment 10
      Bias and leakage curents ok - attachment 11
      System wide checks:
      WR
      		 Base 		Current 	Difference
      aida07 fault 	 0xc53d : 	 0xc540 : 	 3  
      aida08 fault 	 0xf1be : 	 0xf1d0 : 	 18  
      White Rabbit error counter test result: Passed 6, Failed 2

      FPGA
      			 Base 		Current 		Difference
      aida07 fault 	 0x2a : 	 0x2b : 	 1  
      FPGA Timestamp error counter test result: Passed 7, Failed 1

      Analysis of R5_33 - attachment 12

15:28 Statistics ok - attachment 13
      Temperature ok - attachment 14
      Bias and leakage current ok - attachment 15
      System wide checks:
      	
		 Base 		Current 	Difference
      aida07 fault 	 0xc53d : 	 0xc546 : 	 9  
      aida08 fault 	 0xf1be : 	 0xf1e0 : 	 34  
      White Rabbit error counter test result: Passed 6, Failed 2

      			 Base 		Current 		Difference
      aida07 fault 	 0x2a : 	 0x2b : 	 1  
      FPGA Timestamp error counter test result: Passed 7, Failed 1

      Analysis of R5_63 - attachment 16

01:30    Attachments 1-5, white rabbit and fpga timestamp failures, otherwise all good

03:30    Attachments 11-15, white rabbit and fpga timestamp failures, otherwise all good

05:30    Attachments 6-10, white rabbit and fpga timestamp failures, otherwise all good

07:30    Attachments 16-20, white rabbit and fpga timestamp failures, otherwise all good

16:20
System wide checks.
Clock status OK
White Rabbit Error status:
		 Base 		Current 	Difference
aida07 fault 	 0xc4fe : 	 0xc525 : 	 39  
aida08 fault 	 0xf0e9 : 	 0xf166 : 	 125  
White Rabbit error counter test result: Passed 6, Failed 2

FPGA error status:
                 Base 		Current 		Difference
aida07 fault 	 0x11 : 	 0x1c : 	 11  
aida08 fault 	 0x1a : 	 0x23 : 	 9  

Temps OK (attachment 1), Stats OK (attachment 2), Bias and leakage OK (attachment 3)


18:05
System wide checks.
Clock status OK
White Rabbit Error status:
		 Base 		Current 	Difference
aida07 fault 	 0xc4fe : 	 0xc527 : 	 41  
aida08 fault 	 0xf0e9 : 	 0xf177 : 	 142  
White Rabbit error counter test result: Passed 6, Failed 2

FPGA error status:
		 Base 		Current 		Difference
aida07 fault 	 0x11 : 	 0x1c : 	 11  
aida08 fault 	 0x1a : 	 0x24 : 	 10  
FPGA Timestamp error counter test result: Passed 6, Failed 2

Temps OK (attachment 4), Stats OK (attachment 5), Bias and leakage OK (attachment 6)


20:22
System wide checks.
Clock status OK
White Rabbit Error status:
		 Base 		Current 	Difference
aida07 fault 	 0xc4fe : 	 0xc529 : 	 43  
aida08 fault 	 0xf0e9 : 	 0xf17f : 	 150  
White Rabbit error counter test result: Passed 6, Failed 2

FPGA error status:
		 Base 		Current 		Difference
aida07 fault 	 0x11 : 	 0x20 : 	 15  
aida08 fault 	 0x1a : 	 0x25 : 	 11  
FPGA Timestamp error counter test result: Passed 6, Failed 2

Temps OK (attachment 7), Stats OK (attachment 8), Bias and leakage OK (attachment 9)

21:56
System wide checks.
Clock status OK
White Rabbit Error status:
		 Base 		Current 	Difference
aida07 fault 	 0xc4fe : 	 0xc529 : 	 43  
aida08 fault 	 0xf0e9 : 	 0xf186 : 	 157  
White Rabbit error counter test result: Passed 6, Failed 2

FPGA error status:	
		 Base 		Current 		Difference
aida07 fault 	 0x11 : 	 0x21 : 	 16  
aida08 fault 	 0x1a : 	 0x26 : 	 12  

Temps OK (attachment 10), Stats OK (attachment 11), Bias and leakage OK (attachment 12)

23:50
System wide checks.
Clock status OK
White Rabbit Error status:
		 Base 		Current 	Difference
aida07 fault 	 0xc4fe : 	 0xc52e : 	 48  
aida08 fault 	 0xf0e9 : 	 0xf18c : 	 163  
White Rabbit error counter test result: Passed 6, Failed 2

FPGA error status:	
	
			 Base 		Current 		Difference
aida07 fault 	 0x11 : 	 0x21 : 	 16  
aida08 fault 	 0x1a : 	 0x27 : 	 13  
FPGA Timestamp error counter test result: Passed 6, Failed 2

Temps OK (attachment 13), Stats OK (attachment 14), Bias and leakage OK (attachment 15)

07:56 Taken over from AM
      Current White Rabbit Error status:
      		 Base 		Current 	Difference
      aida07 fault 	 0xc4fe : 	 0xc51a : 	 28  
      aida08 fault 	 0xf0e9 : 	 0xf132 : 	 73  
      White Rabbit error counter test result: Passed 6, Failed 2

      FPGA error status:
      			 Base 		Current 		Difference
      aida07 fault 	 0x11 : 	 0x18 : 	 7  
      aida08 fault 	 0x1a : 	 0x20 : 	 6  
      FPGA Timestamp error counter test result: Passed 6, Failed 2

      Likely that at some point AIDA08 will drop out again. I think this occurs if there is a sudden burst of WR errors which causes the merger to drop the link

      Statistics ok - attachment 1
      FEE temperatures ok - attachment 2
      Bias and leakage currents ok - attachment 3
      Analysis of file R3_93 - attachment 4
      Still deadtime in aida02 of ~15%
      Patrick has said he will look into whether we are still sending info code 4 and 5 for every ADC event which could be contributing to the deadtime.
      
      Currently 2.2 TB of space on /media/SecondDrive
      Current data rate of 6.23MB/s
      At current data rate this should last 4 days

08:27 Currently no beam
      Problem with Alvarez 3 now
      Tomorrow will have a 1 hour break due to changing of the source

09:36 Beam back

09:59 System wide checks
		 Base 		Current 	Difference
      aida07 fault 	 0xc4fe : 	 0xc51b : 	 29  
      aida08 fault 	 0xf0e9 : 	 0xf138 : 	 79  
      White Rabbit error counter test result: Passed 6, Failed 2

      			 Base 		Current 		Difference
      aida07 fault 	 0x11 : 	 0x18 : 	 7  
      aida08 fault 	 0x1a : 	 0x20 : 	 6  

      Stats ok - attachment 5
      Temperatures ok - attachment 6
      Bias and leakage currents ok - attachment 7

11:14 FRS group is changing from Hg setting back to Pt

11:36 Back on Pt R3_129

14:14 Stats ok - attachment 8
      Temperature ok - attachment 9
      Bias and leakage currents ok - attachment 10
      System wide checks:
      WR:
      		 Base 		Current 	Difference
      aida07 fault 	 0xc4fe : 	 0xc523 : 	 37  
      aida08 fault 	 0xf0e9 : 	 0xf156 : 	 109  
      White Rabbit error counter test result: Passed 6, Failed 2

      FPGA
      	
			 Base 		Current 		Difference
      aida07 fault 	 0x11 : 	 0x1b : 	 10  
      aida08 fault 	 0x1a : 	 0x20 : 	 6

      Analysis of R3_156 - attachment 11  

00:00 TD Noticed on the last stats uploaded that AIDA08 had stopped sending signals.
      Attempted to recover restarting merger but this caused 01 to drop out.

00:10 DAQ recovered and AIDA01 recovered with a reboot


01:00 Attachments 1-4, white rabbit failures on aida07 and aida08, otherwise all good

03:00 Attachments 5-9, aida07 and aida08 failures on white rabbit and fpga timestamp, otherwise all good

05:00 Attachments 10-14, aida07 and aida08 failures on white rabbit and fpga timestamp continue, otherwise all good

07:00 Attachments 15-19, aida07 and aida08 failures on white rabbit and fpga timestamp continue, otherwise all good

16:00 Take over from Magda mrning shift

Beam was off of sometime for tunning to Mercurey setting.

17:00 attached 1-3

19:00 attached 4-6

21:00 attached 7-9

23:30 10-12

Study of the FEE64 power supply distribution has yielded the following :-

The most sensitive regulator, as regards the device voltage input, is the TPS51100 that supplies the DDR memory reference voltages. This requires +4.75 to +5.25 volts. This is supplied from the Power input connector +5v input.

The common LT3080 regulator used over much of the board is a Low Voltage Dropout regulator. This requires 0.5v difference between input and output voltage as minimum. This is not a problem with the +4.75v minimum for the TPS51100 requirement setting the voltage for the board.

The supply to the mezzanine is direct from the power connector +5v input. On the mezzanine there is an LT3080 for each ASIC supplying the required 3.3v. These regulators would possibly benefit from a 1uF capacitor at the Control voltage input.

The simplest approach would be to add a capacitor to the bottom layer where the Control voltage enters the mezzanine. 

The power cable has a nominal resistance of 13.3ohms/km. The 3 conductors of the cable are supplying 10A when all is in operation. So the expected voltage drop would thus be ( 10 x 13.3 x 0.007 ) /3 => 0.3v each core. 

The conclusion would be that the voltage at the power supply should drop to 5.25 v thus ensuring the TPS51100 is supplied as required regardless of the operation of the FEE. 

This will be tested at Daresbury.

Took over the night shift from Tom.

8:10 : attachments 1-3

10:00 : attachments 4-6

12:00 : attachments 7-9

The beam was off for around 20' starting at around 11:30-11:50 due to ion source issues.

12:19 OH Disabled all discriminators to reduce the data rate.

13:06 Analysis of R2_104 15.66% deadtime on AIDA04- attachment 10

14:00 : attachments 11-13

16:00 : attachments 14-16

00.28 all histograms & stats zero'd
      baseline counters

ASIC settings 2021Apr29-13-16-00
 *except* slow comparator 0x64 -> 0xa

BNC PB-5 pulser
 to p+n FEE64s only
  amplitude 1.0V
  attenuator x1
  decay time 1ms
  polarity +
  frequency 2Hz

all AD9252 waveform ADCs disabled

Tape Server -> no storage mode

00.36 check ASIC control all FEE64s all ASICs

      all system wide checks OK

00.41 BNC PB-5 pulser rate increased from 2Hz to 22Hz per request by HA/NH - to ensure DTAS correlation checker has sufficient rate of identified pulser events 

per p+n FEE64 1.8.L spectra - attachment 1
 aida01 pulser peak width 88 ch FWHM

per FEE64 rate spectra - attachment 2

adc, pause and correlation scaler data items - push, aida01 and aida02 stats - attachments 3-8

FEE64 temps OK - attachment 9

DSSSD bias & leakage current OK - attachments 10 & 11

02.28 aida05 zero adc data items, unable to halt aida05 - all other FEE64s OK

      restart NewMerger -> now able to halt aida05

      DAQ go OK, merge rate c. 1.4M data items/s

per FEE64 1.8.H spectra - attachments 12 & 13
 approx flat distribution to c. 10GeV


adc data items stats OK - attachments 14

FEE64 temps OK - attachment 15

DSSSD bias & leakage current OK - attachments 16 & 17

02.55 Some problems with GO4 data analysis for PID (FRS PID looks OK). 204Pt setting. Will commence writing data to disk file S505/R2_0

04.12 per FEE64 1.8.H spectra - attachments 18 & 19
      c. 9GeV peak in upstream DSSSDs (aida01, 2, 3 & 4) indicating (high energy) heavy fragments are stopping
      Low energy events near c. 200MeV lower mass, high energy, low energy loss - fission fragments?

04.20 per FEE64 stats spectra - attachment 20
      indicates implant x-y distribution - off centre (more significantly in x than in y) but DSSSDs appear to see most of distribution
      remember raw distro will have significant fraction of fission fragments - may be misleading - need to gate on high energy events to see x-y distro of ions of 
interest

adc data items stats OK - attachments 21

FEE64 temps OK - attachment 22

DSSSD bias & leakage current OK - attachments 23



06.10 per FEE64 rate spectra - attachment 24

per FEE64 1.8.H spectra - attachments 25 & 26

per FEE64 stats spectra - attachment 27
     
adc data items stats OK - attachments 28

FEE64 temps OK - attachment 29

system wide checks OK *except* WR & FPGA errors - see attachments 30 & 31

DSSSD bias & leakage current OK - attachments 32

07.52 analysis of file S505/R2_47 - attachment 33
      all FEE64 dead times < 1% *except* aida02 c. 7%

11:19 Stats ok 0xa on all FEE - attachment 1
      Current pulser settings - attachment 2
      FEE Temperatures ok - attachment 3
      Noticed both AIDA03 and AIDA07 had ASICs with missing channels. Did a check ASIC control on them to recover
      Layout 1 after recovering channels - attachment 4
      New stats (Note increased rate on 3 and 7) attachment 5
      Bias and leakage currents ok - attachment 6

      S4 now in closed access. Trying to find out if the beam dump has been put in front of AIDA. Will make sure this is checked before beam to S4.

14:05 Have had it confirmed that the flange beam dump is in position in S4

15:43 Stats ok -attachment 7
      Temp ok - attachment 8
      Bias and Leakage current ok - attachment 9


17:45 Currently the time machine trigger(for correlation) is connected to the DTAS or.
      As the beam is tuning they are triggering at 150k. This is 450k worth of scalers in AIDA
      During the experiment this will be changed to SC1 trigger to solve this.

19:49: Stats ok - attachment 10
       temperatures ok - attachment 11
       Bias - Leakage current ok - attachment 12

10:26 MACB HDMI for AIDA03 and AIDA04 reseated.
      Merger and DAQ restarted - All happily sending data
      Has been running for 10 minutes with no WR errors.
      Rates from yesterday achieved again.

      DTAS will move back into position at some point.
      Once this is done, will perform pulser walkthrough.


11:14 DTAS now back in position for experiment
      Slight increase in noise noticed across a few FEEs
      Over time this has decreased on all but AIDA07 which is now running at ~80k - attachment 1
      AIDA03 is now however gaining white rabbit errors at a rate of 1 every few minutes.
      Less frequently gaining FPGA errors
    
12:20 AIDA power cycle to reseat the HDMI for AIDA03 - It would eventually crash if not done
      Upon powering up again rates in AIDA08 slightly worse - attachment 2

13:26 No WR errors since the last power cycle.

21:16 Baseline system wide checks, zero histograms & stats

All system wide checks OK

per p+n FEE64 1.8.L spectra - attachment 3
 aida01 pulser peak width 82 channels FWHM

per FEE64 rate spectra - attachment 4

adc, pause, resume and correlation scaler data item stats - attachments 5-8

FEE64 temps OK - attachment 9

DSSSD bias & leakage currents OK - attachments 10 & 11

aida03 and aida04 seem to stop sending data  quickly

stopping DAQ + merger and restarting got them resending data, but quickly stopped again

both are showing large WR errors that increases approx every second.

Assume HDMI cables not well seated and causing all sorts of odd behaviour

Yesterday the sum/inverter was found to be a considerable source of noise in the system.

Plugging it into an oscilloscope this morning it was clear there was a Vpp 17mv 60kHz sin wave in its signal - attachment 1 yellow trace

Moved the inverter to a crate in the messheute and did not see same behaviour. This time see Vpp 5mV and a ~265Hz wave that looks much less clean. - attachment 2 yellow

Put inverter back in CAEN crate in S4 and confirm similar behavior.

Move CAEN crate to new mains port, same behaviour

Move CAEN crate to AC voltage stabiliser, same behaviour

Remove module one by one from CAEN crate.
Removing BIAS supply - No change
Remove Pulser - No change
Remove right most MACB - No change
Remove second right most MACB - Signal disappears  - attachment 3 blue trace
Remove second right most MACB from the rack entirely
Put right most MACB back - Signal comes back - attachment 4 blue trace
Test MACB with and without HDMI connectors connected signal remains
Remove rightmost MACB from the rack - Signal disappears

Put pulser and bias back - Signal does not return
Signal also doesn't return when ramping up bias or running the pulser
Power DAQ up - No change in performance from yesterday with inverter removed.

MACB and Inverter don't share any voltages on NIM:
MACB +-6V and Gnd
Inverter +-24V and Gnd

CAEN Crate does not have any easily accessible monitoring points. Anyone have any ideas?

MACBs and inverter checked on another crate - no signal

Inverter put back in CAEN crate and no noise observed on scope.
With waveforms disabled am able to get <100kHz on all FEEs with the pulser connected

Tested adding bias filters - No change

15:30 Nic realised that one of the mains boxes is labelled measurement network so we connect the AIDA PSU to this via the ac stabiliser
      We get ok stats even with waveforms on at 0xa- attachment 5

      Waveforms still have the 100kHz oscillation in them though attachments 6 and 7

      Without waveforms we get similar performance to before the switch to the new mains - attachment 8

15:56 Move CAEN NIM crate across to this mains network
      300+kHz on all FEEs with waveforms on. Can recover waveform off performance though

16:01 Place CAEN NIM crate back onto platform mains but don't recover the ealier rates with waveform. Everything still around 300kHz.

17:43 Current best settings are waveforms off and we can run at 0xa on all FEEs - attachment 9

01.20 

ASIC settings 2021Apr29-13-16-00
 *except* slow comparator 0x64 -> 0xa

All waveform ADCs disabled

BNC PB-5 pulser
 to p+n FEE64s only

all system wide checks OK *except* ADC calibration, WR decoder & FPGA timestamp errors - see attachments 1 & 2

DSSSD bias & leakage current OK - see attachments 3 & 4

FEE64 temps OK - see attachment 5

statistics OK - ADC data, pause, resume & correlation scaler data items
                data push, data flush, aida01, aida02 - see attachments 6-13

per FEE64 1.8.L spectra - see attachment 14
 aida01 pulser peak width 73 ch FWHM