 Mon Jun 27 23:11:47 2022, TD, Tuesday 28 June 00:00-08:00 30x
|
00:07 Zero stats & all histograms
ASIC settings 2021Apr29-13-16-00
slow comparator 0x64 -> 0xa
all waveform AD9252 ADCs disabled
all fast discs disabled
BNC PB-5 settings (to p+n FEE64s only)
amplitude 1.0V
attenuator x1
decay time 1ms
polarity +
frequency 22Hz
analysis of file S505/R5_896 - attachment 1
zero timewarps
deadtime all FEE64s << 1%
All system wide checks OK *except* WR & FPGA errors - attachments 2 & 3
adc data item stats - attachment 4
FEE64 temps OK - attachment 5
DSSSD bias & leakage currents OK - attachment 6 & 7
00:15 Check ASIC control all FEE64s, all ASICs
02:03
analysis of file S505/R5_914 - attachment 8
zero timewarps
deadtime all FEE64s << 1%
per FEE64 1.8.H spectra - attachments 9 & 10
per p+n FEE64 1.8.L spectra - attachment 11
aida01 pulser peak width 94 ch FWHM
per FEE64 stat & rate spectra - attachments 12 & 13
All system wide checks OK *except* WR & FPGA errors - attachments 14 & 15
adc data item stats - attachment 16
FEE64 temps OK - attachment 17
DSSSD bias & leakage currents OK - attachment 18
03:31 S505 PI Anabel declares experiment end - following periods of beam loss and FRS DAQ issues today
03:41
analysis of file S505/R5_926 - attachment 19
zero timewarps
deadtime all FEE64s << 1%
All system wide checks OK *except* WR & FPGA errors - attachments 20 & 21
adc data item stats - attachment 22
FEE64 temps OK - attachment 23
DSSSD bias & leakage currents OK - attachment 24
07:00
analysis of file S505/R5_954 - attachment 25
zero timewarps
deadtime all FEE64s << 1%
All system wide checks OK *except* WR & FPGA errors - attachments 26 & 27
adc data item stats - attachment 28
FEE64 temps OK - attachment 29
DSSSD bias & leakage currents OK - attachment 30 |
|
Mon Jun 27 17:03:09 2022, MA, Monday 27th June 16:00-00:00 6x
|
16:00 Took over the shift from OH no beam yet.
18:00 Still no beam yet.
Statistics, Temperature, Current are checked and attached 1-3
system wide checks same as last updated in the previoues shift.
22:00 The beam is back but not taking data yet! FRS team doing some checkings
Statistics, Temperature, Current are checked and attached 4-6
system wide checks same as last updated in the previoues shift.
23:30 beam is back and taking data |
|
Mon Jun 27 06:45:22 2022, OH, Monday 27th June 08:00-16:00
|
07:45 Spoke to David and the beam has been gone since about 05:30
Reason for the loss of beam is a vacuum issue before the FRS
They are waiting for the experts
08:31 Statistics ok - attachment 1
Temperature ok - attachment 2
Bias and leakage currrents ok - attachment 3
ASIC clock check ok
Base Current Difference
aida07 fault 0xc53d : 0xc5cf : 146
aida08 fault 0xf1be : 0xf2ba : 252
White Rabbit error counter test result: Passed 6, Failed 2
Base Current Difference
aida07 fault 0x2a : 0x41 : 23
Currently on file R5_771
Analysis of file R5_770 (No beam) - attachment 4
Around 0.25% deadtime on AIDA02 rest even less
09:02 Current free HDD space 965 GB
Current tape server rate 4979 kB/s
Free space taking data rate at 5700 kB/s (Closer to beam value) 47 hours
11:41 Statistics ok - attachment 5
Temperatures ok - attachment 6
Bias and leakage currents ok - attachment 7
ASIC clock check ok
Base Current Difference
aida07 fault 0xc53d : 0xc5cf : 146
aida08 fault 0xf1be : 0xf2ba : 252
White Rabbit error counter test result: Passed 6, Failed 2
Base Current Difference
aida07 fault 0x2a : 0x41 : 23
FPGA Timestamp error counter test result: Passed 7, Failed 1
Note that there has been no change in the White Rabbit errors or FPGA faults since very early morning.
Could the rate of accrual in errors be proportional to the data rate. Faster data rate, errors occur more frequently? |
|
Sun Jun 26 23:04:30 2022, Marc, new shift - Monday 27 June 0:00 to 8:00 12x
|
0:00 Experiment continues to run smoothly. AIDA DAQ has been rather stable. Leakage current has gone up for the past two days in both DSSDs (combination of high-Z and high-intensity beam and external high temperature).
Stats & Temperatures (VIRTEX,PSU, ASICs) all ok.
At 0:30
Stats ok - Attachment 1
Temp ok - Attachement 2
HV-LC -Attachment 3
At 2:20
Stats ok - Attachment 4
Temp ok - Attachement 5
HV-LC -Attachment 6
Wide Checks:
Clock status test result: Passed 8, Failed 0
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable
ADC Calibration (same as before):
FEE64 module aida01 failed
FEE64 module aida02 failed
FEE64 module aida03 failed
FEE64 module aida04 failed
FEE64 module aida05 failed
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida08 failed
Calibration test result: Passed 0, Failed 8
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
WR decoder status:
Base Current Difference
aida07 fault 0xc53d : 0xc5c9 : 140
aida08 fault 0xf1be : 0xf2b2 : 244
White Rabbit error counter test result: Passed 6, Failed 2
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
FPGA timestamp check:
Base Current Difference
aida07 fault 0x2a : 0x41 : 23
FPGA Timestamp error counter test result: Passed 7, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
At 4:15:
Stats ok - Attachment 7
Temp ok - Attachement 8
HV-LC -Attachment 9
Wide Checks: No change
At 7:15: (no beam since ~6am -> background run)
Stats ok - Attachment 10
Temp ok - Attachement 11
HV-LC -Attachment 12
Wide Checks: No change |
|
Sun Jun 26 08:51:20 2022, OH, NH, Sunday 26 June 08:00-24:00 27x
|
09:51 Taken over from Tom following the night shift
Experiment is still running smoothly
Compression of the files is complete up to the start of R5.
Have started compression of files in R5 which should run up to R5_499
Currently on R5_528
Statistics ok - attachment 1
Temperatures ok - attachment 2
Bias and leakage currents ok - attachment 3
System wide checks:
Base Current Difference
aida07 fault 0xc53d : 0xc594 : 87
aida08 fault 0xf1be : 0xf271 : 179
White Rabbit error counter test result: Passed 6, Failed 2
Base Current Difference
aida07 fault 0x2a : 0x3b : 17
FPGA Timestamp error counter test result: Passed 7, Failed 1
Current merger rate is 2E6-4E6 events per second
Current tapeserver rate is 5800 kB/s
Free HDD space is 1.1 TB
At current rates will last 54 hours which should see out the experiment
Experiment currently scheduled to finish at 6am on Tuesday morning (May get until 8am)
Analysis of R5_528 - attachment 4
Deadtime of AIDA02 currently around 15%
11:56 Statistics ok - attachment 5
Temps ok - attachment 6
Bias and leakage currents ok - attachment 7
System wide checks:
Clocks all ok
Base Current Difference
aida07 fault 0xc53d : 0xc59c : 95
aida08 fault 0xf1be : 0xf27a : 188
White Rabbit error counter test result: Passed 6, Failed 2
Base Current Difference
aida07 fault 0x2a : 0x3b : 17
FPGA Timestamp error counter test result: Passed 7, Failed 1
Analysis of file R5_458 - attachment 8
Deadtime in AIDA02 only 11.5% in this file
TapeServer rate still 5.5 MB/s
14:36 Has been no beam for the last while or so.
With no beam AIDA02 has 0.25% deadtime so the deadtime is almost entirely due to the spill on time
Stats- attachment 9
Temp - attachment 10
Bias and leakage currents ok - attachment 11
System wide checks:
Clock ok
Base Current Difference
aida07 fault 0xc53d : 0xc59d : 96
aida08 fault 0xf1be : 0xf27c : 190
White Rabbit error counter test result: Passed 6, Failed 2
Base Current Difference
aida07 fault 0x2a : 0x3c : 18
FPGA Timestamp error counter test result: Passed 7, Failed 1
13:30 Beam taken to change an ion source
14:57 Beam back
16:21 Statistics ok - attachment 12
Temperatures ok - attachment 13
Bias and leakage currents ok - attachment 14
System wide checks - ASIC clocks ok
Base Current Difference
aida07 fault 0xc53d : 0xc5a3 : 102
aida08 fault 0xf1be : 0xf285 : 199
White Rabbit error counter test result: Passed 6, Failed 2
Base Current Difference
aida07 fault 0x2a : 0x3e : 20
FPGA Timestamp error counter test result: Passed 7, Failed 1
Currently on file R5_592
Analysis of R5_591 - Attachment 15
Deadtime of AIDA02 sitting at 17%
[NH taking over for OH so he can get home]
18:53 - FRS DAQ problems mean we weren't taking DESPEC data for the past hour or so... now all back
Statitics ok - attachement 16
Temps ok - attachement 17
Bias & leakage ok - attachement 18
System wide checks -
Clocks OK
ADC Calibration N/A
WR Decoder -
Base Current Difference
aida07 fault 0xc53d : 0xc5ac : 111
aida08 fault 0xf1be : 0xf28c : 206
White Rabbit error counter test result: Passed 6, Failed 2
FPGA -
Base Current Difference
aida07 fault 0x2a : 0x3f : 21
PLL OK
Currently on file R5_621
Analysis of R5_620 - Attachement 19
aida02 deadtime only 5%, all others negligible
20:20 Stats ok - attachment 20
Temps ok - attachment 21
Bias and leakage currents ok - attachment 22
System wide checks:
Clock ok
Base Current Difference
aida07 fault 0xc53d : 0xc5ad : 112
aida08 fault 0xf1be : 0xf293 : 213
White Rabbit error counter test result: Passed 6, Failed 2
Base Current Difference
aida07 fault 0x2a : 0x3f : 21
FPGA Timestamp error counter test result: Passed 7, Failed 1
Analysis of R6_637 - attachment 23
22:13 Statistics ok - attachment 24
Temperatures ok - attachment 25
Bias and leakage currents ok - attachment 26
ASIC clock check ok
Base Current Difference
aida07 fault 0xc53d : 0xc5ba : 125
aida08 fault 0xf1be : 0xf2a2 : 228
White Rabbit error counter test result: Passed 6, Failed 2
Base Current Difference
aida07 fault 0x2a : 0x40 : 22
FPGA Timestamp error counter test result: Passed 7, Failed 1
Analysis of R5_658 - attachment 27 |
|
Sat Jun 25 22:48:57 2022, TD, Sunday 26 June 00:00-08:00 30x
|
23.43 Check ASIC control
Zero stats & all histograms
ASIC settings 2021Apr29-13-16-00
slow comparator 0x64 -> 0xa
all waveform AD9252 ADCs disabled
all fast discs disabled
BNC PB-5 settings (to p+n FEE64s only)
amplitude 1.0V
attenuator x1
decay time 1ms
polarity +
frequency 22Hz
All system wide checks OK *except* WR & FPGA errors - attachments 1 & 2
DSSSD bias & leakage currents OK - attachment 3
adc data item stats - attachment 4
FEE64 temps OK - attachment 5
DSSSD bias & leakage currents OK - attachment 6
analysis of file S505/R5_415 - attachment 7
zero timewarps
max deadtime aida02 c. 17%, all other FEE64s < 1%
02:36
analysis of file S505/R5_449 - attachment 8
zero timewarps
max deadtime aida02 c. 15%, all other FEE64s < 1%
per FEE64 rate & stat spectra - attachments 9 & 10
per p+n FEE64 1.8.L spectra - attachment 11
aida01 pulser peak width 99 ch FWHM
per FEE64 1.8.H spectra - attachments 12 & 13
All system wide checks OK *except* WR & FPGA errors - attachments 14 & 15
adc data item stats - attachment 16
FEE64 temps OK - attachment 17
DSSSD bias & leakage currents OK - attachment 18
05:38
analysis of file S505/R5_483 - attachment 19
zero timewarps
max deadtime aida02 c. 16%, all other FEE64s < 2%
All system wide checks OK *except* WR & FPGA errors - attachments 20 & 21
adc data item stats - attachment 22
FEE64 temps OK - attachment 23
DSSSD bias & leakage currents OK - attachment 24
07:27
analysis of file S505/R5_502 - attachment 25
zero timewarps
max deadtime aida02 c. 16%, all other FEE64s < 2%
All system wide checks OK *except* WR & FPGA errors - attachments 26 & 27
adc data item stats - attachment 28
FEE64 temps OK - attachment 29
DSSSD bias & leakage currents OK - attachment 30 |
|
Sat Jun 25 17:06:42 2022, BA, AA, Saturday 25 June 2022 16:00-00:00 9x
|
Took over the night shift from Magda
18:00 : attachments 1-3
20:00 : attachments 4-6
22:00 : attachments 7-9
00:00 : attachments 10-12
|
|
Sat Jun 25 06:55:07 2022, MS, Saturday 25 June 2022 8:00-16:00 15x
|
Took over the night shift from Tom.
7:00 The beam is back.
8:00 : attachments 1-3
10:00 : attachments 4-6
12:00 : attachments 7-9
14:00 : attachments 10-12
16:00 : attachments 13-15
|
|
Fri Jun 24 22:36:45 2022, TD, Saturday 25 June 00:00-08:00 31x
|
23:36 Check ASIC contorol
Zero stats & all histograms
ASIC settings 2021Apr29-13-16-00
slow comparator 0x64 -> 0xa
all waveform AD9252 ADCs disabled
all fast discs disabled
BNC PB-5 settings (to p+n FEE64s only)
amplitude 1.0V
attenuator x1
decay time 1ms
polarity +
frequency 22Hz
analysis of file S505/R5_154 - attachment 1
zero timewarps
max deadtime aida02 c. 17%, all other FEE64s < 5%
stats - attachments 2-6
per FEE64 1.8.H spectra - attachments 7 & 8
per p+n FEE64 1.8.L spectra - attachment 9
aida01 pulser peak width 101 ch FWHM
per FEE64 stat & rate spectra - attachments 10 & 11
adc data item stats - attachment 12
FEE64 temps OK - attachment 13
DSSSD bias & leakage currents OK - attachments 14 & 15
00:20 all system wide checks OK *except* WR and FPGA errors - attachments 16 & 17
03:03
analysis of file S505/R5_189 - attachment 18
zero timewarps
max deadtime aida02 c. 17%, all other FEE64s < 5%
all system wide checks OK *except* WR and FPGA errors - attachments 19 & 20
adc data item stats - attachment 21
FEE64 temps OK - attachment 22
DSSSD bias & leakage currents OK - attachment 23
03:30
per FEE64 1.8.H spectra - attcahments 24 & 25
06:09 no beam
analysis of file S505/R5_221 - attachment 26
zero timewarps
all dead times << 1%
all system wide checks OK *except* WR and FPGA errors - attachments 27 & 28
adc data item stats - attachment 29
FEE64 temps OK - attachment 30
DSSSD bias & leakage currents OK - attachment 31 |
|
Fri Jun 24 15:04:06 2022, Marc, Friday 24th June - evening shift 11x
|
16:05 - Last checked was at 15:30. (see previous entry. All running smoothly.
Next wide check will be in about an hour.
17:00
Stats -ok - attachment 1
Temperatures ok - attachement 2
Leakage current ok but - attachment 3
ucesb screen-shot - attachment 4
Wide check completed. Nothing different.
WR status decoder status:
Base Current Difference
aida07 fault 0xc53d : 0xc547 : 10
aida08 fault 0xf1be : 0xf1e9 : 43
White Rabbit error counter test result: Passed 6, Failed 2
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
FPGA timestamp::
Base Current Difference
aida07 fault 0x2a : 0x2c : 2
FPGA Timestamp error counter test result: Passed 7, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
Note that the leakage current is ok but has increased since yesterday , See graphana - attachement 5 - Prossibly due to higher beam intensity.
19:20 -
Stats ok - attachment 6
Temp ok - attachment 7
Leakage current ok - attachement 8
Wide check completed and same output as above.
22:00
Stats ok - attachment 9
Temp ok - attachment 10
Leakage current ok - attachement 11
Wide check completed and same output as above.
|
|
Fri Jun 24 06:56:06 2022, OH, Friday 24 June 008:00-16:00 16x
|
07:56 Took over from AM. No issues reported overnight
Stats ok - attachment 1
Temperature ok - attachment 2
Bias and leakage current ok - attachment 3
System wide checks:
Clocks ok
WR
Base Current Difference
aida07 fault 0xc4fe : 0xc53b : 61
aida08 fault 0xf0e9 : 0xf1b7 : 206
White Rabbit error counter test result: Passed 6, Failed 2
FPGA Errors
Base Current Difference
aida07 fault 0x11 : 0x29 : 24
aida08 fault 0x1a : 0x2c : 18
FPGA Timestamp error counter test result: Passed 6, Failed 2
Analysis of R3_359 - attachment 4
Dead time still around 15.9%
Merger item rate around 2E6-4E6
Tape server rate at 7750 kB/s
Current HDD free space 1.7 TB
Time left in HDD 2.59 days
Will run out of space at some point on Sunday
Have started compression of uncompressed raw data on the HDD. Using nice +10
08:45 AIDA07 dropped out from the merger at some point in the preceeding 5-10 minutes. - attachment 5
Am regularly checking the statistics
Was able to stop the DAQ by relaunching the merger.
Reset the FEEs and recovered without a powercylce
Started R4 following this stop
08:58 They have taken the beam to change the ion source for maybe 2 hours
Will stop writing data but continue forwarding to MBS
No storage ticked. Following break will be on R5
11:10 Beam is starting to come back R5 started
Stats ok - attachment 6
Temps ok - attachment 7
Bias and leakage currents ok - attachment 8
13:05 Statistics ok - attachment 9
Temps ok - attachment 10
Bias and leakage curents ok - attachment 11
System wide checks:
WR
Base Current Difference
aida07 fault 0xc53d : 0xc540 : 3
aida08 fault 0xf1be : 0xf1d0 : 18
White Rabbit error counter test result: Passed 6, Failed 2
FPGA
Base Current Difference
aida07 fault 0x2a : 0x2b : 1
FPGA Timestamp error counter test result: Passed 7, Failed 1
Analysis of R5_33 - attachment 12
15:28 Statistics ok - attachment 13
Temperature ok - attachment 14
Bias and leakage current ok - attachment 15
System wide checks:
Base Current Difference
aida07 fault 0xc53d : 0xc546 : 9
aida08 fault 0xf1be : 0xf1e0 : 34
White Rabbit error counter test result: Passed 6, Failed 2
Base Current Difference
aida07 fault 0x2a : 0x2b : 1
FPGA Timestamp error counter test result: Passed 7, Failed 1
Analysis of R5_63 - attachment 16 |
|
Fri Jun 24 00:34:55 2022, AM, Friday 24 June 00:00-08:00 20x
|
01:30 Attachments 1-5, white rabbit and fpga timestamp failures, otherwise all good
03:30 Attachments 11-15, white rabbit and fpga timestamp failures, otherwise all good
05:30 Attachments 6-10, white rabbit and fpga timestamp failures, otherwise all good
07:30 Attachments 16-20, white rabbit and fpga timestamp failures, otherwise all good |
|
Thu Jun 23 15:10:17 2022, Philippos Papadakis, Thursday 23 June 16:00-24:00 15x
|
16:20
System wide checks.
Clock status OK
White Rabbit Error status:
Base Current Difference
aida07 fault 0xc4fe : 0xc525 : 39
aida08 fault 0xf0e9 : 0xf166 : 125
White Rabbit error counter test result: Passed 6, Failed 2
FPGA error status:
Base Current Difference
aida07 fault 0x11 : 0x1c : 11
aida08 fault 0x1a : 0x23 : 9
Temps OK (attachment 1), Stats OK (attachment 2), Bias and leakage OK (attachment 3)
18:05
System wide checks.
Clock status OK
White Rabbit Error status:
Base Current Difference
aida07 fault 0xc4fe : 0xc527 : 41
aida08 fault 0xf0e9 : 0xf177 : 142
White Rabbit error counter test result: Passed 6, Failed 2
FPGA error status:
Base Current Difference
aida07 fault 0x11 : 0x1c : 11
aida08 fault 0x1a : 0x24 : 10
FPGA Timestamp error counter test result: Passed 6, Failed 2
Temps OK (attachment 4), Stats OK (attachment 5), Bias and leakage OK (attachment 6)
20:22
System wide checks.
Clock status OK
White Rabbit Error status:
Base Current Difference
aida07 fault 0xc4fe : 0xc529 : 43
aida08 fault 0xf0e9 : 0xf17f : 150
White Rabbit error counter test result: Passed 6, Failed 2
FPGA error status:
Base Current Difference
aida07 fault 0x11 : 0x20 : 15
aida08 fault 0x1a : 0x25 : 11
FPGA Timestamp error counter test result: Passed 6, Failed 2
Temps OK (attachment 7), Stats OK (attachment 8), Bias and leakage OK (attachment 9)
21:56
System wide checks.
Clock status OK
White Rabbit Error status:
Base Current Difference
aida07 fault 0xc4fe : 0xc529 : 43
aida08 fault 0xf0e9 : 0xf186 : 157
White Rabbit error counter test result: Passed 6, Failed 2
FPGA error status:
Base Current Difference
aida07 fault 0x11 : 0x21 : 16
aida08 fault 0x1a : 0x26 : 12
Temps OK (attachment 10), Stats OK (attachment 11), Bias and leakage OK (attachment 12)
23:50
System wide checks.
Clock status OK
White Rabbit Error status:
Base Current Difference
aida07 fault 0xc4fe : 0xc52e : 48
aida08 fault 0xf0e9 : 0xf18c : 163
White Rabbit error counter test result: Passed 6, Failed 2
FPGA error status:
Base Current Difference
aida07 fault 0x11 : 0x21 : 16
aida08 fault 0x1a : 0x27 : 13
FPGA Timestamp error counter test result: Passed 6, Failed 2
Temps OK (attachment 13), Stats OK (attachment 14), Bias and leakage OK (attachment 15) |
|
Thu Jun 23 06:56:02 2022, OH, Thursday 23 June 08:00-16:00 11x
|
07:56 Taken over from AM
Current White Rabbit Error status:
Base Current Difference
aida07 fault 0xc4fe : 0xc51a : 28
aida08 fault 0xf0e9 : 0xf132 : 73
White Rabbit error counter test result: Passed 6, Failed 2
FPGA error status:
Base Current Difference
aida07 fault 0x11 : 0x18 : 7
aida08 fault 0x1a : 0x20 : 6
FPGA Timestamp error counter test result: Passed 6, Failed 2
Likely that at some point AIDA08 will drop out again. I think this occurs if there is a sudden burst of WR errors which causes the merger to drop the link
Statistics ok - attachment 1
FEE temperatures ok - attachment 2
Bias and leakage currents ok - attachment 3
Analysis of file R3_93 - attachment 4
Still deadtime in aida02 of ~15%
Patrick has said he will look into whether we are still sending info code 4 and 5 for every ADC event which could be contributing to the deadtime.
Currently 2.2 TB of space on /media/SecondDrive
Current data rate of 6.23MB/s
At current data rate this should last 4 days
08:27 Currently no beam
Problem with Alvarez 3 now
Tomorrow will have a 1 hour break due to changing of the source
09:36 Beam back
09:59 System wide checks
Base Current Difference
aida07 fault 0xc4fe : 0xc51b : 29
aida08 fault 0xf0e9 : 0xf138 : 79
White Rabbit error counter test result: Passed 6, Failed 2
Base Current Difference
aida07 fault 0x11 : 0x18 : 7
aida08 fault 0x1a : 0x20 : 6
Stats ok - attachment 5
Temperatures ok - attachment 6
Bias and leakage currents ok - attachment 7
11:14 FRS group is changing from Hg setting back to Pt
11:36 Back on Pt R3_129
14:14 Stats ok - attachment 8
Temperature ok - attachment 9
Bias and leakage currents ok - attachment 10
System wide checks:
WR:
Base Current Difference
aida07 fault 0xc4fe : 0xc523 : 37
aida08 fault 0xf0e9 : 0xf156 : 109
White Rabbit error counter test result: Passed 6, Failed 2
FPGA
Base Current Difference
aida07 fault 0x11 : 0x1b : 10
aida08 fault 0x1a : 0x20 : 6
Analysis of R3_156 - attachment 11 |
|
Wed Jun 22 23:18:45 2022, AM, OH, TD, MA, Thursday 23 June 19x
|
00:00 TD Noticed on the last stats uploaded that AIDA08 had stopped sending signals.
Attempted to recover restarting merger but this caused 01 to drop out.
00:10 DAQ recovered and AIDA01 recovered with a reboot
01:00 Attachments 1-4, white rabbit failures on aida07 and aida08, otherwise all good
03:00 Attachments 5-9, aida07 and aida08 failures on white rabbit and fpga timestamp, otherwise all good
05:00 Attachments 10-14, aida07 and aida08 failures on white rabbit and fpga timestamp continue, otherwise all good
07:00 Attachments 15-19, aida07 and aida08 failures on white rabbit and fpga timestamp continue, otherwise all good |
|
Wed Jun 22 15:09:37 2022, MA, Wednesday 22 June 16:00-00:00 12x
|
16:00 Take over from Magda mrning shift
Beam was off of sometime for tunning to Mercurey setting.
17:00 attached 1-3
19:00 attached 4-6
21:00 attached 7-9
23:30 10-12
|
|
Wed Jun 22 10:09:12 2022, PJCS, INFO: FEE64 supply Voltages
|
Study of the FEE64 power supply distribution has yielded the following :-
The most sensitive regulator, as regards the device voltage input, is the TPS51100 that supplies the DDR memory reference voltages. This requires +4.75 to +5.25 volts. This is supplied from the Power input connector +5v input.
The common LT3080 regulator used over much of the board is a Low Voltage Dropout regulator. This requires 0.5v difference between input and output voltage as minimum. This is not a problem with the +4.75v minimum for the TPS51100 requirement setting the voltage for the board.
The supply to the mezzanine is direct from the power connector +5v input. On the mezzanine there is an LT3080 for each ASIC supplying the required 3.3v. These regulators would possibly benefit from a 1uF capacitor at the Control voltage input.
The simplest approach would be to add a capacitor to the bottom layer where the Control voltage enters the mezzanine.
The power cable has a nominal resistance of 13.3ohms/km. The 3 conductors of the cable are supplying 10A when all is in operation. So the expected voltage drop would thus be ( 10 x 13.3 x 0.007 ) /3 => 0.3v each core.
The conclusion would be that the voltage at the power supply should drop to 5.25 v thus ensuring the TPS51100 is supplied as required regardless of the operation of the FEE.
This will be tested at Daresbury. |
|
Wed Jun 22 07:12:46 2022, MS, Wednesday 22 June 8:00-16:00 16x
|
Took over the night shift from Tom.
8:10 : attachments 1-3
10:00 : attachments 4-6
12:00 : attachments 7-9
The beam was off for around 20' starting at around 11:30-11:50 due to ion source issues.
12:19 OH Disabled all discriminators to reduce the data rate.
13:06 Analysis of R2_104 15.66% deadtime on AIDA04- attachment 10
14:00 : attachments 11-13
16:00 : attachments 14-16
|
|
Tue Jun 21 23:29:16 2022, TD, Wednesday 22 June 00:00-08:00 33x
|
00.28 all histograms & stats zero'd
baseline counters
ASIC settings 2021Apr29-13-16-00
*except* slow comparator 0x64 -> 0xa
BNC PB-5 pulser
to p+n FEE64s only
amplitude 1.0V
attenuator x1
decay time 1ms
polarity +
frequency 2Hz
all AD9252 waveform ADCs disabled
Tape Server -> no storage mode
00.36 check ASIC control all FEE64s all ASICs
all system wide checks OK
00.41 BNC PB-5 pulser rate increased from 2Hz to 22Hz per request by HA/NH - to ensure DTAS correlation checker has sufficient rate of identified pulser events
per p+n FEE64 1.8.L spectra - attachment 1
aida01 pulser peak width 88 ch FWHM
per FEE64 rate spectra - attachment 2
adc, pause and correlation scaler data items - push, aida01 and aida02 stats - attachments 3-8
FEE64 temps OK - attachment 9
DSSSD bias & leakage current OK - attachments 10 & 11
02.28 aida05 zero adc data items, unable to halt aida05 - all other FEE64s OK
restart NewMerger -> now able to halt aida05
DAQ go OK, merge rate c. 1.4M data items/s
per FEE64 1.8.H spectra - attachments 12 & 13
approx flat distribution to c. 10GeV
adc data items stats OK - attachments 14
FEE64 temps OK - attachment 15
DSSSD bias & leakage current OK - attachments 16 & 17
02.55 Some problems with GO4 data analysis for PID (FRS PID looks OK). 204Pt setting. Will commence writing data to disk file S505/R2_0
04.12 per FEE64 1.8.H spectra - attachments 18 & 19
c. 9GeV peak in upstream DSSSDs (aida01, 2, 3 & 4) indicating (high energy) heavy fragments are stopping
Low energy events near c. 200MeV lower mass, high energy, low energy loss - fission fragments?
04.20 per FEE64 stats spectra - attachment 20
indicates implant x-y distribution - off centre (more significantly in x than in y) but DSSSDs appear to see most of distribution
remember raw distro will have significant fraction of fission fragments - may be misleading - need to gate on high energy events to see x-y distro of ions of
interest
adc data items stats OK - attachments 21
FEE64 temps OK - attachment 22
DSSSD bias & leakage current OK - attachments 23
06.10 per FEE64 rate spectra - attachment 24
per FEE64 1.8.H spectra - attachments 25 & 26
per FEE64 stats spectra - attachment 27
adc data items stats OK - attachments 28
FEE64 temps OK - attachment 29
system wide checks OK *except* WR & FPGA errors - see attachments 30 & 31
DSSSD bias & leakage current OK - attachments 32
07.52 analysis of file S505/R2_47 - attachment 33
all FEE64 dead times < 1% *except* aida02 c. 7% |
|
Tue Jun 21 10:19:05 2022, OH ML, Tuesday 21st June 08:00-00:00 12x
|
11:19 Stats ok 0xa on all FEE - attachment 1
Current pulser settings - attachment 2
FEE Temperatures ok - attachment 3
Noticed both AIDA03 and AIDA07 had ASICs with missing channels. Did a check ASIC control on them to recover
Layout 1 after recovering channels - attachment 4
New stats (Note increased rate on 3 and 7) attachment 5
Bias and leakage currents ok - attachment 6
S4 now in closed access. Trying to find out if the beam dump has been put in front of AIDA. Will make sure this is checked before beam to S4.
14:05 Have had it confirmed that the flange beam dump is in position in S4
15:43 Stats ok -attachment 7
Temp ok - attachment 8
Bias and Leakage current ok - attachment 9
17:45 Currently the time machine trigger(for correlation) is connected to the DTAS or.
As the beam is tuning they are triggering at 150k. This is 450k worth of scalers in AIDA
During the experiment this will be changed to SC1 trigger to solve this.
19:49: Stats ok - attachment 10
temperatures ok - attachment 11
Bias - Leakage current ok - attachment 12 |
|