| |
ID |
Date |
Author |
Subject |
|
|
486
|
Mon Jun 20 09:26:23 2022 |
OH, TD | Monday 20 June |
10:26 MACB HDMI for AIDA03 and AIDA04 reseated.
Merger and DAQ restarted - All happily sending data
Has been running for 10 minutes with no WR errors.
Rates from yesterday achieved again.
DTAS will move back into position at some point.
Once this is done, will perform pulser walkthrough.
11:14 DTAS now back in position for experiment
Slight increase in noise noticed across a few FEEs
Over time this has decreased on all but AIDA07 which is now running at ~80k - attachment 1
AIDA03 is now however gaining white rabbit errors at a rate of 1 every few minutes.
Less frequently gaining FPGA errors
12:20 AIDA power cycle to reseat the HDMI for AIDA03 - It would eventually crash if not done
Upon powering up again rates in AIDA08 slightly worse - attachment 2
13:26 No WR errors since the last power cycle.
21:16 Baseline system wide checks, zero histograms & stats
All system wide checks OK
per p+n FEE64 1.8.L spectra - attachment 3
aida01 pulser peak width 82 channels FWHM
per FEE64 rate spectra - attachment 4
adc, pause, resume and correlation scaler data item stats - attachments 5-8
FEE64 temps OK - attachment 9
DSSSD bias & leakage currents OK - attachments 10 & 11 |
| Attachment 1: 220620_1116_stats_no_waveform.png
|
|
| Attachment 2: 220620_1223_stats_no_waveform.png
|
|
| Attachment 3: Screenshot_from_2022-06-20_21-17-41.png
|
|
| Attachment 4: Screenshot_from_2022-06-20_21-14-12.png
|
|
| Attachment 5: Screenshot_from_2022-06-20_21-12-06.png
|
|
| Attachment 6: Screenshot_from_2022-06-20_21-10-55.png
|
|
| Attachment 7: Screenshot_from_2022-06-20_21-08-58.png
|
|
| Attachment 8: Screenshot_from_2022-06-20_21-04-58.png
|
|
| Attachment 9: Screenshot_from_2022-06-20_21-04-00.png
|
|
| Attachment 10: Screenshot_from_2022-06-20_21-02-07.png
|
|
| Attachment 11: Screenshot_from_2022-06-20_21-01-07.png
|
|
|
|
307
|
Wed May 12 06:59:55 2021 |
OH, RDP | Wed 12 May 08:00-16:00 |
08:00 While the rates appeared to have quietened around 7am they are back to their previous rate now.
They still seem to be working on the setting. Implant rate in AIDA about 8 per spill. Thick degrader is still in place
They have collected 25 minutes of implantation data. They are analysing this data to confirm the PID
08:38 The PID confirmation was apparently run number 4. During this run they were implanting Cd in AIDA DSSD2
09:09 They are now taking data implanting somewhere in the snout. I have turned off no storage.
Unfortunately we are writing to NOTAPE R3. I was unable to change it as they were already taking data
Data rate 16.3MB/s
ucesb shows similar rates in SC42 and AIDA01. We may be implanting Te in AIDA then.
09:35 They stopped the DAQ as their is no beam. Switched the tapeserver over to R1 in directory S496. Setting to no storage
09:47 Beam back turned off no storage. On file R1_10
11:04 Now that we have implants on the detector we get a better look at the strips
From the 1d pattern it is obvious that there is an adapter card misaligned in DSSD1- attachments 1 and 2
This adaptor board corresponds to FEE11
12:17 Leakage currents screenshot 512.png
13:30 Beam off. Expected back at 15:00...
14:09 Making use of the beam off time NH and HA entered S4 to investigate FEE1 and FEE11.
They noticed FEE11 was off and have realigned that. All channels are now observed in that FEE
They couldn't see anything wrong with the connector for FEE1. We are still missing a block of channels there.
Rates are as before. No better, no worse.
15:31 Leakage currents screenshot 513.png. They are continuing to rise following the intervention in S4. It could just be because the temperature is rising... |
| Attachment 1: 1dImplants.png
|
|
| Attachment 2: 1dImplants2.png
|
|
| Attachment 3: 512.png
|
|
| Attachment 4: 513.png
|
|
|
|
323
|
Tue May 18 06:50:53 2021 |
OH, PP | Tuesday 18th May 08:00 - 16:00 |
08:00 OH Takes over
Statistics still look good from yesterday. Rate is still down.
Dead time in FEEs 2 and 4 is 2.04%
System wide checks clock ok
Base Current Difference
aida05 fault 0x4da : 0x500 : 38
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
*Baseline reset*
FPGA check ok
Stats - attachment 1
Temp - attachment 2
Bias - attachment 3
08:11 With the decreased noise I am investigating whether we can lower the thresholds on DSSD2 y strips.
I have set them to 0x19 on ASICS 1-3 and 0x64 on ASIC4
The results of the test appear to indicate that it is the network link that we are saturating causing dead time
*** Before After
FEE DT (s) Dead time % DT (s) Dead time %
0 0.013 0.004184451 0.026 0.010025681
1 6.352 2.04458693 4.097 1.57981599
2 0.295 0.09495484 0.292 0.112596112
3 6.349 2.043621288 8.255 3.183153771
4 0.021 0.006759497 0.03 0.011568094
5 0.285 0.091736032 3.388 1.306423377
6 0.02 0.006437616 0.149 0.057454865
7 1.219 0.392372712 2.133 0.822491459
8 1.137 0.365978485 0.457 0.176220627
9 0 0 0 0
10 1.633 0.525631369 1.545 0.595756823
11 0.061 0.01963473 0.166 0.064010118
12 0.059 0.018990968 0.085 0.032776265
13 0 0 0 0
14 0.016 0.005150093 0.023 0.008868872
15 0.282 0.09077039 0.168 0.064781324
Note that the dead time for FEE6 and 8 remains fairly low but the dead time for FEE4 has increased
09:00 They are stopping the run to decrease the bias on the bPlast detector
09:25 No noticable change in AIDA
They will now begin to change the spill structure.
09:46 While they are optimising the 1s spill structure raised the thresholds of FEE6 and 8 back to 0x20. Will start from here when optimising the thresholds post spill change
10:21 Beam is back on AIDA
10:39 System wide checks.
- ADC calibration check failed (this is anticipated because waveforms are off).
- 1 Failed test in White Rabbit:
Base Current Difference
aida05 fault 0x500 : 0x506 : 6
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
- All other systems passed
11:00
Stats - attachment 4
Temp - attachment 5
Bias - attachment 6
12:46 System wide checks
- ADC calibration check failed (this is anticipated because waveforms are off).
- 1 Failed test in White Rabbit:
Base Current Difference
aida05 fault 0x500 : 0x50b : 11
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
- All other systems passed
14:36 System wide checks
- ADC calibration check failed (this is anticipated because waveforms are off).
- 1 Failed test in White Rabbit:
Base Current Difference
aida05 fault 0x500 : 0x50f : 15
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
- All other systems passed |
| Attachment 1: 210518_0757_Stats.png
|
|
| Attachment 2: 210518_0758_Temp.png
|
|
| Attachment 3: 210518_0759_Bias.png
|
|
| Attachment 4: 210518_1059_Stats.png
|
|
| Attachment 5: 210518_1100_Temp.png
|
|
| Attachment 6: 210518_1101_Bias.png
|
|
|
|
412
|
Wed Apr 6 10:09:21 2022 |
OH, NH, TD | Wednesday 6 April |
DSSSD bias OFF. DAQ STOPped
Attachment 6 FEE64 temps OK
Attachment 1-5 system wide checks OK |
| Attachment 1: Screenshot_from_2022-04-06_11-12-59.png
|
|
| Attachment 2: Screenshot_from_2022-04-06_11-12-48.png
|
|
| Attachment 3: Screenshot_from_2022-04-06_11-12-38.png
|
|
| Attachment 4: Screenshot_from_2022-04-06_11-10-06.png
|
|
| Attachment 5: Screenshot_from_2022-04-06_11-09-57.png
|
|
| Attachment 6: Screenshot_from_2022-04-06_11-09-29.png
|
|
|
|
413
|
Thu Apr 7 12:42:14 2022 |
OH, NH, TD | Thursday 7 April |
Re-assembled triple snout re-installed on the AIDA stand - DSSSD ribbon cables and grounds re-connected.
FEE64 adaptor PCB LKs per https://elog.ph.ed.ac.uk/DESPEC/409
Attachments 1 & 2 1.8.L spectra
p+n FEE64 aida10 pulser peak width 97 ch FWHM
n+n FEE64 aida04 pulser peak width 550 ch FWHM
Attachment 3 rate spectra
missing channels aida01, aida07 & aida11 - misaligned ERNI and/or Samtec connectors?
Attachments 4 & 5 1.8.W spectra
Attachment 6 ADC data item rates
aida10 OK, all others c. 300-400k
Attachment 7 FEE64 temperatures OK
Attachment 8 DSSSD bias & leakage currents OK |
| Attachment 1: Screenshot_from_2022-04-07_13-40-55.png
|
|
| Attachment 2: Screenshot_from_2022-04-07_13-40-02.png
|
|
| Attachment 3: Screenshot_from_2022-04-07_13-37-17.png
|
|
| Attachment 4: Screenshot_from_2022-04-07_13-35-43.png
|
|
| Attachment 5: Screenshot_from_2022-04-07_13-35-23.png
|
|
| Attachment 6: Screenshot_from_2022-04-07_13-34-34.png
|
|
| Attachment 7: Screenshot_from_2022-04-07_13-34-12.png
|
|
| Attachment 8: Screenshot_from_2022-04-07_13-33-53.png
|
|
|
|
428
|
Wed May 11 07:04:13 2022 |
OH, NH, TD | Wednesday 11th May |
08:00 Overnight at some point it appears that aida14 has stopped producing any ASIC data
It is currently sending data pause requests and data push requests but nothing else - attachment 1
telnet into aida14 and run top to see what the current state of the FEE is. AIDAExec currently using 0% CPU usage.
FEE appears to have hung. - attachment 2
Still passes all system wide checks.
When performing a check load can see the tclsh process take up CPU usage
09:02 AIDA out of time sorter. Going to perform a power cycle to see if it recovers FEE14
AIDA14 recovered but ASIC2 still missing
Current working hypothesis is ASIC2 on AIDA14 is dead and that the problem with AIDA05 is either DSSD, cable or adaptor board.
Given access would change the adapter board on AIDA05.
Could consider installing the old AIDA05 FEE above AIDA14 and moving the cables up 1.
Would retain the option of a speedybacktrack
Alternatively the HEC events in AIDA05 are just above threshold. Can raise the HEC threshold
A threshold of 0x4 reduced the rate in the HEC channels considerably
A threshold of 0x6 removes entirely
Note when the rate of HEC channels decreases can see the pulser in all channels of AIDA05 again. Showing that the adapter board is properly aligned
09:59 Had access so replaced the adapter card on AIDA05
Did note that while replacing the card the right hand edge of the right hand ribbon cable was lifted slightly.
This could also have been the cause
10:09 Restarted DAQ and there is no change to the behavior of AIDA05
Still see events in HEC
Again raising threshold to 0x5 in ASIC1 largely removes them.
13.10 Attachments 3-5
DAQ histogramming only enabled, waveforms/data transfer disabled, sampling ADCs OFF
ASIC check load
ASIC slow comparator all FEE64s all ASICS 0xa
ADC data item stats
13.17 Attachments 6-9
DAQ histogramming & data transfer enabled, waveforms disabled, sampling ADCs OFF
ADC data item stats
NewMerger/Tape Server
18:00 As part of tests in the afternoon it was investigated powering only one DSSD worth of FEEs at a time
It was found that reasonable rates could be found for DSSD1 but poor rates only for DSSD2
To try and help isolate the two detectors the jumpers 2 and 4 were removed from FEEs 13, 5 and 14 (Top, DSSD2)
After powering on it was found there was no improvement and possible worse noise than before
We were able to recover the rates in the 1DSSD setup so instead have removed FEE power from FEE5 and FEE14 |
| Attachment 1: 220511_0806_aida14.png
|
|
| Attachment 2: 220511_0817_aida14_top.png
|
|
| Attachment 3: Screenshot_from_2022-05-11_13-11-09.png
|
|
| Attachment 4: Screenshot_from_2022-05-11_13-10-29.png
|
|
| Attachment 5: Screenshot_from_2022-05-11_13-09-12.png
|
|
| Attachment 6: Screenshot_from_2022-05-11_13-16-32.png
|
|
| Attachment 7: Screenshot_from_2022-05-11_13-16-24.png
|
|
| Attachment 8: Screenshot_from_2022-05-11_13-16-01.png
|
|
| Attachment 9: Screenshot_from_2022-05-11_13-15-45.png
|
|
|
|
92
|
Tue Nov 12 13:22:19 2019 |
OH, NH, CA | AIDA09 Replacement |
ASIC 1 on FEE9 was not producing signals.
A new HDMI cable was installed but still no signals.
AIDA09 was replaced with a new FEE.
Current list of FEE to serial numbers installed attachment 1
MAC address of new FEE was obtained.
Backup of DHCPD.conf was made dhcpd.confBACKUP191112
dhcpd.conf was updated with new MAC address.
All FEEs powered on and 09 was seen to mount and is seen by AIDAServer.
AIDA09 flashed to newest version of firmware. (0x18430701) - attachment 2 |
| Attachment 1: FEE-SerialNumber.txt
|
FEE# Messanie Serial No. MAC Address
4 026-1690101-0016 d8-80-39-41-f6-b7
8 026-1690101-0013 d8-80-39-41-ba-2b
12 026-1690101-0030 d8-80-39-41-cf-ac
2 026-1690101-0011 d8-80-39-41-ba-22
6 026-1690101-0035 d8-80-39-41-ee-72
10 026-1690101-0032 d8-80-39-41-ee-10
1 026-1690101-0027 d8-80-39-41-ba-8a
5 026-1690101-0008 d8-80-39-41-d7-cc
9 026-1690101-0015 d8-80-39-41-f6-ee
7 Not checked d8-80-39-41-b4-0c
11 026-1690101-0006 d8-80-39-41-f6-5a
3 026-1690101-0019 d8-80-39-41-d8-21
Dead FEE Messanie Serial No. MAC Address
ASIC 1 Dead 026-1690101-0005 d8-80-39-42-0d-0c
|
|
|
390
|
Wed Oct 27 15:12:50 2021 |
OH, NH | Noise tests - 27th October |
Began by installing the triple.
Only the upstream detector was connected to the FEEs the downstream detector cables were left loose. The two detectors are isotolated though.
Initial cabling configuration
No pulser connections to any of the FEEs connected to the DSSD.
Bias core connected to FEE 13 - FEE 3 and FEE 11 with connections made by T pieces
Bias braid connected to FEE 4 - FEE 2 with connection made by T piece
Adaptor boards grounded to FEE cooling plate via lemo
DSSD cable drain wires also connected to LEMOs
LK1 not on any adaptor boards.
LK2, 3 and 4 on all
System started and rates very high in all of the FEEs connected to DSSD.
N.B FEE 8 connected to pulser but not to cable and adaptor board not grounded to FEE cooling plate
FEE14 connected to pulser and adaptor board grounded to FEE cooling plate
Statistics - Attachment 1
It was noted one of the ribbon cable connectors for FEE13 had come off
Statistics fixed - Attachment 2
Representative waveforms showing the noise - Attachments 3 and 4
Bias turned off and HV filters connected
Issue at first as the solder connection in one filter had broken. Filter was replaced and the DSSD bias
Statistics are unchanged
Waveforms with the HV filters installed - Attachments 5 and 6 |
| Attachment 1: 271021-waveaida10-pndssd-filter.png
|
|
| Attachment 2: 271021-dssds-fixed12.png
|
|
| Attachment 3: 271021-waves-dssd-nn.png
|
|
| Attachment 4: 271021-waveaida10-pndssd.png
|
|
| Attachment 5: 271021-waves-aida02-dssdnn-filter.png
|
|
| Attachment 6: 271021-waveaida10-pndssd-filter.png
|
|
|
|
397
|
Tue Mar 1 08:29:19 2022 |
OH, NH | Findings from the upgrade to centos7 from SL6 |
Required packages not included as default
- csh (For running MIDAS scripts)
- xinetd (Runs the rdate server for the FEEs)
- Telnet (Communication with the FEEs)
- dhcp
Configuration changes
Optional extras
- For top bar icons similar to SL6 can use the gnome extension "Frippery panel favorites"
- Then just need to create desktop icons for each of the startup scripts within /usr/share/applications - Attachment 1
- Can also add screenshot to this - Attachment 2
- Gnome tweaks also allows the number of workspaces to be changed
- Also recommend setting workspaces to span displays
|
| Attachment 1: TapeServer.desktop
|
[Desktop Entry]
Name=TapeServer
Exec=/MIDAS/Linux/startup/TapeServer
Terminal=true
Icon=/home/npg/Downloads/midas100.gif
Type=Application
Comment=TapeServer for AIDA
|
| Attachment 2: Screenshot_from_2022-03-01_09-40-18.png
|
|
|
|
400
|
Wed Mar 2 16:00:01 2022 |
OH, NH | Noise tests - 2nd March |
An interesting find with the new sum inverter is that with the pulser at default settings the pulser peak on the n+n side is above the range of the ADC.
We have checked signals from the sum inverter and pulser and both are equal in magnitude.
We therefor determine it is over range as the charge is spread over 4 FEEs vs 12 on the p+n side.
This possibly wasn't an issue with the previous inverter if it produced a lower amplitude.
For tests going forward p+n voltage will be 2V. n+n voltage will be 0.5V - Attachment 1
Going to look into a 20dB attenuator
Seems there were issues with the lemo cables in the n+n pulser chain |
|
|
401
|
Thu Mar 3 08:45:03 2022 |
OH, NH | AIDA Noise and grounding tests 3rd March |
Pulser spectra
- Changes to the nn pulser circuit yesterday have improved the peak widths on the p+n side
- p+n peak widths averact around 100 channels
- n+n around 1000 at 0xa threshold
- n+n 1.8.L spectra - attachment 1
- p+n 1.8.L spectra attachment 2
p+n peak widths
| FEE |
Width |
| 9 |
105 |
| 1 |
82 |
| 10 |
71 |
| 11 |
79 |
| 3 |
96 |
| 12 |
72 |
| 13 |
191 |
| 5 |
101 |
| 14 |
129 |
| 15 |
118 |
| 7 |
110 |
| 16 |
100
|
n+n peak widths
| FEE |
Width |
| 2 |
1200 |
| 4 |
1179 |
| 6 |
- |
| 8 |
1813 |
Daily checks
- Fee temperatures ok - attachment 3
- Bias and leakage currents ok - attachment 4
- Stats spectra with current grounding - attachment 5
- Current grounding is just a cable going between each adaptor board and its respective FEE
- Interfee grounding done by power supply and also pulser circuit
- Rates for each of the asics - attachment 6
- p+n waveforms - attachment 7
- n+n waveforms - attachment 8
- Lots of noise evident on them
Plans for the day
- Connect FEEs up to the copper earth bars installed
- Powercycle and check noise levels
- If n+n remains significantly higher investigate the n+n pulser circuit further
- Investigate the effects of different jumper connetions At present n+n have all LK jumpers on and p+n have lk2-4
|
| Attachment 1: 220303_0927_nn_18L.png
|
|
| Attachment 2: 220303_0928_pn_18L.png
|
|
| Attachment 3: 220303_0932_TEMP.png
|
|
| Attachment 4: 220303_0933_Bias.png
|
|
| Attachment 5: 220303_0933_Stats.png
|
|
| Attachment 6: 220303_0936_Layout1.png
|
|
| Attachment 7: 220303_0941_pn_waveform.png
|
|
| Attachment 8: 220303_0942_nn_waveform.png
|
|
|
|
402
|
Fri Mar 4 08:40:36 2022 |
OH, NH | AIDA Noise and grounding tests 3rd March - Results |
Grounding implemented
- Each FEE given direct connection to heavy duty earth bar via adaptor board lemo ground
- Earth bars are then connected to platform ground on AIDA frame
- Photo of grounding - attachment 1
- Improvements not as large as expected from previous experience though
- Particularly in the n+n side where widths still appear awful
- While implementing the grounding it was noticed the ground lemo socket on FEE7 was damaged and no longer gripped onto a lemo cable
- It was noted that it would need changing at some point but continued with it for the time being
- Jumper configuration was n+n LK1-4 all on, p+n LK2-4 all on
p+n widths
| FEE |
Width |
| 9 |
147 |
| 1 |
116 |
| 10 |
92 |
| 11 |
77 |
| 3 |
128 |
| 12 |
72 |
| 13 |
75 |
| 5 |
59 |
| 14 |
75 |
| 15 |
80 |
| 7 |
150 |
| 16 |
75 |
- Compared to the width with no grounding the FEEs for the first DSSD (First 6) range from no change to slightly worse
- 1.8.L spectra - attachment 3
n+n widths
- Widths still very large
- Width for 8 is around 500
- 1.8.L attachment 2
Additional info
- Statistics - attachment 4
Jumper optimisation
- Went to our previously found optimal jumper configuration for the trips
- n+n LK1 only
- p+n LK 2 and 4 on all FEE
- p+n LK3 on non isolated cables (PCB ground on lower middle cable and n+n side bias on top middle cable)
- While changing the jumpers we also swapped out the damaged adaptor board on aida07 for a new one
- Saw an improvement for most FEEs
| FEE |
Width |
| 9 |
134 |
| 1 |
83 |
| 10 |
80 |
| 11 |
68 |
| 3 |
60 |
| 12 |
60 |
| 13 |
71 |
| 5 |
60 |
| 14 |
75 |
| 15 |
82 |
| 7 |
97 |
| 16 |
77 |
- Unfortunately saw no improvement for the n+n
- We event tested having the pulser only going into a single FEE (aida08)
- Still have a width of 622 for that FEE
- To compare FEEs we have layout 1 - attachment 5
- Layouot 1 on a common timescale - attachment 6
- Note that the rate is typically dominated by a single strip or a couple of strips within a FEE
- Many channels are just showing the pulser rate 25Hz
- p+n 1.8.L attachment 7
- p+n waveforms attachment 8
- n+n waveforms attachment 9
- Testing the noise on the n+n strips we observe noise with slow comparator set to 200 (2MeV)
- Meanwhile the p+n strips are silent at 0x64 for the most part.
- DAQ left running overnight with 0x64 on ASIC settings to see if we observe any alphas
Plan for the day
- Test daisy chaining the n+n across to the other n+n strips
- Currently the n+n braid is only plugged into either aida06 or aida08 (depending on DSSD)
- All of the jumpers are however grounded
- Will still test
- May investigate also trying this on LK3 on adaptors for aida01 and aida03
- Observe the output of the new sum inverter on the scope to check for variation in amplitude
|
| Attachment 1: PXL_20220303_104253926.jpg
|
|
| Attachment 2: 220303_1310_nn_18l.png
|
|
| Attachment 3: 220303_1313_pn_18L.png
|
|
| Attachment 4: 220303_1313_stats.png
|
|
| Attachment 5: 220303_1438_layout1.png
|
|
| Attachment 6: 220303_1439_layout1common.png
|
|
| Attachment 7: 220303_1446_pn_18L.png
|
|
| Attachment 8: 220303_1452_pn_waveform.png
|
|
| Attachment 9: 220303_1453_nn_waveforms.png
|
|
|
|
403
|
Sat Mar 5 10:43:45 2022 |
OH, NH | Grounding and noise tests 4th-5th March Summary |
Tests performed
- After leaving overnight noticed the pulser was not fully inserted into aida09
- Little change in rates observed in 9 but otherwise nothing else
- Connected bias of aida02 (n+n ground) to aida06 (n+n ground) and likewise 4-8
- No change observed in rates
- Also tested covering the snout in light tight material
- Unplugged the system monitor from 6 and 8, no change
- Connected the snout to ground with copper tape
- Rates became very unpredictable in the FEEs unsure if this was snout grounding related though
- Removed the ground and powercycled and rates restored to where previously
- Changed the bias supply jumper inside to have the bias line connected to the rack ground
- At the same time noticed that the ground cable connecting the adaptor board of aida06 to the FEE had become disconnected from the ring terminal
- Recrimped and inserted again
- Saw an improvment in the rates of the p+n strips and also the n+n strips - attachment 1
- Pulser widths however larger than yesterday
p+n width
| FEE |
Width |
| 9 |
118 |
| 1 |
88 |
| 10 |
75 |
| 11 |
77 |
| 3 |
67 |
| 12 |
66 |
| 13 |
96 |
| 5 |
89 |
| 14 |
107 |
| 15 |
93 |
| 7 |
118 |
| 16 |
87 |
n+n width
- Connected up entire pulser circuit to n+n FEEs
| FEE |
Width |
| 2 |
428 |
| 4 |
N/A |
| 6 |
428 |
| 8 |
343 |
Switching bias supply back
- because changes were made to the n+n ground, and the pulser circuit it was decided to go back and check the previous bias configuration with the module set in a floating configuration
- DAQ was then left overnight as had to leave to catch the bus
- Rates are observed to be back at their higher values - attachment 2
- n+n 1.8.L spectra - attachment 3
- p+n 1.8.L spectra - attachment 4
p+n widths
| FEE |
Wdith |
| 9 |
130 |
| 1 |
128 |
| 10 |
75 |
| 11 |
78 |
| 3 |
71 |
| 12 |
71 |
| 13 |
96 |
| 5 |
85 |
| 14 |
110 |
| 15 |
99 |
| 7 |
123 Dbl |
| 16 |
92 |
n+n widths
| FEE |
width |
| 2 |
56 |
| 4 |
983 |
| 6 |
523 |
| 8 |
358 |
- The results show no improvment to slightly worse performance so will go back to having the bias module grounded internally and check the previous conclusions hold before making further changes
Return to grounded bias module
Widths
p+n
| FEE |
Width |
| 9 |
106 |
| 1 |
79 |
| 10 |
72 |
| 11 |
78 |
| 3 |
70 |
| 12 |
73 |
| 13 |
102 |
| 5 |
112 |
| 14 |
105 |
| 15 |
110 |
| 7 |
131 |
| 16 |
104 |
n+n
| FEE |
Width |
| 2 |
- |
| 4 |
298 |
| 6- |
- |
| 8 |
278 |
- Don't see a pulser peak in 4 or 2
- Will replace cable between 4 and 2 and 2 and 6
- Statistics at 0x64 with pulser on - atachment 5
- Now see peaks - attachment 6
- Statistics at 0xa - attachment 7
| FEE |
Width |
| 2 |
284 |
| 4 |
1089 (Triple) |
| 6 |
371 |
| 8 |
318 |
With the grounded bias in place in the module move the braid cable to the frame rather than being connected to bias line
- Still observe a triple peak in 4 and have worse perfromance in 2 and no improvement in p+n
- 0x64 stats - attachment 7
- 0xa statistics - attachment 8
- p+n 1.8.L - attachment 9
- p+n waveform - attachment 10
- n+n waveform - attachment 11
- Same high frequency component in both
- 35 channel spacing corresponds to a frequency of ~1.4MHz
| FEE p+n |
Width |
| 9 |
109 |
| 1 |
118 |
| 10 |
78 |
| 11 |
80 |
| 3 |
70 |
| 12 |
73 |
| 13 |
92 |
| 5 |
85 |
| 14 |
99 |
| 15 |
100 |
| 7 |
123 |
| 16 |
92 |
| FEE n+n |
Width |
| 2 |
531 |
| 4 |
1185 (Triple) |
| 6 |
536 |
| 8 |
376 |
Next tests
- Returned braid to FEE 4 and 8
- Recovered earlier rates and performance
- Re-added the ground cable to the snout
- Much worse noise performance across all FEEs
- p+n rates in 200k
- Removed the ground from the snout
- Got really good rates originally - attachment 12
- Noticed 1 and 5 failed ADC calibration
- Calibrated them and there rates increased massively - attachment 13
- Notice that the pulser peak in FEE2 and 6 keeps dropping out
- As the cables have been changed multiple times thought it could be that an adaptor board was broken
- 4 seemed the most likely suspect with its high rates and triple peaks
- Replaced the adaptor board with one missing an outer ERNI pin (PIN doesn't interface with the FEE so didn't see any issues)
- Pulser peaks were still muissing
- Issue could be then with adaptor board 2 possibly
Conclusions
- Rates on the FEEs are in a better place now than they were at the start of the week
- p+n in particular have shown a large amount of improvement
- There are still however issues with the n+n FEEs
- I think there is likely an issue with one of the adaptor boards
- Possible aida02 at the pulser connection point
- Could explain the triple peaking in aida04 and the pulser issues with aida02 and aida06 dropping out.
- Would recomend swapping out aida02 as a test next week
- Other possible issues coiuld be the clingfilm around the cables has torn and we are grounding the cables to the inside of the snout
- Nic would like to use something other than clingfilm in future
- One suggestion would be mylar which could be held in place with strips of double sided tape
- This would make the cables less likely to stick to each other which is an issue with the tightness of the snout
- Could also try removing LK3 from FEE3 and 7 leaving the PCB ground floating
- I am at a slight loss of what else to try
- I have left the daq running at 0x64 overnight to check for alphas
- Histograms and statistics zeroed
|
| Attachment 1: 220304_1704_groundedbias_stats.png
|
|
| Attachment 2: 220305_1154_Stats.png
|
|
| Attachment 3: 220305_1157_nn_18L.png
|
|
| Attachment 4: 220305_1200_pn_18L.png
|
|
| Attachment 5: 220305_1336_0x64Stats.png
|
|
| Attachment 6: 220305_1341_nn_18L.png
|
|
| Attachment 7: 220305_1342_Stats.png
|
|
| Attachment 8: 220305_1405_0x64_stats.png
|
|
| Attachment 9: 220305_1421_pn_18L.png
|
|
| Attachment 10: 220305_1424_pn_waveform.png
|
|
| Attachment 11: 220305_1429_nn_waveform.png
|
|
| Attachment 12: 220305_1518_stats.png
|
|
| Attachment 13: 220305_1519_waveform_enabled.png
|
|
|
|
414
|
Fri Apr 8 08:59:42 2022 |
OH, NH | Friday 8th April |
09:59 Noticed that the right hand ribbon cable for aida07 was offset vertically by 1. i.e. Only 1 of the 2 rows of pins in. Unbiassed detectors and reconnected.
Rebiassed detectors - leakage current on DSSD2 increased to 9.96uA - Attachment 1
Power cycling FEEs
Following restart the FEEs show some improvement in rates - attachment 2
Rate spectra - attachment 3
Still issues with aida11 in the rate spectra (Likely a misaligned ERNI connector)
Pulser on rate spectra - attachment 4
Pulser off rate spectra - attachment 5
We can see that with the pulser on there are events in the channels in aida01 and aida11
With the pulser off these channels disappear.
ERNI connector is then likely aligned the issue is more likely the DSSD ribbon cable is misaligned.
Taking bias off to investigate.
10:39 Could not observe anything out of alignement about either the connectors or the adaptor boards
Note that after pushing on the connector AIDA01 had largely recovered
13:30 Top of snout taken off while mounted in S4. It was observed that one of the ribbon cables for aida11 was off by 1 row.
This was corrected in situ and the snout was put back together and made light tight again.
Oddly leakage current still the same - DSSD2 has come down though - attachment 6
FEEs powercycled for new tests
Half the channels in aida11 had appeared again
Took apart once more and found the other header also off by one row which has been corrected
FEEs powercycled for new test
All of aida11 now showing - attachment 7
Statistics at 0xa shown in attachment 8
All p+n for DSSD1 are good (aida01, 03, 09, 10, 11 and 12)
p+n for DSSD2 are so so (aida05, 07, 13, 14, 15 and 16)
All n+n 2, 4, 6 and 8 are poor still
14:50 p+n waveforms - attachment 9
n+n waveforms - attachment 10
Current jumper configuration
LK1 (Bias ground) on FEEs 6 and 8
LK2&4 on all p+n
LK3 on aida03 and aida07 (Bottom middle)
15:00 Investigated whether there was a light leak. Placed black material over snout.
Leakage on DSSD2 dropped to 5.6 - attachment 11
There was a light leak
Stats have decreased slightly - attachment 12
16:00 Move raspberry pis atop 5mm alminimum plate to keep away from FEE64 PSUs
No change
16:30 Remove pulser network from all FEEs
Rates get noticably worse in many FEEs... why?
Stats - attachment 13
Possible ideas include braid touching things it shouldn't (Adapter PCBs?) |
| Attachment 1: 220408_1000_bias.png
|
|
| Attachment 2: 220409_1010_Stats.png
|
|
| Attachment 3: 220409_1011_rate.png
|
|
| Attachment 4: 220408_1358_bias.png
|
|
| Attachment 5: 220408_1444_rate.png
|
|
| Attachment 6: 220408_1444_stats.png
|
|
| Attachment 7: 220408_1449_layout7.png
|
|
| Attachment 8: 220408_1451_layout8.png
|
|
| Attachment 9: 220308_1459_bias.png
|
|
| Attachment 10: 220408_1503_stats_post_light.png
|
|
| Attachment 11: 2022-04-08_16-26-30_Stats_No_Pulser.png
|
|
|
|
482
|
Sat Jun 18 09:46:57 2022 |
OH, NH | Friday 17th June |
Summary of the day:
Small improvements were made throughout the day but by far the biggest change to the rate in the FEEs came from disabling the waveforms
This was done by setting ADC con
Initial jumper configuration:
FEE LK
1 2 4
2 1 2 4
3 2 3 4
4 2 4
5 2 4
6 1 2 4
7 2 3 4
8 1 2 4
13:22 LK2, 3 and 4 on all p+n - attachment 1
14:20 AC mains relay connected to AC voltage stabilisers + on separate mains socket from DESPEC - attachment 2
Same configuration but waveforms disabled (p+n improvment) - attachment 3
Layout 1 - attachment 4
14:32 all FEE power cables moved to PSU 2 (Lower one in rack, upper one was slow to start for 1 FEE pair, likely the 5V slow to start)
Previously all FEEs on their own pair within a psu
New order 1,3 2,4 5,7 6,8
Waveforms on - all bad - attachment 5
Waveforms off - all p+n better - attachment 6
14:54 interlock removed from AC mains relay
Also noticed adaptor board on 2 not in fully -> Now in
p+n generally better - attachment 7
Without waveforms - p+n better again - attachment 8
16:04 Platform moved into position
Interlock power cable removed from extension lead
w/ waveform - p+n slightly worse
w/o waveform - p+n same as beffore
16:14 LK3 removed from 1 and 5
w/ waveform -> p+n slightly worse - attachment 9
w/o waveform -> p+n best yet - attachment 10
16:33 LK1 wasn't on all n+n was only on 2, 6 and 8
Removed from 6 -> only one LK1 per DSSD
w/ waveforms - all bad - attachment 11
w/o waveforms - no change - attachment 12
16:59 LK1 removed from 2->4 This is the FEE with the BIAS braid
w/ waveform p+n better - attachment 13
w/0 waveform p+n better, n+n worse? - attachment 14
17:21 Bias filters added to n+n
w/ waveform - no change - attachment 15
w/o wwaveofrm - P=n worse - attachment 16
Stats 0x21 - attachment 17
Stats 0x14 - attachment 18
Stats 0x12 - attachment 19
17:42 Bias filter removed and bias floated (LK1 removed)
w/ waveform all worse - attachment 20
w/o waveform -> p+n worse, n+n similar
18:29 LK1 back on 4+8
w/ waveform -> poor
w/0 waveform - recovered previous - attachment 20
18:32 LK 2 and 4 removed from 2 +6
Also noticed ground lemo on aida02 not fully inserted which has been corrected
w/ waveform poor - attachment 21
Threshold 0xF - attachment 22
w/o waveform - attachment 23 |
| Attachment 1: 220617_1322_stats.png
|
|
| Attachment 2: 220617_1420_stats.png
|
|
| Attachment 3: 220617_11428_stats.png
|
|
| Attachment 4: 220617_1428_layout1.png
|
|
| Attachment 5: 220617_1440_stats_waveform_on.png
|
|
| Attachment 6: 220617_1444_stats_no_waveform.png
|
|
| Attachment 7: 220617_1459_stats_waveform_on.png
|
|
| Attachment 8: 220617_1517_stats_no_waveform.png
|
|
| Attachment 9: 220617_1624_stats_waveform_on.png
|
|
| Attachment 10: 220617_1626_stats_waveform_off.png
|
|
| Attachment 11: 220617_1649_stats_waveform_on.png
|
|
| Attachment 12: 220617_1652_no_waveform.png
|
|
| Attachment 13: 220617_1709_waveform_on.png
|
|
| Attachment 14: 220617_1724_stats_no_waveform.png
|
|
| Attachment 15: 220617_1729_stats_waveform_on.png
|
|
| Attachment 16: 220617_1733_stats_no_waveform.png
|
|
| Attachment 17: 220617_1736_0x21.png
|
|
| Attachment 18: 220617_1738_0x14.png
|
|
| Attachment 19: 220617_1738_0x12.png
|
|
| Attachment 20: 220617_1807_stats_waveform_on.png
|
|
| Attachment 21: 220617_1832_stats_no_waveform.png
|
|
| Attachment 22: 220617_1844_0xf.png
|
|
| Attachment 23: 220617_1845_stats_waveform_on.png
|
|
|
|
502
|
Sun Jun 26 08:51:20 2022 |
OH, NH | Sunday 26 June 08:00-24:00 |
09:51 Taken over from Tom following the night shift
Experiment is still running smoothly
Compression of the files is complete up to the start of R5.
Have started compression of files in R5 which should run up to R5_499
Currently on R5_528
Statistics ok - attachment 1
Temperatures ok - attachment 2
Bias and leakage currents ok - attachment 3
System wide checks:
Base Current Difference
aida07 fault 0xc53d : 0xc594 : 87
aida08 fault 0xf1be : 0xf271 : 179
White Rabbit error counter test result: Passed 6, Failed 2
Base Current Difference
aida07 fault 0x2a : 0x3b : 17
FPGA Timestamp error counter test result: Passed 7, Failed 1
Current merger rate is 2E6-4E6 events per second
Current tapeserver rate is 5800 kB/s
Free HDD space is 1.1 TB
At current rates will last 54 hours which should see out the experiment
Experiment currently scheduled to finish at 6am on Tuesday morning (May get until 8am)
Analysis of R5_528 - attachment 4
Deadtime of AIDA02 currently around 15%
11:56 Statistics ok - attachment 5
Temps ok - attachment 6
Bias and leakage currents ok - attachment 7
System wide checks:
Clocks all ok
Base Current Difference
aida07 fault 0xc53d : 0xc59c : 95
aida08 fault 0xf1be : 0xf27a : 188
White Rabbit error counter test result: Passed 6, Failed 2
Base Current Difference
aida07 fault 0x2a : 0x3b : 17
FPGA Timestamp error counter test result: Passed 7, Failed 1
Analysis of file R5_458 - attachment 8
Deadtime in AIDA02 only 11.5% in this file
TapeServer rate still 5.5 MB/s
14:36 Has been no beam for the last while or so.
With no beam AIDA02 has 0.25% deadtime so the deadtime is almost entirely due to the spill on time
Stats- attachment 9
Temp - attachment 10
Bias and leakage currents ok - attachment 11
System wide checks:
Clock ok
Base Current Difference
aida07 fault 0xc53d : 0xc59d : 96
aida08 fault 0xf1be : 0xf27c : 190
White Rabbit error counter test result: Passed 6, Failed 2
Base Current Difference
aida07 fault 0x2a : 0x3c : 18
FPGA Timestamp error counter test result: Passed 7, Failed 1
13:30 Beam taken to change an ion source
14:57 Beam back
16:21 Statistics ok - attachment 12
Temperatures ok - attachment 13
Bias and leakage currents ok - attachment 14
System wide checks - ASIC clocks ok
Base Current Difference
aida07 fault 0xc53d : 0xc5a3 : 102
aida08 fault 0xf1be : 0xf285 : 199
White Rabbit error counter test result: Passed 6, Failed 2
Base Current Difference
aida07 fault 0x2a : 0x3e : 20
FPGA Timestamp error counter test result: Passed 7, Failed 1
Currently on file R5_592
Analysis of R5_591 - Attachment 15
Deadtime of AIDA02 sitting at 17%
[NH taking over for OH so he can get home]
18:53 - FRS DAQ problems mean we weren't taking DESPEC data for the past hour or so... now all back
Statitics ok - attachement 16
Temps ok - attachement 17
Bias & leakage ok - attachement 18
System wide checks -
Clocks OK
ADC Calibration N/A
WR Decoder -
Base Current Difference
aida07 fault 0xc53d : 0xc5ac : 111
aida08 fault 0xf1be : 0xf28c : 206
White Rabbit error counter test result: Passed 6, Failed 2
FPGA -
Base Current Difference
aida07 fault 0x2a : 0x3f : 21
PLL OK
Currently on file R5_621
Analysis of R5_620 - Attachement 19
aida02 deadtime only 5%, all others negligible
20:20 Stats ok - attachment 20
Temps ok - attachment 21
Bias and leakage currents ok - attachment 22
System wide checks:
Clock ok
Base Current Difference
aida07 fault 0xc53d : 0xc5ad : 112
aida08 fault 0xf1be : 0xf293 : 213
White Rabbit error counter test result: Passed 6, Failed 2
Base Current Difference
aida07 fault 0x2a : 0x3f : 21
FPGA Timestamp error counter test result: Passed 7, Failed 1
Analysis of R6_637 - attachment 23
22:13 Statistics ok - attachment 24
Temperatures ok - attachment 25
Bias and leakage currents ok - attachment 26
ASIC clock check ok
Base Current Difference
aida07 fault 0xc53d : 0xc5ba : 125
aida08 fault 0xf1be : 0xf2a2 : 228
White Rabbit error counter test result: Passed 6, Failed 2
Base Current Difference
aida07 fault 0x2a : 0x40 : 22
FPGA Timestamp error counter test result: Passed 7, Failed 1
Analysis of R5_658 - attachment 27 |
| Attachment 1: 220626_0949_stats.png
|
|
| Attachment 2: 220626_0950_Temps.png
|
|
| Attachment 3: 220626_0951_Bias.png
|
|
| Attachment 4: R5_528_analysis.txt
|
*** TDR format 3.3.0 analyser - TD - May 2021
*** ERROR: READ I/O error: 5002
blocks: 32000
ADC data format: 260639000 ( 776334.1 Hz)
Other data format: 1281002 ( 3815.6 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 1133 ( 3.4 Hz)
RESUME: 1133 ( 3.4 Hz)
SYNC100: 33251 ( 99.0 Hz)
WR48-63: 33251 ( 99.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 1212234 ( 3610.7 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 393457 ( 1171.9 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
WR48-63: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
*** Timestamp elapsed time: 335.730 s
FEE elapsed dead time(s) elapsed idle time(s)
0 8.363 0.000
1 44.893 0.000
2 0.179 0.000
3 3.779 0.000
4 0.000 44.522
5 0.034 0.000
6 0.166 0.000
7 0.017 0.000
8 0.000 0.000
9 0.000 0.000
10 0.000 0.000
11 0.000 0.000
12 0.000 0.000
13 0.000 0.000
14 0.000 0.000
15 0.000 0.000
16 0.000 0.000
17 0.000 0.000
18 0.000 0.000
19 0.000 0.000
20 0.000 0.000
21 0.000 0.000
22 0.000 0.000
23 0.000 0.000
24 0.000 0.000
25 0.000 0.000
26 0.000 0.000
27 0.000 0.000
28 0.000 0.000
29 0.000 0.000
30 0.000 0.000
31 0.000 0.000
32 0.000 0.000
*** Statistics
FEE ADC Data Other Data Sample Undefined Pause Resume SYNC100 WR48-63 Disc MBS Other HEC Data
0 31317521 31305 0 0 207 207 4035 4035 0 22821 0 98396
1 68454688 19188 0 0 693 693 8901 8901 0 0 0 59818
2 19369571 305849 0 0 35 35 2468 2468 0 300843 0 63063
3 45019317 306201 0 0 130 130 5636 5636 0 294669 0 32623
4 4769649 1252 0 0 0 0 626 626 0 0 0 31000
5 17853070 4600 0 0 15 15 2285 2285 0 0 0 37328
6 42448852 303192 0 0 44 44 5356 5356 0 292392 0 54344
7 31406332 309415 0 0 9 9 3944 3944 0 301509 0 16885
8 0 0 0 0 0 0 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0 0 0 0 0
11 0 0 0 0 0 0 0 0 0 0 0 0
12 0 0 0 0 0 0 0 0 0 0 0 0
13 0 0 0 0 0 0 0 0 0 0 0 0
14 0 0 0 0 0 0 0 0 0 0 0 0
15 0 0 0 0 0 0 0 0 0 0 0 0
16 0 0 0 0 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0 0 0 0
*** Timewarps
FEE ADC Pause Resume SYNC100 WR48-63 Disc MBS Undefined Samples
0 0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 0 0
2 0 0 0 0 0 0 0 0 0
3 0 0 0 0 0 0 0 0 0
4 0 0 0 0 0 0 0 0 0
5 0 0 0 0 0 0 0 0 0
6 0 0 0 0 0 0 0 0 0
7 0 0 0 0 0 0 0 0 0
8 0 0 0 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0 0
11 0 0 0 0 0 0 0 0 0
12 0 0 0 0 0 0 0 0 0
13 0 0 0 0 0 0 0 0 0
14 0 0 0 0 0 0 0 0 0
15 0 0 0 0 0 0 0 0 0
16 0 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0
*** Program elapsed time: 41.891s ( 763.894 blocks/s, 47.743 Mb/s)
|
| Attachment 5: 220626_1155_stats.png
|
|
| Attachment 6: 220626_1155_Temp.png
|
|
| Attachment 7: 220626_1156_Bias.png
|
|
| Attachment 8: R5_548_analysis.txt
|
*** TDR format 3.3.0 analyser - TD - May 2021
*** ERROR: READ I/O error: 5002
blocks: 32000
ADC data format: 260758427 ( 740797.4 Hz)
Other data format: 1161573 ( 3300.0 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 1084 ( 3.1 Hz)
RESUME: 1084 ( 3.1 Hz)
SYNC100: 33311 ( 94.6 Hz)
WR48-63: 33311 ( 94.6 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 1092783 ( 3104.5 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 356241 ( 1012.1 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
WR48-63: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
*** Timestamp elapsed time: 351.997 s
FEE elapsed dead time(s) elapsed idle time(s)
0 5.340 0.000
1 40.660 0.000
2 0.175 0.000
3 2.385 0.000
4 0.000 67.319
5 0.032 0.000
6 0.142 0.000
7 0.019 0.000
8 0.000 0.000
9 0.000 0.000
10 0.000 0.000
11 0.000 0.000
12 0.000 0.000
13 0.000 0.000
14 0.000 0.000
15 0.000 0.000
16 0.000 0.000
17 0.000 0.000
18 0.000 0.000
19 0.000 0.000
20 0.000 0.000
21 0.000 0.000
22 0.000 0.000
23 0.000 0.000
24 0.000 0.000
25 0.000 0.000
26 0.000 0.000
27 0.000 0.000
28 0.000 0.000
29 0.000 0.000
30 0.000 0.000
31 0.000 0.000
32 0.000 0.000
*** Statistics
FEE ADC Data Other Data Sample Undefined Pause Resume SYNC100 WR48-63 Disc MBS Other HEC Data
0 29258900 32253 0 0 166 166 3875 3875 0 24171 0 88948
1 70599060 19658 0 0 716 716 9113 9113 0 0 0 56350
2 19313501 275533 0 0 28 28 2458 2458 0 270561 0 56248
3 44887469 278330 0 0 112 112 5682 5682 0 266742 0 29387
4 4430708 1118 0 0 0 0 559 559 0 0 0 27825
5 17540711 4456 0 0 11 11 2217 2217 0 0 0 33663
6 43048879 271297 0 0 40 40 5377 5377 0 260463 0 48745
7 31679199 278928 0 0 11 11 4030 4030 0 270846 0 15075
8 0 0 0 0 0 0 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0 0 0 0 0
11 0 0 0 0 0 0 0 0 0 0 0 0
12 0 0 0 0 0 0 0 0 0 0 0 0
13 0 0 0 0 0 0 0 0 0 0 0 0
14 0 0 0 0 0 0 0 0 0 0 0 0
15 0 0 0 0 0 0 0 0 0 0 0 0
16 0 0 0 0 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0 0 0 0
*** Timewarps
FEE ADC Pause Resume SYNC100 WR48-63 Disc MBS Undefined Samples
0 0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 0 0
2 0 0 0 0 0 0 0 0 0
3 0 0 0 0 0 0 0 0 0
4 0 0 0 0 0 0 0 0 0
5 0 0 0 0 0 0 0 0 0
6 0 0 0 0 0 0 0 0 0
7 0 0 0 0 0 0 0 0 0
8 0 0 0 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0 0
11 0 0 0 0 0 0 0 0 0
12 0 0 0 0 0 0 0 0 0
13 0 0 0 0 0 0 0 0 0
14 0 0 0 0 0 0 0 0 0
15 0 0 0 0 0 0 0 0 0
16 0 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0
*** Program elapsed time: 40.309s ( 793.875 blocks/s, 49.617 Mb/s)
|
| Attachment 9: 220626_1435_stats.png
|
|
| Attachment 10: 220626_1435_Temp.png
|
|
| Attachment 11: 220626_1436_Bias.png
|
|
| Attachment 12: 220626_1620_Stats.png
|
|
| Attachment 13: 220627_1620_Temp.png
|
|
| Attachment 14: 220626_1621_Bias.png
|
|
| Attachment 15: R5_591_analysis.txt
|
*** TDR format 3.3.0 analyser - TD - May 2021
*** ERROR: READ I/O error: 5002
blocks: 32000
ADC data format: 260421862 ( 840783.5 Hz)
Other data format: 1498138 ( 4836.8 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 1449 ( 4.7 Hz)
RESUME: 1448 ( 4.7 Hz)
SYNC100: 33154 ( 107.0 Hz)
WR48-63: 33154 ( 107.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 1428933 ( 4613.4 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 452287 ( 1460.2 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
WR48-63: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
*** Timestamp elapsed time: 309.737 s
FEE elapsed dead time(s) elapsed idle time(s)
0 13.828 0.000
1 52.434 0.000
2 0.366 0.000
3 9.346 0.000
4 0.001 63.918
5 0.097 0.000
6 0.290 0.000
7 0.023 0.000
8 0.000 0.000
9 0.000 0.000
10 0.000 0.000
11 0.000 0.000
12 0.000 0.000
13 0.000 0.000
14 0.000 0.000
15 0.000 0.000
16 0.000 0.000
17 0.000 0.000
18 0.000 0.000
19 0.000 0.000
20 0.000 0.000
21 0.000 0.000
22 0.000 0.000
23 0.000 0.000
24 0.000 0.000
25 0.000 0.000
26 0.000 0.000
27 0.000 0.000
28 0.000 0.000
29 0.000 0.000
30 0.000 0.000
31 0.000 0.000
32 0.000 0.000
*** Statistics
FEE ADC Data Other Data Sample Undefined Pause Resume SYNC100 WR48-63 Disc MBS Other HEC Data
0 31248979 29393 0 0 283 282 4085 4085 0 20658 0 112900
1 66285610 18734 0 0 794 794 8573 8573 0 0 0 62107
2 22411069 363765 0 0 53 53 2890 2890 0 357879 0 73563
3 44202913 357046 0 0 221 221 5556 5556 0 345492 0 37941
4 5178577 1284 0 0 1 1 641 641 0 0 0 37680
5 19493707 4858 0 0 26 26 2403 2403 0 0 0 44391
6 39593301 355717 0 0 57 57 5081 5081 0 345441 0 63694
7 32007706 367341 0 0 14 14 3925 3925 0 359463 0 20011
8 0 0 0 0 0 0 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0 0 0 0 0
11 0 0 0 0 0 0 0 0 0 0 0 0
12 0 0 0 0 0 0 0 0 0 0 0 0
13 0 0 0 0 0 0 0 0 0 0 0 0
14 0 0 0 0 0 0 0 0 0 0 0 0
15 0 0 0 0 0 0 0 0 0 0 0 0
16 0 0 0 0 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0 0 0 0
*** Timewarps
FEE ADC Pause Resume SYNC100 WR48-63 Disc MBS Undefined Samples
0 0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 0 0
2 0 0 0 0 0 0 0 0 0
3 0 0 0 0 0 0 0 0 0
4 0 0 0 0 0 0 0 0 0
5 0 0 0 0 0 0 0 0 0
6 0 0 0 0 0 0 0 0 0
7 0 0 0 0 0 0 0 0 0
8 0 0 0 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0 0
11 0 0 0 0 0 0 0 0 0
12 0 0 0 0 0 0 0 0 0
13 0 0 0 0 0 0 0 0 0
14 0 0 0 0 0 0 0 0 0
15 0 0 0 0 0 0 0 0 0
16 0 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0
*** Program elapsed time: 41.789s ( 765.751 blocks/s, 47.859 Mb/s)
|
| Attachment 16: tmp_aida_stats.png
|
|
| Attachment 17: tmp_aida_temps.png
|
|
| Attachment 18: tmp_aida_hv.png
|
|
| Attachment 19: R5_620
|
*** TDR format 3.3.0 analyser - TD - May 2021
*** ERROR: READ I/O error: 5002
blocks: 32000
ADC data format: 261173934 ( 672570.7 Hz)
Other data format: 746069 ( 1921.3 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 773 ( 2.0 Hz)
RESUME: 773 ( 2.0 Hz)
SYNC100: 33446 ( 86.1 Hz)
WR48-63: 33446 ( 86.1 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 677631 ( 1745.0 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 225736 ( 581.3 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
WR48-63: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
*** Timestamp elapsed time: 388.322 s
FEE elapsed dead time(s) elapsed idle time(s)
0 1.039 0.000
1 21.090 0.000
2 0.051 0.000
3 0.148 0.000
4 0.000 87.597
5 0.001 0.000
6 0.083 0.000
7 0.003 0.000
8 0.000 0.000
9 0.000 0.000
10 0.000 0.000
11 0.000 0.000
12 0.000 0.000
13 0.000 0.000
14 0.000 0.000
15 0.000 0.000
16 0.000 0.000
17 0.000 0.000
18 0.000 0.000
19 0.000 0.000
20 0.000 0.000
21 0.000 0.000
22 0.000 0.000
23 0.000 0.000
24 0.000 0.000
25 0.000 0.000
26 0.000 0.000
27 0.000 0.000
28 0.000 0.000
29 0.000 0.000
30 0.000 0.000
31 0.000 0.000
32 0.000 0.000
*** Statistics
FEE ADC Data Other Data Sample Undefined Pause Resume SYNC100 WR48-63 Disc MBS Other HEC Data
0 25216948 33824 0 0 63 63 3334 3334 0 27030 0 55557
1 75946108 20872 0 0 626 626 9810 9810 0 0 0 40146
2 18060200 168976 0 0 23 23 2304 2304 0 164322 0 34386
3 41452092 174351 0 0 26 26 5287 5287 0 163725 0 18377
4 3680704 948 0 0 0 0 474 474 0 0 0 17387
5 16518678 4150 0 0 1 1 2074 2074 0 0 0 20823
6 44456577 169523 0 0 30 30 5686 5686 0 158091 0 29842
7 35842627 173425 0 0 4 4 4477 4477 0 164463 0 9218
8 0 0 0 0 0 0 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0 0 0 0 0
11 0 0 0 0 0 0 0 0 0 0 0 0
12 0 0 0 0 0 0 0 0 0 0 0 0
13 0 0 0 0 0 0 0 0 0 0 0 0
14 0 0 0 0 0 0 0 0 0 0 0 0
15 0 0 0 0 0 0 0 0 0 0 0 0
16 0 0 0 0 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0 0 0 0
*** Timewarps
FEE ADC Pause Resume SYNC100 WR48-63 Disc MBS Undefined Samples
0 0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 0 0
2 0 0 0 0 0 0 0 0 0
3 0 0 0 0 0 0 0 0 0
4 0 0 0 0 0 0 0 0 0
5 0 0 0 0 0 0 0 0 0
6 0 0 0 0 0 0 0 0 0
7 0 0 0 0 0 0 0 0 0
8 0 0 0 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0 0
11 0 0 0 0 0 0 0 0 0
12 0 0 0 0 0 0 0 0 0
13 0 0 0 0 0 0 0 0 0
14 0 0 0 0 0 0 0 0 0
15 0 0 0 0 0 0 0 0 0
16 0 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0
*** Program elapsed time: 41.547s ( 770.214 blocks/s, 48.138 Mb/s)
|
| Attachment 20: 220624_2021_Stats.png
|
|
| Attachment 21: 220626_2021_Temp.png
|
|
| Attachment 22: 220626_2022_Bias.png
|
|
| Attachment 23: R5_637_analysis.txt
|
*** TDR format 3.3.0 analyser - TD - May 2021
*** ERROR: READ I/O error: 5002
blocks: 32000
ADC data format: 260438015 ( 849636.0 Hz)
Other data format: 1481985 ( 4834.7 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 1460 ( 4.8 Hz)
RESUME: 1460 ( 4.8 Hz)
SYNC100: 33142 ( 108.1 Hz)
WR48-63: 33142 ( 108.1 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 1412781 ( 4609.0 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 447675 ( 1460.5 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
WR48-63: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
*** Timestamp elapsed time: 306.529 s
FEE elapsed dead time(s) elapsed idle time(s)
0 15.951 0.000
1 53.516 0.000
2 0.642 0.000
3 7.248 0.000
4 0.000 53.241
5 0.100 0.000
6 0.344 0.000
7 0.040 0.000
8 0.000 0.000
9 0.000 0.000
10 0.000 0.000
11 0.000 0.000
12 0.000 0.000
13 0.000 0.000
14 0.000 0.000
15 0.000 0.000
16 0.000 0.000
17 0.000 0.000
18 0.000 0.000
19 0.000 0.000
20 0.000 0.000
21 0.000 0.000
22 0.000 0.000
23 0.000 0.000
24 0.000 0.000
25 0.000 0.000
26 0.000 0.000
27 0.000 0.000
28 0.000 0.000
29 0.000 0.000
30 0.000 0.000
31 0.000 0.000
32 0.000 0.000
*** Statistics
FEE ADC Data Other Data Sample Undefined Pause Resume SYNC100 WR48-63 Disc MBS Other HEC Data
0 31824689 29138 0 0 291 291 4129 4129 0 20298 0 110511
1 66116092 18806 0 0 788 788 8615 8615 0 0 0 62706
2 22609638 359670 0 0 68 68 2899 2899 0 353736 0 72536
3 44231727 352857 0 0 199 199 5696 5696 0 341067 0 37873
4 5159378 1270 0 0 0 0 635 635 0 0 0 37425
5 19491876 4836 0 0 27 27 2391 2391 0 0 0 43832
6 39561529 352144 0 0 68 68 4899 4899 0 342210 0 62927
7 31443086 363264 0 0 19 19 3878 3878 0 355470 0 19865
8 0 0 0 0 0 0 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0 0 0 0 0
11 0 0 0 0 0 0 0 0 0 0 0 0
12 0 0 0 0 0 0 0 0 0 0 0 0
13 0 0 0 0 0 0 0 0 0 0 0 0
14 0 0 0 0 0 0 0 0 0 0 0 0
15 0 0 0 0 0 0 0 0 0 0 0 0
16 0 0 0 0 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0 0 0 0
*** Timewarps
FEE ADC Pause Resume SYNC100 WR48-63 Disc MBS Undefined Samples
0 0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 0 0
2 0 0 0 0 0 0 0 0 0
3 0 0 0 0 0 0 0 0 0
4 0 0 0 0 0 0 0 0 0
5 0 0 0 0 0 0 0 0 0
6 0 0 0 0 0 0 0 0 0
7 0 0 0 0 0 0 0 0 0
8 0 0 0 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0 0
11 0 0 0 0 0 0 0 0 0
12 0 0 0 0 0 0 0 0 0
13 0 0 0 0 0 0 0 0 0
14 0 0 0 0 0 0 0 0 0
15 0 0 0 0 0 0 0 0 0
16 0 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0
*** Program elapsed time: 42.891s ( 746.084 blocks/s, 46.630 Mb/s)
|
| Attachment 24: 220626_2211_Stats.png
|
|
| Attachment 25: 220626_2211_Temps.png
|
|
| Attachment 26: 220626_2212_Bias.png
|
|
| Attachment 27: R5_658_analysis.txt
|
*** TDR format 3.3.0 analyser - TD - May 2021
*** ERROR: READ I/O error: 5002
blocks: 32000
ADC data format: 260445182 ( 832450.2 Hz)
Other data format: 1474818 ( 4713.9 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 1484 ( 4.7 Hz)
RESUME: 1484 ( 4.7 Hz)
SYNC100: 33166 ( 106.0 Hz)
WR48-63: 33166 ( 106.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 1405518 ( 4492.4 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 448659 ( 1434.0 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
WR48-63: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
*** Timestamp elapsed time: 312.866 s
FEE elapsed dead time(s) elapsed idle time(s)
0 15.080 0.000
1 57.798 0.000
2 0.343 0.000
3 7.278 0.000
4 0.000 44.445
5 0.093 0.000
6 0.265 0.000
7 0.017 0.000
8 0.000 0.000
9 0.000 0.000
10 0.000 0.000
11 0.000 0.000
12 0.000 0.000
13 0.000 0.000
14 0.000 0.000
15 0.000 0.000
16 0.000 0.000
17 0.000 0.000
18 0.000 0.000
19 0.000 0.000
20 0.000 0.000
21 0.000 0.000
22 0.000 0.000
23 0.000 0.000
24 0.000 0.000
25 0.000 0.000
26 0.000 0.000
27 0.000 0.000
28 0.000 0.000
29 0.000 0.000
30 0.000 0.000
31 0.000 0.000
32 0.000 0.000
*** Statistics
FEE ADC Data Other Data Sample Undefined Pause Resume SYNC100 WR48-63 Disc MBS Other HEC Data
0 32528426 29869 0 0 310 310 4234 4234 0 20781 0 110841
1 66604779 18608 0 0 825 825 8479 8479 0 0 0 61991
2 23919816 357449 0 0 63 63 3136 3136 0 351051 0 72944
3 47288432 353667 0 0 196 196 5939 5939 0 341397 0 38095
4 5032393 1218 0 0 0 0 609 609 0 0 0 37603
5 19089927 4824 0 0 28 28 2384 2384 0 0 0 44213
6 39392581 350027 0 0 52 52 5076 5076 0 339771 0 63206
7 26588828 359156 0 0 10 10 3309 3309 0 352518 0 19766
8 0 0 0 0 0 0 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0 0 0 0 0
11 0 0 0 0 0 0 0 0 0 0 0 0
12 0 0 0 0 0 0 0 0 0 0 0 0
13 0 0 0 0 0 0 0 0 0 0 0 0
14 0 0 0 0 0 0 0 0 0 0 0 0
15 0 0 0 0 0 0 0 0 0 0 0 0
16 0 0 0 0 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0 0 0 0
*** Timewarps
FEE ADC Pause Resume SYNC100 WR48-63 Disc MBS Undefined Samples
0 0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 0 0
2 0 0 0 0 0 0 0 0 0
3 0 0 0 0 0 0 0 0 0
4 0 0 0 0 0 0 0 0 0
5 0 0 0 0 0 0 0 0 0
6 0 0 0 0 0 0 0 0 0
7 0 0 0 0 0 0 0 0 0
8 0 0 0 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0 0
11 0 0 0 0 0 0 0 0 0
12 0 0 0 0 0 0 0 0 0
13 0 0 0 0 0 0 0 0 0
14 0 0 0 0 0 0 0 0 0
15 0 0 0 0 0 0 0 0 0
16 0 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0
*** Program elapsed time: 42.102s ( 760.067 blocks/s, 47.504 Mb/s)
|
|
|
508
|
Tue Jun 28 10:11:35 2022 |
OH, NH | MIDAS Data Aq V10 |
11:11 Rebooted FEEs and changed aidacommon in /MIDAS/linux-ppc_4xx/startup to point to the new V10 DataAq that Patrick produced
When using V9 the Merger statistics reported WR items at twice the rate of ADC data items.
i.e for ever data item we were sending and info code 4 and info code 5 item sending 192 bits of data vs 64 for just the data word
This was causing significant deadtime when FEEs were running in the range of around 200kHz. These WR items were not reported by the MIDAS Acquisition server but were in the Merger statistics
Patrick has produced V10 which removes these.
When running V10 we can confirm in the Merger statistics that this rate is no longer determined by the ADC data rate and instead controlled via Sync Rollover Target in GSI WhiteRabbit Control.
WR items for 0xE - attachment 1
WR items for 0x7 - attachment 2
However we see in the NewMerger terminal the message shown in attachment 3 frequently.
Also we note that the merger time error counter is also going up.
Our thoughts for this are we have a rollover issue (Is the merger expecting the rollover of the LSB to be one value when the MSB is updated but MIDAS is happening on another?)
Are we having dead time issues which is causing time warps?
Does each buffer from the MIDAS Data Acq start with a full WR timestamp?
aidacommon has been changed back to point to V9 to not cause issues when we run the DAQ and forget we changed it to be this way? |
| Attachment 1: 220628_1117_Rollover0xE.png
|
|
| Attachment 2: 220628_1119_Merger_Stats_0x7.png
|
|
| Attachment 3: NewMerger_Dump.txt
|
MERGE Data Link (17671): block sequence error: ID 2, expected 462; received 0
MERGE Data Link (17669): block sequence error: ID 0, expected 461; received 0
MERGE Data Link (17672): block sequence error: ID 3, expected 462; received 0
MERGE Data Link (17670): block sequence error: ID 1, expected 462; received 0
MERGE Data Link (17674): block sequence error: ID 5, expected 463; received 0
MERGE Data Link (17676): block sequence error: ID 7, expected 462; received 0
MERGE Data Link (17675): block sequence error: ID 6, expected 461; received 0
MERGE Data Link (17673): block sequence error: ID 4, expected 462; received 0
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
MERGE Actor (17699): Working with 0 from 8 data sources
|
|
|
229
|
Fri Apr 16 07:19:00 2021 |
OH, MA | Friday 16th April 08:00-16:00 |
08:19 System wide checks all ok
Baselines reset on WR and FPGA checks for the start of the day
Statistics ok - attachment 1
Temperatures ok - attachment 2
Bias and leakage currents ok - attachment 3
09:03 During the reset last night I repaired the OptionsDB for aida02 from the Options.Pristine directory.
No corruptions have been noted since
10:10 System check, clcock and ADC ok
Base Current Difference
aida07 fault 0xfb3a : 0xfb3d : 3
White Rabbit error counter test result: Passed 11, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
FPGA and memory check ok
Statistics ok - attachment 4
Temperatures ok - attachment 5
Bias and leakage currents ok - attachment 6
10:39 Fission fragments were punching through AIDA
This was even with the maxmimum degraders in place
They are now placing a thick plate in front of AIDA to block the fragments while they beam tune.
11:38 They have found the issue with the degrader and have corrected it.
We now see next to no implants in AIDA as expect.
Histograms reset at around 11:35
Layout 2 shows very few implants in 5 minutes - attachment 7
12:08 The high implantation rates of ions into AIDA is clearly seen in the large transients on the leakage currents - attachment 8
This was a large amount of dose going into the detectors which we should try to avoid
13:25 system check, clock and ADC ok
Base Current Difference
aida05 fault 0xc879 : 0xc87b : 2
aida06 fault 0x323c : 0x323e : 2
aida07 fault 0xfb3a : 0xfb3f : 5
aida08 fault 0xd3d6 : 0xd3d8 : 2
White Rabbit error counter test result: Passed 8, Failed 4
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Statistics ok - attachment 9
Temperatures ok - attachment 10
Bias and leakage currents ok - attachment 11
FPGA and memory check ok
16:10 system check, clock and ADC ok
Base Current Difference
aida05 fault 0xc879 : 0xc87b : 2
aida06 fault 0x323c : 0x323e : 2
aida07 fault 0xfb3a : 0xfb3f : 5
aida08 fault 0xd3d6 : 0xd3d8 : 2
White Rabbit error counter test result: Passed 8, Failed 4
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Statistics ok - attachment 12
Temperatures ok - attachment 13
Bias and leakage currents ok - attachment 14
FPGA and memory check ok |
| Attachment 1: 210416_0820_Stats.png
|
|
| Attachment 2: 210416_0821_Temp.png
|
|
| Attachment 3: 210416_0824_Bias.png
|
|
| Attachment 4: 210416_1003Stats.png
|
|
| Attachment 5: 210416_1007Temp.png
|
|
| Attachment 6: 210416_1019_voltages.png
|
|
| Attachment 7: 210416_1139_layout2.png
|
|
| Attachment 8: 210416_1207_Leakage.png
|
|
| Attachment 9: 210416_1311_Stats.png
|
|
| Attachment 10: 210416_1312_Temp.png
|
|
| Attachment 11: 210416_1315_Voltages.png
|
|
| Attachment 12: 210416_1412_Voltages.png
|
|
| Attachment 13: 210416_1548_Stats.png
|
|
| Attachment 14: 210416_1553_Temp.png
|
|
|
|
175
|
Sat Mar 6 06:52:06 2021 |
OH, LS | Saturday 6th March |
07:50 Pulser settings - attachment 1
07:52 System wide checks all ok
Statistics ok - attachment 2
Bias ok - attachment 3
09:30 The intensity of fragments is being increased
They are trying to do a factor of 10 increase first which should take us to around 100Hz
10:00 System wide checks all ok
Statistics - attachment 4
Layout 1 showing implants - attachment 5
Fee temperatures ok - attachment 6
12.00 System wide checks all okay, no failures
Statistics (screenshot7)
Spectrum rate (screenshot8)
FEE temps were normal except AIDA04 had "no response" (screenshot9), reloaded several minutes later seems to read normal
(screenshot10)
Leakage currents okay, increase from previous values written to sheets (screenshot12)
Rechecked FEETemps this time AIDA03 had "no response" (screenshot11), seems this occurs when the reload takes longer
than
usual, another reload shows all FEE temps as normal.
Merger and tape server okay
14.00 System wide checks all okay, no failures
Statistics (screenshot13)
Spectrum rate (screenshot14)
FEE temps normal, no issue with reload this time (screenshot15)
Leakage currents okay still increasing, written to sheets (screenshot16)
Merger okay ~4.5M items/sec
Tape server ~4.5Mb/sec
14.50 Beam has been shifted to centre beam more onto AIDA (corresponds to run S452f011)
16.00 System wide checks all okay, no failures
Statistics (screenshot17)
Spectrum rate (screenshot18)
FEE temps normal except AIDA08 "no response" (screenshot19), reload returns to normal (screenshot20)
Leakage currents added to sheet (screenshot21)
Merger okay ~4.5M items/sec
Tape server ~4.5Mb/sec
17.17 no beam, preparing for the 190Ta setting
17:44 OH Takes over
System wide checks all ok
Statistics - attachment 22
Temp ok - Attachment 23
Bias and leakage currents ok - Attatchment 24
While MBS was not currently writing data the merger -> tapeserver link was briefly toggle
The Directory for the experiment was then setup S452
The merger -> Tapeserver link was then re-enabled and data forwarding to MBS has continued.
ASIC Check all ok
18:10 Started writing data to file
File directory S452
Run number R1_0
Seems to have skipped a few files First full file R1_10
Currently writing to /media/SecondDrive/TapeData
Current free space 4211084180 kB
Current write speed 46163 kB/s
Time remaining until full 91222 seconds
25 hours remaining space
18:34 DESPEC starting new MBS file
AIDA Currently on file R1_41
19:00 DESPEC Closing file
AIDA Currently on R1_75
19:05 Comparing last years statistics to this years
FEE Last year This year Factor increase
1 60960 183111 3.00
2 183494 201256 1.09
3 91506 196647 2.15
4 73914 208290 2.81
5 31979 63571 1.99
6 54084 144638 2.67
7 35621 137605 3.86
8 38626 95858 2.48
9 264436 225790 0.85
10 209709 198896 0.94
11 88443 133899 1.51
12 183493 204996 1.12
Sum 1316265 1994557 1.51
19:22 DESPEC opens a new file (They forgot to mention they were closing the previous file)
AIDA on R1_104
Looking int to high write rate to see if we can do anything to reduce it
It isn't the correlation scaler as that is coming in at a rate of 16kHz which amounts to around 1.5MB/s - attachment 24
19:46 Much of the rate is coming from a small number of channels - attachment 25
19:17 WR Timestamp error.
Mentioned to Nic and he said that sometimes small glitches like this are observed
Base Current Difference
aida01 fault 0x9e8b : 0x9e8d : 2
aida02 fault 0x9a8f : 0x9a91 : 2
aida03 fault 0x29e9 : 0x29eb : 2
aida04 fault 0x7f66 : 0x7f68 : 2
aida05 fault 0x1f4f : 0x1f51 : 2
aida06 fault 0x8faa : 0x8fac : 2
aida07 fault 0x53f0 : 0x53f2 : 2
aida08 fault 0x7066 : 0x7068 : 2
White Rabbit error counter test result: Passed 4, Failed 8
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
All other checks passed
20:52 DESPEC Stopping file
AIDA on R1_221
20:53 DESPEC Stopped file because UCESB crashed
AIDA on file R1_237
21:20 All system wide checks ok
Statistics DISC info- Attachment 27
FEE Temp ok - Attachment 28
Bias and leakage current ok - Attachment 29
Statistics good events - Attachment 30
20:51 Started compressing data with command nice -n 10 gzip -v R1_*
Have discussed with Helena and Nic about the potential issues we could run into with the high data rate
I am not sure if MBS can keep up with this rate
We are still writing to file though.
23:00 Correlations have been lost with the FRS DAQ and the rest of the DESPEC analysis
23:10 UCESB restarted and correlations have been regained. Verified with the implant-FRS time difference peak at 13us |
| Attachment 1: Pulser.png
|
|
| Attachment 2: 210306_0752_Stats.png
|
|
| Attachment 3: 210306_0754_Bias.png
|
|
| Attachment 4: 210306_0958_Stats.png
|
|
| Attachment 5: 210306_0959_Layout1.png
|
|
| Attachment 6: 210306_1000_Temp.png
|
|
| Attachment 7: 1102_Statistics.PNG
|
|
| Attachment 8: 1102_SpectrumRate.PNG
|
|
| Attachment 9: 1102_FEETemps.PNG
|
|
| Attachment 10: 1102_FEETemps2.PNG
|
|
| Attachment 11: 1102_FEETemps3.PNG
|
|
| Attachment 12: 1102_LeakageCurrents.PNG
|
|
| Attachment 13: 1300_Statistics.PNG
|
|
| Attachment 14: 1300_SpectrumRate.PNG
|
|
| Attachment 15: 1300_FEETemps.PNG
|
|
| Attachment 16: 1300_LeakageCurrents.PNG
|
|
| Attachment 17: 1500_Statistics.PNG
|
|
| Attachment 18: 1500_SpectrumRate.PNG
|
|
| Attachment 19: 1500_FEETemps.PNG
|
|
| Attachment 20: 1500_FEETemps2.PNG
|
|
| Attachment 21: 1500_LeakageCurrents.PNG
|
|
| Attachment 22: 210306_1726_Stats.png
|
|
| Attachment 23: 210306_1747_Temp.png
|
|
| Attachment 24: 210327_1748_Bias.png
|
|
| Attachment 25: 210326_1920_MergerStats.png
|
|
| Attachment 26: 210306_1946_Layout1.png
|
|
| Attachment 27: 210306_2120_Stats.png
|
|
| Attachment 28: 210306_2121_Temp.png
|
|
| Attachment 29: 210306_2123_Bias.png
|
|
| Attachment 30: 210326_1920_MergerStats.png
|
|
|
|
187
|
Wed Mar 10 07:16:02 2021 |
OH, LS | Wednesday 10th March 08:00-24:00 |
08:16 System wide checks all ok
Statistics - attachment 1
Temperatures - attachment 2
bias - attachment 3
Merger running 4.5e6 events per second
Tape data writing at 42MB per second
Currently no beam
09:45 While there is no beam we will perform a longer test of the new merger.
Run R20 stopped and merger changed to neew version
Started R21.
The now usual behaviour of additional files at start of merge observed.
N.B. That there was no toggling of the merger no storage or tapeserver no storage. Both were running before the DAQ was
set to going
-rw-rw-r--. 1 npg npg 64K Mar 10 09:50 R21_0
-rw-rw-r--. 1 npg npg 64K Mar 10 09:50 R21_1
-rw-rw-r--. 1 npg npg 64K Mar 10 09:50 R21_2
-rw-rw-r--. 1 npg npg 64K Mar 10 09:50 R21_3
-rw-rw-r--. 1 npg npg 64K Mar 10 09:50 R21_4
-rw-rw-r--. 1 npg npg 64K Mar 10 09:50 R21_5
-rw-rw-r--. 1 npg npg 64K Mar 10 09:50 R21_6
-rw-rw-r--. 1 npg npg 64K Mar 10 09:50 R21_7
-rw-rw-r--. 1 npg npg 64K Mar 10 09:50 R21_8
-rw-rw-r--. 1 npg npg 64K Mar 10 09:50 R21_9
-rw-rw-r--. 1 npg npg 64K Mar 10 09:50 R21_10
-rw-rw-r--. 1 npg npg 64K Mar 10 09:50 R21_11
-rw-rw-r--. 1 npg npg 64K Mar 10 09:50 R21_12
-rw-rw-r--. 1 npg npg 64K Mar 10 09:50 R21_13
-rw-rw-r--. 1 npg npg 64K Mar 10 09:50 R21_14
-rw-rw-r--. 1 npg npg 64K Mar 10 09:50 R21_15
-rw-rw-r--. 1 npg npg 64K Mar 10 09:50 R21_16
-rw-rw-r--. 1 npg npg 407M Mar 10 09:50 R21_17
-rw-rw-r--. 1 npg npg 1.6G Mar 10 09:52 R21_18
TapeData rate of 13280 kB/sec
No errors observed in ucesb
Correlations seen between time machine in AIDA and Ge in online
Pulser rate in the online makes sense
With the current data rate remaining HDD space on current drive will last around 89 hours.
09.20 analysis of file R21_27
ignore rates/elapsed idle time - timestamp incomplete until first info code 4 & 5 data
10:24 Message while performing system wide checks:
Get returned with an error
error: SOAP http transport timed out after 10000 ms
NONE
error: SOAP http transport timed out after 10000 ms
while executing
"$transport $procVarName $url $req"
(procedure "::SOAP::invoke" line 18)
invoked from within
"::SOAP::invoke ::SOAP::_XAIDAAccessClient__Get 10"
("eval" body line 1)
invoked from within
"eval ::SOAP::invoke ::SOAP::_XAIDAAccessClient__Get $args"
(procedure "XAIDAAccessClient__Get" line 1)
invoked from within
"XAIDAAccessClient__Get $Addr"
aida02 restarted itself during the reset process
Looking at the log messages on aida02 cannot see any reason for the cause of the restart.
System wide checks following the restart all ok.
When restarting the MBS relay errors observed until a timestamp was observed:
Warning: MBSTimeF = 0; 0x0000000000000000 0x00000000 0x00000000 0x00000000
10:55 Statistics - attachment 5
Temperature - attachment 6
Bias - attachment 7
11:03 Time machine correlation spectra with new merger
AIDA - FATIMA - Attachment 8
AIDA - Ge - Attachment 9
11:30 An implant rate observed in DSSD2. With no beam. A check of the ASIC control restored the rate to 0. Did not check the
layout to determine which FEE/ASIC caused the events before checking ASIC control
12:12 System wide checks all ok *except adc calibration which is same as before
Statistics - attachment 10
Temperature - attachment 11
Bias and leakage currents ok - attachment 12
A note on the statistics. aida11 has doubled in rate today and aida07 has gone down somewhat.
After talking with the DESPEC locals. Helena and Juergen entere at S4 at 9:40 German time.
They stood on the platform but stayed away from the snout, they added two channels to the scope and adjusted the ribbon
cable from the VME scalers.
Looking at the leakage currents on Grafana at 9:50 German time a fluctuation can be observed in the leakage currents of
DSSD3.
12:50 We have waveforms for some FEES
Layout 7 attachment -14
Layout 8 - attachment 15
14.00 (LS)
Still no beam
System wide checks okay except same as before:
**FEE64 module aida07 failed
FEE64 module aida10 failed
Calibration test result: Passed 10, Failed 2
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun
calibration for that module**
Statistics (attachment16)
Checked rate in aida07 and aida11 following Oscars previous comment, last four statistics attachments(1,5,10,16):
aida07 - 124772, 81773, 95526, 101493 - seems to be increasing back up
aida11 - 72901, 95942, 176139, 93799 - more than doubled but large drop in rate back below 100k will keep an eye on
FEE Temps (attachment17)
Leakage currents written to sheets(attachment18)
Merger ~ 45M items/s
TapeServer~ 14MB/s
14.10 While performing checks informed that some beam is back, and they have started a run file (no. S452f113), AIDA on file
R22_98, rate spectra attached (attachment19)
14.10 During the meeting errors appeared in ucesb, checked these timestamps with the corresponding AIDA files and saw
no timewarps so we are not losing anything.
Restarted MBS relay, errors have not reappeared so far
16.10 System wide checks all okay except aida07 and aida10 fail calibration (same as previous checks)
Statistics (attachment20)
Rate spectra (attachment21)
FEE Temps (attachment22)
Leakage currents written to sheets(attachment23)
Merger ~ 4.3M items/s
TapeServer~ 14MB/s
Current file R22_148
16.26 Increase rate in AIDA as target slits have been opened wider (attachment24), corresponding to runs starting from
S452f115
Around R22_150
18.00 Analysis of R22_122 (before slit adjustments think +-3mm) and R22_178 (after slit adjustment to +-5mm)
Rates of R22_122 (attachment25):
First DSSSD (fee0-fee3) ~66/s
Second DSSSD (fee4-fee7) ~49/s
Third DSSSD (fee8-fee11) ~22/s
Rates of R22_178 (attachment26):
First DSSSD (fee0-fee3) ~122/s
Second DSSSD (fee4-fee7) ~94/s
Third DSSSD (fee8-fee11) ~49/s
18.10 System wide checks all okay except aida07 and aida10 fail calibration (same as previous checks)
Statistics (attachment27)
Rate spectra (attachment28)
FEE Temps (attachment29)
Leakage currents written to sheets(attachment30), look to be on the way down
Merger ~ 4.5M items/s
TapeServer~ 15MB/s
18.24 Several timestamp errors again which also happened earlier (14.10) restarted MBS relay like earlier
19:34 MBS DAQ Crashed
20:00 DAQ is still down. They have Sultan working on it.
20:21 DAQ is back but there are issues with land04 (The raid array data is written to. It was taken out during the lustre
reboot)
We are borrowing a HDD from the SHIP group
20:30 System wide checks all ok
Statistics - attachment 31
Temperature - attachment 32
Bias - attachment 33
22:10 MBS DAQ has crashed
Merger terminal is now showing a large number of bad merge events
It's taken a while but we have managed to get the DAQ back. We had to reset the AIDA MBS.
22:42 System wide checks all ok
Statistics - attachment 34
Temperature - Attachment 35
Bias and leakage currents ok - attachment 36
23:33 System wide checks all ok - attachment 37
Temperatures - attachment 38
Bias and leakage currents ok - attachment 39 |
| Attachment 1: 210310_0813_Stats.png
|
|
| Attachment 2: 210310_0813_Temp.png
|
|
| Attachment 3: 210310_0714_Bias.png
|
|
| Attachment 4: R21_27
|
*** TDR format 3.3.0 analyser - TD - January 2019
*** ERROR: READ I/O error: 5002
blocks: 32000
ADC data format: 259319480 ( 1687258.2 Hz)
Other data format: 2600520 ( 16920.2 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 1272 ( 8.3 Hz)
RESUME: 1272 ( 8.3 Hz)
SYNC100: 573 ( 3.7 Hz)
WR48-63: 573 ( 3.7 Hz)
FEE64 disc: 302889 ( 1970.7 Hz)
MBS info: 2293941 ( 14925.5 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 0 ( 0.0 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
WR48-63: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
*** Timestamp elapsed time: 153.693 s
FEE elapsed dead time(s) elapsed idle time(s)
1 0.000 150.055
2 0.129 153.277
3 1.695 152.203
4 0.000 97.174
5 15.630 152.471
6 0.533 122.407
7 0.048 144.955
8 17.221 151.934
9 26.571 143.076
10 0.063 148.982
11 11.592 141.197
12 0.000 0.000
13 0.000 0.000
14 0.000 0.000
15 0.000 0.000
16 0.000 0.000
17 0.000 0.000
18 0.000 0.000
19 0.000 0.000
20 0.000 0.000
21 0.000 0.000
22 0.000 0.000
23 0.000 0.000
24 0.000 0.000
25 0.000 0.000
26 0.000 0.000
27 0.000 0.000
28 0.000 0.000
29 0.000 0.000
30 0.000 0.000
31 0.000 0.000
32 0.000 0.000
*** Statistics
FEE ADC Data Other Data Sample Undefined Pause Resume SYNC100 WR48-63 Disc MBS Other HEC Data
0 20482535 1736 0 0 381 381 52 52 0 870 0 0
1 26566394 1030 0 0 0 0 56 56 0 918 0 0
2 26980761 1455064 0 0 2 2 87 87 302889 1151997 0 0
3 26884821 1140284 0 0 5 5 65 65 0 1140144 0 0
4 7516935 23 0 0 0 0 7 7 0 9 0 0
5 16098094 1219 0 0 577 577 31 31 0 3 0 0
6 9441595 40 0 0 2 2 18 18 0 0 0 0
7 8769482 38 0 0 2 2 17 17 0 0 0 0
8 32618578 336 0 0 98 98 70 70 0 0 0 0
9 32790396 426 0 0 136 136 77 77 0 0 0 0
10 16087963 82 0 0 1 1 40 40 0 0 0 0
11 35081926 242 0 0 68 68 53 53 0 0 0 0
12 0 0 0 0 0 0 0 0 0 0 0 0
13 0 0 0 0 0 0 0 0 0 0 0 0
14 0 0 0 0 0 0 0 0 0 0 0 0
15 0 0 0 0 0 0 0 0 0 0 0 0
16 0 0 0 0 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0 0 0 0
*** Timewarps
FEE ADC Pause Resume SYNC100 WR48-63 Disc MBS Undefined Samples
0 0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 0 0
2 0 0 0 0 0 0 0 0 0
3 0 0 0 0 0 0 0 0 0
4 0 0 0 0 0 0 0 0 0
5 0 0 0 0 0 0 0 0 0
6 0 0 0 0 0 0 0 0 0
7 0 0 0 0 0 0 0 0 0
8 0 0 0 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0 0
11 0 0 0 0 0 0 0 0 0
12 0 0 0 0 0 0 0 0 0
13 0 0 0 0 0 0 0 0 0
14 0 0 0 0 0 0 0 0 0
15 0 0 0 0 0 0 0 0 0
16 0 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0
*** Program elapsed time:37059.496s ( 0.863 blocks/s, 0.054 Mb/s)
|
| Attachment 5: 210310_1054_Stats.png
|
|
| Attachment 6: 210310_1055_Temp.png
|
|
| Attachment 7: 210310_1055_Bias.png
|
|
| Attachment 8: AIDA_Fatima_Correlation.PNG
|
|
| Attachment 9: AIDA_Ge_Correlation.PNG
|
|
| Attachment 10: 210310_1210_Stats.png
|
|
| Attachment 11: 210310_1211_Temp.png
|
|
| Attachment 12: 210310_1212_Bias.png
|
|
| Attachment 13: AIDA_leakage.PNG
|
|
| Attachment 14: 210310_1348_Layout7.png
|
|
| Attachment 15: 210310_1252_Layout8.png
|
|
| Attachment 16: 1400_Statistics.PNG
|
|
| Attachment 17: 1400_FEETemps.PNG
|
|
| Attachment 18: 1400_LeakageCurrents.PNG
|
|
| Attachment 19: 1400_SpectraRate.PNG
|
|
| Attachment 20: 1600_Statistics.PNG
|
|
| Attachment 21: 1600_SpectraRate.PNG
|
|
| Attachment 22: 1600_FEETemps.PNG
|
|
| Attachment 23: 1600_LeakageCurrents.PNG
|
|
| Attachment 24: 1640_SpectraRate.PNG
|
|
| Attachment 25: 1700_R22_122_Rates.PNG
|
|
| Attachment 26: 1700_R22_178_Rates.PNG
|
|
| Attachment 27: 1800_Statistics.PNG
|
|
| Attachment 28: 1800_SpectraRate.PNG
|
|
| Attachment 29: 1800_FEETemps.PNG
|
|
| Attachment 30: 1800_LeakageCurrents.PNG
|
|
| Attachment 31: 210310_2039_Stats.png
|
|
| Attachment 32: 210310_2040_Temp.png
|
|
| Attachment 33: 210310_2041_Bias.png
|
|
| Attachment 34: 210310_2241_Stats.png
|
|
| Attachment 35: 210310_2241_Temp.png
|
|
| Attachment 36: 210310_2242_Bias.png
|
|
| Attachment 37: 210310_2331_Stats.png
|
|
| Attachment 38: 210310_2332_Temp.png
|
|
| Attachment 39: 210310_2333_Bias.png
|
|