| |
ID |
Date |
Author |
Subject |
|
|
416
|
Tue Nov 1 23:55:33 2016 |
TD, CG | Wednesday 2 November |
08.56 DAQ continues OK
DSSSD bias & leakage current OK - see attachments 1 & 2
FEE64 temperatures OK - see attachment 3
System wide checks OK *except* SYNC pulses - see attachments 4-6
Most recent file R101_249 ADC & disc data synchronised - see attachments 7-8
spectrum s1000 - per ASIC data items as function of timestamp (20us/channel)
spectra 1-32 and 100-132 per FEE64 data items as function of timestamp (20us/channel)
09.03 DAQ stopped
17.10 DAQ started (R103) no storage mode
Master MACB module #1 mode 0, 50MHz TTL clock signal cable disconnected from MACB rear panel.
18.55 Checked DAQ status and it has stopped. No file write times so can't easily check exactly when (see SYNC
stat page to calculate).
All system checks passed except SYNC errors (see attachments 9-12)
Upon attempting to Stop the DAQ, could not contact nnaida11.
AIDA SYNC statistics c. 1.76M SYNCs per FEE64. SYNC rate = 381Hz => elapsed time c. 1.3h => DAQ stalled c.
18.30
23.05 DAQ started (R104) no storage mode
Master MACB module #1 mode 0, 50MHz TTL clock signal cable disconnected from MACB rear panel. |
| Attachment 1: 1.png
|
|
| Attachment 2: 2.png
|
|
| Attachment 3: 3.png
|
|
| Attachment 4: 4.png
|
|
| Attachment 5: 5.png
|
|
| Attachment 6: 6.png
|
|
| Attachment 7: 7.png
|
|
| Attachment 8: 8.png
|
|
| Attachment 9: daqStop1.png
|
|
| Attachment 10: daqStop2.png
|
|
| Attachment 11: daqStop3.png
|
|
| Attachment 12: daqStop_SYNCstat.png
|
|
|
|
417
|
Thu Nov 3 01:25:30 2016 |
TD, CG | Thursday 3 November |
10.20 DAQ found stalled
DSSSD bias & leakage current OK - see attachments 1 & 2
FEE64 temperatures OK - see attachment 3
AIDA SYNC stats - see attachment 4 - stats 10.1M/381/3600 => DAQ stalled at 23.05 + 7.4h => c.06.29
System wide checks OK *except* SYNC pulses & SYNC errors - see attachments 5-8
Conclude DAQ stalling at c. 06.30/18.30 due to start of ORNL LN2 dewar refill cycle
but unrelated to either the BRIKEN-supplied 50MHz TTL clock signal to external clock
input of Master MACB or the ground connection provided by this coaxial cable.
Which implies the problem is either EMI/RFI or ac mains electrical ground.
AIDA powered by two single phase outlets from one of four sections of a single phase ac mains distribution
box with power supplied by two heavy duty (3cm dia) cables from Denkenseiki Noisecut AVR type PMN (AVR4, B3F
104V/51A). AIDA is connected to both NEMA 5-15R-style outlets of section A, LN2 refill to section D, BRIKEN
to sections B, C and D.
Ambient temperature +24.8 deg C, d.p. +4.1 deg C, RH 26.3%
15.00 DSSSD cabling drain wire attachment to AIDA adaptor PCB (FEE64 # 5 & 6) not tightly secured.
CG re-secured drain wire and this has significantly reduced 100kHz period transient noise
as observed at preamplifier outputs.
14.48 Restart DAQ (R110)
Master MACB mode 5, 50MHz TTL clock to external clock input
14.57 DAQ stalled
All system wide checks OK *except* SYNC errors - see attachment 9
ASIC clock timestamp - nnaida19 fails
N.B. SYNC pulses check OK, counters incrementing between successive tests
The latter is a problem we have seen previously but has not stopped DAQ running OK
aidas1> ./analyser /TapeData/Sept2016/R110_2
*** TDR format 3.3.0 analyser - TD - October 2016
*** ERROR: READ I/O error: 5002
blocks: 11861
ADC data format: 82556471 ( 877427.0 Hz)
Other data format: 14217428 ( 151105.7 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 27 ( 0.3 Hz)
RESUME: 27 ( 0.3 Hz)
SYNC100: 861083 ( 9151.8 Hz)
FEE64 disc: 13353468 ( 141923.4 Hz)
MBS info: 2823 ( 30.0 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 1976 ( 21.0 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
*** Timestamp elapsed time: 94.089 s
FEE elapsed dead time(s) elapsed idle time(s)
1 0.084 0.000
2 0.076 0.000
3 0.000 0.000
4 0.000 0.000
5 0.109 0.000
6 0.012 0.000
7 0.000 0.000
8 0.000 0.000
9 0.003 0.000
10 0.000 0.000
11 0.000 0.000
12 0.000 0.000
13 0.000 0.000
14 0.000 0.000
15 0.032 0.000
16 0.013 0.000
17 0.000 0.000
18 0.000 0.000
19 0.125 0.000
20 0.000 0.000
21 0.000 0.000
22 0.000 0.000
23 0.142 0.149
24 0.076 0.000
25 0.000 0.000
26 0.000 0.000
27 0.000 0.000
28 0.000 0.000
29 0.000 0.000
30 0.000 0.000
31 0.000 0.000
32 0.000 0.000
*** Statistics
FEE ADC Data Other Data Sample Undefined Pause Resume SYNC100 Disc MBS Other
HEC Data
0 0 0 0 0 0 0 0 0 0 0
0
1 942953 5806860 0 0 3 3 35855 5770999 0 0
42
2 4036003 4268394 0 0 2 2 35861 4232529 0 0
0
3 3038422 39132 0 0 0 0 35892 3240 0 0
0
4 1192440 42063 0 0 0 0 35889 6174 0 0
68
5 5685966 39310 0 0 5 5 35849 3451 0 0
0
6 3670394 46605 0 0 2 2 35885 7893 2823 0
70
7 2661022 43508 0 0 0 0 35890 7618 0 0
231
8 2165335 41382 0 0 0 0 35890 5492 0 0
359
9 3037520 863964 0 0 1 1 35889 828073 0 0
0
10 2616647 179334 0 0 0 0 35889 143445 0 0
0
11 1083633 39064 0 0 0 0 35889 3175 0 0
0
12 1759629 38368 0 0 0 0 35888 2480 0 0
0
13 2419715 280498 0 0 0 0 35891 244607 0 0
0
14 3189205 1325559 0 0 0 0 35887 1289672 0 0
747
15 4886680 39482 0 0 2 2 35871 3607 0 0
7
16 5087977 38869 0 0 1 1 35886 2981 0 0
0
17 2905638 329411 0 0 0 0 35888 293523 0 0
0
18 3327386 46016 0 0 0 0 35888 10128 0 0
0
19 6770177 40711 0 0 4 4 35842 4861 0 0
7
20 2225755 38524 0 0 0 0 35891 2633 0 0
344
21 2061728 40299 0 0 0 0 35887 4412 0 0
90
22 3052329 512109 0 0 0 0 35890 476219 0 0
7
23 9325689 38673 0 0 3 3 35834 2833 0 0
3
24 5414228 39293 0 0 4 4 35862 3423 0 0
1
25 0 0 0 0 0 0 0 0 0 0
0
26 0 0 0 0 0 0 0 0 0 0
0
27 0 0 0 0 0 0 0 0 0 0
0
28 0 0 0 0 0 0 0 0 0 0
0
29 0 0 0 0 0 0 0 0 0 0
0
30 0 0 0 0 0 0 0 0 0 0
0
31 0 0 0 0 0 0 0 0 0 0
0
32 0 0 0 0 0 0 0 0 0 0
0
*** Timewarps
FEE ADC Pause Resume SYNC100 Disc MBS Undefined Samples
0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 0
2 0 0 0 0 0 0 0 0
3 0 0 0 0 0 0 0 0
4 0 0 0 0 0 0 0 0
5 0 0 0 0 0 0 0 0
6 0 0 0 0 0 0 0 0
7 0 0 0 0 0 0 0 0
8 0 0 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0
11 0 0 0 0 0 0 0 0
12 0 0 0 0 0 0 0 0
13 0 0 0 0 0 0 0 0
14 0 0 0 0 0 0 0 0
15 0 0 0 0 0 0 0 0
16 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0
*** Program elapsed time: 1.500s ( 7907.333 blocks/s, 494.208 Mb/s)
15.48 Restart aidas1
Power cycle FEE64s
Restart AIDA DAQ
Same problem - DAQ stall (no output from merger) at, or shortly after, Start ASIC Readout with
multiple FEE64s with zero AIDA SYNC rate
*All* system wide checks OK
Cannot recover DAQ control of FEE64s with merger restart - power cycle FEE64s again
23.58 Attempt to change master MACB from MACB module #11 mode 5 to MACB module #13 mode 0
Successive fuse failures of USB ac mains relay
System wide ASIC clock timestamp check nnaida 5/6/7/8 13/14/15/16 21/22/23/24 i.e. first half of MACB tree
Re-seat HDMI cabling from master MACB root module to first half of MACB tree - ReSYNC fails - see attachments
10 & 11
|
| Attachment 1: 1.png
|
|
| Attachment 2: 2.png
|
|
| Attachment 3: 3.png
|
|
| Attachment 4: 4.png
|
|
| Attachment 5: 5.png
|
|
| Attachment 6: 6.png
|
|
| Attachment 7: 7.png
|
|
| Attachment 8: 8.png
|
|
| Attachment 9: 9.png
|
|
| Attachment 10: 200.png
|
|
| Attachment 11: 201.png
|
|
|
|
459
|
Sat Nov 19 02:46:09 2016 |
TD, CG | Saturday 19 November |
11.45 DSSSD bias & leakage currents OK - see attachments 1 & 2
FEE64 temperatures OK - see attachment 3
Updated copy of GoogleDocs leakage current spreadsheet - attachment 4
20:17 DSSD bias and leakage currents ok (attachments 4+5).
FEE temps ok (attachment 6) |
| Attachment 1: 1.png
|
|
| Attachment 2: 2.png
|
|
| Attachment 3: 3.png
|
|
| Attachment 4: AIDA_leakage_current(3).xlsx
|
| Attachment 5: bias1.png
|
|
| Attachment 6: bias2.png
|
|
| Attachment 7: temps.png
|
|
|
|
464
|
Wed Nov 23 01:47:28 2016 |
TD, CG | Wednesday 23 November |
10.45 DSSSD bias & leakage currents - see attachments 1 & 2
note - significant increase in leakage current (~0.8uA) since previous readings
may be temperature transient or charging effect due to delta electrons due to BigRIPS tuning to F11
15cm Pb installed upstream of AIDA+BRIKEN some days ago
FEE64 temperatures OK - see attachment 3
15.35 DSSD bias and leakage currents ok (attachments 4+5).
Marginal increase over this morning.
FEE temps ok (attachment 6).
|
| Attachment 1: 1.png
|
|
| Attachment 2: 2.png
|
|
| Attachment 3: 3.png
|
|
| Attachment 4: bias1.png
|
|
| Attachment 5: bias2.png
|
|
| Attachment 6: temps.png
|
|
|
|
466
|
Thu Nov 24 01:07:14 2016 |
TD, CG | Thursday 24 November |
10.05 DSSSD bias & leakage currents OK - see attachments 1 & 2
note - the significant increase observed yesterday appears to be plateau'ing
FEE64 temperatures OK - see attachment 3
10.19 System-wide tests - see attachments 4-17
Clock status fails for varying combinations of nnaida5, 7, 8, 15 & 16
ADC calibration fails for nnaida23
All other tests OK
10.27 LOCAL controls for nnaida5, 6, 7, 8, 15 & 16 - see attachments 18-23
10.37 Start DAQ (Nov2016/R1)
Analysis of R1_0 demonstrates ADC & disc data synchronised - see attachments 24 & 25
Analysis of R1_0 - see attachment 26
20.10 Successfully restarted the DAQ after fill cycle stop.
Bias and leakage currents ok (attachments 27+28).
Temps ok (attachment 29). |
| Attachment 1: 1.png
|
|
| Attachment 2: 2.png
|
|
| Attachment 3: 3.png
|
|
| Attachment 4: 4.png
|
|
| Attachment 5: 5.png
|
|
| Attachment 6: 6.png
|
|
| Attachment 7: 7.png
|
|
| Attachment 8: 8.png
|
|
| Attachment 9: 9.png
|
|
| Attachment 10: 10.png
|
|
| Attachment 11: 11.png
|
|
| Attachment 12: 12.png
|
|
| Attachment 13: 13.png
|
|
| Attachment 14: 14.png
|
|
| Attachment 15: 15.png
|
|
| Attachment 16: 16.png
|
|
| Attachment 17: 17.png
|
|
| Attachment 18: 18.png
|
|
| Attachment 19: 19.png
|
|
| Attachment 20: 20.png
|
|
| Attachment 21: 21.png
|
|
| Attachment 22: 22.png
|
|
| Attachment 23: 23.png
|
|
| Attachment 24: 30.png
|
|
| Attachment 25: 31.png
|
|
| Attachment 26: R1.out
|
*** TDR format 3.3.0 analyser - TD - October 2016
*** TDR format 3.3.0 analyser - TD - October 2016
*** ERROR: READ I/O error: 5002
blocks: 32000
ADC data format: 223675520 ( 579476.2 Hz)
Other data format: 37404321 ( 96903.4 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 94 ( 0.2 Hz)
RESUME: 94 ( 0.2 Hz)
SYNC100: 3533023 ( 9153.0 Hz)
FEE64 disc: 33860100 ( 87721.4 Hz)
MBS info: 11010 ( 28.5 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 5196 ( 13.5 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
*** Timestamp elapsed time: 385.996 s
FEE elapsed dead time(s) elapsed idle time(s)
1 0.529 0.267
2 0.000 0.000
3 0.000 0.000
4 0.000 0.000
5 0.094 0.000
6 0.000 0.000
7 0.000 0.000
8 0.000 0.000
9 0.040 0.000
10 0.000 0.000
11 0.000 0.000
12 0.000 0.000
13 0.000 0.000
14 0.054 0.000
15 0.000 0.000
16 0.000 0.000
17 0.112 0.000
18 0.001 0.000
19 0.011 0.000
20 0.000 0.000
21 0.000 0.000
22 0.000 0.000
23 0.255 0.000
24 0.000 0.000
25 0.000 0.000
26 0.000 0.000
27 0.000 0.000
28 0.000 0.000
29 0.000 0.000
30 0.000 0.000
31 0.000 0.000
32 0.000 0.000
*** Statistics
FEE ADC Data Other Data Sample Undefined Pause Resume SYNC100 Disc MBS Other HEC Data
0 0 0 0 0 0 0 0 0 0 0 0
1 16554209 17740898 0 0 13 13 147026 17593846 0 0 269
2 3734361 238045 0 0 0 0 147239 90806 0 0 0
3 10586532 159695 0 0 0 0 147235 12460 0 0 0
4 1610610 157325 0 0 0 0 147224 10101 0 0 11
5 15070140 162253 0 0 26 26 147203 14998 0 0 0
6 2663164 167198 0 0 0 0 147213 8975 11010 0 47
7 3770260 157365 0 0 0 0 147230 10135 0 0 53
8 2733567 159510 0 0 0 0 147232 12278 0 0 1329
9 11962290 4242132 0 0 7 7 147197 4094921 0 0 0
10 10178199 539710 0 0 0 0 147234 392476 0 0 0
11 1325094 159566 0 0 0 0 147225 12341 0 0 0
12 2646658 156577 0 0 0 0 147230 9347 0 0 0
13 9431851 553540 0 0 0 0 147226 406314 0 0 0
14 11713269 6046015 0 0 6 6 147202 5898801 0 0 1886
15 9547982 159903 0 0 0 0 147225 12678 0 0 0
16 9189286 158739 0 0 0 0 147218 11521 0 0 0
17 12814991 2179220 0 0 26 26 147178 2031990 0 0 0
18 13439977 392095 0 0 1 1 147223 244870 0 0 0
19 14542981 164981 0 0 5 5 147220 17751 0 0 1
20 6034681 157665 0 0 0 0 147229 10436 0 0 1167
21 9567778 190348 0 0 0 0 147234 43114 0 0 405
22 11388041 3044033 0 0 0 0 147237 2896796 0 0 17
23 25281301 157383 0 0 10 10 147122 10241 0 0 11
24 7888298 160125 0 0 0 0 147221 12904 0 0 0
25 0 0 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0 0 0
*** Timewarps
FEE ADC Pause Resume SYNC100 Disc MBS Undefined Samples
0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 0
2 0 0 0 0 0 0 0 0
3 0 0 0 0 0 0 0 0
4 0 0 0 0 0 0 0 0
5 0 0 0 0 0 0 0 0
6 0 0 0 0 0 0 0 0
7 0 0 0 0 0 0 0 0
8 0 0 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0
11 0 0 0 0 0 0 0 0
12 0 0 0 0 0 0 0 0
13 0 0 0 0 0 0 0 0
14 0 0 0 0 0 0 0 0
15 0 0 0 0 0 0 0 0
16 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0
*** Program elapsed time: 16.387s ( 1952.801 blocks/s, 122.050 Mb/s)
|
| Attachment 27: bias1.png
|
|
| Attachment 28: bias2.png
|
|
| Attachment 29: temps.png
|
|
|
|
846
|
Sun Jun 16 10:06:21 2019 |
TD, CG | FEE64 nnaida14 noise issues |
Attachments 1, 3, 5 & 7 *.L spectra log scale
Attachments 2, 4, 6 & 8 *.L spectra linear scale (expanded around pulser peak)
Attachments 9-12 *.W spectra 20us FSR
Observe
2.13.L low amplitude
2.14.L & 3.3.L large number of counts and 'sinusoidal' continuum
... but ... the corresponding waveform spectra look OK
Zero'd all histograms to confirm observations - confirmed - see attachment 16
Pulser peak width spectrum 3.8.L 71 ch FWHM |
| Attachment 1: 100.png
|
|
| Attachment 2: 101.png
|
|
| Attachment 3: 102.png
|
|
| Attachment 4: 103.png
|
|
| Attachment 5: 104.png
|
|
| Attachment 6: 105.png
|
|
| Attachment 7: 106.png
|
|
| Attachment 8: 107.png
|
|
| Attachment 9: 110.png
|
|
| Attachment 10: 111.png
|
|
| Attachment 11: 112.png
|
|
| Attachment 12: 113.png
|
|
| Attachment 13: 120.png
|
|
| Attachment 14: 121.png
|
|
| Attachment 15: 122.png
|
|
| Attachment 16: 130.png
|
|
|
|
922
|
Fri Nov 17 09:19:41 2023 |
TD, CC, VP | AIDA test 17/11/2023 |
We restarted AIDA (a10277ttachment 6) this morning and checked rates (attachments 3/4) and waveforms (attachments 1/2). Temperatures are shown in attachment 5.
It seems consistent with what was observed yesterday.
AIDA stopped and platform moved.
The power cable from USB main power relay was connected to the wall and not the DESPEC 19" racks as expected. It has been corrected : it is now connected to the DESPEC 19" rack, has it should be.
AIDA was restarted after that. bPlast was disconnected from 2 booster cards.
We checked the rates (attachments 7,10 and 11 [linear scale]). They are quite similar, although aida01 rate has increased (maybe a cable issue), and rate from aida07 significantly decreased (it was really low yesterday as well).
The waveforms (attachments 8 & 9) are similar as the ones obtained earlier.
bPlast has been reconnected to all booster cards. Rate increased a little (attachment 12) but broadly similar to that observed first thing this morning.
Then 2 booster cards were grounded, and the rates went up (attachments 13-16)
Afterwards, one power cable was connected to the booster card on the short side, increasing the noise, as observed yesterday (attachments 17-20)
The Rohde Schwarz PSU output was referenced to PSU ground ( 3mm cable to ground socket front panel ). It did not reduced much the noise (attachment 21)
Finally, the power supply was switched on, increasing the noise in AIDA (attachment 22).
After the meeting, we have chosen to reconnect bPlast to the booster : now all LIMOs are connected to the booster cards, the cards are powered and the SiPM are biased. The alarm for the short side has triggered (not full bias)
The Rohde & Scwarz PSU outputs is till referenced to its chassis ground.
See attachments 23-26 for the rates and waveforms in AIDA
14.57 JG/JB added additional black cloth to reduce nPlast SiPM dark current : now it is around 16uA. Updated noise observations
per FEE64 1.8.W spectra 20us FSR - attachments 27-28
per FEE64 1Rate spectra - attachment 29
ADC data item stats - attachment 30
aida02 asics #2-4 remain noisy c. kHz/channel
Most other p+n FEE64s are OK - the usual 'hot' channels but most other channel rates low
15.07 JG removed ground reference for Rohde & Schwarz PSU +4V output to bPlast booster cards
ADC data items atst - attachment 31
Most FEE64s have significantly higher rates so the ground reference is doing 'something'
15.12 JG re-ground ref'd Rohde & Schwarz PSU +4V output to bPlast booster cards
ADC data item stats - attachment 32
Rates for most FEE64s are reduced - some significantly - PSU ground ref is doing something reproducible
15.25 Christoph Scheidenberger reports FRS magnets ON
16.01 JG/JB TAMEX & NIM bin electrically isolated from AIDA stand/DESPEC platform. Screens of flat cables from bPlast to booster cards grounded to snout.
ADC data item stats - attachment 33
per FEE64 1.8.W spectra 20us FSR - attachments 36-37
per FEE64 Rate spectra - attachments 34-35
16.20 TAMEX & NIM bin electrically isolated from AIDA stand/DESPEC platform. Screens of flat cables from bPlast to booster cards now connected together but electrically floating
ADC data item stats - attachment 40
per FEE64 1.8.W spectra 20us FSR - attachments 38-39
per FEE64 Rate spectra - attachment 41
Noted improvement in most FEE64 rates - aida04 improved significantly
16.31 TAMEX & NIM bin electrically isolated from AIDA stand/DESPEC platform. Screens of flat cables from bPlast to booster cards now connected together and grounded to NIM bin chassis.
ADC data item stats - attachment 42
per FEE64 Rate spectra - attachment 43-44
per FEE64 1.8.W spectra 20us FSR - attachments 44-46
Continued, imcremental improvement in FEE64 rates |
| Attachment 1: Screenshot_from_2023-11-17_10-17-43.png
|
|
| Attachment 2: Screenshot_from_2023-11-17_10-15-44.png
|
|
| Attachment 3: Screenshot_from_2023-11-17_10-14-52.png
|
|
| Attachment 4: Screenshot_from_2023-11-17_10-13-44.png
|
|
| Attachment 5: Screenshot_from_2023-11-17_10-04-17.png
|
|
| Attachment 6: Screenshot_from_2023-11-17_10-03-50.png
|
|
| Attachment 7: Screenshot_from_2023-11-17_11-40-57.png
|
|
| Attachment 8: Screenshot_from_2023-11-17_11-40-16.png
|
|
| Attachment 9: Screenshot_from_2023-11-17_11-39-30.png
|
|
| Attachment 10: Screenshot_from_2023-11-17_11-38-44.png
|
|
| Attachment 11: Screenshot_from_2023-11-17_11-48-50.png
|
|
| Attachment 12: Screenshot_from_2023-11-17_11-53-54.png
|
|
| Attachment 13: Screenshot_from_2023-11-17_12-01-25.png
|
|
| Attachment 14: Screenshot_from_2023-11-17_12-00-29.png
|
|
| Attachment 15: Screenshot_from_2023-11-17_11-59-44.png
|
|
| Attachment 16: Screenshot_from_2023-11-17_11-59-20.png
|
|
| Attachment 17: Screenshot_from_2023-11-17_12-04-44.png
|
|
| Attachment 18: Screenshot_from_2023-11-17_12-03-47.png
|
|
| Attachment 19: Screenshot_from_2023-11-17_12-03-06.png
|
|
| Attachment 20: Screenshot_from_2023-11-17_12-02-57.png
|
|
| Attachment 21: Screenshot_from_2023-11-17_12-06-27.png
|
|
| Attachment 22: Screenshot_from_2023-11-17_12-10-02.png
|
|
| Attachment 23: Screenshot_from_2023-11-17_14-34-19.png
|
|
| Attachment 24: Screenshot_from_2023-11-17_14-34-10.png
|
|
| Attachment 25: Screenshot_from_2023-11-17_14-33-48.png
|
|
| Attachment 26: Screenshot_from_2023-11-17_14-33-11.png
|
|
| Attachment 27: Screenshot_from_2023-11-17_14-59-43.png
|
|
| Attachment 28: Screenshot_from_2023-11-17_14-58-52.png
|
|
| Attachment 29: Screenshot_from_2023-11-17_14-57-50.png
|
|
| Attachment 30: Screenshot_from_2023-11-17_14-57-30.png
|
|
| Attachment 31: Screenshot_from_2023-11-17_15-07-28.png
|
|
| Attachment 32: Screenshot_from_2023-11-17_15-12-27.png
|
|
| Attachment 33: Screenshot_from_2023-11-17_16-04-29.png
|
|
| Attachment 34: Screenshot_from_2023-11-17_16-03-56.png
|
|
| Attachment 35: Screenshot_from_2023-11-17_16-03-51.png
|
|
| Attachment 36: Screenshot_from_2023-11-17_16-03-32.png
|
|
| Attachment 37: Screenshot_from_2023-11-17_16-03-14.png
|
|
| Attachment 38: Screenshot_from_2023-11-17_16-21-23.png
|
|
| Attachment 39: Screenshot_from_2023-11-17_16-20-49.png
|
|
| Attachment 40: Screenshot_from_2023-11-17_16-20-22.png
|
|
| Attachment 41: Screenshot_from_2023-11-17_16-20-13.png
|
|
| Attachment 42: Screenshot_from_2023-11-17_16-31-51.png
|
|
| Attachment 43: Screenshot_from_2023-11-17_16-33-29.png
|
|
| Attachment 44: Screenshot_from_2023-11-17_16-33-23.png
|
|
| Attachment 45: Screenshot_from_2023-11-17_16-32-47.png
|
|
| Attachment 46: Screenshot_from_2023-11-17_16-30-01.png
|
|
|
|
576
|
Fri May 19 02:44:14 2017 |
TD, CB, OH | Friday 19th May |
10:44 System wide checks all ok
Leakages and biases all ok (see attachment 1)
Ambient Temperature 24.4C - Humidity 38.1% - Dew Point +9.1
FEE temperatures ok (see attachment 2)
DAQ, Merger and Tape server all still running ok
Good events statistics (see attachment 3)
Orientation of RS-232 cards was wrong. They have now been corrected will enable logging then do a halt and
go on the daq.
Power supply also moved onto AIDA circuit.
11:00 Current file R15_333.
11:04 halt and go performed
24 Log files produced on Rasp Pi console monitor. |
| Attachment 1: 190517_bias1045.png
|
|
| Attachment 2: 190517_feetemp1045.png
|
|
| Attachment 3: 190517_goodevents1056.png
|
|
|
|
604
|
Sun May 28 02:16:04 2017 |
TD, CB | Sunday 28 May |
10.15 DAQ continues OK
Current file May2017/R31_239
Detector biases; leakage currents OK - see attachments 1 & 2
FEE64 Temperatures OK - see attachment 3
good events; statistics OK - see attachment 4
No warnings/errors reported by merger since DAQ GO - see attachment 5
*all* system wide checks OK *except* ADC calibration of nnaida1, 21 and nnaida1 clock status (status=6)
10.30 E17/F11 ambient temperature +23.8 deg C, RH 40.9%, d.p. +9.6 deg C
Julabo FL11006 set point +20 deg C, actual +20.0 deg C, water level OK (c. 95% - refilled)
10.35 Layouts 1, 3, 4, 7 and 8 - see attachments 6-13
Note - as expected, zero HEC events, layouts 5 & 6 not shown
10.58 DAQ stop - transient timeout nnaida1 - followed by successful stop
File May2017/R31_248
Delete R31
11.12 Disable data transfer
DAQ GO - transient timeout nnaida1 - followed by successful start
nnaida1 system console - see attachment 14
12:00 Moved plastic veto scintillator downstream in an attempt to reduce noise
Rates in noisy FEEs reduced by 20-30%
14:30 DAQ continues OK
Found that pulser induces counts in some HEC (nnaida20, 14 in particular). See attachment 15. Pulser off
removes these events completely.
These events could be mistaken for implantation events during data acquisition.
Increasing the fast comparator threshold HEC from 2 -> 3 suppresses the events. Increasing the threshold
to 4 does not appear to have further effects.
Even with the increased threshold some HEC (e.g. 1.0H) still see the pulser. See attachment 16 (fast
comparator threshold HEC was 0x3).
Checked data is synchronised in file R32_33
15:29 Set the threshold back to 2. We will perform this cut off-line in the analysis. |
| Attachment 1: 1.png
|
|
| Attachment 2: 2.png
|
|
| Attachment 3: 3.png
|
|
| Attachment 4: 4.png
|
|
| Attachment 5: 5.png
|
|
| Attachment 6: 6.png
|
|
| Attachment 7: 7.png
|
|
| Attachment 8: 8.png
|
|
| Attachment 9: 9.png
|
|
| Attachment 10: 10.png
|
|
| Attachment 11: 11.png
|
|
| Attachment 12: 12.png
|
|
| Attachment 13: 13.png
|
|
| Attachment 14: ttyUSB7
|
executing generic doStop
EXPERIMENTS/AIDA/Options/nnaida1 Stat.offset NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Aida_Hist_W_Enable NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 TS_SYNC_PHASE NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 ExtClk NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Aida.shift NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 MACB_TRIG_MODE NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Aida.offset NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Aida_GroupBase NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Rate.channels NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Stat.channels NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Aida.Vchannels NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 ASIC.settings NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 WAVE_DMA_HWM NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Aida_Hist_D_Enable NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Stat.shift NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Aida.Wchannels NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Aida_Hist_V_Enable NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Include.Aida NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Aida.channels NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 DataAcqPgm NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Aida_Hist_H_Enable NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 DataFormat NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 ASIC_DMA_HWM NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Aida_Hist_L_Enable NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Index NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
DataAcqCommand 1 Halt
Aida state now Stopped. AidaExecV8_52: Build 170816
Halt (1)
DataAcqCommand 1 disconnect xfer stream #0
disconnect xfer stream #0 (1)
completed generic doStop
executing generic doGo
EXPERIMENTS/AIDA/Options/nnaida1 Stat.offset NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Aida_Hist_W_Enable NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 TS_SYNC_PHASE NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 ExtClk NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Aida.shift NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 MACB_TRIG_MODE NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Aida.offset NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Aida_GroupBase NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Rate.channels NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Stat.channels NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Aida.Vchannels NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 ASIC.settings NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 WAVE_DMA_HWM NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Aida_Hist_D_Enable NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Stat.shift NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Aida.Wchannels NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Aida_Hist_V_Enable NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Include.Aida NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Aida.channels NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 DataAcqPgm NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Aida_Hist_H_Enable NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 DataFormat NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 ASIC_DMA_HWM NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Aida_Hist_L_Enable NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Index NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
DataAcqCommand 3 connect xfer stream #0
Initialising communication with storage server myserver on port 11001
Setting Transfer Block Size 65536
Setting Transfer Mode 3
Setting Overlap Mode 0
Setting Blocking Mode 0
Setting nice 0
TCP transfer library version 3.12
TCP socket send buffer was 16384 - now 221184
TCP socket receive buffer was 87380 - now 221184
TCP socket created OK - now connecting to myserver port 11001
Connected to myserver port 11001
connect xfer stream #0 (1)
DataAcqCommand 2 Go
Data Acquisition Statistics counters now cleared
ASIC DMA buffer size 2097152; WAVE DMA buffer size 1048576
Aida state now Going. AidaExecV8_52: Build 170816
Go (1)
Running Go_Electronics for nnaida1 with TSMaster nnaida6
ADCs Calibrated.
get_WAVEBlk (A) which = 0 ---- status = 0048, State machines = 0201
WV:0:Buffer info. Length = 64 : Offset = 0
WV:1:Buffer info. Length = 62 : Offset = 2
WV:2:Buffer info. Length = 60 : Offset = 4
WV:3:Buffer info. Length = 58 : Offset = 6
WV:4:Buffer info. Length = 56 : Offset = 8
WV:5:Buffer info. Length = 54 : Offset = 10
WV:6:Buffer info. Length = 52 : Offset = 12
WV:7:Buffer info. Length = 50 : Offset = 14
WV:8:Buffer info. Length = 48 : Offset = 16
WV:9:Buffer info. Length = 46 : Offset = 18
Finished Go_Electronics for AIDA
get_WAVEBlk (A) which = 1 ---- status = 0088, State machines = 0504
completed generic doGo
get_WAVEBlk (A) which = 0 ---- status = 0048, State machines = 0201
get_WAVEBlk (A) which = 1 ---- status = 0088, State machines = 0201
get_WAVEBlk (A) which = 0 ---- status = 0048, State machines = 0201
get_WAVEBlk (A) which = 1 ---- status = 0088, State machines = 0201
get_WAVEBlk (A) which = 0 ---- status = 0048, State machines = 0201
get_WAVEBlk (A) which = 1 ---- status = 0088, State machines = 0201
get_WAVEBlk (A) which = 0 ---- status = 0048, State machines = 0201
get_WAVEBlk (A) which = 1 ---- status = 0088, State machines = 0404
In RDOGo_Operate: Enabled Correlation, ASIC and discriminator readout
executing generic doStop
EXPERIMENTS/AIDA/Options/nnaida1 Stat.offset NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Aida_Hist_W_Enable NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 TS_SYNC_PHASE NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 ExtClk NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Aida.shift NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 MACB_TRIG_MODE NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Aida.offset NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Aida_GroupBase NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Rate.channels NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Stat.channels NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Aida.Vchannels NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 ASIC.settings NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 WAVE_DMA_HWM NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Aida_Hist_D_Enable NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Stat.shift NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Aida.Wchannels NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Aida_Hist_V_Enable NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Include.Aida NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Aida.channels NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 DataAcqPgm NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Aida_Hist_H_Enable NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 DataFormat NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 ASIC_DMA_HWM NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Aida_Hist_L_Enable NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
EXPERIMENTS/AIDA/Options/nnaida1 Index NOT available: error 0x10004; unable to access CONTENTS for node /MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1: couldn't open "/MIDAS/DB/EXPERIMENTS/AIDA/Options/nnaida1/CONTENTS": permission denied
DataAcqCommand 1 Halt
Aida state now Stopped. AidaExecV8_52: Build 170816
Halt (1)
DataAcqCommand 1 disconnect xfer stream #0
disconnect xfer stream #0 (1)
completed generic doStop
Flash base address=FC000000
Set Flash to ASync Mode
XFLASH_PART_NOT_SUPPORTED
Finished copying zImage to RAM
Found 0 errors checking kernel image
VHDL version number 0X01900C1F
Bootloader version number 1.2.7 -- 22nd August 2012
Starting LMK 3200 setup
Setting LMK03200 to standard clock settings -- External Clock 23Nov15
.... SPI Base Address=0x81400000
clk_control_reg=0x4
Next step is SPIconfig
Control 32(0x81400000)=0x180
SlaveSel(0x81400000)=0x3
Ctrl(0x81400000)=0xE6
Ctrl(0x81400000)=0x86
SPIconfig done now to set up the LMK3200 registers
LMK #0 : regInit[0]=0x80000000
LMK #0 : regInit[1]=0x10070600
LMK #0 : regInit[2]=0x60601
LMK #0 : regInit[3]=0x60602
LMK #0 : regInit[4]=0x60603
LMK #0 : regInit[5]=0x70624
LMK #0 : regInit[6]=0x70605
LMK #0 : regInit[7]=0x70606
LMK #0 : regInit[8]=0x70627
LMK #0 : regInit[9]=0x10000908
LMK #0 : regInit[10]=0xA0022A09
LMK #0 : regInit[11]=0x82800B
LMK #0 : regInit[12]=0x28C800D
LMK #0 : regInit[13]=0x830020E
LMK #0 : regInit[14]=0xC800180F
Calibrate completed at 942 counts
Setting Clock Control =0x0000000B, to set GOE and sync bit
Ctrl @ SPIstop (0x81400000)=0x186
0 delay setup completed at 0 counts
Finished Clock setup LMK03200
completed LMK 3200 setup
Loaded all four ASICs with default settings
Setting the ADCs into calibration mode
Control 32(0x81400400)=0x180
SlaveSel(0x81400400)=0xFF
Ctrl(0x81400400)=0xE6
Ctrl(0x81400400)=0x86
Init : Config of AD9252 SPI ok
Ctrl @ SPIstop (0x81400400)=0x186
ADCs initialised
waiting for 10 seconds
ADCs calibrated
Control 32(0x81400400)=0x186
SlaveSel(0x81400400)=0xFF
Ctrl(0x81400400)=0xE6
Ctrl(0x81400400)=0x86Config of AD9252 SPI ok
Ctrl @ SPIstop (0x81400400)=0x186
ADCs set for PN-9
Jumping to kernel simpleboot...
zImage starting: loaded at 0x00a00000 (sp: 0x00bc4eb0)
Allocating 0x3b78cc bytes for kernel ...
gunzipping (0x00000000 <- 0x00a0f000:0x00bc380e)...done 0x39604c bytes
Linux/PowerPC load: console=ttyS0 root=/dev/nfs ip=on rw mem=112M
Finalizing device tree... flat tree at 0xbd1300
Probing IIC bus for MAC... MAC address = 0x0 0x4 0xa3 0x2a 0xd0 0x1b
Using Xilinx Virtex440 machine description
Linux version 2.6.31 (nf@nnlxb.dl.ac.uk) (gcc version 4.2.2) #34 PREEMPT Tue Nov 15 15:57:04 GMT 2011
Zone PFN ranges:
DMA 0x00000000 -> 0x00007000
Normal 0x00007000 -> 0x00007000
Movable zone start PFN for each node
early_node_map[1] active PFN ranges
0: 0x00000000 -> 0x00007000
MMU: Allocated 1088 bytes of context maps for 255 contexts
Built 1 zonelists in Zone order, mobility grouping on. Total pages: 28448
Kernel command line: console=ttyS0 root=/dev/nfs ip=on rw mem=112M
PID hash table entries: 512 (order: 9, 2048 bytes)
Dentry cache hash table entries: 16384 (order: 4, 65536 bytes)
Inode-cache hash table entries: 8192 (order: 3, 32768 bytes)
Memory: 109680k/114688k available (3500k kernel code, 4852k reserved, 144k data, 130k bss, 168k init)
Kernel virtual memory layout:
* 0xffffe000..0xfffff000 : fixmap
* 0xfde00000..0xfe000000 : consistent mem
* 0xfde00000..0xfde00000 : early ioremap
* 0xd1000000..0xfde00000 : vmalloc & ioremap
NR_IRQS:512
clocksource: timebase mult[a00000] shift[22] registered
Console: colour dummy device 80x25
Mount-cache hash table entries: 512
NET: Registered protocol family 16
PCI: Probing PCI hardware
bio: create slab <bio-0> at 0
NET: Registered protocol family 2
IP route cache hash table entries: 1024 (order: 0, 4096 bytes)
TCP established hash table entries: 4096 (order: 3, 32768 bytes)
TCP bind hash table entries: 4096 (order: 2, 16384 bytes)
TCP: Hash tables configured (established 4096 bind 4096)
TCP reno registered
NET: Registered protocol family 1
ROMFS MTD (C) 2007 Red Hat, Inc.
msgmni has been set to 214
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered
io scheduler cfq registered (default)
Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
83e00000.serial: ttyS0 at MMIO 0x83e01003 (irq = 16) is a 16550
console [ttyS0] enabled
brd: module loaded
loop: module loaded
Device Tree Probing 'ethernet'
xilinx_lltemac 81c00000.ethernet: MAC address is now 0: 4:a3:2a:d0:1b
xilinx_lltemac 81c00000.ethernet: XLlTemac: using DMA mode.
XLlTemac: DCR address: 0x80
XLlTemac: buffer descriptor size: 32768 (0x8000)
XLlTemac: Allocating DMA descriptors with kmalloc
XLlTemac: (buffer_descriptor_init) phy: 0x6938000, virt: 0xc6938000, size: 0x8000
XTemac: PHY detected at address 7.
xilinx_lltemac 81c00000.ethernet: eth0: Xilinx TEMAC at 0x81C00000 mapped to 0xD1024000, irq=17
fc000000.flash: Found 1 x16 devices at 0x0 in 16-bit bank
Intel/Sharp Extended Query Table at 0x010A
Intel/Sharp Extended Query Table at 0x010A
Intel/Sharp Extended Query Table at 0x010A
Intel/Sharp Extended Query Table at 0x010A
Intel/Sharp Extended Query Table at 0x010A
Intel/Sharp Extended Query Table at 0x010A
Using buffer write method
cfi_cmdset_0001: Erase suspend on write enabled
cmdlinepart partition parsing not available
RedBoot partition parsing not available
Creating 5 MTD partitions on "fc000000.flash":
0x000000000000-0x000000500000 : "golden_firmware"
0x000000500000-0x000000800000 : "golden_kernel"
0x000000800000-0x000000d00000 : "user_firmware"
0x000000d00000-0x000000fe0000 : "user_kernel"
0x000000fe0000-0x000001000000 : "env_variables"
xilinx-xps-spi 81400400.hd-xps-spi: at 0x81400400 mapped to 0xD1028400, irq=20
SPI: XIlinx spi: bus number now 32766
xilinx-xps-spi 81400000.xps-spi: at 0x81400000 mapped to 0xD102C000, irq=21
SPI: XIlinx spi: bus number now 32765
mice: PS/2 mouse device common for all mice
Device Tree Probing 'i2c'
#0 at 0x81600000 mapped to 0xD1030000, irq=22
at24 0-0050: 1024 byte 24c08 EEPROM (writable)
TCP cubic registered
NET: Registered protocol family 17
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
eth0: XLlTemac: Options: 0x3fa
eth0: XLlTemac: allocating interrupt 19 for dma mode tx.
eth0: XLlTemac: allocating interrupt 18 for dma mode rx.
eth0: XLlTemac: speed set to 1000Mb/s
eth0: XLlTemac: Send Threshold = 24, Receive Threshold = 4
eth0: XLlTemac: Send Wait bound = 254, Receive Wait bound = 254
Sending DHCP requests ., OK
IP-Config: Got DHCP answer from 10.1.1.250, my address is 10.1.1.1
IP-Config: Complete:
device=eth0, addr=10.1.1.1, mask=255.255.255.0, gw=10.1.1.250,
host=nnaida1, domain=, nis-domain=(none),
bootserver=10.1.1.250, rootserver=10.1.1.250, rootpath=/MIDAS@aidas/XilinxLinux/ppc_4xx/rfs/nnaida1,tcp
... 4312 more lines ...
|
| Attachment 15: 100.png
|
|
| Attachment 16: 101.png
|
|
|
|
613
|
Tue May 30 00:58:10 2017 |
TD, CB | Tuesday 30 May - day shift |
08.58 DAQ continues OK
File RIBF127R1/R1_99
BigRIPS run 1009
BRIKEN run 9
BNC PB-4
Amplitude 90,000
x5 attenuator IN
t_r 100ns tau_d 50us
- polarity
frequency 25Hz range
delay MIN, tail pulse, INT ref
ASIC settings 2017Mar27-16.12.39
08.59 *online* analysis - ADC & disc data synchronised - see attachment 1
analysis of R1_99 OK - see attachment 2 - deadtime <0.5%
09.11 BigRIPS & BRIKEN DAQ stop
09.13 AIDA RIBF127R1/R1_103
BRIKEN 10
BigRIPS 1010
Fixed Degrader in place 13m Al (8mm + 3mm + 2mm Plates)
Variable degraders
1mm Al - in
1mm Al - in
1mm Al - out
09.30 Detector biases; leakage currents OK - see attachments 3 & 4
FEE64 Temperatures OK - see attachment 5
good events; statistics OK - see attachment 6
No warnings/errors reported by merger since DAQ GO - see attachment 7
*all* system wide checks OK *except* ADC calibration of nnaida1, 21 and nnaida1 clock status (status=6)
09.50 layout 1-8 spectra - see attachments 8-17
Note pulser amplitude drifting for + polarity
*all* spectra zero'd
10.00 BigRIPS & BRIKEN DAQ stop
AIDA RIBF127R1/R1_114
Check ASIC control x2
10.03 AIDA RIBF127R1/R1_115
DAQ start BRIKEN 11
DAQ start BigRIPS 1011
11.12 Changing degraders (16 mm Al plate check?)
E17/F11 ambient temperature +24.7 deg C, RH 38.7%, d.p. +9.7 deg C
Julabo FL11006 set point +20 deg C, actual +20.0 deg C, water level OK (c. 70%)
HEC plot attached - see attach 8
11.17 AIDA RIBF127R1/R1_132
DAQ start BRIKEN 12
DAQ start BigRIPS 1012
Attempted to start new run. Lost beam. BigRIPS re-optimising
11.40 Found beam
Zero'd all histograms
Cleared online sorting histograms
AIDA RIBF127R1/R1_139
DAQ start BRIKEN 13
DAQ start BigRIPS 1013
12.14 Machine is down on AIDA run RIBF127R1/R1_145
HEC plot attached - see attach 9
Implants moved one detector upstream
12.24 Check ASIC Control x2
12.30 Beam back
12.31 AIDA RIBF127R1/R1_149
DAQ start BRIKEN 14
DAQ start BigRIPS 1014
12.35 Detector biases; leakage currents OK - see attachments 22 & 23
*online* analysis - ADC & disc data synchronised
analysis of R1_149 OK - see attachment 24 - deadtime <0.5%
12.39 Beam off RIBF127R1/R1_151
SRC cooling issues
13.55 Background run
AIDA RIBF127R1/R1_170
DAQ start BRIKEN 15, BigRIPS 1015
14.01 Beam still off
14.20 Stop background run
AIDA RIBF127R1/R1_175
15.30 Beam still off
DAQ continues OK
FEE64 temperatures OK - attach 24
Statistics OK - attach 25
Leakage currents OK - attach 26
No merger errors. No nnaida panics from USB console terminal.
|
| Attachment 1: 20.png
|
|
| Attachment 2: R1_99.txt
|
*** TDR format 3.3.0 analyser - TD - October 2016
*** ERROR: READ I/O error: 5002
blocks: 32000
ADC data format: 256830028 ( 1027076.3 Hz)
Other data format: 4257972 ( 17027.8 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 108 ( 0.4 Hz)
RESUME: 108 ( 0.4 Hz)
SYNC100: 2288212 ( 9150.7 Hz)
FEE64 disc: 1921853 ( 7685.6 Hz)
MBS info: 47691 ( 190.7 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 223829 ( 895.1 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
*** Timestamp elapsed time: 250.059 s
FEE elapsed dead time(s) elapsed idle time(s)
1 0.582 0.380
2 0.576 0.532
3 0.000 0.000
4 0.000 0.000
5 0.047 0.000
6 0.020 0.000
7 0.000 0.000
8 0.000 0.000
9 0.135 0.000
10 0.348 0.000
11 0.000 0.000
12 0.000 0.000
13 0.000 0.000
14 0.001 0.000
15 0.000 0.000
16 0.000 0.000
17 0.254 0.000
18 0.045 0.000
19 0.107 0.000
20 0.000 0.000
21 0.000 0.000
22 0.023 0.000
23 0.117 0.000
24 0.068 0.000
25 0.000 0.000
26 0.000 0.000
27 0.000 0.000
28 0.000 0.000
29 0.000 0.000
30 0.000 0.000
31 0.000 0.000
32 0.000 0.000
*** Statistics
FEE ADC Data Other Data Sample Undefined Pause Resume SYNC100 Disc MBS Other HEC Data
0 0 0 0 0 0 0 0 0 0 0 0
1 22632852 177587 0 0 14 14 95165 82394 0 0 16079
2 24293474 169442 0 0 12 12 95164 74254 0 0 13229
3 7417276 107694 0 0 0 0 95376 12318 0 0 211
4 2035272 114874 0 0 0 0 95379 19495 0 0 1114
5 10162120 111937 0 0 13 13 95359 16552 0 0 424
6 10390930 169891 0 0 5 5 95372 26818 47691 0 2658
7 3012146 127459 0 0 0 0 95378 32081 0 0 2911
8 2241979 110413 0 0 0 0 95380 15033 0 0 596
9 17563766 203048 0 0 5 5 95326 107712 0 0 10078
10 19344280 156484 0 0 11 11 95241 61221 0 0 6122
11 4355815 132661 0 0 0 0 95381 37280 0 0 3373
12 5194452 167277 0 0 0 0 95386 71891 0 0 6532
13 8267978 179106 0 0 0 0 95382 83724 0 0 8402
14 9477534 238576 0 0 1 1 95377 143197 0 0 15595
15 7882947 217644 0 0 0 0 95375 122269 0 0 13167
16 7145417 160919 0 0 0 0 95379 65540 0 0 8445
17 12564302 120789 0 0 20 20 95280 25469 0 0 1707
18 16578975 112209 0 0 2 2 95363 16842 0 0 364
19 12528230 205508 0 0 8 8 95337 110155 0 0 10356
20 9291671 269370 0 0 0 0 95376 173994 0 0 13983
21 8697538 208246 0 0 0 0 95381 112865 0 0 23157
22 10491278 243668 0 0 5 5 95374 148284 0 0 28233
23 13296224 308097 0 0 7 7 95333 212750 0 0 19778
24 11963572 245073 0 0 5 5 95348 149715 0 0 17315
25 0 0 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0 0 0
*** Timewarps
FEE ADC Pause Resume SYNC100 Disc MBS Undefined Samples
0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 0
2 0 0 0 0 0 0 0 0
3 0 0 0 0 0 0 0 0
4 0 0 0 0 0 0 0 0
5 0 0 0 0 0 0 0 0
6 0 0 0 0 0 0 0 0
7 0 0 0 0 0 0 0 0
8 0 0 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0
11 0 0 0 0 0 0 0 0
12 0 0 0 0 0 0 0 0
13 0 0 0 0 0 0 0 0
14 0 0 0 0 0 0 0 0
15 0 0 0 0 0 0 0 0
16 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0
*** Program elapsed time: 17.172s ( 1863.512 blocks/s, 116.470 Mb/s)
|
| Attachment 3: 1.png
|
|
| Attachment 4: 2.png
|
|
| Attachment 5: 3.png
|
|
| Attachment 6: 4.png
|
|
| Attachment 7: 5.png
|
|
| Attachment 8: 10.png
|
|
| Attachment 9: 11.png
|
|
| Attachment 10: 12.png
|
|
| Attachment 11: 13.png
|
|
| Attachment 12: 14.png
|
|
| Attachment 13: 15.png
|
|
| Attachment 14: 16.png
|
|
| Attachment 15: 17.png
|
|
| Attachment 16: 18.png
|
|
| Attachment 17: 19.png
|
|
| Attachment 18: 20.png
|
|
| Attachment 19: 21.png
|
|
| Attachment 20: 31.png
|
|
| Attachment 21: 32.png
|
|
| Attachment 22: 22.png
|
|
| Attachment 23: 23.png
|
|
| Attachment 24: R1_149.txt
|
*** TDR format 3.3.0 analyser - TD - October 2016
*** ERROR: READ I/O error: 5002
blocks: 32000
ADC data format: 257668700 ( 1002066.1 Hz)
Other data format: 3419300 ( 13297.6 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 88 ( 0.3 Hz)
RESUME: 87 ( 0.3 Hz)
SYNC100: 2353332 ( 9152.0 Hz)
FEE64 disc: 1034292 ( 4022.3 Hz)
MBS info: 31501 ( 122.5 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 109827 ( 427.1 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
*** Timestamp elapsed time: 257.137 s
FEE elapsed dead time(s) elapsed idle time(s)
1 0.433 0.181
2 0.214 0.107
3 0.000 0.000
4 0.000 0.000
5 0.075 0.000
6 0.011 0.000
7 0.000 0.000
8 0.000 0.000
9 0.079 0.000
10 0.187 0.000
11 0.000 0.000
12 0.000 0.000
13 0.000 0.000
14 0.003 0.000
15 0.000 0.000
16 0.000 0.000
17 0.143 0.000
18 0.172 0.000
19 0.055 0.000
20 0.000 0.000
21 0.000 0.000
22 0.006 0.000
23 0.038 0.000
24 0.039 0.000
25 0.000 0.000
26 0.000 0.000
27 0.000 0.000
28 0.000 0.000
29 0.000 0.000
30 0.000 0.000
31 0.000 0.000
32 0.000 0.000
*** Statistics
FEE ADC Data Other Data Sample Undefined Pause Resume SYNC100 Disc MBS Other HEC Data
0 0 0 0 0 0 0 0 0 0 0 0
1 23227381 160791 0 0 11 11 97918 62851 0 0 11793
2 25090880 150100 0 0 5 5 98001 52089 0 0 8070
3 7603975 108944 0 0 0 0 98083 10861 0 0 69
4 1984568 108068 0 0 0 0 98074 9994 0 0 176
5 10433003 111169 0 0 19 19 98055 13076 0 0 110
6 10351472 140263 0 0 4 4 98077 10677 31501 0 287
7 2921588 108193 0 0 0 0 98081 10112 0 0 237
8 2264804 109110 0 0 0 0 98079 11031 0 0 270
9 17788282 158950 0 0 3 3 98046 60898 0 0 4875
10 19797226 123674 0 0 6 6 98010 25652 0 0 1556
11 4261628 114669 0 0 0 0 98083 16586 0 0 791
12 4992060 129927 0 0 0 0 98084 31843 0 0 2760
13 8026423 122645 0 0 0 0 98079 24566 0 0 1682
14 9355506 164384 0 0 2 2 98070 66310 0 0 6365
15 7580242 154096 0 0 0 0 98081 56015 0 0 6001
16 6933617 121232 0 0 0 0 98075 23157 0 0 1935
17 12772446 108482 0 0 14 14 98021 10433 0 0 139
18 17050877 112187 0 0 7 7 98010 14163 0 0 82
19 12642907 157093 0 0 4 4 98051 59034 0 0 5050
20 9217625 203618 0 0 0 0 98080 105538 0 0 9243
21 8166268 156839 0 0 0 0 98074 58765 0 0 10813
22 10375114 181556 0 0 5 5 98081 83465 0 0 15364
23 13110784 232215 0 0 3 3 98067 134142 0 0 12627
24 11720024 181095 0 0 5 4 98052 83034 0 0 9532
25 0 0 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0 0 0
*** Timewarps
FEE ADC Pause Resume SYNC100 Disc MBS Undefined Samples
0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 0
2 0 0 0 0 0 0 0 0
3 0 0 0 0 0 0 0 0
4 0 0 0 0 0 0 0 0
5 0 0 0 0 0 0 0 0
6 0 0 0 0 0 0 0 0
7 0 0 0 0 0 0 0 0
8 0 0 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0
11 0 0 0 0 0 0 0 0
12 0 0 0 0 0 0 0 0
13 0 0 0 0 0 0 0 0
14 0 0 0 0 0 0 0 0
15 0 0 0 0 0 0 0 0
16 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0
*** Program elapsed time: 23.754s ( 1347.147 blocks/s, 84.197 Mb/s)
|
| Attachment 25: 41.png
|
|
| Attachment 26: 42.png
|
|
| Attachment 27: 43.png
|
|
|
|
706
|
Mon Nov 20 00:53:27 2017 |
TD, CB | Monday 20 November |
09.50 Detector biases & leakage currents OK - see attachments 1 & 2
FEE64 Temperatures OK - see attachment 3
DAQ statistics OK - see attachment 4
7x FEE64s <10k
4x FEE64s 10-20k
5x FEE64s 20-30k
3x FEE64s 30-40k
1x FEE64s 40-50k
4x FEE64 >50k (nnaida13 255k)
PLL & DCM Clock Lock Monitor OK - see attachment 5 & https://elog.ph.ed.ac.uk/AIDA/704 attachment 14
*all* system wide checks OK *except* nnaida7 fails ADC calibration
16.39 Detector biases & leakage currents OK - see attachments 5 & 6
FEE64 Temperatures OK - see attachment 7
DAQ statistics OK - see attachment 8
7x FEE64s <10k
4x FEE64s 10-20k
5x FEE64s 20-30k
3x FEE64s 30-40k
1x FEE64s 40-50k
4x FEE64 >50k (nnaida13 255k)
*all* system wide checks OK *except* nnaida7 fails ADC calibration |
| Attachment 1: 20.png
|
|
| Attachment 2: 22.png
|
|
| Attachment 3: 23.png
|
|
| Attachment 4: 24.png
|
|
| Attachment 5: 25.png
|
|
| Attachment 6: 30.png
|
|
| Attachment 7: 31.png
|
|
| Attachment 8: 32.png
|
|
| Attachment 9: 34.png
|
|
|
|
707
|
Tue Nov 21 00:30:31 2017 |
TD, CB | Tuesday 21 November |
09.27 Detector biases & leakage currents OK - see attachments 1 & 2
FEE64 Temperatures OK - see attachment 3
DAQ statistics OK - see attachment 4
8x FEE64s <10k
2x FEE64s 10-20k
5x FEE64s 20-30k
3x FEE64s 30-40k
1x FEE64s 40-50k
5x FEE64 >50k (nnaida13 255k)
*all* system wide checks OK *except* nnaida7 fails ADC calibration
09.42 per FEE64 rate, 1.8.L, 1.8.W (20ms FSR) spectra - see attachments 5-10
all spectra zero'd
17.05 Detector biases & leakage currents OK - see attachments 11 & 12
FEE64 Temperatures OK - see attachment 13
DAQ statistics OK - see attachment 14
7x FEE64s <10k
4x FEE64s 10-20k
2x FEE64s 20-30k
4x FEE64s 30-40k
2x FEE64s 40-50k
5x FEE64 >50k (nnaida13 261k)
*all* system wide checks OK *except* nnaida7 fails ADC calibration
per FEE64 rate, 1.8.L, 1.8.W (20ms FSR) spectra - see attachments 15-20
pulser peak width nnaida17 69 ch FWHM, nnaida8 72 ch FWHM |
| Attachment 1: 1.png
|
|
| Attachment 2: 2.png
|
|
| Attachment 3: 3.png
|
|
| Attachment 4: 4.png
|
|
| Attachment 5: 7.png
|
|
| Attachment 6: 11.png
|
|
| Attachment 7: 12.png
|
|
| Attachment 8: 13.png
|
|
| Attachment 9: 14.png
|
|
| Attachment 10: 18.png
|
|
| Attachment 11: 1.png
|
|
| Attachment 12: 2.png
|
|
| Attachment 13: 3.png
|
|
| Attachment 14: 4.png
|
|
| Attachment 15: 11.png
|
|
| Attachment 16: 12.png
|
|
| Attachment 17: 13.png
|
|
| Attachment 18: 14.png
|
|
| Attachment 19: 17.png
|
|
| Attachment 20: 18.png
|
|
|
|
708
|
Thu Nov 23 00:41:43 2017 |
TD, CB | Thursday 23 November |
09.33 Detector biases & leakage currents OK - see attachments 1 & 2
FEE64 Temperatures OK - see attachment 3
DAQ statistics OK - see attachment 4
7x FEE64s <10k
4x FEE64s 10-20k
6x FEE64s 20-30k
2x FEE64s 30-40k
0x FEE64s 40-50k
5x FEE64 >50k (nnaida13 280k)
*all* system wide checks OK *except* nnaida7 fails ADC calibration
per FEE64 rate, 1.8.L, 1.8.W (20ms FSR) spectra - see attachments 5-10
pulser peak width nnaida17 70 ch FWHM, nnaida8 72 ch FWHM |
| Attachment 1: 1.png
|
|
| Attachment 2: 2.png
|
|
| Attachment 3: 3.png
|
|
| Attachment 4: 4.png
|
|
| Attachment 5: 11.png
|
|
| Attachment 6: 12.png
|
|
| Attachment 7: 13.png
|
|
| Attachment 8: 14.png
|
|
| Attachment 9: 17.png
|
|
| Attachment 10: 18.png
|
|
|
|
710
|
Thu Nov 23 01:10:43 2017 |
TD, CB | Detector bias & leakage currents - October & November 2017 |
https://docs.google.com/spreadsheets/d/12hgbrywB10hGsKt5uc2HnLfZymh8_uvM6cEASbbqnBE/edit#gid=282765719 |
| Attachment 1: AIDA_leakage_current(16).xlsx
|
|
|
712
|
Thu Nov 23 01:12:33 2017 |
TD, CB | AIDA to safe shutdown state |
10.11 Detector bias OFF
FEE64 power OFF
BNC PB-4 OFF
2x NIM bins OFF - fan tray OFF
Julabo FL11006 chiller OFF - 3ph mains switch OFF
USB ac mains relay OFF - flow interlock enabled - USB disconnected
Raspberry Pis ON
Tektronix AFG3102C ON |
|
|
31
|
Sun Feb 15 06:12:00 2015 |
TD, AEV & CG | How to mount USB disk with NTFS |
Mount external USB expansion disk with NTFS filesystems
with the (superuser) command:
su root
password:
mount -t ntfs-3g /dev/sdd1 /ntfs
[root@aidas1 /]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_aidas1-lv_root
50G 33G 15G 69% /
tmpfs 3.9G 4.3M 3.9G 1% /dev/shm
/dev/sda1 485M 61M 399M 14% /boot
/dev/mapper/vg_aidas1-lv_home
860G 201M 816G 1% /home
/dev/mapper/Data1-data10
1.8T 1.4T 309G 83% /data10
/dev/sdd1 3.7T 1.3T 2.4T 36% /ntfs |
|
|
33
|
Mon Feb 16 06:17:31 2015 |
TD, AEV & CG | Monday 16th Feb progress |
16/02/2015
1500: AIDA moved into operational position.
Water cooling circuit visually checked, fittings, etc for loose fittings/snags/kinks. No problems
Julabo FL11006 chiller moved into position with mains transformer, filled and turned on without problem (small leak from rear overflow, due to overfilling).
Took roughly 2 ~30L water containers to fill reservoir.
Brief leak at connection between back of chiller and yellow hose, bolt tightened and it stopped.
Having been turned on and loop filled, reservoir at ~50% full. Pressure ~10 psi.
No leaks observed in main coolant loop.
When standing downstream of AIDA RH flow gauge blurred due to high rotational speed, LH flow gauge rotating at ~1 rev/s.
Two FEE modules closest to centre in each block checked for power cabling, TS distribution and network cables. All ok.
MACBs front panel cabling checked, no obvious problems. TS distribution tree in order.
Power on for NIM bins and fan tray, network switches and Raspberry Pi. Not yet for the FEE module PSUs.
1 corner of each new USB power supply damaged in transit.
Fuses changed from 10A -> 16A in each PSU. Old 10A fuses taped to back of each module.
1730: Turned on nnaida1-16. Temperatures stable. All on firmware version 0xa4ed006.
Input pulser into all FEE cards. All FEEs bar nnaida5 well-behaving. nnaida12 has several missing channels.
Typical pulser peak width for -ve input modules was ~14ch FWHM with shaping time of 1us.
Typical pulser peak width for +ve input modules was ~27ch FWHM with shaping time of 1us.
Changing shaping time of nnaida10 from 1->2.5us reduced FWHM from 27ch to 23ch.
Attachment 1: temperatures & FEE firmware revisions nnaida1-16
|
| Attachment 1: 1.png
|
|
| Attachment 2: nnaida4_sample.png
|
|
| Attachment 3: nnaida16_sample.png
|
|
| Attachment 4: nnaida5_stats.png
|
|
| Attachment 5: nnaida13_sample.png
|
|
| Attachment 6: nnaida4_params.png
|
|
| Attachment 7: nnaida13_params.png
|
|
| Attachment 8: 2015-02-16_14.52.39.jpg
|
|
| Attachment 9: 2015-02-16_14.53.50.jpg
|
|
| Attachment 10: 2015-02-16_14.54.48.jpg
|
|
|
|
288
|
Tue Jun 7 05:48:13 2016 |
TD, AE, ML & PJW | HEC spectra & implant profile |
DSSSD#1 and DSSSD#6 ADC hit patterns (profile) and 1*H (ASIC#1 20GeV FSR) energy spectra |
| Attachment 1: 30.png
|
|
| Attachment 2: 36.png
|
|
| Attachment 3: 40.png
|
|
|
|
311
|
Sun Jun 12 03:05:37 2016 |
TD, AE, CG, MA | Sunday 12 June 2016 part 2 |
BigRIPS switched from 60Ti to 64V
AIDA48 (AIDA)2034 (EURICA) 2153 (BigRIPS)
start 08.44
stop 09.08 (FEE64 synchronisation lost -> power cycle & reboot FEE64s)
AIDA49 (AIDA) 2036 (EURICA) 2154 (BigRIPS)
Power cycle and reboot FEE64s
FADC power down (per PJCS recommendation)
All FEE64s pass all system wide checks *except* nnaida16 which failed the ADC calibration (not significant
as waveforms are disabled and FADC powered down anyway)
start 09.30
stop 09.58
ASIC settings 2016Jun12-09.09.37
AIDA50 (AIDA) 2037(EURICA) 2155 (BigRIPS)
start 10.02
stop 10.09 (beam lost)
ASIC settings 2016Jun12-10.10.08
changes to nnaida15 discriminator settings to reduce deadtime (pause-resume rate)
AIDA51 (AIDA) 2038 (EURICA) 2156 (BigRIPS)
start 10.45
stop 11.55
ASIC settings 2016Jun12-10.10.08
aidas1> ~/td/GREAT/analyser2 /TapeData/NP1512/AIDA51_0
*** GREAT format 3.2.0 analyser - TD - May 2014
*** ERROR: READ I/O error: 5002
blocks: 32000
ADC data format: 163625493 ( 974305.9 Hz)
Other data format: 97494507 ( 580529.8 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 1227 ( 7.3 Hz)
RESUME: 1227 ( 7.3 Hz)
SYNC100: 64065 ( 381.5 Hz)
FEE64 disc: 96631002 ( 575388.1 Hz)
MBS info: 796986 ( 4745.6 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 213982 ( 1274.2 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
Timestamp elapsed time: 167.941 s
FEE module #: 1 elapsed dead time 0.142 s
FEE module #: 2 elapsed dead time 1.125 s
FEE module #: 3 elapsed dead time 0.000 s
FEE module #: 4 elapsed dead time 0.000 s
FEE module #: 5 elapsed dead time 0.307 s
FEE module #: 6 elapsed dead time 0.579 s
FEE module #: 7 elapsed dead time 0.000 s
FEE module #: 8 elapsed dead time 0.000 s
FEE module #: 9 elapsed dead time 0.082 s
FEE module #: 10 elapsed dead time 0.112 s
FEE module #: 11 elapsed dead time 3.204 s
FEE module #: 12 elapsed dead time 0.005 s
FEE module #: 13 elapsed dead time 0.212 s
FEE module #: 14 elapsed dead time 0.426 s
FEE module #: 15 elapsed dead time 0.000 s
FEE module #: 16 elapsed dead time 0.000 s
FEE module #: 17 elapsed dead time 0.000 s
FEE module #: 18 elapsed dead time 0.046 s
FEE module #: 19 elapsed dead time 0.000 s
FEE module #: 20 elapsed dead time 0.064 s
FEE module #: 21 elapsed dead time 0.319 s
FEE module #: 22 elapsed dead time 0.248 s
FEE module #: 23 elapsed dead time 0.010 s
FEE module #: 24 elapsed dead time 0.190 s
FEE module #: 25 elapsed dead time 0.000 s
FEE module #: 26 elapsed dead time 0.000 s
FEE module #: 27 elapsed dead time 0.000 s
FEE module #: 28 elapsed dead time 0.000 s
FEE module #: 29 elapsed dead time 0.000 s
FEE module #: 30 elapsed dead time 0.000 s
FEE module #: 31 elapsed dead time 0.000 s
FEE module #: 32 elapsed dead time 0.000 s
*** Program elapsed time: 60.988s ( 524.691 blocks/s, 32.793 Mb/s)
See attachments 1-7 for AIDA51 rates etc
See attachment 8 for FEE64 synchronisation check (file AIDA51_0) - OK
AIDA52 (AIDA) 2039 (EURICA) 2158 (BigRIPS)
Start 11.56
Stop 12.54 (AIDA FEE lose sync)
Settings file 2016Jun12-10.10.08
Checked correlation of FEEs in file AIDA52_0 at 12.06 and everything seems ok.
aidas1> ~/td/GREAT/analyser2 /TapeData/NP1512/AIDA52_9
*** GREAT format 3.2.0 analyser - TD - May 2014
*** ERROR: READ I/O error: 5002
blocks: 32000
ADC data format: 163879044 ( 985540.7 Hz)
Other data format: 97240956 ( 584790.6 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 1315 ( 7.9 Hz)
RESUME: 1315 ( 7.9 Hz)
SYNC100: 63433 ( 381.5 Hz)
FEE64 disc: 96378426 ( 579603.4 Hz)
MBS info: 796467 ( 4789.8 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 215707 ( 1297.2 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 1 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
Timestamp elapsed time: 166.283 s
FEE module #: 1 elapsed dead time 0.088 s
FEE module #: 2 elapsed dead time 0.724 s
FEE module #: 3 elapsed dead time 0.000 s
FEE module #: 4 elapsed dead time 0.000 s
FEE module #: 5 elapsed dead time 0.510 s
FEE module #: 6 elapsed dead time 1.337 s
FEE module #: 7 elapsed dead time 0.000 s
FEE module #: 8 elapsed dead time 0.000 s
FEE module #: 9 elapsed dead time 0.087 s
FEE module #: 10 elapsed dead time 0.121 s
FEE module #: 11 elapsed dead time 2.706 s
FEE module #: 12 elapsed dead time 0.000 s
FEE module #: 13 elapsed dead time 0.127 s
FEE module #: 14 elapsed dead time 0.191 s
FEE module #: 15 elapsed dead time 0.000 s
FEE module #: 16 elapsed dead time 0.000 s
FEE module #: 17 elapsed dead time 0.000 s
FEE module #: 18 elapsed dead time 0.058 s
FEE module #: 19 elapsed dead time 0.000 s
FEE module #: 20 elapsed dead time 0.071 s
FEE module #: 21 elapsed dead time 0.399 s
FEE module #: 22 elapsed dead time 0.222 s
FEE module #: 23 elapsed dead time 0.015 s
FEE module #: 24 elapsed dead time 0.061 s
FEE module #: 25 elapsed dead time 0.000 s
FEE module #: 26 elapsed dead time 0.000 s
FEE module #: 27 elapsed dead time 0.000 s
FEE module #: 28 elapsed dead time 0.000 s
FEE module #: 29 elapsed dead time 0.000 s
FEE module #: 30 elapsed dead time 0.000 s
FEE module #: 31 elapsed dead time 0.000 s
FEE module #: 32 elapsed dead time 0.000 s
*** Program elapsed time: 47.215s ( 677.753 blocks/s, 42.360 Mb/s)
12.20 FEE64 temperatures - see attachment 9
Note with all FADC powered off all temperatures now green cf. attachment 8 https://elog.ph.ed.ac.uk/AIDA/305
Julabo FL11006 set point +17 deg C (B3F ambient temperature +25.1 deg C, d.p. +11.2 deg C, RH 40.3%)
13.11 AIDA52_13 per FEE64 ADC timestamp activity OK
12.54 AIDA FEE64 lose synchronisation
No change in system-wide checks
AIDA53 (AIDA) 2040 (EURICA) 2159 (BigRIPS)
Power cycled and rebooted FEE64s
System-wide checks all OK except 6 failures for ADC calibration - not retried
Start 13.09
Stop 13.44 (stop to reset EURICA VME crate)
ASIC Settings file 2016Jun12-10.10.08
13.17 AIDA53_0 FEE64 sync check - OK
aidas1> ~/td/GREAT/analyser2 /TapeData/NP1512/AIDA53_0
*** GREAT format 3.2.0 analyser - TD - May 2014
*** ERROR: READ I/O error: 5002
blocks: 32000
ADC data format: 167386044 ( 1100597.1 Hz)
Other data format: 93733956 ( 616319.7 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 714 ( 4.7 Hz)
RESUME: 714 ( 4.7 Hz)
SYNC100: 58017 ( 381.5 Hz)
FEE64 disc: 92936706 ( 611077.6 Hz)
MBS info: 737805 ( 4851.2 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 166106 ( 1092.2 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 1 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
Timestamp elapsed time: 152.087 s
FEE module #: 1 elapsed dead time 0.020 s
FEE module #: 2 elapsed dead time 0.209 s
FEE module #: 3 elapsed dead time 0.000 s
FEE module #: 4 elapsed dead time 0.000 s
FEE module #: 5 elapsed dead time 0.107 s
FEE module #: 6 elapsed dead time 0.824 s
FEE module #: 7 elapsed dead time 0.000 s
FEE module #: 8 elapsed dead time 0.008 s
FEE module #: 9 elapsed dead time 0.003 s
FEE module #: 10 elapsed dead time 0.000 s
FEE module #: 11 elapsed dead time 0.378 s
FEE module #: 12 elapsed dead time 0.017 s
FEE module #: 13 elapsed dead time 0.000 s
FEE module #: 14 elapsed dead time 0.059 s
FEE module #: 15 elapsed dead time 0.172 s
FEE module #: 16 elapsed dead time 1.101 s
FEE module #: 17 elapsed dead time 0.000 s
FEE module #: 18 elapsed dead time 0.018 s
FEE module #: 19 elapsed dead time 0.000 s
FEE module #: 20 elapsed dead time 0.093 s
FEE module #: 21 elapsed dead time 0.216 s
FEE module #: 22 elapsed dead time 0.161 s
FEE module #: 23 elapsed dead time 0.665 s
FEE module #: 24 elapsed dead time 0.201 s
FEE module #: 25 elapsed dead time 0.000 s
FEE module #: 26 elapsed dead time 0.000 s
FEE module #: 27 elapsed dead time 0.000 s
FEE module #: 28 elapsed dead time 0.000 s
FEE module #: 29 elapsed dead time 0.000 s
FEE module #: 30 elapsed dead time 0.000 s
FEE module #: 31 elapsed dead time 0.000 s
FEE module #: 32 elapsed dead time 0.000 s
*** Program elapsed time: 25.180s ( 1270.866 blocks/s, 79.429 Mb/s)
AIDA54 (AIDA) 2041 (EURICA) 2160 (BigRIPS)
Start 13.46
Stop 13.53 (loss of sync AIDA54_0)
ASIC Settings file 2016Jun12-10.10.08
System-wide checks OK following loss of sync
19:25
Stable operation most of the afternoon through multiple DAQ start/stops.
FEEs remained synchronised throughout.
SYNCs lost in run AIDA57_37 leading to run stop/start.
-> in AIDA57_37 noticed relatively large chunks of data lost in nnaida6. (see attachments 10+11
19.40 Cumulative pause-resume time for AIDA57_37 (see above) and most recent
complete data file. The pause-resume times are relatively large compared to
other FEE64s but low relative to the measurement time. But we should remember
that we have observed periods of inactivity not bounded by pause-resume tags.
aidas1> ./analyser2 /TapeData/NP1512/AIDA57_37
*** GREAT format 3.2.0 analyser - TD - May 2014
*** ERROR: READ I/O error: 5002
blocks: 32000
ADC data format: 164401851 ( 929891.0 Hz)
Other data format: 96718149 ( 547057.9 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 1081 ( 6.1 Hz)
RESUME: 1081 ( 6.1 Hz)
SYNC100: 67440 ( 381.5 Hz)
FEE64 disc: 95842999 ( 542107.9 Hz)
MBS info: 805548 ( 4556.3 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 194112 ( 1097.9 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
Timestamp elapsed time: 176.797 s
FEE module #: 1 elapsed dead time 0.149 s
FEE module #: 2 elapsed dead time 0.741 s
FEE module #: 3 elapsed dead time 0.004 s
FEE module #: 4 elapsed dead time 0.000 s
FEE module #: 5 elapsed dead time 0.425 s
FEE module #: 6 elapsed dead time 1.697 s
FEE module #: 7 elapsed dead time 0.000 s
FEE module #: 8 elapsed dead time 0.000 s
FEE module #: 9 elapsed dead time 0.118 s
FEE module #: 10 elapsed dead time 0.084 s
FEE module #: 11 elapsed dead time 1.064 s
FEE module #: 12 elapsed dead time 0.014 s
FEE module #: 13 elapsed dead time 0.063 s
FEE module #: 14 elapsed dead time 0.267 s
FEE module #: 15 elapsed dead time 0.000 s
FEE module #: 16 elapsed dead time 0.000 s
FEE module #: 17 elapsed dead time 0.000 s
FEE module #: 18 elapsed dead time 0.010 s
FEE module #: 19 elapsed dead time 0.000 s
FEE module #: 20 elapsed dead time 0.059 s
FEE module #: 21 elapsed dead time 0.125 s
FEE module #: 22 elapsed dead time 0.225 s
FEE module #: 23 elapsed dead time 0.000 s
FEE module #: 24 elapsed dead time 0.205 s
FEE module #: 25 elapsed dead time 0.000 s
FEE module #: 26 elapsed dead time 0.000 s
FEE module #: 27 elapsed dead time 0.000 s
FEE module #: 28 elapsed dead time 0.000 s
FEE module #: 29 elapsed dead time 0.000 s
FEE module #: 30 elapsed dead time 0.000 s
FEE module #: 31 elapsed dead time 0.000 s
FEE module #: 32 elapsed dead time 0.000 s
*** Program elapsed time: 23.047s ( 1388.475 blocks/s, 86.780 Mb/s)
aidas1> ./analyser2 /TapeData/NP1512/AIDA58_6
*** GREAT format 3.2.0 analyser - TD - May 2014
*** ADC data: block: 14849 ptr: 14685 data: 0xC40181C9 module: 16 fail: 0 range: 0 id: 1025 channel:
1 adc: 33225 ts: 0x01AB000000A0 OLD: ts: 0x01AB1000003E tag: 3 type: 0
*** ERROR: READ I/O error: 5002
blocks: 32000
ADC data format: 160685457 ( 937303.1 Hz)
Other data format: 100434543 ( 585850.2 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 1282 ( 7.5 Hz)
RESUME: 1282 ( 7.5 Hz)
SYNC100: 65397 ( 381.5 Hz)
FEE64 disc: 99536560 ( 580612.1 Hz)
MBS info: 830022 ( 4841.6 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 213213 ( 1243.7 Hz)
Timewarps: ADC: 1 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
Timestamp elapsed time: 171.434 s
FEE module #: 1 elapsed dead time 0.091 s
FEE module #: 2 elapsed dead time 0.870 s
FEE module #: 3 elapsed dead time 0.000 s
FEE module #: 4 elapsed dead time 0.000 s
FEE module #: 5 elapsed dead time 0.107 s
FEE module #: 6 elapsed dead time 0.518 s
FEE module #: 7 elapsed dead time 0.000 s
FEE module #: 8 elapsed dead time 0.000 s
FEE module #: 9 elapsed dead time 0.058 s
FEE module #: 10 elapsed dead time 0.095 s
FEE module #: 11 elapsed dead time 2.397 s
FEE module #: 12 elapsed dead time 0.055 s
FEE module #: 13 elapsed dead time 0.157 s
FEE module #: 14 elapsed dead time 0.238 s
FEE module #: 15 elapsed dead time 0.000 s
FEE module #: 16 elapsed dead time 0.000 s
FEE module #: 17 elapsed dead time 0.000 s
FEE module #: 18 elapsed dead time 0.013 s
FEE module #: 19 elapsed dead time 0.000 s
FEE module #: 20 elapsed dead time 0.174 s
FEE module #: 21 elapsed dead time 0.298 s
FEE module #: 22 elapsed dead time 0.280 s
FEE module #: 23 elapsed dead time 0.004 s
FEE module #: 24 elapsed dead time 0.071 s
FEE module #: 25 elapsed dead time 0.000 s
FEE module #: 26 elapsed dead time 0.000 s
FEE module #: 27 elapsed dead time 0.000 s
FEE module #: 28 elapsed dead time 0.000 s
FEE module #: 29 elapsed dead time 0.000 s
FEE module #: 30 elapsed dead time 0.000 s
FEE module #: 31 elapsed dead time 0.000 s
FEE module #: 32 elapsed dead time 0.000 s
*** Program elapsed time: 18.234s ( 1754.927 blocks/s, 109.683 Mb/s) |
| Attachment 1: 10.png
|
|
| Attachment 2: 11.png
|
|
| Attachment 3: 12.png
|
|
| Attachment 4: 13.png
|
|
| Attachment 5: 14.png
|
|
| Attachment 6: 15.png
|
|
| Attachment 7: 16.png
|
|
| Attachment 8: 17.png
|
|
| Attachment 9: 20.png
|
|
| Attachment 10: AIDA57_37-pause.png
|
|
| Attachment 11: ADIA57_37.png
|
|
|
|
273
|
Sat Jun 4 04:53:25 2016 |
TD, AE, CG, DK, GL YL & PJW (Team AIDA) | Saturday 4 June 2016 |
12.27 DAQ run R61 started 01.30
Appears to have stopped abruptly at ~06.55
All FEE64 AIDA SYNCs zero
Attempted normal halt to run - DAQ STOP, toggle merge pause, stop tape server - OK
Attempted normal start to run - start tape server, DAQ GO, toggle merge pause - OK
Note during DAQ GO briefly obtained 'unable to connect' error from nnaida1 which resolved
itself a few seconds later
Atachment 1 - AIDA SYNCs
2 - MERGE and server terminal sessions
3 - TapeServer
4 - R61 disk files
5 - Detector biases and leakage currents - OK
Leakage current has decreased since yesterday. B2F/F11 ambient temperature
~25 deg C but EURICA now closed.
6 - AIDA discriminators
7 - AIDA good events
From the MERGE server terminal session ...
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Timestamp (4d087c824ee)[Q 5, Stream 5] is older than previous (4d097c800a0) [Q 5, Stream 5]
Merge: Attempting resynchronisation to timestamp 4d097c800a1 or greater
Merge: Last sync pulse time: 4d097c400a0
Merge: Last timestamp seen: 4d097c800a0
Merge: Restarting at timestamp 4d097cc00a0
Merge: Timestamp (4e122b40d1e)[Q 5, Stream 5] is older than previous (4e132b400a0) [Q 5, Stream 5]
Merge: Attempting resynchronisation to timestamp 4e132b400a1 or greater
Merge: Last sync pulse time: 4e132b000a0
Merge: Last timestamp seen: 4e132b400a0
Merge: Restarting at timestamp 4e132b800a0
Merge: Timestamp (4e1662818a6)[Q 5, Stream 5] is older than previous (4e1762800a0) [Q 5, Stream 5]
Merge: Attempting resynchronisation to timestamp 4e1762800a1 or greater
Merge: Last sync pulse time: 4e1762400a0
Merge: Last timestamp seen: 4e1762800a0
Merge: Restarting at timestamp 4e1762c00a0
Merge: Timestamp (4e1b89803c6)[Q 5, Stream 5] is older than previous (4e1c89800a0) [Q 5, Stream 5]
Merge: Attempting resynchronisation to timestamp 4e1c89800a1 or greater
Merge: Last sync pulse time: 4e1c89400a0
Merge: Last timestamp seen: 4e1c89800a0
Merge: Restarting at timestamp 4e1c89c00a0
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Trying to add items to Q index 7 (Link 7) but queue already full
Merge: Timestamp (4f2580400c6)[Q 5, Stream 5] is older than previous (4f2680400a0) [Q 5, Stream 5]
Merge: Attempting resynchronisation to timestamp 4f2680400a1 or greater
Merge: Last sync pulse time: 4f2680000a0
Merge: Last timestamp seen: 4f2680400a0
Merge: Restarting at timestamp 4f2680800a0
Merge: Timestamp (5026a041c06)[Q 5, Stream 5] is older than previous (5027a0400a0) [Q 5, Stream 5]
Merge: Attempting resynchronisation to timestamp 5027a0400a1 or greater
Merge: Last sync pulse time: 5027a0000a0
Merge: Last timestamp seen: 5027a0400a0
Merge: Restarting at timestamp 5027a0800a0
Merge: Timestamp (5083c7008c6)[Q 5, Stream 5] is older than previous (5084c7000a0) [Q 5, Stream 5]
Merge: Attempting resynchronisation to timestamp 5084c7000a1 or greater
Merge: Last sync pulse time: 5084c6c00a0
Merge: Last timestamp seen: 5084c7000a0
Merge: Restarting at timestamp 5084c7400a0
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Trying to add items to Q index 2 (Link 2) but queue already full
Merge: Timestamp (523c5c80b4e)[Q 5, Stream 5] is older than previous (523d5c800a0) [Q 5, Stream 5]
Merge: Attempting resynchronisation to timestamp 523d5c800a1 or greater
Merge: Last sync pulse time: 523d5c400a0
Merge: Last timestamp seen: 523d5c800a0
Merge: Restarting at timestamp 523d5cc00a0
Merge: Timestamp (527ddc03916)[Q 5, Stream 5] is older than previous (527edc000a0) [Q 5, Stream 5]
Merge: Attempting resynchronisation to timestamp 527edc000a1 or greater
Merge: Last sync pulse time: 527edbc00a0
Merge: Last timestamp seen: 527edc000a0
Merge: Restarting at timestamp 527edc400a0
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Trying to add items to Q index 4 (Link 4) but queue already full
Merge: Timestamp (52e9c2c0456)[Q 5, Stream 5] is older than previous (52eac2c00a0) [Q 5, Stream 5]
Merge: Attempting resynchronisation to timestamp 52eac2c00a1 or greater
Merge: Last sync pulse time: 52eac2800a0
Merge: Last timestamp seen: 52eac2c00a0
Merge: Restarting at timestamp 52eac3000a0
Queue full messages do not appear to be correlated (currently at least) with the busiest FEE64s
12.50 AIDA trigger to EURICA OR(DSSSD#1 - #5) .AND.NOT. DSSSD#6 ~1.8kHz
.AND.EURICA ~50Hz (no significant change in rate when DSSSD#6 veto removed)
15.30 DAQ found stopped - AIDA SYNCs zero - data transfer disabled, histogramming enabled
System wide checks
nnaida19 fails 'ASIC clock timestamp' and 'SYNC errors not counting tests'
all other FEE64s pass all tests |
| Attachment 1: 21.png
|
|
| Attachment 2: 22.png
|
|
| Attachment 3: 23.png
|
|
| Attachment 4: 24.png
|
|
| Attachment 5: 25.png
|
|
| Attachment 6: 26.png
|
|
| Attachment 7: 27.png
|
|
| Attachment 8: 28.png
|
|