| ID |
Date |
Author |
Subject |
|
404
|
Thu Oct 27 10:57:49 2016 |
TD | AIDA data readout |
Developing real-time AIDA data synchronisation test using MIDASsort
To illustrate AIDA readout, ADC and discriminator data from a pulser event is plotted as a function
of the 100MHz AIDA internal timestamp (1us/channel)
Attachment 1 - nnaida8 number of ADC and discriminator data items
Attachment 2 - nnaida1-24 number of ADC and discriminator data items
Attachment 3 - 4*(FEE64 module # - 1) + ASIC # (ADC channels 0-95, disc channels 100-195) versus timestamp
(1us/channel)
The spectra clearly illustrate
- 500kHz ADC readout cycle
- discriminator readout precedes ADC readout by ~18us (this figure will be shaping time dependent)
- after-pulse due to pulser injection via simple charge terminator (no pole zero correction)
~84us after start of ADC readout (this figure will be pulser decay time and shaping time dependent)
Note events were identified (delimited) by time periods > 2us between successive ADC data items |
| Attachment 1: 100.png
|
|
| Attachment 2: 101.png
|
|
| Attachment 3: 102.png
|
|
|
403
|
Thu Oct 27 10:54:14 2016 |
TD | Thursday 27 October |
14.20 DSSSD bias & leakage currents OK - see attachments 1 & 2
FEE64 temperatures OK - see attachment 3
14.35 DAQ restarted for test purposes
18.55 DAQ stalled as expected
Most recent file R83_125 last accessed 18.30
DAQ STOP OK
DAQ RESET & SETUP OK
ReSYNC OK
All system-wide checks OK
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
20.48 DAQ starts (R86)
LEC/MEC fast comparator 0xff -> 0x20 |
| Attachment 1: 1.png
|
|
| Attachment 2: 2.png
|
|
| Attachment 3: 3.png
|
|
|
402
|
Tue Oct 25 02:06:34 2016 |
TD | Tuesday 25 October |
10.10 DAQ found stopped
Most recent file R81_169, last accessed 06.26
R81_169 ADC data synchronised
aidas1> ./analyser /TapeData/Sept2016/R81_169
*** TDR format 3.3.0 analyser - TD - October 2016
*** ERROR: READ I/O error: 5002
blocks: 8369
ADC data format: 67712004 ( 1105598.7 Hz)
Other data format: 570667 ( 9317.8 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 17 ( 0.3 Hz)
RESUME: 17 ( 0.3 Hz)
SYNC100: 560442 ( 9150.9 Hz)
FEE64 disc: 8352 ( 136.4 Hz)
MBS info: 1839 ( 30.0 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 81 ( 1.3 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
*** Timestamp elapsed time: 61.245 s
FEE elapsed dead time(s) elapsed idle time(s)
1 0.000 0.000
2 0.000 0.000
3 0.000 0.000
4 0.000 0.000
5 0.244 0.254
6 0.181 0.191
7 0.000 0.000
8 0.000 0.000
9 0.000 0.000
10 0.000 0.000
11 0.000 0.000
12 0.000 0.000
13 0.000 0.000
14 0.000 0.000
15 0.002 0.000
16 0.000 0.000
17 0.001 0.000
18 0.002 0.000
19 0.003 0.000
20 0.000 0.000
21 0.000 0.000
22 0.000 0.000
23 0.085 0.000
24 0.000 0.000
25 0.000 0.000
26 0.000 0.000
27 0.000 0.000
28 0.000 0.000
29 0.000 0.000
30 0.000 0.000
31 0.000 0.000
32 0.000 0.000
*** Statistics
FEE ADC Data Other Data Sample Undefined Pause Resume SYNC100 Disc MBS
Other HEC Data
0 0 0 0 0 0 0 0 0 0
0 0
1 229453 23628 0 0 0 0 23359 269 0
0 22
2 2464367 23972 0 0 0 0 23362 610 0
0 0
3 1700233 23656 0 0 0 0 23358 298 0
0 0
4 650519 23359 0 0 0 0 23359 0 0
0 0
5 15435615 25773 0 0 4 4 23267 2498 0
0 0
6 15768889 25301 0 0 3 3 23291 165 1839
0 0
7 963142 23361 0 0 0 0 23361 0 0
0 0
8 773768 23661 0 0 0 0 23361 300 0
0 0
9 2244651 23523 0 0 0 0 23359 164 0
0 0
10 1833324 24095 0 0 0 0 23362 733 0
0 0
11 276357 23657 0 0 0 0 23362 295 0
0 0
12 542691 23359 0 0 0 0 23359 0 0
0 0
13 1695447 23996 0 0 0 0 23363 633 0
0 0
14 2181674 23530 0 0 0 0 23361 169 0
0 0
15 2203091 23362 0 0 1 1 23360 0 0
0 0
16 2179412 23361 0 0 0 0 23361 0 0
0 0
17 2317043 23549 0 0 1 1 23359 188 0
0 0
18 2579367 23969 0 0 1 1 23356 611 0
0 0
19 2589829 23664 0 0 3 3 23358 300 0
0 0
20 1296289 23357 0 0 0 0 23357 0 0
0 0
21 1624570 24024 0 0 0 0 23361 663 0
0 59
22 513131 23525 0 0 0 0 23360 165 0
0 0
23 3691921 23335 0 0 4 4 23327 0 0
0 0
24 1957221 23650 0 0 0 0 23359 291 0
0 0
25 0 0 0 0 0 0 0 0 0
0 0
26 0 0 0 0 0 0 0 0 0
0 0
27 0 0 0 0 0 0 0 0 0
0 0
28 0 0 0 0 0 0 0 0 0
0 0
29 0 0 0 0 0 0 0 0 0
0 0
30 0 0 0 0 0 0 0 0 0
0 0
31 0 0 0 0 0 0 0 0 0
0 0
32 0 0 0 0 0 0 0 0 0
0 0
*** Timewarps
FEE ADC Pause Resume SYNC100 Disc MBS Undefined Samples
0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 0
2 0 0 0 0 0 0 0 0
3 0 0 0 0 0 0 0 0
4 0 0 0 0 0 0 0 0
5 0 0 0 0 0 0 0 0
6 0 0 0 0 0 0 0 0
7 0 0 0 0 0 0 0 0
8 0 0 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0
11 0 0 0 0 0 0 0 0
12 0 0 0 0 0 0 0 0
13 0 0 0 0 0 0 0 0
14 0 0 0 0 0 0 0 0
15 0 0 0 0 0 0 0 0
16 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0
*** Program elapsed time: 0.957s ( 8744.751 blocks/s, 546.547 Mb/s)
DSSSD bias & leakage currents OK - see attachments 1 & 2
FEE64 temperatures OK - see attachment 3
System-wide checks OK *except* SYNC errors & SYNC pulses - see attachments 4-7
Merger error messages - see attachment 8
Restart merger OK
DAQ STOP OK
DAQ RESET & SETUP OK
ReSYNC OK
All system-wide checks OK
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
10.22 DAQ starts (R82)
16.07 DAQ continues OK
All system-wide checks OK
17.16 DAQ continues OK
All system-wide checks OK
18.27 All system wide checks OK *except* SYNC pulses - see attachment 9
18.34 DAQ stalled
Most recent file R82_125 last accessed 18.30
All system wide checks OK *except* SYNC errors and SYNC pulses - see attachments 10-13
18.47 AIDA -> safe state
FEE64 power OFF
DSSSD bias OFF |
| Attachment 1: 1.png
|
|
| Attachment 2: 2.png
|
|
| Attachment 3: 3.png
|
|
| Attachment 4: 4.png
|
|
| Attachment 5: 5.png
|
|
| Attachment 6: 6.png
|
|
| Attachment 7: 7.png
|
|
| Attachment 8: 8.png
|
|
| Attachment 9: 10.png
|
|
| Attachment 10: 11.png
|
|
| Attachment 11: 12.png
|
|
| Attachment 12: 13.png
|
|
| Attachment 13: 14.png
|
|
|
401
|
Mon Oct 24 02:46:57 2016 |
TD | Monday 24 October |
10.47 DAQ found stopped, no merger error messages
Most recent file R78_281, last accessed 06.46
R78_281 ADC data synchronised
DSSSD bias & leakage currents OK - see attachments 1 & 2
FEE64 temperatures OK - see attachment 3
System-wide checks - see attachments 4-11
Restart merger OK
DAQ STOP OK
DAQ RESET & SETUP OK
ReSYNC OK
System-wide checks OK
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
11.07 DAQ starts (R79)
18.39 DAQ OK
System wide checks OK *except* SYNC errors & SYNC pulses - see attachments 12-15
18.47 DAQ stalls
Most recent file R79_182, last accessed 18.46
System wide checks OK *except* SYNC errors & SYNC pulses - see attachments 16-19
18.56 DAQ starts (R80) |
| Attachment 1: 1.png
|
|
| Attachment 2: 2.png
|
|
| Attachment 3: 3.png
|
|
| Attachment 4: 4.png
|
|
| Attachment 5: 5.png
|
|
| Attachment 6: 6.png
|
|
| Attachment 7: 7.png
|
|
| Attachment 8: 8.png
|
|
| Attachment 9: 9.png
|
|
| Attachment 10: 10.png
|
|
| Attachment 11: 11.png
|
|
| Attachment 12: 20.png
|
|
| Attachment 13: 21.png
|
|
| Attachment 14: 22.png
|
|
| Attachment 15: 23.png
|
|
| Attachment 16: 24.png
|
|
| Attachment 17: 25.png
|
|
| Attachment 18: 26.png
|
|
| Attachment 19: 27.png
|
|
|
400
|
Sun Oct 23 10:22:36 2016 |
TD | Report - high - merger SEGV |
Merger crashed during operation - see https://elog.ph.ed.ac.uk/AIDA/399 |
|
399
|
Sun Oct 23 02:34:12 2016 |
TD | Sunday 23 October |
10.34 DAQ found stopped, no merger error messages
Most recent file R74_296, last accessed 06.31
R70_315 ADC data synchronised
DSSSD bias & leakage currents OK - see attachments 1 & 2
FEE64 temperatures OK - see attachment 3
Some merger errors - see attachment 4
Restart merger OK
DAQ STOP OK
DAQ RESET & SETUP OK
ReSYNC OK
System-wide checks OK
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
10.42 DAQ start (R75)
nnaida11 drops data transfer #1 tag at SAQ GO
DAQ STOP OK
DAQ RESET & SETUP OK
ReSYNC OK
System-wide checks OK
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
10.46 DAQ start (R76)
18.18 DAQ found stopped
Most recent file R76_29 last accessed 11.52
Merger crashed - see attachment 5
aidas1> ./analyser /TapeData//Sept2016/R76_29
*** GREAT format 3.2.2 analyser - TD - October 2016
*** ERROR: READ I/O error: 5002
blocks: 24384
ADC data format: 197754460 ( 1539809.2 Hz)
Other data format: 1194596 ( 9301.7 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 74 ( 0.6 Hz)
RESUME: 74 ( 0.6 Hz)
SYNC100: 1172671 ( 9131.0 Hz)
FEE64 disc: 18000 ( 140.2 Hz)
MBS info: 3777 ( 29.4 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 190 ( 1.5 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
*** Timestamp elapsed time: 128.428 s
FEE elapsed dead time(s)
1 0.000
2 0.007
3 0.000
4 0.000
5 0.674
6 2.583
7 0.000
8 0.000
9 0.000
10 0.000
11 0.000
12 0.000
13 0.000
14 0.000
15 0.276
16 0.124
17 0.435
18 0.050
19 0.622
20 0.009
21 0.000
22 0.000
23 0.352
24 0.000
25 0.000
26 0.000
27 0.000
28 0.000
29 0.000
30 0.000
31 0.000
32 0.000
*** Statistics
FEE ADC Data Other Data Sample Undefined Pause Resume SYNC100 Disc MBS
Other HEC Data
0 0 0 0 0 0 0 0 0 0
0 0
1 566974 49807 0 0 0 0 48983 824 0
0 62
2 5617753 50240 0 0 2 2 48984 1252 0
0 0
3 4298949 49615 0 0 0 0 48983 632 0
0 0
4 1980475 48985 0 0 0 0 48985 0 0
0 0
5 31381665 53750 0 0 12 12 48726 5000 0
0 0
6 35397222 52131 0 0 12 12 48004 326 3777
0 0
7 3679182 48987 0 0 0 0 48987 0 0
0 0
8 3893833 49613 0 0 0 0 48983 630 0
0 0
9 4909194 49313 0 0 0 0 48987 326 0
0 0
10 4099868 50957 0 0 0 0 48984 1973 0
0 0
11 1590517 49623 0 0 0 0 48987 636 0
0 0
12 2103665 48985 0 0 0 0 48985 0 0
0 0
13 3909540 50385 0 0 0 0 48990 1395 0
0 0
14 5011678 49344 0 0 0 0 48988 356 0
0 0
15 8074527 48904 0 0 10 10 48883 1 0
0 2
16 8822855 48949 0 0 5 5 48939 0 0
0 0
17 11581998 49241 0 0 10 10 48820 401 0
0 0
18 17158740 50227 0 0 1 1 48968 1257 0
0 0
19 14684270 49399 0 0 11 11 48747 630 0
0 0
20 6793305 47946 0 0 1 1 47944 0 0
0 0
21 4052765 50377 0 0 0 0 48986 1391 0
0 126
22 1773964 49339 0 0 0 0 48988 351 0
0 0
23 10790840 48872 0 0 10 10 48852 0 0
0 0
24 5580681 49607 0 0 0 0 48988 619 0
0 0
25 0 0 0 0 0 0 0 0 0
0 0
26 0 0 0 0 0 0 0 0 0
0 0
27 0 0 0 0 0 0 0 0 0
0 0
28 0 0 0 0 0 0 0 0 0
0 0
29 0 0 0 0 0 0 0 0 0
0 0
30 0 0 0 0 0 0 0 0 0
0 0
31 0 0 0 0 0 0 0 0 0
0 0
32 0 0 0 0 0 0 0 0 0
0 0
*** Timewarps
FEE ADC Pause Resume SYNC100 Disc MBS Undef-d Samples
0 0 0 0 0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 0 0 0 0 0
2 0 0 0 0 0 0 0 0 0 0 0 0
3 0 0 0 0 0 0 0 0 0 0 0 0
4 0 0 0 0 0 0 0 0 0 0 0 0
5 0 0 0 0 0 0 0 0 0 0 0 0
6 0 0 0 0 0 0 0 0 0 0 0 0
7 0 0 0 0 0 0 0 0 0 0 0 0
8 0 0 0 0 0 0 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0 0 0 0 0
11 0 0 0 0 0 0 0 0 0 0 0 0
12 0 0 0 0 0 0 0 0 0 0 0 0
13 0 0 0 0 0 0 0 0 0 0 0 0
14 0 0 0 0 0 0 0 0 0 0 0 0
15 0 0 0 0 0 0 0 0 0 0 0 0
16 0 0 0 0 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0 0 0 0
*** Program elapsed time: 3.586s ( 6799.896 blocks/s, 424.993 Mb/s)
Restart merger OK
DAQ STOP OK
DAQ RESET & SETUP OK
ReSYNC OK
System-wide checks OK
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
18.50 DAQ starts (R77)
19.17 DAQ stalled due to disk housekeeping
Restart merger OK
DAQ STOP fails - nnaida1 timeout
Power cycle FEE64s
19.47 DAQ starts (R78) |
| Attachment 1: 1.png
|
|
| Attachment 2: 2.png
|
|
| Attachment 3: 3.png
|
|
| Attachment 4: 4.png
|
|
| Attachment 5: 5.png
|
|
|
398
|
Sat Oct 22 02:32:37 2016 |
TD | Saturday 22 October |
10.32 DAQ found stopped, no merger error messages
Most recent file R70_315, last accessed 06.46
R70_315 ADC data synchronised
DSSSD bias & leakage currents OK - see attachments 1 & 2
FEE64 temperatures OK - see attachment 3
System-wide checks - SYNC errors & SYNC pulses fail - other checks OK - see attachments 4-7
AIDA SYNC stats - see attachment 8
Restart merger OK
DAQ STOP OK
DAQ RESET & SETUP OK
ReSYNC OK
System-wide checks OK
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
10.42 DAQ starts (R71)
18.25 DAQ stop OK
System wide test all OK *except* SYNC errors - see attachments 8-11
New SYNC error baseline
18.54 Repeat system wide checks whilst DAQ is still halted - SYNC errors *and* SYNC pulse now fail - see
attachments 12-15
This may suggest that the clock signal itself is affected by LN2 fill cycle
DAQ RESET & SETUP OK
ReSYNC OK
System-wide checks OK
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
18.59 DAQ starts (R72)
nnaida15 drops data transfer #1 tag at DAQ GO
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
19.07 DAQ starts (R73)
19.17 DAQ stalled
Restart merger OK
DAQ STOP OK
Syten wide checks - SYNC error and SYNC pulse fail - dissimilar FEE64 counts in SYNC pulse
DAQ RESET & SETUP OK
ReSYNC OK
System-wide checks OK
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
19.37 DAQ starts (R74)
|
| Attachment 1: 1.png
|
|
| Attachment 2: 2.png
|
|
| Attachment 3: 3.png
|
|
| Attachment 4: 4.png
|
|
| Attachment 5: 5.png
|
|
| Attachment 6: 6.png
|
|
| Attachment 7: 7.png
|
|
| Attachment 8: 10.png
|
|
| Attachment 9: 11.png
|
|
| Attachment 10: 12.png
|
|
| Attachment 11: 13.png
|
|
| Attachment 12: 20.png
|
|
| Attachment 13: 21.png
|
|
| Attachment 14: 22.png
|
|
| Attachment 15: 23.png
|
|
|
397
|
Fri Oct 21 01:41:54 2016 |
TD | Friday 21 October |
09.38 DAQ (R66) continues OK
Detector bias and leakage currents OK - see attachments 1&2
FEE64 temperatures OK - see attachment 3
System wide checks SYNC count & SYNC errors fail - see attachments 4-7 - other checks OK
R66_367 ADC data synchronised
AIDA SYNC stats - attachment 8
Some merger errors - see attachment 9
13.53 DAQ found stalled
Most recent files R66_396 last accessed 10.55
System wide checks SYNC count & SYNC errors fail - see attachments 10-13 - other checks OK
AIDA SYNC stats - attachment 14
No further merger error messages
Restart merger OK
DAQ STOP OK
DAQ RESET & SETUP OK
ReSYNC OK
System-wide checks OK
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
14.02 DAQ starts (R67)
18.27 DAQ found stalled
Most recent file R67_121 last accessed 18.26
Restart merger
DAQ STOP
System wide checks
DAQ RESET & SETUP
System-wide checks
18.39 DAQ start (R68)
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
18.44 DAQ stalled
Most recent file R68_1 last accessed 18.44
Restart merger
DAQ STOP
System wide checks
DAQ RESET & SETUP
System-wide checks
18.53 DAQ starts (R70) |
| Attachment 1: 1.png
|
|
| Attachment 2: 2.png
|
|
| Attachment 3: 3.png
|
|
| Attachment 4: 4.png
|
|
| Attachment 5: 5.png
|
|
| Attachment 6: 6.png
|
|
| Attachment 7: 7.png
|
|
| Attachment 8: 8.png
|
|
| Attachment 9: 9.png
|
|
| Attachment 10: 10.png
|
|
| Attachment 11: 11.png
|
|
| Attachment 12: 12.png
|
|
| Attachment 13: 13.png
|
|
| Attachment 14: 14.png
|
|
|
396
|
Thu Oct 20 20:00:14 2016 |
CG | Returned module tests @ DL |
Original system configuration before my arrival: nnaida11 - 2b:22:6e
nnaida12 - 2b:33:15
nnaida13 - 2b:09:ce
All on firmware version 0x1900c1f.
With 8us shaping time and pulser amplitude @ 5V, FWHM [ch] were:
Ch | nnaida11 | nnaida12 | nnaida13 | nnaida14
1.4.L | 38 | 33 | 38 | 34
1.6.L | 24 | 25 | 28 | 23
1.8.L | 20 | 17 | 17 | 16
1.10.L | 21 | 17 | 17 | 16
Slightly higher than expected for a few channels, but FWHM fall within ~16-40ch for all channels in all FEEs.
All system checks passed.
nnaida13/14 kept constant throughout as control.
(1) Swapped nnaida11/12: nnaida11 -> 2b:33:0c nnaida12 -> 2a:d0:1a
Initially failed SYNC error test - consequence of old firmware version.
Updated firmware from 0x10300c12 to 0x1900c1f.
All system tests passed.
FWHM on 8us shaping time and same pulser settings.
Ch | nnaida11 | nnaida12 | nnaida13 | nnaida14
1.4.L | 37 | 40 | 42 | 35
1.6.L | 31 | 28 | 29 | 23
1.8.L | 22 | 18 | 15 | 15
1.10.L | 23 | 20 | 16 | 15
(2) Swapped nnaida11/12: nnaida11 -> 2a:b8:90 nnaida12 -> 2a:b6:45
Updated firmware from 0x10300c12 to 0x1900c1f.
All system checks passed.
No ASIC temp in nnaida11.
FWHM on 8us shaping time and same pulser settings.
Ch | nnaida11 | nnaida12 | nnaida13 | nnaida14
1.4.L | 40 | 32 | 40 | 34
1.6.L | 29 | 25 | 29 | 23
1.8.L | 21 | 18 | 17 | 16
1.10.L | 21 | 18 | 17 | 16
(3) Swapped nnaida11/12: nnaida11 -> 2a:ed:9a nnaida12 -> 2b:11:c5
Updated firmware from 0x10300c12 to 0x1900c1f.
All system checks passed.
No ASIC temp in nnaida12.
FWHM on 8us shaping time and same pulser settings.
Ch | nnaida11 | nnaida12 | nnaida13 | nnaida14
1.4.L | 44 | 36 | 40 | 35
1.6.L | 30 | 27 | 29 | 24
1.8.L | 22 | 18 | 16 | 17
1.10.L | 23 | 18 | 17 | 16
The above were all the repaired modules and seem to be operating normally.
Following are the 2 modules for which no fault report was provided. I only had time to complete testing on one, the other remains unknown.
(4) Swapped nnaida11/12: nnaida11 -> 2a:ed:8f nnaida12 -> 2b:09:da
Updated firmware from 0x10300c12 to 0x1900c1f.
nnaida11 failed clock status tests. Tried 'Perform resync' from Master Timestamp window with no effect.
No SYNC errors reported. All other system tests passed.
nnaida12 missing lots of channels but all system checks passed.
FWHM on 8us shaping time and same pulser settings.
Ch | nnaida11 | nnaida12 | nnaida13 | nnaida14
1.4.L | 46 | 46 | 41 | 36
1.6.L | 31 | 26 | 28 | 24
1.8.L | 22 | 19 | 17 | 16
1.10.L | 22 | 20 | 17 | 16
**
Overall, the first three modules which have been repaired seem to be operating normally. The fourth seems not be and the last module remains unknown.
Ch 1.4.L displays greater than expected FWHM throughout in all nnaida11/12, but FWHM were largely constant (to within a few channels) throughout and all FWHM fell in the range ~15-40ch apart from 1.1.L in *all* nnaida. |
| Attachment 1: test1_hitPatterns.png
|
|
| Attachment 2: test1_updatedFirmware.png
|
|
| Attachment 3: test2_hitPattern.png
|
|
| Attachment 4: test2_hitPattern_updatedFirmware.png
|
|
| Attachment 5: test3_hitPattern.png
|
|
| Attachment 6: test4_hitPat.png
|
|
| Attachment 7: test4_clockStatusError.png
|
|
|
395
|
Thu Oct 20 01:30:10 2016 |
TD | Thursday 20 October |
09.30 DAQ found stopped, no merger error messages
Most recent file R63_291, last accessed 06.46
R63_291 ADC data synchronised
aidas1> ./analyser2 /TapeData/Sept2016/R63_291
*** GREAT format 3.2.0 analyser - TD - May 2014
*** ERROR: READ I/O error: 5002
blocks: 29203
ADC data format: 237170622 ( 2006571.1 Hz)
Other data format: 1096655 ( 9278.2 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 113 ( 1.0 Hz)
RESUME: 113 ( 1.0 Hz)
SYNC100: 1077835 ( 9119.0 Hz)
FEE64 disc: 15061 ( 127.4 Hz)
MBS info: 3533 ( 29.9 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 175 ( 1.5 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
Timestamp elapsed time: 118.197 s
FEE module #: 1 elapsed dead time 0.000 s
FEE module #: 2 elapsed dead time 0.000 s
FEE module #: 3 elapsed dead time 0.000 s
FEE module #: 4 elapsed dead time 0.000 s
FEE module #: 5 elapsed dead time 2.763 s
FEE module #: 6 elapsed dead time 0.275 s
FEE module #: 7 elapsed dead time 0.004 s
FEE module #: 8 elapsed dead time 0.003 s
FEE module #: 9 elapsed dead time 0.001 s
FEE module #: 10 elapsed dead time 0.000 s
FEE module #: 11 elapsed dead time 0.000 s
FEE module #: 12 elapsed dead time 0.000 s
FEE module #: 13 elapsed dead time 0.000 s
FEE module #: 14 elapsed dead time 0.013 s
FEE module #: 15 elapsed dead time 0.028 s
FEE module #: 16 elapsed dead time 0.275 s
FEE module #: 17 elapsed dead time 6.010 s
FEE module #: 18 elapsed dead time 0.382 s
FEE module #: 19 elapsed dead time 0.555 s
FEE module #: 20 elapsed dead time 0.378 s
FEE module #: 21 elapsed dead time 0.000 s
FEE module #: 22 elapsed dead time 0.035 s
FEE module #: 23 elapsed dead time 0.168 s
FEE module #: 24 elapsed dead time 0.043 s
FEE module #: 25 elapsed dead time 0.000 s
FEE module #: 26 elapsed dead time 0.000 s
FEE module #: 27 elapsed dead time 0.000 s
FEE module #: 28 elapsed dead time 0.000 s
FEE module #: 29 elapsed dead time 0.000 s
FEE module #: 30 elapsed dead time 0.000 s
FEE module #: 31 elapsed dead time 0.000 s
FEE module #: 32 elapsed dead time 0.000 s
*** Program elapsed time: 28.188s ( 1036.027 blocks/s, 64.752 Mb/s)
DSSSD bias & leakage currents OK - see attachments 1 & 2
FEE64 temperatures OK - see attachment 3
System-wide checks - SYNC errors & SYNC pulses fail - other checks OK - see attachments 4-7
AIDA SYNC stats - see attachment 8
Restart merger OK
DAQ STOP OK
DAQ RESET & SETUP OK
ReSYNC OK
System-wide checks OK
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
09.43 DAQ starts (R64)
18.25 DAQ stop OK
Stop before 'scheduled' 18.30 DAQ stall
System wide checks - SYNC count & SYNC errors fail - other checks OK - see attachments 8-11
18.49 DAQ start (R65)
N.B. normal DAQ start - no DAQ RESET, SETUP, restart of merger & tapeserver etc
18.53 SYNC error counter baseline
19.21 DAQ found stalled
Most recent files R65_12 last access 19.17
Soft reset failed - nnaida4 connect timeouts - power cycle FEE64s
20.02 DAQQ start (R66) |
| Attachment 1: 1.png
|
|
| Attachment 2: 2.png
|
|
| Attachment 3: 3.png
|
|
| Attachment 4: 4.png
|
|
| Attachment 5: 5.png
|
|
| Attachment 6: 6.png
|
|
| Attachment 7: 7.png
|
|
| Attachment 8: 8.png
|
|
| Attachment 9: 10.png
|
|
| Attachment 10: 11.png
|
|
| Attachment 11: 12.png
|
|
| Attachment 12: 13.png
|
|
|
394
|
Wed Oct 19 01:14:10 2016 |
TD | Wednesday 19 October |
09.13 DAQ found stopped, no merger error messages
Most recent file R55_10, last accessed 19.17
Restart merger OK
DAQ STOP OK
aidas1> ./analyser2 /TapeData/Sept2016/R55_10
*** GREAT format 3.2.0 analyser - TD - May 2014
*** ERROR: READ I/O error: 5002
blocks: 23037
ADC data format: 187091367 ( 2006959.5 Hz)
Other data format: 867516 ( 9306.0 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 62 ( 0.7 Hz)
RESUME: 62 ( 0.7 Hz)
SYNC100: 851796 ( 9137.4 Hz)
FEE64 disc: 12836 ( 137.7 Hz)
MBS info: 2760 ( 29.6 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 124 ( 1.3 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
Timestamp elapsed time: 93.221 s
FEE module #: 1 elapsed dead time 0.000 s
FEE module #: 2 elapsed dead time 0.000 s
FEE module #: 3 elapsed dead time 0.000 s
FEE module #: 4 elapsed dead time 0.000 s
FEE module #: 5 elapsed dead time 0.922 s
FEE module #: 6 elapsed dead time 1.113 s
FEE module #: 7 elapsed dead time 0.017 s
FEE module #: 8 elapsed dead time 0.000 s
FEE module #: 9 elapsed dead time 0.001 s
FEE module #: 10 elapsed dead time 0.000 s
FEE module #: 11 elapsed dead time 0.000 s
FEE module #: 12 elapsed dead time 0.000 s
FEE module #: 13 elapsed dead time 0.000 s
FEE module #: 14 elapsed dead time 0.005 s
FEE module #: 15 elapsed dead time 0.016 s
FEE module #: 16 elapsed dead time 0.000 s
FEE module #: 17 elapsed dead time 0.446 s
FEE module #: 18 elapsed dead time 0.435 s
FEE module #: 19 elapsed dead time 0.925 s
FEE module #: 20 elapsed dead time 0.015 s
FEE module #: 21 elapsed dead time 0.000 s
FEE module #: 22 elapsed dead time 0.000 s
FEE module #: 23 elapsed dead time 0.221 s
FEE module #: 24 elapsed dead time 0.000 s
FEE module #: 25 elapsed dead time 0.000 s
FEE module #: 26 elapsed dead time 0.000 s
FEE module #: 27 elapsed dead time 0.000 s
FEE module #: 28 elapsed dead time 0.000 s
FEE module #: 29 elapsed dead time 0.000 s
FEE module #: 30 elapsed dead time 0.000 s
FEE module #: 31 elapsed dead time 0.000 s
FEE module #: 32 elapsed dead time 0.000 s
*** Program elapsed time: 22.266s ( 1034.644 blocks/s, 64.665 Mb/s)
R55_10 ADC data synchronised
DSSSD bias & leakage currents OK - see attachments 1 & 2
FEE64 temperatures OK - see attachment 3
SYNC statistics, System Wide Checks SYNC error & SYNC pulses fail - see attachments 4-8
DAQ RESET & SETUP
System-wide checks
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
#1 DAQ start (R58)
nnaida7 data transfer #1 tag (X) drops at DAQ GO
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
#2 DAQ start (R59)
nnaida15 data transfer #1 tag (X) drops at DAQ GO
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
09.35 DAQ start (R60)
16.28 DAQ stop OK
Final file R60_169
16.30 DAQ start (R61)
BNC PB-4 pulser OFF
Background run - files R61_0 - R61_31
+ 60Co (ID# 0115) - files R61_33 - R61_49
60Co source placed on top of AIDA snout ~ above DSSSD #3
DAQ stalled at 18.28 (during R61_49) - as expected! See https://elog.ph.ed.ac.uk/AIDA/393
DAQ RESET & SETUP
System-wide checks
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
20.05 DAQ start (R63)
BNC PB-4 pulser ON, 60Co source removed |
| Attachment 1: 1.png
|
|
| Attachment 2: 2.png
|
|
| Attachment 3: 3.png
|
|
| Attachment 4: 4.png
|
|
| Attachment 5: 5.png
|
|
| Attachment 6: 6.png
|
|
| Attachment 7: 7.png
|
|
| Attachment 8: 8.png
|
|
|
393
|
Tue Oct 18 06:56:52 2016 |
TD | DAQ status |
From the installation of the RG58 cable for the 50MHz TTL clock from BRIKEN DAQ
to AIDA Master MACB external clock input we observe DAQ stalls at c. 06.30 and
18.30 daily. This coincides with the nominal start times of the ORNL Clover detector
LN2 fill cycles.
run start stop files recovery
AIDA Master MACB internal clock
R1 28.9.16 15.02 11.20 R1_304 OK
RG174 cable from BRIKEN DAQ 50MHz clock (TTL) to AIDA Master MACB external clock input
9.10.16 16.55 17.30 no data transfer, TS_SYNC phase = 0, OK
9.10.16 17.40 19.15 est no data transfer, TS_SYNC phase = 0, ReSYNC
9.10.16 22.20 01.40 no data transfer, TS_SYNC phase = 1, ?
10.10.16 14.20 16.50 no data transfer, TS_SYNC phase = 2, ?
10.10.16 18.35 18.40/19.40 no data transfer, TS_SYNC phase = 3, some FEE64s stopped later than others, ReSYNC
10.10.16 20.36 06.35 no data transfer, TS_SYNC phase = 0, power cycle
RG174 cable from BRIKEN DAQ 50MHz clock (TTL) to AIDA Master MACB external clock input
replaced by RG58 cable
run start stop files recovery
R17 11.10.16 15.46 08.22 R17_440 soft reset
R21 12.10.16 14.05 15.30 power dip - power cycle
R22 12.10.16 16.20 19.05 OK
R24 13.10.16 OK
R25 13.10.16 OK
R26 13.10.16 OK
R27 13.10.16 OK
R28 13.10.16 OK
R29 13.10.16 OK
R32 13.10.16 21.20 22.34 R32_26 soft reset
R33 14.10.16 00.01 00.25 ? power cycle
R40 14.10.16 01.49 06.26 R40_91 soft reset
R41 14.10.16 OK
R42 15.10.16 22.45 07.18 R42_164 System-wide tests - SYNC errors fail *but* SYNC pulses OK - ReSYNC only
R43 15.10.16 11.39 12.43 disk housekeeping caused merger/tapeserver problem
R46 15.10.16 12.43 18.26 R46_109 soft reset
R47 16.10.16 11.12 18.26 R47_120 power cycle
R49 16.10.16 21.05 06.26 R49_226 soft reset
R51 17.10.16 12.21 18.30 R51_145 soft reset
R52 17.10.16 20.41 06.30 R52_211 soft reset
R53 18.10.16 10.02 18.30 R53_193 soft reset
R55 18.10.16 18.52 19.17 R55_10 soft reset
R60 19.10.16 09.35 16.28 R60_169 OK
R61 19.10.16 16.30 18.26 R61_69 power cycle
R63 19.10.16 20.05 06.46 R63_291 soft reset
R64 20.10.16 09.45 18.25 R64_240 OK - System-wide tests - SYNC errors fail & SYNC pulses fail - other checks OK
R65 20.10.16 18.49 19.17 R65_12 power cycle
R66 20.10.16 20.02 10.55 R66_396 soft reset auto LN2 fill may have failed, manually selected c. 11.00
R67 21.10.16 14.02 18.26 R67_121 soft reset
R68 21.10.16 18.40 18.44 R68_1 soft reset
R70 21.10.16 18.53 06.46 R70_315 soft reset
R71 22.10.16 10.42 18.25 OK
R73 22.10.16 19.07 19.17 R73_4 soft reset
R74 22.10.16 19.38 06.31 R74_296 soft reset
R76 23.10.16 10.46 11.52 R76_29 merger crashed, soft reset
R77 23.10.16 18.50 19.17 disk housekeeping caused merger/tapeserver problem
R78 23.10.16 19.47 06.46 R78_291 soft reset
R79 24.10.16 11.07 18.46 R79_182 soft reset
R80 24.10.16 18.56 19.17 R80_5 DAQ stall, soft reset
R81 24.10.16 19.29 06.26 R81_169 soft reset
R82 25.10.16 10.22 18.30 R82_125 -> safe state
R83 27.10.16 14.35 18.30 R83_125 soft reset
R86 27.10.16 20.48 06.26 R86_245 soft reset
R87 28.10.16 14.34 18.47 R87_274 soft reset
R89 29.10.16 19.10 07.17 R89_102 power cycle
soft reset => no FEE64 power cycle
Restart merger
DAQ STOP
System wide checks
DAQ RESET & SETUP
System-wide checks
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1 |
|
391
|
Tue Oct 18 01:43:04 2016 |
TD | Tuesday 18 October |
09/42 DAQ found stopped, no merger error messages
Most recent file R52_211, last accessed 06.30
Restart merger OK
DAQ STOP OK
aidas1> ./analyser2 /TapeData/Sept2016/R52_211
*** GREAT format 3.2.0 analyser - TD - May 2014
*** ERROR: READ I/O error: 5002
blocks: 7619
ADC data format: 61809254 ( 1624588.3 Hz)
Other data format: 354167 ( 9308.9 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 28 ( 0.7 Hz)
RESUME: 28 ( 0.7 Hz)
SYNC100: 347607 ( 9136.5 Hz)
FEE64 disc: 5364 ( 141.0 Hz)
MBS info: 1140 ( 30.0 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 88 ( 2.3 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
Timestamp elapsed time: 38.046 s
FEE module #: 1 elapsed dead time 0.000 s
FEE module #: 2 elapsed dead time 0.011 s
FEE module #: 3 elapsed dead time 0.000 s
FEE module #: 4 elapsed dead time 0.000 s
FEE module #: 5 elapsed dead time 0.058 s
FEE module #: 6 elapsed dead time 0.165 s
FEE module #: 7 elapsed dead time 0.000 s
FEE module #: 8 elapsed dead time 0.000 s
FEE module #: 9 elapsed dead time 0.000 s
FEE module #: 10 elapsed dead time 0.000 s
FEE module #: 11 elapsed dead time 0.000 s
FEE module #: 12 elapsed dead time 0.000 s
FEE module #: 13 elapsed dead time 0.000 s
FEE module #: 14 elapsed dead time 0.002 s
FEE module #: 15 elapsed dead time 0.055 s
FEE module #: 16 elapsed dead time 0.000 s
FEE module #: 17 elapsed dead time 0.057 s
FEE module #: 18 elapsed dead time 0.089 s
FEE module #: 19 elapsed dead time 1.021 s
FEE module #: 20 elapsed dead time 0.000 s
FEE module #: 21 elapsed dead time 0.000 s
FEE module #: 22 elapsed dead time 0.000 s
FEE module #: 23 elapsed dead time 0.085 s
FEE module #: 24 elapsed dead time 0.004 s
FEE module #: 25 elapsed dead time 0.000 s
FEE module #: 26 elapsed dead time 0.000 s
FEE module #: 27 elapsed dead time 0.000 s
FEE module #: 28 elapsed dead time 0.000 s
FEE module #: 29 elapsed dead time 0.000 s
FEE module #: 30 elapsed dead time 0.000 s
FEE module #: 31 elapsed dead time 0.000 s
FEE module #: 32 elapsed dead time 0.000 s
*** Program elapsed time: 1.793s ( 4249.377 blocks/s, 265.586 Mb/s)
R51_211 ADC data synchronised
DSSSD bias & leakage currents OK - see attachments 1 & 2
FEE64 temperatures OK - see attachment 3
System-wide checks - see attachments 4 & 5
SYNC errors and SYNC pulses fail, all other tests OK
Statistics - see attachments 6-16
DAQ RESET & SETUP OK
ReSYNC OK
System-wide checks OK
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
10.02 DAQ start (R53)
18.20 DAQ OK
System-wide checks *all* OK
Statistics - see attachments 17-27
18.30 DAQ stalled on schedule
Most recent file R53_193 last accessed 18.30
AIDA SYNC rates zero in large fraction of FEE64s - see attachment 28
SYNC errors and SYNC pulses fail - see attachments 29-32
DAQ STOP
DAQ RESET
Restart merger
DAQ RESET OK
DAQ SETUP OK
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
System-wide checks *all* OK
18.45 DAQ starts (R54)
merging stops shortly after start of run - see attachment 33
DAQ HALT, RESET, SETUP OK
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
18.52 DAQ starts (R55) |
| Attachment 1: 1.png
|
|
| Attachment 2: 2.png
|
|
| Attachment 3: 3.png
|
|
| Attachment 4: 4.png
|
|
| Attachment 5: 5.png
|
|
| Attachment 6: 6.png
|
|
| Attachment 7: 7.png
|
|
| Attachment 8: 8.png
|
|
| Attachment 9: 9.png
|
|
| Attachment 10: 10.png
|
|
| Attachment 11: 11.png
|
|
| Attachment 12: 12.png
|
|
| Attachment 13: 13.png
|
|
| Attachment 14: 14.png
|
|
| Attachment 15: 15.png
|
|
| Attachment 16: 16.png
|
|
| Attachment 17: 20.png
|
|
| Attachment 18: 21.png
|
|
| Attachment 19: 22.png
|
|
| Attachment 20: 23.png
|
|
| Attachment 21: 24.png
|
|
| Attachment 22: 25.png
|
|
| Attachment 23: 26.png
|
|
| Attachment 24: 27.png
|
|
| Attachment 25: 28.png
|
|
| Attachment 26: 29.png
|
|
| Attachment 27: 30.png
|
|
| Attachment 28: 31.png
|
|
| Attachment 29: 32.png
|
|
| Attachment 30: 33.png
|
|
| Attachment 31: 34.png
|
|
| Attachment 32: 35.png
|
|
| Attachment 33: 36.png
|
|
|
390
|
Mon Oct 17 04:10:30 2016 |
TD | Monday 17 October |
12.05 DAQ found stopped
Most recent file R49_226, last accessed 06.26
Restart merger OK
DAQ STOP OK
System-wide checks - see attachments 1-2
SYNC errors and SYNC pulses fail
ReSYNC OK
System-wide checks OK
Restart merger & tapeserver
DSSSD bias & leakage currents OK - see attachments 3 & 4
FEE64 temperatures OK - see attachment 5
DAQ start
#1 nnaida15 data transfer #1 tag drops at DAQ GO
disable H & data transfer #1
restart merger & tapeserver
enable H & data transfer #1
12.22 #2 DAQ start (R51) OK
20.33 DAQ found stopped
Most recent file R51_145, last accessed 18.30
Restart merger OK
DAQ STOP OK
System-wide checks - see attachments 6 & 7
SYNC errors and SYNC pulses fail
DAQ RESET & SETUP OK
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
20.41 DAQ GO (R52) |
| Attachment 1: 20.png
|
|
| Attachment 2: 21.png
|
|
| Attachment 3: 23.png
|
|
| Attachment 4: 24.png
|
|
| Attachment 5: 26.png
|
|
| Attachment 6: 27.png
|
|
| Attachment 7: 28.png
|
|
|
388
|
Sun Oct 16 03:14:41 2016 |
TD | Sunday 16 October |
11.12 DAQ start (R47)
Detector bias and leakage currents OK - see attachments 1 & 2
FEE64 temperatures OK - see attachment 3
Statistics - see attachments 4-14
11.56 Temperature +25.0 deg C, d.p. +8.4 deg C, RH 34.5%
20.33 DAQ found stopped
Most recent file R47_120 (last access 18.26)
R47_120 ADC data synchronised
Restart merger and request DAQ STOP - DAQ stops OK
System-wide tests SYNC errors and SYNC pulses - see attachments 15 & 16 - all other tests OK
DAQ RESET & SETUP OK
Normal restart which failed shortly after enabling ASIC readout - restarting merger did not
recover DAQ control - power cycle required
21.05 DAQ starts (R49)
|
| Attachment 1: 1.png
|
|
| Attachment 2: 2.png
|
|
| Attachment 3: 3.png
|
|
| Attachment 4: 4.png
|
|
| Attachment 5: 5.png
|
|
| Attachment 6: 6.png
|
|
| Attachment 7: 7.png
|
|
| Attachment 8: 8.png
|
|
| Attachment 9: 9.png
|
|
| Attachment 10: 10.png
|
|
| Attachment 11: 11.png
|
|
| Attachment 12: 12.png
|
|
| Attachment 13: 13.png
|
|
| Attachment 14: 14.png
|
|
| Attachment 15: 15.png
|
|
| Attachment 16: 16.png
|
|
|
387
|
Sat Oct 15 03:27:31 2016 |
TD | Saturday 15 October |
11.27 DAQ found stopped
Most recent file R42_164, last accessed 07.18
Restart merger OK
DAQ STOP OK
System-wide checks - see attachments 1-5
ASIC clock timestamp, SYNC errors fail *but* SYNC pulses OK
ReSYNC OK
System-wide checks OK
DSSSD bias & leakage currents OK - see attachments 6 & 7
FEE64 temperatures OK - see attachment 8
R42_164 ADC data synchronised
aidas1> ./analyser2 /TapeData/Sept2016/R42_164
*** GREAT format 3.2.0 analyser - TD - May 2014
*** ERROR: READ I/O error: 5002
blocks: 2444
ADC data format: 19809141 ( 1404089.4 Hz)
Other data format: 131455 ( 9317.6 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 7 ( 0.5 Hz)
RESUME: 7 ( 0.5 Hz)
SYNC100: 129011 ( 9144.4 Hz)
FEE64 disc: 2010 ( 142.5 Hz)
MBS info: 420 ( 29.8 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 21 ( 1.5 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
Timestamp elapsed time: 14.108 s
FEE module #: 1 elapsed dead time 0.000 s
FEE module #: 2 elapsed dead time 0.000 s
FEE module #: 3 elapsed dead time 0.000 s
FEE module #: 4 elapsed dead time 0.000 s
FEE module #: 5 elapsed dead time 0.334 s
FEE module #: 6 elapsed dead time 0.042 s
FEE module #: 7 elapsed dead time 0.000 s
FEE module #: 8 elapsed dead time 0.000 s
FEE module #: 9 elapsed dead time 0.000 s
FEE module #: 10 elapsed dead time 0.000 s
FEE module #: 11 elapsed dead time 0.000 s
FEE module #: 12 elapsed dead time 0.000 s
FEE module #: 13 elapsed dead time 0.000 s
FEE module #: 14 elapsed dead time 0.000 s
FEE module #: 15 elapsed dead time 0.000 s
FEE module #: 16 elapsed dead time 0.000 s
FEE module #: 17 elapsed dead time 0.000 s
FEE module #: 18 elapsed dead time 0.005 s
FEE module #: 19 elapsed dead time 0.000 s
FEE module #: 20 elapsed dead time 0.000 s
FEE module #: 21 elapsed dead time 0.000 s
FEE module #: 22 elapsed dead time 0.000 s
FEE module #: 23 elapsed dead time 0.000 s
FEE module #: 24 elapsed dead time 0.000 s
FEE module #: 25 elapsed dead time 0.000 s
FEE module #: 26 elapsed dead time 0.000 s
FEE module #: 27 elapsed dead time 0.000 s
FEE module #: 28 elapsed dead time 0.000 s
FEE module #: 29 elapsed dead time 0.000 s
FEE module #: 30 elapsed dead time 0.000 s
FEE module #: 31 elapsed dead time 0.000 s
FEE module #: 32 elapsed dead time 0.000 s
*** Program elapsed time: 0.586s ( 4171.093 blocks/s, 260.693 Mb/s)
11.39 DAQ starts (R43)
12.07 Disk housekeeping caused TapeServer problem
DAQ stopped OK after some minutes and restarting merger
12.43 DAQ starts (R46)
18.26 DAQ stalled
Latest file R46_109 (most recent access 18.26)
Merger error messages - see attachment 9
Restart merger - error messages - see attachment 10
Analysis of R46_109
ADC data synchronised
aidas1> ./analyser2 /TapeData/Sept2016/R46_109
*** GREAT format 3.2.0 analyser - TD - May 2014
*** ERROR: READ I/O error: 5002
blocks: 4688
ADC data format: 37985730 ( 1342216.8 Hz)
Other data format: 263662 ( 9316.4 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 25 ( 0.9 Hz)
RESUME: 25 ( 0.9 Hz)
SYNC100: 258788 ( 9144.2 Hz)
FEE64 disc: 3981 ( 140.7 Hz)
MBS info: 843 ( 29.8 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 41 ( 1.4 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
Timestamp elapsed time: 28.301 s
FEE module #: 1 elapsed dead time 0.000 s
FEE module #: 2 elapsed dead time 0.013 s
FEE module #: 3 elapsed dead time 0.000 s
FEE module #: 4 elapsed dead time 0.000 s
FEE module #: 5 elapsed dead time 0.298 s
FEE module #: 6 elapsed dead time 0.200 s
FEE module #: 7 elapsed dead time 0.000 s
FEE module #: 8 elapsed dead time 0.000 s
FEE module #: 9 elapsed dead time 0.001 s
FEE module #: 10 elapsed dead time 0.000 s
FEE module #: 11 elapsed dead time 0.000 s
FEE module #: 12 elapsed dead time 0.000 s
FEE module #: 13 elapsed dead time 0.000 s
FEE module #: 14 elapsed dead time 0.003 s
FEE module #: 15 elapsed dead time 0.000 s
FEE module #: 16 elapsed dead time 0.000 s
FEE module #: 17 elapsed dead time 0.086 s
FEE module #: 18 elapsed dead time 0.054 s
FEE module #: 19 elapsed dead time 0.097 s
FEE module #: 20 elapsed dead time 0.000 s
FEE module #: 21 elapsed dead time 0.000 s
FEE module #: 22 elapsed dead time 0.001 s
FEE module #: 23 elapsed dead time 0.000 s
FEE module #: 24 elapsed dead time 0.000 s
FEE module #: 25 elapsed dead time 0.000 s
FEE module #: 26 elapsed dead time 0.000 s
FEE module #: 27 elapsed dead time 0.000 s
FEE module #: 28 elapsed dead time 0.000 s
FEE module #: 29 elapsed dead time 0.000 s
FEE module #: 30 elapsed dead time 0.000 s
FEE module #: 31 elapsed dead time 0.000 s
FEE module #: 32 elapsed dead time 0.000 s
*** Program elapsed time: 2.102s ( 2230.721 blocks/s, 139.420 Mb/s)
11.00 Sunday 16 October
DAQ STOP requested - nnaida1-24 all stop *except* nnaida6 which remains going - Run Control displays RESET
only
DAQ RESET
System-wide checks all OK
Normal setup and DAQ restart (see https://elog.ph.ed.ac.uk/AIDA/388 )
DAQ SETUP
|
| Attachment 1: 20.png
|
|
| Attachment 2: 21.png
|
|
| Attachment 3: 22.png
|
|
| Attachment 4: 23.png
|
|
| Attachment 5: 24.png
|
|
| Attachment 6: 25.png
|
|
| Attachment 7: 26.png
|
|
| Attachment 8: 27.png
|
|
| Attachment 9: 30.png
|
|
| Attachment 10: 31.png
|
|
|
385
|
Thu Oct 13 16:12:06 2016 |
TD & CG | Friday 14 October |
DAQ continues (R33)
aidas1> ./analyser2 /TapeData/Sept2016/R33_0
*** GREAT format 3.2.0 analyser - TD - May 2014
*** ERROR: READ I/O error: 5002
blocks: 32000
ADC data format: 258854184 ( 1075951.6 Hz)
Other data format: 2233816 ( 9285.1 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 95 ( 0.4 Hz)
RESUME: 95 ( 0.4 Hz)
SYNC100: 2200965 ( 9148.5 Hz)
FEE64 disc: 26965 ( 112.1 Hz)
MBS info: 5696 ( 23.7 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 273 ( 1.1 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
Timestamp elapsed time: 240.582 s
FEE module #: 1 elapsed dead time 0.000 s
FEE module #: 2 elapsed dead time 0.026 s
FEE module #: 3 elapsed dead time 0.000 s
FEE module #: 4 elapsed dead time 0.000 s
FEE module #: 5 elapsed dead time 0.278 s
FEE module #: 6 elapsed dead time 1.219 s
FEE module #: 7 elapsed dead time 0.000 s
FEE module #: 8 elapsed dead time 0.000 s
FEE module #: 9 elapsed dead time 0.000 s
FEE module #: 10 elapsed dead time 0.000 s
FEE module #: 11 elapsed dead time 0.000 s
FEE module #: 12 elapsed dead time 0.000 s
FEE module #: 13 elapsed dead time 0.000 s
FEE module #: 14 elapsed dead time 0.001 s
FEE module #: 15 elapsed dead time 0.014 s
FEE module #: 16 elapsed dead time 0.015 s
FEE module #: 17 elapsed dead time 0.703 s
FEE module #: 18 elapsed dead time 0.196 s
FEE module #: 19 elapsed dead time 1.037 s
FEE module #: 20 elapsed dead time 0.063 s
FEE module #: 21 elapsed dead time 0.000 s
FEE module #: 22 elapsed dead time 0.006 s
FEE module #: 23 elapsed dead time 0.073 s
FEE module #: 24 elapsed dead time 0.000 s
FEE module #: 25 elapsed dead time 0.000 s
FEE module #: 26 elapsed dead time 0.000 s
FEE module #: 27 elapsed dead time 0.000 s
FEE module #: 28 elapsed dead time 0.000 s
FEE module #: 29 elapsed dead time 0.000 s
FEE module #: 30 elapsed dead time 0.000 s
FEE module #: 31 elapsed dead time 0.000 s
FEE module #: 32 elapsed dead time 0.000 s
*** Program elapsed time: 22.278s ( 1436.394 blocks/s, 89.775 Mb/s)
00.16 DSSSD bias and leakage currents OK - see attachments 1 & 2
FEE64 temperatures OK - see attachment 3
Temperature +25.0 deg C, d.p. +8.2 deg C, RH 33.8%
00.25 DAQ stopped
Unable to restart merger - see attachment 4 - DAQ STOP OK
System wide tests fail - see attachments 5 & 6
DAQ RESET & SETUP, ReSYNC OK
System wide checks fail - as above
Several FEE64 power cycles and eventually an aidas1 reboot due to
1) timeouts from random FEE64s following power cycle + 10 mins
2) data transfer links (X) dropping at DAQ GO
01.49 DAQ starts (R40)
08.31 DAQ found stopped
Most recently accessed file R40_91 @ 06.26
Restart merger & request DAQ STOP
Transient timeout from nnaida8 but STOP completed OK
11.10 DSSSD bias and leakage currents OK - see attachments 7 & 8
FEE64 temperatures OK - see attachment 9
System-wide checks - system clock, SYNC errors and SUNC count fail - see attachments 10-13
ReSYNC fails - see attachment 14
11.30 Analysis of R40 data
R40_91 ADC data synchronised
aidas1> ./analyser2 /TapeData/Sept2016/R40_91
*** GREAT format 3.2.0 analyser - TD - May 2014
*** ERROR: READ I/O error: 5002
blocks: 30880
ADC data format: 249997809 ( 1194334.5 Hz)
Other data format: 1952111 ( 9326.0 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 55 ( 0.3 Hz)
RESUME: 55 ( 0.3 Hz)
SYNC100: 1915685 ( 9152.0 Hz)
FEE64 disc: 30044 ( 143.5 Hz)
MBS info: 6272 ( 30.0 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 303 ( 1.4 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
Timestamp elapsed time: 209.320 s
FEE module #: 1 elapsed dead time 0.000 s
FEE module #: 2 elapsed dead time 0.000 s
FEE module #: 3 elapsed dead time 0.000 s
FEE module #: 4 elapsed dead time 0.000 s
FEE module #: 5 elapsed dead time 0.167 s
FEE module #: 6 elapsed dead time 0.529 s
FEE module #: 7 elapsed dead time 0.000 s
FEE module #: 8 elapsed dead time 0.000 s
FEE module #: 9 elapsed dead time 0.000 s
FEE module #: 10 elapsed dead time 0.000 s
FEE module #: 11 elapsed dead time 0.000 s
FEE module #: 12 elapsed dead time 0.000 s
FEE module #: 13 elapsed dead time 0.000 s
FEE module #: 14 elapsed dead time 0.002 s
FEE module #: 15 elapsed dead time 0.018 s
FEE module #: 16 elapsed dead time 0.043 s
FEE module #: 17 elapsed dead time 0.098 s
FEE module #: 18 elapsed dead time 0.035 s
FEE module #: 19 elapsed dead time 0.378 s
FEE module #: 20 elapsed dead time 0.010 s
FEE module #: 21 elapsed dead time 0.000 s
FEE module #: 22 elapsed dead time 0.000 s
FEE module #: 23 elapsed dead time 0.046 s
FEE module #: 24 elapsed dead time 0.000 s
FEE module #: 25 elapsed dead time 0.000 s
FEE module #: 26 elapsed dead time 0.000 s
FEE module #: 27 elapsed dead time 0.000 s
FEE module #: 28 elapsed dead time 0.000 s
FEE module #: 29 elapsed dead time 0.000 s
FEE module #: 30 elapsed dead time 0.000 s
FEE module #: 31 elapsed dead time 0.000 s
FEE module #: 32 elapsed dead time 0.000 s
*** Program elapsed time: 6.613s ( 4669.392 blocks/s, 291.837 Mb/s)
21.52 DAQ starts (Sept2016/R41) BRIKEN filename 252CfwithAida75mm
Correlation scaler reset requested by BRIKEN DAQ following start of run to disk
End of AIDA snout positioned at edge of (CH2)n moderator aperture for
ORNL HPGe clover detectors
RIKEN 252Cf source ~25k neutrons/s
FEE64 temperatures OK
*All* system-wide checks OK
B2F/F11 temperature + deg C, d.p. + deg C,RH %
ASIC settings 2016Sep02-18.17.01
slow comparator 0x10 -> 0x0a
LEC/MEC fast comparators # 1, 55 & 58 disabled for all FEE64s
Master MACB serial #13 (see https://elog.ph.ed.ac.uk/AIDA/367)
Master MACB mode 5
TS_SYNC_PHASE 0
22.17 DAQ stops OK
BRIKEN DAQ stopped at about same time
22.45 DAQ starts (R42)
AIDA withdrawn from (CH2)n moderator, 252Cf source removed |
| Attachment 1: 10.png
|
|
| Attachment 2: 11.png
|
|
| Attachment 3: 12.png
|
|
| Attachment 4: 13.png
|
|
| Attachment 5: 14.png
|
|
| Attachment 6: 15.png
|
|
| Attachment 7: 7.png
|
|
| Attachment 8: 8.png
|
|
| Attachment 9: 9.png
|
|
| Attachment 10: 10.png
|
|
| Attachment 11: 11.png
|
|
| Attachment 12: 12.png
|
|
| Attachment 13: 14.png
|
|
| Attachment 14: 15.png
|
|
|
384
|
Thu Oct 13 06:07:43 2016 |
TD & CG | Thursday 13 October |
11.40 Power cycle AIDA FEE64s
Set TS_SYNC_PHASE 0
DSSSD bias/leakage current OK - see attachment 1-2
FEE64 temperatures OK - see attachment 3
*All* system-wide checks OK
B2F/F11 temperature + deg C, d.p. + deg C,RH %
ASIC settings 2016Sep02-18.17.01
slow comparator 0x10 -> 0x0a
LEC/MEC fast comparators # 1, 55 & 58 disabled for all FEE64s
Master MACB serial #13 (see https://elog.ph.ed.ac.uk/AIDA/367)
Master MACB mode 5
12.15 DAQ starts (file Sept2016/R24)
15.16 DAQ stops OK
15.18 DAQ starts (R25)
Pulser walkthrough
BNC PB-4 amplitudes 10,000 - 90,000 @ 10,000 step
See attachment 4
Spectra saved 13.10.16 15.32 (08.32 FEE64s)
15.30 DAQ stops OK
15.38 DAQ starts (R26)
Pulser walkthrough
BNC PB-4 amplitudes 8,000 - 2,000 @ 2,000 step
See attachment 5
Spectra saved 13.10.16 15.47 (08.47 FEE64s)
15.45 DAQ stops
15.50 DAQ starts (R27)
LEC/MEC FSR 20MeV -> 1GeV
BNC PB-4 amplitude 90,000
Spectra saved 13.10.16 16.07 (09.05 FEE64s)
16.05 DAQ stops
16.12 DAQ starts (R28)
LEC/MEC FSR 1GeV -> 20MeV
BNC PB-4 off (noise + background activity only)
Spectra saved 13.10.16 16:49 (saved 09.49 FEE64s)
16.48 DAQ stops
16.57 DAQ starts (R29)
BNC PB-4 90,000 amplitude ~23Hz freq.
17.26 Lost SYNC pulses. Merger+TapeServer stopped writing to file (noticed 18.20)
18.25 R29 closed.
19:20 NIM bin supplying TTL clock to master MACB failed at or around the same time as the DAQ stop occurred.
Need to replace NIM bin before clock can be supplied.
21.20 DAQ starts (R32)
23.51 DAQ found stopped, most recent file R32_26 (last access 22.34 13.10.16)
23.56 Restarted merger & immediately requested DAQ STOP - OK
DAQ reset & setup
System-wide tests all OK
00.01 DAQ starts (R33) |
| Attachment 1: 1.png
|
|
| Attachment 2: 2.png
|
|
| Attachment 3: 3.png
|
|
| Attachment 4: 4.png
|
|
| Attachment 5: 5.png
|
|
|
383
|
Wed Oct 12 02:58:19 2016 |
TD & CG | Wednesday 12 October |
10.50 Merger & TapeServer found stopped
connect errors - see attachment 1
DSSSD bias and leakage currents OK - see attachments 2 & 3
Most recent data file R17_440 last accessed 08.22 12.10.16
R17_440 ADC data synchronised
aidas1> ./analyser2 /TapeData/Sept2016/R17_440
*** GREAT format 3.2.0 analyser - TD - May 2014
*** ERROR: READ I/O error: 5002
blocks: 30546
ADC data format: 147630891 ( 1189229.1 Hz)
Other data format: 101593923 ( 818381.9 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 101 ( 0.8 Hz)
RESUME: 101 ( 0.8 Hz)
SYNC100: 1133804 ( 9133.3 Hz)
FEE64 disc: 100456290 ( 809217.8 Hz)
MBS info: 3627 ( 29.2 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 151 ( 1.2 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
Timestamp elapsed time: 124.140 s
FEE module #: 1 elapsed dead time 0.728 s
FEE module #: 2 elapsed dead time 0.184 s
FEE module #: 3 elapsed dead time 0.000 s
FEE module #: 4 elapsed dead time 0.000 s
FEE module #: 5 elapsed dead time 1.034 s
FEE module #: 6 elapsed dead time 3.462 s
FEE module #: 7 elapsed dead time 0.000 s
FEE module #: 8 elapsed dead time 0.000 s
FEE module #: 9 elapsed dead time 0.064 s
FEE module #: 10 elapsed dead time 0.003 s
FEE module #: 11 elapsed dead time 0.000 s
FEE module #: 12 elapsed dead time 0.000 s
FEE module #: 13 elapsed dead time 0.000 s
FEE module #: 14 elapsed dead time 0.062 s
FEE module #: 15 elapsed dead time 0.018 s
FEE module #: 16 elapsed dead time 0.005 s
FEE module #: 17 elapsed dead time 0.817 s
FEE module #: 18 elapsed dead time 0.035 s
FEE module #: 19 elapsed dead time 0.048 s
FEE module #: 20 elapsed dead time 0.000 s
FEE module #: 21 elapsed dead time 0.000 s
FEE module #: 22 elapsed dead time 0.103 s
FEE module #: 23 elapsed dead time 0.216 s
FEE module #: 24 elapsed dead time 0.000 s
FEE module #: 25 elapsed dead time 0.000 s
FEE module #: 26 elapsed dead time 0.000 s
FEE module #: 27 elapsed dead time 0.000 s
FEE module #: 28 elapsed dead time 0.000 s
FEE module #: 29 elapsed dead time 0.000 s
FEE module #: 30 elapsed dead time 0.000 s
FEE module #: 31 elapsed dead time 0.000 s
FEE module #: 32 elapsed dead time 0.000 s
*** Program elapsed time: 14.129s ( 2161.951 blocks/s, 135.122 Mb/s)
Conclude DAQ ran without problem for c. 16.5h which is a significant improvement
assuming that it can be consistently reproduced.
11.20 Merger restart did not clear connect/timeout issues
Re-route BNC - BNC RG58 cable with 50MHz clock from BRIKEN to
AIDA Master MACB external clock input (rear panel) so that it will
not be stepped upon.
Power cycle AIDA FEE64s
N.B. I selected 'Switch ALL On' rather than 'Sequence ALL On' by mistake and one
of the USB ac mains controller output 250VAC/16A fuses blew - use 'Sequence ALL On'
TS_SYNC_PHASE 0
FEE64 temperatures OK
*All* system-wide checks OK
B2F/F11 temperature +24.6 deg C, d.p. +8.9 deg C,RH 36.6%
ASIC settings 2016Sep02-18.17.01
slow comparator 0x10 -> 0x0a
LEC/MEC fast comparators # 1, 55 & 58 disabled for all FEE64s
Master MACB serial #13 (see https://elog.ph.ed.ac.uk/AIDA/367)
Master MACB mode 5
14.05 DAQ starts (file Sept2016/R21)
15.30 Sequence of three electrical power dips over 30 minute period. RIKEN public address
system reports that TEPCO (Tokyo Electric Power Company) - the site supplier - is
having region-wide problems.
The third power dip stopped DAQ.
Unable to recover by restarting merger
Power cycle FEE64s
16.20 DAQ restarts (file Sept2016/R22)
19.05 DAQ stops
Because of risk of power outages due to TEPCO distribution problems we have powered
down AIDA and Julabo FL1006 chiller and disconnected from ac mains. |
| Attachment 1: 1.png
|
|
| Attachment 2: 2.png
|
|
| Attachment 3: 3.png
|
|
| Attachment 4: 4.png
|
|
| Attachment 5: 5.png
|
|
|
382
|
Tue Oct 11 03:34:02 2016 |
CG & TD | Tuesday 11 October |
10.40 DAQ found stopped
All FEE64 AIDA SYNC rates zero, AIDA SYNC count ~ 13.8M => DAQ stopped @ 06.35
System wide checks - see attachments 1-4 - all other system-wide checks OK
11.15 Replace cable for Master MACB external clock signal
- change from c. 4m RG174 (Lemo-00 to Lemo-00) to c. 4m RG58 (BNC to BNC)
Power cycle AIDA FEE64s
Set TS_SYNC_PHASE 0
DSSSD bias/leakage current OK - see attacment 5-6
FEE64 temperatures OK - see attachment 7
*All* system-wide checks OK (N.B. several ReSYNCs required)
B2F/F11 temperature + deg C, d.p. + deg C,RH %
ASIC settings 2016Sep02-18.17.01
slow comparator 0x10 -> 0x0a
Master MACB serial #13 (see https://elog.ph.ed.ac.uk/AIDA/367)
Master MACB mode 5
11.58 DAQ starts
15.41 DAQ OK
15.44 DAQ stop requested
15.46 DAQ start
Data transfer to disk enabled to test DAQ correlation real-time monitor
22.50 AIDA SYNCs OK
# Master SYNCs *not equal* to # FEE64 SYNCs - all FEE64 SYNCs # equal
- all other system-wide checks OK - see attachments 8-15
Merger and TapeServer OK
R17_197 ADC data synchonised
aidas1> ./analyser2 /TapeData/Sept2016/R17_197
*** GREAT format 3.2.0 analyser - TD - May 2014
*** ERROR: READ I/O error: 5002
blocks: 32000
ADC data format: 165074646 ( 1182230.4 Hz)
Other data format: 96013354 ( 687627.8 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 113 ( 0.8 Hz)
RESUME: 113 ( 0.8 Hz)
SYNC100: 1275326 ( 9133.6 Hz)
FEE64 disc: 94733738 ( 678463.4 Hz)
MBS info: 4064 ( 29.1 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 191 ( 1.4 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
Timestamp elapsed time: 139.630 s
FEE module #: 1 elapsed dead time 0.000 s
FEE module #: 2 elapsed dead time 0.206 s
FEE module #: 3 elapsed dead time 0.000 s
FEE module #: 4 elapsed dead time 0.000 s
FEE module #: 5 elapsed dead time 1.084 s
FEE module #: 6 elapsed dead time 4.015 s
FEE module #: 7 elapsed dead time 0.000 s
FEE module #: 8 elapsed dead time 0.000 s
FEE module #: 9 elapsed dead time 0.201 s
FEE module #: 10 elapsed dead time 0.000 s
FEE module #: 11 elapsed dead time 0.000 s
FEE module #: 12 elapsed dead time 0.000 s
FEE module #: 13 elapsed dead time 0.003 s
FEE module #: 14 elapsed dead time 0.193 s
FEE module #: 15 elapsed dead time 0.024 s
FEE module #: 16 elapsed dead time 0.000 s
FEE module #: 17 elapsed dead time 1.198 s
FEE module #: 18 elapsed dead time 0.040 s
FEE module #: 19 elapsed dead time 0.124 s
FEE module #: 20 elapsed dead time 0.000 s
FEE module #: 21 elapsed dead time 0.000 s
FEE module #: 22 elapsed dead time 0.187 s
FEE module #: 23 elapsed dead time 0.258 s
FEE module #: 24 elapsed dead time 0.000 s
FEE module #: 25 elapsed dead time 0.000 s
FEE module #: 26 elapsed dead time 0.000 s
FEE module #: 27 elapsed dead time 0.000 s
FEE module #: 28 elapsed dead time 0.000 s
FEE module #: 29 elapsed dead time 0.000 s
FEE module #: 30 elapsed dead time 0.000 s
FEE module #: 31 elapsed dead time 0.000 s
FEE module #: 32 elapsed dead time 0.000 s
*** Program elapsed time: 19.555s ( 1636.436 blocks/s, 102.277 Mb/s) |
| Attachment 1: 50.png
|
|
| Attachment 2: 51.png
|
|
| Attachment 3: 53.png
|
|
| Attachment 4: 54.png
|
|
| Attachment 5: 55.png
|
|
| Attachment 6: 56.png
|
|
| Attachment 7: 57.png
|
|
| Attachment 8: 60.png
|
|
| Attachment 9: 61.png
|
|
| Attachment 10: 62.png
|
|
| Attachment 11: 63.png
|
|
| Attachment 12: 64.png
|
|
| Attachment 13: 65.png
|
|
| Attachment 14: 66.png
|
|
| Attachment 15: 67.png
|
|
| Attachment 16: 69.png
|
|