| ID |
Date |
Author |
Subject |
|
357
|
Sun Sep 4 04:35:42 2016 |
TD | FEE64 & MACB configuration |
UP
nnaida1 nnaida9 nnaida17
LEFT nnaida2 nnaida10 nnaida18 RIGHT
nnaida23 nnaida15 nnaida7 nnaida3 nnaida11 nnaida19
nnaida24 nnaida16 nnaida8 nnaida4 nnaida12 nnaida20
nnaida5 nnaida13 nnaida21
nnaida6 nnaida14 nnaida22
DOWN
<LOOKING UPSTREAM>
LEFT-RIGHT n+n ohmic strips
UP-DOWN p+n junction strips
1 2 3 4 5 6 7 8 9 10 11 MACB NIM Module # spare 1 spare 2
<- #1 <- #2 <- #2 <- #2 <- #2 <- #1 <- #7 <- #7 <- #7 <- #7 Next
-> #2 -> #3 6* 21 7 23 -> #8 1 17 3 19 Port 1
-> #7 -> #4 5 22 8 24 -> #9 2 18 4 20 Port 2
-> #5 13 29 15 31 -> #10 9 25 11 27 Port 3
-> #6 14 30 16 32 -> #11 10 26 12 28 Port 4
** 2 2 3 3 3 3 3 3 3 3 Mode
y y Rev B y
* = Master Timestamp FEE64
** MACB (rev A serial #13 mode 5) external 50MHz clock
or
MACB (rev B serial #11 mode 0) internal 50MHz cloxk
MACB HDMI cabling mechanically secured by 1U cable organiser above MACB NIM bin and by AIDA frame
- see attachments 1-3 |
| Attachment 1: IMG_6502.JPG
|
|
| Attachment 2: IMG_6503.JPG
|
|
| Attachment 3: IMG_6504.JPG
|
|
|
358
|
Mon Sep 5 06:54:02 2016 |
TD | Monday 5 September |
14.20 R17 continues
DSSSD bias/leakage current OK - see attachments 1-2
FEE64 temperatures OK - see attachment 3
System-wide checks OK *except* nnaida19 fails ASIC timestamp check (no change)
No merger error messages
AIDA SYNC, PAUSE, RESUME, Correlation L, SYNC time warp, Good Events, AIDA disc, nnaida6, nnaida5 &
nnaida20 statistics - see attachments 4-14
Analysis of R17_584 - FEE64 data synaidas1> ./analyser2 /TapeData/Aug2016/R17_584
*** GREAT format 3.2.0 analyser - TD - May 2014
*** ERROR: READ I/O error: 5002
blocks: 32000
ADC data format: 258868072 ( 1733415.6 Hz)
Other data format: 2219928 ( 14864.9 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 94 ( 0.6 Hz)
RESUME: 94 ( 0.6 Hz)
SYNC100: 1365786 ( 9145.5 Hz)
FEE64 disc: 842248 ( 5639.8 Hz)
MBS info: 11706 ( 78.4 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 400 ( 2.7 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
Timestamp elapsed time: 149.340 s
FEE module #: 1 elapsed dead time 0.000 s
FEE module #: 2 elapsed dead time 0.026 s
FEE module #: 3 elapsed dead time 0.007 s
FEE module #: 4 elapsed dead time 0.000 s
FEE module #: 5 elapsed dead time 0.439 s
FEE module #: 6 elapsed dead time 0.329 s
FEE module #: 7 elapsed dead time 0.000 s
FEE module #: 8 elapsed dead time 0.000 s
FEE module #: 9 elapsed dead time 0.104 s
FEE module #: 10 elapsed dead time 0.000 s
FEE module #: 11 elapsed dead time 0.000 s
FEE module #: 12 elapsed dead time 0.001 s
FEE module #: 13 elapsed dead time 0.515 s
FEE module #: 14 elapsed dead time 0.290 s
FEE module #: 15 elapsed dead time 0.078 s
FEE module #: 16 elapsed dead time 0.036 s
FEE module #: 17 elapsed dead time 0.379 s
FEE module #: 18 elapsed dead time 0.044 s
FEE module #: 19 elapsed dead time 0.122 s
FEE module #: 20 elapsed dead time 0.063 s
FEE module #: 21 elapsed dead time 0.218 s
FEE module #: 22 elapsed dead time 0.467 s
FEE module #: 23 elapsed dead time 0.232 s
FEE module #: 24 elapsed dead time 0.059 s
FEE module #: 25 elapsed dead time 0.000 s
FEE module #: 26 elapsed dead time 0.000 s
FEE module #: 27 elapsed dead time 0.000 s
FEE module #: 28 elapsed dead time 0.000 s
FEE module #: 29 elapsed dead time 0.000 s
FEE module #: 30 elapsed dead time 0.000 s
FEE module #: 31 elapsed dead time 0.000 s
FEE module #: 32 elapsed dead time 0.000 s
*** Program elapsed time: 26.172s ( 1222.687 blocks/s, 76.418 Mb/s)
chronised - see attachment 16
aidas1> ./analyser2 /TapeData/Aug2016/R17_584
*** GREAT format 3.2.0 analyser - TD - May 2014
*** ERROR: READ I/O error: 5002
blocks: 32000
ADC data format: 258868072 ( 1733415.6 Hz)
Other data format: 2219928 ( 14864.9 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 94 ( 0.6 Hz)
RESUME: 94 ( 0.6 Hz)
SYNC100: 1365786 ( 9145.5 Hz)
FEE64 disc: 842248 ( 5639.8 Hz)
MBS info: 11706 ( 78.4 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 400 ( 2.7 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
Timestamp elapsed time: 149.340 s
FEE module #: 1 elapsed dead time 0.000 s
FEE module #: 2 elapsed dead time 0.026 s
FEE module #: 3 elapsed dead time 0.007 s
FEE module #: 4 elapsed dead time 0.000 s
FEE module #: 5 elapsed dead time 0.439 s
FEE module #: 6 elapsed dead time 0.329 s
FEE module #: 7 elapsed dead time 0.000 s
FEE module #: 8 elapsed dead time 0.000 s
FEE module #: 9 elapsed dead time 0.104 s
FEE module #: 10 elapsed dead time 0.000 s
FEE module #: 11 elapsed dead time 0.000 s
FEE module #: 12 elapsed dead time 0.001 s
FEE module #: 13 elapsed dead time 0.515 s
FEE module #: 14 elapsed dead time 0.290 s
FEE module #: 15 elapsed dead time 0.078 s
FEE module #: 16 elapsed dead time 0.036 s
FEE module #: 17 elapsed dead time 0.379 s
FEE module #: 18 elapsed dead time 0.044 s
FEE module #: 19 elapsed dead time 0.122 s
FEE module #: 20 elapsed dead time 0.063 s
FEE module #: 21 elapsed dead time 0.218 s
FEE module #: 22 elapsed dead time 0.467 s
FEE module #: 23 elapsed dead time 0.232 s
FEE module #: 24 elapsed dead time 0.059 s
FEE module #: 25 elapsed dead time 0.000 s
FEE module #: 26 elapsed dead time 0.000 s
FEE module #: 27 elapsed dead time 0.000 s
FEE module #: 28 elapsed dead time 0.000 s
FEE module #: 29 elapsed dead time 0.000 s
FEE module #: 30 elapsed dead time 0.000 s
FEE module #: 31 elapsed dead time 0.000 s
FEE module #: 32 elapsed dead time 0.000 s
*** Program elapsed time: 26.172s ( 1222.687 blocks/s, 76.418 Mb/s)
15.08 R17 stops
Detector bias OFF
FEE64 power OFF
USB-controlled ac mains relay OFF
Julabo FL11006 recirculating chiller OFF
ac mains cables to ac mains distributor disconnected
network cables aida1 and aida2 disconnected, rolled up and secured to stairwell rails
207Bi source removed and returned to B2F/F11 storage location |
| Attachment 1: 1.png
|
|
| Attachment 2: 2.png
|
|
| Attachment 3: 3.png
|
|
| Attachment 4: 4.png
|
|
| Attachment 5: 5.png
|
|
| Attachment 6: 6.png
|
|
| Attachment 7: 7.png
|
|
| Attachment 8: 8.png
|
|
| Attachment 9: 9.png
|
|
| Attachment 10: 10.png
|
|
| Attachment 11: 11.png
|
|
| Attachment 12: 12.png
|
|
| Attachment 13: 13.png
|
|
| Attachment 14: 14.png
|
|
| Attachment 15: 15.png
|
|
| Attachment 16: 16.png
|
|
|
363
|
Fri Sep 30 03:35:12 2016 |
TD | Friday 30 September |
11.37 Attempt to use external 50MHz clock for Master MACB (w/ 50MHz front panel label)
NIM 50MHz -> NIM-TTL translator -> Master MACB (rear panel)
Master MACB mode 0 -> 5
Fails to ReSync, clock errors etc - see attachments 1-5
Change NIM 50MHz -> 25MHz (to improve signal quality) and attempt ReSync (no power FEE64 power cycle) - no
change
14.30 Master MACB mode -> 0
NIM 25MHz -> Master MACB correlation clock input (front panel)
BNC PB_4 frequency c. 26Hz
Trigger output -> SCA -> LeCroy 429A -> Master MACB accepted trigger input
./analyser2 v /TapeData/Sept2016/R5_0 | grep MBS
:
:
*** MBS information: block: 471 ptr: 7203 data: 0x86807CAB module: 6 information type: 8 information
index: 0 MBS data:7CAB ts: 0x00000017AC2C4582
*** MBS information: block: 471 ptr: 7205 data: 0x8681575E module: 6 information type: 8 information
index: 1 MBS data:575E ts: 0x00000017AC2C4582
*** MBS information: block: 471 ptr: 7207 data: 0x86820000 module: 6 information type: 8 information
index: 2 MBS data:0000 ts: 0x00000017AC2C4582
*** MBS information: block: 471 ptr: 15767 data: 0x86800FA8 module: 6 information type: 8 information
index: 0 MBS data:0FA8 ts: 0x00000017AC669176
*** MBS information: block: 471 ptr: 15769 data: 0x8681576D module: 6 information type: 8 information
index: 1 MBS data:576D ts: 0x00000017AC669176
*** MBS information: block: 471 ptr: 15771 data: 0x86820000 module: 6 information type: 8 information
index: 2 MBS data:0000 ts: 0x00000017AC669176
0x576D0FA8 - 0x575E7CAB = 0xE92FD = 955133 * 40ns = 38205320ns = 38.2ms => 26Hz
0x00000017AC669176 - 0x00000017AC2C4582 = 0x3a4bf4 = 3820532 * 10ns = 38205320ns QED
15.45 Master MACB mode 0 -> 5
NIM 50MHz -> Phillips 726 NIM-TTL-NIM -> Master MACB external clock (rear panel)
Phillips 726 provides better quality TTL signal with c. 3ns rise/falltimes and 3.5V amplitude into 50 Ohm
cf. NIM-TTL-NIM module used previously
See http://www.phillipsscientific.com/pdf/726ds.pdf
RESET/SETUP DAQ OK
System-wide checks OK *except* nnaida19 fails ASIC timestamp check (no change)
Start test run R6_0 to disk but no AIDA issue correlation triggers are recorded
Can manually issue correlation triggers and resets via Correlation Status & Control tab
Observe statistics count 3x for each trigger and reset - see https://elog.ph.ed.ac.uk/AIDA/362
aidas1> ./analyser2 v /TapeData/Sept2016/R6_0 | grep MBS
:
:
Test #1 - manual correlation trigger
*** MBS information: block: 9626 ptr: 9991 data: 0x86807774 module: 6 information type: 8 information
index: 0 MBS data:7774 ts: 0x00000077517D2816
*** MBS information: block: 9626 ptr: 9993 data: 0x86817213 module: 6 information type: 8 information
index: 1 MBS data:7213 ts: 0x00000077517D2816
*** MBS information: block: 9626 ptr: 9995 data: 0x86820003 module: 6 information type: 8 information
index: 2 MBS data:0003 ts: 0x00000077517D2816
:
:
Test #2 - manual correlation trigger x2 separated by ~10s
*** MBS information: block: 16751 ptr: 10787 data: 0x86807285 module: 6 information type: 8 information
index: 0 MBS data:7285 ts: 0x000000833335145A
*** MBS information: block: 16751 ptr: 10789 data: 0x86816A81 module: 6 information type: 8 information
index: 1 MBS data:6A81 ts: 0x000000833335145A
*** MBS information: block: 16751 ptr: 10791 data: 0x86820006 module: 6 information type: 8 information
index: 2 MBS data:0006 ts: 0x000000833335145A
*** MBS information: block: 16929 ptr: 8209 data: 0x86807BC5 module: 6 information type: 8 information
index: 0 MBS data:7BC5 ts: 0x000000837F39395A
*** MBS information: block: 16929 ptr: 8211 data: 0x86817D82 module: 6 information type: 8 information
index: 1 MBS data:7D82 ts: 0x000000837F39395A
*** MBS information: block: 16929 ptr: 8213 data: 0x86820006 module: 6 information type: 8 information
index: 2 MBS data:0006 ts: 0x000000837F39395A
0x67D827BC5 − 0x66A817285 = 0x13010940 = 318835008 × 20ns = 6376700160ns = 6.4s
0x837F39395A - 0x833335145A = 0x4C042500 = 1275340032 * 10ns = 12753400320ns = 12.8s => correlation clock
frequency = 25MHz *not* 50MHz
:
:
Test #3 - manual correlation reset, followed by correlation trigger ~2s later
*** MBS information: block: 17065 ptr: 15987 data: 0x86804D22 module: 6 information type: 8 information
index: 0 MBS data:4D22 ts: 0x00000083B96F05A6
*** MBS information: block: 17065 ptr: 15989 data: 0x86810344 module: 6 information type: 8 information
index: 1 MBS data:0344 ts: 0x00000083B96F05A6
*** MBS information: block: 17065 ptr: 15991 data: 0x86820000 module: 6 information type: 8 information
index: 2 MBS data:0000 ts: 0x00000083B96F05A6
0x3444d22 = 54807842 * 40ns = 2192313680ns = 2.2s
*re-check*
Correlation trigger input OK, pulse width 500ns
External clock is 50MHz
MACB master mode 6
So ... correlation clock frequency is 25MHz and Correlation DAQ Accepted Trigger not working
18.30 Per PCS request
Master MACB clock output 25Mhz
reset output not observed at DAQ GO as expected |
| Attachment 1: 20.png
|
|
| Attachment 2: 21.png
|
|
| Attachment 3: 22.png
|
|
| Attachment 4: 23.png
|
|
| Attachment 5: 24.png
|
|
|
364
|
Fri Sep 30 07:52:56 2016 |
TD | Report - low - AIDA correlation L/M/H statistics count 3x |
The AIDA correlation L, M & H statistics count 3x for each correlation trigger,
i.e. counts and rates displayed by statistics are 3x actual counts and rates |
|
367
|
Sat Oct 1 09:34:23 2016 |
TD | Installing latest (16 April 2015) MACB firmware |
Installed Xilinx 14.7 LabTools + (free) WebPack license
Start iMPACT
Connect Xilinx programmer via 14-way ribbon cable to MACB PCB jtag connector
and connect USB cable to laptop
Powerup NIM bin with MACB (serial #13)
Auto-install Xilinx programmer driver
Use iMPACT Wizard to automagically find Xilinx device of MACB
Load new *.jed file
Operations -> Program
Operations -> Verify
TTY console log
Welcome to iMPACT
iMPACT Version: 14.7
Project: C:\Xilinx\14.7\LabTools\LabTools\auto_project.ipf created.
// *** BATCH CMD : setMode -bs
// *** BATCH CMD : setMode -bs
// *** BATCH CMD : setMode -bs
// *** BATCH CMD : setMode -bs
GUI --- Auto connect to cable...
// *** BATCH CMD : setCable -port auto
INFO:iMPACT - Digilent Plugin: Plugin Version: 2.4.4
INFO:iMPACT - Digilent Plugin: no JTAG device was found.
AutoDetecting cable. Please wait.
*** WARNING ***: When port is set to auto detect mode, cable speed is set to default 6 MHz regardless of explicit arguments supplied for
setting the baud rates
PROGRESS_START - Starting Operation.
Connecting to cable (Usb Port - USB21).
Checking cable driver.
Driver file xusb_xp2.sys found.
Driver version: src=2301, dest=2301.
Driver windrvr6.sys version = 10.2.1.0. WinDriver v10.21 Jungo (c) 1997 - 2010 Build Date: Aug 31 2010 x86_64 64bit SYS 14:14:44,
version = 1021.
Cable PID = 0008.
Max current requested during enumeration is 300 mA.
Type = 0x0005.
write (count, cmdBuffer, dataBuffer) failed C0000004.
Cable Type = 3, Revision = 0.
Setting cable speed to 6 MHz.
Cable connection established.
Firmware version = 2301.
File version of C:/Xilinx/14.7/LabTools/LabTools/data/xusb_xp2.hex = 2401.
Firmware hex file version = 2401.
Downloading C:/Xilinx/14.7/LabTools/LabTools/data/xusb_xp2.hex.
Downloaded firmware version = 2401.
PLD file version = 200Dh.
PLD version = 200Dh.
PROGRESS_END - End Operation.
Elapsed time = 1 sec.
Type = 0x0005.
ESN option: 000013E7C24E01.
Attempting to identify devices in the boundary-scan chain configuration...
INFO:iMPACT - Current time: 01/10/2016 09:30:42
// *** BATCH CMD : Identify -inferir
PROGRESS_START - Starting Operation.
Identifying chain contents...'0': : Manufacturer's ID = Xilinx xc2c128, Version : 1
INFO:iMPACT:1777 -
Reading C:/Xilinx/14.7/LabTools/LabTools/xbr/data/xc2c128.bsd...
INFO:iMPACT:501 - '1': Added Device xc2c128 successfully.
----------------------------------------------------------------------
----------------------------------------------------------------------
done.
PROGRESS_END - End Operation.
Elapsed time = 6 sec.
// *** BATCH CMD : identifyMPM
// *** BATCH CMD : assignFile -p 1 -file "C:/Users/td/Local Documents/misc/macb_top_revB.jed"
'1': Loading file 'C:/Users/td/Local Documents/misc/macb_top_revB.jed' ...
done.
INFO:iMPACT:501 - '1': Added Device xc2c128 successfully.
----------------------------------------------------------------------
----------------------------------------------------------------------
INFO:iMPACT - Current time: 01/10/2016 09:32:54
// *** BATCH CMD : Program -p 1 -e -v
PROGRESS_START - Starting Operation.
Maximum TCK operating frequency for this device chain: 33000000.
Validating chain...
Boundary-scan chain validated successfully.
'1': Erasing device...
'1': Erasure completed successfully.
'1': Programming device...
done.
'1': Verifying device...
done.
'1': Setting ISC done bits...done
'1': Programming completed successfully.
PROGRESS_END - End Operation.
Elapsed time = 2 sec.
INFO:iMPACT - Current time: 01/10/2016 09:33:01
// *** BATCH CMD : Verify -p 1
PROGRESS_START - Starting Operation.
Maximum TCK operating frequency for this device chain: 33000000.
Validating chain...
Boundary-scan chain validated successfully.
'1': Verifying device...
done.
'1': Verification completed successfully.
PROGRESS_END - End Operation.
Elapsed time = 0 sec. |
| Attachment 1: Untitled.png
|
|
|
368
|
Sun Oct 2 06:12:54 2016 |
TD | Sunday 2 October |
14.15 DSSSD bias/leakage current OK - see attachments 1-2
FEE64 temperatures OK - see attachment 3
B2F/F11 temperature +27.5 deg C, d.p. +10.6 deg C, 34.3 RH %
14.24 R8 starts
ASIC settings 2016Sep02-18.17.01
Master MACB serial #13 (see https://elog.ph.ed.ac.uk/AIDA/367)
Master MACB mode 5
NIM 50MHz -> Phillips 726 NIM-TTL-NIM -> Master MACB - external clock (rear panel)
Note Fuji Diamond Tool Ltd 1020 NIM-TTL level adaptor (NIM, pink front panel) with signal
rise and falltimes quoted as <10ns produced poor quality 25/50MHz clock - the Phillips 726
has significantly faster rise and falltimes and drices a higher amplitude signal to 50R load.
BNC PB-4 trigger output -> Ortec SCA -> LeCroy 429A (500ns width) -> Master MACB - Correlation DAQ
Accepted Trigger
BNC PB-4 frequency c. 26Hz
Statistics - see attachments 4-14
Master Timestamp - see attachment 15
Correlation Status & Control - see attachment 16
Successive correlation scaler events from R8_0
*** MBS information: block: 120 ptr: 7827 data: 0x868068B4 module: 6 information type: 8 information
index: 0 MBS data:68B4 ts: 0x00000019979ADAF8
*** MBS information: block: 120 ptr: 7829 data: 0x86812F00 module: 6 information type: 8 information
index: 1 MBS data:2F00 ts: 0x00000019979ADAF8
*** MBS information: block: 120 ptr: 7831 data: 0x86820000 module: 6 information type: 8 information
index: 2 MBS data:0000 ts: 0x00000019979ADAF8
*** MBS information: block: 126 ptr: 7005 data: 0x8680FDF4 module: 6 information type: 8 information
index: 0 MBS data:FDF4 ts: 0x0000001997D52FF8
*** MBS information: block: 126 ptr: 7007 data: 0x86812F0E module: 6 information type: 8 information
index: 1 MBS data:2F0E ts: 0x0000001997D52FF8
*** MBS information: block: 126 ptr: 7009 data: 0x86820000 module: 6 information type: 8 information
index: 2 MBS data:0000 ts: 0x0000001997D52FF8
0x2F0EFDF4 - 0x2F0068B4 = 0xE9540 = 955712 * 40ns (assume 25MHz) = 38228480ns = 38.2ms
0x1997D52FF8 - 0x19979ADAF8 = 0x3A5500 = 3822848 * 10ns = 38228480ns = 38.2ms
=> correlation clock frequency = 25MHz
Master MACB (serial #13) mode = 5, 50MHz clock to external clock input (rear panel)
clock output 25MHz
reset output reset not observed at DAQ GO
reset observed using Correlation status & control tab -> Force
Correlation Reset
reset request output not tested
correlation DAQ accepted trigger output c.26Hz
Could not persuade MACB serial #13 to operate in mode 0, external clock input (rear panel) disconnected)
Persistent failures to ReSYNC by nnaida11, 12, 17, 18, 19, & 20 (see https://elog.ph.ed.ac.uk/AIDA/357)
following RESET/SETUP, FEE64 power cycle, MACB NIM bin power cycle and re-seating master MACB HDMI cables
Switch back to previous Master MACB serial #11
Master MACB (serial #11) mode = 0
clock output 25MHz
reset output reset not observed at DAQ GO
reset not observed using Correlation status & control tab -> Force
Correlation Reset
reset request output not tested
correlation DAQ accepted trigger output not observed |
| Attachment 1: 1.png
|
|
| Attachment 2: 2.png
|
|
| Attachment 3: 3.png
|
|
| Attachment 4: 4.png
|
|
| Attachment 5: 5.png
|
|
| Attachment 6: 6.png
|
|
| Attachment 7: 7.png
|
|
| Attachment 8: 8.png
|
|
| Attachment 9: 9.png
|
|
| Attachment 10: 10.png
|
|
| Attachment 11: 11.png
|
|
| Attachment 12: 12.png
|
|
| Attachment 13: 13.png
|
|
| Attachment 14: 14.png
|
|
| Attachment 15: 15.png
|
|
| Attachment 16: 16.png
|
|
|
372
|
Thu Oct 6 12:39:14 2016 |
TD | Images of setup at F11 |
|
| Attachment 1: IMG_6687.JPG
|
|
| Attachment 2: IMG_6689.JPG
|
|
| Attachment 3: IMG_6690.JPG
|
|
|
373
|
Thu Oct 6 13:09:50 2016 |
TD | Images of setup at F11 - 2 |
|
| Attachment 1: IMG_6692.JPG
|
|
| Attachment 2: IMG_6693.JPG
|
|
| Attachment 3: IMG_6694.JPG
|
|
|
375
|
Thu Oct 6 13:20:34 2016 |
TD | Images of setup at F11 - 3 |
|
| Attachment 1: IMG_6695.JPG
|
|
| Attachment 2: IMG_6696.JPG
|
|
| Attachment 3: IMG_6701.JPG
|
|
|
377
|
Thu Oct 6 13:22:24 2016 |
TD | Images of setup at F11 - 4 |
|
| Attachment 1: IMG_6702.JPG
|
|
|
387
|
Sat Oct 15 03:27:31 2016 |
TD | Saturday 15 October |
11.27 DAQ found stopped
Most recent file R42_164, last accessed 07.18
Restart merger OK
DAQ STOP OK
System-wide checks - see attachments 1-5
ASIC clock timestamp, SYNC errors fail *but* SYNC pulses OK
ReSYNC OK
System-wide checks OK
DSSSD bias & leakage currents OK - see attachments 6 & 7
FEE64 temperatures OK - see attachment 8
R42_164 ADC data synchronised
aidas1> ./analyser2 /TapeData/Sept2016/R42_164
*** GREAT format 3.2.0 analyser - TD - May 2014
*** ERROR: READ I/O error: 5002
blocks: 2444
ADC data format: 19809141 ( 1404089.4 Hz)
Other data format: 131455 ( 9317.6 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 7 ( 0.5 Hz)
RESUME: 7 ( 0.5 Hz)
SYNC100: 129011 ( 9144.4 Hz)
FEE64 disc: 2010 ( 142.5 Hz)
MBS info: 420 ( 29.8 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 21 ( 1.5 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
Timestamp elapsed time: 14.108 s
FEE module #: 1 elapsed dead time 0.000 s
FEE module #: 2 elapsed dead time 0.000 s
FEE module #: 3 elapsed dead time 0.000 s
FEE module #: 4 elapsed dead time 0.000 s
FEE module #: 5 elapsed dead time 0.334 s
FEE module #: 6 elapsed dead time 0.042 s
FEE module #: 7 elapsed dead time 0.000 s
FEE module #: 8 elapsed dead time 0.000 s
FEE module #: 9 elapsed dead time 0.000 s
FEE module #: 10 elapsed dead time 0.000 s
FEE module #: 11 elapsed dead time 0.000 s
FEE module #: 12 elapsed dead time 0.000 s
FEE module #: 13 elapsed dead time 0.000 s
FEE module #: 14 elapsed dead time 0.000 s
FEE module #: 15 elapsed dead time 0.000 s
FEE module #: 16 elapsed dead time 0.000 s
FEE module #: 17 elapsed dead time 0.000 s
FEE module #: 18 elapsed dead time 0.005 s
FEE module #: 19 elapsed dead time 0.000 s
FEE module #: 20 elapsed dead time 0.000 s
FEE module #: 21 elapsed dead time 0.000 s
FEE module #: 22 elapsed dead time 0.000 s
FEE module #: 23 elapsed dead time 0.000 s
FEE module #: 24 elapsed dead time 0.000 s
FEE module #: 25 elapsed dead time 0.000 s
FEE module #: 26 elapsed dead time 0.000 s
FEE module #: 27 elapsed dead time 0.000 s
FEE module #: 28 elapsed dead time 0.000 s
FEE module #: 29 elapsed dead time 0.000 s
FEE module #: 30 elapsed dead time 0.000 s
FEE module #: 31 elapsed dead time 0.000 s
FEE module #: 32 elapsed dead time 0.000 s
*** Program elapsed time: 0.586s ( 4171.093 blocks/s, 260.693 Mb/s)
11.39 DAQ starts (R43)
12.07 Disk housekeeping caused TapeServer problem
DAQ stopped OK after some minutes and restarting merger
12.43 DAQ starts (R46)
18.26 DAQ stalled
Latest file R46_109 (most recent access 18.26)
Merger error messages - see attachment 9
Restart merger - error messages - see attachment 10
Analysis of R46_109
ADC data synchronised
aidas1> ./analyser2 /TapeData/Sept2016/R46_109
*** GREAT format 3.2.0 analyser - TD - May 2014
*** ERROR: READ I/O error: 5002
blocks: 4688
ADC data format: 37985730 ( 1342216.8 Hz)
Other data format: 263662 ( 9316.4 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 25 ( 0.9 Hz)
RESUME: 25 ( 0.9 Hz)
SYNC100: 258788 ( 9144.2 Hz)
FEE64 disc: 3981 ( 140.7 Hz)
MBS info: 843 ( 29.8 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 41 ( 1.4 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
Timestamp elapsed time: 28.301 s
FEE module #: 1 elapsed dead time 0.000 s
FEE module #: 2 elapsed dead time 0.013 s
FEE module #: 3 elapsed dead time 0.000 s
FEE module #: 4 elapsed dead time 0.000 s
FEE module #: 5 elapsed dead time 0.298 s
FEE module #: 6 elapsed dead time 0.200 s
FEE module #: 7 elapsed dead time 0.000 s
FEE module #: 8 elapsed dead time 0.000 s
FEE module #: 9 elapsed dead time 0.001 s
FEE module #: 10 elapsed dead time 0.000 s
FEE module #: 11 elapsed dead time 0.000 s
FEE module #: 12 elapsed dead time 0.000 s
FEE module #: 13 elapsed dead time 0.000 s
FEE module #: 14 elapsed dead time 0.003 s
FEE module #: 15 elapsed dead time 0.000 s
FEE module #: 16 elapsed dead time 0.000 s
FEE module #: 17 elapsed dead time 0.086 s
FEE module #: 18 elapsed dead time 0.054 s
FEE module #: 19 elapsed dead time 0.097 s
FEE module #: 20 elapsed dead time 0.000 s
FEE module #: 21 elapsed dead time 0.000 s
FEE module #: 22 elapsed dead time 0.001 s
FEE module #: 23 elapsed dead time 0.000 s
FEE module #: 24 elapsed dead time 0.000 s
FEE module #: 25 elapsed dead time 0.000 s
FEE module #: 26 elapsed dead time 0.000 s
FEE module #: 27 elapsed dead time 0.000 s
FEE module #: 28 elapsed dead time 0.000 s
FEE module #: 29 elapsed dead time 0.000 s
FEE module #: 30 elapsed dead time 0.000 s
FEE module #: 31 elapsed dead time 0.000 s
FEE module #: 32 elapsed dead time 0.000 s
*** Program elapsed time: 2.102s ( 2230.721 blocks/s, 139.420 Mb/s)
11.00 Sunday 16 October
DAQ STOP requested - nnaida1-24 all stop *except* nnaida6 which remains going - Run Control displays RESET
only
DAQ RESET
System-wide checks all OK
Normal setup and DAQ restart (see https://elog.ph.ed.ac.uk/AIDA/388 )
DAQ SETUP
|
| Attachment 1: 20.png
|
|
| Attachment 2: 21.png
|
|
| Attachment 3: 22.png
|
|
| Attachment 4: 23.png
|
|
| Attachment 5: 24.png
|
|
| Attachment 6: 25.png
|
|
| Attachment 7: 26.png
|
|
| Attachment 8: 27.png
|
|
| Attachment 9: 30.png
|
|
| Attachment 10: 31.png
|
|
|
388
|
Sun Oct 16 03:14:41 2016 |
TD | Sunday 16 October |
11.12 DAQ start (R47)
Detector bias and leakage currents OK - see attachments 1 & 2
FEE64 temperatures OK - see attachment 3
Statistics - see attachments 4-14
11.56 Temperature +25.0 deg C, d.p. +8.4 deg C, RH 34.5%
20.33 DAQ found stopped
Most recent file R47_120 (last access 18.26)
R47_120 ADC data synchronised
Restart merger and request DAQ STOP - DAQ stops OK
System-wide tests SYNC errors and SYNC pulses - see attachments 15 & 16 - all other tests OK
DAQ RESET & SETUP OK
Normal restart which failed shortly after enabling ASIC readout - restarting merger did not
recover DAQ control - power cycle required
21.05 DAQ starts (R49)
|
| Attachment 1: 1.png
|
|
| Attachment 2: 2.png
|
|
| Attachment 3: 3.png
|
|
| Attachment 4: 4.png
|
|
| Attachment 5: 5.png
|
|
| Attachment 6: 6.png
|
|
| Attachment 7: 7.png
|
|
| Attachment 8: 8.png
|
|
| Attachment 9: 9.png
|
|
| Attachment 10: 10.png
|
|
| Attachment 11: 11.png
|
|
| Attachment 12: 12.png
|
|
| Attachment 13: 13.png
|
|
| Attachment 14: 14.png
|
|
| Attachment 15: 15.png
|
|
| Attachment 16: 16.png
|
|
|
390
|
Mon Oct 17 04:10:30 2016 |
TD | Monday 17 October |
12.05 DAQ found stopped
Most recent file R49_226, last accessed 06.26
Restart merger OK
DAQ STOP OK
System-wide checks - see attachments 1-2
SYNC errors and SYNC pulses fail
ReSYNC OK
System-wide checks OK
Restart merger & tapeserver
DSSSD bias & leakage currents OK - see attachments 3 & 4
FEE64 temperatures OK - see attachment 5
DAQ start
#1 nnaida15 data transfer #1 tag drops at DAQ GO
disable H & data transfer #1
restart merger & tapeserver
enable H & data transfer #1
12.22 #2 DAQ start (R51) OK
20.33 DAQ found stopped
Most recent file R51_145, last accessed 18.30
Restart merger OK
DAQ STOP OK
System-wide checks - see attachments 6 & 7
SYNC errors and SYNC pulses fail
DAQ RESET & SETUP OK
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
20.41 DAQ GO (R52) |
| Attachment 1: 20.png
|
|
| Attachment 2: 21.png
|
|
| Attachment 3: 23.png
|
|
| Attachment 4: 24.png
|
|
| Attachment 5: 26.png
|
|
| Attachment 6: 27.png
|
|
| Attachment 7: 28.png
|
|
|
391
|
Tue Oct 18 01:43:04 2016 |
TD | Tuesday 18 October |
09/42 DAQ found stopped, no merger error messages
Most recent file R52_211, last accessed 06.30
Restart merger OK
DAQ STOP OK
aidas1> ./analyser2 /TapeData/Sept2016/R52_211
*** GREAT format 3.2.0 analyser - TD - May 2014
*** ERROR: READ I/O error: 5002
blocks: 7619
ADC data format: 61809254 ( 1624588.3 Hz)
Other data format: 354167 ( 9308.9 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 28 ( 0.7 Hz)
RESUME: 28 ( 0.7 Hz)
SYNC100: 347607 ( 9136.5 Hz)
FEE64 disc: 5364 ( 141.0 Hz)
MBS info: 1140 ( 30.0 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 88 ( 2.3 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
Timestamp elapsed time: 38.046 s
FEE module #: 1 elapsed dead time 0.000 s
FEE module #: 2 elapsed dead time 0.011 s
FEE module #: 3 elapsed dead time 0.000 s
FEE module #: 4 elapsed dead time 0.000 s
FEE module #: 5 elapsed dead time 0.058 s
FEE module #: 6 elapsed dead time 0.165 s
FEE module #: 7 elapsed dead time 0.000 s
FEE module #: 8 elapsed dead time 0.000 s
FEE module #: 9 elapsed dead time 0.000 s
FEE module #: 10 elapsed dead time 0.000 s
FEE module #: 11 elapsed dead time 0.000 s
FEE module #: 12 elapsed dead time 0.000 s
FEE module #: 13 elapsed dead time 0.000 s
FEE module #: 14 elapsed dead time 0.002 s
FEE module #: 15 elapsed dead time 0.055 s
FEE module #: 16 elapsed dead time 0.000 s
FEE module #: 17 elapsed dead time 0.057 s
FEE module #: 18 elapsed dead time 0.089 s
FEE module #: 19 elapsed dead time 1.021 s
FEE module #: 20 elapsed dead time 0.000 s
FEE module #: 21 elapsed dead time 0.000 s
FEE module #: 22 elapsed dead time 0.000 s
FEE module #: 23 elapsed dead time 0.085 s
FEE module #: 24 elapsed dead time 0.004 s
FEE module #: 25 elapsed dead time 0.000 s
FEE module #: 26 elapsed dead time 0.000 s
FEE module #: 27 elapsed dead time 0.000 s
FEE module #: 28 elapsed dead time 0.000 s
FEE module #: 29 elapsed dead time 0.000 s
FEE module #: 30 elapsed dead time 0.000 s
FEE module #: 31 elapsed dead time 0.000 s
FEE module #: 32 elapsed dead time 0.000 s
*** Program elapsed time: 1.793s ( 4249.377 blocks/s, 265.586 Mb/s)
R51_211 ADC data synchronised
DSSSD bias & leakage currents OK - see attachments 1 & 2
FEE64 temperatures OK - see attachment 3
System-wide checks - see attachments 4 & 5
SYNC errors and SYNC pulses fail, all other tests OK
Statistics - see attachments 6-16
DAQ RESET & SETUP OK
ReSYNC OK
System-wide checks OK
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
10.02 DAQ start (R53)
18.20 DAQ OK
System-wide checks *all* OK
Statistics - see attachments 17-27
18.30 DAQ stalled on schedule
Most recent file R53_193 last accessed 18.30
AIDA SYNC rates zero in large fraction of FEE64s - see attachment 28
SYNC errors and SYNC pulses fail - see attachments 29-32
DAQ STOP
DAQ RESET
Restart merger
DAQ RESET OK
DAQ SETUP OK
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
System-wide checks *all* OK
18.45 DAQ starts (R54)
merging stops shortly after start of run - see attachment 33
DAQ HALT, RESET, SETUP OK
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
18.52 DAQ starts (R55) |
| Attachment 1: 1.png
|
|
| Attachment 2: 2.png
|
|
| Attachment 3: 3.png
|
|
| Attachment 4: 4.png
|
|
| Attachment 5: 5.png
|
|
| Attachment 6: 6.png
|
|
| Attachment 7: 7.png
|
|
| Attachment 8: 8.png
|
|
| Attachment 9: 9.png
|
|
| Attachment 10: 10.png
|
|
| Attachment 11: 11.png
|
|
| Attachment 12: 12.png
|
|
| Attachment 13: 13.png
|
|
| Attachment 14: 14.png
|
|
| Attachment 15: 15.png
|
|
| Attachment 16: 16.png
|
|
| Attachment 17: 20.png
|
|
| Attachment 18: 21.png
|
|
| Attachment 19: 22.png
|
|
| Attachment 20: 23.png
|
|
| Attachment 21: 24.png
|
|
| Attachment 22: 25.png
|
|
| Attachment 23: 26.png
|
|
| Attachment 24: 27.png
|
|
| Attachment 25: 28.png
|
|
| Attachment 26: 29.png
|
|
| Attachment 27: 30.png
|
|
| Attachment 28: 31.png
|
|
| Attachment 29: 32.png
|
|
| Attachment 30: 33.png
|
|
| Attachment 31: 34.png
|
|
| Attachment 32: 35.png
|
|
| Attachment 33: 36.png
|
|
|
393
|
Tue Oct 18 06:56:52 2016 |
TD | DAQ status |
From the installation of the RG58 cable for the 50MHz TTL clock from BRIKEN DAQ
to AIDA Master MACB external clock input we observe DAQ stalls at c. 06.30 and
18.30 daily. This coincides with the nominal start times of the ORNL Clover detector
LN2 fill cycles.
run start stop files recovery
AIDA Master MACB internal clock
R1 28.9.16 15.02 11.20 R1_304 OK
RG174 cable from BRIKEN DAQ 50MHz clock (TTL) to AIDA Master MACB external clock input
9.10.16 16.55 17.30 no data transfer, TS_SYNC phase = 0, OK
9.10.16 17.40 19.15 est no data transfer, TS_SYNC phase = 0, ReSYNC
9.10.16 22.20 01.40 no data transfer, TS_SYNC phase = 1, ?
10.10.16 14.20 16.50 no data transfer, TS_SYNC phase = 2, ?
10.10.16 18.35 18.40/19.40 no data transfer, TS_SYNC phase = 3, some FEE64s stopped later than others, ReSYNC
10.10.16 20.36 06.35 no data transfer, TS_SYNC phase = 0, power cycle
RG174 cable from BRIKEN DAQ 50MHz clock (TTL) to AIDA Master MACB external clock input
replaced by RG58 cable
run start stop files recovery
R17 11.10.16 15.46 08.22 R17_440 soft reset
R21 12.10.16 14.05 15.30 power dip - power cycle
R22 12.10.16 16.20 19.05 OK
R24 13.10.16 OK
R25 13.10.16 OK
R26 13.10.16 OK
R27 13.10.16 OK
R28 13.10.16 OK
R29 13.10.16 OK
R32 13.10.16 21.20 22.34 R32_26 soft reset
R33 14.10.16 00.01 00.25 ? power cycle
R40 14.10.16 01.49 06.26 R40_91 soft reset
R41 14.10.16 OK
R42 15.10.16 22.45 07.18 R42_164 System-wide tests - SYNC errors fail *but* SYNC pulses OK - ReSYNC only
R43 15.10.16 11.39 12.43 disk housekeeping caused merger/tapeserver problem
R46 15.10.16 12.43 18.26 R46_109 soft reset
R47 16.10.16 11.12 18.26 R47_120 power cycle
R49 16.10.16 21.05 06.26 R49_226 soft reset
R51 17.10.16 12.21 18.30 R51_145 soft reset
R52 17.10.16 20.41 06.30 R52_211 soft reset
R53 18.10.16 10.02 18.30 R53_193 soft reset
R55 18.10.16 18.52 19.17 R55_10 soft reset
R60 19.10.16 09.35 16.28 R60_169 OK
R61 19.10.16 16.30 18.26 R61_69 power cycle
R63 19.10.16 20.05 06.46 R63_291 soft reset
R64 20.10.16 09.45 18.25 R64_240 OK - System-wide tests - SYNC errors fail & SYNC pulses fail - other checks OK
R65 20.10.16 18.49 19.17 R65_12 power cycle
R66 20.10.16 20.02 10.55 R66_396 soft reset auto LN2 fill may have failed, manually selected c. 11.00
R67 21.10.16 14.02 18.26 R67_121 soft reset
R68 21.10.16 18.40 18.44 R68_1 soft reset
R70 21.10.16 18.53 06.46 R70_315 soft reset
R71 22.10.16 10.42 18.25 OK
R73 22.10.16 19.07 19.17 R73_4 soft reset
R74 22.10.16 19.38 06.31 R74_296 soft reset
R76 23.10.16 10.46 11.52 R76_29 merger crashed, soft reset
R77 23.10.16 18.50 19.17 disk housekeeping caused merger/tapeserver problem
R78 23.10.16 19.47 06.46 R78_291 soft reset
R79 24.10.16 11.07 18.46 R79_182 soft reset
R80 24.10.16 18.56 19.17 R80_5 DAQ stall, soft reset
R81 24.10.16 19.29 06.26 R81_169 soft reset
R82 25.10.16 10.22 18.30 R82_125 -> safe state
R83 27.10.16 14.35 18.30 R83_125 soft reset
R86 27.10.16 20.48 06.26 R86_245 soft reset
R87 28.10.16 14.34 18.47 R87_274 soft reset
R89 29.10.16 19.10 07.17 R89_102 power cycle
soft reset => no FEE64 power cycle
Restart merger
DAQ STOP
System wide checks
DAQ RESET & SETUP
System-wide checks
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1 |
|
394
|
Wed Oct 19 01:14:10 2016 |
TD | Wednesday 19 October |
09.13 DAQ found stopped, no merger error messages
Most recent file R55_10, last accessed 19.17
Restart merger OK
DAQ STOP OK
aidas1> ./analyser2 /TapeData/Sept2016/R55_10
*** GREAT format 3.2.0 analyser - TD - May 2014
*** ERROR: READ I/O error: 5002
blocks: 23037
ADC data format: 187091367 ( 2006959.5 Hz)
Other data format: 867516 ( 9306.0 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 62 ( 0.7 Hz)
RESUME: 62 ( 0.7 Hz)
SYNC100: 851796 ( 9137.4 Hz)
FEE64 disc: 12836 ( 137.7 Hz)
MBS info: 2760 ( 29.6 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 124 ( 1.3 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
Timestamp elapsed time: 93.221 s
FEE module #: 1 elapsed dead time 0.000 s
FEE module #: 2 elapsed dead time 0.000 s
FEE module #: 3 elapsed dead time 0.000 s
FEE module #: 4 elapsed dead time 0.000 s
FEE module #: 5 elapsed dead time 0.922 s
FEE module #: 6 elapsed dead time 1.113 s
FEE module #: 7 elapsed dead time 0.017 s
FEE module #: 8 elapsed dead time 0.000 s
FEE module #: 9 elapsed dead time 0.001 s
FEE module #: 10 elapsed dead time 0.000 s
FEE module #: 11 elapsed dead time 0.000 s
FEE module #: 12 elapsed dead time 0.000 s
FEE module #: 13 elapsed dead time 0.000 s
FEE module #: 14 elapsed dead time 0.005 s
FEE module #: 15 elapsed dead time 0.016 s
FEE module #: 16 elapsed dead time 0.000 s
FEE module #: 17 elapsed dead time 0.446 s
FEE module #: 18 elapsed dead time 0.435 s
FEE module #: 19 elapsed dead time 0.925 s
FEE module #: 20 elapsed dead time 0.015 s
FEE module #: 21 elapsed dead time 0.000 s
FEE module #: 22 elapsed dead time 0.000 s
FEE module #: 23 elapsed dead time 0.221 s
FEE module #: 24 elapsed dead time 0.000 s
FEE module #: 25 elapsed dead time 0.000 s
FEE module #: 26 elapsed dead time 0.000 s
FEE module #: 27 elapsed dead time 0.000 s
FEE module #: 28 elapsed dead time 0.000 s
FEE module #: 29 elapsed dead time 0.000 s
FEE module #: 30 elapsed dead time 0.000 s
FEE module #: 31 elapsed dead time 0.000 s
FEE module #: 32 elapsed dead time 0.000 s
*** Program elapsed time: 22.266s ( 1034.644 blocks/s, 64.665 Mb/s)
R55_10 ADC data synchronised
DSSSD bias & leakage currents OK - see attachments 1 & 2
FEE64 temperatures OK - see attachment 3
SYNC statistics, System Wide Checks SYNC error & SYNC pulses fail - see attachments 4-8
DAQ RESET & SETUP
System-wide checks
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
#1 DAQ start (R58)
nnaida7 data transfer #1 tag (X) drops at DAQ GO
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
#2 DAQ start (R59)
nnaida15 data transfer #1 tag (X) drops at DAQ GO
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
09.35 DAQ start (R60)
16.28 DAQ stop OK
Final file R60_169
16.30 DAQ start (R61)
BNC PB-4 pulser OFF
Background run - files R61_0 - R61_31
+ 60Co (ID# 0115) - files R61_33 - R61_49
60Co source placed on top of AIDA snout ~ above DSSSD #3
DAQ stalled at 18.28 (during R61_49) - as expected! See https://elog.ph.ed.ac.uk/AIDA/393
DAQ RESET & SETUP
System-wide checks
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
20.05 DAQ start (R63)
BNC PB-4 pulser ON, 60Co source removed |
| Attachment 1: 1.png
|
|
| Attachment 2: 2.png
|
|
| Attachment 3: 3.png
|
|
| Attachment 4: 4.png
|
|
| Attachment 5: 5.png
|
|
| Attachment 6: 6.png
|
|
| Attachment 7: 7.png
|
|
| Attachment 8: 8.png
|
|
|
395
|
Thu Oct 20 01:30:10 2016 |
TD | Thursday 20 October |
09.30 DAQ found stopped, no merger error messages
Most recent file R63_291, last accessed 06.46
R63_291 ADC data synchronised
aidas1> ./analyser2 /TapeData/Sept2016/R63_291
*** GREAT format 3.2.0 analyser - TD - May 2014
*** ERROR: READ I/O error: 5002
blocks: 29203
ADC data format: 237170622 ( 2006571.1 Hz)
Other data format: 1096655 ( 9278.2 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 113 ( 1.0 Hz)
RESUME: 113 ( 1.0 Hz)
SYNC100: 1077835 ( 9119.0 Hz)
FEE64 disc: 15061 ( 127.4 Hz)
MBS info: 3533 ( 29.9 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 175 ( 1.5 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
Timestamp elapsed time: 118.197 s
FEE module #: 1 elapsed dead time 0.000 s
FEE module #: 2 elapsed dead time 0.000 s
FEE module #: 3 elapsed dead time 0.000 s
FEE module #: 4 elapsed dead time 0.000 s
FEE module #: 5 elapsed dead time 2.763 s
FEE module #: 6 elapsed dead time 0.275 s
FEE module #: 7 elapsed dead time 0.004 s
FEE module #: 8 elapsed dead time 0.003 s
FEE module #: 9 elapsed dead time 0.001 s
FEE module #: 10 elapsed dead time 0.000 s
FEE module #: 11 elapsed dead time 0.000 s
FEE module #: 12 elapsed dead time 0.000 s
FEE module #: 13 elapsed dead time 0.000 s
FEE module #: 14 elapsed dead time 0.013 s
FEE module #: 15 elapsed dead time 0.028 s
FEE module #: 16 elapsed dead time 0.275 s
FEE module #: 17 elapsed dead time 6.010 s
FEE module #: 18 elapsed dead time 0.382 s
FEE module #: 19 elapsed dead time 0.555 s
FEE module #: 20 elapsed dead time 0.378 s
FEE module #: 21 elapsed dead time 0.000 s
FEE module #: 22 elapsed dead time 0.035 s
FEE module #: 23 elapsed dead time 0.168 s
FEE module #: 24 elapsed dead time 0.043 s
FEE module #: 25 elapsed dead time 0.000 s
FEE module #: 26 elapsed dead time 0.000 s
FEE module #: 27 elapsed dead time 0.000 s
FEE module #: 28 elapsed dead time 0.000 s
FEE module #: 29 elapsed dead time 0.000 s
FEE module #: 30 elapsed dead time 0.000 s
FEE module #: 31 elapsed dead time 0.000 s
FEE module #: 32 elapsed dead time 0.000 s
*** Program elapsed time: 28.188s ( 1036.027 blocks/s, 64.752 Mb/s)
DSSSD bias & leakage currents OK - see attachments 1 & 2
FEE64 temperatures OK - see attachment 3
System-wide checks - SYNC errors & SYNC pulses fail - other checks OK - see attachments 4-7
AIDA SYNC stats - see attachment 8
Restart merger OK
DAQ STOP OK
DAQ RESET & SETUP OK
ReSYNC OK
System-wide checks OK
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
09.43 DAQ starts (R64)
18.25 DAQ stop OK
Stop before 'scheduled' 18.30 DAQ stall
System wide checks - SYNC count & SYNC errors fail - other checks OK - see attachments 8-11
18.49 DAQ start (R65)
N.B. normal DAQ start - no DAQ RESET, SETUP, restart of merger & tapeserver etc
18.53 SYNC error counter baseline
19.21 DAQ found stalled
Most recent files R65_12 last access 19.17
Soft reset failed - nnaida4 connect timeouts - power cycle FEE64s
20.02 DAQQ start (R66) |
| Attachment 1: 1.png
|
|
| Attachment 2: 2.png
|
|
| Attachment 3: 3.png
|
|
| Attachment 4: 4.png
|
|
| Attachment 5: 5.png
|
|
| Attachment 6: 6.png
|
|
| Attachment 7: 7.png
|
|
| Attachment 8: 8.png
|
|
| Attachment 9: 10.png
|
|
| Attachment 10: 11.png
|
|
| Attachment 11: 12.png
|
|
| Attachment 12: 13.png
|
|
|
397
|
Fri Oct 21 01:41:54 2016 |
TD | Friday 21 October |
09.38 DAQ (R66) continues OK
Detector bias and leakage currents OK - see attachments 1&2
FEE64 temperatures OK - see attachment 3
System wide checks SYNC count & SYNC errors fail - see attachments 4-7 - other checks OK
R66_367 ADC data synchronised
AIDA SYNC stats - attachment 8
Some merger errors - see attachment 9
13.53 DAQ found stalled
Most recent files R66_396 last accessed 10.55
System wide checks SYNC count & SYNC errors fail - see attachments 10-13 - other checks OK
AIDA SYNC stats - attachment 14
No further merger error messages
Restart merger OK
DAQ STOP OK
DAQ RESET & SETUP OK
ReSYNC OK
System-wide checks OK
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
14.02 DAQ starts (R67)
18.27 DAQ found stalled
Most recent file R67_121 last accessed 18.26
Restart merger
DAQ STOP
System wide checks
DAQ RESET & SETUP
System-wide checks
18.39 DAQ start (R68)
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
18.44 DAQ stalled
Most recent file R68_1 last accessed 18.44
Restart merger
DAQ STOP
System wide checks
DAQ RESET & SETUP
System-wide checks
18.53 DAQ starts (R70) |
| Attachment 1: 1.png
|
|
| Attachment 2: 2.png
|
|
| Attachment 3: 3.png
|
|
| Attachment 4: 4.png
|
|
| Attachment 5: 5.png
|
|
| Attachment 6: 6.png
|
|
| Attachment 7: 7.png
|
|
| Attachment 8: 8.png
|
|
| Attachment 9: 9.png
|
|
| Attachment 10: 10.png
|
|
| Attachment 11: 11.png
|
|
| Attachment 12: 12.png
|
|
| Attachment 13: 13.png
|
|
| Attachment 14: 14.png
|
|
|
398
|
Sat Oct 22 02:32:37 2016 |
TD | Saturday 22 October |
10.32 DAQ found stopped, no merger error messages
Most recent file R70_315, last accessed 06.46
R70_315 ADC data synchronised
DSSSD bias & leakage currents OK - see attachments 1 & 2
FEE64 temperatures OK - see attachment 3
System-wide checks - SYNC errors & SYNC pulses fail - other checks OK - see attachments 4-7
AIDA SYNC stats - see attachment 8
Restart merger OK
DAQ STOP OK
DAQ RESET & SETUP OK
ReSYNC OK
System-wide checks OK
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
10.42 DAQ starts (R71)
18.25 DAQ stop OK
System wide test all OK *except* SYNC errors - see attachments 8-11
New SYNC error baseline
18.54 Repeat system wide checks whilst DAQ is still halted - SYNC errors *and* SYNC pulse now fail - see
attachments 12-15
This may suggest that the clock signal itself is affected by LN2 fill cycle
DAQ RESET & SETUP OK
ReSYNC OK
System-wide checks OK
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
18.59 DAQ starts (R72)
nnaida15 drops data transfer #1 tag at DAQ GO
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
19.07 DAQ starts (R73)
19.17 DAQ stalled
Restart merger OK
DAQ STOP OK
Syten wide checks - SYNC error and SYNC pulse fail - dissimilar FEE64 counts in SYNC pulse
DAQ RESET & SETUP OK
ReSYNC OK
System-wide checks OK
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
19.37 DAQ starts (R74)
|
| Attachment 1: 1.png
|
|
| Attachment 2: 2.png
|
|
| Attachment 3: 3.png
|
|
| Attachment 4: 4.png
|
|
| Attachment 5: 5.png
|
|
| Attachment 6: 6.png
|
|
| Attachment 7: 7.png
|
|
| Attachment 8: 10.png
|
|
| Attachment 9: 11.png
|
|
| Attachment 10: 12.png
|
|
| Attachment 11: 13.png
|
|
| Attachment 12: 20.png
|
|
| Attachment 13: 21.png
|
|
| Attachment 14: 22.png
|
|
| Attachment 15: 23.png
|
|
|
399
|
Sun Oct 23 02:34:12 2016 |
TD | Sunday 23 October |
10.34 DAQ found stopped, no merger error messages
Most recent file R74_296, last accessed 06.31
R70_315 ADC data synchronised
DSSSD bias & leakage currents OK - see attachments 1 & 2
FEE64 temperatures OK - see attachment 3
Some merger errors - see attachment 4
Restart merger OK
DAQ STOP OK
DAQ RESET & SETUP OK
ReSYNC OK
System-wide checks OK
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
10.42 DAQ start (R75)
nnaida11 drops data transfer #1 tag at SAQ GO
DAQ STOP OK
DAQ RESET & SETUP OK
ReSYNC OK
System-wide checks OK
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
10.46 DAQ start (R76)
18.18 DAQ found stopped
Most recent file R76_29 last accessed 11.52
Merger crashed - see attachment 5
aidas1> ./analyser /TapeData//Sept2016/R76_29
*** GREAT format 3.2.2 analyser - TD - October 2016
*** ERROR: READ I/O error: 5002
blocks: 24384
ADC data format: 197754460 ( 1539809.2 Hz)
Other data format: 1194596 ( 9301.7 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 74 ( 0.6 Hz)
RESUME: 74 ( 0.6 Hz)
SYNC100: 1172671 ( 9131.0 Hz)
FEE64 disc: 18000 ( 140.2 Hz)
MBS info: 3777 ( 29.4 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 190 ( 1.5 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
*** Timestamp elapsed time: 128.428 s
FEE elapsed dead time(s)
1 0.000
2 0.007
3 0.000
4 0.000
5 0.674
6 2.583
7 0.000
8 0.000
9 0.000
10 0.000
11 0.000
12 0.000
13 0.000
14 0.000
15 0.276
16 0.124
17 0.435
18 0.050
19 0.622
20 0.009
21 0.000
22 0.000
23 0.352
24 0.000
25 0.000
26 0.000
27 0.000
28 0.000
29 0.000
30 0.000
31 0.000
32 0.000
*** Statistics
FEE ADC Data Other Data Sample Undefined Pause Resume SYNC100 Disc MBS
Other HEC Data
0 0 0 0 0 0 0 0 0 0
0 0
1 566974 49807 0 0 0 0 48983 824 0
0 62
2 5617753 50240 0 0 2 2 48984 1252 0
0 0
3 4298949 49615 0 0 0 0 48983 632 0
0 0
4 1980475 48985 0 0 0 0 48985 0 0
0 0
5 31381665 53750 0 0 12 12 48726 5000 0
0 0
6 35397222 52131 0 0 12 12 48004 326 3777
0 0
7 3679182 48987 0 0 0 0 48987 0 0
0 0
8 3893833 49613 0 0 0 0 48983 630 0
0 0
9 4909194 49313 0 0 0 0 48987 326 0
0 0
10 4099868 50957 0 0 0 0 48984 1973 0
0 0
11 1590517 49623 0 0 0 0 48987 636 0
0 0
12 2103665 48985 0 0 0 0 48985 0 0
0 0
13 3909540 50385 0 0 0 0 48990 1395 0
0 0
14 5011678 49344 0 0 0 0 48988 356 0
0 0
15 8074527 48904 0 0 10 10 48883 1 0
0 2
16 8822855 48949 0 0 5 5 48939 0 0
0 0
17 11581998 49241 0 0 10 10 48820 401 0
0 0
18 17158740 50227 0 0 1 1 48968 1257 0
0 0
19 14684270 49399 0 0 11 11 48747 630 0
0 0
20 6793305 47946 0 0 1 1 47944 0 0
0 0
21 4052765 50377 0 0 0 0 48986 1391 0
0 126
22 1773964 49339 0 0 0 0 48988 351 0
0 0
23 10790840 48872 0 0 10 10 48852 0 0
0 0
24 5580681 49607 0 0 0 0 48988 619 0
0 0
25 0 0 0 0 0 0 0 0 0
0 0
26 0 0 0 0 0 0 0 0 0
0 0
27 0 0 0 0 0 0 0 0 0
0 0
28 0 0 0 0 0 0 0 0 0
0 0
29 0 0 0 0 0 0 0 0 0
0 0
30 0 0 0 0 0 0 0 0 0
0 0
31 0 0 0 0 0 0 0 0 0
0 0
32 0 0 0 0 0 0 0 0 0
0 0
*** Timewarps
FEE ADC Pause Resume SYNC100 Disc MBS Undef-d Samples
0 0 0 0 0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 0 0 0 0 0
2 0 0 0 0 0 0 0 0 0 0 0 0
3 0 0 0 0 0 0 0 0 0 0 0 0
4 0 0 0 0 0 0 0 0 0 0 0 0
5 0 0 0 0 0 0 0 0 0 0 0 0
6 0 0 0 0 0 0 0 0 0 0 0 0
7 0 0 0 0 0 0 0 0 0 0 0 0
8 0 0 0 0 0 0 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0 0 0 0 0
11 0 0 0 0 0 0 0 0 0 0 0 0
12 0 0 0 0 0 0 0 0 0 0 0 0
13 0 0 0 0 0 0 0 0 0 0 0 0
14 0 0 0 0 0 0 0 0 0 0 0 0
15 0 0 0 0 0 0 0 0 0 0 0 0
16 0 0 0 0 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0 0 0 0
*** Program elapsed time: 3.586s ( 6799.896 blocks/s, 424.993 Mb/s)
Restart merger OK
DAQ STOP OK
DAQ RESET & SETUP OK
ReSYNC OK
System-wide checks OK
Disable H & data transfer #1
Restart merger & tapeserver
Enable H & data transfer #1
18.50 DAQ starts (R77)
19.17 DAQ stalled due to disk housekeeping
Restart merger OK
DAQ STOP fails - nnaida1 timeout
Power cycle FEE64s
19.47 DAQ starts (R78) |
| Attachment 1: 1.png
|
|
| Attachment 2: 2.png
|
|
| Attachment 3: 3.png
|
|
| Attachment 4: 4.png
|
|
| Attachment 5: 5.png
|
|