We have observed a number of AIDA FEE64s stop producing data.
Summary of dates & (approximate) times
aida03 21.4.24 08.20
aida04 22.4.24 09.14 https://elog.ph.ed.ac.uk/DESPEC/586
aida01 22.4.24 16.00
aida02 22.4.24 22.00
aida02 23.4.24 10.21
aida02 23.4.24 11.25
aida04 23.4.24 16.53? attachment 2
aida04/aida05 25.4.24 22.28 no messages in aida04 system console
aida04 27.8.24 c.23:00
aida04 28.8.24 c. 12-13:00 power cycle
aida04 28.8.24 c. 16.00 reset per Elog 618
aida02 28.4.24 c. 22.20 reset per Elog 618, no error messages in system console
aida04 29.4.24 01.15
aida04 30.4.24 <03.30
aida02 30.4.24 <10.00
aida02 2.5.24 c. 19.30 reset
In future, it would be useful to copy system console reports to the Elog to check whether we can identify common error messages etc.
ssh pi@nnrpi1
<password>
cd logs
cat /ttyUSBx
The current mapping of USB port # to AIDA FEE64s - attachment 1
aida02 system console c. 19.30 2.5.24
02:05:24/14:40:51|executing generic doStop
02:05:24/19:27:28|Halt (1)
02:05:24/19:27:29|Halt (2)
02:05:24/19:27:29|Halt (3)
02:05:24/19:27:30|Halt (4)
02:05:24/19:27:30|Halt (5)
02:05:24/19:27:31|Halt (6)
02:05:24/19:27:31|Halt (7)
02:05:24/19:27:32|Halt (8)
02:05:24/19:27:32|Halt (9)
02:05:24/19:27:33|Halt (10)
02:05:24/19:27:33|Halt (11)
02:05:24/19:27:34|Halt (12)
02:05:24/19:27:34|Halt (13)
02:05:24/19:27:35|Halt (14)
02:05:24/19:27:35|Halt (15)
02:05:24/19:27:36|Halt (16)
02:05:24/19:27:36|Halt (17)
02:05:24/19:27:37|Halt (18)
02:05:24/19:27:38|Halt (19)
02:05:24/19:27:38|Action has not completed
02:05:24/19:27:39|Giving up waiting
02:05:24/19:27:39|disconnect xfer stream #0 (1)
02:05:24/19:27:39|disconnect xfer stream #0 (2)
02:05:24/19:27:39|disconnect xfer stream #0 (3)
02:05:24/19:27:40|disconnect xfer stream #0 (4)
02:05:24/19:27:40|disconnect xfer stream #0 (5)
02:05:24/19:27:41|disconnect xfer stream #0 (6)
02:05:24/19:27:41|disconnect xfer stream #0 (7)
02:05:24/19:27:42|disconnect xfer stream #0 (8)
02:05:24/19:27:42|disconnect xfer stream #0 (9)
02:05:24/19:27:43|disconnect xfer stream #0 (10)
02:05:24/19:27:43|disconnect xfer stream #0 (11)
02:05:24/19:27:44|disconnect xfer stream #0 (12)
02:05:24/19:27:44|disconnect xfer stream #0 (13)
02:05:24/19:27:45|disconnect xfer stream #0 (14)
02:05:24/19:27:46|disconnect xfer stream #0 (15)
02:05:24/19:27:46|disconnect xfer stream #0 (16)
02:05:24/19:27:47|disconnect xfer stream #0 (17)
02:05:24/19:27:47|disconnect xfer stream #0 (18)
02:05:24/19:27:48|disconnect xfer stream #0 (19)
02:05:24/19:27:48|Action has not completed
02:05:24/19:27:49|Giving up waiting
02:05:24/19:27:49|executing generic Stop_Merger
02:05:24/19:27:49|executing generic Stop_DataStorageServer
02:05:24/19:27:49|completed generic doStop
02:05:24/19:27:49|do_GetState returned z=0 and 1
02:05:24/19:27:49|do_GetState returned z=0 and 1
02:05:24/19:28:01|Transfer Error - : Connection reset by peer
02:05:24/19:28:08|send() failed:
02:05:24/19:28:08|Aida state now Stopped. AidaExecV10.0_Jul 6 2022: Build 15:10:57
02:05:24/19:28:08|do_GetState returned z=0 and 0
02:05:24/19:28:13|executing generic doGo
02:05:24/19:28:45|Initialising communication with storage server 192.168.11.99 on port 11002
02:05:24/19:28:45|Setting Transfer Block Size 65536
02:05:24/19:28:45|Setting Transfer Mode 3
02:05:24/19:28:45|Setting Overlap Mode 0
02:05:24/19:28:45|Setting Blocking Mode 0
02:05:24/19:28:45|Setting nice 0
02:05:24/19:28:45|TCP transfer library version 3.12
02:05:24/19:28:45|TCP socket send buffer was 16384 - now 221184
02:05:24/19:28:45|TCP socket receive buffer was 87380 - now 221184
02:05:24/19:28:45|TCP socket created OK - now connecting to 192.168.11.99 port 11002
02:05:24/19:28:45|Connected to 192.168.11.99 port 11002
02:05:24/19:28:45|connect xfer stream #0 (1)
02:05:24/19:28:45|Data Acquisition Statistics counters now cleared
02:05:24/19:28:46|ASIC DMA buffer size 2097152; WAVE DMA buffer size 1048576
02:05:24/19:28:46|Aida state now Going. AidaExecV10.0: Build Jul 6 2022_15:10:57
02:05:24/19:28:46|Go (1)
02:05:24/19:28:46|Running Go_Electronics for aida02 with TSMaster aida01
02:05:24/19:28:46|ADCs Calibrated.
02:05:24/19:28:46|Finished Go_Electronics for AIDA
02:05:24/19:28:46|executing generic Go_Merger
02:05:24/19:28:46|executing generic Go_DataStorageServer
02:05:24/19:28:46|completed generic doGo
02:05:24/19:28:46|do_GetState returned z=0 and 1
02:05:24/19:28:46|In RDOGo_Operate: Enabled Correlation, ASIC and discriminator readout
02:05:24/19:28:51|Also in RDOGo_Operate: Enabled waveform readout
02:05:24/19:28:51|get_WAVEBlk (A) which = 0 ---- status = 0048, State machines = 0201
02:05:24/19:28:51|WV:0:Buffer info. Length = 64 : Offset = 0
02:05:24/19:28:51|WV:1:Buffer info. Length = 62 : Offset = 2
02:05:24/19:28:51|WV:2:Buffer info. Length = 60 : Offset = 4
02:05:24/19:28:52|WV:3:Buffer info. Length = 58 : Offset = 6
02:05:24/19:28:52|WV:4:Buffer info. Length = 56 : Offset = 8
02:05:24/19:28:52|WV:5:Buffer info. Length = 54 : Offset = 10
02:05:24/19:28:52|WV:6:Buffer info. Length = 52 : Offset = 12
02:05:24/19:28:52|WV:7:Buffer info. Length = 50 : Offset = 14
02:05:24/19:28:52|WV:8:Buffer info. Length = 48 : Offset = 16
02:05:24/19:28:52|WV:9:Buffer info. Length = 46 : Offset = 18
02:05:24/19:28:52|get_WAVEBlk (A) which = 1 ---- status = 0088, State machines = 0201
02:05:24/19:28:52|get_WAVEBlk (A) which = 0 ---- status = 0048, State machines = 0908
02:05:24/19:28:52|get_WAVEBlk (A) which = 1 ---- status = 0088, State machines = 0908
02:05:24/19:28:52|get_WAVEBlk (A) which = 0 ---- status = 0048, State machines = 0908
02:05:24/19:28:52|get_WAVEBlk (A) which = 1 ---- status = 0088, State machines = 0908
02:05:24/19:28:52|get_WAVEBlk (A) which = 0 ---- status = 0048, State machines = 0908
02:05:24/19:28:52|get_WAVEBlk (A) which = 1 ---- status = 0088, State machines = 0908
02:05:24/19:28:52|get_WAVEBlk (A) which = 0 ---- status = 1048, State machines = 0201
02:05:24/19:28:52|get_WAVEBlk (A) which = 1 ---- status = 0088, State machines = 0908
02:05:24/19:28:52| |