09.25 All system wide checks OK *except*
Base Current Difference
aida01 fault 0xc960 : 0xc963 : 3
aida02 fault 0x8bbf : 0x8bc2 : 3
aida03 fault 0xae9e : 0xaea1 : 3
aida04 fault 0x6e3 : 0x6e6 : 3
aida05 fault 0x31a4 : 0x31ac : 8
aida05 : WR status 0x10
aida06 fault 0x22ea : 0x22f2 : 8
aida07 fault 0x438f : 0x4397 : 8
aida08 fault 0x61ed : 0x61f5 : 8
aida09 fault 0x1c32 : 0x1c33 : 1
White Rabbit error counter test result: Passed 3, Failed 9
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Returned 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 16 18 24 16 3 1 3 2 2 3 4 : 27792
aida02 : 25 6 3 5 1 4 2 2 3 3 6 : 36260
aida03 : 19 10 3 6 3 3 1 3 3 3 6 : 36556
aida04 : 21 7 3 4 2 4 2 2 3 3 6 : 36284
aida05 : 40 37 16 13 4 3 2 2 3 3 4 : 28904
aida06 : 32 4 4 3 0 4 1 3 3 3 6 : 36416
aida07 : 26 22 18 13 3 3 2 4 3 4 3 : 27672
aida08 : 15 9 4 1 2 3 1 3 3 3 6 : 36324
aida09 : 29 41 11 19 3 4 3 4 3 3 4 : 30348
aida10 : 17 5 8 4 3 3 3 3 2 3 6 : 36012
aida11 : 25 9 3 5 1 1 2 4 2 3 6 : 35900
aida12 : 80 43 12 7 4 2 3 2 2 3 4 : 27960
Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same
FEE : aida01 => Options file size is 1026 Last changed Sat Apr 24 03:57:22 CEST 2021
FEE : aida02 => Options file size is 1014 Last changed Fri Apr 16 00:56:20 CEST 2021
FEE : aida03 => Options file size is 1014 Last changed Wed Apr 14 21:52:04 CEST 2021
FEE : aida04 => Options file size is 1025 Last changed Sat Apr 17 06:07:36 CEST 2021
FEE : aida05 => Options file size is 1025 Last changed Fri Apr 16 00:53:25 CEST 2021
FEE : aida06 => Options file size is 1014 Last changed Wed Apr 14 21:52:04 CEST 2021
FEE : aida07 => Options file size is 1014 Last changed Wed Apr 14 21:52:04 CEST 2021
FEE : aida08 => Options file size is 1014 Last changed Wed Apr 14 21:52:04 CEST 2021
FEE : aida09 => Options file size is 1014 Last changed Wed Apr 14 21:52:05 CEST 2021
FEE : aida10 => Options file size is 1014 Last changed Wed Apr 14 21:52:06 CEST 2021
FEE : aida11 => Options file size is 1014 Last changed Wed Apr 14 21:52:05 CEST 2021
FEE : aida12 => Options file size is 1025 Last changed Mon Apr 19 09:05:25 CEST 2021
09.25 DSSSD bias |& leakage currents Ok - attachment 1
FEE64 temps OK - attachment 2
good event stats - attachment 3
aida09 zero rate
merger time seq errors - attachment 4
Grafana DSSSD bias |& leakage currents for previous 7 days - attachment 5
DAQ continues OK *except* aida09 - no data storage
10.00 from aida09 system console
24/03:04:39|In RDOGo_Operate: Enabled Correlation, ASIC and discriminator readout
24/03:04:39|Also in RDOGo_Operate: Enabled waveform readout
24/03:04:39|
24/03:04:39|get_WAVEBlk (C) : which = 0 , Status = 00001031 , Count = 0
24/03:04:39|get_WAVEBlk (A) which = 0 ---- status = 0540, State machines = 050A
24/03:04:39|WV:0:Buffer info. Length = 131072 : Offset = 0
24/03:04:40|WV:1:Buffer info. Length = 131070 : Offset = 2
24/03:04:40|WV:2:Buffer info. Length = 131068 : Offset = 4
24/03:04:40|WV:3:Buffer info. Length = 131066 : Offset = 6
24/03:04:40|WV:4:Buffer info. Length = 131064 : Offset = 8
24/03:04:40|WV:5:Buffer info. Length = 131062 : Offset = 10
24/03:04:40|WV:6:Buffer info. Length = 131060 : Offset = 12
24/03:04:40|WV:7:Buffer info. Length = 131058 : Offset = 14
24/03:04:40|WV:8:Buffer info. Length = 131056 : Offset = 16
24/03:04:40|WV:9:Buffer info. Length = 131054 : Offset = 18
24/03:04:40|get_WAVEBlk (A) which = 1 ---- status = 0088, State machines = 0908
24/03:04:40|get_ASICBlk : Blk=1 : bytes = 1048576 : Blk_Status = 00000088 : Offset = 00100000 : Databuffer : 273090556 : 0 => 804441A5 : 1 => 09CD0000
24/03:04:40|Last Data : Databuffer 274139128 : 0 => C01B7F88 : 1 => 0C05B135
24/03:04:40| 1FA61FB2 : 1F951F66 | 1F361F5D : 1F751F0F | 1F191EF0 : 1EF31EE9 | 1E701E8E : 1EEB1F51
24/03:04:40| 1F521F99 : 1FF52005 | 1FE52027 : 20171FCC | 1FAD1F3D : 1ED41E7B | 1EB11E77 : 3FAB373A
24/03:04:40| 8050023F : 0AE0C000 | 804441A5 : 0AE0C000 | 400003FC : 0AE0C4D0 | 1E581F6F : 1E451D26
24/03:04:41| 1FB01E60 : 1D2E1F80 | 1E251D88 : 1FA81F4A | 1DD51F1F : 1E1D1DF7 | 1EBC1DF0 : 1DF41E7E
24/03:04:41| 1DEA1DF5 : 1E9E1E83 | 1E231E9C : 1E661E60 | 1EA41E6F : 1E101D9C | 1D751D92 : 1DFC1DF2
24/03:04:41| 1E201E47 : 1E7C1E6C | 1E351E00 : 1DD91DFE | 1DD81DB5 : 1DE91E1C | 1E261DD6 : 1E1A1E0B
24/03:04:41| 1DFB1DCC : 1DEA1E11 | 1E201E39 : 1E351E7A | 1E7B1DBE : 1E7A1E81 | 1E261DFC : 1DEA1DAC
24/03:04:41| 1D831DAC : 1DF51E1F | 1E4E1E48 : 1E421E50 | 1E861DDB : 1E361E18 | 1D551DB9 : 1DC51D8E
24/03:04:41|get_ASICBlk : Blk=0 : bytes = 1048576 : Blk_Status = 00000048 : Offset = 00000000 : Databuffer : 273090556 : 0 => C03A7F13 : 1 => 0C05B455
24/03:04:41|Last Data : Databuffer 274139128 : 0 => C0127D9D : 1 => 0DB9DEED
24/03:04:41| 1FA61FB2 : 1F951F66 | 1F361F5D : 1F751F0F | 1F191EF0 : 1EF31EE9 | 1E701E8E : 1EEB1F51
24/03:04:41| 1F521F99 : 1FF52005 | 1FE52027 : 20171FCC | 1FAD1F3D : 1ED41E7B | 1EB11E77 : 3FAB373A
24/03:04:41| 8050023F : 0AE0C000 | 804441A5 : 0AE0C000 | 400003FC : 0AE0C4D0 | 1E581F6F : 1E451D26
24/03:04:42| 1FB01E60 : 1D2E1F80 | 1E251D88 : 1FA81F4A | 1DD51F1F : 1E1D1DF7 | 1EBC1DF0 : 1DF41E7E
24/03:04:42| 1DEA1DF5 : 1E9E1E83 | 1E231E9C : 1E661E60 | 1EA41E6F : 1E101D9C | 1D751D92 : 1DFC1DF2
24/03:04:42| 1E201E47 : 1E7C1E6C | 1E351E00 : 1DD91DFE | 1DD81DB5 : 1DE91E1C | 1E261DD6 : 1E1A1E0B
24/03:04:42| 1DFB1DCC : 1DEA1E11 | 1E201E39 : 1E351E7A | 1E7B1DBE : 1E7A1E81 | 1E261DFC : 1DEA1DAC
24/03:04:42| 1D831DAC : 1DF51E1F | 1E4E1E48 : 1E421E50 | 1E861DDB : 1E361E18 | 1D551DB9 : 1DC51D8E
24/03:04:42|get_WAVEBlk (A) which = 0 ---- status = 0048, State machines = 0908
24/03:04:42|get_ASICBlk : Blk=1 : bytes = 1048576 : Blk_Status = 00000088 : Offset = 00100000 : Databuffer : 273090556 : 0 => C02D7E26 : 1 => 0DB9DEED
24/03:04:42|Last Data : Databuffer 274139128 : 0 => C0217F73 : 1 => 0F816B3D
24/03:04:42| 1EFB1F2D : 1F311F1E | 1F3B1F1C : 1EDD1EC4 | 1ED71EE6 : 1F301F09 | 1F4D1FAA : 1F1E1F81
24/03:04:42| 1F491F0C : 1F481F31 | 1F6C1F7E : 1F601F8E | 1FC71FA9 : 1F451F9F | 1F8C1F1D : 1F551FA9
24/03:04:42| 1F671E98 : 1F421F47 | 1ED91F67 : 1F911F70 | 1DFD1F98 : 224A1D5B | 1CF821BC : 1F771E94
24/03:04:43| 213F1EE7 : 1DBC20F2 | 1F961D9F : 1FC81FE3 | 1EA0206E : 1F9D1DD9 | 20042040 : 1E6A1ECA
24/03:04:43| 1F7E1FD0 : 1F981EC7 | 1E7A1F1B : 1F5F1F27 | 1F121F10 : 1F531F92 | 1F5B1EF8 : 1FA12003
24/03:04:43| 1F7A1F97 : 1F561F26 | 200C1F80 : 1FC3201A | 1FA81FA4 : 1FEB1F99 | 1F5C1F9D : 1F8F1F98
24/03:04:43| 1FCC1FC5 : 1FB01F92 | 1FA21F6D : 1F261F0A | 1F141F1C : 1F461F24 | 1EEE1FDB : 20221F2F
24/03:04:43| 1F741FF1 : 1F891F8A | 1FC71F5E : 1F961FF2 | 1F6B1F6C : 1FB11F74 | 1F821FA8 : 1F841F54
24/03:04:43|get_ASICBlk : Blk=0 : bytes = 1048576 : Blk_Status = 00000048 : Offset = 00000000 : Databuffer : 273090556 : 0 => C02D7EB4 : 1 => 0F816D95
24/03:04:43|Last Data : Databuffer 274139128 : 0 => C0167F53 : 1 => 0148F2DD
24/03:04:43| 1EFB1F2D : 1F311F1E | 1F3B1F1C : 1EDD1EC4 | 1ED71EE6 : 1F301F09 | 1F4D1FAA : 1F1E1F81
24/03:04:43| 1F491F0C : 1F481F31 | 1F6C1F7E : 1F601F8E | 1FC71FA9 : 1F451F9F | 1F8C1F1D : 1F551FA9
24/03:04:43| 1F671E98 : 1F421F47 | 1ED91F67 : 1F911F70 | 1DFD1F98 : 224A1D5B | 1CF821BC : 1F771E94
24/03:04:44| 213F1EE7 : 1DBC20F2 | 1F961D9F : 1FC81FE3 | 1EA0206E : 1F9D1DD9 | 20042040 : 1E6A1ECA
24/03:04:44| 1F7E1FD0 : 1F981EC7 | 1E7A1F1B : 1F5F1F27 | 1F121F10 : 1F531F92 | 1F5B1EF8 : 1FA12003
24/03:04:44| 1F7A1F97 : 1F561F26 | 200C1F80 : 1FC3201A | 1FA81FA4 : 1FEB1F99 | 1F5C1F9D : 1F8F1F98
24/03:04:44| 1FCC1FC5 : 1FB01F92 | 1FA21F6D : 1F261F0A | 1F141F1C : 1F461F24 | 1EEE1FDB : 20221F2F
24/03:04:44| 1F741FF1 : 1F891F8A | 1FC71F5E : 1F961FF2 | 1F6B1F6C : 1FB11F74 | 1F821FA8 : 1F841F54
24/03:04:44|get_ASICBlk : Blk=1 : bytes = 1048576 : Blk_Status = 00000088 : Offset = 00100000 : Databuffer : 273090556 : 0 => C0207E17 : 1 => 0148F2DD
24/03:04:44|Last Data : Databuffer 274139128 : 0 => C0177F46 : 1 => 047FF0CD
24/03:04:44| 1EFB1F2D : 1F311F1E | 1F3B1F1C : 1EDD1EC4 | 1ED71EE6 : 1F301F09 | 1F4D1FAA : 1F1E1F81
24/03:04:44| 1F491F0C : 1F481F31 | 1F6C1F7E : 1F601F8E | 1FC71FA9 : 1F451F9F | 1F8C1F1D : 1F551FA9
24/03:04:44| 1F671E98 : 1F421F47 | 1ED91F67 : 1F911F70 | 1DFD1F98 : 224A1D5B | 1CF821BC : 1F771E94
24/03:04:44| 213F1EE7 : 1DBC20F2 | 1F961D9F : 1FC81FE3 | 1EA0206E : 1F9D1DD9 | 20042040 : 1E6A1ECA
24/03:04:45| 1F7E1FD0 : 1F981EC7 | 1E7A1F1B : 1F5F1F27 | 1F121F10 : 1F531F92 | 1F5B1EF8 : 1FA12003
24/03:04:45| 1F7A1F97 : 1F561F26 | 200C1F80 : 1FC3201A | 1FA81FA4 : 1FEB1F99 | 1F5C1F9D : 1F8F1F98
24/03:04:45| 1FCC1FC5 : 1FB01F92 | 1FA21F6D : 1F261F0A | 1F141F1C : 1F461F24 | 1EEE1FDB : 20221F2F
24/03:04:45| 1F741FF1 : 1F891F8A | 1FC71F5E : 1F961FF2 | 1F6B1F6C : 1FB11F74 | 1F821FA8 : 1F841F54
24/03:04:45|get_WAVEBlk (A) which = 1 ---- status = 0088, State machines = 0908
24/03:04:45|get_WAVEBlk (A) which = 0 ---- status = 0048, State machines = 0908
24/03:04:45|get_WAVEBlk (A) which = 1 ---- status = 0088, State machines = 0908
24/03:04:47|get_WAVEBlk (A) which = 0 ---- status = 0048, State machines = 0908
24/03:04:56|get_WAVEBlk (A) which = 1 ---- status = 0088, State machines = 0908
24/03:05:05|get_WAVEBlk (A) which = 0 ---- status = 0048, State machines = 0908
24/03:05:15|get_WAVEBlk (A) which = 1 ---- status = 0088, State machines = 0908
24/03:05:24|197 spectra zeroed
25/10:21:10|do_GetState returned z=0 and 1
25/10:21:11|Data Acquisition Statistics counters now cleared
25/10:24:01|Clear Statistics (1)
25/10:24:01|bad event data (0x45000034 0x84e64000)
26/16:52:50|bad event data (0x40061e21 0xc0a80b63)
26/16:52:50|received undefined information code: 1
26/16:52:50|bad event data (0x0101080a 0xe9b68def)
26/16:52:50|bad event data (0x03527827 0x0101050a)
26/16:52:50|received undefined information code: 9
From system console
Apr 24 02:52:42 aida09 kernel: IP-Config: Got DHCP answer from 192.168.11.99, my address is 192.168.11.9
Apr 24 02:52:42 aida09 kernel: IP-Config: Complete:
Apr 24 02:52:42 aida09 kernel: device=eth0, addr=192.168.11.9, mask=255.255.255.0, gw=255.255.255.255,
Apr 24 02:52:42 aida09 kernel: host=aida09, domain=dl.ac.uk, nis-domain=nuclear.physics,
Apr 24 02:52:42 aida09 kernel: bootserver=192.168.11.99, rootserver=192.168.11.99, rootpath=/home/Embedded/XilinxLinux/ppc_4xx/rfs/aida09
Apr 24 02:52:42 aida09 kernel: Looking up port of RPC 100003/2 on 192.168.11.99
Apr 24 02:52:42 aida09 kernel: Looking up port of RPC 100005/1 on 192.168.11.99
Apr 24 02:52:43 aida09 kernel: VFS: Mounted root (nfs filesystem) on device 0:12.
Apr 24 02:52:43 aida09 kernel: Freeing unused kernel memory: 168k init
Apr 24 02:52:43 aida09 kernel: xaida: device parameters: base=0x81000000 size=0x200000
Apr 24 02:52:44 aida09 kernel: Trying to free nonexistent resource <0000000081000000-00000000811fffff>
Apr 24 02:52:44 aida09 kernel: xaida: mem region start 0x81000000 for 0x200000 mapped at 0xd2100000
Apr 24 02:52:44 aida09 kernel: xaida: driver assigned major number 254
Apr 24 02:52:44 aida09 xinetd[273]: IPv6 socket creation failed for service telnet, trying IPv4
Apr 24 02:52:44 aida09 xinetd[273]: IPv6 socket creation failed for service ftp, trying IPv4
Apr 24 02:52:44 aida09 xinetd[273]: xinetd Version 2.3.14 started with libwrap loadavg options compiled in.
Apr 24 02:52:44 aida09 xinetd[273]: Started working: 2 available services
Apr 24 02:52:48 aida09 kernel: Trying to free nonexistent resource <0000000007000000-0000000007ffffff>
Apr 24 02:52:48 aida09 kernel: AIDAMEM: aidamem: mem region start 0x7000000 for 0x1000000 mapped at 0xd2380000
Apr 24 02:52:48 aida09 kernel: AIDAMEM: aidamem: driver assigned major number 253
Apr 24 02:53:04 aida09 kernel: xaida: open:
Apr 24 02:53:04 aida09 kernel: AIDAMEM: aidamem_open:
Apr 24 02:57:33 aida09 kernel: xaida: open:
Apr 24 13:39:36 aida09 xinetd[273]: START: telnet pid=396 from=192.168.11.99
Apr 24 13:42:05 aida09 xinetd[273]: EXIT: telnet status=0 pid=396 duration=149(sec)
Apr 27 09:01:50 aida09 xinetd[273]: START: telnet pid=422 from=192.168.11.99 |