| ID |
Date |
Author |
Subject |
|
356
|
Fri Jun 4 09:47:44 2021 |
TD | aida15 - system console output - 00.02 Friday 4 June reboot | 03/23:58:55|kernel BUG at mm/slab.c:2974!
03/23:58:55|Oops: Exception in kernel mode, sig: 5 [#1]
03/23:58:55|PREEMPT Xilinx Virtex440
03/23:58:55|Modules linked in: aidamem xdriver xh_spidev_register
03/23:58:55|NIP: c009211c LR: c00920b0 CTR: c0038e70
03/23:58:55|REGS: c631fc60 TRAP: 0700 Not tainted (2.6.31)
03/23:58:55|MSR: 00021000 <ME,CE> CR: 22022048 XER: 00000000
03/23:58:56|TASK = c627a0c0[375] 'AidaExecV9' THREAD: c631e000
03/23:58:56|GPR00: 00000001 c631fd10 c627a0c0 c680daf0 00000020 00000001 c6940020 00000000
03/23:58:56|GPR08: fffffff6 00010002 c680e400 c631e000 22008022 1005520c c03a0000 00000020
03/23:58:56|GPR16: c0390000 c03a069c c03a0000 c038c384 c038cc18 00000020 00000000 00200200
03/23:58:56|GPR24: 00100100 c631e000 00000000 c680dae8 c680dae0 c680ae00 00000010 c680e400
03/23:58:56|NIP [c009211c] cache_alloc_refill+0x130/0x608
03/23:58:56|LR [c00920b0] cache_alloc_refill+0xc4/0x608
03/23:58:56|Call Trace:
03/23:58:56|[c631fd10] [c00920b0] cache_alloc_refill+0xc4/0x608 (unreliable)
03/23:58:56|[c631fd70] [c00927d8] kmem_cache_alloc+0xc4/0xcc
03/23:58:56|[c631fd90] [c0042420] __sigqueue_alloc+0x50/0xb8
03/23:58:56|[c631fdb0] [c0042938] __send_signal+0x78/0x260
03/23:58:56|[c631fde0] [c0042f78] group_send_sig_info+0x70/0x9c
03/23:58:56|[c631fe10] [c00438a8] kill_pid_info+0x48/0x8c
03/23:58:56|[c631fe30] [c0038e8c] it_real_fn+0x1c/0x30
03/23:58:56|[c631fe40] [c0050c40] hrtimer_run_queues+0x184/0x240
03/23:58:56|[c631fea0] [c0040ba8] run_local_timers+0x10/0x2c
03/23:58:56|[c631feb0] [c0040bf4] update_process_times+0x30/0x70
03/23:58:57|[c631fed0] [c005a000] tick_periodic+0x34/0xe8
03/23:58:57|[c631fee0] [c005a0d4] tick_handle_periodic+0x20/0x120
03/23:58:57|[c631ff20] [c000af70] timer_interrupt+0xa4/0x10c
03/23:58:57|[c631ff40] [c000e9c4] ret_from_except+0x0/0x18
03/23:58:57|Instruction dump:
03/23:58:57|2f1e0000 409900f4 387c0010 3b7c0008 80dc0000 7f9c3000 419e014c 81060010
03/23:58:57|801d001c 7c004010 38000000 7c000114 <0f000000> 81260010 801d001c 7f804840
03/23:58:57|Kernel panic - not syncing: Fatal exception in interrupt
03/23:58:57|Call Trace:
03/23:58:57|[c631fab0] [c0005de8] show_stack+0x44/0x16c (unreliable)
03/23:58:57|[c631faf0] [c00345bc] panic+0x94/0x168
03/23:58:57|[c631fb40] [c000bd44] die+0x178/0x18c
03/23:58:57|[c631fb60] [c000c000] _exception+0x164/0x1b4
03/23:58:57|[c631fc50] [c000e978] ret_from_except_full+0x0/0x4c
03/23:58:57|[c631fd10] [c00920b0] cache_alloc_refill+0xc4/0x608
03/23:58:57|[c631fd70] [c00927d8] kmem_cache_alloc+0xc4/0xcc
03/23:58:57|[c631fd90] [c0042420] __sigqueue_alloc+0x50/0xb8
03/23:58:57|[c631fdb0] [c0042938] __send_signal+0x78/0x260
03/23:58:57|[c631fde0] [c0042f78] group_send_sig_info+0x70/0x9c
03/23:58:58|[c631fe10] [c00438a8] kill_pid_info+0x48/0x8c
03/23:58:58|[c631fe30] [c0038e8c] it_real_fn+0x1c/0x30
03/23:58:58|[c631fe40] [c0050c40] hrtimer_run_queues+0x184/0x240
03/23:58:58|[c631fea0] [c0040ba8] run_local_timers+0x10/0x2c
03/23:58:58|[c631feb0] [c0040bf4] update_process_times+0x30/0x70
03/23:58:58|[c631fed0] [c005a000] tick_periodic+0x34/0xe8
03/23:58:58|[c631fee0] [c005a0d4] tick_handle_periodic+0x20/0x120
03/23:58:58|[c631ff20] [c000af70] timer_interrupt+0xa4/0x10c
03/23:58:58|[c631ff40] [c000e9c4] ret_from_except+0x0/0x18
03/23:58:58|Rebooting in 180 seconds..
ISOL Version 1.00 Date 9th January 2017
Flash base address=FC000000
Set Flash to ASync Mode
XST_SUCCESS|
Finished copying zImage to RAM
04/00:01:59|
Found 0 errors checking kernel image
04/00:02:00|VHDL version number 0X03350706
Based on AIDA Bootloader version number 1.2.0 -- 16th August 2012
Starting LMK 3200 setup
04/00:02:00|
Setting LMK03200 to standard clock settings -- External Clock 23Nov15
.... SPI Base Address=0x81400000
clk_control_reg=0x4
04/00:02:00|Next step is SPIconfig
Control 32(0x81400000)=0x180
SlaveSel(0x81400000)=0x3
Ctrl(0x81400000)=0xE6
Ctrl(0x81400000)=0x86
04/00:02:00|SPIconfig done now to set up the LMK3200 registers
04/00:02:00|LMK #0 : regInit[0]=0x80000000
04/00:02:00|LMK #0 : regInit[1]=0x10070600
04/00:02:00|LMK #0 : regInit[2]=0x60601
04/00:02:00|LMK #0 : regInit[3]=0x60602
04/00:02:00|LMK #0 : regInit[4]=0x60603
04/00:02:00|LMK #0 : regInit[5]=0x70624
04/00:02:00|LMK #0 : regInit[6]=0x70605
04/00:02:00|LMK #0 : regInit[7]=0x70606
04/00:02:00|LMK #0 : regInit[8]=0x70627
04/00:02:00|LMK #0 : regInit[9]=0x10000908
04/00:02:00|LMK #0 : regInit[10]=0xA0022A09
04/00:02:00|LMK #0 : regInit[11]=0x82800B
04/00:02:01|LMK #0 : regInit[12]=0x28C800D
04/00:02:01|LMK #0 : regInit[13]=0x830020E
04/00:02:01|LMK #0 : regInit[14]=0xC800180F
Calibrate completed at 941 counts
Setting Clock Control =0x0000000B, to set GOE and sync bit
Ctrl @ SPIstop (0x81400000)=0x186
Timeout waiting for Lock detect Stage 2 (Zero Delay), PWR_DWN=0x00000004
04/00:02:01|
Finished Clock setup LMK03200
completed LMK 3200 setup
Loaded all four ASICs with default settings
Setting the ADCs into calibration mode
04/00:02:01|
Control 32(0x81400400)=0x180
SlaveSel(0x81400400)=0xFF
Ctrl(0x81400400)=0xE6
Ctrl(0x81400400)=0x86
Init : Config of AD9252 SPI ok
04/00:02:01|
Ctrl @ SPIstop (0x81400400)=0x186ADCs initialised
Cal DCMs not locked
ADC calibrate failed
Jumping to kernel simpleboot...
04/00:02:01|
zImage starting: loaded at 0x00a00000 (sp: 0x00bc4eb0)
Allocating 0x3b78cc bytes for kernel ...
gunzipping (0x00000000 <- 0x00a0f000:0x00bc380e)...done 0x39604c bytes
04/00:02:05|
Linux/PowerPC load: console=ttyS0 root=/dev/nfs ip=on rw mem=112M
Finalizing device tree... flat tree at 0xbd1300
Probing IIC bus for MAC... MAC address = 0xd8 0x80 0x39 0x41 0xee 0x10
04/00:02:11|Using Xilinx Virtex440 machine description
04/00:02:12|Linux version 2.6.31 (nf@nnlxb.dl.ac.uk) (gcc version 4.2.2) #34 PREEMPT Tue Nov 15 15:57:04 GMT 2011
04/00:02:12|Zone PFN ranges:
04/00:02:12| DMA 0x00000000 -> 0x00007000
04/00:02:12| Normal 0x00007000 -> 0x00007000
04/00:02:12|Movable zone start PFN for each node
04/00:02:12|early_node_map[1] active PFN ranges
04/00:02:12| 0: 0x00000000 -> 0x00007000
04/00:02:12|MMU: Allocated 1088 bytes of context maps for 255 contexts
04/00:02:12|Built 1 zonelists in Zone order, mobility grouping on. Total pages: 28448
04/00:02:12|Kernel command line: console=ttyS0 root=/dev/nfs ip=on rw mem=112M
04/00:02:12|PID hash table entries: 512 (order: 9, 2048 bytes)
04/00:02:12|Dentry cache hash table entries: 16384 (order: 4, 65536 bytes)
04/00:02:12|Inode-cache hash table entries: 8192 (order: 3, 32768 bytes)
04/00:02:12|Memory: 109680k/114688k available (3500k kernel code, 4852k reserved, 144k data, 130k bss, 168k init)
04/00:02:12|Kernel virtual memory layout:
04/00:02:12| * 0xffffe000..0xfffff000 : fixmap
04/00:02:12| * 0xfde00000..0xfe000000 : consistent mem
04/00:02:12| * 0xfde00000..0xfde00000 : early ioremap
04/00:02:13| * 0xd1000000..0xfde00000 : vmalloc & ioremap
04/00:02:13|NR_IRQS:512
04/00:02:13|clocksource: timebase mult[a00000] shift[22] registered
04/00:02:13|Console: colour dummy device 80x25
04/00:02:13|Mount-cache hash table entries: 512
04/00:02:13|NET: Registered protocol family 16
04/00:02:13|PCI: Probing PCI hardware
04/00:02:13|bio: create slab <bio-0> at 0
04/00:02:13|NET: Registered protocol family 2
04/00:02:13|IP route cache hash table entries: 1024 (order: 0, 4096 bytes)
04/00:02:13|TCP established hash table entries: 4096 (order: 3, 32768 bytes)
04/00:02:13|TCP bind hash table entries: 4096 (order: 2, 16384 bytes)
04/00:02:13|TCP: Hash tables configured (established 4096 bind 4096)
04/00:02:13|TCP reno registered
04/00:02:13|NET: Registered protocol family 1
04/00:02:13|ROMFS MTD (C) 2007 Red Hat, Inc.
04/00:02:13|msgmni has been set to 214
04/00:02:13|io scheduler noop registered
04/00:02:13|io scheduler anticipatory registered
04/00:02:13|io scheduler deadline registered
04/00:02:13|io scheduler cfq registered (default)
04/00:02:13|Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
04/00:02:13|83e00000.serial: ttyS0 at MMIO 0x83e01003 (irq = 16) is a 16550
04/00:02:14|console [ttyS0] enabled
04/00:02:14|brd: module loaded
04/00:02:14|loop: module loaded
04/00:02:14|Device Tree Probing 'ethernet'
04/00:02:14|xilinx_lltemac 81c00000.ethernet: MAC address is now d8:80:39:41:ee:10
04/00:02:14|xilinx_lltemac 81c00000.ethernet: XLlTemac: using DMA mode.
04/00:02:14|XLlTemac: DCR address: 0x80
04/00:02:14|XLlTemac: buffer descriptor size: 32768 (0x8000)
04/00:02:14|XLlTemac: Allocating DMA descriptors with kmalloc
04/00:02:14|XLlTemac: (buffer_descriptor_init) phy: 0x6938000, virt: 0xc6938000, size: 0x8000
04/00:02:14|XTemac: PHY detected at address 7.
04/00:02:14|xilinx_lltemac 81c00000.ethernet: eth0: Xilinx TEMAC at 0x81C00000 mapped to 0xD1024000, irq=17
04/00:02:14|fc000000.flash: Found 1 x16 devices at 0x0 in 16-bit bank
04/00:02:14| Intel/Sharp Extended Query Table at 0x010A
04/00:02:14| Intel/Sharp Extended Query Table at 0x010A
04/00:02:14| Intel/Sharp Extended Query Table at 0x010A
04/00:02:14| Intel/Sharp Extended Query Table at 0x010A
04/00:02:14| Intel/Sharp Extended Query Table at 0x010A
04/00:02:14| Intel/Sharp Extended Query Table at 0x010A
04/00:02:14|Using buffer write method
04/00:02:15|cfi_cmdset_0001: Erase suspend on write enabled
04/00:02:15|cmdlinepart partition parsing not available
04/00:02:15|RedBoot partition parsing not available
04/00:02:15|Creating 5 MTD partitions on "fc000000.flash":
04/00:02:15|0x000000000000-0x000000500000 : "golden_firmware"
04/00:02:15|0x000000500000-0x000000800000 : "golden_kernel"
04/00:02:15|0x000000800000-0x000000d00000 : "user_firmware"
04/00:02:15|0x000000d00000-0x000000fe0000 : "user_kernel"
04/00:02:15|0x000000fe0000-0x000001000000 : "env_variables"
04/00:02:15|xilinx-xps-spi 81400400.hd-xps-spi: at 0x81400400 mapped to 0xD1028400, irq=20
04/00:02:15|SPI: XIlinx spi: bus number now 32766
04/00:02:15|xilinx-xps-spi 81400000.xps-spi: at 0x81400000 mapped to 0xD102C000, irq=21
04/00:02:15|SPI: XIlinx spi: bus number now 32765
04/00:02:15|mice: PS/2 mouse device common for all mice
04/00:02:15|Device Tree Probing 'i2c'
04/00:02:15| #0 at 0x81600000 mapped to 0xD1030000, irq=22
04/00:02:15|at24 0-0050: 1024 byte 24c08 EEPROM (writable)
04/00:02:15|TCP cubic registered
04/00:02:15|NET: Registered protocol family 17
04/00:02:15|RPC: Registered udp transport module.
04/00:02:15|RPC: Registered tcp transport module.
04/00:02:16|eth0: XLlTemac: Options: 0x3fa
04/00:02:16|eth0: XLlTemac: allocating interrupt 19 for dma mode tx.
04/00:02:16|eth0: XLlTemac: allocating interrupt 18 for dma mode rx.
04/00:02:16|eth0: XLlTemac: speed set to 1000Mb/s
04/00:02:18|eth0: XLlTemac: Send Threshold = 24, Receive Threshold = 4
04/00:02:18|eth0: XLlTemac: Send Wait bound = 254, Receive Wait bound = 254
04/00:02:18|Sending DHCP requests ., OK
04/00:02:19|IP-Config: Got DHCP answer from 192.168.11.99, my address is 192.168.11.15
04/00:02:20|IP-Config: Complete:
04/00:02:20| device=eth0, addr=192.168.11.15, mask=255.255.255.0, gw=255.255.255.255,
04/00:02:20| host=aida15, domain=dl.ac.uk, nis-domain=nuclear.physics,
04/00:02:20| bootserver=192.168.11.99, rootserver=192.168.11.99, rootpath=/home/Embedded/XilinxLinux/ppc_4xx/rfs/aida15
04/00:02:20|Looking up port of RPC 100003/2 on 192.168.11.99
04/00:02:20|Looking up port of RPC 100005/1 on 192.168.11.99
04/00:02:20|VFS: Mounted root (nfs filesystem) on device 0:12.
04/00:02:20|Freeing unused kernel memory: 168k init
INIT: version 2.86 booting
04/00:02:20|Starting sysinit...
04/00:02:21| Welcome to DENX & STFC Daresbury Embedded Linux Environment
04/00:02:22| Press 'I' to enter interactive startup.
04/00:02:22|Setting clock (utc): Thu Jun 3 23:02:22 BST 2021 [ OK ]
04/00:02:22|Building the cache [ OK ]
04/00:02:22|Setting hostname aida15: [ OK ]
04/00:02:23|Mounting local filesystems: [ OK ]
04/00:02:25|Enabling /etc/fstab swaps: [ OK ]
04/00:02:27|Finishing sysinit...
INIT: Entering runlevel: 3
04/00:02:30|Entering non-interactive startup
04/00:02:30|FATAL: Module ipv6 not found.
04/00:02:31|Bringing up loopback interface: [ OK ]
04/00:02:33|FATAL: Module ipv6 not found.
04/00:02:34|Starting system logger: [ OK ]
04/00:02:34|Starting kernel logger: [ OK ]
04/00:02:35|Starting rpcbind: [ OK ]
04/00:02:36|Mounting NFS filesystems: [ OK ]
04/00:02:37|Mounting other filesystems: [ OK ]
04/00:02:37|Starting xinetd: [ OK ]
04/00:02:38|Starting midas: Starting MIDAS Data Acquisition for aida15
04/00:02:38|xaida: device parameters: base=0x81000000 size=0x200000
04/00:02:44|Trying to free nonexistent resource <0000000081000000-00000000811fffff>
04/00:02:44|xaida: mem region start 0x81000000 for 0x200000 mapped at 0xd2100000
04/00:02:44|xaida: driver assigned major number 254
04/00:02:44|Trying to free nonexistent resource <0000000007000000-0000000007ffffff>
04/00:02:50|AIDAMEM: aidamem: mem region start 0x7000000 for 0x1000000 mapped at 0xd2380000
04/00:02:50|AIDAMEM: aidamem: driver assigned major number 253
04/00:02:50|System identified is CPU ppc; Platform is unix; OS is Linux and Version is 2.6.31
04/00:02:56|Environment selected is CPU ppc; Platform unix; OS Linux and Operating System linux-ppc_4xx
04/00:02:57|MIDASBASE = /MIDAS and MIDAS_LIBRARY = /MIDAS/TclHttpd/linux-ppc_4xx
04/00:02:57|PATH = /MIDAS/bin_linux-ppc_4xx:/MIDAS/TclHttpd/linux-ppc_4xx:/MIDAS/linux-ppc_4xx/bin:/MIDAS/linux-ppc_4xx/bin:/sbin:/usr/sbin:/bin:/usr/bin
04/00:02:57|Computer Name = aida15; Temp Directory = /tmp/tcl361
04/00:03:01|package limit is not available: can't find package limit
04/00:03:04|Running with default file descriptor limit
04/00:03:04|package setuid is not available: can't find package setuid
04/00:03:05|Running as user 0 group 0
04/00:03:05|
04/00:03:06|AIDA Data Acquisition Program Release 9_10.Apr 3 2019_11:34:31 starting
04/00:03:06|
04/00:03:06|Built without pthreads
04/00:03:06|
04/00:03:06|Creating NetVars
04/00:03:06|Output buffer length = 65xaida: open:
04/00:03:06|504; format option = 4AIDAMEM: aidamem_open:
04/00:03:06|; transfer option = 3
04/00:03:06|EB transfer option = 3
04/00:03:06|NetVars created and initialised
04/00:03:06|Statistics thread starting
04/00:03:06|Statistics thread created
04/00:03:06|Stat/Rate creation thread starting
04/00:03:06|Data Acquisition task has PID 375
04/00:03:06|Stat/Rate creation thread created
04/00:03:06|Hit/Rate creation thread starting
04/00:03:06|Hit/Rate creation thread created
04/00:03:07|AIDA Heartbeat thread starting
04/00:03:07|Heartbeat thread created
04/00:03:07|Installing signal handlers
04/00:03:07|Done
04/00:03:07|ModuleNum = 0
04/00:03:07|Aida Initialise complete. AidaExecV9_10: Build Apr 3 2019_11:34:31. HDL version : 03350706
04/00:03:07|Spectra table initialised
04/00:03:07|AIDA Data Acquisition now all ready to start
04/00:03:07|SIGBUS, SIGSEGV and SIGPIPE traps setup
04/00:03:07|/debug user "debug" password " bjkhcfyuqis"
04/00:03:14|httpd started on port 8015
04/00:03:14|
04/00:03:14|Cannot use /MIDAS/config/TclHttpd/aida15@8015/startup.tcl
04/00:03:14|Custom startup from /MIDAS/config/TclHttpd/aida15/startup.tcl
04/00:03:14|XAIDA Access package 1.0
04/00:03:16|/XAIDAAccessServer
04/00:03:16|XAD9252 Access package 1.0
04/00:03:16|/XAD9252AccessServer
04/00:03:16|/DataBaseAccessServer
04/00:03:16|[ OK ]
04/00:03:16|/NetVarService
04/00:03:17|/SigTaskService
04/00:03:17|Loaded MemSasAccess
04/00:03:17|/SpectrumService
04/00:03:17|loading tcl/AIDARunControl.tcl for namespace ::
04/00:03:17|
04/00:03:17|DENX ELDK version 4.2 build 2008-04-01
04/00:03:17|Linux 2.6.31 on a ppc
04/00:03:17|
04/00:03:17|aida15 login: /DataAcquisitionControlServer
04/00:03:17|DefineMessage unknown
04/00:03:17|Run Control Server Implementation for AIDA
04/00:03:17|RunControlServer loaded
04/00:03:17|loading Html/RunControl/implementation.tcl
04/00:03:18|/MIDAS/TclHttpd/Html/RunControl/common.tcl returned z=1 and couldn't read file "/MIDAS/TclHttpd/Html/RunControl/common.tcl": no such file or directory
04/00:03:18|ReadRegister failed: Name=NetVar.EXEC.ID; Code= 0x10004; Info= Register name does not exist
04/00:03:18|Created UI registers
04/00:03:18|RunControl loaded
04/00:03:19|loading Html/AIDA/RunControl/implementation.tcl for namespace ::
04/00:03:19|AIDA RunControl loaded
04/00:03:19|Completed custom startup from /MIDAS/TclHttpd/Html/AIDA/RunControl/stats.defn.tcl
04/00:03:19|do_GetState returned z=0 and 8
04/09:31:05|do_GetState returned z=0 and 8
04/09:31:17|do_GetState returned z=0 and 8
04/09:31:30|do_GetState returned z=0 and 8
04/09:31:43|do_GetState returned z=0 and 8
04/09:31:55|do_GetState returned z=0 and 8
04/09:32:07|do_GetState returned z=0 and 8
04/09:32:19|do_GetState returned z=0 and 8
04/09:32:32|do_GetState returned z=0 and 8
04/09:32:45|do_GetState returned z=0 and 8
04/09:32:57|do_GetState returned z=0 and 8
04/09:33:10|do_GetState returned z=0 and 8
04/09:33:22|do_GetState returned z=0 and 8
04/09:33:35|do_GetState returned z=0 and 8
04/09:33:47|do_GetState returned z=0 and 8
04/09:34:00|do_GetState returned z=0 and 8
04/09:34:13|do_GetState returned z=0 and 8
04/09:34:25|do_GetState returned z=0 and 8
04/09:34:38|do_GetState returned z=0 and 8
04/09:34:50|do_GetState returned z=0 and 8
04/09:35:03|do_GetState returned z=0 and 8
04/09:35:16|do_GetState returned z=0 and 8
04/09:35:28|do_GetState returned z=0 and 8
04/09:35:41|do_GetState returned z=0 and 8
04/09:35:53|do_GetState returned z=0 and 8
04/09:36:06|do_GetState returned z=0 and 8
04/09:36:18|do_GetState returned z=0 and 8
04/09:36:31|do_GetState returned z=0 and 8
04/09:36:43|do_GetState returned z=0 and 8
04/09:36:56|do_GetState returned z=0 and 8
04/09:37:09|do_GetState returned z=0 and 8
04/09:37:21|do_GetState returned z=0 and 8
04/09:37:34|do_GetState returned z=0 and 8
04/09:37:46|do_GetState returned z=0 and 8
04/09:37:59|do_GetState returned z=0 and 8
04/09:38:12|do_GetState returned z=0 and 8
04/09:38:24|do_GetState returned z=0 and 8
04/09:38:37|do_GetState returned z=0 and 8
04/09:38:49|do_GetState returned z=0 and 8
04/09:39:02|do_GetState returned z=0 and 8
04/09:39:15|do_GetState returned z=0 and 8
04/09:39:27|do_GetState returned z=0 and 8
04/09:39:40|do_GetState returned z=0 and 8
04/09:39:52|do_GetState returned z=0 and 8
04/09:40:05|do_GetState returned z=0 and 8
04/09:40:18|do_GetState returned z=0 and 8
04/09:40:30|do_GetState returned z=0 and 8
04/09:40:43|do_GetState returned z=0 and 8
04/09:40:55|do_GetState returned z=0 and 8
04/09:41:08|do_GetState returned z=0 and 8
04/09:41:21|do_GetState returned z=0 and 8
04/09:41:33|do_GetState returned z=0 and 8
04/09:41:46|do_GetState returned z=0 and 8
04/09:41:58|do_GetState returned z=0 and 8
04/09:42:11|do_GetState returned z=0 and 8
04/09:42:23|do_GetState returned z=0 and 8
04/09:42:36|do_GetState returned z=0 and 8
04/09:42:48|do_GetState returned z=0 and 8
04/09:43:01|do_GetState returned z=0 and 8
04/09:43:14|do_GetState returned z=0 and 8
04/09:43:26|do_GetState returned z=0 and 8
04/09:43:39|do_GetState returned z=0 and 8
04/09:43:51|do_GetState returned z=0 and 8
04/09:44:04|do_GetState returned z=0 and 8
04/09:44:17|do_GetState returned z=0 and 8
04/09:44:29|do_GetState returned z=0 and 8
04/09:44:42|do_GetState returned z=0 and 8
04/09:44:54|do_GetState returned z=0 and 8
04/09:45:07|do_GetState returned z=0 and 8
04/09:45:19|do_GetState returned z=0 and 8
04/09:45:32|do_GetState returned z=0 and 8
04/09:45:44|do_GetState returned z=0 and 8
04/09:45:57|do_GetState returned z=0 and 8
04/09:46:09|do_GetState returned z=0 and 8
04/09:46:22|do_GetState returned z=0 and 8
04/09:46:34|do_GetState returned z=0 and 8
04/09:46:47|do_GetState returned z=0 and 8
04/09:47:00|do_GetState returned z=0 and 8
04/09:47:12|do_GetState returned z=0 and 8
04/09:47:25|do_GetState returned z=0 and 8
04/09:47:37|do_GetState returned z=0 and 8
04/09:47:50|do_GetState returned z=0 and 8
04/09:48:03|do_GetState returned z=0 and 8
04/09:48:15|do_GetState returned z=0 and 8
04/09:48:28|do_GetState returned z=0 and 8
04/09:48:40|do_GetState returned z=0 and 8
04/09:48:53|do_GetState returned z=0 and 8
04/09:49:05|do_GetState returned z=0 and 8
04/09:49:18|do_GetState returned z=0 and 8
04/09:49:31|do_GetState returned z=0 and 8
04/09:49:43|do_GetState returned z=0 and 8
04/09:49:56|do_GetState returned z=0 and 8
04/09:50:08|do_GetState returned z=0 and 8
04/09:50:21|do_GetState returned z=0 and 8
04/09:50:33|do_GetState returned z=0 and 8
04/09:50:46|do_GetState returned z=0 and 8
04/09:50:58|do_GetState returned z=0 and 8
04/09:51:11|do_GetState returned z=0 and 8
04/09:51:23|do_GetState returned z=0 and 8
04/09:51:36|do_GetState returned z=0 and 8
04/09:51:49|do_GetState returned z=0 and 8
04/09:52:01|do_GetState returned z=0 and 8
04/09:52:14|do_GetState returned z=0 and 8
04/09:52:26|do_GetState returned z=0 and 8
04/09:52:39|do_GetState returned z=0 and 8
04/09:52:51|do_GetState returned z=0 and 8
04/09:53:04|do_GetState returned z=0 and 8
04/09:53:16|do_GetState returned z=0 and 8
04/09:53:29|do_GetState returned z=0 and 8
04/09:53:42|do_GetState returned z=0 and 8
04/09:53:54|do_GetState returned z=0 and 8
04/09:54:07|do_GetState returned z=0 and 8
04/09:54:19|do_GetState returned z=0 and 8
04/09:54:32|do_GetState returned z=0 and 8
04/09:54:44|do_GetState returned z=0 and 8
04/09:54:57|do_GetState returned z=0 and 8
04/09:55:09|do_GetState returned z=0 and 8
04/09:55:22|do_GetState returned z=0 and 8
04/09:55:34|do_GetState returned z=0 and 8
04/09:55:47|do_GetState returned z=0 and 8
04/09:56:00|do_GetState returned z=0 and 8
04/09:56:12|do_GetState returned z=0 and 8
04/09:56:25|do_GetState returned z=0 and 8
04/09:56:37|do_GetState returned z=0 and 8
04/09:56:50|do_GetState returned z=0 and 8
04/09:57:02|do_GetState returned z=0 and 8
04/09:57:15|do_GetState returned z=0 and 8
04/09:57:27|do_GetState returned z=0 and 8
04/09:57:40|do_GetState returned z=0 and 8
04/09:57:53|do_GetState returned z=0 and 8
04/09:58:05|do_GetState returned z=0 and 8
04/09:58:18|do_GetState returned z=0 and 8
04/09:58:30|do_GetState returned z=0 and 8
04/09:58:43|do_GetState returned z=0 and 8
04/09:58:56|do_GetState returned z=0 and 8
04/09:59:08|do_GetState returned z=0 and 8
04/09:59:21|do_GetState returned z=0 and 8
04/09:59:33|do_GetState returned z=0 and 8
04/09:59:46|do_GetState returned z=0 and 8
04/09:59:58|do_GetState returned z=0 and 8
04/10:00:11|do_GetState returned z=0 and 8
04/10:00:23|do_GetState returned z=0 and 8
04/10:00:36|do_GetState returned z=0 and 8
04/10:00:48|do_GetState returned z=0 and 8
04/10:01:01|do_GetState returned z=0 and 8
04/10:01:13|do_GetState returned z=0 and 8
04/10:01:26|do_GetState returned z=0 and 8
04/10:01:38|do_GetState returned z=0 and 8
04/10:01:51|do_GetState returned z=0 and 8
04/10:02:04|do_GetState returned z=0 and 8
04/10:02:16|do_GetState returned z=0 and 8
04/10:02:29|do_GetState returned z=0 and 8
04/10:02:41|do_GetState returned z=0 and 8
04/10:02:54|do_GetState returned z=0 and 8
04/10:03:06|do_GetState returned z=0 and 8
04/10:03:19|do_GetState returned z=0 and 8
04/10:03:31|do_GetState returned z=0 and 8
04/10:03:44|do_GetState returned z=0 and 8
04/10:03:57|do_GetState returned z=0 and 8
04/10:04:09|do_GetState returned z=0 and 8
04/10:04:22|do_GetState returned z=0 and 8
04/10:04:34|do_GetState returned z=0 and 8
04/10:04:47|do_GetState returned z=0 and 8
04/10:04:59|do_GetState returned z=0 and 8
04/10:05:12|do_GetState returned z=0 and 8
04/10:05:25|do_GetState returned z=0 and 8
04/10:05:37|do_GetState returned z=0 and 8
04/10:05:50|do_GetState returned z=0 and 8
04/10:06:02|do_GetState returned z=0 and 8
04/10:06:15|do_GetState returned z=0 and 8
04/10:06:27|do_GetState returned z=0 and 8
04/10:06:40|do_GetState returned z=0 and 8
04/10:06:52|do_GetState returned z=0 and 8
04/10:07:05|do_GetState returned z=0 and 8
04/10:07:18|do_GetState returned z=0 and 8
04/10:07:30|do_GetState returned z=0 and 8 |
|
355
|
Fri Jun 4 08:40:52 2021 |
TD | Friday 4 June |
09.20 DAQ found stopped
files in /TapeData/S496 indicate increased data rate yesterday evening from c. 18.00 until data stopped at c. midnight
aida15 rebooted at 00.02
regained control of DAQ and issued DAQ STOP by restarting NewMerger *twice*
11.00 DAQ reset and setup
All system wide checks OK *except*
Base Current Difference
aida05 fault 0x4d3c : 0x4d3c : 0
aida05 : WR status 0x10
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 2 4 2 8 5 4 1 2 2 4 7 : 41352
aida02 : 0 3 3 6 6 4 1 2 2 4 7 : 41352
aida03 : 2 4 2 12 1 3 2 2 2 4 7 : 41352
aida04 : 2 2 1 7 4 3 2 2 2 4 7 : 41352
aida05 : 2 3 1 5 3 2 3 2 2 4 7 : 41360
aida06 : 2 4 2 6 4 3 2 2 2 4 7 : 41352
aida07 : 1 5 2 9 4 2 4 3 1 4 7 : 41324
aida08 : 0 3 1 7 4 3 4 3 1 4 7 : 41352
aida09 : 3 3 0 8 2 2 3 2 2 4 7 : 41380
aida10 : 1 2 2 11 3 2 3 3 3 3 7 : 41044
aida11 : 2 3 1 6 6 4 2 3 3 3 7 : 41072
aida12 : 4 2 1 8 3 3 3 3 1 4 7 : 41072
aida13 : 2 4 1 8 4 4 2 3 3 3 7 : 41016
aida14 : 1 4 1 9 4 4 2 3 3 3 7 : 41044
aida15 : 0 5 1 2 2 1 2 4 3 4 7 : 42872
aida16 : 1 3 2 9 4 4 4 4 2 3 7 : 41052
Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same
FEE : aida01 => Options file size is 1026 Last changed Fri Jun 04 09:30:42 CEST 2021
FEE : aida02 => Options file size is 1025 Last changed Sun May 23 00:19:21 CEST 2021
FEE : aida03 => Options file size is 1014 Last changed Thu Apr 29 14:43:50 CEST 2021
FEE : aida04 => Options file size is 1025 Last changed Fri May 14 16:54:56 CEST 2021
FEE : aida05 => Options file size is 1025 Last changed Mon May 17 06:25:41 CEST 2021
FEE : aida06 => Options file size is 1014 Last changed Thu Apr 29 14:43:59 CEST 2021
FEE : aida07 => Options file size is 1014 Last changed Thu Apr 29 14:44:02 CEST 2021
FEE : aida08 => Options file size is 1025 Last changed Sun May 23 00:16:54 CEST 2021
FEE : aida09 => Options file size is 1014 Last changed Thu Apr 29 14:44:08 CEST 2021
FEE : aida10 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida11 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida12 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida13 => Options file size is 1025 Last changed Fri May 07 19:40:34 CEST 2021
FEE : aida14 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida15 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida16 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
11.03 DAQ resumes file S496/R32
alpha background
slow comparator 0x64
BNC PB-5 pulser OFF
11.09 1.8.W spectra - 20us FSR - attachments 1 & 2
Grafana - DSSSD bias & leakage current - most recent 7 days - attachment 3
ADC data items - attachment 4
FEE64 temperatures OK - attachment 5
DSSSD bias & leakage currents OK - attachment 6
Merger/Tape Server/Merger statistics - attachment 7
no merger errors reported since previous restart
13.07 analysis of file S496/R32_16 - attachment 8
max deadtime 0.18% aida04 |
|
354
|
Thu Jun 3 10:16:39 2021 |
TD | Thursday 3 June | 11.19 Hot HEC channels multiple FEE64s
ASIC check fixes
System wide checks OK * except*
Base Current Difference
aida01 fault 0x8940 : 0x8941 : 1
aida02 fault 0x9b32 : 0x9b33 : 1
aida03 fault 0xf266 : 0xf267 : 1
aida04 fault 0x579b : 0x579c : 1
White Rabbit error counter test result: Passed 12, Failed 4
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 11 4 2 2 2 2 2 3 2 4 7 : 41516
aida02 : 10 7 2 0 1 2 2 3 2 4 7 : 41408
aida03 : 10 8 2 2 2 4 1 4 2 3 7 : 40008
aida04 : 5 4 4 2 2 3 1 4 2 3 7 : 39860
aida05 : 12 3 2 2 1 2 2 3 2 4 7 : 41448
aida06 : 15 9 2 2 3 2 1 3 2 4 7 : 41380
aida07 : 12 8 1 2 2 3 3 2 2 4 7 : 41408
aida08 : 9 6 3 1 2 3 3 2 2 4 7 : 41380
aida09 : 15 6 1 1 2 2 2 3 2 4 7 : 41500
aida10 : 15 6 3 2 1 3 3 2 1 4 7 : 40348
aida11 : 8 2 1 1 1 3 1 3 2 4 7 : 41248
aida12 : 18 3 2 1 3 3 2 2 2 4 7 : 41184
aida13 : 7 4 1 4 3 4 3 3 1 4 7 : 41100
aida14 : 18 6 1 4 3 4 2 2 1 4 7 : 40392
aida15 : 7 7 1 3 1 2 1 3 2 4 7 : 41220
aida16 : 4 4 2 3 3 3 2 4 1 4 7 : 41200
Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same
FEE : aida01 => Options file size is 1026 Last changed Wed Jun 02 16:27:30 CEST 2021
FEE : aida02 => Options file size is 1025 Last changed Sun May 23 00:19:21 CEST 2021
FEE : aida03 => Options file size is 1014 Last changed Thu Apr 29 14:43:50 CEST 2021
FEE : aida04 => Options file size is 1025 Last changed Fri May 14 16:54:56 CEST 2021
FEE : aida05 => Options file size is 1025 Last changed Mon May 17 06:25:41 CEST 2021
FEE : aida06 => Options file size is 1014 Last changed Thu Apr 29 14:43:59 CEST 2021
FEE : aida07 => Options file size is 1014 Last changed Thu Apr 29 14:44:02 CEST 2021
FEE : aida08 => Options file size is 1025 Last changed Sun May 23 00:16:54 CEST 2021
FEE : aida09 => Options file size is 1014 Last changed Thu Apr 29 14:44:08 CEST 2021
FEE : aida10 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida11 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida12 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida13 => Options file size is 1025 Last changed Fri May 07 19:40:34 CEST 2021
FEE : aida14 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida15 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida16 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
Grafana - DSSSD bias & leakage current - most recent 7 days - attachment 1
1.8.W spectra - 20us FSR - attachments 2 & 3
ADC data items - attachment 4
FEE64 temperatures OK - attachment 5
DSSSD bias & leakage currents OK - attachment 6
Merger/Tape Server/Merger statistics - attachment 7
no merger errors reported since previous restart
11.43 Fixed merger issue which stopped output from merger - see https://elog.ph.ed.ac.uk/DESPEC/36
13.09 analysis of file S496/R31_16 - attachment 8
max deadtime 0.31% aida04 |
|
353
|
Wed Jun 2 14:53:05 2021 |
OH, TD | Merger/Tapeserver issues after reboot of aida-3 | On 01/06/21 aida-3 was rebooted as the graphical user interface had frozen.
We were also unable to connected to MIDAS via ssh port forwarding.
Since the restart we have been unable to write to file.
The tapeserver will allocate/mount/open a file but nothing will be written to it.
The merger will be receiving data but never shows as having any "Current links with data".
/MIDAS currently points to lrwxrwxrwx. 1 root root 45 Jan 23 2019 /MIDAS -> /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119
The current disk structure is:
[npg@aidas-gsi npg]$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sr0 11:0 1 1024M 0 rom
sdb 8:16 0 7.3T 0 disk
├─sdb1 8:17 0 200M 0 part
├─sdb2 8:18 0 500M 0 part
└─sdb3 8:19 0 7.3T 0 part
├─vg_aidas2-lv_root (dm-2)
253:2 0 50G 0 lvm
├─vg_aidas2-lv_home (dm-3)
253:3 0 7.2T 0 lvm /media/1e121361-83d3-4825-b6ae-8700b07e0ca7
└─vg_aidas2-lv_swap (dm-4)
253:4 0 7.8G 0 lvm
sdc 8:32 0 7.3T 0 disk
└─sdc1 8:33 0 7.3T 0 part /media/ThirdDrive
sdd 8:48 0 7.3T 0 disk
└─sdd1 8:49 0 7.3T 0 part /media/SecondDrive
sda 8:0 0 465.8G 0 disk
├─sda1 8:1 0 200M 0 part /boot/efi
├─sda2 8:2 0 500M 0 part /boot
└─sda3 8:3 0 465.1G 0 part
├─vg_aidasgsi-lv_root (dm-0)
253:0 0 50G 0 lvm /
├─vg_aidasgsi-lv_swap (dm-1)
253:1 0 7.8G 0 lvm [SWAP]
└─vg_aidasgsi-lv_home (dm-5)
253:5 0 407.3G 0 lvm /home
Terminal outputs for the HTTPd, Merger and TapeServer terminals - attachments 1-3
TapeServer config file - attachment 4
Merger options - attachment 5
Note in the merger window the top bar is different to what we have seen previously - Attachment 6 and https://elog.ph.ed.ac.uk/DESPEC/210418_124830/51.png |
|
352
|
Sun May 30 08:57:53 2021 |
TD | Sunday 30 May | 09.55 DAQ continues - file S496/R18_588
alpha background
increased data rate due to hot HEC channels since c. 06.00 this am - ASIC check fixes problem
All system wide checks OK *except*
Base Current Difference
aida01 fault 0x82b0 : 0x82b7 : 7
aida02 fault 0xfe72 : 0xfe79 : 7
aida03 fault 0x97f4 : 0x97fb : 7
aida04 fault 0x950 : 0x957 : 7
aida05 fault 0x8e93 : 0x8e98 : 5
aida06 fault 0x545a : 0x545f : 5
aida07 fault 0xc04d : 0xc052 : 5
aida08 fault 0xa0f7 : 0xa0fc : 5
White Rabbit error counter test result: Passed 8, Failed 8
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 6 9 5 1 3 2 2 2 2 4 6 : 37008
aida02 : 3 5 5 2 2 4 3 3 2 4 6 : 37956
aida03 : 22 12 7 5 3 3 2 3 3 3 6 : 36872
aida04 : 40 13 5 5 3 4 2 3 3 3 6 : 37048
aida05 : 15 5 5 4 2 2 2 2 2 4 6 : 37044
aida06 : 11 6 2 1 2 3 3 4 2 4 6 : 38300
aida07 : 17 5 4 3 1 3 3 2 2 4 6 : 37324
aida08 : 1 8 1 2 2 4 4 3 2 4 6 : 38164
aida09 : 10 8 6 1 2 4 3 3 1 4 6 : 36968
aida10 : 14 7 5 2 1 3 3 3 1 4 6 : 36800
aida11 : 7 9 3 1 2 3 3 3 1 4 6 : 36788
aida12 : 3 5 2 3 1 4 2 3 2 4 6 : 37620
aida13 : 12 4 4 2 1 3 2 2 2 4 6 : 37008
aida14 : 11 10 5 1 1 3 2 2 2 4 6 : 37036
aida15 : 11 5 1 2 1 3 3 4 2 4 6 : 38244
aida16 : 10 4 8 1 2 3 2 2 2 4 6 : 37096
Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same
FEE : aida01 => Options file size is 1026 Last changed Sun May 23 12:04:25 CEST 2021
FEE : aida02 => Options file size is 1025 Last changed Sun May 23 00:19:21 CEST 2021
FEE : aida03 => Options file size is 1014 Last changed Thu Apr 29 14:43:50 CEST 2021
FEE : aida04 => Options file size is 1025 Last changed Fri May 14 16:54:56 CEST 2021
FEE : aida05 => Options file size is 1025 Last changed Mon May 17 06:25:41 CEST 2021
FEE : aida06 => Options file size is 1014 Last changed Thu Apr 29 14:43:59 CEST 2021
FEE : aida07 => Options file size is 1014 Last changed Thu Apr 29 14:44:02 CEST 2021
FEE : aida08 => Options file size is 1025 Last changed Sun May 23 00:16:54 CEST 2021
FEE : aida09 => Options file size is 1014 Last changed Thu Apr 29 14:44:08 CEST 2021
FEE : aida10 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida11 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida12 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida13 => Options file size is 1025 Last changed Fri May 07 19:40:34 CEST 2021
FEE : aida14 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida15 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida16 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
Grafana - DSSSD bias & leakage current - most recent 7 days - attachment 1
Lost activity monitor - attachment 2
1.8.W spectra - attachments 3 & 4
ADC data item stats - attachment 5
FEE64 temps OK - attachment 6
DSSSD bias & leakage current OK - attachment 7
Merger/Tape Server/Merger stats - attachment 8
no merger errors reported since previous restart
14.24 All histograms, statistics and merger statistics zero'd
16.16 analysis of file S496/R18_541 - attachment 9
max deadtime 0.02% aida03
18.40 DAQ continues - file S496/R18_589
alpha background
increased data rate due to hot HEC channels since c. 18.00 this am - ASIC check fixes problem |
|
351
|
Sat May 29 13:13:45 2021 |
TD | Saturday 29 May | 14.10 DAQ continues - file S496/R18_539
alpha background
increased data rate due to ht HEC channels since c. 06.00 this am - ASIC check fixes problem
All system wide checks OK *except*
Base Current Difference
aida01 fault 0x82b0 : 0x82b7 : 7
aida02 fault 0xfe72 : 0xfe79 : 7
aida03 fault 0x97f4 : 0x97fb : 7
aida04 fault 0x950 : 0x957 : 7
aida05 fault 0x8e93 : 0x8e97 : 4
aida06 fault 0x545a : 0x545e : 4
aida07 fault 0xc04d : 0xc051 : 4
aida08 fault 0xa0f7 : 0xa0fb : 4
White Rabbit error counter test result: Passed 8, Failed 8
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 15 8 7 2 2 2 2 2 2 4 6 : 37036
aida02 : 14 3 5 2 2 4 3 3 2 4 6 : 37984
aida03 : 15 11 8 6 3 3 2 3 3 3 6 : 36884
aida04 : 43 13 5 5 3 4 2 3 3 3 6 : 37060
aida05 : 19 3 5 4 2 2 2 2 2 4 6 : 37044
aida06 : 17 4 1 3 2 3 3 4 2 4 6 : 38356
aida07 : 16 6 2 1 0 2 2 3 2 4 6 : 37296
aida08 : 21 8 1 1 3 4 4 3 2 4 6 : 38276
aida09 : 13 10 2 1 2 3 2 2 2 4 6 : 37060
aida10 : 13 9 4 1 1 3 3 3 1 4 6 : 36764
aida11 : 13 9 2 1 2 3 3 3 1 4 6 : 36796
aida12 : 11 6 4 2 2 4 2 3 2 4 6 : 37724
aida13 : 7 6 6 2 1 3 2 2 2 4 6 : 37036
aida14 : 9 7 7 1 1 3 2 2 2 4 6 : 37036
aida15 : 14 5 2 2 1 3 3 4 2 4 6 : 38272
aida16 : 9 4 10 1 2 3 2 2 2 4 6 : 37124
Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same
FEE : aida01 => Options file size is 1026 Last changed Sun May 23 12:04:25 CEST 2021
FEE : aida02 => Options file size is 1025 Last changed Sun May 23 00:19:21 CEST 2021
FEE : aida03 => Options file size is 1014 Last changed Thu Apr 29 14:43:50 CEST 2021
FEE : aida04 => Options file size is 1025 Last changed Fri May 14 16:54:56 CEST 2021
FEE : aida05 => Options file size is 1025 Last changed Mon May 17 06:25:41 CEST 2021
FEE : aida06 => Options file size is 1014 Last changed Thu Apr 29 14:43:59 CEST 2021
FEE : aida07 => Options file size is 1014 Last changed Thu Apr 29 14:44:02 CEST 2021
FEE : aida08 => Options file size is 1025 Last changed Sun May 23 00:16:54 CEST 2021
FEE : aida09 => Options file size is 1014 Last changed Thu Apr 29 14:44:08 CEST 2021
FEE : aida10 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida11 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida12 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida13 => Options file size is 1025 Last changed Fri May 07 19:40:34 CEST 2021
FEE : aida14 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida15 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida16 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
Grafana - DSSSD bias & leakage current - most recent 7 days - attachment 1
Lost activity monitor - attachment 2
1.8.W spectra - attachments 3 & 4
ADC data item stats - attachment 5
FEE64 temps OK - attachment 6
DSSSD bias & leakage current OK - attachment 7
Merger/Tape Server/Merger stats - attachment 8
no merger errors reported since previous restart
14.24 All histograms, statistics and merger statistics zero'd
16.16 analysis of file S496/R18_541 - attachment 9
max deadtime 0.12% aida04
18.40 DAQ continues - file S496/R18_551
alpha background
increased data rate due to ht HEC channels since c. 18.00 this am - ASIC check fixes problem |
|
350
|
Fri May 28 10:29:07 2021 |
TD | Friday 28 May | 11.30 DAQ continues - file S496/R18_472
alpha background
All system wide checks OK *except*
FEE64 module aida06 global clocks failed, 6
Clock status test result: Passed 15, Failed 1
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable
Base Current Difference
aida01 fault 0x82b0 : 0x82b2 : 2
aida02 fault 0xfe72 : 0xfe74 : 2
aida03 fault 0x97f4 : 0x97f6 : 2
aida04 fault 0x950 : 0x952 : 2
aida05 fault 0x8e93 : 0x8e95 : 2
aida06 fault 0x545a : 0x545c : 2
aida07 fault 0xc04d : 0xc04f : 2
aida08 fault 0xa0f7 : 0xa0f9 : 2
White Rabbit error counter test result: Passed 8, Failed 8
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 1 4 7 1 3 2 2 2 2 4 6 : 36980
aida02 : 3 1 5 1 2 4 3 3 2 4 6 : 37892
aida03 : 8 8 6 6 3 3 2 3 3 3 6 : 36800
aida04 : 37 12 5 6 3 4 2 3 3 3 6 : 37060
aida05 : 11 5 6 4 2 2 2 2 2 4 6 : 37044
aida06 : 4 3 3 3 2 2 2 3 3 4 6 : 38456
aida07 : 1 12 3 2 0 3 3 2 2 4 6 : 37204
aida08 : 13 8 1 2 3 4 3 3 2 4 6 : 38020
aida09 : 7 7 5 1 1 3 2 2 2 4 6 : 36996
aida10 : 12 7 9 2 1 3 3 3 1 4 6 : 36856
aida11 : 2 9 3 1 2 3 3 3 1 4 6 : 36768
aida12 : 11 4 3 3 2 4 2 3 2 4 6 : 37724
aida13 : 10 8 6 2 2 3 2 2 2 4 6 : 37128
aida14 : 16 7 7 1 0 4 2 2 2 4 6 : 37128
aida15 : 12 6 2 2 1 3 3 4 2 4 6 : 38272
aida16 : 9 4 10 1 2 3 2 2 2 4 6 : 37124
Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same
FEE : aida01 => Options file size is 1026 Last changed Sun May 23 12:04:25 CEST 2021
FEE : aida02 => Options file size is 1025 Last changed Sun May 23 00:19:21 CEST 2021
FEE : aida03 => Options file size is 1014 Last changed Thu Apr 29 14:43:50 CEST 2021
FEE : aida04 => Options file size is 1025 Last changed Fri May 14 16:54:56 CEST 2021
FEE : aida05 => Options file size is 1025 Last changed Mon May 17 06:25:41 CEST 2021
FEE : aida06 => Options file size is 1014 Last changed Thu Apr 29 14:43:59 CEST 2021
FEE : aida07 => Options file size is 1014 Last changed Thu Apr 29 14:44:02 CEST 2021
FEE : aida08 => Options file size is 1025 Last changed Sun May 23 00:16:54 CEST 2021
FEE : aida09 => Options file size is 1014 Last changed Thu Apr 29 14:44:08 CEST 2021
FEE : aida10 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida11 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida12 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida13 => Options file size is 1025 Last changed Fri May 07 19:40:34 CEST 2021
FEE : aida14 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida15 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida16 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
Grafana - DSSSD bias & leakage current - most recent 7 days - attachment 1
Lost activity monitor - attachment 2
1.8.W spectra - attachments 3 & 4
ADC data item stats - attachment 5
FEE64 temps OK - attachment 6
DSSSD bias & leakage current OK - attachment 7
Merger/Tape Server/Merger stats - attachment 8
no merger errors reported since previous restart
14.30 analysis of file S496/R18_471 - attachment 9
max deadtime 0.18% aida04 |
|
349
|
Thu May 27 12:25:53 2021 |
TD | Thursday 27 May | 13.29 DAQ continues - file S496/R18_443
alpha background
All system wide checks OK *except*
FEE64 module aida06 global clocks failed, 6
Clock status test result: Passed 15, Failed 1
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable
Base Current Difference
aida01 fault 0x82b0 : 0x82b2 : 2
aida02 fault 0xfe72 : 0xfe74 : 2
aida03 fault 0x97f4 : 0x97f6 : 2
aida04 fault 0x950 : 0x952 : 2
aida05 fault 0x8e93 : 0x8e95 : 2
aida06 fault 0x545a : 0x545c : 2
aida07 fault 0xc04d : 0xc04f : 2
aida08 fault 0xa0f7 : 0xa0f9 : 2
White Rabbit error counter test result: Passed 8, Failed 8
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 2 7 7 1 3 2 2 2 2 4 6 : 37008
aida02 : 3 1 5 1 3 4 3 3 2 4 6 : 37956
aida03 : 2 6 5 6 2 4 2 3 3 3 6 : 36808
aida04 : 24 12 5 5 4 3 2 3 3 3 6 : 36912
aida05 : 8 1 9 3 2 2 2 2 2 4 6 : 37016
aida06 : 3 6 2 2 2 2 2 3 3 4 6 : 38428
aida07 : 6 17 1 2 0 3 3 2 2 4 6 : 37232
aida08 : 4 2 1 2 3 4 3 3 2 4 6 : 37936
aida09 : 15 7 5 2 3 4 3 3 1 4 6 : 37060
aida10 : 7 6 9 2 0 3 3 3 1 4 6 : 36764
aida11 : 19 10 2 0 2 3 3 3 1 4 6 : 36796
aida12 : 11 6 4 2 2 4 2 3 2 4 6 : 37724
aida13 : 17 9 2 0 0 4 2 2 2 4 6 : 37036
aida14 : 15 4 7 1 1 3 2 2 2 4 6 : 37036
aida15 : 9 6 1 2 1 3 3 4 2 4 6 : 38244
aida16 : 11 11 8 2 0 3 2 2 2 4 6 : 37060
Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same
FEE : aida01 => Options file size is 1026 Last changed Sun May 23 12:04:25 CEST 2021
FEE : aida02 => Options file size is 1025 Last changed Sun May 23 00:19:21 CEST 2021
FEE : aida03 => Options file size is 1014 Last changed Thu Apr 29 14:43:50 CEST 2021
FEE : aida04 => Options file size is 1025 Last changed Fri May 14 16:54:56 CEST 2021
FEE : aida05 => Options file size is 1025 Last changed Mon May 17 06:25:41 CEST 2021
FEE : aida06 => Options file size is 1014 Last changed Thu Apr 29 14:43:59 CEST 2021
FEE : aida07 => Options file size is 1014 Last changed Thu Apr 29 14:44:02 CEST 2021
FEE : aida08 => Options file size is 1025 Last changed Sun May 23 00:16:54 CEST 2021
FEE : aida09 => Options file size is 1014 Last changed Thu Apr 29 14:44:08 CEST 2021
FEE : aida10 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida11 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida12 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida13 => Options file size is 1025 Last changed Fri May 07 19:40:34 CEST 2021
FEE : aida14 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida15 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida16 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
Grafana - DSSSD bias & leakage current - most recent 7 days - attachment 1
Lost activity monitor - attachment 2
1.8.W spectra - attachments 3 & 4
ADC data item stats - attachment 5
FEE64 temps OK - attachment 6
DSSSD bias & leakage current OK - attachment 7
Merger/Tape Server/Merger stats - attachment 8
no merger errors reported since previous restart
14.30 analysis of file S496/R18_442 - attachment 9
max deadtime 0.28% aida04 |
|
348
|
Wed May 26 13:21:59 2021 |
TD | Wednesday 26 May |
14.22 DAQ continues - file S496/R18_369
alpha background
All system wide checks OK *except*
Base Current Difference
aida01 fault 0x82b0 : 0x82b1 : 1
aida02 fault 0xfe72 : 0xfe73 : 1
aida03 fault 0x97f4 : 0x97f5 : 1
aida04 fault 0x950 : 0x951 : 1
aida05 fault 0x8e93 : 0x8e94 : 1
aida06 fault 0x545a : 0x545b : 1
aida07 fault 0xc04d : 0xc04e : 1
aida08 fault 0xa0f7 : 0xa0f8 : 1
White Rabbit error counter test result: Passed 8, Failed 8
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 12 4 6 1 2 3 2 2 2 4 6 : 37072
aida02 : 12 2 4 1 3 4 3 3 2 4 6 : 37984
aida03 : 12 6 7 6 2 4 3 2 3 3 6 : 36624
aida04 : 36 11 4 6 4 3 2 3 3 3 6 : 36968
aida05 : 2 4 6 3 2 2 2 2 2 4 6 : 36968
aida06 : 8 5 3 2 2 2 2 3 3 4 6 : 38456
aida07 : 10 4 6 4 3 3 3 3 2 4 6 : 37992
aida08 : 21 8 3 2 3 3 3 3 2 4 6 : 37956
aida09 : 17 10 3 2 3 4 3 3 1 4 6 : 37060
aida10 : 3 6 10 2 1 3 3 3 1 4 6 : 36828
aida11 : 12 6 4 2 2 3 3 3 1 4 6 : 36832
aida12 : 2 9 5 1 3 3 2 3 2 4 6 : 37632
aida13 : 4 7 2 1 3 3 3 3 2 4 6 : 37832
aida14 : 15 6 6 1 0 4 2 2 2 4 6 : 37100
aida15 : 5 4 3 2 1 3 3 4 2 4 6 : 38244
aida16 : 17 6 6 2 1 3 2 2 2 4 6 : 37076
Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same
FEE : aida01 => Options file size is 1026 Last changed Sun May 23 12:04:25 CEST 2021
FEE : aida02 => Options file size is 1025 Last changed Sun May 23 00:19:21 CEST 2021
FEE : aida03 => Options file size is 1014 Last changed Thu Apr 29 14:43:50 CEST 2021
FEE : aida04 => Options file size is 1025 Last changed Fri May 14 16:54:56 CEST 2021
FEE : aida05 => Options file size is 1025 Last changed Mon May 17 06:25:41 CEST 2021
FEE : aida06 => Options file size is 1014 Last changed Thu Apr 29 14:43:59 CEST 2021
FEE : aida07 => Options file size is 1014 Last changed Thu Apr 29 14:44:02 CEST 2021
FEE : aida08 => Options file size is 1025 Last changed Sun May 23 00:16:54 CEST 2021
FEE : aida09 => Options file size is 1014 Last changed Thu Apr 29 14:44:08 CEST 2021
FEE : aida10 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida11 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida12 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida13 => Options file size is 1025 Last changed Fri May 07 19:40:34 CEST 2021
FEE : aida14 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida15 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida16 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
Grafana - DSSSD bias & leakage cuuent - most recent 7 days - attachment 1
Lost activity monitor - attachment 2
1.8.W spectra - attachments 3 & 4
ADC data item stats - attachment 5
FEE64 temps OK - attachment 6
DSSSD bias & leakage current OK - attachment 7
Merger/Tape Server/Merger stats - attachment 8
no merger errors reported since previous restart
14.30 analysis of file S496/R18_340 - attachment 9
max deadtime 0.31% aida04 |
|
347
|
Tue May 25 11:16:54 2021 |
TD | Tuesday 25 May | 12.16 DAQ continues file S496/R18_324
alpha background
Tape server c. 2.5Mb/s
ASIC check
Tape server c. 300kb/s - presumably hot HEC channels
12.20 All system wide checks OK *except*
Base Current Difference
aida01 fault 0x82b0 : 0x82b1 : 1
aida02 fault 0xfe72 : 0xfe73 : 1
aida03 fault 0x97f4 : 0x97f5 : 1
aida04 fault 0x950 : 0x951 : 1
aida05 fault 0x8e93 : 0x8e94 : 1
aida06 fault 0x545a : 0x545b : 1
aida07 fault 0xc04d : 0xc04e : 1
aida08 fault 0xa0f7 : 0xa0f8 : 1
White Rabbit error counter test result: Passed 8, Failed 8
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 2 6 6 2 1 3 2 2 2 4 6 : 37016
aida02 : 3 6 5 3 3 4 3 2 1 3 7 : 38572
aida03 : 21 10 8 6 3 3 2 3 2 4 6 : 37924
aida04 : 29 15 4 5 3 4 2 3 3 3 6 : 37004
aida05 : 17 5 2 2 2 2 2 2 2 4 6 : 36940
aida06 : 1 6 0 1 2 2 2 3 3 4 6 : 38356
aida07 : 11 6 7 4 3 3 3 3 2 4 6 : 38028
aida08 : 13 8 3 2 2 3 3 2 1 3 7 : 38372
aida09 : 7 9 2 1 3 3 3 3 2 4 6 : 37860
aida10 : 9 8 4 2 0 3 3 3 1 4 6 : 36708
aida11 : 16 6 3 2 2 3 3 3 1 4 6 : 36832
aida12 : 21 10 7 4 2 4 2 2 1 3 7 : 38420
aida13 : 8 3 3 1 3 3 3 3 2 4 6 : 37832
aida14 : 14 3 6 1 0 4 2 2 2 4 6 : 37072
aida15 : 2 4 2 2 1 3 3 4 2 4 6 : 38216
aida16 : 16 5 5 3 0 3 2 2 2 4 6 : 37016
Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same
FEE : aida01 => Options file size is 1026 Last changed Sun May 23 12:04:25 CEST 2021
FEE : aida02 => Options file size is 1025 Last changed Sun May 23 00:19:21 CEST 2021
FEE : aida03 => Options file size is 1014 Last changed Thu Apr 29 14:43:50 CEST 2021
FEE : aida04 => Options file size is 1025 Last changed Fri May 14 16:54:56 CEST 2021
FEE : aida05 => Options file size is 1025 Last changed Mon May 17 06:25:41 CEST 2021
FEE : aida06 => Options file size is 1014 Last changed Thu Apr 29 14:43:59 CEST 2021
FEE : aida07 => Options file size is 1014 Last changed Thu Apr 29 14:44:02 CEST 2021
FEE : aida08 => Options file size is 1025 Last changed Sun May 23 00:16:54 CEST 2021
FEE : aida09 => Options file size is 1014 Last changed Thu Apr 29 14:44:08 CEST 2021
FEE : aida10 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida11 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida12 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida13 => Options file size is 1025 Last changed Fri May 07 19:40:34 CEST 2021
FEE : aida14 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida15 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida16 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
Grafana - DSSSD bias & leakage cuuent - most recent 7 days - attachment 1
Rate spectra - attachment 2
ADC data item stats - attachment 3
FEE64 temps OK - attachment 4
DSSSD bias & leakage current OK - attachment 5
Merger/Tape Server/Merger stats - attachment 6
no merger errors reported since previous restart
17.30 analysis of file S496/R18_340
max deadtime 0.36% aida04 |
|
346
|
Mon May 24 09:57:07 2021 |
TD | Monday 24 May |
07.30 Hot HEC channels multiple FEE64s
ASIC check fixes
10.56 DAQ continues OK file S496/R18_75
System wide checks OK * except*
FEE64 module aida06 global clocks failed, 6
Clock status test result: Passed 15, Failed 1
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable
Base Current Difference
aida01 fault 0x82b0 : 0x82b1 : 1
aida02 fault 0xfe72 : 0xfe73 : 1
aida03 fault 0x97f4 : 0x97f5 : 1
aida04 fault 0x950 : 0x951 : 1
aida05 fault 0x8e93 : 0x8e94 : 1
aida06 fault 0x545a : 0x545b : 1
aida07 fault 0xc04d : 0xc04e : 1
aida08 fault 0xa0f7 : 0xa0f8 : 1
White Rabbit error counter test result: Passed 8, Failed 8
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 1 2 7 2 1 4 3 2 1 4 7 : 40452
aida02 : 13 3 3 1 3 4 2 2 2 4 7 : 41308
aida03 : 17 6 2 2 1 4 2 3 2 3 7 : 39700
aida04 : 8 6 1 0 1 3 1 4 2 3 7 : 39712
aida05 : 10 3 5 3 4 3 3 2 1 4 7 : 40560
aida06 : 4 4 2 0 2 3 2 3 3 4 7 : 42576
aida07 : 5 3 2 3 1 3 1 3 2 4 7 : 41324
aida08 : 11 5 1 1 2 3 3 2 2 4 7 : 41348
aida09 : 7 4 3 0 2 3 1 3 2 4 7 : 41324
aida10 : 11 4 2 2 2 3 2 4 1 4 7 : 41132
aida11 : 18 5 0 2 2 4 2 2 1 4 7 : 40240
aida12 : 2 3 0 1 2 3 2 2 2 4 7 : 41024
aida13 : 4 5 1 2 2 3 2 4 1 4 7 : 41096
aida14 : 9 3 1 2 1 4 2 2 2 4 7 : 41164
aida15 : 2 3 1 3 2 2 3 3 2 4 7 : 41744
aida16 : 5 5 2 0 3 3 2 4 1 4 7 : 41116
Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same
FEE : aida01 => Options file size is 1026 Last changed Sun May 23 12:04:25 CEST 2021
FEE : aida02 => Options file size is 1025 Last changed Sun May 23 00:19:21 CEST 2021
FEE : aida03 => Options file size is 1014 Last changed Thu Apr 29 14:43:50 CEST 2021
FEE : aida04 => Options file size is 1025 Last changed Fri May 14 16:54:56 CEST 2021
FEE : aida05 => Options file size is 1025 Last changed Mon May 17 06:25:41 CEST 2021
FEE : aida06 => Options file size is 1014 Last changed Thu Apr 29 14:43:59 CEST 2021
FEE : aida07 => Options file size is 1014 Last changed Thu Apr 29 14:44:02 CEST 2021
FEE : aida08 => Options file size is 1025 Last changed Sun May 23 00:16:54 CEST 2021
FEE : aida09 => Options file size is 1014 Last changed Thu Apr 29 14:44:08 CEST 2021
FEE : aida10 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida11 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida12 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida13 => Options file size is 1025 Last changed Fri May 07 19:40:34 CEST 2021
FEE : aida14 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida15 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida16 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
Grafana - DSSSD bias & leakage current - most recent 7 days - attachment 1
Statistics - disc, pause, WR 28-47, correlation scaler - attachments 2-5
1.8.H spectra - attachments 6-8
1.8.L spectra - attachments 9 & 10
Stat spectra - attachments 11 & 12
1.8.W spectra - 20us FSR - attachments 13 & 14
ADC data items - attachment 15
FEE64 temperatures OK - attachment 16
DSSSD bias & leakage currents OK - attachment 17
Merger/Tape Server/Merger statistics - attachment 18
no merger errors reported since previous restart
analysis of file R18_74
max dead time 0.05%
10.17 ASIC check
All histograms, statistics, merger statistics zero'd |
|
345
|
Sat May 22 23:40:32 2021 |
TD | Sunday 23 May |
00.40 Unable to connect to FEE64s 1-7
DAQ power cycled OK
ASIC settings 2021Apr29-13-16-00
slow comparator 0x64 *all* FEE64s
BNC PB-5 OFF
file S496/R16
alpha background
07.17 Some hot HEC channels
ASIC check fixes issue
System wide checks OK *except*
Base Current Difference
aida06 fault 0x5457 : 0x5457 : 0
aida06 : WR status 0x10
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Base Current Difference
aida12 fault 0x0 : 0x1cd : 461
FPGA Timestamp error counter test result: Passed 15, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 16 9 4 3 1 4 3 2 1 4 7 : 40552
aida02 : 3 2 2 1 2 3 3 2 2 4 7 : 41308
aida03 : 14 5 0 1 1 3 2 4 2 3 7 : 40000
aida04 : 14 4 5 1 3 3 1 4 2 3 7 : 39944
aida05 : 12 4 1 3 2 3 3 2 2 4 7 : 41408
aida06 : 8 3 2 1 2 4 2 3 3 4 7 : 42744
aida07 : 10 4 3 2 1 2 2 3 2 4 7 : 41464
aida08 : 7 5 2 0 3 3 3 2 2 4 7 : 41380
aida09 : 10 5 1 1 1 2 2 3 2 4 7 : 41408
aida10 : 11 6 3 2 1 3 2 2 2 4 7 : 41100
aida11 : 12 8 3 2 2 2 3 2 1 4 7 : 40288
aida12 : 9 9 3 1 2 3 3 2 1 4 7 : 40380
aida13 : 10 4 0 1 1 2 1 3 2 4 7 : 41128
aida14 : 9 4 2 1 1 2 1 3 2 4 7 : 41156
aida15 : 5 4 0 3 3 3 3 3 2 4 7 : 41940
aida16 : 9 6 3 2 0 2 1 3 2 4 7 : 41156
Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same
FEE : aida01 => Options file size is 1026 Last changed Sun May 23 00:29:48 CEST 2021
FEE : aida02 => Options file size is 1025 Last changed Sun May 23 00:19:21 CEST 2021
FEE : aida03 => Options file size is 1014 Last changed Thu Apr 29 14:43:50 CEST 2021
FEE : aida04 => Options file size is 1025 Last changed Fri May 14 16:54:56 CEST 2021
FEE : aida05 => Options file size is 1025 Last changed Mon May 17 06:25:41 CEST 2021
FEE : aida06 => Options file size is 1014 Last changed Thu Apr 29 14:43:59 CEST 2021
FEE : aida07 => Options file size is 1014 Last changed Thu Apr 29 14:44:02 CEST 2021
FEE : aida08 => Options file size is 1025 Last changed Sun May 23 00:16:54 CEST 2021
FEE : aida09 => Options file size is 1014 Last changed Thu Apr 29 14:44:08 CEST 2021
FEE : aida10 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida11 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida12 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida13 => Options file size is 1025 Last changed Fri May 07 19:40:34 CEST 2021
FEE : aida14 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida15 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida16 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE64 temperatures OK - attachment 1
ADC data item stats - attachment 2
Grafana DSSSD bias & leakage currents most recent 7 days - attachment 3
DSSSD bias & leakage currents OK - attachment 4
11.18 Analysis of file S496/R16_33 - attachment 5
[npg@aidas-gsi S496]$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_aidasgsi-lv_root
50G 18G 30G 37% /
tmpfs 7.9G 685M 7.2G 9% /dev/shm
/dev/sda2 477M 42M 410M 10% /boot
/dev/sda1 200M 264K 200M 1% /boot/efi
/dev/mapper/vg_aidasgsi-lv_home
401G 90G 292G 24% /home
/dev/mapper/vg_aidas2-lv_home
7.2T 6.0T 867G 88% /media/1e121361-83d3-4825-b6ae-8700b07e0ca7
/dev/sdd1 7.2T 5.6T 1.3T 82% /media/SecondDrive
/dev/sdc1 7.2T 6.6T 259G 97% /media/ThirdDrive
867/2 x 3279s = 16.5 days to filesystem full - OK
12.11 aida15 zero data/poll stats - re-loaded DAQ program OK
aida04 unable to connect - multiple merger starts ineffective - aida04 rebooted and DAQ re-started OK
DAQ continues file S496/R18 |
|
344
|
Sat May 22 13:40:56 2021 |
TD | Saturday 22 May |
14.37 DAQ continues file S496/R15_14
Grafana - DSSSD bias & leakage current - attachment 1
1.8.L spectra - attachments 2 & 3
Stat spectra - attachments 4 & 5
FEE64 temperatures OK - attachment 6
stats - ADC data items, pause items OK - attachments 7 & 8
DSSSD bias & leakage current OK - attachment 9
Merger/Tape Server/Merger stats - attachment 10
analysis of file S496/R14_1920 - attachment 11
all FEE64 deadtime 0% as expected
17.30 aida01, 3, 5, 9 & 11 have hot HEC channels
ASIC check fixes
All histogram, statistics & merger statistics zero'd
DAQ continues
Note gamma-ray source tests have started with FATIMA and DEGAS
20.20 Analysis file R15_19 - attachment 12
[npg@aidas-gsi S496]$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_aidasgsi-lv_root
50G 18G 30G 37% /
tmpfs 7.9G 665M 7.2G 9% /dev/shm
/dev/sda2 477M 42M 410M 10% /boot
/dev/sda1 200M 264K 200M 1% /boot/efi
/dev/mapper/vg_aidasgsi-lv_home
401G 90G 292G 24% /home
/dev/mapper/vg_aidas2-lv_home
7.2T 6.3T 530G 93% /media/1e121361-83d3-4825-b6ae-8700b07e0ca7
/dev/sdd1 7.2T 5.6T 1.3T 82% /media/SecondDrive
/dev/sdc1 7.2T 6.6T 259G 97% /media/ThirdDrive
estimate 90/2 x 3168s = 40h to filesystem full - OK
20.22 All system wide checks OK *except*
Base Current Difference
aida01 fault 0xf82c : 0xf82f : 3
aida02 fault 0xdda6 : 0xdda9 : 3
aida03 fault 0xb13d : 0xb140 : 3
aida04 fault 0xf4cd : 0xf4d0 : 3
aida05 fault 0x500 : 0x575 : 117
aida06 fault 0xc6e7 : 0xc6e9 : 2
aida07 fault 0x4920 : 0x4922 : 2
aida08 fault 0xbd9a : 0xbd9c : 2
White Rabbit error counter test result: Passed 8, Failed 8
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 8 4 6 6 1 4 3 3 4 3 6 : 38048
aida02 : 11 8 10 3 4 4 3 4 3 3 6 : 37740
aida03 : 2 8 3 3 2 2 1 4 2 4 6 : 37720
aida04 : 26 15 17 4 4 2 1 4 2 4 6 : 38256
aida05 : 15 6 4 1 1 2 1 3 3 4 6 : 38156
aida06 : 7 7 16 6 3 4 2 2 3 4 6 : 38612
aida07 : 11 5 12 3 3 3 2 3 1 3 7 : 38836
aida08 : 13 6 7 2 1 2 2 2 3 4 6 : 37972
aida09 : 23 3 8 5 0 5 3 3 4 3 6 : 38164
aida10 : 12 5 7 1 2 2 4 3 2 4 6 : 37992
aida11 : 19 2 7 1 2 2 1 2 3 4 6 : 37740
aida12 : 21 5 3 3 3 3 1 2 3 4 6 : 37964
aida13 : 18 7 7 1 2 4 2 3 2 4 6 : 37776
aida14 : 8 3 13 7 2 4 2 3 2 4 6 : 37992
aida15 : 11 5 14 1 3 4 2 3 2 4 6 : 37908
aida16 : 17 6 6 2 2 3 1 2 3 4 6 : 37908
Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same
FEE : aida01 => Options file size is 1026 Last changed Mon May 17 11:43:04 CEST 2021
FEE : aida02 => Options file size is 1014 Last changed Thu Apr 29 14:43:46 CEST 2021
FEE : aida03 => Options file size is 1014 Last changed Thu Apr 29 14:43:50 CEST 2021
FEE : aida04 => Options file size is 1025 Last changed Fri May 14 16:54:56 CEST 2021
FEE : aida05 => Options file size is 1025 Last changed Mon May 17 06:25:41 CEST 2021
FEE : aida06 => Options file size is 1014 Last changed Thu Apr 29 14:43:59 CEST 2021
FEE : aida07 => Options file size is 1014 Last changed Thu Apr 29 14:44:02 CEST 2021
FEE : aida08 => Options file size is 1025 Last changed Wed May 05 12:15:54 CEST 2021
FEE : aida09 => Options file size is 1014 Last changed Thu Apr 29 14:44:08 CEST 2021
FEE : aida10 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida11 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida12 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida13 => Options file size is 1025 Last changed Fri May 07 19:40:34 CEST 2021
FEE : aida14 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida15 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida16 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
19.30 Lost activity monitors - attachments 13-15
20.54 1.8.W spectra 20us FSR attachments 16 & 23
1.8.W spectra 200us FSR attachments 17 & 22
1.8.W spectra 2ms FSR attachments 18 & 21
1.8.W spectra 20ms FSR attachments 19 & 20 |
|
343
|
Fri May 21 23:19:23 2021 |
TD | Saturday 22 May 00:00-08:00 |
00.16 413Gb remaining /TapeData filesystem
estimate 413/2 x 177s = 10.15h to filesystem full - OK
00.18 All histograms, statistics and merger statistics zero'd
00.57 All system wide checks OK *except*
Base Current Difference
aida05 fault 0x500 : 0x572 : 114
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 19 9 3 4 1 5 2 2 3 3 6 : 36356
aida02 : 4 8 7 2 4 3 3 3 2 3 6 : 35968
aida03 : 12 9 6 2 5 3 2 3 3 3 6 : 36824
aida04 : 11 5 9 2 2 2 2 4 3 3 6 : 37028
aida05 : 24 9 2 1 2 3 2 3 4 3 6 : 37608
aida06 : 25 6 3 4 2 4 3 3 3 3 6 : 37060
aida07 : 3 3 3 1 3 3 2 2 2 2 7 : 37044
aida08 : 20 4 2 3 1 3 2 2 3 3 6 : 36016
aida09 : 15 8 2 5 0 3 3 2 3 3 6 : 36284
aida10 : 16 6 4 1 2 3 3 2 2 4 6 : 37328
aida11 : 20 2 2 1 1 3 2 2 3 3 6 : 35936
aida12 : 1 2 1 2 3 4 1 2 4 3 6 : 36900
aida13 : 25 10 4 1 0 3 2 3 3 3 6 : 36500
aida14 : 20 6 9 2 2 5 1 2 2 4 6 : 37200
aida15 : 31 9 5 2 3 3 1 3 3 3 6 : 36500
aida16 : 24 4 2 3 1 4 2 3 3 3 6 : 36672
Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same
FEE : aida01 => Options file size is 1026 Last changed Mon May 17 11:43:04 CEST 2021
FEE : aida02 => Options file size is 1014 Last changed Thu Apr 29 14:43:46 CEST 2021
FEE : aida03 => Options file size is 1014 Last changed Thu Apr 29 14:43:50 CEST 2021
FEE : aida04 => Options file size is 1025 Last changed Fri May 14 16:54:56 CEST 2021
FEE : aida05 => Options file size is 1025 Last changed Mon May 17 06:25:41 CEST 2021
FEE : aida06 => Options file size is 1014 Last changed Thu Apr 29 14:43:59 CEST 2021
FEE : aida07 => Options file size is 1014 Last changed Thu Apr 29 14:44:02 CEST 2021
FEE : aida08 => Options file size is 1025 Last changed Wed May 05 12:15:54 CEST 2021
FEE : aida09 => Options file size is 1014 Last changed Thu Apr 29 14:44:08 CEST 2021
FEE : aida10 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida11 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida12 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida13 => Options file size is 1025 Last changed Fri May 07 19:40:34 CEST 2021
FEE : aida14 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida15 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida16 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
01.12 Lost activity monitor - attachment 1
1.8.H spectra - attachments 2 & 3
1.8.L spectra - attachments 4-6
aida01 pulser peak width 120 ch FWHM
Rate specttra - attachments 7 & 8
01.06 Lost activity monitor - attachment 9
ucesb - attachment 10
Grafana - DSSSD bias & leakage currents past 7 days - attachment 11
statistics - adc data, disc data, WR 28-47, pause, correlation scaler, good wave events - attachments 12-17
FEE64 temperatures OK - attachment 18
DSSSD bias & leakage currents OK - attachment 19
Merger/Tape Server/Merger stats - attachment 20
no merger warning/error messages since last restart
01.26 Analysis file S496/R14_1798 - attachment 21
aida08 5.5% deadtime
01.17 Disk space remaining /TapeData filesystem
[npg@aidas-gsi S496]$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_aidasgsi-lv_root
50G 18G 30G 37% /
tmpfs 7.9G 706M 7.2G 9% /dev/shm
/dev/sda2 477M 42M 410M 10% /boot
/dev/sda1 200M 264K 200M 1% /boot/efi
/dev/mapper/vg_aidasgsi-lv_home
401G 90G 292G 24% /home
/dev/mapper/vg_aidas2-lv_home
7.2T 6.4T 395G 95% /media/1e121361-83d3-4825-b6ae-8700b07e0ca7
/dev/sdd1 7.2T 5.6T 1.3T 82% /media/SecondDrive
/dev/sdc1 7.2T 6.6T 259G 97% /media/ThirdDrive
estimate 395/2 x 177s = 9.7h to filesystem full - OK
01.30 despec shift reports no data reported by ucesb
aida stats & tape server ok - no data being forwarded to MBS by data relay
01.39 despec shift restart FATIMA DAQ - data forwarding to MBS by data relay resumes - AIDA file S496/R14_1811
04.35 System wide checks all OK *except*
Base Current Difference
aida05 fault 0x500 : 0x573 : 115
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
[npg@aidas-gsi S496]$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_aidasgsi-lv_root
50G 18G 30G 37% /
tmpfs 7.9G 661M 7.2G 9% /dev/shm
/dev/sda2 477M 42M 410M 10% /boot
/dev/sda1 200M 264K 200M 1% /boot/efi
/dev/mapper/vg_aidasgsi-lv_home
401G 90G 292G 24% /home
/dev/mapper/vg_aidas2-lv_home
7.2T 6.5T 337G 96% /media/1e121361-83d3-4825-b6ae-8700b07e0ca7
/dev/sdd1 7.2T 5.6T 1.3T 82% /media/SecondDrive
/dev/sdc1 7.2T 6.6T 259G 97% /media/ThirdDrive
estimate 337/2 x 177s = 8.3h to filesystem full - OK
FEE64 temperatures OK - attachment 22
ADC data item statistics OK - attachment 24
DSSSD bias & leakage current OK - attachment 26
analysis of file S496/R14_1920 - attachment 28
aida04 deadtime 5.0%
07.10 Ssystem wide checks all OK *except*
Base Current Difference
aida05 fault 0x500 : 0x573 : 115
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
[npg@aidas-gsi S496]$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_aidasgsi-lv_root
50G 18G 30G 37% /
tmpfs 7.9G 662M 7.2G 9% /dev/shm
/dev/sda2 477M 42M 410M 10% /boot
/dev/sda1 200M 264K 200M 1% /boot/efi
/dev/mapper/vg_aidasgsi-lv_home
401G 90G 292G 24% /home
/dev/mapper/vg_aidas2-lv_home
7.2T 6.5T 287G 96% /media/1e121361-83d3-4825-b6ae-8700b07e0ca7
/dev/sdd1 7.2T 5.6T 1.3T 82% /media/SecondDrive
/dev/sdc1 7.2T 6.6T 259G 97% /media/ThirdDrive
estimate 287/2 x 178s = 7.1h to filesystem full - OK
FEE64 temperatures OK - attachment 23
ADC data item statistics OK - attachment 25
DSSSD bias & leakage current OK - attachment 27
analysis of file S496/R14_1920 - attachment 29
aida04 deadtime 4.4%
07.35 Found MBS data monitor kill'd
Restarted MBS data relay and data monitor OK
08.00 beam off - end of S496
08.28 Tape Server -> no storage mode
DAQ continues
09.58 All slow comparators -> 0x64
BNC PB-5 OFF
file S496/R15
alpha background
Tape server 200kb/s, merger 260k data items/s |
|
342
|
Fri May 21 15:03:48 2021 |
CA | Friday 21st May 16:00 - 00:00 shift | 16:00 CA takes over
DSSD1 n+n sides 0x20 on ASICs 1-3 and 0x64 on ASIC 4
DSSD2 n+n sides 0x1b on ASICs 1-3 and 0x64 on ASIC 4
PULSER SETTINGS
---------------------------
Pulse is ON
Positive Tail Pulse
Trigger Source is Internal Clock
Trigger Threshold is 3.5
Amplitude : 2.0 Volts
Rep Rate : 2.0 hZ
Delay : 250.0 ns
Fall Time : 1 ms
Attenuation : 1
Display is : Volts
Equivalent keV is : 200.0
Ramp Start at 0.01 Volts
Ramp Stop at 9.99 Volts
Ramp Start at 1.0 keV
Ramp Stop at 999.0 keV
Ramp Time is 60 seconds
# Ramp Cycles is 1
16:14 system wide checks all ok *except*
Base Current Difference
aida05 fault 0x500 : 0x570 : 112
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
16:17 temperatures ok - attachment 1
statistics ok - attachment 2
detector bias / leakage currents ok - attachment 3
analysis of R14_1625 - FEE7 deadtime at 4.5%
18:06 system wide checks all ok *except*
Base Current Difference
aida05 fault 0x500 : 0x571 : 113
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
18:07 temperatures ok - attachment 4
statistics ok - attachment 5
detector bias / leakage currents ok - attachment 6
analysis of R14_1662 - FEE7 deadtime at 4.3%
19:07 DAQ continues ok - writing to file R14_1682
20:07 system wide checks all ok *except*
Base Current Difference
aida05 fault 0x500 : 0x572 : 114
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
20:10 temperatures ok - attachment 7
statistics ok - attachment 8
detector bias / leakage currents ok - attachment 9
analysis of R14_1703 - max deadtime at 4.0% - attachment 10
20:35 rate spectra - attachment 11
HEC spectra - attachment 12 & 13
20:36 all histograms zeroed
20:37 DAQ continues ok - writing to file R14_1712
20:45 /media/1e121361-83d3-4825-b6ae-9700b07e0ca7 at 94% use.
~ 12 hours until 100% at 10 MB/s data rate, not accounting for compression of older files
21:12 no beam - writing to file R14_1723
21:17 beam is back
22:07 system wide checks all ok - WR decoder fault same as entry at 20:07
22:08 temperatures ok - attachment 14
statistics ok - attachment 15
detector bias / leakage currents ok - attachment 16
analysis of R14_1740 - max deadtime at 4.2% - attachment 17
|
|
341
|
Fri May 21 07:03:50 2021 |
OH, TD | Friday 21 May 08:00-16:00 | 08:00 OH and TD take over
Current settings p+n sides at 0xc
DSSD1 n+n sides 0x20 on ASICs 1-3 and 0x64 on ASIC 4
DSSD2 n+n sides 0x1b on ASICs 1-3 and 0x64 on ASIC 4
PULSER SETTINGS
---------------------------
Pulse is ON
Positive Tail Pulse
Trigger Source is Internal Clock
Trigger Threshold is 3.5
Amplitude : 2.0 Volts
Rep Rate : 2.0 hZ
Delay : 250.0 ns
Fall Time : 1 ms
Attenuation : 1
Display is : Volts
Equivalent keV is : 200.0
Ramp Start at 0.01 Volts
Ramp Stop at 9.99 Volts
Ramp Start at 1.0 keV
Ramp Stop at 999.0 keV
Ramp Time is 60 seconds
# Ramp Cycles is 1
08:29 System wide checks ok - WR difference at 111
Statistics ok (Still high following the sharp rise at 18:30 yesterday) - attachment 1
Temperature ok - attachment 2
Bias and leakage current ok - attachment 3
09:58 System wide checks ok - WR difference at 111
Statistics ok - attachment 4
Temperature ok - attachment 5
Bias and leakage current ok - attachment 6
Max deadtime this morning has been around 5.5% on FEE8
12:32 System wide checks ok - WR difference at 111
Statistics ok - attachment 7
Temperature ok - attachment 9
Bias and leakage current ok - attachment 8 |
|
340
|
Fri May 21 06:09:28 2021 |
ML | system wide checks | System wide checks at 7am CET
All seems fine, nothing significant to report
Slight increase of dead time noted in FEE 3, 5 and 7.
-----
Clock status test result: Passed 16, Failed 0
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable
-----
ADC calibration
FEE64 module aida01 failed
FEE64 module aida02 failed
FEE64 module aida03 failed
FEE64 module aida04 failed
FEE64 module aida05 failed
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida08 failed
FEE64 module aida09 failed
FEE64 module aida10 failed
FEE64 module aida11 failed
FEE64 module aida12 failed
FEE64 module aida13 failed
FEE64 module aida14 failed
FEE64 module aida15 failed
FEE64 module aida16 failed
Calibration test result: Passed 0, Failed 16
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
-----
White Rabbit:
Base Current Difference
aida05 fault 0x500 : 0x56e : 110
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
-----
FPGA Timestamp error counter test result: Passed 16, Failed 0
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
-----
Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 23 5 6 5 0 4 1 3 3 3 6 : 36484
aida02 : 10 3 3 2 3 2 2 2 3 3 6 : 35952
aida03 : 2 5 1 4 5 2 2 3 3 3 6 : 36608
aida04 : 1 3 4 0 4 3 2 3 3 3 6 : 36572
aida05 : 19 9 3 2 2 3 1 3 4 3 6 : 37380
aida06 : 13 11 1 5 3 4 4 2 3 3 6 : 36860
aida07 : 12 2 4 3 3 2 1 3 2 2 7 : 37280
aida08 : 25 5 7 1 1 3 3 3 3 3 6 : 36828
aida09 : 22 5 6 4 1 3 2 3 3 3 6 : 36640
aida10 : 23 8 6 3 2 3 2 2 2 4 6 : 37212
aida11 : 24 14 7 3 3 2 3 3 3 3 6 : 36960
aida12 : 16 10 5 3 1 2 1 2 4 3 6 : 36736
aida13 : 23 6 4 2 2 3 2 3 3 3 6 : 36620
aida14 : 22 7 8 4 3 4 1 2 2 4 6 : 37200
aida15 : 10 12 9 2 1 3 1 3 3 3 6 : 36376
aida16 : 18 5 6 2 0 3 2 4 3 3 6 : 37008
|
|
339
|
Fri May 21 02:17:47 2021 |
ML | system wide checks | Wide check at 3am -
All fine.
Same output as at the beginning of the shift (see my previous entry) and attachments.
|
|
338
|
Thu May 20 23:26:11 2021 |
Marc Labiche (ML) | system wide checks | System Wide check at begining of the night shift:
-----
Clock status test result: Passed 16, Failed 0
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable
-----
ADC calibration:
FEE64 module aida01 failed
FEE64 module aida02 failed
FEE64 module aida03 failed
FEE64 module aida04 failed
FEE64 module aida05 failed
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida08 failed
FEE64 module aida09 failed
FEE64 module aida10 failed
FEE64 module aida11 failed
FEE64 module aida12 failed
FEE64 module aida13 failed
FEE64 module aida14 failed
FEE64 module aida15 failed
FEE64 module aida16 failed
Calibration test result: Passed 0, Failed 16
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
-----
White rabbit status:
Base Current Difference
aida05 fault 0x500 : 0x56d : 109
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
-----
FPGA Timestamp error counter test result: Passed 16, Failed 0
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
-----
Memory info from FEE cards
Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 20 7 7 6 1 5 2 2 3 3 6 : 36472
aida02 : 19 5 4 4 4 3 3 3 2 3 6 : 36020
aida03 : 22 10 2 2 5 2 2 3 3 3 6 : 36680
aida04 : 12 7 1 0 4 3 1 4 3 3 6 : 36856
aida05 : 16 7 5 3 1 3 2 3 3 3 6 : 36584
aida06 : 20 7 5 6 2 4 4 2 3 3 6 : 36888
aida07 : 13 7 5 2 3 2 1 3 2 2 7 : 37308
aida08 : 18 7 5 3 0 2 3 3 3 3 6 : 36656
aida09 : 21 6 4 4 1 4 2 3 3 3 6 : 36740
aida10 : 21 10 2 3 2 3 2 2 2 4 6 : 37156
aida11 : 23 2 4 4 3 2 2 3 2 4 6 : 37612
aida12 : 24 1 4 3 2 2 1 2 4 3 6 : 36744
aida13 : 26 9 4 2 1 4 3 2 3 3 6 : 36464
aida14 : 19 9 9 2 3 4 1 2 2 4 6 : 37156
aida15 : 10 7 10 0 1 3 1 3 3 3 6 : 36288
aida16 : 28 7 3 2 1 3 3 3 3 3 6 : 36824 |
|
337
|
Thu May 20 14:56:33 2021 |
OH | Thursday 20th May 16:00 - 24:00 | 16:00 p+n threshold 0xc (120keV)
n+n threshold DSSD2 0x20 (250keV) ASICS 1-3, 0x64 ASIC4
n+n threshold DSSD2 0x19 (250keV) ASICS 1-3, 0x64 ASIC4
Dead time has increased from yesterday. From the elog this appears to coincide with their changing of the spill structure.
17:30 Grafana informs us we have hit 10uA on DSSD2. This is fine with the size of the detector
18:12 Gregor noticed that at 18:02 the rate in AIDA increased. This could be related to the autofill starting around now.
The deadtimes in FEE6 and 8 have risen to around 6.25%.
Looking at this mornings and yesterday evenings fill times this behavior isn't observed
18:30 The rates still seem to be increased so is unlikely to be due to the filling which finished at least 10 minutes ago
18:49 Have tried raising the thresholds to 0x1a for the y side strips of DSSD2
This has brought the deadtime of AIDA06 down to 5.53% (This seems to have just been a statistical fluctuation and the rate went back up again in the next file)
18:12 Raised threshold to 0x1b for ASIC1-3 of FEE6 and 8
18:15 System wide checks ok except WR as usual
Base Current Difference
aida05 fault 0x500 : 0x56d : 109
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Statistics increased at 18:02 still working on - attachment 2
FEE temp - attachment 3
Bias and leakage current - attachment 4
20:51 System wide checks ok - WR same as before
Statistics still high - attachment 5
Temperatures ok - attachment 6
Bias and leakage currents ok - attachment 7
22:52 Statistics still high - attachment 8
Temp ok - attachment 9
Bias and leakage current ok - attachment 10
System wide checks ok - WR difference still 109 |
|