ELOG DESPEC

AIDA GELINA BRIKEN nToF CRIB ISOLDE CIRCE nTOFCapture DESPEC DTAS EDI_PSA 179Ta CARME StellarModelling DCF K40

DESPEC

Fields marked with * are required
Entry time:	Sat Mar 29 11:27:22 2025
Author*:
Subject*:
> > Some of the FEE64s aida01 .. aida12 panic during boot > > Frequencies of panics for each FEE64 can be seen in attachment 1 > > Below is an example of an aida04 panic following a power cycle and an automatic reboot > > pi@raspberrypi:~/logs $ ./tail_aida aida04 > aida04 > ttyUSB7 > 19/11:07:04\|LR [203f5680] 0x203f5680 > 19/11:07:04\|Call Trace: > 19/11:07:04\|Kernel panic - not syncing: Fatal exception in interrupt > 19/11:07:04\|Call Trace: > 19/11:07:04\|[c6941de0] [c0005de8] show_stack+0x44/0x16c (unreliable) > 19/11:07:04\|[c6941e20] [c00345bc] panic+0x94/0x168 > 19/11:07:04\|[c6941e70] [c000bd44] die+0x178/0x18c > 19/11:07:04\|[c6941e90] [c0011a28] do_page_fault+0xc4/0x458 > 19/11:07:04\|[c6941f40] [c000e7c4] handle_page_fault+0xc/0x80 > 19/11:07:04\|Rebooting in 180 seconds.. > ISOL Version 1.00 Date 9th January 2017 > Flash base address=FC000000 > Set Flash to ASync Mode > XST_SUCCESS\| > Finished copying zImage to RAM > 19/11:10:05\| > Found 0 errors checking kernel image > 19/11:10:06\|VHDL version number 0X18430701 > Based on AIDA Bootloader version number 1.2.0 -- 16th August 2012 > Starting LMK 3200 setup > 19/11:10:06\| > Setting LMK03200 to standard clock settings -- External Clock 23Nov15 > .... SPI Base Address=0x81400000 > clk_control_reg=0x4 > 19/11:10:06\|Next step is SPIconfig > Control 32(0x81400000)=0x180 > SlaveSel(0x81400000)=0x3 > Ctrl(0x81400000)=0xE6 > Ctrl(0x81400000)=0x86 > 19/11:10:06\|SPIconfig done now to set up the LMK3200 registers > 19/11:10:06\|LMK #0 : regInit[0]=0x80000000 > 19/11:10:07\|LMK #0 : regInit[1]=0x10070600 > 19/11:10:07\|LMK #0 : regInit[2]=0x60601 > 19/11:10:07\|LMK #0 : regInit[3]=0x60602 > 19/11:10:07\|LMK #0 : regInit[4]=0x60603 > 19/11:10:07\|LMK #0 : regInit[5]=0x70624 > 19/11:10:07\|LMK #0 : regInit[6]=0x70605 > 19/11:10:07\|LMK #0 : regInit[7]=0x70606 > 19/11:10:07\|LMK #0 : regInit[8]=0x70627 > 19/11:10:07\|LMK #0 : regInit[9]=0x10000908 > 19/11:10:07\|LMK #0 : regInit[10]=0xA0022A09 > 19/11:10:07\|LMK #0 : regInit[11]=0x82800B > 19/11:10:07\|LMK #0 : regInit[12]=0x28C800D > 19/11:10:07\|LMK #0 : regInit[13]=0x830020E > 19/11:10:07\|LMK #0 : regInit[14]=0xC800180F > Calibrate completed at 941 counts > Setting Clock Control =0x0000000B, to set GOE and sync bit > Ctrl @ SPIstop (0x81400000)=0x186 > Timeout waiting for Lock detect Stage 2 (Zero Delay), PWR_DWN=0x00000004 > 19/11:10:07\| > Finished Clock setup LMK03200 > completed LMK 3200 setup > Loaded all four ASICs with default settings > Setting the ADCs into calibration mode > 19/11:10:07\| > Control 32(0x81400400)=0x180 > SlaveSel(0x81400400)=0xFF > Ctrl(0x81400400)=0xE6 > Ctrl(0x81400400)=0x86 > Init : Config of AD9252 SPI ok > 19/11:10:08\| > Ctrl @ SPIstop (0x81400400)=0x186ADCs initialised > Cal DCMs not locked > ADC calibrate failed > Jumping to kernel simpleboot... > 19/11:10:08\| > zImage starting: loaded at 0x00a00000 (sp: 0x00bc4eb0) > Allocating 0x3b78cc bytes for kernel ... > gunzipping (0x00000000 <- 0x00a0f000:0x00bc380e)...done 0x39604c bytes > 19/11:10:11\| > Linux/PowerPC load: console=ttyS0 root=/dev/nfs ip=on rw mem=112M > Finalizing device tree... flat tree at 0xbd1300 > Probing IIC bus for MAC... MAC address = 0xd8 0x80 0x39 0x41 0xf6 0xb7 > 19/11:10:17\|Using Xilinx Virtex440 machine description > 19/11:10:18\|Linux version 2.6.31 (nf@nnlxb.dl.ac.uk) (gcc version 4.2.2) #34 PREEMPT Tue Nov 15 15:57:04 GMT 2011 > 19/11:10:18\|Zone PFN ranges: > 19/11:10:18\| DMA 0x00000000 -> 0x00007000 > 19/11:10:18\| Normal 0x00007000 -> 0x00007000 > 19/11:10:18\|Movable zone start PFN for each node > 19/11:10:18\|early_node_map[1] active PFN ranges > 19/11:10:18\| 0: 0x00000000 -> 0x00007000 > 19/11:10:18\|MMU: Allocated 1088 bytes of context maps for 255 contexts > 19/11:10:18\|Built 1 zonelists in Zone order, mobility grouping on. Total pages: 28448 > 19/11:10:18\|Kernel command line: console=ttyS0 root=/dev/nfs ip=on rw mem=112M > 19/11:10:18\|PID hash table entries: 512 (order: 9, 2048 bytes) > 19/11:10:19\|Dentry cache hash table entries: 16384 (order: 4, 65536 bytes) > 19/11:10:19\|Inode-cache hash table entries: 8192 (order: 3, 32768 bytes) > 19/11:10:19\|Memory: 109680k/114688k available (3500k kernel code, 4852k reserved, 144k data, 130k bss, 168k init) > 19/11:10:19\|Kernel virtual memory layout: > 19/11:10:19\| * 0xffffe000..0xfffff000 : fixmap > 19/11:10:19\| * 0xfde00000..0xfe000000 : consistent mem > 19/11:10:19\| * 0xfde00000..0xfde00000 : early ioremap > 19/11:10:19\| * 0xd1000000..0xfde00000 : vmalloc & ioremap > 19/11:10:19\|NR_IRQS:512 > 19/11:10:19\|clocksource: timebase mult[a00000] shift[22] registered > 19/11:10:19\|Console: colour dummy device 80x25 > 19/11:10:19\|Mount-cache hash table entries: 512 > 19/11:10:19\|NET: Registered protocol family 16 > 19/11:10:19\|PCI: Probing PCI hardware > 19/11:10:19\|bio: create slab <bio-0> at 0 > 19/11:10:19\|NET: Registered protocol family 2 > 19/11:10:19\|IP route cache hash table entries: 1024 (order: 0, 4096 bytes) > 19/11:10:19\|TCP established hash table entries: 4096 (order: 3, 32768 bytes) > 19/11:10:19\|TCP bind hash table entries: 4096 (order: 2, 16384 bytes) > 19/11:10:19\|TCP: Hash tables configured (established 4096 bind 4096) > 19/11:10:20\|TCP reno registered > 19/11:10:20\|NET: Registered protocol family 1 > 19/11:10:20\|ROMFS MTD (C) 2007 Red Hat, Inc. > 19/11:10:20\|msgmni has been set to 214 > 19/11:10:20\|io scheduler noop registered > 19/11:10:20\|io scheduler anticipatory registered > 19/11:10:20\|io scheduler deadline registered > 19/11:10:20\|io scheduler cfq registered (default) > 19/11:10:20\|Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled > 19/11:10:20\|83e00000.serial: ttyS0 at MMIO 0x83e01003 (irq = 16) is a 16550 > 19/11:10:20\|console [ttyS0] enabled > 19/11:10:20\|brd: module loaded > 19/11:10:20\|loop: module loaded > 19/11:10:20\|Device Tree Probing 'ethernet' > 19/11:10:20\|xilinx_lltemac 81c00000.ethernet: MAC address is now d8:80:39:41:f6:b7 > 19/11:10:20\|xilinx_lltemac 81c00000.ethernet: XLlTemac: using DMA mode. > 19/11:10:20\|XLlTemac: DCR address: 0x80 > 19/11:10:20\|XLlTemac: buffer descriptor size: 32768 (0x8000) > 19/11:10:20\|XLlTemac: Allocating DMA descriptors with kmalloc > 19/11:10:20\|XLlTemac: (buffer_descriptor_init) phy: 0x6938000, virt: 0xc6938000, size: 0x8000 > 19/11:10:20\|XTemac: PHY detected at address 7. > 19/11:10:20\|xilinx_lltemac 81c00000.ethernet: eth0: Xilinx TEMAC at 0x81C00000 mapped to 0xD1024000, irq=17 > 19/11:10:21\|fc000000.flash: Found 1 x16 devices at 0x0 in 16-bit bank > 19/11:10:21\| Intel/Sharp Extended Query Table at 0x010A > 19/11:10:21\| Intel/Sharp Extended Query Table at 0x010A > 19/11:10:21\| Intel/Sharp Extended Query Table at 0x010A > 19/11:10:21\| Intel/Sharp Extended Query Table at 0x010A > 19/11:10:21\| Intel/Sharp Extended Query Table at 0x010A > 19/11:10:21\| Intel/Sharp Extended Query Table at 0x010A > 19/11:10:21\|Using buffer write method > 19/11:10:21\|cfi_cmdset_0001: Erase suspend on write enabled > 19/11:10:21\|cmdlinepart partition parsing not available > 19/11:10:21\|RedBoot partition parsing not available > 19/11:10:21\|Creating 5 MTD partitions on "fc000000.flash": > 19/11:10:21\|0x000000000000-0x000000500000 : "golden_firmware" > 19/11:10:21\|0x000000500000-0x000000800000 : "golden_kernel" > 19/11:10:21\|0x000000800000-0x000000d00000 : "user_firmware" > 19/11:10:21\|0x000000d00000-0x000000fe0000 : "user_kernel" > 19/11:10:21\|0x000000fe0000-0x000001000000 : "env_variables" > 19/11:10:21\|xilinx-xps-spi 81400400.hd-xps-spi: at 0x81400400 mapped to 0xD1028400, irq=20 > 19/11:10:21\|SPI: XIlinx spi: bus number now 32766 > 19/11:10:21\|xilinx-xps-spi 81400000.xps-spi: at 0x81400000 mapped to 0xD102C000, irq=21 > 19/11:10:22\|SPI: XIlinx spi: bus number now 32765 > 19/11:10:22\|mice: PS/2 mouse device common for all mice > 19/11:10:22\|Device Tree Probing 'i2c' > 19/11:10:22\| #0 at 0x81600000 mapped to 0xD1030000, irq=22 > 19/11:10:22\|at24 0-0050: 1024 byte 24c08 EEPROM (writable) > 19/11:10:22\|TCP cubic registered > 19/11:10:22\|NET: Registered protocol family 17 > 19/11:10:22\|RPC: Registered udp transport module. > 19/11:10:22\|RPC: Registered tcp transport module. > 19/11:10:22\|eth0: XLlTemac: Options: 0x3fa > 19/11:10:22\|eth0: XLlTemac: allocating interrupt 19 for dma mode tx. > 19/11:10:23\|eth0: XLlTemac: allocating interrupt 18 for dma mode rx. > 19/11:10:23\|eth0: XLlTemac: speed set to 1000Mb/s > 19/11:10:25\|eth0: XLlTemac: Send Threshold = 24, Receive Threshold = 4 > 19/11:10:25\|eth0: XLlTemac: Send Wait bound = 254, Receive Wait bound = 254 > 19/11:10:25\|Sending DHCP requests ., OK > 19/11:10:26\|IP-Config: Got DHCP answer from 192.168.11.99, my address is 192.168.11.4 > 19/11:10:26\|IP-Config: Complete: > 19/11:10:26\| device=eth0, addr=192.168.11.4, mask=255.255.255.0, gw=255.255.255.255, > 19/11:10:26\| host=aida04, domain=dl.ac.uk, nis-domain=nuclear.physics, > 19/11:10:26\| bootserver=192.168.11.99, rootserver=192.168.11.99, rootpath=/home/Embedded/XilinxLinux/ppc_4xx/rfs/aida04 > 19/11:10:26\|Looking up port of RPC 100003/2 on 192.168.11.99 > 19/11:10:26\|Looking up port of RPC 100005/1 on 192.168.11.99 > 19/11:10:26\|VFS: Mounted root (nfs filesystem) on device 0:12. > 19/11:10:26\|Freeing unused kernel memory: 168k init > INIT: version 2.86 booting > 19/11:10:27\|Starting sysinit... > 19/11:10:27\| Welcome to DENX & STFC Daresbury Embedded Linux Environment > 19/11:10:27\| Press 'I' to enter interactive startup. > 19/11:10:27\|Setting clock (utc): Thu Dec 19 10:10:28 GMT 2019 [ OK ] > 19/11:10:28\|Building the cache [ OK ] > 19/11:10:28\|Setting hostname aida04: [ OK ] > 19/11:10:29\|Mounting local filesystems: [ OK ] > 19/11:10:30\|Enabling /etc/fstab swaps: [ OK ] > 19/11:10:32\|Finishing sysinit... > INIT: Entering runlevel: 3 > 19/11:10:35\|Entering non-interactive startup > 19/11:10:36\|FATAL: Module ipv6 not found. > 19/11:10:37\|Bringing up loopback interface: [ OK ] > 19/11:10:39\|FATAL: Module ipv6 not found. > 19/11:10:39\|Starting system logger: [ OK ] > 19/11:10:40\|Starting kernel logger: [ OK ] > 19/11:10:40\|Starting rpcbind: [ OK ] > 19/11:10:41\|Mounting NFS filesystems: [ OK ] > 19/11:10:42\|Mounting other filesystems: [ OK ] > 19/11:10:42\|Starting xinetd: [ OK ] > 19/11:10:43\|Starting midas: Starting MIDAS Data Acquisition for aida04 > 19/11:10:43\|xaida: device parameters: base=0x81000000 size=0x200000 > 19/11:10:48\|Trying to free nonexistent resource <0000000081000000-00000000811fffff> > 19/11:10:49\|xaida: mem region start 0x81000000 for 0x200000 mapped at 0xd2100000 > 19/11:10:49\|xaida: driver assigned major number 254 > 19/11:10:49\|Trying to free nonexistent resource <0000000007000000-0000000007ffffff> > 19/11:10:54\|AIDAMEM: aidamem: mem region start 0x7000000 for 0x1000000 mapped at 0xd2380000 > 19/11:10:54\|AIDAMEM: aidamem: driver assigned major number 253 > 19/11:10:54\|System identified is CPU ppc; Platform is unix; OS is Linux and Version is 2.6.31 > 19/11:11:01\|Environment selected is CPU ppc; Platform unix; OS Linux and Operating System linux-ppc_4xx > 19/11:11:01\|MIDASBASE = /MIDAS and MIDAS_LIBRARY = /MIDAS/TclHttpd/linux-ppc_4xx > 19/11:11:01\|PATH = /MIDAS/bin_linux-ppc_4xx:/MIDAS/TclHttpd/linux-ppc_4xx:/MIDAS/linux-ppc_4xx/bin:/MIDAS/linux-ppc_4xx/bin:/sbin:/usr/sbin:/bin:/usr/bin > 19/11:11:01\|Computer Name = aida04; Temp Directory = /tmp/tcl361 > 19/11:11:05\|package limit is not available: can't find package limit > 19/11:11:07\|Running with default file descriptor limit > 19/11:11:07\|package setuid is not available: can't find package setuid > 19/11:11:09\|Running as user 0 group 0 > 19/11:11:09\| > 19/11:11:10\|AIDA Data Acquisition Program Release 9_10.Apr 3 2019_11:34:31 starting > 19/11:11:10\| > 19/11:11:10\|Built without pthreaxaida: open: > 19/11:11:10\|ds > 19/11:11:10\| > 19/11:11:10\|Creating NetAIDAMEM: aidamem_open: > 19/11:11:10\|Vars > 19/11:11:10\|Output buffer length = 65504; format option = 4; transfer option = 3 > 19/11:11:11\|EB transfer option = 3 > 19/11:11:11\|NetVars created and initialised > 19/11:11:11\|Statistics thread starting > 19/11:11:11\|Statistics thread created > 19/11:11:11\|Stat/Rate creation thread starting > 19/11:11:11\|Data Acquisition task has PID 375 > 19/11:11:11\|Stat/Rate creation thread created > 19/11:11:11\|Hit/Rate creation thread starting > 19/11:11:11\|Hit/Rate creation thread created > 19/11:11:11\|AIDA Heartbeat thread starting > 19/11:11:11\|Heartbeat thread created > 19/11:11:11\|Installing signal handlers > 19/11:11:11\|Done > 19/11:11:11\|ModuleNum = 0 > 19/11:11:11\|Aida Initialise complete. AidaExecV9_10: Build Apr 3 2019_11:34:31. HDL version : 18430701 > 19/11:11:11\|Spectra table initialised > 19/11:11:11\|AIDA Data Acquisition now all ready to start > 19/11:11:11\|SIGBUS, SIGSEGV and SIGPIPE traps setup > 19/11:11:11\|/debug user "debug" password "-f9x7ruru8cg" > 19/11:11:17\|httpd started on port 8015 > 19/11:11:18\| > 19/11:11:18\|Cannot use /MIDAS/config/TclHttpd/aida04@8015/startup.tcl > 19/11:11:18\|Custom startup from /MIDAS/config/TclHttpd/aida04/startup.tcl > 19/11:11:18\|XAIDA Access package 1.0 > 19/11:11:19\|/XAIDAAccessServer > 19/11:11:19\|XAD9252 Access package 1.0 > 19/11:11:20\|/XAD9252AccessServer > 19/11:11:20\|/DataBaseAccessServer > 19/11:11:20\|/NetVarService > 19/11:11:20\|/SigTaskService > 19/11:11:20\|Loaded MemSasAccess > 19/11:11:20\|/SpectrumService > 19/11:11:20\|loading tcl/AIDARunControl.tcl for namespace :: > 19/11:11:20\|/DataAcquisitionControlServer > 19/11:11:20\|DefineMessage unknown > 19/11:11:20\|Run Control Server Implementation for AIDA > 19/11:11:21\|RunControlServer loaded > 19/11:11:21\|loading Html/RunControl/implementation.tcl > 19/11:11:21\|[ OK ] > 19/11:11:21\|/MIDAS/TclHttpd/Html/RunControl/common.tcl returned z=1 and couldn't read file "/MIDAS/TclHttpd/Html/RunControl/common.tcl": no such file or directory > 19/11:11:21\|ReadRegister failed: Name=NetVar.EXEC.ID; Code= 0x10004; Info= Register name does not exist > 19/11:11:21\| > 19/11:11:21\|DENX ELDK version 4.2 build 2008-04-01 > 19/11:11:21\|Linux 2.6.31 on a ppc > 19/11:11:21\| > 19/11:11:21\|aida04 login: Created UI registers > 19/11:11:22\|RunControl loaded > 19/11:11:22\|loading Html/AIDA/RunControl/implementation.tcl for namespace :: > 19/11:11:22\|AIDA RunControl loaded > 19/11:11:24\|Completed custom startup from /MIDAS/TclHttpd/Html/AIDA/RunControl/stats.defn.tcl Encoding: HTML ELCode plain Suppress Email notification

Attachment 1:
Drop attachments here...

ELOG V3.1.4-unknown