| ID |
Date |
Author |
Subject |
|
274
|
Sat Apr 24 04:50:01 2021 |
ML | Spectra |
Some screenshot of spectra attached |
| Attachment 1: 24-04-2021_540am_SpecLayoutId3.png
|
|
| Attachment 2: 24-04-2021_540am_SpecLayId4.png
|
|
| Attachment 3: 24-04-2021_540am_SpecLayId5.png
|
|
| Attachment 4: 24-04-2021_540am_SpecLayId6.png
|
|
| Attachment 5: 24-04-2021_540am_SpecLayId7.png
|
|
| Attachment 6: 24-04-2021_540am_SpecLayId8.png
|
|
| Attachment 7: 24-04-2021_540am_SpecLayId1.png
|
|
| Attachment 8: 24-04-2021_540am_SpecLayId2.png
|
|
|
273
|
Sat Apr 24 04:07:23 2021 |
ML | ucesb scalers |
ucesb is back - see attachment |
| Attachment 1: 24-04-2021_5am_ucesb.png
|
|
|
272
|
Sat Apr 24 03:32:11 2021 |
ML | quick checks |
Quick system checks after crash
Temperature ok.
Still cannot connect remotely to /ucesb with my gsi web login and password (?).
Clock status test result: Passed 12, Failed 0
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable
Calibration test result: Passed 12, Failed 0
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
Base Current Difference
aida05 fault 0x31a2 : 0x31a2 : 0
aida05 : WR status 0x10
White Rabbit error counter test result: Passed 11, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Returned 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 9 3 4 2 3 2 1 4 2 3 7 : 39804
aida02 : 29 4 2 3 0 3 1 4 2 3 7 : 39828
aida03 : 20 6 1 2 2 3 1 4 2 3 7 : 39888
aida04 : 19 5 4 2 1 3 1 4 2 3 7 : 39860
aida05 : 27 5 4 1 2 3 2 3 1 4 7 : 40692
aida06 : 17 10 3 2 1 3 1 4 2 3 7 : 39876
aida07 : 22 7 2 2 2 3 1 4 2 3 7 : 39920
aida08 : 12 6 5 1 3 2 1 4 2 3 7 : 39824
aida09 : 0 9 1 3 1 3 1 4 2 3 7 : 39800
aida10 : 2 2 2 3 3 2 3 2 2 3 7 : 39256
aida11 : 21 7 4 3 0 3 2 3 2 3 7 : 39596
aida12 : 18 8 3 0 2 3 2 3 2 3 7 : 39608
|
| Attachment 1: 24-04-2021_430am_GoodEvent.png
|
|
|
271
|
Sat Apr 24 03:16:19 2021 |
TD | aida-gsi window manager/login session crashed |
Looks like the aida-gsi window manager/login session crashed.
On site S460 shift restarted AnyDesk. Upon re-connection *all* browsers, terminals, applications were gone
I have re-started the basics but BNC PB-5 pulser, CAEN NIM bin, MBS rate monitor, system console monitor etc
still need to be re-started.
Extract from /var/log/messages
Apr 24 03:19:01 aidas-gsi kernel: gnome-terminal invoked oom-killer: gfp_mask=0xd0, order=0, oom_adj=0, oom_score_adj=0
Apr 24 03:19:01 aidas-gsi kernel: gnome-terminal cpuset=/ mems_allowed=0
Apr 24 03:19:01 aidas-gsi kernel: Pid: 18955, comm: gnome-terminal Not tainted 2.6.32-696.el6.x86_64 #1
Apr 24 03:19:01 aidas-gsi kernel: Call Trace:
Apr 24 03:19:01 aidas-gsi kernel: [<ffffffff81131550>] ? dump_header+0x90/0x1b0
Apr 24 03:19:01 aidas-gsi kernel: [<ffffffff8123c1dc>] ? security_real_capable_noaudit+0x3c/0x70
Apr 24 03:19:01 aidas-gsi kernel: [<ffffffff811319d2>] ? oom_kill_process+0x82/0x2a0
Apr 24 03:19:01 aidas-gsi kernel: [<ffffffff81131911>] ? select_bad_process+0xe1/0x120
Apr 24 03:19:01 aidas-gsi kernel: [<ffffffff81131e10>] ? out_of_memory+0x220/0x3c0
Apr 24 03:19:01 aidas-gsi kernel: [<ffffffff8113e7ec>] ? __alloc_pages_nodemask+0x93c/0x950
Apr 24 03:19:01 aidas-gsi kernel: [<ffffffff81015533>] ? native_sched_clock+0x13/0x80
Apr 24 03:19:01 aidas-gsi kernel: [<ffffffff810aaea1>] ? lock_hrtimer_base+0x31/0x60
Apr 24 03:19:01 aidas-gsi kernel: [<ffffffff81177a3a>] ? alloc_pages_current+0xaa/0x110
Apr 24 03:19:01 aidas-gsi kernel: [<ffffffff8113bb4e>] ? __get_free_pages+0xe/0x50
Apr 24 03:19:01 aidas-gsi kernel: [<ffffffff811b1944>] ? __pollwait+0xb4/0xf0
Apr 24 03:19:01 aidas-gsi kernel: [<ffffffff81348684>] ? n_tty_poll+0x194/0x1d0
Apr 24 03:19:01 aidas-gsi kernel: [<ffffffff813440aa>] ? tty_poll+0x8a/0xa0
Apr 24 03:19:01 aidas-gsi kernel: [<ffffffff811b1db5>] ? do_sys_poll+0x2d5/0x5b0
Apr 24 03:19:01 aidas-gsi kernel: [<ffffffff811b1890>] ? __pollwait+0x0/0xf0
Apr 24 03:19:01 aidas-gsi kernel: [<ffffffff811b1980>] ? pollwake+0x0/0x60
Apr 24 03:19:01 aidas-gsi kernel: [<ffffffff811b1980>] ? pollwake+0x0/0x60
Apr 24 03:19:01 aidas-gsi kernel: [<ffffffff811b1980>] ? pollwake+0x0/0x60
Apr 24 03:19:01 aidas-gsi kernel: [<ffffffff811b1980>] ? pollwake+0x0/0x60
Apr 24 03:19:01 aidas-gsi kernel: [<ffffffff811b1980>] ? pollwake+0x0/0x60
Apr 24 03:19:01 aidas-gsi kernel: [<ffffffff811b1980>] ? pollwake+0x0/0x60
Apr 24 03:19:01 aidas-gsi kernel: [<ffffffff811b1980>] ? pollwake+0x0/0x60
Apr 24 03:19:01 aidas-gsi kernel: [<ffffffff811b1980>] ? pollwake+0x0/0x60
Apr 24 03:19:01 aidas-gsi kernel: [<ffffffff811b1980>] ? pollwake+0x0/0x60
Apr 24 03:19:01 aidas-gsi kernel: [<ffffffff811b1745>] ? poll_select_set_timeout+0x95/0xb0
Apr 24 03:19:01 aidas-gsi kernel: [<ffffffff811b2281>] ? sys_poll+0x71/0x100
Apr 24 03:19:01 aidas-gsi kernel: [<ffffffff8100b0d2>] ? system_call_fastpath+0x16/0x1b
Apr 24 03:19:01 aidas-gsi kernel: Mem-Info:
Apr 24 03:19:01 aidas-gsi kernel: Node 0 DMA per-cpu:
Apr 24 03:19:01 aidas-gsi kernel: CPU 0: hi: 0, btch: 1 usd: 0
Apr 24 03:19:01 aidas-gsi kernel: CPU 1: hi: 0, btch: 1 usd: 0
Apr 24 03:19:01 aidas-gsi kernel: CPU 2: hi: 0, btch: 1 usd: 0
Apr 24 03:19:01 aidas-gsi kernel: CPU 3: hi: 0, btch: 1 usd: 0
Apr 24 03:19:01 aidas-gsi kernel: CPU 4: hi: 0, btch: 1 usd: 0
Apr 24 03:19:01 aidas-gsi kernel: CPU 5: hi: 0, btch: 1 usd: 0
Apr 24 03:19:01 aidas-gsi kernel: CPU 6: hi: 0, btch: 1 usd: 0
Apr 24 03:19:01 aidas-gsi kernel: CPU 7: hi: 0, btch: 1 usd: 0
Apr 24 03:19:01 aidas-gsi kernel: CPU 8: hi: 0, btch: 1 usd: 0
Apr 24 03:19:01 aidas-gsi kernel: CPU 9: hi: 0, btch: 1 usd: 0
Apr 24 03:19:01 aidas-gsi kernel: CPU 10: hi: 0, btch: 1 usd: 0
Apr 24 03:19:01 aidas-gsi kernel: CPU 11: hi: 0, btch: 1 usd: 0
Apr 24 03:19:01 aidas-gsi kernel: CPU 12: hi: 0, btch: 1 usd: 0
Apr 24 03:19:01 aidas-gsi kernel: CPU 13: hi: 0, btch: 1 usd: 0
Apr 24 03:19:01 aidas-gsi kernel: CPU 14: hi: 0, btch: 1 usd: 0
Apr 24 03:19:01 aidas-gsi kernel: CPU 15: hi: 0, btch: 1 usd: 0
Apr 24 03:19:01 aidas-gsi kernel: Node 0 DMA32 per-cpu:
Apr 24 03:19:01 aidas-gsi kernel: CPU 0: hi: 186, btch: 31 usd: 172
Apr 24 03:19:01 aidas-gsi kernel: CPU 1: hi: 186, btch: 31 usd: 45
Apr 24 03:19:01 aidas-gsi kernel: CPU 2: hi: 186, btch: 31 usd: 97
Apr 24 03:19:01 aidas-gsi kernel: CPU 3: hi: 186, btch: 31 usd: 142
Apr 24 03:19:01 aidas-gsi kernel: CPU 4: hi: 186, btch: 31 usd: 49
Apr 24 03:19:01 aidas-gsi kernel: CPU 5: hi: 186, btch: 31 usd: 56
Apr 24 03:19:01 aidas-gsi kernel: CPU 6: hi: 186, btch: 31 usd: 179
Apr 24 03:19:01 aidas-gsi kernel: CPU 7: hi: 186, btch: 31 usd: 171
Apr 24 03:19:01 aidas-gsi kernel: CPU 8: hi: 186, btch: 31 usd: 167
Apr 24 03:19:01 aidas-gsi kernel: CPU 9: hi: 186, btch: 31 usd: 67
Apr 24 03:19:01 aidas-gsi kernel: CPU 10: hi: 186, btch: 31 usd: 107
Apr 24 03:19:01 aidas-gsi kernel: CPU 11: hi: 186, btch: 31 usd: 170
Apr 24 03:19:01 aidas-gsi kernel: CPU 12: hi: 186, btch: 31 usd: 54
Apr 24 03:19:01 aidas-gsi kernel: CPU 13: hi: 186, btch: 31 usd: 118
Apr 24 03:19:01 aidas-gsi kernel: CPU 14: hi: 186, btch: 31 usd: 171
Apr 24 03:19:01 aidas-gsi kernel: CPU 15: hi: 186, btch: 31 usd: 196
Apr 24 03:19:01 aidas-gsi kernel: Node 0 Normal per-cpu:
Apr 24 03:19:01 aidas-gsi kernel: CPU 0: hi: 186, btch: 31 usd: 165
Apr 24 03:19:01 aidas-gsi kernel: CPU 1: hi: 186, btch: 31 usd: 36
Apr 24 03:19:01 aidas-gsi kernel: CPU 2: hi: 186, btch: 31 usd: 165
Apr 24 03:19:01 aidas-gsi kernel: CPU 3: hi: 186, btch: 31 usd: 167
Apr 24 03:19:01 aidas-gsi kernel: CPU 4: hi: 186, btch: 31 usd: 37
Apr 24 03:19:01 aidas-gsi kernel: CPU 5: hi: 186, btch: 31 usd: 128
Apr 24 03:19:01 aidas-gsi kernel: CPU 6: hi: 186, btch: 31 usd: 173
Apr 24 03:19:01 aidas-gsi kernel: CPU 7: hi: 186, btch: 31 usd: 181
Apr 24 03:19:01 aidas-gsi kernel: CPU 8: hi: 186, btch: 31 usd: 181
Apr 24 03:19:01 aidas-gsi kernel: CPU 9: hi: 186, btch: 31 usd: 161
Apr 24 03:19:01 aidas-gsi kernel: CPU 10: hi: 186, btch: 31 usd: 177
Apr 24 03:19:01 aidas-gsi kernel: CPU 11: hi: 186, btch: 31 usd: 178
Apr 24 03:19:01 aidas-gsi kernel: CPU 12: hi: 186, btch: 31 usd: 99
Apr 24 03:19:01 aidas-gsi kernel: CPU 13: hi: 186, btch: 31 usd: 183
Apr 24 03:19:01 aidas-gsi kernel: CPU 14: hi: 186, btch: 31 usd: 165
Apr 24 03:19:01 aidas-gsi kernel: CPU 15: hi: 186, btch: 31 usd: 162
Apr 24 03:19:01 aidas-gsi kernel: active_anon:3408671 inactive_anon:401328 isolated_anon:128
Apr 24 03:19:01 aidas-gsi kernel: active_file:22995 inactive_file:61739 isolated_file:0
Apr 24 03:19:01 aidas-gsi kernel: unevictable:0 dirty:59856 writeback:22 unstable:0
Apr 24 03:19:01 aidas-gsi kernel: free:35843 slab_reclaimable:15134 slab_unreclaimable:65491
Apr 24 03:19:01 aidas-gsi kernel: mapped:129529 shmem:249820 pagetables:24586 bounce:0
Apr 24 03:19:01 aidas-gsi kernel: Node 0 DMA free:15752kB min:60kB low:72kB high:88kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB
unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15368kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB
slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Apr 24 03:19:01 aidas-gsi kernel: lowmem_reserve[]: 0 874 16024 16024
Apr 24 03:19:01 aidas-gsi kernel: Node 0 DMA32 free:64204kB min:3680kB low:4600kB high:5520kB active_anon:369500kB inactive_anon:377436kB active_file:10988kB
inactive_file:26828kB unevictable:0kB isolated(anon):384kB isolated(file):0kB present:895084kB mlocked:0kB dirty:26856kB writeback:88kB mapped:35896kB shmem:39156kB
slab_reclaimable:4224kB slab_unreclaimable:4344kB kernel_stack:3344kB pagetables:2032kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:65792 all_unreclaimable?
no
Apr 24 03:19:01 aidas-gsi kernel: lowmem_reserve[]: 0 0 15150 15150
Apr 24 03:19:01 aidas-gsi kernel: Node 0 Normal free:63416kB min:63836kB low:79792kB high:95752kB active_anon:13265184kB inactive_anon:1227876kB active_file:80992kB
inactive_file:220128kB unevictable:0kB isolated(anon):128kB isolated(file):0kB present:15513600kB mlocked:0kB dirty:212568kB writeback:0kB mapped:482220kB
shmem:960124kB slab_reclaimable:56312kB slab_unreclaimable:257620kB kernel_stack:17600kB pagetables:96312kB unstable:0kB bounce:0kB writeback_tmp:0kB
pages_scanned:17824 all_unreclaimable? no
Apr 24 03:19:01 aidas-gsi kernel: lowmem_reserve[]: 0 0 0 0
Apr 24 03:19:01 aidas-gsi kernel: Node 0 DMA: 0*4kB 1*8kB 2*16kB 1*32kB 1*64kB 0*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15752kB
Apr 24 03:19:01 aidas-gsi kernel: Node 0 DMA32: 373*4kB 559*8kB 428*16kB 296*32kB 189*64kB 111*128kB 43*256kB 5*512kB 2*1024kB 0*2048kB 0*4096kB = 64204kB
Apr 24 03:19:01 aidas-gsi kernel: Node 0 Normal: 16100*4kB 1*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 64408kB
Apr 24 03:19:01 aidas-gsi kernel: 357788 total pagecache pages
Apr 24 03:19:01 aidas-gsi kernel: 22930 pages in swap cache
Apr 24 03:19:01 aidas-gsi kernel: Swap cache stats: add 3720478, delete 3697548, find 5026852/5228898
Apr 24 03:19:01 aidas-gsi kernel: Free swap = 1826100kB
Apr 24 03:19:01 aidas-gsi kernel: Total swap = 8200188kB
Apr 24 03:19:01 aidas-gsi kernel: 4194303 pages RAM
Apr 24 03:19:01 aidas-gsi kernel: 97868 pages reserved
Apr 24 03:19:01 aidas-gsi kernel: 578727 pages shared
Apr 24 03:19:01 aidas-gsi kernel: 3846484 pages non-shared
Apr 24 03:19:01 aidas-gsi kernel: [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name
Apr 24 03:19:01 aidas-gsi kernel: [ 1388] 0 1388 2819 130 2 -17 -1000 udevd
Apr 24 03:19:01 aidas-gsi kernel: [ 3927] 0 3927 7442 128 12 -17 -1000 auditd
Apr 24 03:19:01 aidas-gsi kernel: [ 3961] 0 3961 62914 755 15 0 0 rsyslogd
Apr 24 03:19:01 aidas-gsi kernel: [ 4029] 0 4029 4604 141 7 0 0 irqbalance
Apr 24 03:19:01 aidas-gsi kernel: [ 4054] 32 4054 4746 124 13 0 0 rpcbind
Apr 24 03:19:01 aidas-gsi kernel: [ 4074] 81 4074 8194 440 6 0 0 dbus-daemon
Apr 24 03:19:01 aidas-gsi kernel: [ 4089] 0 4089 26490 2647 4 0 0 NetworkManager
Apr 24 03:19:01 aidas-gsi kernel: [ 4096] 0 4096 14534 328 12 0 0 modem-manager
Apr 24 03:19:01 aidas-gsi kernel: [ 4112] 29 4112 5839 97 0 0 0 rpc.statd
Apr 24 03:19:01 aidas-gsi kernel: [ 4151] 0 4151 47297 117 15 0 0 cupsd
Apr 24 03:19:01 aidas-gsi kernel: [ 4171] 0 4171 11251 75 9 0 0 wpa_supplicant
Apr 24 03:19:01 aidas-gsi kernel: [ 4180] 0 4180 2280 162 9 0 0 dhclient
Apr 24 03:19:01 aidas-gsi kernel: [ 4210] 0 4210 1020 82 8 0 0 acpid
Apr 24 03:19:01 aidas-gsi kernel: [ 4222] 68 4222 9968 569 15 0 0 hald
Apr 24 03:19:01 aidas-gsi kernel: [ 4223] 0 4223 5100 184 12 0 0 hald-runner
Apr 24 03:19:01 aidas-gsi kernel: [ 4271] 0 4271 5630 157 12 0 0 hald-addon-inpu
Apr 24 03:19:01 aidas-gsi kernel: [ 4272] 68 4272 4502 120 0 0 0 hald-addon-acpi
Apr 24 03:19:01 aidas-gsi kernel: [ 4306] 0 4306 96540 161 0 0 0 automount
Apr 24 03:19:01 aidas-gsi kernel: [ 4492] 0 4492 26831 15 9 0 0 rpc.rquotad
Apr 24 03:19:01 aidas-gsi kernel: [ 4497] 0 4497 5450 213 4 0 0 rpc.mountd
Apr 24 03:19:01 aidas-gsi kernel: [ 4542] 0 4542 5776 42 4 0 0 rpc.idmapd
Apr 24 03:19:01 aidas-gsi kernel: [ 4567] 0 4567 16573 113 3 -17 -1000 sshd
Apr 24 03:19:01 aidas-gsi kernel: [ 4578] 0 4578 5429 117 7 0 0 xinetd
Apr 24 03:19:01 aidas-gsi kernel: [ 4589] 38 4589 7686 213 10 0 0 ntpd
Apr 24 03:19:01 aidas-gsi kernel: [ 4675] 0 4675 20260 147 5 0 0 master
Apr 24 03:19:01 aidas-gsi kernel: [ 4682] 89 4682 20323 141 5 0 0 qmgr
Apr 24 03:19:01 aidas-gsi kernel: [ 4696] 0 4696 27153 253 0 0 0 ksmtuned
Apr 24 03:19:01 aidas-gsi kernel: [ 4709] 0 4709 29222 137 10 0 0 crond
Apr 24 03:19:01 aidas-gsi kernel: [ 4724] 0 4724 5277 50 12 0 0 atd
Apr 24 03:19:01 aidas-gsi kernel: [ 4741] 0 4741 172796 2328 0 0 0 virt-who
Apr 24 03:19:01 aidas-gsi kernel: [ 4762] 0 4762 256599 407 10 0 0 libvirtd
Apr 24 03:19:01 aidas-gsi kernel: [ 4858] 0 4858 16126 150 10 0 0 certmonger
Apr 24 03:19:01 aidas-gsi kernel: [ 4903] 99 4903 3223 102 2 0 0 dnsmasq
Apr 24 03:19:01 aidas-gsi kernel: [ 4918] 0 4918 4539 223 5 0 0 dhcpd
Apr 24 03:19:01 aidas-gsi kernel: [ 4920] 0 4920 33509 312 11 0 0 gdm-binary
Apr 24 03:19:01 aidas-gsi kernel: [ 4927] 0 4927 1016 61 11 0 0 mingetty
Apr 24 03:19:01 aidas-gsi kernel: [ 4929] 0 4929 1016 61 12 0 0 mingetty
Apr 24 03:19:01 aidas-gsi kernel: [ 4931] 0 4931 1016 61 9 0 0 mingetty
Apr 24 03:19:01 aidas-gsi kernel: [ 4933] 0 4933 1016 61 15 0 0 mingetty
Apr 24 03:19:01 aidas-gsi kernel: [ 4935] 0 4935 1016 61 1 0 0 mingetty
Apr 24 03:19:01 aidas-gsi kernel: [ 4944] 0 4944 3148 440 2 -17 -1000 udevd
Apr 24 03:19:01 aidas-gsi kernel: [ 4945] 0 4945 3148 438 2 -17 -1000 udevd
Apr 24 03:19:01 aidas-gsi kernel: [ 5050] 0 5050 1029198 377 3 0 0 console-kit-dae
Apr 24 03:19:01 aidas-gsi kernel: [ 5126] 0 5126 12508 365 3 0 0 devkit-power-da
Apr 24 03:19:01 aidas-gsi kernel: [ 5164] 0 5164 45435 581 1 0 0 polkitd
Apr 24 03:19:01 aidas-gsi kernel: [ 5175] 499 5175 42114 146 0 0 0 rtkit-daemon
Apr 24 03:19:01 aidas-gsi kernel: [ 5702] 0 5702 14009 514 1 0 0 udisks-daemon
Apr 24 03:19:01 aidas-gsi kernel: [ 5725] 0 5725 11306 74 1 0 0 udisks-daemon
Apr 24 03:19:01 aidas-gsi kernel: [18391] 0 18391 41240 293 8 0 0 gdm-simple-slav
Apr 24 03:19:01 aidas-gsi kernel: [18393] 0 18393 3476635 2412883 13 0 0 Xorg
Apr 24 03:19:01 aidas-gsi kernel: [18427] 42 18427 5017 83 0 0 0 dbus-launch
Apr 24 03:19:01 aidas-gsi kernel: [18464] 0 18464 47926 269 0 0 0 gdm-session-wor
Apr 24 03:19:01 aidas-gsi kernel: [18510] 500 18510 57281 424 1 0 0 gnome-keyring-d
Apr 24 03:19:01 aidas-gsi kernel: [18520] 500 18520 64493 751 0 0 0 gnome-session
Apr 24 03:19:01 aidas-gsi kernel: [18528] 500 18528 5017 83 0 0 0 dbus-launch
Apr 24 03:19:01 aidas-gsi kernel: [18529] 500 18529 8211 379 8 0 0 dbus-daemon
Apr 24 03:19:01 aidas-gsi kernel: [18604] 500 18604 33379 1113 13 0 0 gconfd-2
Apr 24 03:19:01 aidas-gsi kernel: [18612] 500 18612 139831 1984 9 0 0 gnome-settings-
Apr 24 03:19:01 aidas-gsi kernel: [18614] 500 18614 71643 696 12 0 0 seahorse-daemon
Apr 24 03:19:01 aidas-gsi kernel: [18616] 500 18616 34351 298 13 0 0 gvfsd
Apr 24 03:19:01 aidas-gsi kernel: [18636] 500 18636 112427 2321 4 0 0 metacity
Apr 24 03:19:01 aidas-gsi kernel: [18637] 500 18637 203008 6568 10 0 0 gnome-panel
Apr 24 03:19:01 aidas-gsi kernel: [18638] 500 18638 326834 16727 10 0 0 nautilus
Apr 24 03:19:01 aidas-gsi kernel: [18640] 500 18640 141364 341 14 0 0 bonobo-activati
Apr 24 03:19:01 aidas-gsi kernel: [18642] 500 18642 65558 741 10 0 0 bluetooth-apple
Apr 24 03:19:01 aidas-gsi kernel: [18645] 500 18645 68687 968 4 0 0 gnome-power-man
Apr 24 03:19:01 aidas-gsi kernel: [18646] 500 18646 7601 812 12 0 0 restorecond
Apr 24 03:19:01 aidas-gsi kernel: [18648] 500 18648 28879 401 5 0 0 im-settings-dae
Apr 24 03:19:01 aidas-gsi kernel: [18649] 500 18649 132278 1323 4 0 0 gnome-volume-co
Apr 24 03:19:01 aidas-gsi kernel: [18654] 500 18654 75318 2282 10 0 0 gnome-terminal
Apr 24 03:19:01 aidas-gsi kernel: [18657] 500 18657 119660 1244 2 0 0 trashapplet
Apr 24 03:19:01 aidas-gsi kernel: [18659] 500 18659 88087 3906 5 0 0 wnck-applet
Apr 24 03:19:01 aidas-gsi kernel: [18660] 500 18660 62988 653 7 0 0 vino-server
Apr 24 03:19:01 aidas-gsi kernel: [18661] 500 18661 72490 958 0 0 0 polkit-gnome-au
Apr 24 03:19:01 aidas-gsi kernel: [18662] 500 18662 82733 1441 5 0 0 nm-applet
Apr 24 03:19:01 aidas-gsi kernel: [18666] 500 18666 126400 635 1 0 0 gvfs-gdu-volume
Apr 24 03:19:01 aidas-gsi kernel: [18667] 500 18667 74660 1042 5 0 0 gpk-update-icon
Apr 24 03:19:01 aidas-gsi kernel: [18669] 500 18669 64033 850 0 0 0 gdu-notificatio
Apr 24 03:19:01 aidas-gsi kernel: [18676] 500 18676 81107 2344 6 0 0 python
Apr 24 03:19:01 aidas-gsi kernel: [18693] 500 18693 131702 2471 3 0 0 pulseaudio
Apr 24 03:19:01 aidas-gsi kernel: [18823] 500 18823 58170 269 4 0 0 gvfs-afc-volume
Apr 24 03:19:01 aidas-gsi kernel: [18836] 500 18836 37789 300 8 0 0 gvfs-gphoto2-vo
Apr 24 03:19:01 aidas-gsi kernel: [18844] 500 18844 36051 437 0 0 0 gvfsd-trash
Apr 24 03:19:01 aidas-gsi kernel: [18868] 500 18868 10275 287 6 0 0 gconf-im-settin
Apr 24 03:19:01 aidas-gsi kernel: [18869] 500 18869 84055 4872 3 0 0 gnome-screensav
Apr 24 03:19:01 aidas-gsi kernel: [18870] 500 18870 2057 88 3 0 0 gnome-pty-helpe
Apr 24 03:19:01 aidas-gsi kernel: [18871] 500 18871 27123 198 0 0 0 bash
Apr 24 03:19:01 aidas-gsi kernel: [18894] 500 18894 98681 1288 0 0 0 gdm-user-switch
Apr 24 03:19:01 aidas-gsi kernel: [18896] 500 18896 72800 911 3 0 0 notification-ar
Apr 24 03:19:01 aidas-gsi kernel: [18897] 500 18897 143660 1846 2 0 0 clock-applet
Apr 24 03:19:01 aidas-gsi kernel: [18900] 500 18900 23718 242 3 0 0 gconf-helper
Apr 24 03:19:01 aidas-gsi kernel: [18918] 500 18918 34358 265 0 0 0 gvfsd-burn
Apr 24 03:19:01 aidas-gsi kernel: [18932] 500 18932 33897 307 4 0 0 gvfsd-metadata
Apr 24 03:19:01 aidas-gsi kernel: [18955] 500 18955 82196 7037 7 0 0 gnome-terminal
Apr 24 03:19:01 aidas-gsi kernel: [18957] 500 18957 2057 97 7 0 0 gnome-pty-helpe
Apr 24 03:19:01 aidas-gsi kernel: [18977] 500 18977 27123 114 8 0 0 bash
Apr 24 03:19:01 aidas-gsi kernel: [18991] 0 18991 47361 130 10 0 0 sudo
Apr 24 03:19:01 aidas-gsi kernel: [18992] 0 18992 27524 290 15 0 0 iptraf
Apr 24 03:19:01 aidas-gsi kernel: [21973] 500 21973 1058234 192783 0 0 0 firefox
Apr 24 03:19:01 aidas-gsi kernel: [21987] 500 21987 68608 284 3 0 0 at-spi-bus-laun
Apr 24 03:19:01 aidas-gsi kernel: [22221] 500 22221 731047 34381 0 0 0 Privileged Cont
Apr 24 03:19:01 aidas-gsi kernel: [22248] 500 22248 705574 10970 0 0 0 WebExtensions
Apr 24 03:19:01 aidas-gsi kernel: [22335] 500 22335 960920 166074 0 0 0 Web Content
Apr 24 03:19:01 aidas-gsi kernel: [22407] 500 22407 1023120 77418 0 0 0 Web Content
Apr 24 03:19:01 aidas-gsi kernel: [22810] 500 22810 27123 114 2 0 0 bash
Apr 24 03:19:01 aidas-gsi kernel: [22826] 500 22826 15487 185 8 0 0 ssh
Apr 24 03:19:01 aidas-gsi kernel: [22830] 500 22830 27123 114 9 0 0 bash
Apr 24 03:19:01 aidas-gsi kernel: [26234] 500 26234 839272 43450 3 0 0 Web Content
Apr 24 03:19:01 aidas-gsi kernel: [26311] 500 26311 27123 282 7 0 0 bash
Apr 24 03:19:01 aidas-gsi kernel: [26884] 500 26884 27123 115 5 0 0 bash
Apr 24 03:19:01 aidas-gsi kernel: [19388] 500 19388 27123 293 0 0 0 bash
Apr 24 03:19:01 aidas-gsi kernel: [11683] 500 11683 791783 18526 0 0 0 Web Content
Apr 24 03:19:01 aidas-gsi kernel: [ 4876] 500 4876 54985 304 5 0 0 gvfsd-computer
Apr 24 03:19:01 aidas-gsi kernel: [17248] 500 17248 29595 100 1 0 0 screen
Apr 24 03:19:01 aidas-gsi kernel: [17249] 500 17249 29631 157 10 0 0 screen
Apr 24 03:19:01 aidas-gsi kernel: [17250] 500 17250 27124 114 7 0 0 bash
Apr 24 03:19:01 aidas-gsi kernel: [17266] 500 17266 15489 181 8 0 0 ssh
Apr 24 03:19:01 aidas-gsi kernel: [19879] 0 19879 25531 154 0 0 0 sshd
Apr 24 03:19:01 aidas-gsi kernel: [19883] 500 19883 25531 163 2 0 0 sshd
Apr 24 03:19:01 aidas-gsi kernel: [19884] 500 19884 27638 106 15 0 0 bash
Apr 24 03:19:01 aidas-gsi kernel: [19907] 500 19907 29595 97 5 0 0 screen
Apr 24 03:19:01 aidas-gsi kernel: [14850] 500 14850 56156 327 2 0 0 gvfsd-network
Apr 24 03:19:01 aidas-gsi kernel: [14855] 500 14855 36479 318 3 0 0 gvfsd-dnssd
Apr 24 03:19:01 aidas-gsi kernel: [22615] 500 22615 27123 117 9 0 0 bash
Apr 24 03:19:01 aidas-gsi kernel: [13867] 0 13867 25531 161 0 0 0 sshd
Apr 24 03:19:01 aidas-gsi kernel: [13880] 500 13880 25645 177 8 0 0 sshd
Apr 24 03:19:01 aidas-gsi kernel: [13881] 500 13881 14400 138 8 0 0 sftp-server
Apr 24 03:19:01 aidas-gsi kernel: [20258] 500 20258 776888 93602 3 0 0 Web Content
Apr 24 03:19:01 aidas-gsi kernel: [25910] 500 25910 15050 209 3 0 0 ssh
Apr 24 03:19:01 aidas-gsi kernel: [26010] 500 26010 2879 95 15 0 0 tinyproxy
Apr 24 03:19:01 aidas-gsi kernel: [26513] 0 26513 25530 204 12 0 0 sshd
Apr 24 03:19:01 aidas-gsi kernel: [26517] 500 26517 25530 177 8 0 0 sshd
Apr 24 03:19:01 aidas-gsi kernel: [29203] 0 29203 25016 164 15 0 0 sshd
Apr 24 03:19:01 aidas-gsi kernel: [29207] 500 29207 25534 158 6 0 0 sshd
Apr 24 03:19:01 aidas-gsi kernel: [ 1105] 500 1105 27123 143 7 0 0 bash
Apr 24 03:19:01 aidas-gsi kernel: [17163] 0 17163 25016 190 15 0 0 sshd
Apr 24 03:19:01 aidas-gsi kernel: [17167] 500 17167 25534 185 0 0 0 sshd
Apr 24 03:19:01 aidas-gsi kernel: [18690] 0 18690 25530 183 8 0 0 sshd
Apr 24 03:19:01 aidas-gsi kernel: [18695] 500 18695 25530 138 8 0 0 sshd
Apr 24 03:19:01 aidas-gsi kernel: [18696] 500 18696 27123 127 0 0 0 bash
Apr 24 03:19:01 aidas-gsi kernel: [20802] 0 20802 25530 173 8 0 0 sshd
Apr 24 03:19:01 aidas-gsi kernel: [20806] 500 20806 25530 142 8 0 0 sshd
Apr 24 03:19:01 aidas-gsi kernel: [20807] 500 20807 27123 172 9 0 0 bash
Apr 24 03:19:01 aidas-gsi kernel: [ 9697] 0 9697 25530 187 0 0 0 sshd
Apr 24 03:19:01 aidas-gsi kernel: [ 9701] 500 9701 25563 184 10 0 0 sshd
Apr 24 03:19:01 aidas-gsi kernel: [ 9703] 500 9703 27123 132 0 0 0 bash
Apr 24 03:19:01 aidas-gsi kernel: [21737] 500 21737 29350 360 10 0 0 python
Apr 24 03:19:01 aidas-gsi kernel: [21860] 500 21860 166837 1623 5 0 0 anydesk
Apr 24 03:19:01 aidas-gsi kernel: [21861] 500 21861 208277 3415 0 0 0 anydesk
Apr 24 03:19:01 aidas-gsi kernel: [21868] 500 21868 470786 1884 2 0 0 anydesk
Apr 24 03:19:01 aidas-gsi kernel: [20297] 500 20297 731145 44730 0 0 0 Web Content
Apr 24 03:19:01 aidas-gsi kernel: [23452] 0 23452 47361 305 5 0 0 sudo
Apr 24 03:19:01 aidas-gsi kernel: [23453] 0 23453 25240 86 2 0 0 tail
Apr 24 03:19:01 aidas-gsi kernel: [23841] 500 23841 27123 243 3 0 0 bash
Apr 24 03:19:01 aidas-gsi kernel: [ 345] 500 345 27156 311 15 0 0 bash
Apr 24 03:19:01 aidas-gsi kernel: [ 4825] 500 4825 682055 25908 0 0 0 Web Content
Apr 24 03:19:01 aidas-gsi kernel: [16950] 500 16950 26921 257 1 0 0 watch
Apr 24 03:19:01 aidas-gsi kernel: [10822] 500 10822 1060110 432573 3 0 0 anydesk
Apr 24 03:19:01 aidas-gsi kernel: [29109] 500 29109 27186 482 0 0 0 bash
Apr 24 03:19:01 aidas-gsi kernel: [ 3754] 500 3754 27123 375 8 0 0 bash
Apr 24 03:19:01 aidas-gsi kernel: [11083] 500 11083 28030 216 3 0 0 HTTPD@8115
Apr 24 03:19:01 aidas-gsi kernel: [11087] 500 11087 27504 195 0 0 0 TclHttpd-server
Apr 24 03:19:01 aidas-gsi kernel: [11089] 500 11089 36731 3211 12 0 0 tclsh8.5_copy2
Apr 24 03:19:01 aidas-gsi kernel: [11097] 500 11097 28030 207 1 0 0 TapeServer
Apr 24 03:19:01 aidas-gsi kernel: [11104] 500 11104 5398 1567 10 0 0 master
Apr 24 03:19:01 aidas-gsi kernel: [11105] 500 11105 25231 134 12 0 0 sleep
Apr 24 03:19:01 aidas-gsi kernel: [11106] 500 11106 4323 187 6 0 0 stats
Apr 24 03:19:01 aidas-gsi kernel: [11107] 500 11107 4918 219 5 0 0 driver
Apr 24 03:19:01 aidas-gsi kernel: [11108] 500 11108 4917 194 9 0 0 driver
Apr 24 03:19:01 aidas-gsi kernel: [11109] 500 11109 4917 194 12 0 0 driver
Apr 24 03:19:01 aidas-gsi kernel: [11110] 500 11110 4630 214 0 0 0 linkTCP
Apr 24 03:19:01 aidas-gsi kernel: [11111] 500 11111 4630 182 9 0 0 linkTCP
Apr 24 03:19:01 aidas-gsi kernel: [11114] 500 11114 28030 217 12 0 0 NewMerger
Apr 24 03:19:01 aidas-gsi kernel: [11124] 500 11124 404629 8857 0 0 0 master64
Apr 24 03:19:01 aidas-gsi kernel: [11125] 500 11125 25231 135 9 0 0 sleep
Apr 24 03:19:01 aidas-gsi kernel: [11126] 500 11126 19761 17318 13 0 0 statrate64
Apr 24 03:19:01 aidas-gsi kernel: [11127] 500 11127 35968 11059 2 0 0 link64
Apr 24 03:19:01 aidas-gsi kernel: [11128] 500 11128 35968 11498 11 0 0 link64
Apr 24 03:19:01 aidas-gsi kernel: [11129] 500 11129 35968 11581 14 0 0 link64
Apr 24 03:19:01 aidas-gsi kernel: [11130] 500 11130 35968 12366 6 0 0 link64
Apr 24 03:19:01 aidas-gsi kernel: [11131] 500 11131 35968 10031 5 0 0 link64
Apr 24 03:19:01 aidas-gsi kernel: [11132] 500 11132 35968 10689 5 0 0 link64
Apr 24 03:19:01 aidas-gsi kernel: [11133] 500 11133 35968 11896 6 0 0 link64
Apr 24 03:19:01 aidas-gsi kernel: [11134] 500 11134 35968 10510 14 0 0 link64
Apr 24 03:19:01 aidas-gsi kernel: [11135] 500 11135 35968 15679 14 0 0 link64
Apr 24 03:19:01 aidas-gsi kernel: [11136] 500 11136 35968 14623 14 0 0 link64
Apr 24 03:19:01 aidas-gsi kernel: [11137] 500 11137 35968 12790 15 0 0 link64
Apr 24 03:19:01 aidas-gsi kernel: [11138] 500 11138 35968 11683 0 0 0 link64
Apr 24 03:19:01 aidas-gsi kernel: [11139] 500 11139 35968 8773 5 0 0 link64
Apr 24 03:19:01 aidas-gsi kernel: [11140] 500 11140 35968 8774 10 0 0 link64
Apr 24 03:19:01 aidas-gsi kernel: [11141] 500 11141 35968 8772 10 0 0 link64
Apr 24 03:19:01 aidas-gsi kernel: [11142] 500 11142 35968 8773 10 0 0 link64
Apr 24 03:19:01 aidas-gsi kernel: [11152] 500 11152 307392 47667 5 0 0 merge64
Apr 24 03:19:01 aidas-gsi kernel: [11200] 500 11200 723346 37902 5 0 0 file:// Content
Apr 24 03:19:01 aidas-gsi kernel: [11338] 500 11338 28030 216 0 0 0 HTTPD
Apr 24 03:19:01 aidas-gsi kernel: [11342] 500 11342 27504 195 0 0 0 TclHttpd-server
Apr 24 03:19:01 aidas-gsi kernel: [11344] 500 11344 46965 14788 3 0 0 tclsh8.5
Apr 24 03:19:01 aidas-gsi kernel: [12835] 500 12835 766232 71066 0 0 0 Web Content
Apr 24 03:19:01 aidas-gsi kernel: [24071] 500 24071 27123 409 9 0 0 bash
Apr 24 03:19:01 aidas-gsi kernel: [24347] 500 24347 28030 231 4 0 0 datarelaymbs
Apr 24 03:19:01 aidas-gsi kernel: [24351] 500 24351 7751 1313 10 0 0 DataRelayMBS
Apr 24 03:19:01 aidas-gsi kernel: [ 407] 89 407 20280 470 13 0 0 pickup
Apr 24 03:19:01 aidas-gsi kernel: [ 1586] 0 1586 4238 156 13 0 0 anacron
Apr 24 03:19:01 aidas-gsi kernel: [ 1876] 0 1876 25231 144 10 0 0 sleep
Apr 24 03:19:01 aidas-gsi kernel: Out of memory: Kill process 18393 (Xorg) score 523 or sacrifice child
Apr 24 03:19:01 aidas-gsi kernel: Killed process 18393, UID 0, (Xorg) total-vm:13906540kB, anon-rss:9558800kB, file-rss:92712kB
Apr 24 03:19:17 aidas-gsi gnome-keyring-daemon[18510]: dbus failure unregistering from session: Connection is closed
Apr 24 03:19:17 aidas-gsi gnome-keyring-daemon[18510]: dbus failure unregistering from session: Connection is closed
Apr 24 03:19:30 aidas-gsi dbus: [system] Rejected send message, 2 matched rules; type="method_call", sender=":1.11471" (uid=500 pid=1884 comm="/usr/libexec/anydesk)
interface="org.freedesktop.DBus.Properties" member="GetAll" error name="(unset)" requested_reply=0 destination=":1.20" (uid=0 pid=5050 comm="/usr/sbin/console-kit-
daemon))
Apr 24 03:19:30 aidas-gsi dbus: [system] Rejected send message, 2 matched rules; type="method_call", sender=":1.11473" (uid=500 pid=1887 comm="/usr/libexec/anydesk)
interface="org.freedesktop.DBus.Properties" member="GetAll" error name="(unset)" requested_reply=0 destination=":1.20" (uid=0 pid=5050 comm="/usr/sbin/console-kit-
daemon))
Apr 24 03:19:45 aidas-gsi kernel: nouveau: evo channel stalled
Apr 24 03:19:47 aidas-gsi rtkit-daemon[5175]: Sucessfully made thread 1961 of process 1961 (/usr/bin/pulseaudio) owned by '42' high priority at nice level -11.
Apr 24 03:19:48 aidas-gsi pulseaudio[1961]: conf-parser.c: [/usr/share/pulseaudio/alsa-mixer/paths/hdmi-output-0.conf:6] Unknown lvalue 'required-any' in section 'Jack
HDMI/DP,pcm=3'.
Apr 24 03:19:48 aidas-gsi rtkit-daemon[5175]: Sucessfully made thread 1965 of process 1961 (/usr/bin/pulseaudio) owned by '42' RT at priority 5.
Apr 24 03:19:48 aidas-gsi rtkit-daemon[5175]: Sucessfully made thread 1966 of process 1961 (/usr/bin/pulseaudio) owned by '42' RT at priority 5.
Apr 24 03:19:48 aidas-gsi rtkit-daemon[5175]: Sucessfully made thread 1967 of process 1961 (/usr/bin/pulseaudio) owned by '42' RT at priority 5.
Apr 24 03:19:52 aidas-gsi dbus: [system] Rejected send message, 2 matched rules; type="method_call", sender=":1.11485" (uid=500 pid=1981 comm="/usr/libexec/anydesk)
interface="org.freedesktop.DBus.Properties" member="GetAll" error name="(unset)" requested_reply=0 destination=":1.20" (uid=0 pid=5050 comm="/usr/sbin/console-kit-
daemon))
Apr 24 03:19:52 aidas-gsi dbus: [system] Rejected send message, 2 matched rules; type="method_call", sender=":1.11487" (uid=500 pid=1983 comm="/usr/libexec/anydesk)
interface="org.freedesktop.DBus.Properties" member="GetAll" error name="(unset)" requested_reply=0 destination=":1.20" (uid=0 pid=5050 comm="/usr/sbin/console-kit-
daemon))
Apr 24 03:30:44 aidas-gsi seahorse-daemon[2215]: DNS-SD initialization failed: Daemon not running
Apr 24 03:30:44 aidas-gsi seahorse-daemon[2215]: init gpgme version 1.1.8
Apr 24 03:30:44 aidas-gsi rtkit-daemon[5175]: Sucessfully made thread 2243 of process 2243 (/usr/bin/pulseaudio) owned by '500' high priority at nice level -11.
Apr 24 03:30:45 aidas-gsi pulseaudio[2243]: conf-parser.c: [/usr/share/pulseaudio/alsa-mixer/paths/hdmi-output-0.conf:6] Unknown lvalue 'required-any' in section 'Jack
HDMI/DP,pcm=3'.
Apr 24 03:30:45 aidas-gsi rtkit-daemon[5175]: Sucessfully made thread 2301 of process 2243 (/usr/bin/pulseaudio) owned by '500' RT at priority 5.
Apr 24 03:30:45 aidas-gsi rtkit-daemon[5175]: Sucessfully made thread 2302 of process 2243 (/usr/bin/pulseaudio) owned by '500' RT at priority 5.
Apr 24 03:30:45 aidas-gsi rtkit-daemon[5175]: Sucessfully made thread 2303 of process 2243 (/usr/bin/pulseaudio) owned by '500' RT at priority 5.
Apr 24 03:30:46 aidas-gsi rtkit-daemon[5175]: Sucessfully made thread 2325 of process 2325 (/usr/bin/pulseaudio) owned by '500' high priority at nice level -11.
Apr 24 03:30:46 aidas-gsi pulseaudio[2325]: pid.c: Daemon already running.
Apr 24 03:30:46 aidas-gsi rtkit-daemon[5175]: Sucessfully made thread 2326 of process 2326 (/usr/bin/pulseaudio) owned by '500' high priority at nice level -11.
Apr 24 03:30:46 aidas-gsi pulseaudio[2326]: pid.c: Daemon already running.
Apr 24 03:30:46 aidas-gsi dbus: [system] Rejected send message, 2 matched rules; type="method_call", sender=":1.11506" (uid=500 pid=2328 comm="/usr/libexec/anydesk)
interface="org.freedesktop.DBus.Properties" member="GetAll" error name="(unset)" requested_reply=0 destination=":1.20" (uid=0 pid=5050 comm="/usr/sbin/console-kit-
daemon))
Apr 24 03:30:46 aidas-gsi rtkit-daemon[5175]: Sucessfully made thread 2418 of process 2418 (/usr/bin/pulseaudio) owned by '500' high priority at nice level -11.
Apr 24 03:30:46 aidas-gsi pulseaudio[2418]: pid.c: Daemon already running.
Apr 24 03:30:46 aidas-gsi dbus: [system] Rejected send message, 2 matched rules; type="method_call", sender=":1.11508" (uid=500 pid=2420 comm="/usr/libexec/anydesk)
interface="org.freedesktop.DBus.Properties" member="GetAll" error name="(unset)" requested_reply=0 destination=":1.20" (uid=0 pid=5050 comm="/usr/sbin/console-kit-
daemon))
Apr 24 03:30:51 aidas-gsi pulseaudio[2243]: ratelimit.c: 25 events suppressed
Apr 24 03:30:51 aidas-gsi kernel: nouveau: evo channel stalled
Apr 24 03:32:31 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.251 from b8:27:eb:bb:46:7b via p4p1
Apr 24 03:32:31 aidas-gsi dhcpd: DHCPACK on 192.168.11.251 to b8:27:eb:bb:46:7b via p4p1
Apr 24 03:33:11 aidas-gsi kernel: nouveau: evo channel stalled
Apr 24 03:38:36 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.117 from b8:27:eb:40:53:e8 via p4p1
Apr 24 03:38:36 aidas-gsi dhcpd: DHCPACK on 192.168.11.117 to b8:27:eb:40:53:e8 via p4p1
Apr 24 03:39:51 aidas-gsi dbus: [system] Rejected send message, 2 matched rules; type="method_call", sender=":1.11521" (uid=500 pid=3005 comm="/usr/libexec/anydesk)
interface="org.freedesktop.DBus.Properties" member="GetAll" error name="(unset)" requested_reply=0 destination=":1.20" (uid=0 pid=5050 comm="/usr/sbin/console-kit-
daemon))
Apr 24 03:39:51 aidas-gsi dbus: [system] Rejected send message, 2 matched rules; type="method_call", sender=":1.11524" (uid=500 pid=3012 comm="/usr/libexec/anydesk)
interface="org.freedesktop.DBus.Properties" member="GetAll" error name="(unset)" requested_reply=0 destination=":1.20" (uid=0 pid=5050 comm="/usr/sbin/console-kit-
daemon))
Apr 24 03:39:51 aidas-gsi dbus: [system] Rejected send message, 2 matched rules; type="method_call", sender=":1.11525" (uid=500 pid=3015 comm="/usr/libexec/anydesk)
interface="org.freedesktop.DBus.Properties" member="GetAll" error name="(unset)" requested_reply=0 destination=":1.20" (uid=0 pid=5050 comm="/usr/sbin/console-kit-
daemon))
Apr 24 03:39:51 aidas-gsi dbus: [system] Rejected send message, 2 matched rules; type="method_call", sender=":1.11527" (uid=500 pid=3018 comm="/usr/libexec/anydesk)
interface="org.freedesktop.DBus.Properties" member="GetAll" error name="(unset)" requested_reply=0 destination=":1.20" (uid=0 pid=5050 comm="/usr/sbin/console-kit-
daemon))
Apr 24 03:39:51 aidas-gsi dbus: [system] Rejected send message, 2 matched rules; type="method_call", sender=":1.11529" (uid=500 pid=3022 comm="/usr/libexec/anydesk)
interface="org.freedesktop.DBus.Properties" member="GetAll" error name="(unset)" requested_reply=0 destination=":1.20" (uid=0 pid=5050 comm="/usr/sbin/console-kit-
daemon))
Apr 24 03:39:51 aidas-gsi dbus: [system] Rejected send message, 2 matched rules; type="method_call", sender=":1.11531" (uid=500 pid=3024 comm="/usr/libexec/anydesk)
interface="org.freedesktop.DBus.Properties" member="GetAll" error name="(unset)" requested_reply=0 destination=":1.20" (uid=0 pid=5050 comm="/usr/sbin/console-kit-
daemon))
Apr 24 03:40:13 aidas-gsi dbus: [system] Rejected send message, 2 matched rules; type="method_call", sender=":1.11533" (uid=500 pid=3100 comm="/usr/libexec/anydesk)
interface="org.freedesktop.DBus.Properties" member="GetAll" error name="(unset)" requested_reply=0 destination=":1.20" (uid=0 pid=5050 comm="/usr/sbin/console-kit-
daemon))
Apr 24 03:40:37 aidas-gsi kernel: nouveau: evo channel stalled
Apr 24 03:40:46 aidas-gsi kernel: nouveau: evo channel stalled
Apr 24 03:41:22 aidas-gsi kernel: nouveau: evo channel stalled
Apr 24 03:42:10 aidas-gsi kernel: nouveau: evo channel stalled
Apr 24 03:42:19 aidas-gsi kernel: nouveau: evo channel stalled
Apr 24 03:43:31 aidas-gsi kernel: nouveau: evo channel stalled
Apr 24 03:47:10 aidas-gsi kernel: nouveau: evo channel stalled
Apr 24 03:49:55 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.117 from b8:27:eb:40:53:e8 via p4p1
Apr 24 03:49:55 aidas-gsi dhcpd: DHCPACK on 192.168.11.117 to b8:27:eb:40:53:e8 via p4p1
Apr 24 03:50:58 aidas-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:b4:0c via p4p1
Apr 24 03:50:58 aidas-gsi dhcpd: DHCPOFFER on 192.168.11.7 to d8:80:39:41:b4:0c via p4p1
Apr 24 03:50:58 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.7 (192.168.11.99) from d8:80:39:41:b4:0c via p4p1
Apr 24 03:50:58 aidas-gsi dhcpd: DHCPACK on 192.168.11.7 to d8:80:39:41:b4:0c via p4p1
Apr 24 03:50:58 aidas-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:f6:b7 via p4p1
Apr 24 03:50:58 aidas-gsi dhcpd: DHCPOFFER on 192.168.11.4 to d8:80:39:41:f6:b7 via p4p1
Apr 24 03:50:58 aidas-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:d8:21 via p4p1
Apr 24 03:50:58 aidas-gsi dhcpd: DHCPOFFER on 192.168.11.3 to d8:80:39:41:d8:21 via p4p1
Apr 24 03:50:58 aidas-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:ba:2b via p4p1
Apr 24 03:50:58 aidas-gsi dhcpd: DHCPOFFER on 192.168.11.8 to d8:80:39:41:ba:2b via p4p1
Apr 24 03:50:58 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.4 (192.168.11.99) from d8:80:39:41:f6:b7 via p4p1
Apr 24 03:50:58 aidas-gsi dhcpd: DHCPACK on 192.168.11.4 to d8:80:39:41:f6:b7 via p4p1
Apr 24 03:50:58 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.3 (192.168.11.99) from d8:80:39:41:d8:21 via p4p1
Apr 24 03:50:58 aidas-gsi dhcpd: DHCPACK on 192.168.11.3 to d8:80:39:41:d8:21 via p4p1
Apr 24 03:50:58 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.8 (192.168.11.99) from d8:80:39:41:ba:2b via p4p1
Apr 24 03:50:58 aidas-gsi dhcpd: DHCPACK on 192.168.11.8 to d8:80:39:41:ba:2b via p4p1
Apr 24 03:50:58 aidas-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:ba:22 via p4p1
Apr 24 03:50:58 aidas-gsi dhcpd: DHCPOFFER on 192.168.11.2 to d8:80:39:41:ba:22 via p4p1
Apr 24 03:50:58 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.2 (192.168.11.99) from d8:80:39:41:ba:22 via p4p1
Apr 24 03:50:58 aidas-gsi dhcpd: DHCPACK on 192.168.11.2 to d8:80:39:41:ba:22 via p4p1
Apr 24 03:50:58 aidas-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:ba:8a via p4p1
Apr 24 03:50:58 aidas-gsi dhcpd: DHCPOFFER on 192.168.11.1 to d8:80:39:41:ba:8a via p4p1
Apr 24 03:50:58 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.1 (192.168.11.99) from d8:80:39:41:ba:8a via p4p1
Apr 24 03:50:58 aidas-gsi dhcpd: DHCPACK on 192.168.11.1 to d8:80:39:41:ba:8a via p4p1
Apr 24 03:50:58 aidas-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:ee:72 via p4p1
Apr 24 03:50:58 aidas-gsi dhcpd: DHCPOFFER on 192.168.11.6 to d8:80:39:41:ee:72 via p4p1
Apr 24 03:50:58 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.6 (192.168.11.99) from d8:80:39:41:ee:72 via p4p1
Apr 24 03:50:58 aidas-gsi dhcpd: DHCPACK on 192.168.11.6 to d8:80:39:41:ee:72 via p4p1
Apr 24 03:50:58 aidas-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:d7:cc via p4p1
Apr 24 03:50:58 aidas-gsi dhcpd: DHCPOFFER on 192.168.11.5 to d8:80:39:41:d7:cc via p4p1
Apr 24 03:50:58 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.5 (192.168.11.99) from d8:80:39:41:d7:cc via p4p1
Apr 24 03:50:58 aidas-gsi dhcpd: DHCPACK on 192.168.11.5 to d8:80:39:41:d7:cc via p4p1
Apr 24 03:50:58 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.8:816 for /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida08
(/home/Embedded/XilinxLinux/ppc_4xx/rfs)
Apr 24 03:50:58 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.7:952 for /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida07
(/home/Embedded/XilinxLinux/ppc_4xx/rfs)
Apr 24 03:50:58 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.3:884 for /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida03
(/home/Embedded/XilinxLinux/ppc_4xx/rfs)
Apr 24 03:50:58 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.4:895 for /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida04
(/home/Embedded/XilinxLinux/ppc_4xx/rfs)
Apr 24 03:50:58 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.2:918 for /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida02
(/home/Embedded/XilinxLinux/ppc_4xx/rfs)
Apr 24 03:50:59 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.1:892 for /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida01
(/home/Embedded/XilinxLinux/ppc_4xx/rfs)
Apr 24 03:50:59 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.6:839 for /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida06
(/home/Embedded/XilinxLinux/ppc_4xx/rfs)
Apr 24 03:50:59 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.5:897 for /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida05
(/home/Embedded/XilinxLinux/ppc_4xx/rfs)
Apr 24 03:51:03 aidas-gsi xinetd[4578]: START: time-stream pid=0 from=::ffff:192.168.11.8
Apr 24 03:51:03 aidas-gsi xinetd[4578]: START: time-stream pid=0 from=::ffff:192.168.11.1
Apr 24 03:51:03 aidas-gsi xinetd[4578]: START: time-stream pid=0 from=::ffff:192.168.11.7
Apr 24 03:51:03 aidas-gsi xinetd[4578]: START: time-stream pid=0 from=::ffff:192.168.11.3
Apr 24 03:51:03 aidas-gsi xinetd[4578]: START: time-stream pid=0 from=::ffff:192.168.11.6
Apr 24 03:51:03 aidas-gsi xinetd[4578]: START: time-stream pid=0 from=::ffff:192.168.11.4
Apr 24 03:51:03 aidas-gsi xinetd[4578]: START: time-stream pid=0 from=::ffff:192.168.11.2
Apr 24 03:51:04 aidas-gsi xinetd[4578]: START: time-stream pid=0 from=::ffff:192.168.11.5
Apr 24 03:51:23 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.7:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119
(/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 24 03:51:23 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.8:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119
(/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 24 03:51:25 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.3:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119
(/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 24 03:51:25 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.1:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119
(/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 24 03:51:29 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.4:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119
(/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 24 03:51:34 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.2:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119
(/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 24 03:51:36 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.6:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119
(/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 24 03:51:44 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.5:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119
(/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 24 03:52:06 aidas-gsi kernel: nouveau: evo channel stalled
Apr 24 03:52:10 aidas-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:f6:ee via p4p1
Apr 24 03:52:10 aidas-gsi dhcpd: DHCPOFFER on 192.168.11.9 to d8:80:39:41:f6:ee via p4p1
Apr 24 03:52:10 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.9 (192.168.11.99) from d8:80:39:41:f6:ee via p4p1
Apr 24 03:52:10 aidas-gsi dhcpd: DHCPACK on 192.168.11.9 to d8:80:39:41:f6:ee via p4p1
Apr 24 03:52:10 aidas-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:ba:89 via p4p1
Apr 24 03:52:10 aidas-gsi dhcpd: DHCPOFFER on 192.168.11.10 to d8:80:39:41:ba:89 via p4p1
Apr 24 03:52:10 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.10 (192.168.11.99) from d8:80:39:41:ba:89 via p4p1
Apr 24 03:52:10 aidas-gsi dhcpd: DHCPACK on 192.168.11.10 to d8:80:39:41:ba:89 via p4p1
Apr 24 03:52:11 aidas-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:f6:5a via p4p1
Apr 24 03:52:11 aidas-gsi dhcpd: DHCPOFFER on 192.168.11.11 to d8:80:39:41:f6:5a via p4p1
Apr 24 03:52:11 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.11 (192.168.11.99) from d8:80:39:41:f6:5a via p4p1
Apr 24 03:52:11 aidas-gsi dhcpd: DHCPACK on 192.168.11.11 to d8:80:39:41:f6:5a via p4p1
Apr 24 03:52:11 aidas-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:cf:ac via p4p1
Apr 24 03:52:11 aidas-gsi dhcpd: DHCPOFFER on 192.168.11.12 to d8:80:39:41:cf:ac via p4p1
Apr 24 03:52:11 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.12 (192.168.11.99) from d8:80:39:41:cf:ac via p4p1
Apr 24 03:52:11 aidas-gsi dhcpd: DHCPACK on 192.168.11.12 to d8:80:39:41:cf:ac via p4p1
Apr 24 03:52:11 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.9:814 for /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida09
(/home/Embedded/XilinxLinux/ppc_4xx/rfs)
Apr 24 03:52:11 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.10:852 for /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida10
(/home/Embedded/XilinxLinux/ppc_4xx/rfs)
Apr 24 03:52:11 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.11:694 for /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida11
(/home/Embedded/XilinxLinux/ppc_4xx/rfs)
Apr 24 03:52:11 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.12:770 for /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida12
(/home/Embedded/XilinxLinux/ppc_4xx/rfs)
Apr 24 03:52:15 aidas-gsi xinetd[4578]: START: time-stream pid=0 from=::ffff:192.168.11.9
Apr 24 03:52:15 aidas-gsi xinetd[4578]: START: time-stream pid=0 from=::ffff:192.168.11.10
Apr 24 03:52:15 aidas-gsi xinetd[4578]: START: time-stream pid=0 from=::ffff:192.168.11.12
Apr 24 03:52:15 aidas-gsi xinetd[4578]: START: time-stream pid=0 from=::ffff:192.168.11.11
Apr 24 03:52:33 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.9:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119
(/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 24 03:52:35 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.11:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119
(/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 24 03:52:36 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.10:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119
(/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 24 03:52:36 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.12:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119
(/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119) |
|
270
|
Sat Apr 24 03:13:05 2021 |
ML | AIDA DAQ Reset |
AIDA PC crashe completely at ~3:30am CET.
Tom was called and he Restarted the AIDA system at ~4am with Run 52.
Not sure what cause the crash but AnyDesk remote connection was suddenly lost and once reconnected all Terminal sessions were gone.
|
|
269
|
Sat Apr 24 01:44:09 2021 |
ML | system wide checks |
system wide checks:
Clock status:
FEE64 module aida06 global clocks failed, 6
Clock status test result: Passed 11, Failed 1
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable
ADC calibration:
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida12 failed
Calibration test result: Passed 9, Failed 3
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
White Rabbit
Base Current Difference
aida06 fault 0x679f : 0x67a4 : 5
White Rabbit error counter test result: Passed 11, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
FPGA timestamp error page not working !!
Memory information:
Returned 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 19 15 4 6 4 3 2 4 2 3 6 : 36420
aida02 : 36 25 24 6 0 4 2 3 2 3 4 : 28056
aida03 : 17 7 4 3 3 5 2 2 3 3 6 : 36444
aida04 : 4 5 4 5 3 4 2 3 3 3 6 : 36824
aida05 : 23 9 4 2 3 3 1 3 3 3 6 : 36452
aida06 : 26 8 5 1 3 4 2 2 3 3 6 : 36312
aida07 : 21 9 3 3 4 5 1 2 3 3 6 : 36268
aida08 : 27 9 2 2 2 4 1 3 3 3 6 : 36500
aida09 : 13 4 3 3 1 2 4 2 3 3 6 : 36388
aida10 : 1 10 0 3 1 2 3 3 2 3 6 : 35572
aida11 : 23 13 8 3 3 4 3 3 2 3 6 : 36196
aida12 : 19 7 4 3 2 3 2 3 3 3 6 : 36644
|
| Attachment 1: 24-04-2021_230am_AIDA_HV.png
|
|
| Attachment 2: 24-04-2021_230am_ucesbStat.png
|
|
| Attachment 3: 24-04-2021_230am_grafana.png
|
|
| Attachment 4: 24-04-2021_230am_AidaGoodEvent.png
|
|
| Attachment 5: 24-04-2021_230am_Correlation#8.png
|
|
| Attachment 6: 24-04-2021_230am_RESUME#3.png
|
|
| Attachment 7: 24-04-2021_230am_PAUSE#2.png
|
|
|
268
|
Fri Apr 23 20:19:52 2021 |
TD | Friday 23 April 20.00-00.00 |
21.19 system wide checks
FEE64 module aida06 global clocks failed, 6
Clock status test result: Passed 11, Failed 1
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida12 failed
Calibration test result: Passed 9, Failed 3
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
Base Current Difference
aida06 fault 0x679f : 0x67a2 : 3
White Rabbit error counter test result: Passed 11, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Returned 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 23 7 5 7 3 3 2 4 2 3 6 : 36356
aida02 : 67 29 27 4 0 4 2 3 2 3 4 : 28196
aida03 : 14 5 2 4 3 5 2 2 3 3 6 : 36416
aida04 : 5 5 0 7 2 3 3 3 3 3 6 : 36892
aida05 : 26 5 4 3 3 3 1 3 3 3 6 : 36464
aida06 : 20 7 5 2 3 4 2 2 3 3 6 : 36312
aida07 : 24 7 1 2 3 4 2 3 3 3 6 : 36776
aida08 : 22 7 6 2 3 3 1 3 3 3 6 : 36464
aida09 : 20 7 1 2 1 3 3 2 3 3 6 : 36248
aida10 : 9 1 6 1 1 2 2 4 2 3 6 : 35820
aida11 : 15 4 2 3 3 4 3 3 2 3 6 : 35996
aida12 : 22 11 6 2 2 3 2 2 3 3 6 : 36176
Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same
FEE : aida01 => Options file size is 1026 Last changed Fri Apr 23 03:51:08 CEST 2021
FEE : aida02 => Options file size is 1014 Last changed Fri Apr 16 00:56:20 CEST 2021
FEE : aida03 => Options file size is 1014 Last changed Wed Apr 14 21:52:04 CEST 2021
FEE : aida04 => Options file size is 1025 Last changed Sat Apr 17 06:07:36 CEST 2021
FEE : aida05 => Options file size is 1025 Last changed Fri Apr 16 00:53:25 CEST 2021
FEE : aida06 => Options file size is 1014 Last changed Wed Apr 14 21:52:04 CEST 2021
FEE : aida07 => Options file size is 1014 Last changed Wed Apr 14 21:52:04 CEST 2021
FEE : aida08 => Options file size is 1014 Last changed Wed Apr 14 21:52:04 CEST 2021
FEE : aida09 => Options file size is 1014 Last changed Wed Apr 14 21:52:05 CEST 2021
FEE : aida10 => Options file size is 1014 Last changed Wed Apr 14 21:52:06 CEST 2021
FEE : aida11 => Options file size is 1014 Last changed Wed Apr 14 21:52:05 CEST 2021
FEE : aida12 => Options file size is 1025 Last changed Mon Apr 19 09:05:25 CEST 2021
21.20 DSSSD biases & leakage currents OK - attachment 1
FEE64 tempos OK - attachment 2
good event stats - attachment 3
Grafana DSSSD biases & leakage currents for most recent 7 days - attachment 4
23.35 System wide checks - no changes to above
DSSSD biases & leakage currents OK - attachment 5
FEE64 tempos OK - attachment 6
good event stats - attachment 7
23.46 DAQ continues file S460/R51_378 |
| Attachment 1: 201.png
|
|
| Attachment 2: Screenshot_2021-04-23_Temperature_and_status_scan_aidas-gsi(7).png
|
.png.png "Screenshot_2021-04-23_Temperature_and_status_scan_aidas-gsi(7).png")
|
| Attachment 3: Screenshot_2021-04-23_Statistics_aidas-gsi(7).png
|
.png.png "Screenshot_2021-04-23_Statistics_aidas-gsi(7).png")
|
| Attachment 4: Screenshot_2021-04-23_AIDA_Alerting_-_Grafana.png
|
|
| Attachment 5: 202.png
|
|
| Attachment 6: Screenshot_2021-04-23_Statistics_aidas-gsi(8).png
|
.png.png "Screenshot_2021-04-23_Statistics_aidas-gsi(8).png")
|
| Attachment 7: Screenshot_2021-04-23_Statistics_aidas-gsi(8).png
|
.png.png "Screenshot_2021-04-23_Statistics_aidas-gsi(8).png")
|
|
267
|
Fri Apr 23 15:13:28 2021 |
DSJ | 23 April 16.00-20.00 shift |
16:00 taking over from Muneerah
16:15 - temp, ucesb, database, stats etc. look ok.
16:41 - Nic restarting MBS
17:05 - temp, ucesb, database, stats etc. look ok.
17:43 - temp, ucesb, database, stats etc. look ok.
18.10 - system wide checks. All same as previous checks - eg elog 264 + 266
FEE64 module aida06 global clocks failed, 6
Clock status test result: Passed 11, Failed 1
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida12 failed
Calibration test result: Passed 9, Failed 3
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
Base Current Difference
aida06 fault 0x679f : 0x67a2 : 3
White Rabbit error counter test result: Passed 11, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Check FPGA Timestamp Error - HTML errors
Returned 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 19 7 4 7 4 3 2 4 2 3 6 : 36388
aida02 : 1 2 5 2 0 4 3 4 3 4 3 : 27044
aida03 : 28 4 4 4 4 4 2 2 3 3 6 : 36432
aida04 : 5 5 2 6 2 3 3 3 3 3 6 : 36892
aida05 : 24 4 7 3 3 3 2 2 3 3 6 : 36240
aida06 : 21 6 6 3 3 4 2 2 3 3 6 : 36356
aida07 : 15 7 4 3 3 4 2 2 3 3 6 : 36308
aida08 : 23 6 7 1 3 4 1 3 3 3 6 : 36572
aida09 : 1 8 2 1 1 2 4 2 3 3 6 : 36292
aida10 : 3 4 6 2 2 2 3 3 2 3 6 : 35660
aida11 : 15 7 4 1 2 3 2 4 2 3 6 : 36052
aida12 : 25 13 5 2 2 3 2 3 3 3 6 : 36700
18:45 - temp, ucesb, database, stats etc. look ok.
19:03 - temp, ucesb, database, stats etc. look ok.
19:26 - temp, ucesb, database, stats etc. look ok.
TapeData (df -h) - 7.2T 3.9T 2.9T 58% /media/1e121361-83d3-4825-b6ae-8700b07e0ca7
19:30 - beam being shared with R3B, implantation rates fluctuation by~ 50% back to normal 19:40
19:50 system checks - all as previous
FEE64 module aida06 global clocks failed, 6
Clock status test result: Passed 11, Failed 1
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida12 failed
Calibration test result: Passed 9, Failed 3
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
Base Current Difference
aida06 fault 0x679f : 0x67a2 : 3
White Rabbit error counter test result: Passed 11, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Check FPGA Timestamp Error - HTML errors
Returned 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 25 6 4 7 4 3 2 4 2 3 6 : 36404
aida02 : 20 7 5 1 0 4 2 3 2 3 4 : 27384
aida03 : 28 6 3 5 3 5 2 2 3 3 6 : 36528
aida04 : 19 8 4 6 2 4 4 2 3 3 6 : 36876
aida05 : 7 3 8 3 4 3 1 3 3 3 6 : 36500
aida06 : 25 8 4 3 3 4 2 3 3 3 6 : 36868
aida07 : 18 7 3 4 3 4 2 2 3 3 6 : 36336
aida08 : 16 10 6 2 3 3 1 3 3 3 6 : 36464
aida09 : 0 3 1 1 1 3 4 2 3 3 6 : 36360
aida10 : 8 1 10 0 2 2 2 4 2 3 6 : 35912
aida11 : 2 2 2 3 3 4 3 3 2 3 6 : 35928
aida12 : 12 7 4 1 3 3 2 3 3 3 6 : 36616
|
| Attachment 1: Screenshot_2021-04-23_Statistics_aidas-gsi(5).png
|
.png.png "Screenshot_2021-04-23_Statistics_aidas-gsi(5).png")
|
| Attachment 2: Screenshot_2021-04-23_Temperature_and_status_scan_aidas-gsi(5).png
|
.png.png "Screenshot_2021-04-23_Temperature_and_status_scan_aidas-gsi(5).png")
|
| Attachment 3: 555.png
|
|
| Attachment 4: Screenshot_2021-04-23_Temperature_and_status_scan_aidas-gsi(6).png
|
.png.png "Screenshot_2021-04-23_Temperature_and_status_scan_aidas-gsi(6).png")
|
| Attachment 5: Screenshot_2021-04-23_Statistics_aidas-gsi(6).png
|
.png.png "Screenshot_2021-04-23_Statistics_aidas-gsi(6).png")
|
| Attachment 6: 556.png
|
|
|
266
|
Fri Apr 23 13:13:46 2021 |
MA, TD | Friday 23 April 12.00-16.00 |
14.15 System wide checks
FEE64 module aida06 global clocks failed, 6
Clock status test result: Passed 11, Failed 1
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida12 failed
Calibration test result: Passed 9, Failed 3
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
Base Current Difference
aida06 fault 0x679f : 0x67a2 : 3
White Rabbit error counter test result: Passed 11, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
FPGA timestamp errors - reports 'server internal error'
14.16 DSSSD bias & leakage currents OK - attachment 1
FEE64 temps OK - attachment 2
good events stats - attachment 3
14.22 DAQ continues file S460/R51_205
beam intensity 2.2e+09/s
15:53 system check
Rates, Temperatures, Voltages are ok and attached 4,5,6
Clock check
FEE64 module aida06 global clocks failed, 6
Clock status test result: Passed 11, Failed 1
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable
ADC check
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida12 failed
Calibration test result: Passed 9, Failed 3
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
White Rabbit
Base Current Difference
aida06 fault 0x679f : 0x67a2 : 3
White Rabbit error counter test result: Passed 11, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
FPGA
Loge page error!
Memory check
Returned 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 23 6 2 1 2 3 2 4 2 3 6 : 36044
aida02 : 15 3 2 2 0 3 2 3 2 3 4 : 27188
aida03 : 29 4 0 3 2 4 2 2 3 3 6 : 36212
aida04 : 4 6 4 4 2 3 3 3 3 3 6 : 36864
aida05 : 19 5 4 2 3 3 1 3 3 3 6 : 36404
aida06 : 24 9 5 2 3 3 3 2 3 3 6 : 36472
aida07 : 20 4 9 0 2 3 2 2 3 3 6 : 36096
aida08 : 22 9 4 3 1 3 1 3 3 3 6 : 36352
aida09 : 20 7 3 2 2 3 3 2 3 3 6 : 36344
aida10 : 25 9 9 5 2 3 2 3 3 3 6 : 36828
aida11 : 24 11 2 0 1 3 1 3 3 3 6 : 36248
aida12 : 26 5 4 2 1 3 2 2 3 3 6 : 36048 |
| Attachment 1: 200.png
|
|
| Attachment 2: Screenshot_2021-04-23_Temperature_and_status_scan_aidas-gsi(4).png
|
.png.png "Screenshot_2021-04-23_Temperature_and_status_scan_aidas-gsi(4).png")
|
| Attachment 3: Screenshot_2021-04-23_Statistics_aidas-gsi(4).png
|
.png.png "Screenshot_2021-04-23_Statistics_aidas-gsi(4).png")
|
| Attachment 4: 210423_1550_Voltages.png
|
|
| Attachment 5: 210423_1550_Stats.png
|
|
| Attachment 6: 210423_1550_Temp.png
|
|
|
265
|
Fri Apr 23 12:12:08 2021 |
TD | aida02 restart - 03.13UTC+2 Friday 23 April |
Overnight aida02 crashed and restarted itself. A power cycle of all FEE64s was required to recover.
The system log appears to show the aida02 embedded processor crashed (note log times are UTC+1
for some reason).
Apr 21 00:41:21 aida02 kernel: xaida: open:
Apr 23 02:09:41 aida02 kernel: Unable to handle kernel paging request for data at address 0x00200200
Apr 23 02:09:41 aida02 kernel: Faulting instruction address: 0xc0091dd8
Apr 23 02:09:41 aida02 kernel: Oops: Kernel access of bad area, sig: 11 [#1]
Apr 23 02:09:41 aida02 kernel: PREEMPT Xilinx Virtex440
Apr 23 02:09:41 aida02 kernel: Modules linked in: aidamem xdriver xh_spidev_register
Apr 23 02:09:42 aida02 kernel: NIP: c0091dd8 LR: c0091e40 CTR: 00000000
Apr 23 02:09:42 aida02 kernel: REGS: c6835e60 TRAP: 0300 Not tainted (2.6.31)
Apr 23 02:09:42 aida02 kernel: MSR: 00021000 <ME,CE> CR: 24000088 XER: 00000000
Apr 23 02:09:42 aida02 kernel: DEAR: 00200200, ESR: 00800000
Apr 23 02:09:42 aida02 kernel: TASK = c6824880[5] 'events/0' THREAD: c6834000
Apr 23 02:09:42 aida02 kernel: GPR00: c0091e40 c6835f10 c6824880 00000001 00000003 c038c3a8 00000000 c0498000
Apr 23 02:09:42 aida02 kernel: GPR08: 00000008 00100100 00000051 00200200 00485e6c 00005aa8 4805de90 00000001
Apr 23 02:09:42 aida02 kernel: GPR16: 00000000 00000000 4801d000 00000003 c031aef4 c02fb040 c02fb008 00000002
Apr 23 02:09:42 aida02 kernel: GPR24: c6834000 00100100 00200200 c680ae00 00000001 c680daf0 c680dae0 c6940020
Apr 23 02:09:42 aida02 kernel: NIP [c0091dd8] drain_freelist+0x54/0x110
Apr 23 02:09:42 aida02 kernel: LR [c0091e40] drain_freelist+0xbc/0x110
Apr 23 02:09:42 aida02 kernel: Call Trace:
Apr 23 02:09:42 aida02 kernel: [c6835f10] [c0091e40] drain_freelist+0xbc/0x110 (unreliable)
Apr 23 02:09:42 aida02 kernel: [c6835f40] [c0093158] cache_reap+0x104/0x138
Apr 23 02:09:42 aida02 kernel: [c6835f60] [c0048908] worker_thread+0x140/0x204
Apr 23 02:09:42 aida02 kernel: [c6835fc0] [c004cfc8] kthread+0x78/0x7c
Apr 23 02:09:42 aida02 kernel: [c6835ff0] [c000e140] kernel_thread+0x4c/0x68
Apr 23 02:09:42 aida02 kernel: Instruction dump:
Apr 23 02:09:43 aida02 kernel: 3ba40010 7f80e800 419e00d4 3f200010 3f400020 54380024 63390100 635a0200
Apr 23 02:09:43 aida02 kernel: 3b800000 48000068 813f0000 817f0004 <912b0000> 91690004 933f0000 935f0004
Apr 23 02:09:43 aida02 kernel: ---[ end trace 6baa5b82e4ef4e92 ]---
Apr 23 02:09:43 aida02 kernel: note: events/0[5] exited with preempt_count 1
Apr 23 02:13:39 aida02 syslogd 1.4.2: restart.
Apr 23 02:13:39 aida02 kernel: klogd 1.4.2, log source = /proc/kmsg started.
Apr 23 02:13:39 aida02 kernel: Using Xilinx Virtex440 machine description
Apr 23 02:13:39 aida02 kernel: Linux version 2.6.31 (nf@nnlxb.dl.ac.uk) (gcc version 4.2.2) #34 PREEMPT Tue Nov 15 15:57:04 GMT 2011 |
|
264
|
Fri Apr 23 07:09:57 2021 |
LJW | Friday 23rd April 08:00-12:00 |
08:30
System Checks
Clock Status error:
FEE64 module aida06 global clocks failed, 6
Clock status test result: Passed 11, Failed 1
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable
ADC Calibration error:
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida12 failed
Calibration test result: Passed 9, Failed 3
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
White Rabbit Decoder Status error:
Base Current Difference
aida06 fault 0x679f : 0x67a2 : 3
White Rabbit error counter test result: Passed 11, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
FPGA Timestamp error:
Showed larger error message begining 'Got the error Server Internal Error
while trying to obtain /AIDA/Check/Check.tml.'
Statistics - See attacnment 1
Temperatures - See attachment 2
Bias & Leakage Currents - See attachment 3
10:02
System Checks:
Clock Status Error:
FEE64 module aida06 global clocks failed, 6
Clock status test result: Passed 11, Failed 1
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable
ADC Calibration Error:
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida12 failed
Calibration test result: Passed 9, Failed 3
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
White Rabbit Decoder Status Error:
Base Current Difference
aida06 fault 0x679f : 0x67a2 : 3
White Rabbit error counter test result: Passed 11, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
FPGA Timestamp error:
Showed larger error message begining 'Got the error Server Internal Error
while trying to obtain /AIDA/Check/Check.tml.'
Statistics - See attacnment 4
Temperatures - See attachment 5
Bias & Leakage Currents - See attachment 6
10:45
System Checks
Clock Status Error:
FEE64 module aida06 global clocks failed, 6
Clock status test result: Passed 11, Failed 1
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable
ADC Calibration check Error:
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida12 failed
Calibration test result: Passed 9, Failed 3
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
White Rabbit Decoder Status Error:
Base Current Difference
aida06 fault 0x679f : 0x67a2 : 3
White Rabbit error counter test result: Passed 11, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
FPGA Timestamp error:
Showed larger error message begining 'Got the error Server Internal Error
while trying to obtain /AIDA/Check/Check.tml.'
Statistics - See attacnment 7
Temperatures - See attachment 8
Bias & Leakage Currents - See attachment 9
|
| Attachment 1: 2304211_1.png
|
|
| Attachment 2: 230421_2.png
|
|
| Attachment 3: 230421_3.png
|
|
| Attachment 4: 230421_4.png
|
|
| Attachment 5: 230421_5.png
|
|
| Attachment 6: 230421_6.png
|
|
| Attachment 7: 230421_7.png
|
|
| Attachment 8: 230421_8.png
|
|
| Attachment 9: 230421_9.png
|
|
|
263
|
Thu Apr 22 23:17:46 2021 |
DJ | Friday 23rd April 00:00-08:00 |
Took over from TD ... All is well.
00:45 System Wide Checks
Clock: 12 passed, 0 failed.
ADC Calibration: Passed 12, 0 failed.
White Rabbit Decoder: Passed 4,failed 8. - See screenshot.
FPGA Passed 11, failed 1. - See screenshot.
FEE64 Linux: See screenshot.
-----
02:49 run closed. Checking beam.
03:00 File 55 opened.
03:13:23 DAQ crashed due to AIDA02 rebooting. TD restarted DAQ. Ready at approx 04:00.
File 56 opened at 4:24.
---------
04:30 System Wide Checks (See screenshots)
White Rabbit Decoder: Passed 12, failed 0.
FPGA: page not loading properly.
----
05:37 System wide checks. See screenshots
Clock: Status: 11 passed. 1 fail.
ADC calibration: Passed 9, failed 3.
White Rabbit decoder: Passed 12, failed 0.
-----
07:32 System wide checks (See screenshots).
|
| Attachment 1: Screenshot_2021-04-23_Statistics_aidas-gsi.png
|
|
| Attachment 2: Screenshot_2021-04-23_Temperature_and_status_scan_aidas-gsi.png
|
|
| Attachment 3: Screenshot-CAEN_HV_Supply.png
|
|
| Attachment 4: Screenshot_2021-04-23_System_wide_Checks_aidas-gsi.png
|
|
| Attachment 5: Screenshot_2021-04-23_System_wide_Checks_aidas-gsi(1).png
|
.png.png "Screenshot_2021-04-23_System_wide_Checks_aidas-gsi(1).png")
|
| Attachment 6: Screenshot_2021-04-23_System_wide_Checks_aidas-gsi(2).png
|
.png.png "Screenshot_2021-04-23_System_wide_Checks_aidas-gsi(2).png")
|
| Attachment 7: Screenshot_2021-04-23_Statistics_aidas-gsi(1).png
|
.png.png "Screenshot_2021-04-23_Statistics_aidas-gsi(1).png")
|
| Attachment 8: Screenshot_2021-04-23_Temperature_and_status_scan_aidas-gsi(1).png
|
.png.png "Screenshot_2021-04-23_Temperature_and_status_scan_aidas-gsi(1).png")
|
| Attachment 9: 60.png
|
|
| Attachment 10: Screenshot_2021-04-23_System_wide_Checks_aidas-gsi(3).png
|
.png.png "Screenshot_2021-04-23_System_wide_Checks_aidas-gsi(3).png")
|
| Attachment 11: Screenshot_2021-04-23_System_wide_Checks_aidas-gsi(4).png
|
.png.png "Screenshot_2021-04-23_System_wide_Checks_aidas-gsi(4).png")
|
| Attachment 12: Screenshot_2021-04-23_System_wide_Checks_aidas-gsi(5).png
|
.png.png "Screenshot_2021-04-23_System_wide_Checks_aidas-gsi(5).png")
|
| Attachment 13: Screenshot_2021-04-23_Spectrum_Browser_aidas-gsi.png
|
|
| Attachment 14: Screenshot_2021-04-23_Spectrum_Browser_aidas-gsi.png
|
|
| Attachment 15: Screenshot_2021-04-23_Temperature_and_status_scan_aidas-gsi(2).png
|
.png.png "Screenshot_2021-04-23_Temperature_and_status_scan_aidas-gsi(2).png")
|
| Attachment 16: Screenshot_2021-04-23_Statistics_aidas-gsi(2).png
|
.png.png "Screenshot_2021-04-23_Statistics_aidas-gsi(2).png")
|
| Attachment 17: 61.png
|
|
| Attachment 18: Screenshot_2021-04-23_System_wide_Checks_aidas-gsi(6).png
|
.png.png "Screenshot_2021-04-23_System_wide_Checks_aidas-gsi(6).png")
|
| Attachment 19: Screenshot_2021-04-23_System_wide_Checks_aidas-gsi(7).png
|
.png.png "Screenshot_2021-04-23_System_wide_Checks_aidas-gsi(7).png")
|
| Attachment 20: Screenshot_2021-04-23_System_wide_Checks_aidas-gsi(8).png
|
.png.png "Screenshot_2021-04-23_System_wide_Checks_aidas-gsi(8).png")
|
| Attachment 21: Screenshot_2021-04-23_Temperature_and_status_scan_aidas-gsi(3).png
|
.png.png "Screenshot_2021-04-23_Temperature_and_status_scan_aidas-gsi(3).png")
|
| Attachment 22: Screenshot_2021-04-23_Statistics_aidas-gsi(3).png
|
.png.png "Screenshot_2021-04-23_Statistics_aidas-gsi(3).png")
|
| Attachment 23: 63.png
|
|
| Attachment 24: Screenshot_2021-04-23_System_wide_Checks_aidas-gsi(9).png
|
.png.png "Screenshot_2021-04-23_System_wide_Checks_aidas-gsi(9).png")
|
| Attachment 25: Screenshot_2021-04-23_System_wide_Checks_aidas-gsi(10).png
|
.png.png "Screenshot_2021-04-23_System_wide_Checks_aidas-gsi(10).png")
|
| Attachment 26: Screenshot_2021-04-23_System_wide_Checks_aidas-gsi(11).png
|
.png.png "Screenshot_2021-04-23_System_wide_Checks_aidas-gsi(11).png")
|
| Attachment 27: Screenshot_2021-04-23_System_wide_Checks_aidas-gsi(12).png
|
.png.png "Screenshot_2021-04-23_System_wide_Checks_aidas-gsi(12).png")
|
|
262
|
Thu Apr 22 15:18:20 2021 |
TD | Thursday 22 April 16.00-00.00 |
16.20 zero all histograms, statistics & merger statistics
most recent merger error messages
MERGE Data Link (29984): bad timestamp 6 3 0xc1ac7f1c 0x0dce7672 0x00002cda5dce7672 0x16782cda5dce7672 0x16782cda5dcf2032
MERGE Data Link (29984): bad timestamp 6 3 0xc18b7e62 0x0dce7e42 0x00002cda5dce7e42 0x16782cda5dce7e42 0x16782cda5dcf2032
MERGE Data Link (29984): bad timestamp 6 3 0xc1957ebc 0x0dce7e42 0x00002cda5dce7e42 0x16782cda5dce7e42 0x16782cda5dcf2032
MERGE Data Link (29984): bad timestamp 6 3 0xc1b67ecb 0x0dcf0ae2 0x00002cda5dcf0ae2 0x16782cda5dcf0ae2 0x16782cda5dcf2032
MERGE Data Link (29984): bad timestamp 6 3 0xc1957ea3 0x0dcf12b2 0x00002cda5dcf12b2 0x16782cda5dcf12b2 0x16782cda5dcf2032
MERGE Data Link (29984): bad timestamp 6 3 0xc1867fba 0x0b2ee142 0x00002dc9eb2ee142 0x16782dc9eb2ee142 0x16782dc9fad22532
MERGE Data Link (29984): bad timestamp 6 3 0x86800889 0x0ad28158 0x00002dc9eb2ee142 0x16782dc9ead28158 0x16782dc9fad22532
MERGE Data Link (29984): bad timestamp 6 3 0xc1be7e6e 0x0b2ef0e2 0x00002dc9eb2ef0e2 0x16782dc9eb2ef0e2 0x16782dc9fad2c172
MERGE Data Link (29984): bad timestamp 6 3 0xc1b08006 0x0b2f5e42 0x00002dc9eb2f5e42 0x16782dc9eb2f5e42 0x16782dc9fad34502
MERGE Data Link (29984): bad timestamp 6 3 0xc1807de7 0x0ad3c342 0x00002dc9eb2f6de2 0x16782dc9ead3c342 0x16782dc9fad34502
MERGE Statistics counters now cleared
16.21 merger server terminal reset & clear
TapeData -> /media/ThirdDRive/TapeData
[npg@aidas-gsi bin64]$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_aidasgsi-lv_root
50G 13G 34G 28% /
tmpfs 7.9G 559M 7.3G 7% /dev/shm
/dev/sda2 477M 42M 410M 10% /boot
/dev/sda1 200M 264K 200M 1% /boot/efi
/dev/mapper/vg_aidasgsi-lv_home
401G 79G 302G 21% /home
/dev/mapper/vg_aidas2-lv_home
7.2T 3.9T 2.9T 58% /media/1e121361-83d3-4825-b6ae-8700b07e0ca7
/dev/sdd1 7.2T 6.3T 605G 92% /media/SecondDrive
/dev/sdc1 7.2T 3.6T 3.3T 52% /media/ThirdDrive
at current rate (10.5Mb/s) we have 3.6d remaining
17.40 *all* system wide checks OK *except* FPGA timestamp errors
Base Current Difference
aida12 fault 0x0 : 0x51 : 81
FPGA Timestamp error counter test result: Passed 11, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same
FEE : aida01 => Options file size is 1026 Last changed Wed Apr 21 01:51:54 CEST 2021
FEE : aida02 => Options file size is 1014 Last changed Fri Apr 16 00:56:20 CEST 2021
FEE : aida03 => Options file size is 1014 Last changed Wed Apr 14 21:52:04 CEST 2021
FEE : aida04 => Options file size is 1025 Last changed Sat Apr 17 06:07:36 CEST 2021
FEE : aida05 => Options file size is 1025 Last changed Fri Apr 16 00:53:25 CEST 2021
FEE : aida06 => Options file size is 1014 Last changed Wed Apr 14 21:52:04 CEST 2021
FEE : aida07 => Options file size is 1014 Last changed Wed Apr 14 21:52:04 CEST 2021
FEE : aida08 => Options file size is 1014 Last changed Wed Apr 14 21:52:04 CEST 2021
FEE : aida09 => Options file size is 1014 Last changed Wed Apr 14 21:52:05 CEST 2021
FEE : aida10 => Options file size is 1014 Last changed Wed Apr 14 21:52:06 CEST 2021
FEE : aida11 => Options file size is 1014 Last changed Wed Apr 14 21:52:05 CEST 2021
FEE : aida12 => Options file size is 1025 Last changed Mon Apr 19 09:05:25 CEST 2021
17.43 FEE64 temps OK - attachment 1
statistics - attachments 2-8
per FEE64 rate spectra - attachments 9 & 10
1.8.L spectra - attachments 11-14
1.8.H spectra - attachments 15-16
1.8.W spectra 20us FSR - attachments 17-18
1.8.W spectra 200us, 2ms & 20ms FSR - attachments 19-21, 30-32
merger, tape server & merger statistics 19-25
ucesb - attachment 26
Grafana DSSD biases & leakage currents - attachment 27
taperserver & tapeserver/merger server terminals - no timewarp errors since c. 15.25 - attachment 28
DSSSD biases & leakage currents - attachment 29
22.17 *all* system wide checks OK *except*
Base Current Difference
aida01 fault 0x3612 : 0x3613 : 1
aida02 fault 0x523c : 0x523d : 1
aida03 fault 0xc5cd : 0xc5ce : 1
aida04 fault 0x14f1 : 0x14f2 : 1
aida05 fault 0x9a4e : 0x9a4f : 1
aida06 fault 0xfdd2 : 0xfdd3 : 1
aida07 fault 0x4a0 : 0x4a1 : 1
aida08 fault 0xc796 : 0xc797 : 1
White Rabbit error counter test result: Passed 4, Failed 8
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Base Current Difference
aida12 fault 0x0 : 0x57 : 87
FPGA Timestamp error counter test result: Passed 11, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same
FEE : aida01 => Options file size is 1026 Last changed Wed Apr 21 01:51:54 CEST 2021
FEE : aida02 => Options file size is 1014 Last changed Fri Apr 16 00:56:20 CEST 2021
FEE : aida03 => Options file size is 1014 Last changed Wed Apr 14 21:52:04 CEST 2021
FEE : aida04 => Options file size is 1025 Last changed Sat Apr 17 06:07:36 CEST 2021
FEE : aida05 => Options file size is 1025 Last changed Fri Apr 16 00:53:25 CEST 2021
FEE : aida06 => Options file size is 1014 Last changed Wed Apr 14 21:52:04 CEST 2021
FEE : aida07 => Options file size is 1014 Last changed Wed Apr 14 21:52:04 CEST 2021
FEE : aida08 => Options file size is 1014 Last changed Wed Apr 14 21:52:04 CEST 2021
FEE : aida09 => Options file size is 1014 Last changed Wed Apr 14 21:52:05 CEST 2021
FEE : aida10 => Options file size is 1014 Last changed Wed Apr 14 21:52:06 CEST 2021
FEE : aida11 => Options file size is 1014 Last changed Wed Apr 14 21:52:05 CEST 2021
FEE : aida12 => Options file size is 1025 Last changed Mon Apr 19 09:05:25 CEST 2021
Most recent merger errors
MERGE Data Link (29979): bad timestamp 1 3 0xc05080ef 0x0c88aaa2 0x000047aa2c88aaa2 0x167847aa2c88aaa2 0x167847aa2c8a34d2
MERGE Data Link (29979): bad timestamp 1 3 0xc06f80d7 0x0c88aaa2 0x000047aa2c88aaa2 0x167847aa2c88aaa2 0x167847aa2c8a34d2
MERGE Data Link (29979): bad timestamp 1 3 0xc0778222 0x0c88b272 0x000047aa2c88b272 0x167847aa2c88b272 0x167847aa2c8a34d2
MERGE Data Link (29979): bad timestamp 1 3 0xc0598098 0x0c88e152 0x000047aa2c88e152 0x167847aa2c88e152 0x167847aa2c8a34d2
MERGE Data Link (29979): bad timestamp 1 3 0xc042815b 0x0c894eb2 0x000047aa2c894eb2 0x167847aa2c894eb2 0x167847aa2c8a34d2
MERGE Data Link (29979): bad timestamp 1 3 0xc05980eb 0x0c899502 0x000047aa2c899502 0x167847aa2c899502 0x167847aa2c8a34d2
MERGE Data Link (29979): bad timestamp 1 3 0xc075813c 0x0c89e322 0x000047aa2c89e322 0x167847aa2c89e322 0x167847aa2c8a34d2
MERGE Data Link (29979): bad timestamp 1 3 0xc04280d5 0x0c89f2c2 0x000047aa2c89f2c2 0x167847aa2c89f2c2 0x167847aa2c8a34d2
MERGE Actor (29995): Working with 0 from 12 data sources
MERGE Actor (29995): Working with 0 from 12 data sources
22.20 DSSSD biases & leakage currents - attachment 33
FEE64 temps OK - attachment 34
statistics - attachments 35
merger statistics 36
|
| Attachment 1: Screenshot_2021-04-22_Temperature_and_status_scan_aidas-gsi(4).png
|
.png.png "Screenshot_2021-04-22_Temperature_and_status_scan_aidas-gsi(4).png")
|
| Attachment 2: Screenshot_2021-04-22_Statistics_aidas-gsi(4).png
|
.png.png "Screenshot_2021-04-22_Statistics_aidas-gsi(4).png")
|
| Attachment 3: Screenshot_2021-04-22_Statistics_aidas-gsi(5).png
|
.png.png "Screenshot_2021-04-22_Statistics_aidas-gsi(5).png")
|
| Attachment 4: Screenshot_2021-04-22_Statistics_aidas-gsi(6).png
|
.png.png "Screenshot_2021-04-22_Statistics_aidas-gsi(6).png")
|
| Attachment 5: Screenshot_2021-04-22_Statistics_aidas-gsi(7).png
|
.png.png "Screenshot_2021-04-22_Statistics_aidas-gsi(7).png")
|
| Attachment 6: Screenshot_2021-04-22_Statistics_aidas-gsi(8).png
|
.png.png "Screenshot_2021-04-22_Statistics_aidas-gsi(8).png")
|
| Attachment 7: Screenshot_2021-04-22_Statistics_aidas-gsi(9).png
|
.png.png "Screenshot_2021-04-22_Statistics_aidas-gsi(9).png")
|
| Attachment 8: Screenshot_2021-04-22_Statistics_aidas-gsi(10).png
|
.png.png "Screenshot_2021-04-22_Statistics_aidas-gsi(10).png")
|
| Attachment 9: Screenshot_2021-04-22_Spectrum_Browser_aidas-gsi.png
|
|
| Attachment 10: Screenshot_2021-04-22_Spectrum_Browser_aidas-gsi(1).png
|
.png.png "Screenshot_2021-04-22_Spectrum_Browser_aidas-gsi(1).png")
|
| Attachment 11: Screenshot_2021-04-22_Spectrum_Browser_aidas-gsi(2).png
|
.png.png "Screenshot_2021-04-22_Spectrum_Browser_aidas-gsi(2).png")
|
| Attachment 12: Screenshot_2021-04-22_Spectrum_Browser_aidas-gsi(3).png
|
.png.png "Screenshot_2021-04-22_Spectrum_Browser_aidas-gsi(3).png")
|
| Attachment 13: Screenshot_2021-04-22_Spectrum_Browser_aidas-gsi(4).png
|
.png.png "Screenshot_2021-04-22_Spectrum_Browser_aidas-gsi(4).png")
|
| Attachment 14: Screenshot_2021-04-22_Spectrum_Browser_aidas-gsi(5).png
|
.png.png "Screenshot_2021-04-22_Spectrum_Browser_aidas-gsi(5).png")
|
| Attachment 15: Screenshot_2021-04-22_Spectrum_Browser_aidas-gsi(6).png
|
.png.png "Screenshot_2021-04-22_Spectrum_Browser_aidas-gsi(6).png")
|
| Attachment 16: Screenshot_2021-04-22_Spectrum_Browser_aidas-gsi(7).png
|
.png.png "Screenshot_2021-04-22_Spectrum_Browser_aidas-gsi(7).png")
|
| Attachment 17: Screenshot_2021-04-22_Spectrum_Browser_aidas-gsi(8).png
|
.png.png "Screenshot_2021-04-22_Spectrum_Browser_aidas-gsi(8).png")
|
| Attachment 18: Screenshot_2021-04-22_Spectrum_Browser_aidas-gsi(9).png
|
.png.png "Screenshot_2021-04-22_Spectrum_Browser_aidas-gsi(9).png")
|
| Attachment 19: Screenshot_2021-04-22_Spectrum_Browser_aidas-gsi(10).png
|
.png.png "Screenshot_2021-04-22_Spectrum_Browser_aidas-gsi(10).png")
|
| Attachment 20: Screenshot_2021-04-22_Spectrum_Browser_aidas-gsi(11).png
|
.png.png "Screenshot_2021-04-22_Spectrum_Browser_aidas-gsi(11).png")
|
| Attachment 21: Screenshot_2021-04-22_Spectrum_Browser_aidas-gsi(12).png
|
.png.png "Screenshot_2021-04-22_Spectrum_Browser_aidas-gsi(12).png")
|
| Attachment 22: Screenshot_2021-04-22_NewMerger_Control_aidas-gsi.png
|
|
| Attachment 23: Screenshot_2021-04-22_Tape_Service_(Expert)_aidas-gsi.png
|
_aidas-gsi.png.png "Screenshot_2021-04-22_Tape_Service_(Expert)_aidas-gsi.png")
|
| Attachment 24: Screenshot_2021-04-22_NewMerger_Statistics_aidas-gsi.png
|
|
| Attachment 25: Screenshot_2021-04-22_NewMerger_Statistics_aidas-gsi(1).png
|
.png.png "Screenshot_2021-04-22_NewMerger_Statistics_aidas-gsi(1).png")
|
| Attachment 26: Screenshot_2021-04-22_ucesb.png
|
|
| Attachment 27: Screenshot_2021-04-22_AIDA_Alerting_-_Grafana(1).png
|
.png.png "Screenshot_2021-04-22_AIDA_Alerting_-_Grafana(1).png")
|
| Attachment 28: 101.png
|
|
| Attachment 29: 100.png
|
|
| Attachment 30: Screenshot_2021-04-22_Spectrum_Browser_aidas-gsi.png
|
|
| Attachment 31: Screenshot_2021-04-22_Spectrum_Browser_aidas-gsi(1).png
|
.png.png "Screenshot_2021-04-22_Spectrum_Browser_aidas-gsi(1).png")
|
| Attachment 32: Screenshot_2021-04-22_Spectrum_Browser_aidas-gsi(2).png
|
.png.png "Screenshot_2021-04-22_Spectrum_Browser_aidas-gsi(2).png")
|
| Attachment 33: 102.png
|
|
| Attachment 34: Screenshot_2021-04-22_Statistics_aidas-gsi.png
|
|
| Attachment 35: Screenshot_2021-04-22_Temperature_and_status_scan_aidas-gsi.png
|
|
| Attachment 36: Screenshot_2021-04-22_NewMerger_Statistics_aidas-gsi.png
|
|
|
261
|
Thu Apr 22 11:03:57 2021 |
JS | Thursday 22nd April: 12:00-16:00 |
12:00 Taking over from Liam, all is well.
12:10 Beam dropped out for a minute, implantation rate is down a bit, peaking around 300 Hz
12:19 Beam stopped (from GSI log "Run stopped because of a call from HKR stating they have a problematic device which they
need to check and will need apprx. 15 min.")
12:30
ucesb ok (no beam)
DB No faults
Stats ok.
12:34 In order to fix some ongoing FRS problem no beam expected for at least an hour.
13:00
We are taking intermittent beam while they work on the FRS
DB No faults
ucesb ok elog:261/1
Current ok elog:261/2
Stats ok elog:261/3
Temps ok elog:261/4
Clock status test result: Passed 12, Failed 0
Calibration test result: Passed 12, Failed 0
White Rabbit error counter test result: Passed 12, Failed 0
Base Current Difference
aida12 fault 0x0 : 0x46 : 70
FPGA Timestamp error counter test result: Passed 11, Failed 1
Returned 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 15 6 5 3 2 2 2 3 2 3 7 : 39580
aida02 : 21 7 4 2 2 3 2 3 2 3 7 : 39692
aida03 : 20 10 3 3 0 3 2 3 2 3 7 : 39600
aida04 : 8 7 4 5 2 4 3 2 2 3 7 : 39608
aida05 : 17 14 1 2 3 3 4 2 2 3 7 : 39748
aida06 : 17 10 2 3 2 2 4 2 2 3 7 : 39572
aida07 : 27 10 8 7 2 3 3 2 2 3 7 : 39708
aida08 : 9 6 2 3 2 1 3 3 2 3 7 : 39636
aida09 : 1 2 1 4 0 3 3 2 2 3 7 : 39204
aida10 : 5 8 4 4 2 2 1 3 2 3 7 : 39316
aida11 : 6 4 5 3 2 3 2 2 2 3 7 : 39144
aida12 : 15 5 8 1 0 4 1 3 2 3 7 : 39428
13:32
ucesb ok
DB No faults
Stats ok
14:01
ucesb ok
DB No faults
Stats ok
14.30 analysis of file /TapeData/S460/R50_528 - attachment 5
cat /var/log/messages | grep /MIDAS - attachments 6 & 7
Grafana DSSSD bias & leakage currents for last 7 days - attachment 8
14:55
DB No faults
ucesb ok
Current ok elog:261/9
Stats ok elog:261/10
Temps ok elog:261/11
FPGA Timestamp Check Error:
Base Current Difference
aida12 fault 0x0 : 0x4d : 77
FPGA Timestamp error counter test result: Passed 11, Failed 1
15:30
ucesb ok
DB No faults
Stats ok
15:59
ucesb ok elog:261/12
DB No faults
Stats ok |
| Attachment 1: 51.png
|
|
| Attachment 2: 52.png
|
|
| Attachment 3: Screenshot_2021-04-22_Statistics_aidas-gsi(2).png
|
.png.png "Screenshot_2021-04-22_Statistics_aidas-gsi(2).png")
|
| Attachment 4: Screenshot_2021-04-22_Temperature_and_status_scan_aidas-gsi(2).png
|
.png.png "Screenshot_2021-04-22_Temperature_and_status_scan_aidas-gsi(2).png")
|
| Attachment 5: R50_528
|
*** TDR format 3.3.0 analyser - TD - January 2019
*** ERROR: READ I/O error: 5002
blocks: 32000
ADC data format: 256073749 ( 1415213.2 Hz)
Other data format: 5846251 ( 32309.8 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 32674 ( 180.6 Hz)
WR48-63: 32674 ( 180.6 Hz)
FEE64 disc: 1081787 ( 5978.6 Hz)
MBS info: 4699116 ( 25970.1 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 342915 ( 1895.1 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
WR48-63: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
*** Timestamp elapsed time: 180.944 s
FEE elapsed dead time(s) elapsed idle time(s)
1 0.000 53.074
2 0.000 66.167
3 0.000 47.044
4 0.000 162.945
5 0.000 122.260
6 0.000 104.653
7 0.000 108.241
8 0.000 75.998
9 0.000 7.078
10 0.000 27.522
11 0.000 39.393
12 0.000 0.000
13 0.000 0.000
14 0.000 0.000
15 0.000 0.000
16 0.000 0.000
17 0.000 0.000
18 0.000 0.000
19 0.000 0.000
20 0.000 0.000
21 0.000 0.000
22 0.000 0.000
23 0.000 0.000
24 0.000 0.000
25 0.000 0.000
26 0.000 0.000
27 0.000 0.000
28 0.000 0.000
29 0.000 0.000
30 0.000 0.000
31 0.000 0.000
32 0.000 0.000
*** Statistics
FEE ADC Data Other Data Sample Undefined Pause Resume SYNC100 WR48-63 Disc MBS Other HEC Data
0 18754736 101009 0 0 0 0 2455 2455 95016 1083 0 35321
1 22221860 61355 0 0 0 0 2741 2741 55873 0 0 20378
2 16643754 1970924 0 0 0 0 2367 2367 197444 1768746 0 33880
3 21495935 1857330 0 0 0 0 2886 2886 82623 1768935 0 33490
4 3525133 403992 0 0 0 0 456 456 99315 303765 0 35967
5 10189380 355501 0 0 0 0 1262 1262 56604 296373 0 20229
6 12085720 387518 0 0 0 0 1592 1592 108226 276108 0 33721
7 11376753 370078 0 0 0 0 1497 1497 82978 284106 0 32056
8 43639711 66166 0 0 0 0 5406 5406 55354 0 0 18854
9 42033424 67528 0 0 0 0 5337 5337 56854 0 0 18886
10 29147293 115191 0 0 0 0 3615 3615 107961 0 0 31625
11 24960050 89659 0 0 0 0 3060 3060 83539 0 0 28508
12 0 0 0 0 0 0 0 0 0 0 0 0
13 0 0 0 0 0 0 0 0 0 0 0 0
14 0 0 0 0 0 0 0 0 0 0 0 0
15 0 0 0 0 0 0 0 0 0 0 0 0
16 0 0 0 0 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0 0 0 0
*** Timewarps
FEE ADC Pause Resume SYNC100 WR48-63 Disc MBS Undefined Samples
0 0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 0 0
2 0 0 0 0 0 0 0 0 0
3 0 0 0 0 0 0 0 0 0
4 0 0 0 0 0 0 0 0 0
5 0 0 0 0 0 0 0 0 0
6 0 0 0 0 0 0 0 0 0
7 0 0 0 0 0 0 0 0 0
8 0 0 0 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0 0
11 0 0 0 0 0 0 0 0 0
12 0 0 0 0 0 0 0 0 0
13 0 0 0 0 0 0 0 0 0
14 0 0 0 0 0 0 0 0 0
15 0 0 0 0 0 0 0 0 0
16 0 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0
*** Program elapsed time:52321.719s ( 0.612 blocks/s, 0.038 Mb/s)
|
| Attachment 6: log2
|
Apr 13 17:43:23 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.7:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 13 17:43:31 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.8:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 13 17:43:32 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.6:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 13 17:43:34 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.4:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 13 17:43:38 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.1:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 13 17:43:39 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.3:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 13 17:43:40 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.5:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 13 17:43:42 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.2:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 13 17:44:56 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.9:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 13 17:44:56 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.11:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 13 17:44:58 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.10:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 13 17:45:01 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.12:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 16 00:58:32 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.8:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 16 00:58:32 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.6:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 16 00:58:34 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.5:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 16 00:58:34 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.4:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 16 00:58:34 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.3:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 16 00:58:35 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.1:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 16 00:58:42 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.2:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 16 00:58:50 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.7:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 17 06:10:30 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.8:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 17 06:10:31 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.3:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 17 06:10:35 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.5:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 17 06:10:38 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.6:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 17 06:10:38 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.7:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 17 06:10:39 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.4:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 17 06:10:39 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.1:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 17 06:10:43 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.2:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 17 06:11:08 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.9:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 17 06:11:09 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.11:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 17 06:11:10 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.12:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 17 06:11:12 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.10:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 17 06:33:31 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.8:857 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
|
| Attachment 7: log1
|
Apr 19 08:33:03 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.12:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 19 09:13:33 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.3:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 19 09:13:36 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.8:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 19 09:13:37 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.6:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 19 09:13:38 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.1:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 19 09:13:40 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.4:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 19 09:13:42 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.5:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 19 09:13:46 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.2:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 19 09:13:56 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.7:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 19 09:14:48 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.11:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 19 09:14:48 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.9:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 19 09:14:50 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.10:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 19 09:14:51 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.12:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 19 09:29:49 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.5:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 19 09:29:49 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.6:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 19 09:29:51 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.3:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 19 09:29:52 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.4:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 19 09:29:53 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.1:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 19 09:29:53 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.7:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 19 09:30:00 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.2:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 19 09:31:16 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.9:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 19 09:31:17 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.12:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 19 09:31:17 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.11:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 19 09:31:17 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.10:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 19 09:33:24 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.8:857 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 19 10:27:23 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.6:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 19 10:27:24 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.5:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 19 10:27:24 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.8:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 19 10:27:26 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.7:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 19 10:27:27 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.4:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 19 10:27:30 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.3:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 19 10:27:30 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.1:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 19 10:27:32 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.2:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 19 10:28:43 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.9:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 19 10:28:45 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.11:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 19 10:28:45 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.12:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 19 10:28:46 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.10:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 00:01:09 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.5:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 00:01:09 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.4:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 00:01:10 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.8:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 00:01:10 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.3:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 00:01:10 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.2:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 00:01:11 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.6:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 00:01:30 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.7:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 00:02:39 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.11:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 00:02:39 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.9:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 00:02:41 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.10:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 00:02:42 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.12:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 00:04:32 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.1:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 00:05:44 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.8:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 00:05:49 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.3:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 00:05:49 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.4:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 00:05:49 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.7:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 00:05:50 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.5:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 00:05:52 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.6:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 00:05:55 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.1:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 00:05:57 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.2:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 08:10:52 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.9:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 08:18:01 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.8:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 08:18:05 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.5:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 08:18:06 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.6:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 08:18:10 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.1:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 08:18:13 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.9:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 08:18:19 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.7:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 08:18:19 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.2:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 08:18:23 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.11:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 08:18:25 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.12:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 08:18:26 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.10:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 08:21:20 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.4:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 08:21:23 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.3:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 10:47:23 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.8:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 10:47:28 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.5:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 10:47:31 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.3:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 10:47:34 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.7:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 10:47:35 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.4:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 10:47:39 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.6:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 10:47:39 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.9:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 10:47:39 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.1:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 10:47:39 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.11:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 10:47:41 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.2:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 10:47:44 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.12:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 10:50:35 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.10:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 11:08:07 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.5:857 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 11:30:31 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.5:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 11:30:33 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.8:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 11:30:33 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.7:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 11:30:41 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.4:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 11:30:41 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.6:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 11:30:42 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.1:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 11:30:42 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.3:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 11:30:46 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.2:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 11:30:49 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.9:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 11:30:49 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.10:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 11:30:55 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.12:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 11:33:56 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.11:857 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 22:07:57 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.1:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 22:34:17 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.8:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 22:34:23 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.1:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 22:34:23 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.6:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 22:34:23 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.4:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 22:34:24 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.3:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 22:34:25 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.5:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 22:34:31 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.2:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 22:34:31 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.7:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 22:35:30 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.9:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 22:35:32 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.11:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 22:35:33 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.10:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 20 22:35:35 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.12:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 21 01:36:20 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.5:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 21 01:36:20 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.3:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 21 01:36:21 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.4:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 21 01:36:22 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.8:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 21 01:36:23 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.1:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 21 01:36:28 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.6:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 21 01:36:47 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.7:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 21 01:36:49 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.9:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 21 01:36:50 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.12:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 21 01:36:50 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.10:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 21 01:36:52 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.11:862 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
Apr 21 01:40:07 aidas-gsi rpc.mountd[4497]: authenticated mount request from 192.168.11.2:857 for /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119 (/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119)
|
| Attachment 8: Screenshot_2021-04-22_AIDA_Alerting_-_Grafana.png
|
|
| Attachment 9: 53.png
|
|
| Attachment 10: Screenshot_2021-04-22_Statistics_aidas-gsi(3).png
|
.png.png "Screenshot_2021-04-22_Statistics_aidas-gsi(3).png")
|
| Attachment 11: Screenshot_2021-04-22_Temperature_and_status_scan_aidas-gsi(3).png
|
.png.png "Screenshot_2021-04-22_Temperature_and_status_scan_aidas-gsi(3).png")
|
| Attachment 12: 54.png
|
|
|
260
|
Thu Apr 22 07:06:37 2021 |
LPG | Thursday 22nd April: 08:00-12:00 |
08:01 - Taking over from Corrigan...
ucesb ok!
DB ok!
Stats ok!
08:30
ucesb ok!
DB ok!
Stats ok!
09:15
ucesb ok! elog:260/4
DB ok!
Stats ok!
Leakage currents ok! elog:260/1
Temperatures ok! elog:260/2
MIDAS Rates ok! elog:260/3
System wide checks
Clock ok!
ADC ok!
WR ok!
FPGA 1 fail
Base Current Difference
aida12 fault 0x0 : 0x40 : 64
FPGA Timestamp error counter test result: Passed 11, Failed 1
Memory ok!
09:45
ucesb ok!
DB ok!
Stats ok!
10:30
ucesb ok!
DB ok!
Stats ok!
11:00
ucesb ok! elog:260/5
DB ok!
Stats ok!
Leakage currents ok! elog:260/6
Temperatures ok! elog:260/7
MIDAS Rates ok! elog:260/8
System wide checks
Clock ok!
ADC ok!
WR ok!
FPGA 1 fail
Base Current Difference
aida12 fault 0x0 : 0x42 : 66
FPGA Timestamp error counter test result: Passed 11, Failed 1
Memory ok!
11:30
ucesb ok! elog:260/5
DB ok!
Stats ok!
11:55
ucesb ok!
DB ok!
Stats ok! |
| Attachment 1: lpg20.png
|
|
| Attachment 2: Screenshot_2021-04-22_Temperature_and_status_scan_aidas-gsi.png
|
|
| Attachment 3: Screenshot_2021-04-22_Statistics_aidas-gsi.png
|
|
| Attachment 4: Screenshot_2021-04-22_at_08.19.51.png
|
|
| Attachment 5: Screenshot_2021-04-22_at_09.57.10.png
|
|
| Attachment 6: lpg21.png
|
|
| Attachment 7: Screenshot_2021-04-22_Temperature_and_status_scan_aidas-gsi(1).png
|
.png.png "Screenshot_2021-04-22_Temperature_and_status_scan_aidas-gsi(1).png")
|
| Attachment 8: Screenshot_2021-04-22_Statistics_aidas-gsi(1).png
|
.png.png "Screenshot_2021-04-22_Statistics_aidas-gsi(1).png")
|
|
259
|
Wed Apr 21 23:05:39 2021 |
CA | 22nd April 00:00 - 08:00 |
00:07 stats/ucesb/DB checks all ok
00:30 all system wide checks ok *except* fpga errors
Base Current Difference
aida12 fault 0x0 : 0x1f : 31
FPGA Timestamp error counter test result: Passed 11, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
statistics ok - attachment 1
temperatures ok - attachment 2
detector bias / leakage currents ok - attachment 3
no error messages in DB terminal & ucesb ok - attachment 4
01:00 stats/ucesb/DB checks all ok
01:30 stats/ucesb/DB checks all ok
01:53 issues with beam dropping in and out - ESR working to stabilise this
02:00 stats/ucesb/DB checks all ok
02:09 beam back in a stable state
02:30 all system wide checks ok *except* fpga errors
Base Current Difference
aida12 fault 0x0 : 0x29 : 41
FPGA Timestamp error counter test result: Passed 11, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
02:33 statistics ok - attachment 5
temperatures ok - attachment 6
detector bias / leakage currents ok - attachment 7
no error messages in DB terminal & ucesb ok - attachment 8
02:45 beam off - potentially serious issue, awaiting update from FRS
03:00 significant issues with UNILAC, beam not expected back for a while
03:05 stats/ucesb/DB checks all ok
03:32 stats/ucesb/DB checks all ok
03:55 beam is back
04:00 stats/ucesb/DB checks all ok
04:30 all system wide checks ok *except* fpga errors
Base Current Difference
aida12 fault 0x0 : 0x2b : 43
FPGA Timestamp error counter test result: Passed 11, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
04:32 statistics ok - attachment 9
temperatures ok - attachment 10
detector bias / leakage currents ok - attachment 11
no error messages in DB terminal & ucesb ok - attachment 12
05:00 stats/ucesb/DB checks all ok
05:30 stats/ucesb/DB checks all ok
06:00 stats/ucesb/DB checks all ok
06:30 all system wide checks ok *except* fpga errors
Base Current Difference
aida12 fault 0x0 : 0x34 : 52
FPGA Timestamp error counter test result: Passed 11, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
06:32 temperatures ok - attachment 13
statistics ok - attachment 14
detector bias / leakage currents ok - attachment 15
no error messages in DB terminal & ucesb ok - attachment 16
07:00 stats/ucesb/DB checks all ok
07:30 stats/ucesb/DB checks all ok
07:59 stats/ucesb/DB checks all ok
|
| Attachment 1: Screenshot_from_2021-04-21_23-31-22.png
|
|
| Attachment 2: Screenshot_from_2021-04-21_23-31-44.png
|
|
| Attachment 3: Screenshot_from_2021-04-21_23-32-03.png
|
|
| Attachment 4: Screenshot_from_2021-04-21_23-34-46.png
|
|
| Attachment 5: Screenshot_from_2021-04-22_01-31-14.png
|
|
| Attachment 6: Screenshot_from_2021-04-22_01-31-37.png
|
|
| Attachment 7: Screenshot_from_2021-04-22_01-34-03.png
|
|
| Attachment 8: Screenshot_from_2021-04-22_01-34-31.png
|
|
| Attachment 9: Screenshot_from_2021-04-22_03-32-36.png
|
|
| Attachment 10: Screenshot_from_2021-04-22_03-32-57.png
|
|
| Attachment 11: Screenshot_from_2021-04-22_03-33-11.png
|
|
| Attachment 12: Screenshot_from_2021-04-22_03-33-57.png
|
|
| Attachment 13: Screenshot_from_2021-04-22_05-33-01.png
|
|
| Attachment 14: Screenshot_from_2021-04-22_05-33-19.png
|
|
| Attachment 15: Screenshot_from_2021-04-22_05-33-30.png
|
|
| Attachment 16: Screenshot_from_2021-04-22_05-33-43.png
|
|
|
258
|
Wed Apr 21 15:01:14 2021 |
LJW, MA | Wednesday 21st April 16:00-20:00 |
16:30
All System checks okay, except for:
FPGA Timestamp error:
Base Current Difference
aida12 fault 0x0 : 0x1b : 27
FPGA Timestamp error counter test result: Passed 11, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
Temperature Checks: See attachment 1
Statistics Checks: See attachment 2
Bias & Leakage Currents: See attachment 3
18:09
All system checks okay, except for:
FPGA Timestamp error:
Base Current Difference
aida12 fault 0x0 : 0x1d : 29
FPGA Timestamp error counter test result: Passed 11, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
Temperature checks: See attachment 4
Statistics Checks: See attachment 5
Bias & Leakage Currents: See attachment 6
19:40
All system checks okay, except for:
FPGA Timestamp error:
Base Current Difference
aida12 fault 0x0 : 0x1d : 29
FPGA Timestamp error counter test result: Passed 11, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
Temperature checks: See attachment 7
Statistics Checks: See attachment 8
Bias & Leakage Currents: See attachment 9
*Beam being tuned to another experiment (R3B) - all DSSD implants fluctuating back and forth to 0 Hz for around the last hour or so*
22:30
General check
Rates, Voltages, Temperatures, Ucesb are attached 10,11,12,13
Its been noticed that aida04 temp. is incresing to about 65 (red), still below 70.
the system wide check are all ok except FPGA
Base Current Difference
aida12 fault 0x0 : 0x1d : 29
FPGA Timestamp error counter test result: Passed 11, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
|
| Attachment 1: 210421_1.png
|
|
| Attachment 2: 210421_2.png
|
|
| Attachment 3: 2104211_3.png
|
|
| Attachment 4: 210421_4.png
|
|
| Attachment 5: 210421_5.png
|
|
| Attachment 6: 210421_6.png
|
|
| Attachment 7: 210421_7.png
|
|
| Attachment 8: 210421_8.png
|
|
| Attachment 9: 210421_9.png
|
|
| Attachment 10: 210421_2230_Stats.png
|
|
| Attachment 11: 210421_2230_Temp.png
|
|
| Attachment 12: 210421_2230_Ucesb.png
|
|
| Attachment 13: 210421_2230_Voltages.png
|
|
|
257
|
Wed Apr 21 11:27:55 2021 |
OH | Wed 21st April 12:00-16:00 |
12:30 MBS has crashed
This time AIDA is still running fine.
Stats ok - attachment 1
Temperature - attachment 2
Bias and leakage currents - attachment 3
All system wide checks ok *except*
Base Current Difference
aida12 fault 0x0 : 0x19 : 25
FPGA Timestamp error counter test result: Passed 11, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
Now also note errors on aida7 and aida03 in the merger TS errors
(0 represents the sum across all FEE)
0 1355800 0x0014b018
1 0 0x00000000
2 0 0x00000000
3 33 0x00000021
4 0 0x00000000
5 0 0x00000000
6 0 0x00000000
7 10 0x0000000a
8 0 0x00000000
9 0 0x00000000
10 0 0x00000000
11 1355757 0x0014afed
15:30 CA takes over
Stats ok - attachment 4
Temperature - attachment 5
Bias and leakage currents - attachment 6
all system wide checks ok *except*
Base Current Difference
aida12 fault 0x0 : 0x19 : 27
FPGA Timestamp error counter test result: Passed 11, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
15:42 db check and ucesb ok |
| Attachment 1: 210421_1338_Stats.png
|
|
| Attachment 2: 210421_1328_Temp.png
|
|
| Attachment 3: 210421_1342_Bias.png
|
|
| Attachment 4: Screenshot_from_2021-04-21_14-33-36.png
|
|
| Attachment 5: Screenshot_from_2021-04-21_14-34-29.png
|
|
| Attachment 6: Screenshot_from_2021-04-21_14-36-08.png
|
|
|
256
|
Wed Apr 21 06:56:42 2021 |
MS | Wednesday 21 April 08:00-12:00 2021 |
8:00
Clock status test result: Passed 12, Failed 0
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable
FEE64 module aida01 failed
FEE64 module aida02 failed
FEE64 module aida03 failed
FEE64 module aida04 failed
FEE64 module aida05 failed
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida08 failed
FEE64 module aida09 failed
FEE64 module aida10 failed
FEE64 module aida11 failed
FEE64 module aida12 failed
Calibration test result: Passed 0, Failed 12
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
White Rabbit error counter test result: Passed 12, Failed 0
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Base Current Difference
aida12 fault 0x0 : 0x15 : 21
FPGA Timestamp error counter test result: Passed 11, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
Returned 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 19 6 7 2 2 4 3 2 2 3 7 : 39596
aida02 : 25 3 2 1 1 3 2 3 2 3 7 : 39548
aida03 : 28 5 3 1 2 2 2 3 2 3 7 : 39528
aida04 : 27 6 2 2 2 4 2 4 2 3 7 : 40316
aida05 : 9 9 4 3 1 3 2 3 3 3 7 : 40652
aida06 : 21 7 6 1 2 2 4 2 2 3 7 : 39564
aida07 : 20 7 5 0 1 2 2 3 2 3 7 : 39448
aida08 : 21 5 4 1 1 2 3 3 2 3 7 : 39708
aida09 : 1 3 1 3 0 2 1 4 2 3 7 : 39564
aida10 : 20 6 3 2 1 2 3 3 2 3 7 : 39728
aida11 : 18 4 2 1 2 2 2 2 2 3 7 : 38952
aida12 : 19 8 7 1 1 2 3 3 2 3 7 : 39772
From the merger statistics during the night there have been 1355757 TS errors in aida12. Corrigan said they have been quiet since around 3am though.
10:00
Clock status test result: Passed 12, Failed 0
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable
FEE64 module aida01 failed
FEE64 module aida02 failed
FEE64 module aida03 failed
FEE64 module aida04 failed
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida08 failed
FEE64 module aida09 failed
FEE64 module aida10 failed
FEE64 module aida11 failed
FEE64 module aida12 failed
Calibration test result: Passed 1, Failed 11
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
White Rabbit error counter test result: Passed 12, Failed 0
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Base Current Difference
aida12 fault 0x0 : 0x15 : 21
FPGA Timestamp error counter test result: Passed 11, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
Returned 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 15 7 6 2 1 4 3 2 2 3 7 : 39508
aida02 : 1 2 1 0 1 2 2 4 2 3 7 : 39780
aida03 : 23 8 2 0 1 3 2 3 2 3 7 : 39548
aida04 : 23 9 1 2 3 3 2 4 2 3 7 : 40244
aida05 : 14 7 2 2 4 3 2 3 1 4 7 : 40784
aida06 : 18 5 5 2 2 2 3 3 2 3 7 : 39808
aida07 : 27 4 3 2 2 3 3 2 2 3 7 : 39420
aida08 : 23 6 3 2 2 2 3 3 2 3 7 : 39804
aida09 : 18 4 6 3 2 4 3 2 2 3 7 : 39592
aida10 : 6 9 5 3 2 3 2 3 2 3 7 : 39696
aida11 : 16 4 1 2 3 4 1 3 2 3 7 : 39536
aida12 : 18 5 7 1 0 3 3 3 2 3 7 : 39808
10:22
NH Noticed that the ASIC clocks had not been synchronised following the last AIDA restart. - attachment 7
As the beam is having issues they are not currently recording
I have now synchronised the ASIC clocks.
All ADC are now calibrated
11:06
Used the beam off time to change the drive writing to. Now writing to /media/ThirdDrive/TapeData/S460/R50
11:10
Beam is back they will start writing to file again
12:00
Clock status test result: Passed 12, Failed 0
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable
Calibration test result: Passed 12, Failed 0
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
White Rabbit error counter test result: Passed 12, Failed 0
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Base Current Difference
aida12 fault 0x0 : 0x17 : 23
FPGA Timestamp error counter test result: Passed 11, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
Returned 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 20 5 5 1 1 4 2 3 2 3 7 : 39720
aida02 : 16 8 4 1 0 2 1 4 2 3 7 : 39648
aida03 : 9 7 3 2 3 3 2 3 2 3 7 : 39692
aida04 : 8 7 5 2 3 5 3 2 2 3 7 : 39720
aida05 : 20 5 3 4 2 3 3 3 2 3 7 : 39976
aida06 : 25 8 0 3 1 2 3 3 2 3 7 : 39748
aida07 : 24 8 4 2 3 3 2 3 2 3 7 : 39776
aida08 : 22 9 0 2 2 2 3 3 2 3 7 : 39776
aida09 : 1 4 4 3 2 3 2 3 2 3 7 : 39620
aida10 : 13 4 4 3 2 3 1 3 2 3 7 : 39412
aida11 : 2 3 1 1 3 4 2 2 2 3 7 : 39184
aida12 : 16 10 7 2 1 3 1 3 2 3 7 : 39424
|
| Attachment 1: Screenshot_2021-04-21_Statistics_aidas-gsi.png
|
|
| Attachment 2: Screenshot_2021-04-21_Temperature_and_status_scan_aidas-gsi.png
|
|
| Attachment 3: 46.png
|
|
| Attachment 4: Screenshot_2021-04-21_Statistics_aidas-gsi(1).png
|
.png.png "Screenshot_2021-04-21_Statistics_aidas-gsi(1).png")
|
| Attachment 5: Screenshot_2021-04-21_Temperature_and_status_scan_aidas-gsi(1).png
|
.png.png "Screenshot_2021-04-21_Temperature_and_status_scan_aidas-gsi(1).png")
|
| Attachment 6: 47.png
|
|
| Attachment 7: Capture.PNG
|
|
| Attachment 8: Screenshot_2021-04-21_Temperature_and_status_scan_aidas-gsi(2).png
|
.png.png "Screenshot_2021-04-21_Temperature_and_status_scan_aidas-gsi(2).png")
|
| Attachment 9: Screenshot_2021-04-21_Statistics_aidas-gsi(2).png
|
.png.png "Screenshot_2021-04-21_Statistics_aidas-gsi(2).png")
|
| Attachment 10: 48.png
|
|
|
255
|
Tue Apr 20 23:08:22 2021 |
CA | Wed Apr 21 00:00-08:00 |
00:00 - CA takes over
00:15 - stats/database/ucesb ok - attachment 1
- another burst of bad timestamp error messages in NewMerger terminal - but all FEE64 still ok - attachment 2
- Merger/TapeService ok
00:21 - beam still off while UNILAC having problems
00:26 - beam is back - rate spectra - attachment 3
00:32 - ucesb following beam return - attachment 4
- implant rates reaching highs of ~1kHz
00:35 - Grafana leakage current monitor last 6 hours - attachment 5
01:00 *all* system wide checks ok
stats ok - attachment 6
temperatures ok - attachment 7
detector bias / leakage currents ok - attachment 8
db check ok / ucesb ok
still observe periodic bursts of bad timestamp errors in NewMerger terminal - FEE64s still linked with merger ok
01:30 AIDA crashes just as I make check, most FEE64 lost connection -> powercycle and restart MIDAS
01:53 AIDA reset completed - writing to file NULL/R48
01:56 all system wide checks ok *except*
FPGA error counter
Base Current Difference
aida12 fault 0x0 : 0x1 : 1
FPGA Timestamp error counter test result: Passed 11, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
ADC calibration
FEE64 module aida04 failed
FEE64 module aida05 failed
Calibration test result: Passed 10, Failed 2
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
02:01 good event statistics ok - attachment 9
temperatures ok - attachment 10
detector bias / leakage currents ok - attachment 11
db check / ucesb ok
02:20 beam stop
02:21 beam back
02:30 stats/ucesb/db checks all ok
03:00 stats/ucesb/db checks all ok - attachment 12
03:30 stats/ucesb/db checks all ok
04:00 system wide checks ok, except;
ADC calibration
FEE64 module aida04 failed
FEE64 module aida05 failed
Calibration test result: Passed 10, Failed 2
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
FPGA errors
Base Current Difference
aida12 fault 0x0 : 0x9 : 9
FPGA Timestamp error counter test result: Passed 11, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
04:03 beam off briefly
04:05 good event statistics ok - attachment 13
04:07 beam back
temperatures ok - attachment 14
bias / leakage currents ok - attachment 15
db check / ucesb ok - attachment 16
implant rates in AIDA peaking at ~1.5kHz
04:30 stats/ucesb/db checks all ok
04:48 no beam
04:58 beam back
05:00 stats/ucesb/db checks all ok - attachment 17
05:30 stats/ucesb/db checks all ok
06:00 system wide checks ok, except;
ADC calibration
FEE64 module aida04 failed
FEE64 module aida05 failed
Calibration test result: Passed 10, Failed 2
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
FPGA errors
Base Current Difference
aida12 fault 0x0 : 0x9 : 9
FPGA Timestamp error counter test result: Passed 11, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
FEE64 Linux Memory Info
Returned 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 5 5 3 3 2 3 2 3 2 3 7 : 39628
aida02 : 22 11 3 0 2 3 2 3 2 3 7 : 39648
aida03 : 20 10 1 1 2 2 1 4 2 3 7 : 39760
aida04 : 17 10 2 2 3 3 2 3 2 3 7 : 39732
aida05 : 1 8 3 2 3 2 2 3 3 3 7 : 40564
aida06 : 5 6 2 3 2 2 3 3 2 3 7 : 39748
aida07 : 18 4 7 0 1 3 1 4 2 3 7 : 39832
aida08 : 22 10 3 3 2 2 3 3 2 3 7 : 39864
aida09 : 3 6 2 2 0 3 1 4 2 3 7 : 39708
aida10 : 2 4 3 2 1 2 2 3 2 3 7 : 39384
aida11 : 20 8 3 1 1 3 3 2 2 3 7 : 39328
aida12 : 16 6 3 1 0 3 2 3 2 3 7 : 39488
06:02 good event statistics ok - attachment 18
FEE64 temperatures ok - attachment 19
detector bias / leakage currents ok - attachment 20
06:06 db check and ucesb ok - attachment 21
NewMerger terminal hasn't flagged up any more errors since ~ 03:00
06:30 stats/ucesb/db checks all ok
DESPEC restarts timesorter
07:00 stats/ucesb/db checks all ok
07:30 stats/ucesb/db checks all ok - attachment 22
grafana leakage currents last 1 hour - attachment 23
|
| Attachment 1: Screenshot_from_2021-04-20_23-15-20.png
|
|
| Attachment 2: Screenshot_from_2021-04-20_23-18-14.png
|
|
| Attachment 3: Screenshot_from_2021-04-20_23-29-06.png
|
|
| Attachment 4: Screenshot_from_2021-04-20_23-32-08.png
|
|
| Attachment 5: Screenshot_from_2021-04-20_23-34-42.png
|
|
| Attachment 6: Screenshot_from_2021-04-21_00-00-06.png
|
|
| Attachment 7: Screenshot_from_2021-04-21_00-00-29.png
|
|
| Attachment 8: Screenshot_from_2021-04-21_00-01-15.png
|
|
| Attachment 9: Screenshot_from_2021-04-21_01-00-03.png
|
|
| Attachment 10: Screenshot_from_2021-04-21_01-00-48.png
|
|
| Attachment 11: Screenshot_from_2021-04-21_01-01-05.png
|
|
| Attachment 12: Screenshot_from_2021-04-21_01-59-30.png
|
|
| Attachment 13: Screenshot_from_2021-04-21_03-05-24.png
|
|
| Attachment 14: Screenshot_from_2021-04-21_03-05-47.png
|
|
| Attachment 15: Screenshot_from_2021-04-21_03-06-07.png
|
|
| Attachment 16: Screenshot_from_2021-04-21_03-11-44.png
|
|
| Attachment 17: Screenshot_from_2021-04-21_04-00-22.png
|
|
| Attachment 18: Screenshot_from_2021-04-21_05-00-18.png
|
|
| Attachment 19: Screenshot_from_2021-04-21_05-00-43.png
|
|
| Attachment 20: Screenshot_from_2021-04-21_05-03-47.png
|
|
| Attachment 21: Screenshot_from_2021-04-21_05-05-43.png
|
|
| Attachment 22: Screenshot_from_2021-04-21_06-30-20.png
|
|
| Attachment 23: Screenshot_from_2021-04-21_06-32-51.png
|
|