ELOG DESPEC

AIDA GELINA BRIKEN nToF CRIB ISOLDE CIRCE nTOFCapture DESPEC DTAS EDI_PSA 179Ta CARME StellarModelling DCF K40

DESPEC, Page 3 of 35

Full | Summary | Threaded | Hide attachments

687 Entries

Goto page Previous 1, 2, 3, 4 ... 33, 34, 35 Next

ID	Date	Author	Subject
568	Tue Apr 9 09:04:07 2024	TD	[How To] Remote control of BNC PB-5
BNC PB-5 set to remote control Login to RPi nnrpi2 ssh -X nnrpi2 -l pi Start PB-5 app - attachment 1 ./BNC-PB-5/pb5.py
Attachment 1: Screenshot_from_2024-04-09_10-03-42.png

618	Sun Apr 28 16:07:59 2024	TD	[How To] Recover DAQ when one FEE64 stops sending data - short(er) version
0) Please infrom the 'on shift' crew that you need to restart the AIDA DAQ 1) STOP DAQ [workspace 2, display 1, browser tab 'Run Control'] all FEE64s should change status from 'going' to 'stopped' except whichever FEE64(s) have a problem - typically their status is 'going' or 'error' 2) Restart Merger by selecting the Merger icon [workspace 3, display 2] terminal window showing the startup will popup 3) Setup Merger by selecting 'SETUP' [workspace 3, display 1, browser tab 'NewMerger Control'] 4) Start Merger by selecting 'GO' [workspace 3, display 1, browser tab 'NewMerger Control'] all data links will be inactive (red) 5) Disable data transfer by selecting 'Data transfer Disable #1 all' [workspace 2, display 1, browser tab 'Run Control'] 6) Start DAQ Start DAQ by selecting 'GO' [workspace 2, display 1, browser tab 'Run Control'] check that DAQ is actually going - if not repeat 'GO' 7) Check WR timestamps by selecting 'Collect all timestamps' [workspace 2, display 1, browser tab 'GSI White Rabbit Control'] readout times will be displayed for each FEE64 check that the readout times are monotonically increasing as you read down the list of readout times - if not, contact expert support 8) Stop DAQ Start DAQ by selecting 'STOP' [workspace 2, display 1, browser tab 'Run Control'] check that DAQ has actually stopped - if not repeat 'STOP 9) Enable data transfer to Merger and TapeServer and start DAQ Enable data transfer by selecting 'Enable data transfer Enable #1 All' [workspace 2, display 1, browser tab 'Run Control'] Start DAQ by selecting 'GO' [workspace 2, display 1, browser tab 'Run Control'] check that DAQ is actually going - if not repeat 'GO' 10) Check data is being transferred to Merger and TapeServer Select 'Reload' [workspace 3, display 1, browser tab 'NewMerger Control'] you should see a non-zero 'current merged data rate' and all links should show as green at least some of the time (requires observation over several reloads) Select 'Reload' [workspace 3, display 1, browser tab 'TapeServer'] you should see 'Blocks written' and 'kb written' counters increase 11) Check AIDA data is being forwarded from the TapeServer to MBS [workspace 3, display 2, command terminal] 12) Restart should now have been successfully completed. Please inform the 'on shift' crew' that AIDA is forwarding data. If not, please request expert assistance.
644	Sun Jun 9 13:32:48 2024	TD	[How To] Recover DAQ when one FEE64 stops sending data - short(er) version
1) Select 'No Storage' [workspace 3, display 1, browser tab 'TapeServer'] If already in 'No Storage' mode please ignore this step 2) STOP DAQ [workspace 2, display 1, browser tab 'Run Control'] all FEE64s should change status from 'going' to 'stopped' except whichever FEE64(s) have a problem - typically their status is 'going' or 'error' 3) Restart Merger by selecting the Merger icon [workspace 3, display 2] terminal window showing the startup will popup 4) Setup Merger by selecting 'SETUP' [workspace 3, display 1, browser tab 'NewMerger Control'] 5) Start Merger by selecting 'GO' [workspace 3, display 1, browser tab 'NewMerger Control'] all data links will be inactive (red) 6) Start DAQ by selecting 'GO' [workspace 2, display 1, browser tab 'Run Control'] check that DAQ is actually going - if not repeat 'GO' 7) Check data is being transferred to Merger and TapeServer Select 'Reload' [workspace 3, display 1, browser tab 'NewMerger Control'] you should see a non-zero 'current merged data rate' and all links should show as green at least some of the time (requires observation over several reloads) Select 'Reload' [workspace 3, display 1, browser tab 'TapeServer'] you should see 'Blocks written' and 'kb written' counters increase 8) Check AIDA data is being forwarded from the TapeServer to MBS [workspace 3, display 2, command terminal] 9) Unselect 'No Storage' [workspace 3, display 1, browser tab 'TapeServer'] If already in 'No Storage' mode at the start of this procedure please ignore this step If you continue to have problems, please request expert assistance.
609	Fri Apr 26 08:15:59 2024	TD	[How To] Recover DAQ when one FEE64 stops sending data
Occasionally, one (sometimes more) FEE64(s) will stop producing data whilst the DAQ is running and data is being transferred to the Merger and TapeServer. Typical symptoms are: - no data in Rate spectrum [workspace 2, display 1, browser tab 'Spectrum Browser'] - zero rate in 'ADC data item' statistics [workspace 2, display 1, browser tab 'Statistics'] - Merger data link inactive [workspace 3, display 1, browser tab 'NewMerger Control'] active data links are green, inactive links are red N.B. data links are numbered from 0 to n-1, FEE64s are numbered from 1 to n 0) Please infrom the 'on shift' crew that you need to restart the AIDA DAQ 1) STOP DAQ [workspace 2, display 1, browser tab 'Run Control'] all FEE64s should change status from 'going' to 'stopped' except whichever FEE64(s) have a problem - typically their status is 'going' or 'error' 2) Restart Merger by selecting the Merger icon [workspace 3, display 2] terminal window showing the startup will popup 3) Setup Merger by selecting 'SETUP' [workspace 3, display 1, browser tab 'NewMerger Control'] 4) Start Merger by selecting 'GO' [workspace 3, display 1, browser tab 'NewMerger Control'] all data links will be inactive (red) 5) Reboot problem FEE64s Launch a command terminal and telnet to the problem FEE64 for example telnet aida04 Enter user 'root' and enter password. When logged in reboot the FEE64 using the command reboot 6) Monitor boot using server system log [workspace 1, display 2, browser tab 'terminal'] you should see two messages showing the FEE64 mounting filesystems wait c. three minutes for boot to complete 7) Restart DAQ Select DAQ status 'Update' [workspace 2, display 1, browser tab 'Run Control'] Select DAQ status 'RESET' [workspace 2, display 1, browser tab 'Run Control'] Select DAQ status 'SETUP' [workspace 2, display 1, browser tab 'Run Control'] Disable histogramming by selecting 'Histogramming Disable all' [workspace 2, display 1, browser tab 'Run Control'] Disable waveforms by selecting 'Enable Waveforms Disable all' [workspace 2, display 1, browser tab 'Run Control'] Disable data transfer by selecting 'Data transfer Disable #1 all' [workspace 2, display 1, browser tab 'Run Control'] 8) Restore ASIC settings Select 'Save/Restore Settings' [workspace 2, display 1, browser tab 'Control'] Select the correct 'Data Base Key' [workspace 2, display 1, browser tab 'Module Settings'] Select 'Restore Settings' [workspace 2, display 1, browser tab 'Module Settings'] Wait 10s 9) Check ASIC settings Select 'Check ASIC Control' [workspace 2, display 1, browser tab 'ASIC Control'] Wait for popup indicating completion of checks and dismiss 10) System wide checks Select 'Synchronise the ASIC clocks' [workspace 2, display 1, browser tab 'System Wide Checks'] Select 'Collect all WR status error counters ...' [workspace 2, display 1, browser tab 'System Wide Checks'] Select 'Collect all FPGA timestamp error counters ...' [workspace 2, display 1, browser tab 'System Wide Checks'] Select 'Check clock status' [workspace 2, display 1, browser tab 'System Wide Checks'] Select 'Check WR decoder status' [workspace 2, display 1, browser tab 'System Wide Checks'] Select 'Check FPGA timestamp errors' [workspace 2, display 1, browser tab 'System Wide Checks'] make a note or take a screenshot of any error messages observed 11) Start DAQ Enable histogramming by selecting 'Histogramming Enable All' [workspace 2, display 1, browser tab 'Run Control'] Enable waveforms by selecting 'Enable Waveforms Enable All' [workspace 2, display 1, browser tab 'Run Control'] Start DAQ by selecting 'GO' [workspace 2, display 1, browser tab 'Run Control'] check that DAQ is actually going - if not repeat 'GO' 12) Check WR timestamps by selecting 'Collect all timestamps' [workspace 2, display 1, browser tab 'GSI White Rabbit Control'] readout times will be displayed for each FEE64 check that the readout times are monotonically increasing as you read down the list of readout times - if not, contact expert support 13) Stop DAQ Start DAQ by selecting 'STOP' [workspace 2, display 1, browser tab 'Run Control'] check that DAQ has actually stopped - if not repeat 'STOP 14) Enable data transfer to Merger and TapeServer and start DAQ Enable data transfer by selecting 'Enable data transfer Enable #1 All' [workspace 2, display 1, browser tab 'Run Control'] Start DAQ by selecting 'GO' [workspace 2, display 1, browser tab 'Run Control'] check that DAQ is actually going - if not repeat 'GO' 15) Check data is being transferred to Merger and TapeServer Select 'Reload' [workspace 3, display 1, browser tab 'NewMerger Control'] you should see a non-zero 'current merged data rate' and all links should show as green at least some of the time (requires observation over several reloads) Select 'Reload' [workspace 3, display 1, browser tab 'TapeServer'] you should see 'Blocks written' and 'kb written' counters increase 16) Check AIDA data is being forwarded from the TapeServer to MBS [workspace 3, display 2, command terminal] 17) Restart should now have been successfully completed. Please inform the 'on shift' crew' that AIDA is forwarding data. If not, please request expert assistance.
139	Tue Mar 10 12:36:44 2020	TD	[How To] Monitor AIDA
All AIDA operations are controlled from the Dell T410 Workstation aidas1 using web browsers. AIDA has multiple, independent data acquisition servers connected by an Ethernet network. Response times to commands, information updates etc are ~10-60s - please be patient! Every 4h 1) Desktop 1 - 1x PUTTY terminal sessions Each PUTTY terminal session is connected to a CAEN N1419 4 ch HV Power Supply Addr 00 Ch 0 = DSSSD # 1 : : Ch 2 = DSSSD # 3 Select the terminal and obtain a screenshot (Fn-Alt-PrtScn) of DSSSD #1 = #3 detector bias voltages and leakage currents Upload screenshots to Elog (elog.ph.ed.ac.uk/DESPEC) Update leakage current spreadsheet (https://docs.google.com/spreadsheets/d/12hgbrywB10hGsKt5uc2HnLfZymh8_uvM6cEASbbqnBE/edit#gid=813167023) If S4 ambient temperature information is available please add this information to the Elog Compare DSSSD leakage currents (and temperatures) with previous observations 2) Desktop 1 - Firefox web browser - Temperatures tab Select 'Reload' Check temperatures Virtex < 70 deg C PSU < 45 deg C ASIC < 70 deg C Note that the ASIC temperatures for some data acquisition servers sometimes read back as 0.0 If the temperature values above are exceeded switch off AIDA FEE64 electrical power as follows Desktop 3 - Firefox web browser From 'Rly16 Control' select 'Switch ALL off' and now seek expert help If DAQ running also 3) Desktop 1 - Firefox web browser - Run Control tab Select 'Update' Check state of all data acquisition servers is 'going' Check histogramming and data transfer #1 are enabled as indicated by 'H W X' tags 4) Desktop 1 - Firefox web browser - Statistics Check 'good event' statistics - screenshot to Elog - compare with previous Elog entries 5) Desktop 1 - Firefox web browser - System Wide Checks Check all system wide checks - compare results with porevious Elog entries 6) Desktop 2 - Firefox web browser - MERGE tab Select 'Reload' Check merge rate (~2-3M data items/s) consistent with value observed at start of run Observe active spinners in NewMerger and TapeServer server terminal windows adjacent to web browser Chweck MBS terminal data rate is around 20000kB/s Bottom right terminal on desktop 2 7) Desktop 2 - Firefox web browser - TapeServer tab Select 'Reload' Check data rate consistent with value observed at start of run
478	Wed Jun 8 09:55:28 2022	NH	Weds 8 May
10.30 : Turn off AIDA FEE64s Prepare a ground cable for snout - M10 terminal on one end for copper bar. Will affix with Cu tape for now (no free screws) Plan for today: Remove extraneous ground cables Check ground of all FEE64s esp. 05 and 07 - include on FEE64 side. 14:42 - Done all grounds seem snug, extra ground cables removed. Try power on and check Try powering just DSSD1 FEE64s - No change (aida01 125k, aida02,03,04 - 400k) Try grounding snout 14:59 Connection to snout very bad as tape does not fix well to thick copper in braid. nO difference either way. Probe PSU output with Battery DSO if available - See figure 2 Try shuffling FEE64 power allocation - 15:32 - Now as 1 X 3 X 2 X 4 X 5 X 7 X 6 X 8 X (Also FEEs on own pair) No difference to rates Multimeter voltages for last socket of PSU1 (paired with aida04): 1 : 6.02 V 2: 6.02 V 3: 6.02 V 4: -6.44 V 5: 7.74 V Similar on PSU2 Voltage between right (GND) pin 5 and 1 (pin 5 is COM) is -0.4 V, similar if pin 4 is COM. Pin1-3 are 0 V when paired
Attachment 1: 2022-06-08_14-48-21_goodevents.png

Attachment 2: rn_image_picker_lib_temp_97373248-bdea-4864-b8a9-a13d0d016e30.jpg

326	Tue May 18 23:11:40 2021	CA	Wednesday May 19th 00:00 - 08:00
00:00 CA takes over LS has performed usual checks, see previous Elog entry - all ok 00:30 stats ok R14_470 deadtime at 4.4% 02:10 system wide checks ok except: all ADC fail calibration (expected, no waveforms) WR decoder status Base Current Difference aida05 fault 0x500 : 0x52f : 47 White Rabbit error counter test result: Passed 15, Failed 1 Understand the status reports as follows:- Status bit 3 : White Rabbit decoder detected an error in the received data Status bit 2 : Firmware registered WR error, no reload of Timestamp Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR 02:12 FEE64 Temps ok - attachment 1 Stats ok - attachment 2 detector bias / leakage currents ok - attachment 3 dead-times ok - R14_497 max 4% - attachment 4 03:27 DAQ continues ok - writing to file R14_517 04:07 system wide checks ok except: all ADC fail calibration (expected, no waveforms) WR decoder status Base Current Difference aida05 fault 0x500 : 0x537 : 55 White Rabbit error counter test result: Passed 15, Failed 1 Understand the status reports as follows:- Status bit 3 : White Rabbit decoder detected an error in the received data Status bit 2 : Firmware registered WR error, no reload of Timestamp Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR FEE64 Temps ok - attachment 6 Stats ok - attachment 5 detector bias / leakage currents ok - attachment 7 dead-times ok - R14_526 max 4% - attachment 8 05:27 bad timestamp error in merger terminal - attachment 9 06:05 all system wide checks ok except; all ADC fail calibration (expected, no waveforms) WR decoder status Base Current Difference aida05 fault 0x500 : 0x53d : 61 White Rabbit error counter test result: Passed 15, Failed 1 Understand the status reports as follows:- Status bit 3 : White Rabbit decoder detected an error in the received data Status bit 2 : Firmware registered WR error, no reload of Timestamp Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR 06:07 FEE64 Temps ok - attachment 10 Stats ok - attachment 11 detector bias / leakage currents ok - attachment 12 dead-times ok - R14_558 max 3.6% - attachment 13 07:01 another bad timestamp error in merger terminal - attachment 14
Attachment 1: Screenshot_from_2021-05-19_01-13-48.png

Attachment 2: Screenshot_from_2021-05-19_01-14-09.png

Attachment 3: Screenshot_from_2021-05-19_01-14-24.png

Attachment 4: Screenshot_from_2021-05-19_01-19-37.png

Attachment 5: Screenshot_from_2021-05-19_03-04-39.png

Attachment 6: Screenshot_from_2021-05-19_03-05-33.png

Attachment 7: Screenshot_from_2021-05-19_03-05-53.png

Attachment 8: Screenshot_from_2021-05-19_03-09-03.png

Attachment 9: Screenshot_from_2021-05-19_04-26-09.png

Attachment 10: Screenshot_from_2021-05-19_05-08-01.png

Attachment 11: Screenshot_from_2021-05-19_05-08-23.png

Attachment 12: Screenshot_from_2021-05-19_05-08-46.png

Attachment 13: Screenshot_from_2021-05-19_05-11-33.png

Attachment 14: Screenshot_from_2021-05-19_06-12-29.png

43	Wed Apr 3 09:28:07 2019	CA, CB, TD	Wednesday April 3rd 2018
Arrived at 09:54 - sequenced all FEE64 on FEE64 not mounted properly - something happened to network overnight, see attachment 1,2 for AIDA system log 10.23 - FEE64 still not mounted properly, power cycling FEE64 10.30 - sequenced all FEE64 on 10.43 - FEE64 mounted successfully - DAQ reset and setup - looks ok - histogram enabled on aida12 - clock ReSYNC - system wide checks - all FEE64 pass clock status check - aida2 and aida12 fail ADC calibration - SYNC pulse and SYNC error checks ok - WR decoder check returns errors (attachment 3) 10.53 - FEE64 temperatures ok (attachment 4) - detector bias/leakage current ok (attachment 5) 11.04 - TapeServer start - not writing to storage - ASIC check load performed - DAQ start - ASIC control tab (attachment 6) - Run control keeps reloading - has returned to 'reset' screen with aida11 showing "undefined" (attachment 7) - good event statistics - all FEE64 showing 0 except aida09 (attachment 8) 11.20 - stopped tape server - reset daq - returns error (attachment 9) 11.41 DAQ off, detector bias off, FEE64 off 13.30 detector bias on, FEE64 sequenced on all browser tabs reopened merger set back up - ok mbs data relay set back up - ok DAQ setup and started, initially ok good event statistics (attachment 10) 16.25 FEE64 restarted, along with Merger, TapeServer, DAQ (not writing to file) thresholds (slow comparator, fast comparator HEC, fast comparator LEC) raised to 0xff for testing. aida10 stats appear reasonable, however wave events still reported despite being deactivated (see report entry on DAQ stall) (attachment 11) 17.00 DAQ, TapeServer, Merger all stopped Detector bias off FEE64s switched off coolant water still running (cannot be disabled due to lack of access) AIDA in safe state
Attachment 1: 030419_aidasystemlog.txt
Apr 3 04:06:56 aidas-gsi NetworkManager[3917]: <info> nameserver '140.181.96.11' Apr 3 04:06:56 aidas-gsi NetworkManager[3917]: <info> nameserver '140.181.96.29' Apr 3 04:06:56 aidas-gsi NetworkManager[3917]: <info> domain name 'gsi.de' Apr 3 04:08:00 aidas-gsi dhcpd: Dynamic and static leases present for 192.168.11.117. Apr 3 04:08:00 aidas-gsi dhcpd: Remove host declaration nnrpi2 or remove 192.168.11.117 Apr 3 04:08:00 aidas-gsi dhcpd: from the dynamic address pool for 192.168.11.0/24 Apr 3 04:08:00 aidas-gsi dhcpd: uid lease 192.168.11.104 for client b8:27:eb:40:53:e8 is duplicate on 192.168.11.0/24 Apr 3 04:08:00 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.117 from b8:27:eb:40:53:e8 via p4p1 Apr 3 04:08:00 aidas-gsi dhcpd: DHCPACK on 192.168.11.117 to b8:27:eb:40:53:e8 via p4p1 Apr 3 04:36:05 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.251 from b8:27:eb:bb:46:7b via p4p1 Apr 3 04:36:05 aidas-gsi dhcpd: DHCPACK on 192.168.11.251 to b8:27:eb:bb:46:7b via p4p1 Apr 3 04:38:00 aidas-gsi dhcpd: Dynamic and static leases present for 192.168.11.117. Apr 3 04:38:00 aidas-gsi dhcpd: Remove host declaration nnrpi2 or remove 192.168.11.117 Apr 3 04:38:00 aidas-gsi dhcpd: from the dynamic address pool for 192.168.11.0/24 Apr 3 04:38:00 aidas-gsi dhcpd: uid lease 192.168.11.104 for client b8:27:eb:40:53:e8 is duplicate on 192.168.11.0/24 Apr 3 04:38:00 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.117 from b8:27:eb:40:53:e8 via p4p1 Apr 3 04:38:00 aidas-gsi dhcpd: DHCPACK on 192.168.11.117 to b8:27:eb:40:53:e8 via p4p1 Apr 3 04:54:49 aidas-gsi dhclient[3982]: DHCPREQUEST on eth0 to 10.99.0.13 port 67 (xid=0x2d1bd666) Apr 3 04:54:49 aidas-gsi dhclient[3982]: DHCPACK from 10.99.0.13 (xid=0x2d1bd666) Apr 3 04:54:49 aidas-gsi dhclient[3982]: bound to 10.99.2.56 -- renewal in 2761 seconds. Apr 3 04:54:49 aidas-gsi NetworkManager[3917]: <info> (eth0): DHCPv4 state changed renew -> renew Apr 3 04:54:49 aidas-gsi NetworkManager[3917]: <info> address 10.99.2.56 Apr 3 04:54:49 aidas-gsi NetworkManager[3917]: <info> prefix 17 (255.255.128.0) Apr 3 04:54:49 aidas-gsi NetworkManager[3917]: <info> gateway 10.99.0.1 Apr 3 04:54:49 aidas-gsi NetworkManager[3917]: <info> hostname 'aida-3' Apr 3 04:54:49 aidas-gsi NetworkManager[3917]: <info> nameserver '140.181.96.11' Apr 3 04:54:49 aidas-gsi NetworkManager[3917]: <info> nameserver '140.181.96.29' Apr 3 04:54:49 aidas-gsi NetworkManager[3917]: <info> domain name 'gsi.de' Apr 3 05:06:05 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.251 from b8:27:eb:bb:46:7b via p4p1 Apr 3 05:06:05 aidas-gsi dhcpd: DHCPACK on 192.168.11.251 to b8:27:eb:bb:46:7b via p4p1 Apr 3 05:07:22 aidas-gsi yum[25016]: Updated: tzdata-2019a-1.el6.noarch Apr 3 05:07:23 aidas-gsi yum[25016]: Updated: tzdata-java-2019a-1.el6.noarch Apr 3 05:08:00 aidas-gsi dhcpd: Dynamic and static leases present for 192.168.11.117. Apr 3 05:08:00 aidas-gsi dhcpd: Remove host declaration nnrpi2 or remove 192.168.11.117 Apr 3 05:08:00 aidas-gsi dhcpd: from the dynamic address pool for 192.168.11.0/24 Apr 3 05:08:00 aidas-gsi dhcpd: uid lease 192.168.11.104 for client b8:27:eb:40:53:e8 is duplicate on 192.168.11.0/24 Apr 3 05:08:00 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.117 from b8:27:eb:40:53:e8 via p4p1 Apr 3 05:08:00 aidas-gsi dhcpd: DHCPACK on 192.168.11.117 to b8:27:eb:40:53:e8 via p4p1 Apr 3 05:36:05 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.251 from b8:27:eb:bb:46:7b via p4p1 Apr 3 05:36:05 aidas-gsi dhcpd: DHCPACK on 192.168.11.251 to b8:27:eb:bb:46:7b via p4p1 Apr 3 05:38:00 aidas-gsi dhcpd: Dynamic and static leases present for 192.168.11.117. Apr 3 05:38:00 aidas-gsi dhcpd: Remove host declaration nnrpi2 or remove 192.168.11.117 Apr 3 05:38:00 aidas-gsi dhcpd: from the dynamic address pool for 192.168.11.0/24 Apr 3 05:38:00 aidas-gsi dhcpd: uid lease 192.168.11.104 for client b8:27:eb:40:53:e8 is duplicate on 192.168.11.0/24 Apr 3 05:38:00 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.117 from b8:27:eb:40:53:e8 via p4p1 Apr 3 05:38:00 aidas-gsi dhcpd: DHCPACK on 192.168.11.117 to b8:27:eb:40:53:e8 via p4p1 Apr 3 05:40:50 aidas-gsi dhclient[3982]: DHCPREQUEST on eth0 to 10.99.0.13 port 67 (xid=0x2d1bd666) Apr 3 05:40:50 aidas-gsi dhclient[3982]: DHCPACK from 10.99.0.13 (xid=0x2d1bd666) Apr 3 05:40:50 aidas-gsi dhclient[3982]: bound to 10.99.2.56 -- renewal in 3003 seconds. Apr 3 05:40:50 aidas-gsi NetworkManager[3917]: <info> (eth0): DHCPv4 state changed renew -> renew Apr 3 05:40:50 aidas-gsi NetworkManager[3917]: <info> address 10.99.2.56 Apr 3 05:40:50 aidas-gsi NetworkManager[3917]: <info> prefix 17 (255.255.128.0) Apr 3 05:40:50 aidas-gsi NetworkManager[3917]: <info> gateway 10.99.0.1 Apr 3 05:40:50 aidas-gsi NetworkManager[3917]: <info> hostname 'aida-3' Apr 3 05:40:50 aidas-gsi NetworkManager[3917]: <info> nameserver '140.181.96.11' Apr 3 05:40:50 aidas-gsi NetworkManager[3917]: <info> nameserver '140.181.96.29' Apr 3 05:40:50 aidas-gsi NetworkManager[3917]: <info> domain name 'gsi.de' Apr 3 06:06:05 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.251 from b8:27:eb:bb:46:7b via p4p1 Apr 3 06:06:05 aidas-gsi dhcpd: DHCPACK on 192.168.11.251 to b8:27:eb:bb:46:7b via p4p1 Apr 3 06:08:00 aidas-gsi dhcpd: Dynamic and static leases present for 192.168.11.117. Apr 3 06:08:00 aidas-gsi dhcpd: Remove host declaration nnrpi2 or remove 192.168.11.117 Apr 3 06:08:00 aidas-gsi dhcpd: from the dynamic address pool for 192.168.11.0/24 Apr 3 06:08:00 aidas-gsi dhcpd: uid lease 192.168.11.104 for client b8:27:eb:40:53:e8 is duplicate on 192.168.11.0/24 Apr 3 06:08:00 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.117 from b8:27:eb:40:53:e8 via p4p1 Apr 3 06:08:00 aidas-gsi dhcpd: DHCPACK on 192.168.11.117 to b8:27:eb:40:53:e8 via p4p1 Apr 3 06:30:53 aidas-gsi dhclient[3982]: DHCPREQUEST on eth0 to 10.99.0.13 port 67 (xid=0x2d1bd666) Apr 3 06:30:53 aidas-gsi dhclient[3982]: DHCPACK from 10.99.0.13 (xid=0x2d1bd666) Apr 3 06:30:53 aidas-gsi dhclient[3982]: bound to 10.99.2.56 -- renewal in 3269 seconds. Apr 3 06:30:53 aidas-gsi NetworkManager[3917]: <info> (eth0): DHCPv4 state changed renew -> renew Apr 3 06:30:53 aidas-gsi NetworkManager[3917]: <info> address 10.99.2.56 Apr 3 06:30:53 aidas-gsi NetworkManager[3917]: <info> prefix 17 (255.255.128.0) Apr 3 06:30:53 aidas-gsi NetworkManager[3917]: <info> gateway 10.99.0.1 Apr 3 06:30:53 aidas-gsi NetworkManager[3917]: <info> hostname 'aida-3' Apr 3 06:30:53 aidas-gsi NetworkManager[3917]: <info> nameserver '140.181.96.11' Apr 3 06:30:53 aidas-gsi NetworkManager[3917]: <info> nameserver '140.181.96.29' Apr 3 06:30:53 aidas-gsi NetworkManager[3917]: <info> domain name 'gsi.de' Apr 3 06:36:05 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.251 from b8:27:eb:bb:46:7b via p4p1 Apr 3 06:36:05 aidas-gsi dhcpd: DHCPACK on 192.168.11.251 to b8:27:eb:bb:46:7b via p4p1 Apr 3 06:38:00 aidas-gsi dhcpd: Dynamic and static leases present for 192.168.11.117. Apr 3 06:38:00 aidas-gsi dhcpd: Remove host declaration nnrpi2 or remove 192.168.11.117 Apr 3 06:38:00 aidas-gsi dhcpd: from the dynamic address pool for 192.168.11.0/24 Apr 3 06:38:00 aidas-gsi dhcpd: uid lease 192.168.11.104 for client b8:27:eb:40:53:e8 is duplicate on 192.168.11.0/24 Apr 3 06:38:00 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.117 from b8:27:eb:40:53:e8 via p4p1 Apr 3 06:38:00 aidas-gsi dhcpd: DHCPACK on 192.168.11.117 to b8:27:eb:40:53:e8 via p4p1 Apr 3 07:06:05 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.251 from b8:27:eb:bb:46:7b via p4p1 Apr 3 07:06:05 aidas-gsi dhcpd: DHCPACK on 192.168.11.251 to b8:27:eb:bb:46:7b via p4p1 Apr 3 07:08:00 aidas-gsi dhcpd: Dynamic and static leases present for 192.168.11.117. Apr 3 07:08:00 aidas-gsi dhcpd: Remove host declaration nnrpi2 or remove 192.168.11.117 Apr 3 07:08:00 aidas-gsi dhcpd: from the dynamic address pool for 192.168.11.0/24 Apr 3 07:08:00 aidas-gsi dhcpd: uid lease 192.168.11.104 for client b8:27:eb:40:53:e8 is duplicate on 192.168.11.0/24 Apr 3 07:08:00 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.117 from b8:27:eb:40:53:e8 via p4p1 Apr 3 07:08:00 aidas-gsi dhcpd: DHCPACK on 192.168.11.117 to b8:27:eb:40:53:e8 via p4p1 Apr 3 07:25:22 aidas-gsi dhclient[3982]: DHCPREQUEST on eth0 to 10.99.0.13 port 67 (xid=0x2d1bd666) Apr 3 07:25:22 aidas-gsi dhclient[3982]: DHCPACK from 10.99.0.13 (xid=0x2d1bd666) Apr 3 07:25:22 aidas-gsi dhclient[3982]: bound to 10.99.2.56 -- renewal in 2773 seconds. Apr 3 07:25:22 aidas-gsi NetworkManager[3917]: <info> (eth0): DHCPv4 state changed renew -> renew Apr 3 07:25:22 aidas-gsi NetworkManager[3917]: <info> address 10.99.2.56 Apr 3 07:25:22 aidas-gsi NetworkManager[3917]: <info> prefix 17 (255.255.128.0) Apr 3 07:25:22 aidas-gsi NetworkManager[3917]: <info> gateway 10.99.0.1 Apr 3 07:25:22 aidas-gsi NetworkManager[3917]: <info> hostname 'aida-3' Apr 3 07:25:22 aidas-gsi NetworkManager[3917]: <info> nameserver '140.181.96.11' Apr 3 07:25:22 aidas-gsi NetworkManager[3917]: <info> nameserver '140.181.96.29' Apr 3 07:25:22 aidas-gsi NetworkManager[3917]: <info> domain name 'gsi.de' Apr 3 07:36:05 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.251 from b8:27:eb:bb:46:7b via p4p1 Apr 3 07:36:05 aidas-gsi dhcpd: DHCPACK on 192.168.11.251 to b8:27:eb:bb:46:7b via p4p1 Apr 3 07:38:00 aidas-gsi dhcpd: Dynamic and static leases present for 192.168.11.117. Apr 3 07:38:00 aidas-gsi dhcpd: Remove host declaration nnrpi2 or remove 192.168.11.117 Apr 3 07:38:00 aidas-gsi dhcpd: from the dynamic address pool for 192.168.11.0/24 Apr 3 07:38:00 aidas-gsi dhcpd: uid lease 192.168.11.104 for client b8:27:eb:40:53:e8 is duplicate on 192.168.11.0/24 Apr 3 07:38:00 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.117 from b8:27:eb:40:53:e8 via p4p1 Apr 3 07:38:00 aidas-gsi dhcpd: DHCPACK on 192.168.11.117 to b8:27:eb:40:53:e8 via p4p1 Apr 3 08:06:05 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.251 from b8:27:eb:bb:46:7b via p4p1 Apr 3 08:06:05 aidas-gsi dhcpd: DHCPACK on 192.168.11.251 to b8:27:eb:bb:46:7b via p4p1 Apr 3 08:08:00 aidas-gsi dhcpd: Dynamic and static leases present for 192.168.11.117. Apr 3 08:08:00 aidas-gsi dhcpd: Remove host declaration nnrpi2 or remove 192.168.11.117 Apr 3 08:08:00 aidas-gsi dhcpd: from the dynamic address pool for 192.168.11.0/24 Apr 3 08:08:00 aidas-gsi dhcpd: uid lease 192.168.11.104 for client b8:27:eb:40:53:e8 is duplicate on 192.168.11.0/24 Apr 3 08:08:00 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.117 from b8:27:eb:40:53:e8 via p4p1 Apr 3 08:08:00 aidas-gsi dhcpd: DHCPACK on 192.168.11.117 to b8:27:eb:40:53:e8 via p4p1 Apr 3 08:11:35 aidas-gsi dhclient[3982]: DHCPREQUEST on eth0 to 10.99.0.13 port 67 (xid=0x2d1bd666) Apr 3 08:11:35 aidas-gsi dhclient[3982]: DHCPACK from 10.99.0.13 (xid=0x2d1bd666) Apr 3 08:11:35 aidas-gsi dhclient[3982]: bound to 10.99.2.56 -- renewal in 3455 seconds. Apr 3 08:11:35 aidas-gsi NetworkManager[3917]: <info> (eth0): DHCPv4 state changed renew -> renew Apr 3 08:11:35 aidas-gsi NetworkManager[3917]: <info> address 10.99.2.56 Apr 3 08:11:35 aidas-gsi NetworkManager[3917]: <info> prefix 17 (255.255.128.0) Apr 3 08:11:35 aidas-gsi NetworkManager[3917]: <info> gateway 10.99.0.1 Apr 3 08:11:35 aidas-gsi NetworkManager[3917]: <info> hostname 'aida-3' Apr 3 08:11:35 aidas-gsi NetworkManager[3917]: <info> nameserver '140.181.96.11' Apr 3 08:11:35 aidas-gsi NetworkManager[3917]: <info> nameserver '140.181.96.29' Apr 3 08:11:35 aidas-gsi NetworkManager[3917]: <info> domain name 'gsi.de' Apr 3 08:36:05 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.251 from b8:27:eb:bb:46:7b via p4p1 Apr 3 08:36:05 aidas-gsi dhcpd: DHCPACK on 192.168.11.251 to b8:27:eb:bb:46:7b via p4p1 Apr 3 08:38:00 aidas-gsi dhcpd: Dynamic and static leases present for 192.168.11.117. Apr 3 08:38:00 aidas-gsi dhcpd: Remove host declaration nnrpi2 or remove 192.168.11.117 Apr 3 08:38:00 aidas-gsi dhcpd: from the dynamic address pool for 192.168.11.0/24 Apr 3 08:38:00 aidas-gsi dhcpd: uid lease 192.168.11.104 for client b8:27:eb:40:53:e8 is duplicate on 192.168.11.0/24 Apr 3 08:38:00 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.117 from b8:27:eb:40:53:e8 via p4p1 Apr 3 08:38:00 aidas-gsi dhcpd: DHCPACK on 192.168.11.117 to b8:27:eb:40:53:e8 via p4p1 Apr 3 09:06:05 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.251 from b8:27:eb:bb:46:7b via p4p1 Apr 3 09:06:05 aidas-gsi dhcpd: DHCPACK on 192.168.11.251 to b8:27:eb:bb:46:7b via p4p1 Apr 3 09:08:00 aidas-gsi dhcpd: Dynamic and static leases present for 192.168.11.117. Apr 3 09:08:00 aidas-gsi dhcpd: Remove host declaration nnrpi2 or remove 192.168.11.117 Apr 3 09:08:00 aidas-gsi dhcpd: from the dynamic address pool for 192.168.11.0/24 Apr 3 09:08:00 aidas-gsi dhcpd: uid lease 192.168.11.104 for client b8:27:eb:40:53:e8 is duplicate on 192.168.11.0/24 Apr 3 09:08:00 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.117 from b8:27:eb:40:53:e8 via p4p1 Apr 3 09:08:00 aidas-gsi dhcpd: DHCPACK on 192.168.11.117 to b8:27:eb:40:53:e8 via p4p1 Apr 3 09:09:10 aidas-gsi dhclient[3982]: DHCPREQUEST on eth0 to 10.99.0.13 port 67 (xid=0x2d1bd666) Apr 3 09:09:10 aidas-gsi dhclient[3982]: DHCPACK from 10.99.0.13 (xid=0x2d1bd666) Apr 3 09:09:10 aidas-gsi dhclient[3982]: bound to 10.99.2.56 -- renewal in 2865 seconds. Apr 3 09:09:10 aidas-gsi NetworkManager[3917]: <info> (eth0): DHCPv4 state changed renew -> renew Apr 3 09:09:10 aidas-gsi NetworkManager[3917]: <info> address 10.99.2.56 Apr 3 09:09:10 aidas-gsi NetworkManager[3917]: <info> prefix 17 (255.255.128.0) Apr 3 09:09:10 aidas-gsi NetworkManager[3917]: <info> gateway 10.99.0.1 Apr 3 09:09:10 aidas-gsi NetworkManager[3917]: <info> hostname 'aida-3' Apr 3 09:09:10 aidas-gsi NetworkManager[3917]: <info> nameserver '140.181.96.11' Apr 3 09:09:10 aidas-gsi NetworkManager[3917]: <info> nameserver '140.181.96.29' Apr 3 09:09:10 aidas-gsi NetworkManager[3917]: <info> domain name 'gsi.de' Apr 3 09:36:05 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.251 from b8:27:eb:bb:46:7b via p4p1 Apr 3 09:36:05 aidas-gsi dhcpd: DHCPACK on 192.168.11.251 to b8:27:eb:bb:46:7b via p4p1 Apr 3 09:38:00 aidas-gsi dhcpd: Dynamic and static leases present for 192.168.11.117. Apr 3 09:38:00 aidas-gsi dhcpd: Remove host declaration nnrpi2 or remove 192.168.11.117 Apr 3 09:38:00 aidas-gsi dhcpd: from the dynamic address pool for 192.168.11.0/24 Apr 3 09:38:00 aidas-gsi dhcpd: uid lease 192.168.11.104 for client b8:27:eb:40:53:e8 is duplicate on 192.168.11.0/24 Apr 3 09:38:00 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.117 from b8:27:eb:40:53:e8 via p4p1 Apr 3 09:38:00 aidas-gsi dhcpd: DHCPACK on 192.168.11.117 to b8:27:eb:40:53:e8 via p4p1 Apr 3 09:53:59 aidas-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:f6:b7 via p4p1 Apr 3 09:53:59 aidas-gsi dhcpd: DHCPOFFER on 192.168.11.4 to d8:80:39:41:f6:b7 via p4p1 Apr 3 09:53:59 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.4 (192.168.11.99) from d8:80:39:41:f6:b7 via p4p1 Apr 3 09:53:59 aidas-gsi dhcpd: DHCPACK on 192.168.11.4 to d8:80:39:41:f6:b7 via p4p1 Apr 3 09:53:59 aidas-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:d8:21 via p4p1 Apr 3 09:53:59 aidas-gsi dhcpd: DHCPOFFER on 192.168.11.3 to d8:80:39:41:d8:21 via p4p1 Apr 3 09:53:59 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.3 (192.168.11.99) from d8:80:39:41:d8:21 via p4p1 Apr 3 09:53:59 aidas-gsi dhcpd: DHCPACK on 192.168.11.3 to d8:80:39:41:d8:21 via p4p1 Apr 3 09:53:59 aidas-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:d7:cc via p4p1 Apr 3 09:53:59 aidas-gsi dhcpd: DHCPOFFER on 192.168.11.5 to d8:80:39:41:d7:cc via p4p1 Apr 3 09:53:59 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.5 (192.168.11.99) from d8:80:39:41:d7:cc via p4p1 Apr 3 09:53:59 aidas-gsi dhcpd: DHCPACK on 192.168.11.5 to d8:80:39:41:d7:cc via p4p1 Apr 3 09:53:59 aidas-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:ba:2b via p4p1 Apr 3 09:53:59 aidas-gsi dhcpd: DHCPOFFER on 192.168.11.8 to d8:80:39:41:ba:2b via p4p1 Apr 3 09:53:59 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.8 (192.168.11.99) from d8:80:39:41:ba:2b via p4p1 Apr 3 09:53:59 aidas-gsi dhcpd: DHCPACK on 192.168.11.8 to d8:80:39:41:ba:2b via p4p1 Apr 3 09:53:59 aidas-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:ba:8a via p4p1 Apr 3 09:53:59 aidas-gsi dhcpd: DHCPOFFER on 192.168.11.1 to d8:80:39:41:ba:8a via p4p1 Apr 3 09:53:59 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.1 (192.168.11.99) from d8:80:39:41:ba:8a via p4p1 Apr 3 09:53:59 aidas-gsi dhcpd: DHCPACK on 192.168.11.1 to d8:80:39:41:ba:8a via p4p1 Apr 3 09:53:59 aidas-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:ba:22 via p4p1 Apr 3 09:53:59 aidas-gsi dhcpd: DHCPOFFER on 192.168.11.2 to d8:80:39:41:ba:22 via p4p1 Apr 3 09:53:59 aidas-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:ee:72 via p4p1 Apr 3 09:53:59 aidas-gsi dhcpd: DHCPOFFER on 192.168.11.6 to d8:80:39:41:ee:72 via p4p1 Apr 3 09:53:59 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.6 (192.168.11.99) from d8:80:39:41:ee:72 via p4p1 Apr 3 09:53:59 aidas-gsi dhcpd: DHCPACK on 192.168.11.6 to d8:80:39:41:ee:72 via p4p1 Apr 3 09:53:59 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.2 (192.168.11.99) from d8:80:39:41:ba:22 via p4p1 Apr 3 09:53:59 aidas-gsi dhcpd: DHCPACK on 192.168.11.2 to d8:80:39:41:ba:22 via p4p1 Apr 3 09:54:00 aidas-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:b4:0c via p4p1 Apr 3 09:54:00 aidas-gsi dhcpd: DHCPOFFER on 192.168.11.7 to d8:80:39:41:b4:0c via p4p1 Apr 3 09:54:00 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.7 (192.168.11.99) from d8:80:39:41:b4:0c via p4p1 Apr 3 09:54:00 aidas-gsi dhcpd: DHCPACK on 192.168.11.7 to d8:80:39:41:b4:0c via p4p1 Apr 3 09:54:00 aidas-gsi rpc.mountd[4324]: authenticated mount request from 192.168.11.4:666 for /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida04 (/home/Embedded/XilinxLinux/ppc_4xx/rfs) Apr 3 09:54:00 aidas-gsi rpc.mountd[4324]: authenticated mount request from 192.168.11.3:1013 for /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida03 (/home/Embedded/XilinxLinux/ppc_4xx/rfs) Apr 3 09:54:00 aidas-gsi rpc.mountd[4324]: authenticated mount request from 192.168.11.5:904 for /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida05 (/home/Embedded/XilinxLinux/ppc_4xx/rfs) Apr 3 09:54:00 aidas-gsi rpc.mountd[4324]: authenticated mount request from 192.168.11.8:890 for /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida08 (/home/Embedded/XilinxLinux/ppc_4xx/rfs) Apr 3 09:54:00 aidas-gsi rpc.mountd[4324]: authenticated mount request from 192.168.11.1:1007 for /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida01 (/home/Embedded/XilinxLinux/ppc_4xx/rfs) Apr 3 09:54:00 aidas-gsi rpc.mountd[4324]: authenticated mount request from 192.168.11.6:710 for /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida06 (/home/Embedded/XilinxLinux/ppc_4xx/rfs) Apr 3 09:54:00 aidas-gsi rpc.mountd[4324]: authenticated mount request from 192.168.11.2:727 for /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida02 (/home/Embedded/XilinxLinux/ppc_4xx/rfs) Apr 3 09:54:00 aidas-gsi rpc.mountd[4324]: authenticated mount request from 192.168.11.7:698 for /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida07 (/home/Embedded/XilinxLinux/ppc_4xx/rfs) Apr 3 09:54:00 aidas-gsi dhcpd: DHCPDISCOVER from d8:80:39:42:0d:0c via p4p1 Apr 3 09:54:00 aidas-gsi dhcpd: DHCPOFFER on 192.168.11.9 to d8:80:39:42:0d:0c via p4p1 Apr 3 09:54:00 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.9 (192.168.11.99) from d8:80:39:42:0d:0c via p4p1 Apr 3 09:54:00 aidas-gsi dhcpd: DHCPACK on 192.168.11.9 to d8:80:39:42:0d:0c via p4p1 Apr 3 09:54:00 aidas-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:cf:ac via p4p1 Apr 3 09:54:00 aidas-gsi dhcpd: DHCPOFFER on 192.168.11.12 to d8:80:39:41:cf:ac via p4p1 Apr 3 09:54:00 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.12 (192.168.11.99) from d8:80:39:41:cf:ac via p4p1 Apr 3 09:54:00 aidas-gsi dhcpd: DHCPACK on 192.168.11.12 to d8:80:39:41:cf:ac via p4p1 Apr 3 09:54:00 aidas-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:f6:5a via p4p1 Apr 3 09:54:00 aidas-gsi dhcpd: DHCPOFFER on 192.168.11.11 to d8:80:39:41:f6:5a via p4p1 Apr 3 09:54:00 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.11 (192.168.11.99) from d8:80:39:41:f6:5a via p4p1 Apr 3 09:54:00 aidas-gsi dhcpd: DHCPACK on 192.168.11.11 to d8:80:39:41:f6:5a via p4p1 Apr 3 09:54:00 aidas-gsi dhcpd: DHCPDISCOVER from d8:80:39:41:ee:10 via p4p1 Apr 3 09:54:00 aidas-gsi dhcpd: DHCPOFFER on 192.168.11.10 to d8:80:39:41:ee:10 via p4p1 Apr 3 09:54:00 aidas-gsi dhcpd: DHCPREQUEST for 192.168.11.10 (192.168.11.99) from d8:80:39:41:ee:10 via p4p1 Apr 3 09:54:00 aidas-gsi dhcpd: DHCPACK on 192.168.11.10 to d8:80:39:41:ee:10 via p4p1 Apr 3 09:54:01 aidas-gsi rpc.mountd[4324]: authenticated mount request from 192.168.11.9:907 for /home/Embedded/XilinxLinux/ppc_4xx/rfs/aida09 (/home/Embedded/XilinxLinux/ppc_4xx/rfs) ... 190 more lines ...
Attachment 2: 030419_usb.png

Attachment 3: WRerrorcheck.png

Attachment 4: 030419_1051_temp.png

Attachment 5: 030419_1052)bias.png

Attachment 6: 030519_asiccontrol.png

Attachment 7: 030419_runcontrol.png

Attachment 8: 030419_1116_stats.png

Attachment 9: 030419_runcontrol_1120.png

Attachment 10: 030419_1343_stats.png

Attachment 11: 030419_1652_stats10.png

406	Wed Mar 9 10:15:56 2022	NH TD	Wednesday 9 March - noise tests contd.
11.15 - S4 is currently closed no access If access possible (should be) plan is Re-attach ribbon cable braids Re-attach DSSSD middle wafer rear g/r (LK3) Re-attach DSSSD ground (LK2?) See difference in rates, expectation is braid should restore most p+n rates Once status quo ante achieved explore ASIC settings to identify preamp feedback and shaping time options to stabilise preamp output and optimise signal:noise. aida12 Peak width = 84.73 aida02 Peak width = 526.68 Attachment 1 - Stats Attachment 2 - Per FEE Rates Attachments 3,4 - Waveforms Attachments 5,6 - Pulser Peaks Attachment 7 - HV Bias. Leakage current recovered and does not seem to be ramping up 16.00 - Attach LK3 jumpers to aidas 1, 3, 5, 7 (DSSD ground and middle rear g/r) Attachment 8 - Stats Attachment 9 - Per FEE Rates Attachment 10, 11 - Waveforms Attachment 12, 13 - Pulser Peaks aida12 Peak width = 78.51 aida02 Peak width = 480.90 No major change, FEES 1-8 are all noisy and FEES 9-16 are quiet aida01, aida05 waveforms very noticably bad (100 kHz), aida03, aida07 waveforms seem OK 16.45 - Attach LK2, LK4 jumpers to all FEEs (field plane grounding) No significant changes Attachment 14 - Stats Attachment 15 - Per FEE Rates Attachment 16, 17 - Pulser Peaks Attachment 18, 19 - Waveforms aida12 Peak width = 71.28 aida02 Peak width = 458.18 Remove LK3 jumpers aida12 Peak width = 71.43 aida02 Peak width = 496.05 Attachment 20 - Stats Attachment 21 - Per FEE Rates Attachment 22, 23 - Pulser Peaks Attachment 24, 25 - Waveforms Re-add jumpers - attachments 26-28 aida12 Peak width = 87.42 aida02 Peak width = 572.24 17.45 Directly bias from the p+n FEEs using SHV->BNC->Lemo - attachments 29-33 aida12 Peak width = 87.87 aida02 Peak width = 504.41 The correlation between high 'ADC data item' rate and the noise observed on the corresponding 1.8.W spectrum is not clear except for aida01 & aida05 - see also attachments 34-35 ADC data item stats - attachment 36 DSSSD bias & leakage current OK - attachment 37 FEE64 temperatures OK - attachment 38 all system wide checks OK *except aida11 clock fail status 6
Attachment 1: 2022-03-09_15-37-08_stats.png

Attachment 2: 2022-03-09_15-37-33_rates.png

Attachment 3: 2022-03-09_15-38-18_pn_waves.png

Attachment 4: 022-03-09_15-38-59_nn_waves.png

Attachment 5: 2022-03-09_15-40-23_pn_widths.png

Attachment 6: 2022-03-09_15-41-26_nn_widths.png

Attachment 7: 2022-03-09_15-42-55_hv_bias.png

Attachment 8: 2022-03-09_16-03-05_stats.png

Attachment 9: 2022-03-09_16-06-30_rates.png

Attachment 10: 2022-03-09_16-07-43_pn_waves.png

Attachment 11: 2022-03-09_16-08-12_nn_waves.png

Attachment 12: 2022-03-09_16-10-10_pn_widths.png

Attachment 13: 2022-03-09_16-11-14_nn_widths.png

Attachment 14: 2022-03-09_16-40-33_stats.png

Attachment 15: 2022-03-09_16-42-44_rates.png

Attachment 16: 2022-03-09_16-43-46_pn_widths.png

Attachment 17: 2022-03-09_16-44-46_nn_widths.png

Attachment 18: 2022-03-09_16-45-39_pn_waves.png

Attachment 19: 2022-03-09_16-46-08_nn_waves.png

Attachment 20: 2022-03-09_17-03-33_stats.png

Attachment 21: 2022-03-09_17-04-22_rates.png

Attachment 22: 2022-03-09_17-07-44_nn_widths.png

Attachment 23: 2022-03-09_17-06-40_pn_widths.png

Attachment 24: 2022-03-09_17-05-41_nn_waves.png

Attachment 25: 2022-03-09_17-05-11_pn_waves.png

Attachment 26: 2022-03-09_17-26-03_nn_widths.png

Attachment 27: 2022-03-09_17-24-58_pn_widths.png

Attachment 28: 2022-03-09_17-23-57_stats.png

Attachment 29: 2022-03-09_17-43-33nn_widths.png

Attachment 30: 2022-03-09_17-42-29_pn_widths.png

Attachment 31: 2022-03-09_17-41-23_stats.png

Attachment 32: 2022-03-09_17-45-55_nn_waves.png

Attachment 33: 2022-03-09_17-45-27_pn_waves.png

Attachment 34: Screenshot_from_2022-03-09_18-12-13.png

Attachment 35: Screenshot_from_2022-03-09_18-11-32.png

Attachment 36: Screenshot_from_2022-03-09_18-27-23.png

Attachment 37: Screenshot_from_2022-03-09_18-32-13.png

Attachment 38: Screenshot_from_2022-03-09_18-35-24.png

361	Wed Jun 9 11:27:47 2021	TD	Wednesday 9 June
12.30 DAQ continues OK file S496/R32_373 System wide checks OK except Base Current Difference aida01 fault 0x8944 : 0x8947 : 3 aida02 fault 0x9b36 : 0x9b39 : 3 aida03 fault 0xf26b : 0xf26e : 3 aida04 fault 0x57a0 : 0x57a3 : 3 aida05 fault 0x4d3c : 0x4d3d : 1 aida05 : WR status 0x10 aida06 fault 0x640c : 0x640d : 1 aida07 fault 0x255e : 0x255f : 1 aida08 fault 0xffbd : 0xffbe : 1 White Rabbit error counter test result: Passed 8, Failed 8 Understand the status reports as follows:- Status bit 3 : White Rabbit decoder detected an error in the received data Status bit 2 : Firmware registered WR error, no reload of Timestamp Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k aida01 : 13 5 12 7 2 4 2 2 1 4 7 : 40572 aida02 : 11 6 8 6 1 3 3 2 1 4 7 : 40540 aida03 : 26 5 11 5 1 2 2 3 2 3 7 : 39712 aida04 : 18 7 8 3 2 3 2 3 2 3 7 : 39776 aida05 : 11 5 9 4 1 2 2 3 1 4 7 : 40612 aida06 : 17 3 6 5 5 2 1 3 1 4 7 : 40604 aida07 : 9 3 4 9 1 3 3 2 1 4 7 : 40540 aida08 : 11 8 5 5 3 3 3 2 1 4 7 : 40604 aida09 : 13 5 8 5 2 3 3 2 1 4 7 : 40572 aida10 : 13 8 13 5 1 2 2 2 1 4 7 : 40228 aida11 : 11 5 14 6 3 3 3 3 2 3 7 : 40244 aida12 : 10 9 5 5 3 3 2 2 1 4 7 : 40352 aida13 : 15 7 7 4 3 4 3 3 2 3 7 : 40228 aida14 : 9 6 7 6 3 4 3 3 2 3 7 : 40260 aida15 : 14 3 8 5 1 2 2 4 1 4 7 : 41136 aida16 : 15 4 9 4 4 4 3 3 2 3 7 : 40300 Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same FEE : aida01 => Options file size is 1026 Last changed Fri Jun 04 09:30:42 CEST 2021 FEE : aida02 => Options file size is 1025 Last changed Sun May 23 00:19:21 CEST 2021 FEE : aida03 => Options file size is 1014 Last changed Thu Apr 29 14:43:50 CEST 2021 FEE : aida04 => Options file size is 1025 Last changed Fri May 14 16:54:56 CEST 2021 FEE : aida05 => Options file size is 1025 Last changed Mon May 17 06:25:41 CEST 2021 FEE : aida06 => Options file size is 1014 Last changed Thu Apr 29 14:43:59 CEST 2021 FEE : aida07 => Options file size is 1014 Last changed Thu Apr 29 14:44:02 CEST 2021 FEE : aida08 => Options file size is 1025 Last changed Sun May 23 00:16:54 CEST 2021 FEE : aida09 => Options file size is 1014 Last changed Thu Apr 29 14:44:08 CEST 2021 FEE : aida10 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021 FEE : aida11 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021 FEE : aida12 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021 FEE : aida13 => Options file size is 1025 Last changed Fri May 07 19:40:34 CEST 2021 FEE : aida14 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021 FEE : aida15 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021 FEE : aida16 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021 1.8.L specttra - attachments 1 & 2 Grafana - DSSSD bias & leakage current - most recent 7 days - attachment 3 Lost activity monitor - attachment 4 1.8.W spectra - 20us FSR - attachments 5 & 6 ADC data items - attachment 7 FEE64 temperatures OK - attachment 8 DSSSD bias & leakage currents OK - attachment 9 Merger/Tape Server/Merger statistics - attachment 10 no merger errors reported since previous restart 16.28 analysis of file S496/R32_372 - attachment 11 max deadtime 0.07% aida04
Attachment 1: Screenshot_2021-06-09_Spectrum_Browser_aidas-gsi(3).png

Attachment 2: Screenshot_2021-06-09_Spectrum_Browser_aidas-gsi(2).png

Attachment 3: Screenshot_2021-06-09_AIDA_Alerting_-_Grafana.png

Attachment 4: Screenshot_2021-06-09_Lost_activity_monitor_aidas-gsi(1).png

Attachment 5: Screenshot_2021-06-09_Spectrum_Browser_aidas-gsi(1).png

Attachment 6: Screenshot_2021-06-09_Spectrum_Browser_aidas-gsi.png

Attachment 7: Screenshot_2021-06-09_Statistics_aidas-gsi.png

Attachment 8: Screenshot_2021-06-09_Temperature_and_status_scan_aidas-gsi.png

Attachment 9: 991.png

Attachment 10: 992.png

Attachment 11: R32_372
* TDR format 3.3.0 analyser - TD - May 2021 * ERROR: READ I/O error: 5002 blocks: 32000 ADC data format: 65571173 ( 25420.1 Hz) Other data format: 196348830 ( 76118.8 Hz) Sample trace data format: 0 ( 0.0 Hz) Undefined format: 0 ( 0.0 Hz) Other data format type: PAUSE: 52 ( 0.0 Hz) RESUME: 52 ( 0.0 Hz) SYNC100: 41608 ( 16.1 Hz) WR48-63: 41608 ( 16.1 Hz) FEE64 disc: 2050316 ( 794.8 Hz) MBS info: 194215194 ( 75291.7 Hz) Other info: 0 ( 0.0 Hz) ADC data range bit set: 284 ( 0.1 Hz) Timewarps: ADC: 0 ( 0.0 Hz) PAUSE: 0 ( 0.0 Hz) RESUME: 0 ( 0.0 Hz) SYNC100: 0 ( 0.0 Hz) WR48-63: 0 ( 0.0 Hz) FEE64 disc: 0 ( 0.0 Hz) MBS info: 0 ( 0.0 Hz) Undefined: 0 ( 0.0 Hz) Sample trace: 0 ( 0.0 Hz) * Timestamp elapsed time: 2579.504 s FEE elapsed dead time(s) elapsed idle time(s) 0 0.000 0.000 1 0.015 1853.986 2 0.548 0.000 3 1.886 0.000 4 0.000 0.000 5 0.000 1457.725 6 0.000 0.000 7 0.000 1391.289 8 0.000 0.000 9 0.000 0.000 10 0.000 0.000 11 0.000 1323.904 12 0.000 0.000 13 0.000 0.000 14 0.000 0.000 15 0.000 836.274 16 0.000 0.000 17 0.000 0.000 18 0.000 0.000 19 0.000 0.000 20 0.000 0.000 21 0.000 0.000 22 0.000 0.000 23 0.000 0.000 24 0.000 0.000 25 0.000 0.000 26 0.000 0.000 27 0.000 0.000 28 0.000 0.000 29 0.000 0.000 30 0.000 0.000 31 0.000 0.000 32 0.000 0.000 * Statistics FEE ADC Data Other Data Sample Undefined Pause Resume SYNC100 WR48-63 Disc MBS Other HEC Data 0 6664 532 0 0 0 0 1 1 530 0 0 3 1 4648488 85466 0 0 1 1 947 947 83570 0 0 92 2 5675 97165745 0 0 27 27 16487 16487 7 97132710 0 2 3 47009038 98898439 0 0 24 24 21346 21346 1773239 97082460 0 42 4 3608 4 0 0 0 0 0 0 4 0 0 1 5 6949043 95991 0 0 0 0 1383 1383 93225 0 0 60 6 5001 30 0 0 0 0 1 1 4 24 0 0 7 6897601 102341 0 0 0 0 1436 1436 99469 0 0 74 8 1797 1 0 0 0 0 0 0 1 0 0 1 9 2911 4 0 0 0 0 1 1 2 0 0 3 10 8722 137 0 0 0 0 1 1 135 0 0 0 11 5985 36 0 0 0 0 2 2 32 0 0 4 12 7238 34 0 0 0 0 1 1 32 0 0 0 13 7345 52 0 0 0 0 0 0 52 0 0 0 14 4898 4 0 0 0 0 0 0 4 0 0 1 15 7159 14 0 0 0 0 2 2 10 0 0 1 16 0 0 0 0 0 0 0 0 0 0 0 0 17 0 0 0 0 0 0 0 0 0 0 0 0 18 0 0 0 0 0 0 0 0 0 0 0 0 19 0 0 0 0 0 0 0 0 0 0 0 0 20 0 0 0 0 0 0 0 0 0 0 0 0 21 0 0 0 0 0 0 0 0 0 0 0 0 22 0 0 0 0 0 0 0 0 0 0 0 0 23 0 0 0 0 0 0 0 0 0 0 0 0 24 0 0 0 0 0 0 0 0 0 0 0 0 25 0 0 0 0 0 0 0 0 0 0 0 0 26 0 0 0 0 0 0 0 0 0 0 0 0 27 0 0 0 0 0 0 0 0 0 0 0 0 28 0 0 0 0 0 0 0 0 0 0 0 0 29 0 0 0 0 0 0 0 0 0 0 0 0 30 0 0 0 0 0 0 0 0 0 0 0 0 31 0 0 0 0 0 0 0 0 0 0 0 0 32 0 0 0 0 0 0 0 0 0 0 0 0 * Timewarps FEE ADC Pause Resume SYNC100 WR48-63 Disc MBS Undefined Samples 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 4 0 0 0 0 0 0 0 0 0 5 0 0 0 0 0 0 0 0 0 6 0 0 0 0 0 0 0 0 0 7 0 0 0 0 0 0 0 0 0 8 0 0 0 0 0 0 0 0 0 9 0 0 0 0 0 0 0 0 0 10 0 0 0 0 0 0 0 0 0 11 0 0 0 0 0 0 0 0 0 12 0 0 0 0 0 0 0 0 0 13 0 0 0 0 0 0 0 0 0 14 0 0 0 0 0 0 0 0 0 15 0 0 0 0 0 0 0 0 0 16 0 0 0 0 0 0 0 0 0 17 0 0 0 0 0 0 0 0 0 18 0 0 0 0 0 0 0 0 0 19 0 0 0 0 0 0 0 0 0 20 0 0 0 0 0 0 0 0 0 21 0 0 0 0 0 0 0 0 0 22 0 0 0 0 0 0 0 0 0 23 0 0 0 0 0 0 0 0 0 24 0 0 0 0 0 0 0 0 0 25 0 0 0 0 0 0 0 0 0 26 0 0 0 0 0 0 0 0 0 27 0 0 0 0 0 0 0 0 0 28 0 0 0 0 0 0 0 0 0 29 0 0 0 0 0 0 0 0 0 30 0 0 0 0 0 0 0 0 0 31 0 0 0 0 0 0 0 0 0 32 0 0 0 0 0 0 0 0 0 * Program elapsed time: 30.055s ( 1064.726 blocks/s, 66.545 Mb/s)
298	Thu May 6 16:59:20 2021	TD, OH	Wednesday 6 May
18.09 NH has installed jumpers LK2-4 for all adaptor PCBs per FEE64 rate spectra - attachments 1-4 attachment 4 with BNC PB-5 pulser 10Hz ADC data items stats - attachment 5 1.8.W spectra - attachments 6 & 7 Observe significant reduction in DSSSD/ribbon cable boundary channel rates - status quo ante Tuesday 4 May https://elog.ph.ed.ac.uk/DESPEC/291 ? 1.8.L spectra - attachments 8 & 9 pulser peak width aida01 85 ch FWHM, aida2 295 ch FWHM 19:21 Alpha run started May21/R6 Statistics dominated by aida04 with a rate of 6k System wide checks all ok except: Base Current Difference aida13 fault 0x0 : 0x17 : 23 FPGA Timestamp error counter test result: Passed 15, Failed 1 If any of these counts are reported as in error The ASIC readout system has detected a timeslip. That is the timestamp read from the time FIFO is not younger than the last Writing to file at ~300kB/s Rate spectra for aida04 shows dominated by first asic - attachment 10 1*L spectra for aida04 asic 1 - attachment 11 20.25 per FEE64 rate spectra - attachment 12 handful of hot channels aida06, 09 & 10 ADC data item stats - attachment 13
Attachment 1: Screenshot_2021-05-06_Spectrum_Browser_aidas-gsi(4).png

Attachment 2: Screenshot_2021-05-06_Spectrum_Browser_aidas-gsi(5).png

Attachment 3: Screenshot_2021-05-06_Spectrum_Browser_aidas-gsi(6).png

Attachment 4: Screenshot_2021-05-06_Spectrum_Browser_aidas-gsi(7).png

Attachment 5: Screenshot_2021-05-06_Statistics_aidas-gsi(1).png

Attachment 6: Screenshot_2021-05-06_Spectrum_Browser_aidas-gsi(8).png

Attachment 7: Screenshot_2021-05-06_Spectrum_Browser_aidas-gsi(9).png

Attachment 8: Screenshot_2021-05-06_Spectrum_Browser_aidas-gsi.png

Attachment 9: Screenshot_2021-05-06_Spectrum_Browser_aidas-gsi(1).png

Attachment 10: 210506_1924_aida04Rate.png

Attachment 11: 210506_1927_1*L.png

Attachment 12: Screenshot_2021-05-06_Spectrum_Browser_aidas-gsi(2).png

Attachment 13: Screenshot_2021-05-06_Statistics_aidas-gsi.png

412	Wed Apr 6 10:09:21 2022	OH, NH, TD	Wednesday 6 April
DSSSD bias OFF. DAQ STOPped Attachment 6 FEE64 temps OK Attachment 1-5 system wide checks OK
Attachment 1: Screenshot_from_2022-04-06_11-12-59.png

Attachment 2: Screenshot_from_2022-04-06_11-12-48.png

Attachment 3: Screenshot_from_2022-04-06_11-12-38.png

Attachment 4: Screenshot_from_2022-04-06_11-10-06.png

Attachment 5: Screenshot_from_2022-04-06_11-09-57.png

Attachment 6: Screenshot_from_2022-04-06_11-09-29.png

297	Wed May 5 20:10:16 2021	OH	Wednesday 5 May Alpha run
21:10 After all of the works in S4 today the noise in the system was considerably worse It was decided that before starting a new alpha run a powercycle would be performed. The powercycle has had no effect on the rates. They are still terrible. The difference between yesterday and today is: - Adaptor cards for FEE9 and 13 were re-aligned as they were previously misaligned - bPlast has been plugged in and biased All system wide checks ok Statistics threshold at 0xa - attachment 1 Rate spectra (FEE5 does have events in all channels just noise is dominated by a few) - attachment 2 Waveforms - attachments 3 and 4 Pulser turned off and threshold raised to 0x64 Statistics - attachment 5 21:19 Alpha run started to May21/R4* 08:51 DAQ continues OK - file May21/R4_23 FEE64 temps OK - attachment 8 ADC data items OK - attachment 9 1.8.W spectra - attachments 6 & 7 no change cf. yesterday pm Grafana - most recent 12h DSSSD bias & leakage current - attachment 10 DSSSD biases & leakage currents - attachment 11 Tape Server/Merger - attachment 12 09:44 R4 stopped
Attachment 1: 210504_2105_Stats.png

Attachment 2: 210504_2107_Layout1.png

Attachment 3: 210504_2108_layout7.png

Attachment 4: 210504_2109_Layout8.png

Attachment 5: 210504_2115_stats.png

Attachment 6: Screenshot_2021-05-06_Spectrum_Browser_aidas-gsi.png

Attachment 7: Screenshot_2021-05-06_Spectrum_Browser_aidas-gsi(1).png

Attachment 8: Screenshot_2021-05-06_Temperature_and_status_scan_aidas-gsi.png

Attachment 9: Screenshot_2021-05-06_Statistics_aidas-gsi.png

Attachment 10: Screenshot_2021-05-06_AIDA_Alerting_-_Grafana.png

Attachment 11: 401.png

Attachment 12: 402.png

285	Wed Apr 28 10:06:11 2021	NH	Wednesday 28 April
11.05 All system wide checks OK except WR Base Current Difference aida01 fault 0xc960 : 0xc971 : 17 aida02 fault 0x8bbf : 0x8bd0 : 17 aida03 fault 0xae9e : 0xaeaf : 17 aida04 fault 0x6e3 : 0x6f4 : 17 aida05 fault 0x31a4 : 0x31bd : 25 aida05 : WR status 0x10 aida06 fault 0x22ea : 0x2303 : 25 aida07 fault 0x438f : 0x43a8 : 25 aida08 fault 0x61ed : 0x6206 : 25 aida09 fault 0x1c32 : 0x1c40 : 14 aida10 fault 0x9332 : 0x933f : 13 aida11 fault 0x227b : 0x2288 : 13 aida12 fault 0xb2fc : 0xb309 : 13 White Rabbit error counter test result: Passed 0, Failed 12 Understand the status reports as follows:- Status bit 3 : White Rabbit decoder detected an error in the received data Status bit 2 : Firmware registered WR error, no reload of Timestamp Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR FPGA Base Current Difference aida01 fault 0x0 : 0x1 : 1 aida02 fault 0x0 : 0x3 : 3 aida03 fault 0x0 : 0x1 : 1 aida04 fault 0x0 : 0x2 : 2 aida06 fault 0x0 : 0x2 : 2 aida08 fault 0x0 : 0x1 : 1 aida11 fault 0x0 : 0x3 : 3 aida12 fault 0x0 : 0x3 : 3 FPGA Timestamp error counter test result: Passed 4, Failed 8 If any of these counts are reported as in error The ASIC readout system has detected a timeslip. That is the timestamp read from the time FIFO is not younger than the last Looks like a big WR glitch happened at some point (master in issue?) Memory Returned 0 0 0 0 0 0 0 0 0 0 0 0 Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k aida01 : 26 20 26 15 3 1 3 2 2 3 4 : 27848 aida02 : 25 7 1 4 1 4 2 2 3 3 6 : 36204 aida03 : 30 5 3 3 3 3 1 3 3 3 6 : 36464 aida04 : 5 10 0 5 2 3 1 3 3 3 6 : 36356 aida05 : 34 40 16 13 4 3 3 2 3 3 4 : 29160 aida06 : 16 8 4 2 1 4 1 3 3 3 6 : 36416 aida07 : 28 23 23 10 4 3 2 4 3 4 3 : 27736 aida08 : 26 9 3 3 1 3 1 3 3 3 6 : 36352 aida09 : 54 42 10 17 3 4 3 4 3 3 4 : 30376 aida10 : 17 5 7 4 3 3 3 3 2 3 6 : 35996 aida11 : 19 10 11 2 1 1 2 4 2 3 6 : 35916 aida12 : 82 42 12 7 4 2 3 2 2 3 4 : 27960 Options unchanged from ELOG 284 DAQ continues OK but New Merger window has disappeared... clearly still running in background! DSSSD leakage currents decreasing as expected - Plan later today: Prepare AIDA Kapton Cables for triple install 4 new FEE64s into S4 (13,14,15,16) Proposed layout of FEE64s (perspective with beam) in fig 5 Cards 9-12 will be moved on Friday when triple is installed When ready MAC addressed will be given to OH/TD who will do the PC configuration including: - Network IP allocation - MIDAS registration - Firmware updating - Checks 17.20: The four new FEEs have been installed and connected to Ethernet, Power, HDMI and Water No front-end adapter is connected yet The MAC addresss are as follows aida13 xilinx_lltemac 81c00000.ethernet: MAC address is now d8:80:39:42: d:15 aida14 xilinx_lltemac 81c00000.ethernet: MAC address is now d8:80:39:42: d: b aida15 xilinx_lltemac 81c00000.ethernet: MAC address is now d8:80:39:41:ee:10 aida16 xilinx_lltemac 81c00000.ethernet: MAC address is now d8:80:39:41:f6:ed
Attachment 1: elog_temps.png

Attachment 2: elog_gswr.png

Attachment 3: elog_tse.png

Attachment 4: elog_hv.png

Attachment 5: AIDA_FEE.png.jpeg

348	Wed May 26 13:21:59 2021	TD	Wednesday 26 May
14.22 DAQ continues - file S496/R18_369 alpha background All system wide checks OK except Base Current Difference aida01 fault 0x82b0 : 0x82b1 : 1 aida02 fault 0xfe72 : 0xfe73 : 1 aida03 fault 0x97f4 : 0x97f5 : 1 aida04 fault 0x950 : 0x951 : 1 aida05 fault 0x8e93 : 0x8e94 : 1 aida06 fault 0x545a : 0x545b : 1 aida07 fault 0xc04d : 0xc04e : 1 aida08 fault 0xa0f7 : 0xa0f8 : 1 White Rabbit error counter test result: Passed 8, Failed 8 Understand the status reports as follows:- Status bit 3 : White Rabbit decoder detected an error in the received data Status bit 2 : Firmware registered WR error, no reload of Timestamp Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k aida01 : 12 4 6 1 2 3 2 2 2 4 6 : 37072 aida02 : 12 2 4 1 3 4 3 3 2 4 6 : 37984 aida03 : 12 6 7 6 2 4 3 2 3 3 6 : 36624 aida04 : 36 11 4 6 4 3 2 3 3 3 6 : 36968 aida05 : 2 4 6 3 2 2 2 2 2 4 6 : 36968 aida06 : 8 5 3 2 2 2 2 3 3 4 6 : 38456 aida07 : 10 4 6 4 3 3 3 3 2 4 6 : 37992 aida08 : 21 8 3 2 3 3 3 3 2 4 6 : 37956 aida09 : 17 10 3 2 3 4 3 3 1 4 6 : 37060 aida10 : 3 6 10 2 1 3 3 3 1 4 6 : 36828 aida11 : 12 6 4 2 2 3 3 3 1 4 6 : 36832 aida12 : 2 9 5 1 3 3 2 3 2 4 6 : 37632 aida13 : 4 7 2 1 3 3 3 3 2 4 6 : 37832 aida14 : 15 6 6 1 0 4 2 2 2 4 6 : 37100 aida15 : 5 4 3 2 1 3 3 4 2 4 6 : 38244 aida16 : 17 6 6 2 1 3 2 2 2 4 6 : 37076 Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same FEE : aida01 => Options file size is 1026 Last changed Sun May 23 12:04:25 CEST 2021 FEE : aida02 => Options file size is 1025 Last changed Sun May 23 00:19:21 CEST 2021 FEE : aida03 => Options file size is 1014 Last changed Thu Apr 29 14:43:50 CEST 2021 FEE : aida04 => Options file size is 1025 Last changed Fri May 14 16:54:56 CEST 2021 FEE : aida05 => Options file size is 1025 Last changed Mon May 17 06:25:41 CEST 2021 FEE : aida06 => Options file size is 1014 Last changed Thu Apr 29 14:43:59 CEST 2021 FEE : aida07 => Options file size is 1014 Last changed Thu Apr 29 14:44:02 CEST 2021 FEE : aida08 => Options file size is 1025 Last changed Sun May 23 00:16:54 CEST 2021 FEE : aida09 => Options file size is 1014 Last changed Thu Apr 29 14:44:08 CEST 2021 FEE : aida10 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021 FEE : aida11 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021 FEE : aida12 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021 FEE : aida13 => Options file size is 1025 Last changed Fri May 07 19:40:34 CEST 2021 FEE : aida14 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021 FEE : aida15 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021 FEE : aida16 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021 Grafana - DSSSD bias & leakage cuuent - most recent 7 days - attachment 1 Lost activity monitor - attachment 2 1.8.W spectra - attachments 3 & 4 ADC data item stats - attachment 5 FEE64 temps OK - attachment 6 DSSSD bias & leakage current OK - attachment 7 Merger/Tape Server/Merger stats - attachment 8 no merger errors reported since previous restart 14.30 analysis of file S496/R18_340 - attachment 9 max deadtime 0.31% aida04
Attachment 1: Screenshot_2021-05-26_AIDA_Alerting_-_Grafana.png

Attachment 2: Screenshot_2021-05-26_Lost_activity_monitor_aidas-gsi.png

Attachment 3: Screenshot_2021-05-26_Spectrum_Browser_aidas-gsi(1).png

Attachment 4: Screenshot_2021-05-26_Spectrum_Browser_aidas-gsi.png

Attachment 5: Screenshot_2021-05-26_Statistics_aidas-gsi.png

Attachment 6: Screenshot_2021-05-26_Temperature_and_status_scan_aidas-gsi.png

Attachment 7: 920.png

Attachment 8: 921.png

Attachment 9: R18_368
* TDR format 3.3.0 analyser - TD - May 2021 * ERROR: READ I/O error: 5002 blocks: 32000 ADC data format: 60865731 ( 22299.7 Hz) Other data format: 201054280 ( 73661.2 Hz) Sample trace data format: 0 ( 0.0 Hz) Undefined format: 0 ( 0.0 Hz) Other data format type: PAUSE: 152 ( 0.1 Hz) RESUME: 152 ( 0.1 Hz) SYNC100: 42166 ( 15.4 Hz) WR48-63: 42166 ( 15.4 Hz) FEE64 disc: 1309767 ( 479.9 Hz) MBS info: 199659877 ( 73150.3 Hz) Other info: 0 ( 0.0 Hz) ADC data range bit set: 107 ( 0.0 Hz) Timewarps: ADC: 0 ( 0.0 Hz) PAUSE: 0 ( 0.0 Hz) RESUME: 0 ( 0.0 Hz) SYNC100: 0 ( 0.0 Hz) WR48-63: 0 ( 0.0 Hz) FEE64 disc: 0 ( 0.0 Hz) MBS info: 0 ( 0.0 Hz) Undefined: 0 ( 0.0 Hz) Sample trace: 0 ( 0.0 Hz) * Timestamp elapsed time: 2729.447 s FEE elapsed dead time(s) elapsed idle time(s) 0 0.000 0.000 1 0.000 2401.079 2 0.453 0.000 3 8.570 0.000 4 0.000 0.000 5 0.000 1507.937 6 0.000 0.000 7 0.000 1499.170 8 0.000 0.000 9 0.000 0.000 10 0.000 0.000 11 0.000 0.000 12 0.000 0.000 13 0.000 0.000 14 0.000 0.000 15 0.000 0.000 16 0.000 0.000 17 0.000 0.000 18 0.000 0.000 19 0.000 0.000 20 0.000 0.000 21 0.000 0.000 22 0.000 0.000 23 0.000 0.000 24 0.000 0.000 25 0.000 0.000 26 0.000 0.000 27 0.000 0.000 28 0.000 0.000 29 0.000 0.000 30 0.000 0.000 31 0.000 0.000 32 0.000 0.000 * Statistics FEE ADC Data Other Data Sample Undefined Pause Resume SYNC100 WR48-63 Disc MBS Other HEC Data 0 1115 29 0 0 0 0 1 1 27 0 0 10 1 2766811 11645 0 0 0 0 574 574 10497 0 0 18 2 2194 100014163 0 0 27 27 17354 17354 0 99979401 0 1 3 44009553 100997602 0 0 125 125 21286 21286 1274310 99680470 0 5 4 902 6 0 0 0 0 0 0 0 6 0 0 5 6872755 14651 0 0 0 0 1451 1451 11749 0 0 10 6 1948 13 0 0 0 0 1 1 11 0 0 9 7 7202790 16106 0 0 0 0 1497 1497 13112 0 0 9 8 531 0 0 0 0 0 0 0 0 0 0 0 9 394 0 0 0 0 0 0 0 0 0 0 3 10 1118 1 0 0 0 0 0 0 1 0 0 0 11 900 26 0 0 0 0 0 0 26 0 0 12 12 1756 13 0 0 0 0 0 0 13 0 0 8 13 1346 11 0 0 0 0 1 1 9 0 0 7 14 0 0 0 0 0 0 0 0 0 0 0 0 15 1618 14 0 0 0 0 1 1 12 0 0 15 16 0 0 0 0 0 0 0 0 0 0 0 0 17 0 0 0 0 0 0 0 0 0 0 0 0 18 0 0 0 0 0 0 0 0 0 0 0 0 19 0 0 0 0 0 0 0 0 0 0 0 0 20 0 0 0 0 0 0 0 0 0 0 0 0 21 0 0 0 0 0 0 0 0 0 0 0 0 22 0 0 0 0 0 0 0 0 0 0 0 0 23 0 0 0 0 0 0 0 0 0 0 0 0 24 0 0 0 0 0 0 0 0 0 0 0 0 25 0 0 0 0 0 0 0 0 0 0 0 0 26 0 0 0 0 0 0 0 0 0 0 0 0 27 0 0 0 0 0 0 0 0 0 0 0 0 28 0 0 0 0 0 0 0 0 0 0 0 0 29 0 0 0 0 0 0 0 0 0 0 0 0 30 0 0 0 0 0 0 0 0 0 0 0 0 31 0 0 0 0 0 0 0 0 0 0 0 0 32 0 0 0 0 0 0 0 0 0 0 0 0 * Timewarps FEE ADC Pause Resume SYNC100 WR48-63 Disc MBS Undefined Samples 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 4 0 0 0 0 0 0 0 0 0 5 0 0 0 0 0 0 0 0 0 6 0 0 0 0 0 0 0 0 0 7 0 0 0 0 0 0 0 0 0 8 0 0 0 0 0 0 0 0 0 9 0 0 0 0 0 0 0 0 0 10 0 0 0 0 0 0 0 0 0 11 0 0 0 0 0 0 0 0 0 12 0 0 0 0 0 0 0 0 0 13 0 0 0 0 0 0 0 0 0 14 0 0 0 0 0 0 0 0 0 15 0 0 0 0 0 0 0 0 0 16 0 0 0 0 0 0 0 0 0 17 0 0 0 0 0 0 0 0 0 18 0 0 0 0 0 0 0 0 0 19 0 0 0 0 0 0 0 0 0 20 0 0 0 0 0 0 0 0 0 21 0 0 0 0 0 0 0 0 0 22 0 0 0 0 0 0 0 0 0 23 0 0 0 0 0 0 0 0 0 24 0 0 0 0 0 0 0 0 0 25 0 0 0 0 0 0 0 0 0 26 0 0 0 0 0 0 0 0 0 27 0 0 0 0 0 0 0 0 0 28 0 0 0 0 0 0 0 0 0 29 0 0 0 0 0 0 0 0 0 30 0 0 0 0 0 0 0 0 0 31 0 0 0 0 0 0 0 0 0 32 0 0 0 0 0 0 0 0 0 * Program elapsed time: 31.695s ( 1009.613 blocks/s, 63.101 Mb/s)
489	Wed Jun 22 07:12:46 2022	MS	Wednesday 22 June 8:00-16:00
Took over the night shift from Tom. 8:10 : attachments 1-3 10:00 : attachments 4-6 12:00 : attachments 7-9 The beam was off for around 20' starting at around 11:30-11:50 due to ion source issues. 12:19 OH Disabled all discriminators to reduce the data rate. 13:06 Analysis of R2_104 15.66% deadtime on AIDA04- attachment 10 14:00 : attachments 11-13 16:00 : attachments 14-16
Attachment 1: Screenshot_from_2022-06-22_08-11-30.png

Attachment 2: Screenshot_2022-06-22_at_08-10-58_Temperature_and_status_scan_aidas-gsi.png

Attachment 3: Screenshot_2022-06-22_at_08-10-16_Statistics_aidas-gsi.png

Attachment 4: Screenshot_2022-06-22_at_09-59-11_Statistics_aidas-gsi.png

Attachment 5: Screenshot_2022-06-22_at_09-59-40_Temperature_and_status_scan_aidas-gsi.png

Attachment 6: Screenshot_from_2022-06-22_10-00-13.png

Attachment 7: Screenshot_2022-06-22_at_11-58-57_Statistics_aidas-gsi.png

Attachment 8: Screenshot_2022-06-22_at_11-59-18_Temperature_and_status_scan_aidas-gsi.png

Attachment 9: Screenshot_from_2022-06-22_11-59-55.png

Attachment 10: R2_104_analysis.txt
* TDR format 3.3.0 analyser - TD - May 2021 * ERROR: READ I/O error: 5002 blocks: 32000 ADC data format: 260553751 ( 867725.3 Hz) Other data format: 1366249 ( 4550.0 Hz) Sample trace data format: 0 ( 0.0 Hz) Undefined format: 0 ( 0.0 Hz) Other data format type: PAUSE: 1464 ( 4.9 Hz) RESUME: 1464 ( 4.9 Hz) SYNC100: 33119 ( 110.3 Hz) WR48-63: 33119 ( 110.3 Hz) FEE64 disc: 0 ( 0.0 Hz) MBS info: 1297083 ( 4319.7 Hz) Other info: 0 ( 0.0 Hz) ADC data range bit set: 518974 ( 1728.3 Hz) Timewarps: ADC: 0 ( 0.0 Hz) PAUSE: 0 ( 0.0 Hz) RESUME: 0 ( 0.0 Hz) SYNC100: 0 ( 0.0 Hz) WR48-63: 0 ( 0.0 Hz) FEE64 disc: 0 ( 0.0 Hz) MBS info: 0 ( 0.0 Hz) Undefined: 0 ( 0.0 Hz) Sample trace: 0 ( 0.0 Hz) * Timestamp elapsed time: 300.272 s FEE elapsed dead time(s) elapsed idle time(s) 0 7.776 0.000 1 47.563 0.000 2 0.466 0.000 3 4.438 0.000 4 0.149 14.780 5 16.612 0.000 6 0.972 0.000 7 0.380 0.000 8 0.000 0.000 9 0.000 0.000 10 0.000 0.000 11 0.000 0.000 12 0.000 0.000 13 0.000 0.000 14 0.000 0.000 15 0.000 0.000 16 0.000 0.000 17 0.000 0.000 18 0.000 0.000 19 0.000 0.000 20 0.000 0.000 21 0.000 0.000 22 0.000 0.000 23 0.000 0.000 24 0.000 0.000 25 0.000 0.000 26 0.000 0.000 27 0.000 0.000 28 0.000 0.000 29 0.000 0.000 30 0.000 0.000 31 0.000 0.000 32 0.000 0.000 * Statistics FEE ADC Data Other Data Sample Undefined Pause Resume SYNC100 WR48-63 Disc MBS Other HEC Data 0 27837591 27876 0 0 187 187 3557 3557 0 20388 0 104437 1 57538232 16015 0 0 651 652 7356 7356 0 0 0 64114 2 23047755 331545 0 0 63 63 2949 2949 0 325521 0 64002 3 44324924 329582 0 0 146 145 5671 5671 0 317949 0 37242 4 11203550 2766 0 0 57 57 1326 1326 0 0 0 90620 5 28839527 7636 0 0 219 219 3599 3599 0 0 0 65648 6 37663072 317251 0 0 87 87 4772 4772 0 307533 0 63360 7 30099100 333578 0 0 54 54 3889 3889 0 325692 0 29551 8 0 0 0 0 0 0 0 0 0 0 0 0 9 0 0 0 0 0 0 0 0 0 0 0 0 10 0 0 0 0 0 0 0 0 0 0 0 0 11 0 0 0 0 0 0 0 0 0 0 0 0 12 0 0 0 0 0 0 0 0 0 0 0 0 13 0 0 0 0 0 0 0 0 0 0 0 0 14 0 0 0 0 0 0 0 0 0 0 0 0 15 0 0 0 0 0 0 0 0 0 0 0 0 16 0 0 0 0 0 0 0 0 0 0 0 0 17 0 0 0 0 0 0 0 0 0 0 0 0 18 0 0 0 0 0 0 0 0 0 0 0 0 19 0 0 0 0 0 0 0 0 0 0 0 0 20 0 0 0 0 0 0 0 0 0 0 0 0 21 0 0 0 0 0 0 0 0 0 0 0 0 22 0 0 0 0 0 0 0 0 0 0 0 0 23 0 0 0 0 0 0 0 0 0 0 0 0 24 0 0 0 0 0 0 0 0 0 0 0 0 25 0 0 0 0 0 0 0 0 0 0 0 0 26 0 0 0 0 0 0 0 0 0 0 0 0 27 0 0 0 0 0 0 0 0 0 0 0 0 28 0 0 0 0 0 0 0 0 0 0 0 0 29 0 0 0 0 0 0 0 0 0 0 0 0 30 0 0 0 0 0 0 0 0 0 0 0 0 31 0 0 0 0 0 0 0 0 0 0 0 0 32 0 0 0 0 0 0 0 0 0 0 0 0 * Timewarps FEE ADC Pause Resume SYNC100 WR48-63 Disc MBS Undefined Samples 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 4 0 0 0 0 0 0 0 0 0 5 0 0 0 0 0 0 0 0 0 6 0 0 0 0 0 0 0 0 0 7 0 0 0 0 0 0 0 0 0 8 0 0 0 0 0 0 0 0 0 9 0 0 0 0 0 0 0 0 0 10 0 0 0 0 0 0 0 0 0 11 0 0 0 0 0 0 0 0 0 12 0 0 0 0 0 0 0 0 0 13 0 0 0 0 0 0 0 0 0 14 0 0 0 0 0 0 0 0 0 15 0 0 0 0 0 0 0 0 0 16 0 0 0 0 0 0 0 0 0 17 0 0 0 0 0 0 0 0 0 18 0 0 0 0 0 0 0 0 0 19 0 0 0 0 0 0 0 0 0 20 0 0 0 0 0 0 0 0 0 21 0 0 0 0 0 0 0 0 0 22 0 0 0 0 0 0 0 0 0 23 0 0 0 0 0 0 0 0 0 24 0 0 0 0 0 0 0 0 0 25 0 0 0 0 0 0 0 0 0 26 0 0 0 0 0 0 0 0 0 27 0 0 0 0 0 0 0 0 0 28 0 0 0 0 0 0 0 0 0 29 0 0 0 0 0 0 0 0 0 30 0 0 0 0 0 0 0 0 0 31 0 0 0 0 0 0 0 0 0 32 0 0 0 0 0 0 0 0 0 * Program elapsed time: 34.320s ( 932.392 blocks/s, 58.275 Mb/s)
Attachment 11: Screenshot_2022-06-22_at_14-01-58_Statistics_aidas-gsi.png

Attachment 12: Screenshot_2022-06-22_at_14-02-23_Temperature_and_status_scan_aidas-gsi.png

Attachment 13: Screenshot_from_2022-06-22_14-03-14.png

Attachment 14: Screenshot_2022-06-22_at_15-57-21_Statistics_aidas-gsi.png

Attachment 15: Screenshot_2022-06-22_at_15-57-42_Temperature_and_status_scan_aidas-gsi.png

Attachment 16: Screenshot_from_2022-06-22_15-58-16.png

491	Wed Jun 22 15:09:37 2022	MA	Wednesday 22 June 16:00-00:00
16:00 Take over from Magda mrning shift Beam was off of sometime for tunning to Mercurey setting. 17:00 attached 1-3 19:00 attached 4-6 21:00 attached 7-9 23:30 10-12
Attachment 1: BiasCurrent2022-06-22_17-11-07.png

Attachment 2: Statistics_2022-06-22_17-13-52.png

Attachment 3: Temperature2022-06-22_17-12-35.png

Attachment 4: BiasCurrent2022-06-22_19-00-31.png

Attachment 5: Temperature2022-06-22_19-02-18.png

Attachment 6: Statics2022-06-22_19-03-11.png

Attachment 7: BiasCurrent2022-06-22_21-04-38.png

Attachment 8: Statistics2022-06-22_21-07-07.png

Attachment 9: Temperature2022-06-22_21-05-51.png

Attachment 10: BiasCurrent2022-06-22_23-29-40.png

Attachment 11: Statistics2022-06-22_23-31-30.png

Attachment 12: Temperature2022-06-22_23-30-28.png

488	Tue Jun 21 23:29:16 2022	TD	Wednesday 22 June 00:00-08:00
00.28 all histograms & stats zero'd baseline counters ASIC settings 2021Apr29-13-16-00 except slow comparator 0x64 -> 0xa BNC PB-5 pulser to p+n FEE64s only amplitude 1.0V attenuator x1 decay time 1ms polarity + frequency 2Hz all AD9252 waveform ADCs disabled Tape Server -> no storage mode 00.36 check ASIC control all FEE64s all ASICs all system wide checks OK 00.41 BNC PB-5 pulser rate increased from 2Hz to 22Hz per request by HA/NH - to ensure DTAS correlation checker has sufficient rate of identified pulser events per p+n FEE64 1.8.L spectra - attachment 1 aida01 pulser peak width 88 ch FWHM per FEE64 rate spectra - attachment 2 adc, pause and correlation scaler data items - push, aida01 and aida02 stats - attachments 3-8 FEE64 temps OK - attachment 9 DSSSD bias & leakage current OK - attachments 10 & 11 02.28 aida05 zero adc data items, unable to halt aida05 - all other FEE64s OK restart NewMerger -> now able to halt aida05 DAQ go OK, merge rate c. 1.4M data items/s per FEE64 1.8.H spectra - attachments 12 & 13 approx flat distribution to c. 10GeV adc data items stats OK - attachments 14 FEE64 temps OK - attachment 15 DSSSD bias & leakage current OK - attachments 16 & 17 02.55 Some problems with GO4 data analysis for PID (FRS PID looks OK). 204Pt setting. Will commence writing data to disk file S505/R2_0 04.12 per FEE64 1.8.H spectra - attachments 18 & 19 c. 9GeV peak in upstream DSSSDs (aida01, 2, 3 & 4) indicating (high energy) heavy fragments are stopping Low energy events near c. 200MeV lower mass, high energy, low energy loss - fission fragments? 04.20 per FEE64 stats spectra - attachment 20 indicates implant x-y distribution - off centre (more significantly in x than in y) but DSSSDs appear to see most of distribution remember raw distro will have significant fraction of fission fragments - may be misleading - need to gate on high energy events to see x-y distro of ions of interest adc data items stats OK - attachments 21 FEE64 temps OK - attachment 22 DSSSD bias & leakage current OK - attachments 23 06.10 per FEE64 rate spectra - attachment 24 per FEE64 1.8.H spectra - attachments 25 & 26 per FEE64 stats spectra - attachment 27 adc data items stats OK - attachments 28 FEE64 temps OK - attachment 29 system wide checks OK except WR & FPGA errors - see attachments 30 & 31 DSSSD bias & leakage current OK - attachments 32 07.52 analysis of file S505/R2_47 - attachment 33 all FEE64 dead times < 1% except aida02 c. 7%
Attachment 1: Screenshot_from_2022-06-22_00-50-20.png

Attachment 2: Screenshot_from_2022-06-22_00-49-07.png

Attachment 3: Screenshot_from_2022-06-22_00-47-56.png

Attachment 4: Screenshot_from_2022-06-22_00-46-57.png

Attachment 5: Screenshot_from_2022-06-22_00-46-17.png

Attachment 6: Screenshot_from_2022-06-22_00-45-33.png

Attachment 7: Screenshot_from_2022-06-22_00-44-58.png

Attachment 8: Screenshot_from_2022-06-22_00-44-13.png

Attachment 9: Screenshot_from_2022-06-22_00-36-06.png

Attachment 10: Screenshot_from_2022-06-22_00-35-17.png

Attachment 11: Screenshot_from_2022-06-22_00-34-01.png

Attachment 12: Screenshot_from_2022-06-22_02-44-02.png

Attachment 13: Screenshot_from_2022-06-22_02-41-13.png

Attachment 14: Screenshot_from_2022-06-22_02-40-14.png

Attachment 15: Screenshot_from_2022-06-22_02-39-42.png

Attachment 16: Screenshot_from_2022-06-22_02-39-01.png

Attachment 17: Screenshot_from_2022-06-22_02-38-24.png

Attachment 18: Screenshot_from_2022-06-22_04-10-47.png

Attachment 19: Screenshot_from_2022-06-22_04-10-00.png

Attachment 20: Screenshot_from_2022-06-22_04-23-13.png

Attachment 21: Screenshot_from_2022-06-22_04-20-06.png

Attachment 22: Screenshot_from_2022-06-22_04-19-34.png

Attachment 23: Screenshot_from_2022-06-22_04-19-00.png

Attachment 24: Screenshot_from_2022-06-22_06-10-41.png

Attachment 25: Screenshot_from_2022-06-22_06-09-33.png

Attachment 26: Screenshot_from_2022-06-22_06-08-08.png

Attachment 27: Screenshot_from_2022-06-22_06-06-50.png

Attachment 28: Screenshot_from_2022-06-22_06-05-58.png

Attachment 29: Screenshot_from_2022-06-22_06-05-20.png

Attachment 30: Screenshot_from_2022-06-22_06-04-27.png

Attachment 31: Screenshot_from_2022-06-22_06-03-53.png

Attachment 32: Screenshot_from_2022-06-22_06-03-05.png

Attachment 33: R2_47
* TDR format 3.3.0 analyser - TD - May 2021 * ERROR: READ I/O error: 5002 blocks: 32000 ADC data format: 259428432 ( 714533.4 Hz) Other data format: 2491568 ( 6862.4 Hz) Sample trace data format: 0 ( 0.0 Hz) Undefined format: 0 ( 0.0 Hz) Other data format type: PAUSE: 881 ( 2.4 Hz) RESUME: 880 ( 2.4 Hz) SYNC100: 33352 ( 91.9 Hz) WR48-63: 33352 ( 91.9 Hz) FEE64 disc: 1422279 ( 3917.3 Hz) MBS info: 1000824 ( 2756.5 Hz) Other info: 0 ( 0.0 Hz) ADC data range bit set: 416179 ( 1146.3 Hz) Timewarps: ADC: 0 ( 0.0 Hz) PAUSE: 0 ( 0.0 Hz) RESUME: 0 ( 0.0 Hz) SYNC100: 0 ( 0.0 Hz) WR48-63: 0 ( 0.0 Hz) FEE64 disc: 0 ( 0.0 Hz) MBS info: 0 ( 0.0 Hz) Undefined: 0 ( 0.0 Hz) Sample trace: 0 ( 0.0 Hz) * Timestamp elapsed time: 363.074 s FEE elapsed dead time(s) elapsed idle time(s) 0 0.857 0.000 1 26.452 0.000 2 0.132 0.000 3 1.570 0.000 4 0.002 46.041 5 0.661 0.000 6 0.312 0.000 7 0.027 0.000 8 0.000 0.000 9 0.000 0.000 10 0.000 0.000 11 0.000 0.000 12 0.000 0.000 13 0.000 0.000 14 0.000 0.000 15 0.000 0.000 16 0.000 0.000 17 0.000 0.000 18 0.000 0.000 19 0.000 0.000 20 0.000 0.000 21 0.000 0.000 22 0.000 0.000 23 0.000 0.000 24 0.000 0.000 25 0.000 0.000 26 0.000 0.000 27 0.000 0.000 28 0.000 0.000 29 0.000 0.000 30 0.000 0.000 31 0.000 0.000 32 0.000 0.000 * Statistics FEE ADC Data Other Data Sample Undefined Pause Resume SYNC100 WR48-63 Disc MBS Other HEC Data 0 25701314 249377 0 0 56 56 3411 3411 217219 25224 0 83012 1 63351849 247459 0 0 567 566 8361 8361 229604 0 0 62244 2 23166597 387869 0 0 37 37 3010 3010 134065 247710 0 48973 3 48876539 363237 0 0 105 105 6171 6171 104940 245745 0 28921 4 8006429 267186 0 0 3 3 1045 1045 265090 0 0 68788 5 22095663 170195 0 0 36 36 2789 2789 164545 0 0 52757 6 40483353 472437 0 0 66 66 5106 5106 227985 234108 0 48544 7 27746688 333808 0 0 11 11 3459 3459 78831 248037 0 22940 8 0 0 0 0 0 0 0 0 0 0 0 0 9 0 0 0 0 0 0 0 0 0 0 0 0 10 0 0 0 0 0 0 0 0 0 0 0 0 11 0 0 0 0 0 0 0 0 0 0 0 0 12 0 0 0 0 0 0 0 0 0 0 0 0 13 0 0 0 0 0 0 0 0 0 0 0 0 14 0 0 0 0 0 0 0 0 0 0 0 0 15 0 0 0 0 0 0 0 0 0 0 0 0 16 0 0 0 0 0 0 0 0 0 0 0 0 17 0 0 0 0 0 0 0 0 0 0 0 0 18 0 0 0 0 0 0 0 0 0 0 0 0 19 0 0 0 0 0 0 0 0 0 0 0 0 20 0 0 0 0 0 0 0 0 0 0 0 0 21 0 0 0 0 0 0 0 0 0 0 0 0 22 0 0 0 0 0 0 0 0 0 0 0 0 23 0 0 0 0 0 0 0 0 0 0 0 0 24 0 0 0 0 0 0 0 0 0 0 0 0 25 0 0 0 0 0 0 0 0 0 0 0 0 26 0 0 0 0 0 0 0 0 0 0 0 0 27 0 0 0 0 0 0 0 0 0 0 0 0 28 0 0 0 0 0 0 0 0 0 0 0 0 29 0 0 0 0 0 0 0 0 0 0 0 0 30 0 0 0 0 0 0 0 0 0 0 0 0 31 0 0 0 0 0 0 0 0 0 0 0 0 32 0 0 0 0 0 0 0 0 0 0 0 0 * Timewarps FEE ADC Pause Resume SYNC100 WR48-63 Disc MBS Undefined Samples 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 4 0 0 0 0 0 0 0 0 0 5 0 0 0 0 0 0 0 0 0 6 0 0 0 0 0 0 0 0 0 7 0 0 0 0 0 0 0 0 0 8 0 0 0 0 0 0 0 0 0 9 0 0 0 0 0 0 0 0 0 10 0 0 0 0 0 0 0 0 0 11 0 0 0 0 0 0 0 0 0 12 0 0 0 0 0 0 0 0 0 13 0 0 0 0 0 0 0 0 0 14 0 0 0 0 0 0 0 0 0 15 0 0 0 0 0 0 0 0 0 16 0 0 0 0 0 0 0 0 0 17 0 0 0 0 0 0 0 0 0 18 0 0 0 0 0 0 0 0 0 19 0 0 0 0 0 0 0 0 0 20 0 0 0 0 0 0 0 0 0 21 0 0 0 0 0 0 0 0 0 22 0 0 0 0 0 0 0 0 0 23 0 0 0 0 0 0 0 0 0 24 0 0 0 0 0 0 0 0 0 25 0 0 0 0 0 0 0 0 0 26 0 0 0 0 0 0 0 0 0 27 0 0 0 0 0 0 0 0 0 28 0 0 0 0 0 0 0 0 0 29 0 0 0 0 0 0 0 0 0 30 0 0 0 0 0 0 0 0 0 31 0 0 0 0 0 0 0 0 0 32 0 0 0 0 0 0 0 0 0 * Program elapsed time: 35.072s ( 912.402 blocks/s, 57.025 Mb/s)
258	Wed Apr 21 15:01:14 2021	LJW, MA	Wednesday 21st April 16:00-20:00
16:30 All System checks okay, except for: FPGA Timestamp error: Base Current Difference aida12 fault 0x0 : 0x1b : 27 FPGA Timestamp error counter test result: Passed 11, Failed 1 If any of these counts are reported as in error The ASIC readout system has detected a timeslip. That is the timestamp read from the time FIFO is not younger than the last Temperature Checks: See attachment 1 Statistics Checks: See attachment 2 Bias & Leakage Currents: See attachment 3 18:09 All system checks okay, except for: FPGA Timestamp error: Base Current Difference aida12 fault 0x0 : 0x1d : 29 FPGA Timestamp error counter test result: Passed 11, Failed 1 If any of these counts are reported as in error The ASIC readout system has detected a timeslip. That is the timestamp read from the time FIFO is not younger than the last Temperature checks: See attachment 4 Statistics Checks: See attachment 5 Bias & Leakage Currents: See attachment 6 19:40 All system checks okay, except for: FPGA Timestamp error: Base Current Difference aida12 fault 0x0 : 0x1d : 29 FPGA Timestamp error counter test result: Passed 11, Failed 1 If any of these counts are reported as in error The ASIC readout system has detected a timeslip. That is the timestamp read from the time FIFO is not younger than the last Temperature checks: See attachment 7 Statistics Checks: See attachment 8 Bias & Leakage Currents: See attachment 9 Beam being tuned to another experiment (R3B) - all DSSD implants fluctuating back and forth to 0 Hz for around the last hour or so 22:30 General check Rates, Voltages, Temperatures, Ucesb are attached 10,11,12,13 Its been noticed that aida04 temp. is incresing to about 65 (red), still below 70. the system wide check are all ok except FPGA Base Current Difference aida12 fault 0x0 : 0x1d : 29 FPGA Timestamp error counter test result: Passed 11, Failed 1 If any of these counts are reported as in error The ASIC readout system has detected a timeslip. That is the timestamp read from the time FIFO is not younger than the last
Attachment 1: 210421_1.png

Attachment 2: 210421_2.png

Attachment 3: 2104211_3.png

Attachment 4: 210421_4.png

Attachment 5: 210421_5.png

Attachment 6: 210421_6.png

Attachment 7: 210421_7.png

Attachment 8: 210421_8.png

Attachment 9: 210421_9.png

Attachment 10: 210421_2230_Stats.png

Attachment 11: 210421_2230_Temp.png

Attachment 12: 210421_2230_Ucesb.png

Attachment 13: 210421_2230_Voltages.png

256	Wed Apr 21 06:56:42 2021	MS	Wednesday 21 April 08:00-12:00 2021
8:00 Clock status test result: Passed 12, Failed 0 Understand status as follows Status bit 3 : firmware PLL that creates clocks from external clock not locked Status bit 2 : always logic '1' Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock If all these bits are not set then the operation of the firmware is unreliable FEE64 module aida01 failed FEE64 module aida02 failed FEE64 module aida03 failed FEE64 module aida04 failed FEE64 module aida05 failed FEE64 module aida06 failed FEE64 module aida07 failed FEE64 module aida08 failed FEE64 module aida09 failed FEE64 module aida10 failed FEE64 module aida11 failed FEE64 module aida12 failed Calibration test result: Passed 0, Failed 12 If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module White Rabbit error counter test result: Passed 12, Failed 0 Understand the status reports as follows:- Status bit 3 : White Rabbit decoder detected an error in the received data Status bit 2 : Firmware registered WR error, no reload of Timestamp Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR Base Current Difference aida12 fault 0x0 : 0x15 : 21 FPGA Timestamp error counter test result: Passed 11, Failed 1 If any of these counts are reported as in error The ASIC readout system has detected a timeslip. That is the timestamp read from the time FIFO is not younger than the last Returned 0 0 0 0 0 0 0 0 0 0 0 0 Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k aida01 : 19 6 7 2 2 4 3 2 2 3 7 : 39596 aida02 : 25 3 2 1 1 3 2 3 2 3 7 : 39548 aida03 : 28 5 3 1 2 2 2 3 2 3 7 : 39528 aida04 : 27 6 2 2 2 4 2 4 2 3 7 : 40316 aida05 : 9 9 4 3 1 3 2 3 3 3 7 : 40652 aida06 : 21 7 6 1 2 2 4 2 2 3 7 : 39564 aida07 : 20 7 5 0 1 2 2 3 2 3 7 : 39448 aida08 : 21 5 4 1 1 2 3 3 2 3 7 : 39708 aida09 : 1 3 1 3 0 2 1 4 2 3 7 : 39564 aida10 : 20 6 3 2 1 2 3 3 2 3 7 : 39728 aida11 : 18 4 2 1 2 2 2 2 2 3 7 : 38952 aida12 : 19 8 7 1 1 2 3 3 2 3 7 : 39772 From the merger statistics during the night there have been 1355757 TS errors in aida12. Corrigan said they have been quiet since around 3am though. 10:00 Clock status test result: Passed 12, Failed 0 Understand status as follows Status bit 3 : firmware PLL that creates clocks from external clock not locked Status bit 2 : always logic '1' Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock If all these bits are not set then the operation of the firmware is unreliable FEE64 module aida01 failed FEE64 module aida02 failed FEE64 module aida03 failed FEE64 module aida04 failed FEE64 module aida06 failed FEE64 module aida07 failed FEE64 module aida08 failed FEE64 module aida09 failed FEE64 module aida10 failed FEE64 module aida11 failed FEE64 module aida12 failed Calibration test result: Passed 1, Failed 11 If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module White Rabbit error counter test result: Passed 12, Failed 0 Understand the status reports as follows:- Status bit 3 : White Rabbit decoder detected an error in the received data Status bit 2 : Firmware registered WR error, no reload of Timestamp Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR Base Current Difference aida12 fault 0x0 : 0x15 : 21 FPGA Timestamp error counter test result: Passed 11, Failed 1 If any of these counts are reported as in error The ASIC readout system has detected a timeslip. That is the timestamp read from the time FIFO is not younger than the last Returned 0 0 0 0 0 0 0 0 0 0 0 0 Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k aida01 : 15 7 6 2 1 4 3 2 2 3 7 : 39508 aida02 : 1 2 1 0 1 2 2 4 2 3 7 : 39780 aida03 : 23 8 2 0 1 3 2 3 2 3 7 : 39548 aida04 : 23 9 1 2 3 3 2 4 2 3 7 : 40244 aida05 : 14 7 2 2 4 3 2 3 1 4 7 : 40784 aida06 : 18 5 5 2 2 2 3 3 2 3 7 : 39808 aida07 : 27 4 3 2 2 3 3 2 2 3 7 : 39420 aida08 : 23 6 3 2 2 2 3 3 2 3 7 : 39804 aida09 : 18 4 6 3 2 4 3 2 2 3 7 : 39592 aida10 : 6 9 5 3 2 3 2 3 2 3 7 : 39696 aida11 : 16 4 1 2 3 4 1 3 2 3 7 : 39536 aida12 : 18 5 7 1 0 3 3 3 2 3 7 : 39808 10:22 NH Noticed that the ASIC clocks had not been synchronised following the last AIDA restart. - attachment 7 As the beam is having issues they are not currently recording I have now synchronised the ASIC clocks. All ADC are now calibrated 11:06 Used the beam off time to change the drive writing to. Now writing to /media/ThirdDrive/TapeData/S460/R50 11:10 Beam is back they will start writing to file again 12:00 Clock status test result: Passed 12, Failed 0 Understand status as follows Status bit 3 : firmware PLL that creates clocks from external clock not locked Status bit 2 : always logic '1' Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock If all these bits are not set then the operation of the firmware is unreliable Calibration test result: Passed 12, Failed 0 If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module White Rabbit error counter test result: Passed 12, Failed 0 Understand the status reports as follows:- Status bit 3 : White Rabbit decoder detected an error in the received data Status bit 2 : Firmware registered WR error, no reload of Timestamp Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR Base Current Difference aida12 fault 0x0 : 0x17 : 23 FPGA Timestamp error counter test result: Passed 11, Failed 1 If any of these counts are reported as in error The ASIC readout system has detected a timeslip. That is the timestamp read from the time FIFO is not younger than the last Returned 0 0 0 0 0 0 0 0 0 0 0 0 Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k aida01 : 20 5 5 1 1 4 2 3 2 3 7 : 39720 aida02 : 16 8 4 1 0 2 1 4 2 3 7 : 39648 aida03 : 9 7 3 2 3 3 2 3 2 3 7 : 39692 aida04 : 8 7 5 2 3 5 3 2 2 3 7 : 39720 aida05 : 20 5 3 4 2 3 3 3 2 3 7 : 39976 aida06 : 25 8 0 3 1 2 3 3 2 3 7 : 39748 aida07 : 24 8 4 2 3 3 2 3 2 3 7 : 39776 aida08 : 22 9 0 2 2 2 3 3 2 3 7 : 39776 aida09 : 1 4 4 3 2 3 2 3 2 3 7 : 39620 aida10 : 13 4 4 3 2 3 1 3 2 3 7 : 39412 aida11 : 2 3 1 1 3 4 2 2 2 3 7 : 39184 aida12 : 16 10 7 2 1 3 1 3 2 3 7 : 39424
Attachment 1: Screenshot_2021-04-21_Statistics_aidas-gsi.png

Attachment 2: Screenshot_2021-04-21_Temperature_and_status_scan_aidas-gsi.png

Attachment 3: 46.png

Attachment 4: Screenshot_2021-04-21_Statistics_aidas-gsi(1).png

Attachment 5: Screenshot_2021-04-21_Temperature_and_status_scan_aidas-gsi(1).png

Attachment 6: 47.png

Attachment 7: Capture.PNG

Attachment 8: Screenshot_2021-04-21_Temperature_and_status_scan_aidas-gsi(2).png

Attachment 9: Screenshot_2021-04-21_Statistics_aidas-gsi(2).png

Attachment 10: 48.png

Goto page Previous 1, 2, 3, 4 ... 33, 34, 35 Next

ELOG V3.1.4-unknown