11.28 DAQ continues OK - no storage
ASIC settings 2019Dec19-16.19.51
DSSSD#1 slow comparator 0xa
DSSSD#2 slow comparator 0xa
DSSSD#3 slow comparator 0xd
BNC PB-5 Pulser
Amplitude1.0V
Attenuation x1
Frequency 2Hz
tau_d 1ms
- polarity
Delay 250ns, tail pulse
10.26 System wide checks
FEE64 module aida09 global clocks failed, 6
Clock status test result: Passed 11, Failed 1
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable
FEE64 module aida07 failed
FEE64 module aida09 failed
Calibration test result: Passed 10, Failed 2
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
Base Current Difference
aida05 fault 0x6acb : 0x6acc : 1
aida06 fault 0x8394 : 0x8395 : 1
aida07 fault 0x8c77 : 0x8c7a : 3
aida08 fault 0x1692 : 0x1693 : 1
White Rabbit error counter test result: Passed 8, Failed 4
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
FPGA Timestamp error counter test result: Passed 12, Failed 0
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
Returned 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 25 16 3 6 1 3 2 2 3 4 10 : 54676
aida02 : 17 13 14 10 5 5 2 3 3 3 6 : 37516
aida03 : 19 10 8 3 2 3 2 4 2 3 6 : 36220
aida04 : 2 6 2 3 3 2 2 3 2 4 15 : 74360
aida05 : 89 48 20 3 3 34 21 20 6 4 7 : 64324
aida06 : 31 11 1 10 0 1 1 3 4 2 5 : 31140
aida07 : 21 13 8 4 3 2 4 4 3 3 6 : 37756
aida08 : 37 10 2 0 1 2 2 4 3 3 6 : 36932
aida09 : 4 18 6 3 1 3 1 2 3 3 6 : 35872
aida10 : 20 4 1 3 0 2 2 3 3 3 6 : 36320
aida11 : 2 6 6 2 2 4 2 3 2 3 6 : 35672
aida12 : 15 10 2 3 1 2 2 3 3 3 6 : 36428
FEE64 Temperatures OK - attachment 1
Good event, disc info #6 & good wave statistics - attachments 2, 3 & 4
note high disc rates (c. 200k) in FEE64s aida03, aida05 & aida07
Detector bias & leakage currents OK - attachment 5
Merger OK - 4.5M data items/s
TapeServer OK - 15Mb/s
Rate spectra - attachment 6
11.41 Merger errors since yesterday's Elog
MERGE Data Link (20502): bad timestamp 2 3 0xc08e4cce 0x0c1c8460 0x0000f14c5c1c8460 0x166cf14c5c1c8460 0x166cf1515c1c74c0
MERGE Data Link (20502): bad timestamp 2 3 0xc0967f4f 0x0c1c8460 0x0000f14c5c1c8460 0x166cf14c5c1c8460 0x166cf1515c1c74c0
MERGE Data Link (20502): bad timestamp 2 3 0xc0807dc7 0x0c1c9bd0 0x0000f14c5c1c9bd0 0x166cf14c5c1c9bd0 0x166cf1515c1c74c0
MERGE Data Link (20502): bad timestamp 2 3 0xc0957ee9 0x0c1c9bd0 0x0000f14c5c1c9bd0 0x166cf14c5c1c9bd0 0x166cf1515c1c74c0
MERGE Data Link (20502): bad timestamp 2 3 0xc0857eea 0x0c1ca3a0 0x0000f14c5c1ca3a0 0x166cf14c5c1ca3a0 0x166cf1515c1c74c0
MERGE Data Link (20502): bad timestamp 2 3 0xc0977df3 0x0c1ca3a0 0x0000f14c5c1ca3a0 0x166cf14c5c1ca3a0 0x166cf1515c1c74c0
MERGE Data Link (20502): bad timestamp 2 3 0xc08a7edd 0x0c1cab70 0x0000f14c5c1cab70 0x166cf14c5c1cab70 0x166cf1515c1c74c0
MERGE Data Link (20502): bad timestamp 2 3 0xc09c7c7f 0x0c1cab70 0x0000f14c5c1cab70 0x166cf14c5c1cab70 0x166cf1515c1c74c0
MERGE Data Link (20502): bad timestamp 2 3 0xc09d7fd8 0x0c1cb340 0x0000f14c5c1cb340 0x166cf14c5c1cb340 0x166cf1515c1c74c0
:
:
MERGE Data Link (20502): bad timestamp 2 3 0x8263ffff 0x0433c338 0x000018ff4433c338 0x166d18ff4433c338 0x166d18ff4e17c2c0
MERGE Data Link (20502): bad timestamp 2 3 0x8263ffff 0x0433c356 0x000018ff4433c356 0x166d18ff4433c356 0x166d18ff4e17c2c0
MERGE Data Link (20502): bad timestamp 2 3 0x8263f080 0x0433c36a 0x000018ff4433c36a 0x166d18ff4433c36a 0x166d18ff4e17c2c0
MERGE Data Link (20502): bad timestamp 2 3 0x82630f7f 0x0433c374 0x000018ff4433c374 0x166d18ff4433c374 0x166d18ff4e17c2c0
MERGE Data Link (20502): bad timestamp 2 3 0x8263ffff 0x0433c388 0x000018ff4433c388 0x166d18ff4433c388 0x166d18ff4e17c2c0
MERGE Data Link (20502): bad timestamp 2 3 0x8263ffff 0x0433c3a6 0x000018ff4433c3a6 0x166d18ff4433c3a6 0x166d18ff4e17c2c0
MERGE Data Link (20502): bad timestamp 2 3 0x8263efff 0x0433c3c4 0x000018ff4433c3c4 0x166d18ff4433c3c4 0x166d18ff4e17c2c0
MERGE Data Link (20502): bad timestamp 2 3 0x8263ffff 0x0433c3d8 0x000018ff4433c3d8 0x166d18ff4433c3d8 0x166d18ff4e17c2c0
MERGE Data Link (20502): bad timestamp 2 3 0x8263ffff 0x0433c3f6 0x000018ff4433c3f6 0x166d18ff4433c3f6 0x166d18ff4e17c2c0
MERGE Data Link (20502): bad timestamp 2 3 0x8263efff 0x0433c414 0x000018ff4433c414 0x166d18ff4433c414 0x166d18ff4e17c2c0
MERGE Data Link (20502): bad timestamp 2 3 0x82218ff4 0x0433c41e 0x000018ff4433c41e 0x166d18ff4433c41e 0x166d18ff4e17c2c0
From Tuesday, March 16, 2021 9:54:36.584 PM to Wednesday, March 17, 2021 10:02:05.935 AM
11.46 Grafana record of bias & leakage current for the 7 days to date - attachment 7
11.51 ASIC check load - all FEE64s, all ASICs
11.51 Disable 'no storage' mode and start writing to disk file S452/R58
12.01 R58_18 & R58_19 *no* change
12.16 R58_24 & R58_25 *all* waveforms disabled (Run Control tab)
12.23 R58_27 & R58_28 *all* discriminators disabled (Discriminator tab) *but* disc rates now low - see attachment 8 cf. attachment 3
12.27 Enable 'no storage' mode
12.38 Analysis of files R58_19, R58_25 & R58_28 - see attachments 9, 10 & 11
Disabling waveforms significantly reduces pause/resume for aida01 - aida08
Effect of disabling discriminators less clear - disc rate zero for all FEE64s except aida03 (2k). |