ELOG AIDA

AIDA GELINA BRIKEN nToF CRIB ISOLDE CIRCE nTOFCapture DESPEC DTAS EDI_PSA 179Ta CARME StellarModelling DCF K40

AIDA

Fields marked with * are required
Entry time:	Sat Mar 29 12:28:14 2025
Author*:
Subject*:
> 10.20 DAQ found stalled > > DSSSD bias & leakage current OK - see attachments 1 & 2 > FEE64 temperatures OK - see attachment 3 > AIDA SYNC stats - see attachment 4 - stats 10.1M/381/3600 => DAQ stalled at 23.05 + 7.4h => c.06.29 > System wide checks OK except SYNC pulses & SYNC errors - see attachments 5-8 > > Conclude DAQ stalling at c. 06.30/18.30 due to start of ORNL LN2 dewar refill cycle > but unrelated to either the BRIKEN-supplied 50MHz TTL clock signal to external clock > input of Master MACB or the ground connection provided by this coaxial cable. > > Which implies the problem is either EMI/RFI or ac mains electrical ground. > > AIDA powered by two single phase outlets from one of four sections of a single phase ac mains distribution > box with power supplied by two heavy duty (3cm dia) cables from Denkenseiki Noisecut AVR type PMN (AVR4, B3F > 104V/51A). AIDA is connected to both NEMA 5-15R-style outlets of section A, LN2 refill to section D, BRIKEN > to sections B, C and D. > > Ambient temperature +24.8 deg C, d.p. +4.1 deg C, RH 26.3% > > 15.00 DSSSD cabling drain wire attachment to AIDA adaptor PCB (FEE64 # 5 & 6) not tightly secured. > CG re-secured drain wire and this has significantly reduced 100kHz period transient noise > as observed at preamplifier outputs. > > 14.48 Restart DAQ (R110) > Master MACB mode 5, 50MHz TTL clock to external clock input > > 14.57 DAQ stalled > > All system wide checks OK except SYNC errors - see attachment 9 > ASIC clock timestamp - nnaida19 fails > N.B. SYNC pulses check OK, counters incrementing between successive tests > The latter is a problem we have seen previously but has not stopped DAQ running OK > > aidas1> ./analyser /TapeData/Sept2016/R110_2 > * TDR format 3.3.0 analyser - TD - October 2016 > * ERROR: READ I/O error: 5002 > blocks: 11861 > ADC data format: 82556471 ( 877427.0 Hz) > Other data format: 14217428 ( 151105.7 Hz) > Sample trace data format: 0 ( 0.0 Hz) > Undefined format: 0 ( 0.0 Hz) > Other data format type: PAUSE: 27 ( 0.3 Hz) > RESUME: 27 ( 0.3 Hz) > SYNC100: 861083 ( 9151.8 Hz) > FEE64 disc: 13353468 ( 141923.4 Hz) > MBS info: 2823 ( 30.0 Hz) > Other info: 0 ( 0.0 Hz) > > ADC data range bit set: 1976 ( 21.0 Hz) > > Timewarps: ADC: 0 ( 0.0 Hz) > PAUSE: 0 ( 0.0 Hz) > RESUME: 0 ( 0.0 Hz) > SYNC100: 0 ( 0.0 Hz) > FEE64 disc: 0 ( 0.0 Hz) > MBS info: 0 ( 0.0 Hz) > Undefined: 0 ( 0.0 Hz) > Sample trace: 0 ( 0.0 Hz) > > * Timestamp elapsed time: 94.089 s > FEE elapsed dead time(s) elapsed idle time(s) > 1 0.084 0.000 > 2 0.076 0.000 > 3 0.000 0.000 > 4 0.000 0.000 > 5 0.109 0.000 > 6 0.012 0.000 > 7 0.000 0.000 > 8 0.000 0.000 > 9 0.003 0.000 > 10 0.000 0.000 > 11 0.000 0.000 > 12 0.000 0.000 > 13 0.000 0.000 > 14 0.000 0.000 > 15 0.032 0.000 > 16 0.013 0.000 > 17 0.000 0.000 > 18 0.000 0.000 > 19 0.125 0.000 > 20 0.000 0.000 > 21 0.000 0.000 > 22 0.000 0.000 > 23 0.142 0.149 > 24 0.076 0.000 > 25 0.000 0.000 > 26 0.000 0.000 > 27 0.000 0.000 > 28 0.000 0.000 > 29 0.000 0.000 > 30 0.000 0.000 > 31 0.000 0.000 > 32 0.000 0.000 > > * Statistics > FEE ADC Data Other Data Sample Undefined Pause Resume SYNC100 Disc MBS Other > HEC Data > 0 0 0 0 0 0 0 0 0 0 0 > 0 > 1 942953 5806860 0 0 3 3 35855 5770999 0 0 > 42 > 2 4036003 4268394 0 0 2 2 35861 4232529 0 0 > 0 > 3 3038422 39132 0 0 0 0 35892 3240 0 0 > 0 > 4 1192440 42063 0 0 0 0 35889 6174 0 0 > 68 > 5 5685966 39310 0 0 5 5 35849 3451 0 0 > 0 > 6 3670394 46605 0 0 2 2 35885 7893 2823 0 > 70 > 7 2661022 43508 0 0 0 0 35890 7618 0 0 > 231 > 8 2165335 41382 0 0 0 0 35890 5492 0 0 > 359 > 9 3037520 863964 0 0 1 1 35889 828073 0 0 > 0 > 10 2616647 179334 0 0 0 0 35889 143445 0 0 > 0 > 11 1083633 39064 0 0 0 0 35889 3175 0 0 > 0 > 12 1759629 38368 0 0 0 0 35888 2480 0 0 > 0 > 13 2419715 280498 0 0 0 0 35891 244607 0 0 > 0 > 14 3189205 1325559 0 0 0 0 35887 1289672 0 0 > 747 > 15 4886680 39482 0 0 2 2 35871 3607 0 0 > 7 > 16 5087977 38869 0 0 1 1 35886 2981 0 0 > 0 > 17 2905638 329411 0 0 0 0 35888 293523 0 0 > 0 > 18 3327386 46016 0 0 0 0 35888 10128 0 0 > 0 > 19 6770177 40711 0 0 4 4 35842 4861 0 0 > 7 > 20 2225755 38524 0 0 0 0 35891 2633 0 0 > 344 > 21 2061728 40299 0 0 0 0 35887 4412 0 0 > 90 > 22 3052329 512109 0 0 0 0 35890 476219 0 0 > 7 > 23 9325689 38673 0 0 3 3 35834 2833 0 0 > 3 > 24 5414228 39293 0 0 4 4 35862 3423 0 0 > 1 > 25 0 0 0 0 0 0 0 0 0 0 > 0 > 26 0 0 0 0 0 0 0 0 0 0 > 0 > 27 0 0 0 0 0 0 0 0 0 0 > 0 > 28 0 0 0 0 0 0 0 0 0 0 > 0 > 29 0 0 0 0 0 0 0 0 0 0 > 0 > 30 0 0 0 0 0 0 0 0 0 0 > 0 > 31 0 0 0 0 0 0 0 0 0 0 > 0 > 32 0 0 0 0 0 0 0 0 0 0 > 0 > > * Timewarps > FEE ADC Pause Resume SYNC100 Disc MBS Undefined Samples > 0 0 0 0 0 0 0 0 0 > 1 0 0 0 0 0 0 0 0 > 2 0 0 0 0 0 0 0 0 > 3 0 0 0 0 0 0 0 0 > 4 0 0 0 0 0 0 0 0 > 5 0 0 0 0 0 0 0 0 > 6 0 0 0 0 0 0 0 0 > 7 0 0 0 0 0 0 0 0 > 8 0 0 0 0 0 0 0 0 > 9 0 0 0 0 0 0 0 0 > 10 0 0 0 0 0 0 0 0 > 11 0 0 0 0 0 0 0 0 > 12 0 0 0 0 0 0 0 0 > 13 0 0 0 0 0 0 0 0 > 14 0 0 0 0 0 0 0 0 > 15 0 0 0 0 0 0 0 0 > 16 0 0 0 0 0 0 0 0 > 17 0 0 0 0 0 0 0 0 > 18 0 0 0 0 0 0 0 0 > 19 0 0 0 0 0 0 0 0 > 20 0 0 0 0 0 0 0 0 > 21 0 0 0 0 0 0 0 0 > 22 0 0 0 0 0 0 0 0 > 23 0 0 0 0 0 0 0 0 > 24 0 0 0 0 0 0 0 0 > 25 0 0 0 0 0 0 0 0 > 26 0 0 0 0 0 0 0 0 > 27 0 0 0 0 0 0 0 0 > 28 0 0 0 0 0 0 0 0 > 29 0 0 0 0 0 0 0 0 > 30 0 0 0 0 0 0 0 0 > 31 0 0 0 0 0 0 0 0 > 32 0 0 0 0 0 0 0 0 > > * Program elapsed time: 1.500s ( 7907.333 blocks/s, 494.208 Mb/s) > > 15.48 Restart aidas1 > Power cycle FEE64s > > Restart AIDA DAQ > > Same problem - DAQ stall (no output from merger) at, or shortly after, Start ASIC Readout with > multiple FEE64s with zero AIDA SYNC rate > All system wide checks OK > > Cannot recover DAQ control of FEE64s with merger restart - power cycle FEE64s again > > > 23.58 Attempt to change master MACB from MACB module #11 mode 5 to MACB module #13 mode 0 > > Successive fuse failures of USB ac mains relay > > System wide ASIC clock timestamp check nnaida 5/6/7/8 13/14/15/16 21/22/23/24 i.e. first half of MACB tree > > Re-seat HDMI cabling from master MACB root module to first half of MACB tree - ReSYNC fails - see attachments > 10 & 11 > > Encoding: HTML ELCode plain Suppress Email notification

Attachment 1:
Drop attachments here...

ELOG V3.1.4-unknown