AIDA GELINA BRIKEN nToF CRIB ISOLDE CIRCE nTOFCapture DESPEC DTAS EDI_PSA 179Ta CARME StellarModelling DCF K40
  DESPEC, Page 22 of 36  ELOG logo
New entries since:Thu Jan 1 01:00:00 1970
ID Date Authordown Subject
  102   Wed Nov 27 16:33:50 2019 NHNew Bias & Waveforms

HV core (-160 V) now attached to bottom 3 FEEs (with grounded kapton)

DSSDs bias OK, leakages unchanged

No appreciable change in waveforms

Some estimates

aida01/aida05/aida09 VHF noise: 5 channels frequency = 10 MHz ?

All FEEs:
Oscillation at ~25-35 channels = 1.4 to 2 MHz ?
Varies a bit with each FEE.

No obvious issue with upper FEEs unsure why 10 MHz is present - fee09 has been replaced but showed it before as well.

Lower frequency noise on every FEE may be ground noise from e.g. switching PSU?

No more access to S4 to check thoroughly again.

 

 

 

 

Attachment 1: waves_new_odd.png
waves_new_odd.png
Attachment 2: waves_new_even.png
waves_new_even.png
  106   Wed Dec 11 10:58:52 2019 NHInitital Dec. Analysis
Fig 1: 2D hit pattern of each AIDA DSSD for MBS run 24 (40Ar primary beam, centred)

Low statistics in DSSD2 partly due to low energy deposition 

Fig 2: Position according to TPC42 (ca. 2 metres before AIDA)

Fig 3: 2D hit pattern of DSSD 1 for MBS run 37 (34Si fragments, no PID gate)

-
Bottom part of AIDA noticably cut off - rotation in DSSD also noticed (not an issue)

Snout was investigated and platform was discovered to not be fully level by Bogdan using a laser level - he has adjusted it so that the AIDA snout is now level to within 1 mm.
(The beam right side was previous 4-6mm below the beam left side)

Y cut in fragment run is currently of unknown origin. Optics are being investigated

-

Fig 4: Hit pattern of 40Ar with strip offsets fixed (no missing strips)

Fig 5: Energy loss in DSSD1 of 40Ar primary beam

Fig 6: TPC 41, TPC 42, AIDA X comparison

Fig 7: " Y comparison

Fig 8-11: As above for 34Si run

Y cut unknown still.


 
Statistics of 40Ar file
======================================
AIDA Unpacker Analysis
--------------------------------------
First event: Fri Dec  6 05:21:42 2019
Last event : Fri Dec  6 05:26:17 2019
Duration   : 274.604 seconds
--------------------------------------
MBS Events   (M): 131808426 (481 kHz)
AIDA Words   (W): 110608065 (403 kHz)
ADC Words    (A): 36869349 (134 kHz)
ADC Decays   (D): 36782621 (134 kHz)
ADC Implants (I):    86728 (316 Hz)
Info Words   (F): 73738716 (269 kHz)
SYNC48 Words (S): 36869354 (134 kHz)
SYNC63 Words (s): 36869354 (134 kHz)
PAUSE Words  (P):        4 (0 Hz)
RESUME Words (R):        4 (0 Hz)
Discrim Words(D):        0 (0 Hz)
Unknown Words(U):        0 (0 Hz)
Timewarps    (T):        0 (0 Hz)
Dead Time    (X): 0.113631 s (0.0413799%)
======================================
======================================
Individual FEE Overview
======================================
 FEE |        W        A        D        I        F        S        s    P    R    D    U    T        X
   1 |  4383018  1461006  1439401    21605  2922012  1461006  1461006    0    0    0    0    0 0
   2 | 11468351  3822783  3814577     8206  7645568  3822783  3822783    1    1    0    0    0 0.00733666
   3 |  4353624  1451208  1428500    22708  2902416  1451208  1451208    0    0    0    0    0 0
   4 |  8355002  2785000  2755870    29130  5570002  2785001  2785001    0    0    0    0    0 0
   5 |  1133544   377848   376489     1359   755696   377848   377848    0    0    0    0    0 0
   6 |  9617940  3205980  3205767      213  6411960  3205980  3205980    0    0    0    0    0 0
   7 |  4385477  1461825  1460962      863  2923652  1461826  1461826    0    0    0    0    0 0
   8 | 15105135  5035043  5032985     2058 10070092  5035045  5035045    1    1    0    0    0 0.0126429
   9 | 37487211 12495735 12495422      313 24991476 12495736 12495736    2    2    0    0    0 0.0936512
  10 |   110316    36772    36772        0    73544    36772    36772    0    0    0    0    0 0
  11 | 12329904  4109968  4109763      205  8219936  4109968  4109968    0    0    0    0    0 0
  12 |  1878543   626181   626113       68  1252362   626181   626181    0    0    0    0    0 0


Statistics of 34Si file
======================================
AIDA Unpacker Analysis
--------------------------------------
First event: Sat Dec  7 05:20:22 2019
Last event : Sat Dec  7 05:23:51 2019
Duration   : 208.635 seconds
--------------------------------------
MBS Events   (M): 148639738 (714 kHz)
AIDA Words   (W): 132033086 (634 kHz)
ADC Words    (A): 44011024 (211 kHz)
ADC Decays   (D): 43874168 (210 kHz)
ADC Implants (I):   136856 (657 Hz)
Info Words   (F): 88022062 (423 kHz)
SYNC48 Words (S): 44011028 (211 kHz)
SYNC63 Words (s): 44011028 (211 kHz)
PAUSE Words  (P):        3 (0 Hz)
RESUME Words (R):        3 (0 Hz)
Discrim Words(D):        0 (0 Hz)
Unknown Words(U):        0 (0 Hz)
Timewarps    (T):        0 (0 Hz)
Dead Time    (X): 0.0806864 s (0.0386734%)
======================================
======================================
Individual FEE Overview
======================================
 FEE |        W        A        D        I        F        S        s    P    R    D    U    T        X
   1 |  6814953  2271651  2255653    15998  4543302  2271651  2271651    0    0    0    0    0 0
   2 | 14012582  4670860  4668280     2580  9341722  4670860  4670860    1    1    0    0    0 0.0088226
   3 |  7618290  2539430  2523614    15816  5078860  2539430  2539430    0    0    0    0    0 0
   4 |  8928934  2976310  2950380    25930  5952624  2976312  2976312    0    0    0    0    0 0
   5 |  1631415   543805   531957    11848  1087610   543805   543805    0    0    0    0    0 0
   6 | 10908021  3636007  3633830     2177  7272014  3636007  3636007    0    0    0    0    0 0
   7 |  5644296  1881432  1864053    17379  3762864  1881432  1881432    0    0    0    0    0 0
   8 | 14863755  4954585  4931620    22965  9909170  4954585  4954585    0    0    0    0    0 0
   9 | 36745143 12248379 12244135     4244 24496764 12248380 12248380    2    2    0    0    0 0.0718638
  10 |   198573    66191    66191        0   132382    66191    66191    0    0    0    0    0 0
  11 | 15101900  5033966  5018830    15136 10067934  5033967  5033967    0    0    0    0    0 0
  12 |  9565224  3188408  3185625     2783  6376816  3188408  3188408    0    0    0    0    0 0

All good (no deadtime issues)
Attachment 1: AIDA_HitDec19.png
AIDA_HitDec19.png
Attachment 2: TPCs.png
TPCs.png
Attachment 3: AIDA-fragments.png
AIDA-fragments.png
Attachment 4: AIDA_XY.png
AIDA_XY.png
Attachment 5: AIDA_E.png
AIDA_E.png
Attachment 6: TPC_41_42_AIDA_X.png
TPC_41_42_AIDA_X.png
Attachment 7: TPC_41_42_AIDA_Y.png
TPC_41_42_AIDA_Y.png
Attachment 8: AIDA_XY_34Si.png
AIDA_XY_34Si.png
Attachment 9: AIDA_E_34Si.png
AIDA_E_34Si.png
Attachment 10: TPC_41_42_AIDA_X_34Si.png
TPC_41_42_AIDA_X_34Si.png
Attachment 11: TPC_41_42_AIDA_Y_34Si.png
TPC_41_42_AIDA_Y_34Si.png
  108   Wed Dec 18 02:44:27 2019 NH17.12.19, 18.12.19
Beam: 238U (850 MeV/u)
Fragment: 168Re

Tuning accelerator and FRS, believed since 3AM we are implanting on AIDA.

FEE issues:
One HEC in aida03 seems to be empty - LEC channel fills
aida09 has a lot of weird channels, check/load doesn't seem to fix
aida10 still no data after cable replaced

DSSD leakages OK - slightly higher now

Temps good


Fig 4: Rates 14:32

Leakages and temps good AS Before

Checking if we are implanting properly - possibly into the first DSSD at the moment

Fig 5 FRS PID
Fig 6 DSSD 
Fig 7 DSSD (PID gated)
Attachment 1: 1812-stats.png
1812-stats.png
Attachment 2: 1812-leakages.png
1812-leakages.png
Attachment 3: 1812-temps.png
1812-temps.png
Attachment 4: 1812v2-Rates.png
1812v2-Rates.png
Attachment 5: nh_1712_zaoq.png
nh_1712_zaoq.png
Attachment 6: nh_1712_dssd.png
nh_1712_dssd.png
Attachment 7: nh_1712_dssd_id.png
nh_1712_dssd_id.png
  118   Fri Dec 20 10:19:29 2019 NHAlpha Analysis (Pre Run)
======================================                                                                                                                                                                                                                                        
AIDA Unpacker Analysis                                                                                                                                                                                                                                                        
--------------------------------------                                                                                                                                                                                                                                        
First event: Fri Dec 13 15:34:12 2019                                                                                                                                                                                                                                         
Last event : Tue Dec 17 08:52:14 2019                                                                                                                                                                                                                                         
Duration   : 321482 seconds  =  90 hours
--------------------------------------                                                                                                                                                                                                                                        
MBS Events   (M): 1748810227 (5 kHz)                                                                                                                                                                                                                                          
AIDA Words   (W): 1748072047 (5 kHz)                                                                                                                                                                                                                                          
ADC Words    (A): 785414339 (2 kHz)                                                                                                                                                                                                                                           
-- Decays    (D): 771101846 (2 kHz)                                                                                                                                                                                                                                           
-- Implants  (I):  14312493 (44 Hz)                                                                                                                                                                                                                                           
Info Words   (F): 962657708 (2 kHz)                                                                                                                                                                                                                                           
-- SYNC48    (S): 2012396012 (6 kHz)                                                                                                                                                                                                                                          
-- SYNC63    (s): 2012396012 (6 kHz)                                                                                                                                                                                                                                          
-- PAUSE     (P):   2925640 (9 Hz)                                                                                                                                                                                                                                            
-- RESUME    (R):   2925667 (9 Hz)                                                                                                                                                                                                                                            
-- Discrim   (D): 1226981673 (3 kHz)                                                                                                                                                                                                                                          
-- Unknown   (U):         0 (0 Hz)                                                                                                                                                                                                                                            
Timewarps    (T):         2 (0 Hz)
Dead Time    (X): 4336.67292 s (1.34896%)
======================================

Possibly one bond wire loss in DSSD 1
Possibly a few lost in DSSD 3 (X)
DSSD 2 is OK.

Will update with post run data for additional statistics
Attachment 1: Alphas.png
Alphas.png
Attachment 2: Alphas_E.png
Alphas_E.png
  120   Wed Jan 15 14:53:41 2020 NHAIDA Noise 14.01.2019
Checking noise

DESPEC crate entirely offline except:
VME Crate (for VETAR2)
AIDA (All)

Figs 1-2: Waveforms (Odd/Even)
Figs 3-4: Zoomed In (Odd/Even)
Fig 5: Good Event Stats
Fig 6: Merger Rate

Noise still present, despite nothing on DESPEC.

Things to check: 
Turn off VME (kills time stamps but might be OK for waveform test?)
Check floor PSU under AIDA is offline too 

Not sure that other FRS systems (e.g. Ion Catcher) is offline - I think RF may be on.
Attachment 1: Waves_Odd.png
Waves_Odd.png
Attachment 2: Waves_even.png
Waves_even.png
Attachment 3: WavesZ_Odd.png
WavesZ_Odd.png
Attachment 4: WavesZ_Even.png
WavesZ_Even.png
Attachment 5: EvtStas0xA.png
EvtStas0xA.png
Attachment 6: Merger0XA.png
Merger0XA.png
  121   Mon Jan 20 14:01:23 2020 NHAIDA Noise 20.01.2019
> Checking noise
> 
> DESPEC crate entirely offline except:
> VME Crate (for VETAR2)
> AIDA (All)
> 
> Figs 1-2: Waveforms (Odd/Even)
> Figs 3-4: Zoomed In (Odd/Even)
> Fig 5: Good Event Stats
> Fig 6: Merger Rate
> 
> Noise still present, despite nothing on DESPEC.
> 
> Things to check: 
> Turn off VME (kills time stamps but might be OK for waveform test?)
> Check floor PSU under AIDA is offline too 
> 
> Not sure that other FRS systems (e.g. Ion Catcher) is offline - I think RF may be on.

Update with different sampling rates (in order)
10 (200 ns)
100 (2 us)
1000 (20 us)

Note that some waveforms didn't update very reliably
In particular aida11 does not seem right at large sample rates
Attachment 1: Odd_10_Z.png
Odd_10_Z.png
Attachment 2: Even_10_Z.png
Even_10_Z.png
Attachment 3: Odd_100_Z.png
Odd_100_Z.png
Attachment 4: Even_100_Z.png
Even_100_Z.png
Attachment 5: Odd_1000_Z.png
Odd_1000_Z.png
Attachment 6: Even_1000_Z.png
Even_1000_Z.png
  122   Wed Jan 29 12:22:13 2020 NHaida10 troubleshooting
aida10 has no ASIC events when running without pulser

DSSD ribbon cable was replaced, no change

29.01.2020 - change AIDA half adapter board, no change

Swapped HV (braid) from aida10 to aida12 - no change in leakage current, now testing data...
 Looks the same (no data) fig 2

Very hot channel in aida08 as well:
 - Channel 104 in histogram Stat = 4.7.L  fig 1

Change FEE64 card?
Any other checks possible?

31.01.2020:

Installed new FEE64 card aida10: d8:80:39:41:ba:89
Updated dhcpd.conf
System boots OK.
returned bias ground to aida10 for symmetry reasons
Firmware updated from 0x12110103 to 0x18430701

Left recording alpha data over the weekend:
See stats in fig 3

DSSD 1 and 2 look excellent (one broken strip in DSSd 1)?

aida10 still no counts?
aida09 looks very... weird as well?
aida11 and aida12 seem ok?

Detector fault?
Attachment 1: aida08-noise.png
aida08-noise.png
Attachment 2: aida10-noitems.png
aida10-noitems.png
Attachment 3: Stats.png
Stats.png
  124   Wed Jan 29 19:09:51 2020 NHaida10 troubleshooting
> aida10 has no ASIC events when running without pulser
> 
> DSSD ribbon cable was replaced, no change
> 
> 29.01.2020 - change AIDA half adapter board, no change
> 
> Swapped HV (braid) from aida10 to aida12 - no change in leakage current, now testing data...
>  Looks the same (no data) fig 2
> 
> Very hot channel in aida08 as well:
>  - Channel 104 in histogram Stat = 4.7.L  fig 1
> 
> Change FEE64 card?
> Any other checks possible?

when running the pulser in 10 do all the channels respond correctly ?
what happens if you drop the shaper threshold into the noise ?
  125   Fri Feb 14 12:50:29 2020 NHAIDA Noise 14.02.2020
Electricians changed the grounding for the DESPEC platform to a large pillar that goes deep into soil

Change in grounding was apparently successful at reducing the noise complaints in Cave C (R3B)

AIDA:

Fig 1 & 2: Waveforms (Odd/Even)
Fig 3 & 4: Waveforms Zoomed
Fig 5: Good Event Rate
Fig 6: Merger Rate: 2 M items/sec
Fig 7: HV supply (no change)

There does seem to be a modest improvement (Merger rate down from 3 million, good event rates seem *better*)
(Reminder: AIDA@GSI sends 3 items per each ADC event, so merger rate is higher)
Not sure if this is still where it should stand?
Attachment 1: wavesOdd.png
wavesOdd.png
Attachment 2: wavesEven.png
wavesEven.png
Attachment 3: wavesOddZ.png
wavesOddZ.png
Attachment 4: stats_goodevents.png
stats_goodevents.png
Attachment 5: mergerstats.png
mergerstats.png
Attachment 6: hv.png
hv.png
  126   Mon Feb 17 09:48:34 2020 NHReport: aida09 Kernel Panic & Lost WR
aida09 crashed over the weekend and automatically rebooted.
After the reboot the WR timestamp sent to the merger is in the future and hence incorrect

Reset/Setup did not fix issue
Sync ASICs did not fix issue
GSI White Rabbit control page shows a correct WR timestamp

Attach 1: ttyUSB12 (aida09 log with kernel panic)
Attach 2: GSI White Rabbit control page
Attach 3: "Collect All WR Timestamps"
Attach 4: RAW Display for aida09
 
WR Time Item 0x80500232 0x0de48000; Time (48:63)=0x232; Time (28:47)=0x20310; Time (0:27)=0x0de48000
WR Time Item 0x80420310 0x0de48000; Time (28:47)=0x20310; Time (0:27)=0x0de48000

WR Timestamp = 0x23220310de48000 * 10 = 0x15F541EA 8AED0000 = 2020-02-21 CET 01:01:59.699537920
c.f. "GSI page" timestamp starting 0x15F427F5 

Attach 5: Timestamp shown by merger
Attachment 1: ttyUSB12
Attachment 2: GSI_WR.png
GSI_WR.png
Attachment 3: GSI_WR2.png
GSI_WR2.png
Attachment 4: GSI_WR4.png
GSI_WR4.png
Attachment 5: GSI_WR3.png
GSI_WR3.png
  129   Fri Feb 21 09:21:10 2020 NH(Early) Alpha Analysis
Overnight alpha data

======================================
AIDA Unpacker Analysis
--------------------------------------
First event: Thu Feb 20 17:46:47 2020
Last event : Fri Feb 21 10:07:57 2020
Duration   : 58869.9 seconds
--------------------------------------
MBS Events   (M):     1248648 (21 Hz)
AIDA Words   (W): 10222680133 (27 kHz)
ADC Words    (A):  3407431088 (-15076 Hz)
-- Decays    (D):  3407017078 (-15083 Hz)
-- Implants  (I):      414010 (7 Hz)
Info Words   (F):  6815249045 (-30146 Hz)
-- SYNC48    (S):  3407559635 (-15074 Hz)
-- SYNC63    (s):  3407559635 (-15074 Hz)
-- PAUSE     (P):         614 (0 Hz)
-- RESUME    (R):         614 (0 Hz)
-- Discrim   (D):      128547 (2 Hz)
-- Unknown   (U):           0 (0 Hz)
Timewarps    (T):          79 (0 Hz)
Dead Time    (X): 26.8658 s (0.04563588%)
======================================
======================================
Individual FEE Overview
======================================
 FEE |           W           A           D           I           F           S           s     P     R       D     U     T  X
   1 |        5025        1246        1093         153        3779        1675        1675     0     0     429     0     0  0
   2 |        6180         991         851         140        5189        2060        2060     0     0    1069     0     0  0
   3 |  7692461404  2564028610  2564028286         324  5128432794  2564153392  2564153392   614   614 	124782     0    58 26.8658
   4 |        2277         642         581          61        1635         759         759     0     0     117     0     0  0
   5 |        6174        1249        1010         239        4925        2058        2058     0     0     809     0     0  0
   6 |     2467062      822244      822191          53     1644818      822354      822354     0     0     110     0     0  0
   7 |        2484         740         701          39        1744         828         828     0     0      88     0     0  0
   8 |   358445004   119481621   119069026      412595   238963383   119481668   119481668     0     0      47     0     1  0
   9 |        7134        1530        1243         287        5604        2378        2378     0     0     848     0     0  0
  10 |  2168031885   722677295   722677295           0  1445354590   722677295   722677295     0     0       0     0    20  0
  11 |        3981        1252        1190          62        2729        1327        1327     0     0      75     0     0  0
  12 |     1241523      413668      413611          57      827855      413841      413841     0     0     173     0     0  0
**** DESPECAnalysis: PostLoop

A few FEEs running "hot", maybe check. 3 had some alignment issues may need rechecking.
Timewarps somehow in aida03 and aida10 but may be code mistake - must check.
Only aida03 ran fast enough to have dead time.

aida10 shows decay data now - DSSD seems ok

Fig 1: DSSD 1D hit pattern 

Fig 2: DSSD Energy

Fig 3: Single channel in aida03 that is hot

Fig 4: Channels 48-64 in aida08 are hot (wiring?)

Fig 5: Channels 17, 19 (crosstalk?) and 27, 33, 38, 39

All otehr channels quiet and OK. 
Attachment 1: DSSD_1d_HP.png
DSSD_1d_HP.png
Attachment 2: DSSD_E.png
DSSD_E.png
Attachment 3: FEE3_43_Hot.png
FEE3_43_Hot.png
Attachment 4: fee8_hot.png
fee8_hot.png
Attachment 5: Fee10_hot.png
Fee10_hot.png
  133   Tue Feb 25 09:55:59 2020 NHHV Scope Traces
Looking at the HV bias in a scope to see if it is the source of noise especially on the p+n side

Channel 1: HV Core channel 4
Channel 2: HV Braid Channel 4
Math: Ch1 - Ch2

100V bias (100 uA leakage current for 1 MOhm)

No obvious frequency components or large frequency components?
I do not know how this compares to RIKEN or STFC-DL

Updated to put the right frequency range (misread before)

Fig 3-4: Normalish looking data 

Fig 5-6: Examples of noise appearing randomly on math line

Fig 7: 1ms long trace, noise stopped appearing

Update 26.02.2020

Fig 8: Trace with all 4 HV channels off

Fig 9: Trace with only oscilloscope biased

Fig 10: Trace with braids unplugged from all 3 FEE64s (only channel 4 connected)

Things to try:

Joining clean earth & chassis ground?

Checking HV jumper?

Finding 'scope probes and testing NIM socket?
Attachment 1: SCRN0057.PNG
SCRN0057.PNG
Attachment 2: SCRN0054.PNG
SCRN0054.PNG
Attachment 3: SCRN0059.PNG
SCRN0059.PNG
Attachment 4: SCRN0060.PNG
SCRN0060.PNG
Attachment 5: SCRN0071.PNG
SCRN0071.PNG
Attachment 6: SCRN0073.PNG
SCRN0073.PNG
Attachment 7: SCRN0075.PNG
SCRN0075.PNG
Attachment 8: SCRN0078.PNG
SCRN0078.PNG
Attachment 9: SCRN0079.PNG
SCRN0079.PNG
Attachment 10: SCRN0080.PNG
SCRN0080.PNG
  134   Sat Feb 29 10:56:38 2020 NHAlpha Hit Pattern
Circa 700 GB of data

A few channels in DSSD 1 missing? Otherwise seems OK
Attachment 1: alphas_1d.pdf
alphas_1d.pdf
  135   Mon Mar 9 12:47:18 2020 NHReport: Unusual AIDA Timestamp Error
Noted using my AIDA event builder code, a timewarp was reported and I noticed the ADC event had a different timestamp LSB to the "WR" markers preceding it, which I didn't think was possible in the merger as it stands

 aida event 875015fa 0f68098c | INFO 5 SYNC6348 08 015fa
 aida event 8744b164 0f68098c | INFO 4 SYNC4828 08 4b164
 event time: 15fa4b164f68098c
 aida event c1f6851b 0f68098c | ADC 08:54 L 851b
 event time: 15fa4b164f68098c
 aida event 875015fa 0f68115c | INFO 5 SYNC6348 08 015fa
 aida event 8744b164 0f68115c | INFO 4 SYNC4828 08 4b164
 event time: 15fa4b164f68115c
 aida event c1ef8238 0f68115c | ADC 08:47 L 8238
 event time: 15fa4b164f68115c
 aida event 875015fa 0f68115c | INFO 5 SYNC6348 08 015fa
 aida event 8744b164 0f68115c | INFO 4 SYNC4828 08 4b164
 event time: 15fa4b164f68115c
 aida event c1f884b0 0f68115c | ADC 08:56 L 84b0
 event time: 15fa4b164f68115c
 aida event 8a5015fa 0f681382 | INFO 5 SYNC6348 11 015fa
 aida event 8a44b164 0f681382 | INFO 4 SYNC4828 11 4b164
 event time: 15fa4b164f681382
 aida event c2817dc8 0f681382 | ADC 11:01 L 7dc8
 event time: 15fa4b164f681382
 aida event 875015fa 0f68192c | INFO 5 SYNC6348 08 015fa
 aida event 8744b164 0f68192c | INFO 4 SYNC4828 08 4b164
 event time: 15fa4b164f68192c
 aida event c1f9849c 0f68192c | ADC 08:57 L 849c
 event time: 15fa4b164f68192c
 aida event 815015fa 0f681f94 | INFO 5 SYNC6348 02 015fa
 aida event 8144b164 0f681f94 | INFO 4 SYNC4828 02 4b164
 event time: 15fa4b164f681f94
 aida event c0568084 04b6854a | ADC 02:22 L 8084
 event time: 15fa4b1644b6854a
AIDA Timewarp (15fa4b1644b6854a before 15fa4b164f68192c)
Event reported errors, skipping this file...
Events: 506459240   506459240             (0 errors)

Unsure if the cause, easily handled by ignoring such events but should be "impossible"
  156   Mon Mar 16 07:25:24 2020 NHEnd of Experiment

08:25 Beam Stopped, end of S480

Tape Server stopped at R12_2097

Started again in no storage mode

Will leave compressing runs

  159   Tue Jul 7 08:04:57 2020 NHPower failure in Messhuette
Last week there was a power failure in Messhuette (FRS control room) - AIDA-3 and MBS were affected.
AIDA itself and S4 equipment is on UPS and was not shut down.

AIDA-3 has been restarted and MIDAS restarted. All seems OK

S4 conditions: 24.2 C / 33.5% RH / Td = 7.1 C 
  160   Tue Jul 7 08:50:27 2020 NHnnrpi2 does not boot
MIDAS was not working properly on nnrpi2 - power cycled. Now system does not boot properly, it gets stuck trying to login claiming "PAM critical error" - probable file system corruption?
Will investigate if SD card can be checked and repaired
  161   Tue Oct 6 15:48:06 2020 NHnnrpi kernel update
FYI:

The SD card on nnrpi2 was full again causing issues, the log files in /var/log (kern.log & messages) were *huge* and filled with the same error many times a second

Oct  4 15:01:22 raspberrypi kernel: [1030048.603846]                                                                                                                                                                                                                                                                         
Oct  4 15:01:22 raspberrypi kernel: [1030048.607742] WARN::dwc_otg_handle_mode_mismatch_intr:68: Mode Mismatch Interrupt: currently in Host mode                                                                                                                                                                             
Oct  4 15:01:22 raspberrypi kernel: [1030048.607742]                                                                                                                                                                                                                                                                         
Oct  4 15:01:22 raspberrypi kernel: [1030048.667755] WARN::dwc_otg_handle_mode_mismatch_intr:68: Mode Mismatch Interrupt: currently in Host mode                                                                                                                                                                             
Oct  4 15:01:22 raspberrypi kernel: [1030048.667755]                                                                                                                                                                                                                                                                         
Oct  4 15:01:22 raspberrypi kernel: [1030048.667848] WARN::dwc_otg_handle_mode_mismatch_intr:68: Mode Mismatch Interrupt: currently in Host mode                                                                                                                                                                             
Oct  4 15:01:22 raspberrypi kernel: [1030048.667848]                                                                                                                                                                                                                                                                         
Oct  4 15:01:22 raspberrypi kernel: [1030048.731753] WARN::dwc_otg_handle_mode_mismatch_intr:68: Mode Mismatch Interrupt: currently in Host mode                                                                                                                                                                             
Oct  4 15:01:22 raspberrypi kernel: [1030048.731753]                                                                                             

A google issue showed that this was fixed in an update to Linux kernel, so I updated the kernel to the latest version.
now the message doesn't seem to appear so hopefully the pis won't break so rapidly.
  162   Tue Nov 24 13:10:42 2020 NHWaveforms with frame grounded
AIDA Frame is now grounded to platform (will add photo later)

Waveforms improved from April (c.f. https://elog.ph.ed.ac.uk/DESPEC/125) but comparison not truly fair (other systems may be offline now)
Attachment 1: waves_odd.png
waves_odd.png
Attachment 2: waves_even.png
waves_even.png
Attachment 3: 20201124_164416.jpg
20201124_164416.jpg
Attachment 4: 20201124_164413.jpg
20201124_164413.jpg
  163   Mon Feb 1 15:19:07 2021 NHAIDA Status Feb 2021
01.02.2021 - Four new sets of PSU, HDMI & Ethernet cables were ran from the DESPEC rack to the AIDA frame, in preparation for 4 new FEE64s.
A new 7 port USB hub has also been placed for the last few serial connections.

For the first experiment this year, we will run with a narrow AIDA configuration as in 2020. (12 FEEs)

The system has been powered up and biased successfully. To be monitored over the next few days.

A lot of the FEE serial monitors are missing, I think a hub has been disconnected from nnrpi1. This will be investigated on Wednesday.
The rack will also be grounded properly on Wednesday to investigate the impact on the noise.

A single new water line (feed and return) needs to be installed for the 16th FEE, this will be done once the hose is found/ordered...

The 4 new FEE64s will be mounted and powered up at some point to obtain the MAC addresses, and possibly check MIDAS merging 16 FEE64s.

Waveform data will be taken wednesday before & after grounding the rack to see noise situation currently
ELOG V3.1.3-7933898