AIDA GELINA BRIKEN nToF CRIB ISOLDE CIRCE nTOFCapture DESPEC DTAS EDI_PSA 179Ta CARME StellarModelling DCF K40
  DESPEC, Page 30 of 35  ELOG logo
ID Date Author Subject
  108   Wed Dec 18 02:44:27 2019 NH17.12.19, 18.12.19
Beam: 238U (850 MeV/u)
Fragment: 168Re

Tuning accelerator and FRS, believed since 3AM we are implanting on AIDA.

FEE issues:
One HEC in aida03 seems to be empty - LEC channel fills
aida09 has a lot of weird channels, check/load doesn't seem to fix
aida10 still no data after cable replaced

DSSD leakages OK - slightly higher now

Temps good


Fig 4: Rates 14:32

Leakages and temps good AS Before

Checking if we are implanting properly - possibly into the first DSSD at the moment

Fig 5 FRS PID
Fig 6 DSSD 
Fig 7 DSSD (PID gated)
  107   Mon Dec 16 14:48:02 2019 NH TDAIDA 16.12.19
DSSD Snout was reassembled to install the second bPlast (downstream) 
 (fig 1)

Ribbon cable for aida10 was replaced (DSSD3 upper)

All DSSDs bias again (see fig 2)

Alpha run in progress - channels OK with 2 exceptions  (fig 3)

- aida10 still recording minimal to no counts

- channel 55 aida08 reading lots of data even with comparator 0x64 - corresponds to DSSD1 strip 0 (bottom strip)
  possible ground/cable issue? will check when access possible (fig 4)

Unusual MIDAS error noticed (fig 5) - restart of HTTPd fixed it
  106   Wed Dec 11 10:58:52 2019 NHInitital Dec. Analysis
Fig 1: 2D hit pattern of each AIDA DSSD for MBS run 24 (40Ar primary beam, centred)

Low statistics in DSSD2 partly due to low energy deposition 

Fig 2: Position according to TPC42 (ca. 2 metres before AIDA)

Fig 3: 2D hit pattern of DSSD 1 for MBS run 37 (34Si fragments, no PID gate)

-
Bottom part of AIDA noticably cut off - rotation in DSSD also noticed (not an issue)

Snout was investigated and platform was discovered to not be fully level by Bogdan using a laser level - he has adjusted it so that the AIDA snout is now level to within 1 mm.
(The beam right side was previous 4-6mm below the beam left side)

Y cut in fragment run is currently of unknown origin. Optics are being investigated

-

Fig 4: Hit pattern of 40Ar with strip offsets fixed (no missing strips)

Fig 5: Energy loss in DSSD1 of 40Ar primary beam

Fig 6: TPC 41, TPC 42, AIDA X comparison

Fig 7: " Y comparison

Fig 8-11: As above for 34Si run

Y cut unknown still.


 
Statistics of 40Ar file
======================================
AIDA Unpacker Analysis
--------------------------------------
First event: Fri Dec  6 05:21:42 2019
Last event : Fri Dec  6 05:26:17 2019
Duration   : 274.604 seconds
--------------------------------------
MBS Events   (M): 131808426 (481 kHz)
AIDA Words   (W): 110608065 (403 kHz)
ADC Words    (A): 36869349 (134 kHz)
ADC Decays   (D): 36782621 (134 kHz)
ADC Implants (I):    86728 (316 Hz)
Info Words   (F): 73738716 (269 kHz)
SYNC48 Words (S): 36869354 (134 kHz)
SYNC63 Words (s): 36869354 (134 kHz)
PAUSE Words  (P):        4 (0 Hz)
RESUME Words (R):        4 (0 Hz)
Discrim Words(D):        0 (0 Hz)
Unknown Words(U):        0 (0 Hz)
Timewarps    (T):        0 (0 Hz)
Dead Time    (X): 0.113631 s (0.0413799%)
======================================
======================================
Individual FEE Overview
======================================
 FEE |        W        A        D        I        F        S        s    P    R    D    U    T        X
   1 |  4383018  1461006  1439401    21605  2922012  1461006  1461006    0    0    0    0    0 0
   2 | 11468351  3822783  3814577     8206  7645568  3822783  3822783    1    1    0    0    0 0.00733666
   3 |  4353624  1451208  1428500    22708  2902416  1451208  1451208    0    0    0    0    0 0
   4 |  8355002  2785000  2755870    29130  5570002  2785001  2785001    0    0    0    0    0 0
   5 |  1133544   377848   376489     1359   755696   377848   377848    0    0    0    0    0 0
   6 |  9617940  3205980  3205767      213  6411960  3205980  3205980    0    0    0    0    0 0
   7 |  4385477  1461825  1460962      863  2923652  1461826  1461826    0    0    0    0    0 0
   8 | 15105135  5035043  5032985     2058 10070092  5035045  5035045    1    1    0    0    0 0.0126429
   9 | 37487211 12495735 12495422      313 24991476 12495736 12495736    2    2    0    0    0 0.0936512
  10 |   110316    36772    36772        0    73544    36772    36772    0    0    0    0    0 0
  11 | 12329904  4109968  4109763      205  8219936  4109968  4109968    0    0    0    0    0 0
  12 |  1878543   626181   626113       68  1252362   626181   626181    0    0    0    0    0 0


Statistics of 34Si file
======================================
AIDA Unpacker Analysis
--------------------------------------
First event: Sat Dec  7 05:20:22 2019
Last event : Sat Dec  7 05:23:51 2019
Duration   : 208.635 seconds
--------------------------------------
MBS Events   (M): 148639738 (714 kHz)
AIDA Words   (W): 132033086 (634 kHz)
ADC Words    (A): 44011024 (211 kHz)
ADC Decays   (D): 43874168 (210 kHz)
ADC Implants (I):   136856 (657 Hz)
Info Words   (F): 88022062 (423 kHz)
SYNC48 Words (S): 44011028 (211 kHz)
SYNC63 Words (s): 44011028 (211 kHz)
PAUSE Words  (P):        3 (0 Hz)
RESUME Words (R):        3 (0 Hz)
Discrim Words(D):        0 (0 Hz)
Unknown Words(U):        0 (0 Hz)
Timewarps    (T):        0 (0 Hz)
Dead Time    (X): 0.0806864 s (0.0386734%)
======================================
======================================
Individual FEE Overview
======================================
 FEE |        W        A        D        I        F        S        s    P    R    D    U    T        X
   1 |  6814953  2271651  2255653    15998  4543302  2271651  2271651    0    0    0    0    0 0
   2 | 14012582  4670860  4668280     2580  9341722  4670860  4670860    1    1    0    0    0 0.0088226
   3 |  7618290  2539430  2523614    15816  5078860  2539430  2539430    0    0    0    0    0 0
   4 |  8928934  2976310  2950380    25930  5952624  2976312  2976312    0    0    0    0    0 0
   5 |  1631415   543805   531957    11848  1087610   543805   543805    0    0    0    0    0 0
   6 | 10908021  3636007  3633830     2177  7272014  3636007  3636007    0    0    0    0    0 0
   7 |  5644296  1881432  1864053    17379  3762864  1881432  1881432    0    0    0    0    0 0
   8 | 14863755  4954585  4931620    22965  9909170  4954585  4954585    0    0    0    0    0 0
   9 | 36745143 12248379 12244135     4244 24496764 12248380 12248380    2    2    0    0    0 0.0718638
  10 |   198573    66191    66191        0   132382    66191    66191    0    0    0    0    0 0
  11 | 15101900  5033966  5018830    15136 10067934  5033967  5033967    0    0    0    0    0 0
  12 |  9565224  3188408  3185625     2783  6376816  3188408  3188408    0    0    0    0    0 0

All good (no deadtime issues)
  105   Sat Dec 7 02:39:02 2019 DK, NH et al.34Si Fragments PID
See attached

Tentative PID (DK) as attachment #3

DESPEC LISE++ file as attachment #4
  104   Fri Dec 6 20:34:56 2019 CA, DK, NHDec 6 Night Shift
21.35 AIDA DAQ currently running, TapeService on, not writing to storage

      System wide checks ok

      Attachments 1-3: temperatures, good event stats, detector bias/leakage currents ok

22.04 waiting for beam. Possibility of fragments, further news to come

22.43 expecting fragments with 300 MeV/u 40Ar primary beam

      histograms zeroed

00.31 still waiting for beam

      detector bias/leakage currents ok (see google spreadsheet)

      FEE64 temps and good event statistics ok

02:02 still awaiting beam - issues with interlock

03:17 beam tuning - centering primary beam on target

03:43 beam reached AIDA, fragmentation beam (see elog entry by DK for PID)

      primary beam 300 Mev/u 40Ar, fragments in region of 34Si

      started writing to file Dec19/R2 at rate ~ 4000 kb/s

      histograms zeroed

03.49 system wide checks all ok

      attachments 4-6:

      temperatures ok

      bias/leakage currents ok

      good event statistics ok, except aida09 running slightly fast

04:03 attachments 7-8 - 1.8.H energy spectra

                      - no high energy events seen in aida10 -> checked ASIC control

                      - still no high energy events in aida10

04:09 attachment 9 - rate spectra for all FEE modules

04.16 attachment 10 - 1D hits for all FEE modules

04.19 beam stop - file R2_3

04:52 beam back on - file R2_6 - rate of ~ 5000 ions per spill

06:13 temperatures ok

      good event statistics ok (aida09 still running fast) - attachment 11

      detector bias/ leakage currents ok - see google spreadsheet

      all system wide checks ok

      1.8.H spectra - attachments 12/13

06:20 attachment 14 - rate spectra for all FEE modules

      attachment 15 - 1D hits for all FEE modules

      still no high energy events in aida10


      
  103   Thu Dec 5 22:00:11 2019 CA, DK, NHDec 5 Night Shift
15:30
  TD found the merger was running at ~43 Mb/s
  The odd-numbered FEE64s all had significant (>200k) disk rates
  ALL LEC/MEC fast discriminators and increased slow comparator from 0xa to 0x14
  Merger rate dropped to a more nominal value like 2.3 Mb/s
  Settings not saved

40Ar beam 300 MeV/u expected, but SIS problem at the moment.

23:00
  Leak currents are stable as previously (20-21 Nov) and entered to Google spreadsheet
  S4 vault is closed.
  Presently AIDA is running without tape storage
  System wide checks: 
    Master clock / aida01 sync reports error as expected; 
    All white rabbit decoders fail, but they are just off by 1 bit (no problem)
    Everything else okay.
  Stats look okay, but keep in mind above discriminator threshold -- see attachment 1
  Temperatures fine -- see attachment 2
  MBS looks okay -- we see a possible hint for the 1 bit difference with an "unexpected data word" -- see attachment 3 
  Data rate is 1.7 Mb/s, merger okay -- see attachment 4

23:30 
  Now we wait for the beam

00:05
  Attachments 5-11: results of system wide checks after collecting sync errors and WR errors from baseline - all ok

00:25
  Histograms zeroed at 00:25.

00:35

  1.8.L pulser peak widths

FEE64       width(ch)
  1            92
  2            131
  3            75
  4            97
  5            89
  6            117
  7            90
  8            87
  9            223
 10            58
 11            112
 12            126 

no peak observed on aida09

For comparison, pulser peaks measured on 13/11/19:

pulser peak widths:
                   FEE      width (ch)

                   aida01 - 130

                   aida02 - 168
             
                   aida03 - 83

                   aida04 - 83

                   aida05 - 200

                   aida06 - 148

                   aida07 - 250

                   aida08 - 81

                   aida09 - 134

                   aida10 - 190

                   aida11 - 333

                   aida12 - 149

2:00
  40Ar beam tuning is finished.  Intensity at S4 is about 3 kHz per spill.  1 spill per / 10 seconds
  MBS system cannot be used for online merging tonight, because NeuLAND group left their connection open
  We can merge offline using the White Rabbit timestamps
  Remove the beam blocking steel flange from in front of AIDA et al.
  Writing to file Dec19/R1

03.20 Rate, Stat, 1.8.L, 1.8.H, 1.8.W spectra - attachments 13-20
      Observe
       light ions in 1.8.L spectra
       low energy (<1.5GeV) heavy ions in 1.8.H spectra with most heavy ions stopping in DSSSDs 1 & 2
       Stat hit patterns indicate broad (c. 5cm FWHM) centred heavy ion peak

03.24 Detector biases & leakage currents OK - attachment 21
      FEE64 temperatures OK - attachment 22
      Good event statistics (12 of 12 < 50k) - attachment 23

      System wide checks
      List All Sync Error Counters - attachment 24
      List all Sync Received Counters - attachment 25

      Baseline invoked earlier c. 00.05
      Collect all SYNC Error Counters for Baseline
      Collect All White Rabbit status Error Counters for Baseline 

04:07 2.6 mm Al degrader. 300Hz beam. 300 MeV/u 40Ar beam.

04.14 pulser peak seen in aida09 after performing ASIC check-load earlier in shift - width = 219 ch

04.45 1.8.H energy spectra (attachment 26/27)

06:08 attachments 28-30: detector bias/leakage currents ok
                         temperatures ok
                         good event statistics ok
                         all system wide checks ok

08:49 DAQ stop/start. TapeServer writing to No Storage. End of beam.


      
  102   Wed Nov 27 16:33:50 2019 NHNew Bias & Waveforms

HV core (-160 V) now attached to bottom 3 FEEs (with grounded kapton)

DSSDs bias OK, leakages unchanged

No appreciable change in waveforms

Some estimates

aida01/aida05/aida09 VHF noise: 5 channels frequency = 10 MHz ?

All FEEs:
Oscillation at ~25-35 channels = 1.4 to 2 MHz ?
Varies a bit with each FEE.

No obvious issue with upper FEEs unsure why 10 MHz is present - fee09 has been replaced but showed it before as well.

Lower frequency noise on every FEE may be ground noise from e.g. switching PSU?

No more access to S4 to check thoroughly again.

 

 

 

 

  101   Mon Nov 25 13:25:51 2019 NHWaveforms
> AIDA Waveforms immediately after beam at 21.11.19 8am or so.

Update 25.11.19 with all FEE64 waveforms added

Things to note:
aida01, aida05, aida09 are all on TOP - show VHF noise, esp. aida01?
TOP is where HV is connected (-ve core)

Everything noisy in general though
  100   Fri Nov 22 08:55:31 2019 NHWaveforms
AIDA Waveforms immediately after beam at 21.11.19 8am or so.
  99   Wed Nov 20 20:36:14 2019 CA, CB, DK, NH20/21 November Night Shift
21:35 (NH) - A few merger timestamp errors were appearing - Decided to powercycle all FEE64s, all checks OK.
 No Screenshot due to forgetting before I closed merger 
 FEEs were often not stopping properly and needing merger reset as well.

Beam Plan:

40Ar 300 MeV/u Primary Beam
and/or
34Si Fragment from Ar

22:45

FEES were restart near these timestamps in the TapeData, evidently
-rw-rw-r--.  1 npg  npg  2.0G Nov 20 21:17 R1_361
-rw-rw-r--.  1 npg  npg  2.0G Nov 20 21:36 R1_362

Run number was kept the same (Tape Server was not stopped @ FEE restart?)

All system wide checks are okay (except of course the master clock fails)
Temps look okay (attachment 1, dated 21/Nov by mistake)
Leak currents all around 0.9 uA as before (attachment 2)
Stats are behaved. We don't have beam yet (attachment 3)
Merger and tape server happy (see attachment 4)
Data rate histos while beam is still off (see attachment 5)

==Nov 21==

0:00
TD noted that the data rates have gone up at some point (about double).  
Now we increased the slow comparator thresholds from 0xa to 0xb (data rate drops from 24 Mb/s to 16 Mb/s).
We increased it again from 0xb to 0xc, but near that time the beam was also turned on, and the data rate was about 25 Mb/s.

Beam was turned on near R1_472 or a bit earlier.

TD requested waveforms, but at the moment they are not appearing.  Will enable WFs after a condition change.

Beam seen in all three DSSSDs, see attachment 6.  Beam was turned off before I could get more screencaps.

Degrader change ... near R1_481

00:32 waveforms now enabled under Run Control.  They also appear to be enabled on ASIC1 undre LED and Waveform controls.  However, we are unable to see the waveforms in the relevant histograms.

00:44 3 g/cm^2 inserted R1_488

00:45 3.2 g/cm^2 degrader in R1_489.

1:00 Several degraders and combinations are being tested since last note.

1:10 It seems 1 g/cm^2 of energy loss is not accounted for by LISE++.  Trying to understand the reason.

1:28 Satisfied with a total degrader thickness of 2.6 g/cm^2 -- seems to calculate that fragments should implant mainly in DSSSD2?  Issue seems to be resolved updating the LISE++ file.
  Near R1_507

Can see the implant samples attached as #7 and #8

1:43 According to the stats, most of the beam energy is being deposited in DSSSD1.  See #9.
  Present beam burst repetition is around 0.1 Hz (each ~10 sec)

2:00 better example of stats histos (see #10)

2:45 tuning the FRS degraders (ones for RIB separation rather than AIDA implantation depth).

3:17 Just started fragments.  Disregard previous filenames with "34Si".  Finally no degrader in the FRS was needed to get Z separation.
  R1_559
  As the DAQs are not synchronized, a quick call to the BNC PB-5 pulser shifiting from 2 -> 50 -> 2 Hz was used to make a signal burst.
  S4 degrader is 5.2 g/cm^2

3:22 Updated biases in Google Sheets.

Having trouble seeing the fragments in the AIDA histograms.  Since we just started fragments, zero the histos.

3:35 Attachment 11, with 34Si fragments as Stats.  Difficult to click Update on the Rates at the right time.  
   From the beam monitors, fragments should be at around 3000 cps.

3:45 (Up to now, some optics tuning with slits, etc)

3:46 Optics settings accepted, AIDA at R1_572, accept this condition for data runs on ~34Si fragments.

3:47 Sent a pulser burst 2 -> 50 -> 2 Hz

3:50 zero histograms again.
   Temperatures attachment 12
   Biases attachment 13
   Statistics attachment 14
   Merger / Tape Server attachment 15

3:56   
   StatHistos attachment 16

4:09 FRS starts new run #56, AIDA R1_583.  Pulser 2-> 50 -> 2 Hz

4:56 Stat histograms in attachment 17.  Not zeroed since noted at 3:50
  98   Tue Nov 19 21:22:06 2019 CA, CB, DK, NH19/20 November 2019 - overnight shift
19 November

Waiting for 40Ar beam to be delivered for tests.
Objective today is to assess degrader thickness suitable to stop 40Ar in 2nd DSSD layer.
Energy expected at 300 MeV/A, around 1E3-4 beam intensity - to be determined.

Tomorrow fragments could be expected.

DSSD / FEE64 pairing as follows

DSSD3 (most downstream): FEEs 9,10,11,12
DSSD2 (middle): FEEs 5,6,7,8
DSSD1 (most upstream): FEEs 1,2,3,4
+plastic scintillator furthest upstream

New AIDA Google spreadsheet created with leakage current, to monitor them during beamtime. First entry added. Bias attached.
https://docs.google.com/spreadsheets/d/12hgbrywB10hGsKt5uc2HnLfZymh8_uvM6cEASbbqnBE/edit#gid=813167023

DESPEC platform has already been moved. No effect on rates - see attached. FEE1 is missing one channel. FEE9, 10 noisy as expected. See attached.


System wide checks *all OK* except
Master clock failed (normal)
ADC calibration fails on 6 and 12. Recalibrated via FADC Align. All OK now.

MBS transfer OK. See attached.

Good event statistics OK. See attached.

FEE64 temperatures OK. See attached.

Merger and data rate OK. See attached.

Waiting for beam.



20 November

01:07 - Still no beam. 

System wide checks OK. 
Stats OK. 
Temps OK. 
MBS OK. 
Merger and data rate OK.


02:35  - Beam is expected soon
Tried enabling writing data to disk. Merger crashed. FEE64 stopped responding to commands.
Reset Merger. FEE64 now respond to commands.
Tried enabling writing to disk again. Merger crashed once more. Reset merger.
Reset Tape Server interface. Now writing to disk.

No data currently being sent to MBS, but operators say they "are adding a new detector". May be normal.

DAQ & Merger going OK. Writing to Nov19/R1

No beam yet.

Temperatures OK
System wide checks OK
Stats OK. See attached.
MBS not working.
Merger and data rate OK. See attached.

~02:48 Beam on. 40Ar. 300 MeV/A, 500 pps. Degrader settings unclear.

03:54 Degrader settings have been changing between 4.5, 1 and 6 ug/cm2
Currently on 6 g/cm2.
AIDA seen through MBS shows no HE on detector 3, and Det 1 and 2 show no counts on positive y (FEE6 and 1). That is not the case from MIDAS.

04:30 The aim of the tests above was to have no 40Ar at all in DSSD3. It does not appear that 6 g/cm2 is sufficient, as we still see events. The reduction between 1 & 2 vs. 3 is about a factor of 2-10 depending on ADC channel. See attached.

~04:30 Moved to 6.2 g/cm2 degrader. See attached.

~04:45 Moved to 5.8 g/cm2. See attached.

~05:09 Moved to 6.4 g/cm2. See attached.

05:31 High energy channel spectra (1.8.H) for all FEE64 w/ 6.4 g/cm2 degrader - See attachments 13 & 14

05:38: Moved to 1.0 g/cm2. See attached. Difference is not huge which may indicate these spectra are misleading, as partly expected. A more detailed analysis will be carried out off line.

06:00: Beam stopped. ~R77. AIDA keeps running background over the day. Beam should restart around 11 pm later today.
  97   Wed Nov 13 15:38:20 2019 NH[HowTo] MBS for AIDA (Nov 19)
For MBS integration AIDA forwards its data to an MBS Foreign Data Receiver (FDR), currently the PC assigned for this task is x86l-94.

To start the MBS system do the following:

Connect to the MBS FDR: ssh despec@x86l-94 (Ask local for password)

cd to the AIDA directory: cd mbsrun/nov19/aida_to_mbs

Start mbs: mbs

Startup the receiver: @startup

Now open the MBS relay for MIDAS (far right icon on the top)

Check both terminals (Relay & MBS) report a connection

In another terminal ssh into the FDR and run rate to monitor the data rate in MBS

For data to be transferred the Tape Server must be GOING but can be in No Storage mode if no MIDAS storage is required.

--

For experiments MBS file saving is handled by the DESPEC time sorter which merges all the subsystems together.
For testing you can write files from x86l-94 using the following commands

connect rfio XXX -disk
Where XXX is an Linux PC (ask local for a PC)

open file /path/to/file_ first=1 size=2000 -auto -rfio
This opens the file, writing in 2GB chunks

clo file
This will close the file when you are done

--

It is possible to restart the MBS relay at any point if the connection seems to have failed, also confirm the Tape Server is GOing
  96   Wed Nov 13 09:10:37 2019 CA, NH, OH13th November 2019
10:10 moved HV cables from 11 and 12 to 10 and 9 - all detectors biased in same configuration

      lowered slow comparator threshold to 0xa
   
      DAQ start

      pulser on

      zero'd histograms

pulser peak widths:
                   FEE      width (ch)

                   aida01 - 130

                   aida02 - 168
             
                   aida03 - 83

                   aida04 - 83

                   aida05 - 200

                   aida06 - 148

                   aida07 - 250

                   aida08 - 81

                   aida09 - 134

                   aida10 - 190

                   aida11 - 333

                   aida12 - 149

For comparison, peak widths from yesterday's measurement:

                   pulser peak widths -   FEE         width(ch)
                                           1             128
                                           2             145
                                           3             84
                                           4             74
                                           5             207
                                           6             125
                                           7             172
                                           8             80
                                           9             193
                                           10            183
                                           11            238
                                           12            138

Attachments 1 & 2 : pulser peak spectra

Attachment 3 - good event statistics

Attachment 4 - detector bias/ leakage currents

                   
  95   Wed Nov 13 01:54:00 2019 TDAida10 Unusual Pulser Peak

Double peaking normally indicates periodic baseline variations with period comparable to shaping time

Quote:

aida10 (no alpha rate recorded) has a strange pulser peak - looks like double peak?

Pulser terminator checked and OK...

All others "OK" but noise quite bad (see previous entry)

 

  94   Tue Nov 12 14:10:18 2019 NH, OH, CAAida10 Unusual Pulser Peak

aida10 (no alpha rate recorded) has a strange pulser peak - looks like double peak?

Pulser terminator checked and OK...

All others "OK" but noise quite bad (see previous entry)

  93   Tue Nov 12 13:59:07 2019 NH, OH, CAAwfully noisy waveforms
Rates in all FEEs much higher than before - waveform shows very noisy in all systems. - attachment 1

Good event statistics - attachment 2

DISC in 7 and 3 above 300k

ASIC check load performed.

Good event statistics - attachment 3

Thesholds at 0xa for all FEE

Disc rate now 0 across all FEE


Pulser settings: attachment 4

pulser peak widths -   FEE         width(ch)
                              1             128
                              2             145
                              3             84
                              4             74
                              5             207
                              6             125
                              7             172
                              8             80
                              9             193
                              10            183
                              11            238
                              12            138

Attachments 5 & 6: pulser peaks for all FEE64

16.24: DAQ stop

       slow comparator threshold changed to 0x64

       ASIC check performed

       Alpha run/DAQ start (w/ data transfer enabled) - writing to file 31Oct19/R13

       Merger and TapeService ok - attachments 7 & 8
  92   Tue Nov 12 13:22:19 2019 OH, NH, CAAIDA09 Replacement
ASIC 1 on FEE9 was not producing signals.

A new HDMI cable was installed but still no signals.

AIDA09 was replaced with a new FEE.
Current list of FEE to serial numbers installed attachment 1

MAC address of new FEE was obtained.

Backup of DHCPD.conf was made dhcpd.confBACKUP191112

dhcpd.conf was updated with new MAC address.

All FEEs powered on and 09 was seen to mount and is seen by AIDAServer.

AIDA09 flashed to newest version of firmware. (0x18430701) - attachment 2
  91   Mon Nov 11 10:34:19 2019 NHAlpha Analysis (11.11.19)

Analysis of approx 20GB of data since Thursday night (not continuous - estimate of hours incominG)

Figures 1-3: Energy Front vs Energy Back (No F-B energy gate)

Figures 4-6: 2D hit pattern

Figures 7-9: 1D hit pattern

aida10 does not seem to be reading out still, unsure of reason? Looks fine with pulser.
Slow comparator not setting or unusual gain?

Otherwise good

  90   Thu Nov 7 10:22:26 2019 Nic, PatrickReply: WR Timestamps
> > > All 12 FEEs have valid WR Timestamps
> > > Had to powercycle aida09 once as before raw readout was displaying upper 12 bits of WR timestamp as 0. Unsure of other method.
> > > 
> > > HDMI cables in aida09 checked and good.
> > 
> > The problem would be the cable , one end or the other. 
> > I think ( if I recall ) a setup would restart the WR decoder.
> > 
> > I notice you have set the WR info word rate to be quite high , 6123/sec typ, is this intentional ?
> 
> Cable will be replaced once a spare is available.
> 
> Setup did not restart the WR decoder when this problem occurred beforehand - Reset/Setup tried.
> 
> WR rate was chosen by Vic I believe, I am unsure of reasoning myself.

An update/clarification:
Although the upper bits are zero I believe actually it is a total failure to synchronise to WR:

aida09
WR Time Item 0x80500000 0x0fbd8000; Time (48:63)=0x0; Time (28:47)=0x249; Time (0:27)=0x0fbd8000
WR Time Item 0x80400249 0x0fbd8000; Time (28:47)=0x249; Time (0:27)=0x0fbd8000
WR Time Item 0x80500000 0x0fbdc000; Time (48:63)=0x0; Time (28:47)=0x249; Time (0:27)=0x0fbdc000
WR Time Item 0x80400249 0x0fbdc000; Time (28:47)=0x249; Time (0:27)=0x0fbdc000
WR Time Item 0x80500000 0x0fbe0000; Time (48:63)=0x0; Time (28:47)=0x249; Time (0:27)=0x0fbe0000
WR Time Item 0x80400249 0x0fbe0000; Time (28:47)=0x249; Time (0:27)=0x0fbe0000

aida10
WR Time Item 0x8050022e 0x0bde8000; Time (48:63)=0x22e; Time (28:47)=0xe29d5; Time (0:27)=0x0bde8000
WR Time Item 0x804e29d5 0x0bde8000; Time (28:47)=0xe29d5; Time (0:27)=0x0bde8000
WR Time Item 0x8050022e 0x0bdec000; Time (48:63)=0x22e; Time (28:47)=0xe29d5; Time (0:27)=0x0bdec000
WR Time Item 0x804e29d5 0x0bdec000; Time (28:47)=0xe29d5; Time (0:27)=0x0bdec000
WR Time Item 0x8050022e 0x0bdf0000; Time (48:63)=0x22e; Time (28:47)=0xe29d5; Time (0:27)=0x0bdf0000
WR Time Item 0x804e29d5 0x0bdf0000; Time (28:47)=0xe29d5; Time (0:27)=0x0bdf0000
  89   Tue Nov 5 11:15:41 2019 NH PJCSReport: aida10 database corruption

If ypou now check the actual values in the aida10 Options screen are the correct ?

It can happen that corruptions once loaded are propagated when Options are generally saved. 

Looking at the dates the files were modified does that make sense to you. It doesn't look as if the corrupted database was changed except when they all were ? Can you check ? Have you checked the other Options files ? I seem to recall that RIKEN AIDA had a program to to this ?

Quote:

Hi, see the attached files in the original comment. I think the BACKUPCorrupt is quite old now.

The Options data gets corrupted and for example the ASIC folder gets set to 'undefined' which I think is why the ASIC data loads incorrectly.

 

Quote:

This looks like the Options not the ASIC data. That will be in the CONTENTS file of the FEE64 named directory within the dated EXPERIMENT directory. Looks like the latest is at /MIDAS/DB/EXPERIMENTS/AIDA/2019Oct31-13.24.23/aida10 ?

A quick look at this file and it looks normal , not all 0xad ? 

Looked at the Options file in the /MIDAS/DB/EXPERIMENTS/AIDA/Options.BACKUPCorrupt/aida10 and the worst bit is the name of the Data Acquistion program.  That could welll cause you problems with readout as it won't understand some of the new data.

 

Quote:

Often aida10 database seems to get corrupted (ASIC values are all 0xad)
Unsure why it's only aida10, might be related to weird alpha rate behaviour at the moment.

 

Attached is corrupted database and correct database

 

 

 

ELOG V3.1.4-unknown