Attachments 1-5 aida01-aida05 options

System wide checks

Attachment 6 - changes to options files - differences reported but none observed (attachments 1-5)

Attachment 7 - collect memory info from FEE64 Linux

Attachment 8 - collect FPGA timestamp errors

Attachment 9 - WR decoder status

Attachment 10 - ADC calibration

Attachment 11 - Clock status

New run R84

Attempting to sync pneumatic movement with ring. Still see something on t out -> p out

Command to move out was being given relative to the wrong time.

Fixed it. Still see one-two events every time we move out around 4-6 MeV. Attempted to greatly increase time before next injection - no difference. Microphonics from moving the detectors? It's just one-two counts on entire detector, acceptable.

New run R85

Now moving in/out

Potentiometer: 23.10 mm to 67.19 mm

11.15 CARME IE514 ion gauge remains OFF
      ML says there are 2x ion gauges with line of sight to the DSSSDs - CARME IE514 (now off) and another controlled by the vacuum group - he will request that this is 
turned off Monday
      ML says that ion gauges within the upstream e-target should not have line of sight to the DSSSDs
 
      FEE64 power ON

      ASIC settings 
       appear to be using mixture of 2022Jan31-11-52-00, 2024Jan20-12.02.08, 2024Feb08-15.59.09 and 2024Apr24-14.14.03 settings

[npg@carme-gsi Options]$ pwd
/home/npg/MIDAS_Releases/23Jan19/MIDAS_200119/DB/EXPERIMENTS/AIDA/Options
[npg@carme-gsi Options]$ foreach f ( `ls` )
foreach? echo $f
foreach? cat $f/CONTENTS | grep settings
foreach? end
aida01
ASIC.settings string 2024Apr24-14.14.03
aida02
ASIC.settings string 2022Jan31-11-52-00
aida03
ASIC.settings string 2024Jan20-12.02.08
aida04
ASIC.settings string 2022Jan31-11-52-00
aida05
ASIC.settings string 2022Jan31-11-52-00
aida06
ASIC.settings string 2022Jan31-11-52-00
aida07
ASIC.settings string 2022Jan31-11-52-00
aida08
ASIC.settings string 2022Jan31-11-52-00
aida09
ASIC.settings string 2022Jan31-11-52-00
aida10
ASIC.settings string 2022Jan31-11-52-00
aida11
ASIC.settings string 2022Jan31-11-52-00
aida12
ASIC.settings string 2022Jan31-11-52-00
aida13
ASIC.settings string 2022Jan31-11-52-00
aida14
ASIC.settings string 2024Feb08-15.59.09
aida15
ASIC.settings string 2022Jan31-11-52-00
aida16
ASIC.settings string 2024Apr24-14.14.03

      slow comparator 0x64 appears to be set for all FEE64s
      positive input (p+n junction strips) FEE64s aida01, 02, 05, 06, 09, 10, 11, 12, 13, 15 & 16
      negative input (n+n Ohmic strips) FEE64s aida03, 04, 07, 08 & 14

      Check ASIC Load all FEE64s

      All system wide checks OK

      FEE64 temps OK - attachment 1

      DSSSD bias & leakage current - attachment 2
       leakage currents remain high cf. June 2024

      per FEE64 Rate spectra - attachment 3
       high rates observed for DSSSD#1 and DSSSD#3 n+n Ohmic strips
       2x hot p+n junction channels aida15 

      per FEE64 1.8.W spectra - 20us FSR - attachments 4-5

      ADC data item stats - attachment 6
       high rates observed for DSSSD#1 and DSSSD#3 n+n Ohmic strips

      CRYRING vacuum and temperature logs - attachments 7-8

12.51 Options DB re-constructed - see https://elog.ph.ed.ac.uk/CARME/665

      ASIC settings for all FEE64s restored from 2024Apr24-14.14.03
       slow comparator 0x64 appears to be set for all FEE64s
       positive input (p+n junction strips) FEE64s aida01, 02, 05, 06, 11, 12, 15 & 16
       negative input (n+n Ohmic strips) FEE64s aida03, 04, 07, 08, 09, 10, 13 & 14

      WR timestamps OK - attachment 9


14.25 EG&G Ortec 448 Pulser settings

      Amplitude 90,000
      Normalise 5.00
      Attenuator x5 IN
      Frequency Relay Line (50Hz)
      t_r 100ns
      tau_d 50us
      Polarity +

      per FEE64 Rate Spectra - attachments 10-11

      per FEE64 1.8.L spectra - attachments 12-14
       Pulser peak width aida06 87 ch FWHM, aida08 94 ch FWHM
       electronic noise poor consistent with high leakage current and 1.8.W spectra above

      ADC data item stats - attachment 15

15.50 Data transfer to Merger/TapeServer started
       Tape Server -> no storage mode
       Data directory APR25

16.30 DAQ STOP
      FEE64 power OFF
      DSSSD bias OFF

00.12 DAQ continues data file R3_15

      DSSSD bias & leakage current OK - attachment 1

      FEE64 temps OK - attachment 2

      ADC data item stats OK - attachment 3

      per FEE64 Rate spectra - attachment 4

      Merger, TapeServer etc - attachments 5-7


05.32 DAQ continues data file R3_16

      DSSSD bias & leakage current OK - attachment 8

      FEE64 temps OK - attachment 9

      ADC data item stats OK - attachment 10

      per FEE64 Rate spectra - attachment 11

      Merger, TapeServer etc - attachments 12-14

11.27 DAQ continues data file R3_17

      DSSSD bias & leakage current OK - attachment 15

      FEE64 temps OK - attachment 16

      ADC data item stats OK - attachment 17

      per FEE64 Rate spectra - attachment 21

      Merger, TapeServer etc - attachments 18-20

11.33 aida16 asic #2 2x Check ASIC control
      aida16 asic #4 2x Check ASIC control


18.53 DAQ continues data file R3_19

      DSSSD bias & leakage current OK - attachment 22

      FEE64 temps OK - attachment 23

      ADC data item stats OK - attachment 24

      per FEE64 Rate spectra - attachment 25

      Merger, TapeServer etc - attachments 26-28

      per FEE64 1.8.H - attachments 29-30

      per FEE64 1.8.L - attachments 31-32

      per FEE64 1.8.W - 20us FSR - attachments 33-36

19.10 CRYRING ambient temperature per https://web-docs.gsi.de/~lestinsk/tempplot.php - attachment 37

13.50 Cryostat set point 120K

13.49 Temperatures OK - attachment 1

13:51 System wide checks OK except WR decoder status

		 Base 		Current 	Difference
aida01 fault 	 0xab60 : 	 0xab86 : 	 38  
aida02 fault 	 0x88d3 : 	 0x88f8 : 	 37  
aida03 fault 	 0x23aa : 	 0x23d0 : 	 38  
aida04 fault 	 0x15d7 : 	 0x15fd : 	 38  
aida05 fault 	 0x2f03 : 	 0x2f0f : 	 12  
White Rabbit error counter test result: Passed 0, Failed 5

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

13.56 set point 95K

      stat, 1.8.L & 1.8.W spectra - attachments 2-6

DSSSD 0 aida01 1*L spectra, aida04 **L spectra

Double peak structure feature of aida04 ( n+n Ohmic side ) strips. Single ( albeit sometimes broad peak ) observed in aida01 ( p+n junction side ) strips.
Not currently observed by DSSSD 1-3. DSSSD 0 high energy peak rate far higher than DSSSD 1-3.

Data from last week did not display this double peak but DSSSDs and beam have been moved since.

Unclear whether this reflects radiation damage and/or wafer/active area edge effects on the n+n side.
Might also reflect beam characteristics throughout cycle.

Requires calibration and dE cut to investigate.

JM comment -> early time cut analysis indicates this is a beam effect. DSSD#0 has majority of halo on it so double peak is very pronounced. 
We observe double peak all DSSDs just to a much lower level. 

Looking downstream the RHS motor assembly has  loose green/yellow (gnd?) see attachment 1

Install blank base plate to trial install in downstream CARME chamber. The rod is just through all of the mounting holes - see attachments 2 & 3

The rod, plate and block are close fitting.

The current position of the TC feedthrough is incompatible wit detector movement- the TC feedthrough should be moved.

Detector baseplate with 1x DSSSD installed - attachments 4 - 6, Cabling length looks OK. 

Should move NEG power and TCs to top & bottom DN40 flanges to avoid conflicts with DSSSDs and cabling.

12.44 DSSSD#3 bias -149.95V leakage current -6.448uA temperature +19.7 deg C ( https://web-docs.gsi.de/~lestinsk/tempplot.php )

CRYRING cave lights ON - leakage current measurements @ 60s intervals

6.448
6.449
6.448
6.448
6.449
6.448
6.448
6.448
6.449
6.448

18.24 DSSSD#3 bias -149.95V leakage current -6.442uA temperature + 21.0 deg C ( https://web-docs.gsi.de/~lestinsk/tempplot.php )

CRYRING cave lights ON - leakage current measurements @ 60s intervals

6.441
6.441
6.441
6.441
6.441
6.442
6.442
6.442
6.440
6.441

07.51 DAQ continues files 22Feb/R97

      ASIC settings 2022Jan31-11-52-00
       slow comparator 0x64

      EG&G Ortec 448 Pulser OFF

      All system wide checks OK

Attachment 1 - ADC data item stats

Attachment 2 - FEE64 temps OK

Attachments 3 & 4, 8 & 9 - per FEE64 1.8.W spectra 20us FSR and 200us FSR

Attachments 5 & 6 - per FEE64 1.8.L spectra

Attachment 7 - per FEE64 stat spectra

Leakage current remains 8.36 uA, 100V. 

Bias power OFF, SIP ion pump power back ON. NIM crate powered off. Motors pi and nnrpi1 still on for remote access. 
Compressed air off, water off. Fee relay power off and power cable disconnected. Motors box off. Safety pin for motors reinserted. 

Checked cable harnesses at cryring. 2 good short sections found and left with boxes in cryring. 2 damaged long sections (bad pin and broken ceramic) taken to repair in 
Edinburgh.

02.02 DAQ continues G22-86/R15_21
      alpha background

      DSSSD leakage current = attachment 1
       DSSSD #2 transient c. few hundred nA continue

      FEE64 temperatures OK - attachment 2

      ADC data item stats - attachment 3
       rates dominated by 'hot' channels aida01 ( 70k ) and aida15 ( 11k ) - all other FEE64s c. 0Hz

      per FEE64 Rate and Stat spectra - attachments 4-5

02.07 All stats zero'd 


05.52 DAQ continues G22-86/R15_28
      alpha background

      FEE64 temperatures OK - attachment 6

      ADC data item stats - attachment 7
       rates dominated by 'hot' channels aida01 ( 72k ), aida15 ( 11k ), aida12 ( 3k ) and aida05 ( 600 )
       all other FEE64s c. 0Hz

      per FEE64 Stat spectra - attachment 8
       aida12 asic #2 HEC channels high rate

      Merger and TapeServer - attachments 9-10

      DSSSD leakage current = attachment 11
       DSSSD #2 transient c. few hundred nA continue

      online per DSSSD 2D x-y spectra - attachnent 12

05.56 All histograms & stats zero'd
      2x Check ASIC load

      ADC data item stats - attachment 7
       rates dominated by 'hot' channels aida01 ( 72k ), aida15 ( 11k )
       all other FEE64s c. 0Hz

10:28 DAQ continues G22-88/R15

      FEE temperatures OK

      ADC data items stats dominated by aida01 (86k) and aida15 (9k) rest of stats approx OHz. 
      Rates dominated by 2 hot channels per FEE 
      Leakage current plot (attachment 15)
      System wide checks OK
      Merger OK (500,000 items per sec), Tape server OK (710 kb/sec being written)
      DSSD XY plot (attachment 16) - some noise coming through in certain strips on DSSD 3
      DSSD Ey energy histograms (attachment 17)
      DSSD Ey vs Ex plots (attachment 18)     -  statistics for DSSD0,1,3 are consistent (DSSD 2 stats appear higher - unknown reason as yet)

11:10 Leakage current for DSSD1 (I2 on leakage current plot) remained 'turbulent'. We dropped the voltage from 150V to 100V. Leakage current dropped from ~4.5uA to ~4.2 uA. We will see if it remains more stable. 
      Rates in DSSD1 increased during change in voltage so all histograms were zero'd and monitoring code reset. Transient also observed for I3 in the leake current plot (attachment 19)

12:10 No progress on the target so far ...
      Noticed in aida03 and aida04 several missing channels in the stats spectrum browser (attachment 20)
      ASIC check load several times on aida03 and aida04. All stats and histograms zero'd 

13.37 Target now available - will attempt beam + target
      New data file G22-88/R16

13.39 per FEE64 Stat spectra

13.43 All histograms and stats zero'd

      multiple channels missing aida03 and aida04 ( and possibly other FEE64s too )

11.20 CARME IE514 3.53e-9 mbar, ambient temperature 20.6 deg C

      CARME bakeout monitor, ELOTECH R4000 temperature controller front panel and Grafana bakeout dashboard- attachments 1-3

      Change ELOTECH R4000 set points for CARME heaters #1 and #3 to 200 deg C, heater #2 to 150 deg C

14.10 CARME IE514 6.12e-9 mbar, ambient temperature 20.7 deg C

      CARME bakeout monitor, ELOTECH R4000 temperature controller front panel and Grafana bakeout dashboard- attachments 4-6

      Change ELOTECH R4000 set points for CARME heaters #1, #2 and #3 to 225, 175 and 225 deg C respectively.


18.10 CARME IE514 1.92e-8 mbar, ambient temperature 20.8 deg C

      CARME bakeout monitor, ELOTECH R4000 temperature controller front panel and Grafana bakeout dashboard- attachments 7-9

      Change ELOTECH R4000 set points for CARME heaters #1, #2 and #3 to 250, 150 and 225 deg C respectively.
      All other ( active ) ELOTECH R4000 set points ( interaction chamber etc ) to 130 deg C.

20.25 CARME IE514 2.14e-8 mbar, ambient temperature 20.8 deg C

      CARME bakeout monitor, ELOTECH R4000 temperature controller front panel and Grafana bakeout dashboard- attachments 10-12

      No change to ELOTECH R4000 set points

Strain reliefs on the flange side of the cable harness have been installed for two detectors (top right and bottom right detectors). 

We require the strain relief parts from edinburgh in order to install strain reliefs for the left two detectors. 

Find attached images for the strain relief installed and a video of the clamped cables during a movement cycle.

A V-I test was performed for the two detectors which have all strain reliefs installed (see attachment 4).  V-I curve appears similar to the previous test (elog 398). Aluminium foil shield is now a bit battered so may have some small light leaks. Detectors were moved using the motors while bias was applied at 150V. Small change in leakage current observed (~0.01-0.02 uA)

The new MACOR strain reliefs on the (detector side) have been installed. See attached photos

The two pieces  of MACOR are not fully closed but we are confident that the kapton wires are sufficiently clamped.

In order for the strain reliefs to be installed in the correct poisition, two nuts on the jacking plate of the cable harness had to be removed. This is as the groove in the strain relief was not deep enough to accomadate the nut. 

Attached is also a video of the wires (not) moving with the strain reliefs mounted

(Thu 28 October)

All strain relief extension plates mounted behind the wires, using SS countersunk bolts (not silver plated).

Inserting bottom part of MACOR strain relief causes wires to bend too much with strain placed on DSSD. No reasonable way to proceed like that.

Opted to move hard kapton coated copper wire to line up with holes in top MACOR strain relief, and clamp the wrapped wire bundles between top MACOR strain relief and extension plate on bottom.

Results good. No significant movement when wires are moved or motors actuated. See videos - password abcd1234

https://datasync.ed.ac.uk/index.php/s/5waHTSY9husEgy9

https://datasync.ed.ac.uk/index.php/s/34293z7kQs2Mppd

https://datasync.ed.ac.uk/index.php/s/xmHFVqqIchtixGT
 

Mounted plate with MACOR shield on other rod. Connected thermocouples. Re-mounted 2x UHV1400 getters.

With help from OH attempted V-I test of detectors. SHV to 2x lemo broken (faulty lemo connector). Had to borrow AID@DESPEC cable.

Diode function OK. VI tests OK but inconclusive. Both DSSDs drain current when voltage is applied, but current not quite stable. Light leak very likely. Will attempt again after closing.

Re-drilled SS supports in order to fit M3 screws.

Used hand tool cleaned with alchool for 6 of the 8.
Had to use machine for another two with thread damaged.

All 8 will be sent for cleaning. Should be able to start mounting at least the 6 ones tomorrow.

Reason for lack of fit still not clear. They were tested in Edinburgh. Possibly result of high-temperature baking?

Check the status of the DAQ system, with Tom. All correct. See attachments:

1 Leackage currents

2 ADC data item stats

3 White rabbit time stamps

Note: At 9:36 we thing they are working on cooling the beam, there is no target and the detectors are fully out.

9:16 cooled beam, carlo is starting new run r21

Target dencity = 1.47E11 atom/cm2 (10:30 am)

Horizon Y09 from 9 mm to 8 mm, checking different beam position (10:55 am)

Right arm is fully in (11:02 am)

By moving in the detector we are reducing the intencity (11:04 am)

Right numatic out, right servo in, potentiiometer at 38.38 mm (11:05 am)

Losing every second shot, tuning the numatic to get better parameters, beam tuning  (11:13 am)

Leaving the detectors out while going for lunch, still beam (11:42 am)

12.23 Both detectors 37mm in ( DSSSD wafer edge c. 5cm from beam axis )

System Wide Checks Update - Run 21

Temperatures ok, leakage current ok

AIDA16 said that there was an FPGA timestamp error, collected again and now all passed - timestamps on merger statistics page all good when checked

Attatchment 4: Temperatures

Attatchment 5: Waveforms layout 7

Attatchment 6: Waveforms layout 8

Attatchment 7: Wavforms layout 2

Attatchment 8: Waveforms layout 1

Attatchment 9: Leakage current ok

16:11 Both detectors 37mm in ( DSSSD wafer edge c. 5cm from beam axis ), target off, beam being tuned, what ever, still run 21

System Wide Checks Update - Run 21

Temperatures ok, leakage current ok

18:00 Run 21 contd. We have spent the afternoon trying to confirm whether we observe an overlap between the target and the beam. We varied the horizontal beam position and used the top left detector around 19-20 MeV region as a gauge for the count rate.

          This should be reasonably safe as the beam halo in this detector is primarily at low energies. We took the count rate over 10 shots into the ring as our gauge. Attached is a plot of the results of varying the beam position. Target density and gas pressure was apporximately constant throughout.

          We estimate -7 mm is the correct horiziontal beam position. We also had a call off Zoran while at this setting as he observed a decrease in the beam lifetime which was observed in HKR. See attached plot of the scan (attachment 10)

07:56 Gleb has taken the beam to try settings for the 5+ charge state. As soon as the target is ready, the 1+ charge state will be given back.

Detectors have been moved out for safety

08:10
      Run 63
   
      DSSSD bias & leakage current OK - attachment 1

      FEE64 temperatures OK - attachment 2

      ADC data item stats - attachment 3

      per FEE64 Rate spectra - attachment 4

      
09:55 
      Run 63 

      DSSSD bias & leakage current OK 

      FEE64 temperatures OK  

      ADC data item stats OK

      per FEE64 Rate spectra OK

10:30

Data rate appeared very high c. 2000 kB/s instead of the usual 200 kB/s.

Rate on first two ASICs of aida02 was high (~2 kHz) but rest of aida01-04 looked fine.

ASIC check load reduced rate back to normal levels. Data rate reduced back to 200 kB/sec.

New ADC offsets loaded into the CARME_Sort code

Aida03 appears to be down. Check load does not resolve issue. Moving detector arm does not resolve issue. No spike in leakage current observed. System checks appear OK.  Zeroing ADC data item statistics keeps aida03 at 606023857 items and does not zero.

Issue resolved by resetting the data links for aida03

13:20

DSSSD bias & leakage current OK

FEE64 temperatures OK

ADC data item stats OK

per FEE64 Rate spectra OK

21:30

Waiting for the target.

Beam has been running, we have been running without storage.

See attachment showing beam stripe on the detectors.

27/05/24

The carme-gsi computer has been moved from the server room into the shifter control room.

Startup of the DAQ looks OK. Settings loaded 2024Apr24

System wide checks OK. Aida16 clock status fails. Aida16 ADC calibration fails. Rest OK.

All FEEs look OK, we again see high rates in aida15 ~ 12k data items/sec, rate in other FEEs is 0.

Testing of the pulser widths has been done. Pulser widths will be uploaded soon.

Background run started. No beam, no target. File G22-201 and R2. Data being written is ~128 kb/s. Run will be left overnight, no night shift.

28/05/24

DAQ continues with alpha background run.

System wide checks OK. Aida16 clock status fails. Aida16 ADC calibration fails. Rest OK. Temps OK

Data can be seen in the spectrum browser on MIDAS. CARME monitoring code was started incorrectly yesterday. It has now be restarted and a small number of events are seen

At 23:22 we started recording data under the new conditions.

Merger, Tape server (WS3): 1408 kB/s writing ok

Leakage current (WS4): 3.27 uA

DAQ (WS2): temperature checked, clock status:ok, adc calibration:ok, white rabbit check:failed

Online monitoring (WS 5): done

10:40 - Status as attached. Heaters setpoint/read at 250 C, 150 C, 225 C. Power stable at 53%, 41%, 63%

10:45 - Informed HKR ramp down / pump activation is starting for CARME only. Setpoints to 200, 125, 200 C. Monitoring slow ramp down.

14:10 - Heater temperatures reached the set points. Power stabilising. CARME internals temperature dropped to around 100 C and still falling. Waiting until later today to further reduce heater setpoint. Will probably aim for 80 C before beginning activation.

15:45 - Heater setpoints to 175, 100, 175 C

15:50 - Interaction chamber jacket setpoints to 100C

18:00 -  CARME temperature decreasing, not stable yet. Will check later tonight.

20:43 - CARME internals around 85C, not quite stable still. Waiting overnight.