CARME setup checked by vaccum group this morning. 

Both IE514 gauges de-gassed. To de-gas on the CM52 controller select the channel and hold the right arrow for two seconds. The reading will flash indicating the de-gassing procedure, this lasts for 2 minutes before completing. Flashing automatically stops once procedure finishes. Attached for convinience is the wire colour-pin assignment for the gauge head again.

Baking jacket fully assembled around CARME and baking code updated. Temperature and rate limits updated to match thermocouples around the chamber.

                        Max Temperature                Max Rate

Chamber                155C                               10C

Floating                  175C                              150C

Octagon (outside tent)  50C                           10C 

 Automatic baking started at 13:50, pressure was 1.69e-7 mbar

Fabre (German)       Color (English)

grau                         grey

blau                         blue

rot                            red

braun                       brown

weiss                        white

rosa                          pink

gelb/gruen                yellow/green

To test effect of the motors on the leakage current, they were cycled once per hour overnight at approximately 50 minutes past the hour. - attachment 1
Leakgage current was seen to decrease slowly overnight before flattening out.

At 11:25 This morning the FEEs were powered on. From this point on the leakage current was observed to start to increase. - Attachment 2
Note the time recorded by the pi logging the leakage current is approximately 2 minutes ahead of the FEE.

15:40 I notice again that powering on the FEEs causes a momentary increase in the leakage current. - Attachment 3

09:00 Detector still biased to 60V and leakage current has been mostly stable overnight.
      Only minor fluctuations observed - attachment 1

      FEE temps all ok - attachment 2
      Stats at 60V - attachment 3
      System wide checks all ok

      Increasing bias to 80V
      60V 9.45uA
      70V 9.7uA
      80V 80 9.82uA Still have diode response

10:15 In prep for taking data with beam tuning have tested the merger and tapeserver.
      To enable writing to file had to implement the method in https://elog.ph.ed.ac.uk/DESPEC/36
      Probably a corrupution caused in the difference between the carme system and the aida system.
      Now can write data to file. To limit the data of FEEs 01-04 have set threshold to 0xFF

10:33 Ion source was started this morning but there was no noticable effect on the leakage currents.

10:56 Leakage current is slowly rising continuing the trend at 60V. - Attachment 4
      This could just be the temperature shift during the day or linked to the ion source being powered on.

11:18 Pulser peaks for the FEEs using layout 3 and 4.
      Layout 3 - attachment 5
      Layout 4 - attachment 6
      The pulser peaks appear to be asymetric and in FEE 7 we are observing a double peak.
      Will check the circuit when we next have access.

11:23 Have started writing data to disk in advance of them putting beam into the ring.
      Writing to R4_8 at around 1.8MB/s. N.B. TapeServer shows the same behavior as AIDA with writing nothing to files R4_0->R4_7
      FEE 01-04 slow comp 0xFF
      FEE 05-08 slow comp 0x64
      Pulser running at 50Hz or so

14:50 Having been writing to DAQ since lunch.
      Leakage currents were mostly stable but we are now seeing an increase in leakage currents at a much greater rate - attachment 7

14:58 Attachment8 Updated leakage currents.

15:00 Due to the rapidly increasing bias we asked them to stop the beam.
      Motors were then cycled but we did not notice the leakage coming down.
      Bias was turned off and then motors cycled again.
      Bias was then set to 60V - Run R5
      Bias raised to 80V - R6
      Carlo requested we stop writing data to disk

16:25 Stopped writing to disk. Will retry bias at 80V and monitor.

18:13 Motors cycled before leaving for night no beam in ring - attachment 9
      to test if the increase is beam related we will leave at 80V overnight with the beam blocked all night.
      To avoid the rapidly increasing leakage current the iSet on the CAEN module is at 11uA with a trip time of 2s.
      

Removed thermocouples, heaters, RE72 modules and leftover baking tent. Support octagon removed except topmost part which requires disconnecting pneumatic hoses going to top gate valve. To be done on Monday with ML assistance.

Attempted to move servo motors once more - did not work. The motors appear stuck but it is not clear how or where. Removed screws on flanged ball nut and on motor back. Still stuck. Moving by hand is not possible. Code likely complains about excessive force on motor startup. Will discuss with PB. Some dismounting once back in air may be required.

Schematic layout of heaters and channel assignment attached. Table kindly provided by AK. I hope the conversion from excel to pdf didn't swallow any important information.

09:00

Came in to see the bias for DSSD4 which has been fluctuating for the past few days had tripped during the night. From the gnuplot it looks like it was fluctuating all night and just after midnight it went above 20uA and tripped.

Other leakage currents are around the value they have been recently, day night cycle still observed. Again perhaps the leakage current of DSSD1 and DSSD3 is slowly decreasing and reaching a lower minimum value. Reset the module, DSSD4 does bias back up and is currently around 10uA. See attachments 1+2.

I have also noticed a small spike in current when starting up the FEE power, see the small spikes in attachment 2.

Continuing with different ground configurations from yesterday.

Ground config 5: I have now removed the grounding cables between non-bias FEE modules. So grounding cable goes between pn and nn bias FEEs and then one grounding cable to the coolant pipe. No grounding on any other FEEs. (attachments 3-8). Effect on pulser widths is hard to discern, some widths have decreased i.e 5 and 6 wheras others have increased such as 11. One interesting note is that the rates on aida03 have decreased by just over a factor 2 so now read ~100k. Whether this is the result of different grounding or maybe similar to what was seen for DSSD3 where rates decreased on there own over time? Attachment 8 shows the rates for each ASIC and most have now reached similar level to the other FEEs (note log scale).

Ground config 6: Have kept grounds to only pn or nn bias FEEs. Have now also included a ground cable from the body of the FEE module into the grounding connection. The ADC data items rate increase from 100k to 150k in aida03, no other impact on rates. Pulser widths remain largely unchanged, although some widths saw increases in the pulser widths. (attachments 9-13)

Ground config 7: Wanted to check impact of having no grounds at all. No impact on leakage currents. Rates look pretty similar for all FEEs, aida03 and aida04 200k and 350k respectively. Pulser widths and waveforms significantly worse than grounded case. See attachments 14-18

Ground config 8: Have reattached all ground cables (back to config1). In addition I have added a ground cable from the coolant pipe to effectively the CARME chamber. Rates appear pretty unchanged. Pulser widths better than no grounding but worse than original configuration. Attachments 19-23

Note since after lunch DSSD4 leakage current no longer fluctuating. Nothing has been changed. As I write that currents start fluctuating again. Maybe some ongoing work in the cave is affecting this?

Removed groud cable going to the chamber.

I wanted to rule out any affect from the test inputs. Have removed all test inputs. Only input is directly from pulser to DSSD3 as this previously had best pulser widths. All other cables removed. See attachments 24-26. Still see very high rates in aida03 and aida04 (250k and 350k). Pulser widths in DSSD3 seem worse now for some reason? While pulser detattched decided to lower slow comparator to 0x32 (500keV) and 0x24 (360 keV) to see rates in all FEEs. Rates still look pretty good, other than aida03 and 04. See attachments 27,29. Putting test inputs back.

Test inputs all back. Ground configuration back to configuration 1. Pulser tests, widths are best so far - will upload them later from all tests.Rates good in all except aida03 and aida04 still.

FEEs off, bias on and monitoring. Have put pins back in motors.

20:50

Bias has tripped again on DSSD4. It had remiained stable with noi fluctuations ~20 mins before leaving.

> Initial grounding see final grounding from elog 187. ASIC settings for all - LEC slow comparator 0x64. Note on 11/02/22 1.8*L widths with same settings were aida1-4: 300,240,250,260

ADC data items <10k

Good events fluctuate 10k-20k

1.8*L pulser widths aida1-4: 433,400,375,375(double peak)

Attachments 1-4

> New grounding implemented. 3mm grounding cable connect fee body to adaptor board. Each adaptor board is then connected directly to the cryring ground. Attachements 10-11

ADC data items <10k

Good events fluctuate 10k-20k

1.8*L pulser widths aida1-4: 340,288,288,295 (reduced double peak)

Noise in the system also seems to have decreaed in zoomed out 1.8*L plots (not attached). Aida1 still shows large noise peak channel 32000 on 1.8*L. 1.8*W no significant change, perhaps slightly reduced.

Attachements 5-9

Screencaps for following movement tests not taken as 1.8*L and 1.8*W spectra appeared the very similar as 5-9 except for slightly different widths.  

> Detectors moved in and out (pneumatic). Leakage current dropped by ~0.2 uA, torque now reads +2.8 previously was ~0. 

no change to ADC data items or good events. During movement aida3 saw momentary increase in good events to ~40k before returning to normal range. Rest of fees good events stats unaffected by movement.

1.8*L pulser widths aida1-4: 413,313,295,306

> servo motor (only) moved, torque reduced back to ~0

no change in good events stats for any fees during servo movement. 

1.8*L pulser widths aida1-4: 425,335,316,288

> Detector moved to IN position (pnuematic and servo). 19.3mm to 81.8mm on potentiometer. 

No change to good events. During movements aida3 saw momentary increase in good events to ~41k, all other fees unaffected. 

1.8*L pulser widths aida1-4: 302,221,305,293 

Noise peak at channel ~32000 on aida 1 (1.8*L) significantly reduced. little noise on other fees 

>Detector moved to fully OUT position (pneumatic and servo). 

no change in good events, during movement aida3 saw momentary increase to ~35k

1.8*L pulser widths aida1-4: 450,350,316,290

Noise peak at chn 32000 on aida 1 has increased to previous size. little noise on other fees. 

> Bias increased to 120V. Leakage current stable however slightly higher ~0.1 uA

no change in good events. 

1.8*L pulser widths aida1-4: 447,345,189,301

> Bias increased to 140V, increase in leakage current ~0.2 uA, appears stable. Increase to 150V, fluctuate in leakage current of +/-0.5uA then huge increase to >20uA trip. 

> Fee power cycle. Bias 100V, leakage current 9.9 uA ~0.1-0.2 uA higher than previous but stable. 

no change to good events

1.8*L pulser widths aida1-4: 468, 350,311, 296

> Some decrease seen with new grounding, however main source of noise still apparent and any decrease seems insignificant compared to changes due to position movement. 

> Water pipes have been zipped tied so they no loger touch motors, no change in pulser widths or noise levels however.  

Grounding

Ground cables have been daisy chained together from FEE to FEE. Ground cables are screwed into a feee bolt hole on the FEE body using an M3 bolt. 

LEMO cables have been connected in a similar way between the grounds on the adaptor boards for each FEE. Short LEMO cables go between J6 on pairs of adaptor boards in the same FEE holder and long LEMO cables go between J7 on adaptor boards in different FEE holders. 

The LEMO grounds from the adaptor boards and the ground cables attached to the FEE bodies are then connected together and then connected to the CRYRING grounding cable. 

Pulser

Jan has kindly lent us a pulser. We send the pulser to an inverter giving us a +ve and -ve signal. 

The +ve signal goes into the pn FEEs (1,2,5,6)

The -ve signal goes into the nn FEEs (3,4,78)

LEMO cable connecting FEEs in the same holder using J3 on adaptor board. J4 is used for recieving the pulse signal, connecting FEEs in different holders or terminating the signal. 

JM plans to install new crimped, 3mm dia copper drain wires (a la AIDA@DESPEC) this afternoon. 

Initial Grounding - Lemo cables connecting adaptor cards ground in daisy chain. Fee bodies connected by 3mm copper drain wires in daisy chain. Two chains brought together 
at final adaptor card which is connected to cryring ground. 

Attachments 1-5

ADC data item rates   all <10k
ASIC settings         all LEC slow comparators 0x64
1.8.L spectra aida05-08 widths 556, 511, ?, 570 ch FWHM respectively
1.8.W spectra 20us FSR
Adaptor card grounding image

1.8L widths appear broader than previous day of ~300 width. Reason unknown, no change made to setup as yet. 

Second Grounding - 3mm copper drain wires connect each adaptor card to Fee body. Fee bodies daisy chained together using 3mm copper drain wire and connected to cryring 
ground. 

ADC data item rates   all <10k
ASIC settings         all LEC slow comparators 0x64

Attachments 6-8
1.8.L spectra aida05-06 widths 620, 550 ch FWHM respectively, 1.1.L aida08 width 350 ch FWHM
1.8.W spectra 20us FSR. Some extra high frequency noise
Adaptor card grounding 
 

Third Grounding - 3mm copper drain wires connect adaptor cards in daisy chains. Fee bodies daisy chained together using 3mm copper drain wire. Two chains brought 
together and connected to cryring ground. 

ADC data item rates   all <10k
ASIC settings         all LEC slow comparators 0x64
1.8.L spectra aida05-08 similar to initial grounding ~500 ch FWHM

Attachments 9-11
1.8.W spectra 20us FSR.
AIDA5-8 channel rates. Rates dominated by noisy strips. Missing strips come and go as ASIC settings checked. 
Adaptor card grounding

Some work still required to reduce noise in the system and reduce pulser widths. Pulser should also be checked on scope 

Gauge turned back ON. Bias 80V, 3.5 uA before. Bias 80V, 5.4 uA after. Bias turned off and on again. NO change to diode characteristic, current ramps high before coming 
down once bias on. Bias left ON 80V to see if stable when second gauge turned on and valves reopened. Pulser ON. Fee power OFF. Water cooling ON. 

No significant changes to waveforms, pulser width or rates with gauge ON. Other noise contributions probably outweigh effect of the gauge at this point. 

3mm copper grounding cable and attachments, crimper and Yorkshire tea left in control room. 

09:00

Leakage currents this morning again appear to have lowered during the night. Current plot shows a day/night variation but I think the minimum is decreasing ever so slightly.

This morning DSSD1=15.1uA and DSSD3=14.4uA, this is ~0.3-0.4 uA lower than yesterday. DSSD4 is still fluctuating in its leakage current and looks to have been all night. See attachment 1.

Will now proceed to systematically check all grounds before restarting AIDA.

Several cables have been re-routed this morning to clean up the area, allow us to see clear what cable goes where, and are avoid any unwanted interference. E.g. ion pump cables have been re-routed to not be near the bias cables, one ion pump cable was running between the power and hdmi on one of the FEEs, this was been sorted.

All grounds have been checked, some loose cables have been tightend up. Current ground configuration has all the adaptor cards (nn and pn) on a single detector connected together. One grounding cable then goes from each detector to the right coolant pipe which all the other detectors are also grounded to. This was the configuration for the previous beamtime. Will start pulser tests after lunch. Bias back on, leakage current slowly increasing again, DSSD3 current no longer fluctuating.

13:00

Ground config 1 - attachments 2-7 (described above)

Ground config 2 - attachments 8-10 (grounds now go to the CRYRING ground cable not the coolant pipe and jumper removed form bias module)

pulser widths 1.8L worse pretty much across the board. No sig impact on rates

Ground config 3 - attachments 11-14 (grounds returned to coolant pipe)

pulser widths 1.8L better than config 2 but still worse than config1

Ground configuration returned to first configuration. Pulser widths consistent with that seen before.

Additional bias tests.

Disconnected the pn bias cable (core) from DSSD1 but left nn bias cable connected. Biased. Observed bias ramp up but no leakage current was seen.

Reconnected pn bias cable but now disconnected nn bias cable. Biased. Observed bias ramp up, leakage current was seen to increase during ramp to ~8uA. Once fully ramped, leakage current started to slowly decrease and eventually went to zero.

Repeated same tests with DSSD2. Same behaviour observed as with DSSD1.

Have removed motors pins and tried some movement again. No effect on leakage current. Checked pulser widths before movement and after movement (with pnuematic IN). No change in pulser widths.

End of day status FEEs off, bias on and monitoring. Motor pins out to allow for last movement check, motors code running in screen session.

Edinburgh group ion gauge turned OFF -> decrease in leakage current from 10.05 uA to 6.5 uA

GSI vacuum group ion gauge turned OFF -> decrease in leakage current from 6.5 uA to 3.63 uA

GSI large ion pump turned ON -> no effect on leakage current

Motors moved IN the same as elog 201. No significant increase in leakage current as moved further into the chamber (attachment 1). Pulser widths follow similar path as previous tests. Aida1 and 2 statrt with larger pulser widths which reduce as are moved into the chamber. Aida3,4 start lower than aida1,2 and remain at same level throughout. (attachment2)

RF ring signal removed from aida5. Data rate dropped back to ~20 kb/sec, Merger rate ~80,000 items/sec

Noise peak on aida1 appears reduced after gauges switched OFF with detector fully out (attachment 3) coompare to elog 201

Noise peak is removed once detector is moved in. (attachment 4)

Ey histogram (attachment 5) shows multiple peaks for the same pulser amplitude. 

XY total pixels plot (attachment 6)

1.8 waveforms for aida 1,2,5 (attachment 7)

Mounted both gate valves and support. Gate 
valves could not be installed one bolt 
rotated from 90 degrees as per drawings 
due to mechanical conflict with back foot.

Using rotating flanges some small rotation 
from 90 was achieved. This may be 
sufficient given the reason for the 
rotation was to avoid mechanical conflicts 
with ring parts that are no longer there.

Valve struts had to be modified as gate 
valve is much lower and close to Al frame 
than in Daresbury.

ML suggested that the reason for the 
change in height mentioned in previous 
entry is that the ring gate valve to YR10 
is not centered. Solutions will be 
considered when the other two setups in 
YR09 are mounted and the alignment 
situation is more clear


Noted a few blind flanges are bluish, 
which indicates SS 304 instead of 316. Did 
we get the wrong ones from ITL? Did we 
swap them in Daresbury by mistake? Is it 
just an odd colour?


Installed turbo pump on top gate valve, 
Pirani gauge provided by ABD (where is 
ours?) and pumped down.

Reached 10-1 mbar in 10 minutes. Turbo on.
Reached 10-5 mbar on IE514 in 17 more 
minutes (where is its manual?)

Will leave pumping overnight.

Dry run complete.

YR09 section valves closed. Section isolated. Manual valve between electron target and CARME remain open.

GSI vacuum division IE514 gauge connected.

Gleb reports gate downstream opened. Vacuum goes up and recovers as expected.

The cave is closed and beam is back

The d intensity is about half of the intensity before the break

The gas target is prepared but the valvas cannot be opened. Anton is coming in at around 1 PM to open the gate valves.

Attachment 1 shows the pump-down of the CARME chamber at GSI before installation of kapton cabling inside the chamber and mounting of the chamber on the ring. Discontinuity in the plot is from switching between the pirani gauge and IE514 gauge. The pumping of the chamber in the turbo regime follows the power law where P ~ 1/(nt) due to outgassing from the chamber being dominated by water outgassing which has an n ~1 dependence (for first order desorption). This is seen in the pumping of the chamber in the turbo regime (Attachment 2) where pressure falls with an n of 1.03. In red the power law is extended to 10^5 minutes, demonstrating the limitations of pumping the chamber without baking. 

Attachment 3 shows the pumpdown pressure and baking temperature (chamber average) of the chamber once mounted on the ring, with all kapton installed in the chamber. Temperature data was lost in the middle of the plot due to the power cut to CRYRING causing temperature data to not be saved to the pi. Some of this data was plotted in elog 87 before beong lost from the pi. Points on the graph correspond to:

Point 1 - Baking started 2pm 23/9/21

Point 2 - Temperature limited to 40C due to high thermal gradients from uneven baking of the volume. Baking stopped 7am 24/9/21 so that heating elements could be repositioned. Baking restarted 10am.

Point 3 - Baking limited to 90C due to tent construction. Negative gradient detected due limitation causing baking to stop around 5pm 25/9/21. Manual mode to 60C. Full power mode restarted 10pm

Point 4 - Temperature of the volume reached 100C. 11am 27/9/21 baking stopped to install new turbo sections. Baking restarted at 3:25 pm Temperature peaks at 130C next morning.

Point 5 - 29/9/21 10am Unexpected power cut at CRYRING, temperature fell to 120C power restored shortly after.

Point 6 - 4/10/21 9am Temperature reduced to 100C and ion pumps outgassed. Activation of NEG pumps

Point 7/8 - Subsequent activation of other NEG pumps.

Point 9 - 7/10/21 CARME reached 9E-12 mbar. Gate valve closed, pressure rises with apparent floor at 1.5E-11 mbar

Some variations in the pressure are seen after reaching the floor, most likely due to sputtering of ion pumps. 

Teething problems with the bakeout such as heater placement and new jacket sections in addition to the power cut prevented continous bakeout for entire period, these issues should not affect the next baking run. As a result of the slightly fractured bakeout the pumpdown power laws have been plotted (logP) against log(T) in sections where temperature was roughly constant with the associated timestamps to gauge the decrease in pressure over time using this power law (Attachment 4). 

Room temp  n ~ 0.75

90C n ~ 9.15

130C (1) n ~ 8.05

130C (2) n ~ 7.37

130C (3) n ~ 4.89

130C (4) n ~ 2.65

Introduction of kapton cabling likely causes change to scaling as the diffusion of gas from kapton not the desorption from the surface dominates the outgassing rate. Diffusion has an n ~ 0.5, likely contributing to the change between the blank chamber and mounted chamber. As the temperature is increased the outgassing is significantly increased causing an initial increase in the temperature followed by a more rapid decrease in the gradient of the pressure decrease over time than at lower temperatures. As time increases, outgassing from water in the chamber dominates less and less with the kapton cabling representing a more significant portion of the total outgassing load likely causing the decrease in n as time increases despite baking at 130C. After ~12000 minutes n decreases significantly and more variations from the linear decrease in pressure are observed. After 17000 seconds the pressure begins to plateu and reached a floor of ~1E-7 mbar over the weekend at which it was assumed outgassing from water in the chamber was no longer significant. Variations could be due to temperature fluctuations which had a greater impact on the pressure as the pressure began to reach the floor.

Activation of NEG pumps was conducted after reaching the floor. The average chamber temperature and air temperature inside the tent was unaffected by the activation of NEG pumps, however the internal thermocouples significantly increased during activation (attachment 5). The internal thermocuples were largely unaffected by the NEG activation until the NEG temperatures were themselves above ~300C after the internal temperatures rose rapidly (more detail in elog 105,108,109,110) . Activating so many NEG elements at once is thus not feasible as temperatures would damage the detectors once mounted. Activating one group at a time (typically 2 NEGs) and D2000's which are away from the detectors to start with and then maintaining at low power before slowly activating NEG's close to the detectors perhaps at a lower temperature will be done to keep temperatures below 130C. A full activation procedure will be posted to the elog before activating again.  

Biased detectors this morning to 150V, the leakage current for all detectors appears stable and is ~3.5 uA. We have also replaced the CAEN bias module so that all channels are now functional. Detectors are connected as follows;

CHN0 -> Top left

CHN1 -> Top Right

CHN2 -> Bottom Left

CHN3 -> Bottom Right

Reconnected the motors and moved the detectors while biased. No observable change in the leakage current during movement at 150V. An increase in the pressure of the electron target was observed during movement (up to 1e-9 mbar). This increase was only momentarily, and the pressure recovered after a couple of seconds. We expect the pressure increase from movement to decrease over time.

10.43 Reboot carme-gsi - unable to open a working terminal

      Post boot found that it was necessary to manually enable WAN network interface (enp2s0) - perhaps it had been disabled previously?

      Started proxy port for remote access via AnyDesk per https://elog.ph.ed.ac.uk/CARME/489 using the command 

         ssh -L 8080:proxy.gsi.de:8080 carme@atppc02

      AnyDesk 978 207 281 now online/available

13.43 carme-gsi moved from 19" rack room to control/counting room (next door)

      19" rack room
       *yellow* network cable disconnected from YR10/2 patch panel, *green* network cable connected to YR10/2 patch panel
      control counting room
       *green* network cable connected to carme-gsi p4p1 network interface (connection to CARME network switch at YR09)
       network cable connected from carme-gsi enp2s0 network interface (port #2) to Netgear MS510TXM EXPSW3 port (connection to WAN)