Found CARME at 50C, a bit cooler than expected, probably due to invasive tent changes yesterday to mount missing wires. Increased heaters temperature to 300C

9:35 Started D-2000 #1,3 (behind the DSSD). T=400, tramp 30, thold 90

11:20 Set D-2000 1,3 to 200C. Turned on D-2000 2,4 as above 

13:15 Set D-2000 2,4 to 200C. Turned on UHV1400 set 1 at 250W, tramp 30, thold infinity (will ramp down manually as required)

13:20 No appreciable effects on DSSD temperature. Turned on set 2 as well, which should be mounted close nearby.

13:45 Both sets reached 250W. No effects on detector temperature. Increased to 300W. Vacuum gets slightly worse, but recovers quickly. Not worth further increasing power. Leaving for 90 minutes.

15:20 Strain relief TC at 75C quite hotter than detector TC at 35C. Might be normal. Vacuum division IE514 must have been degassed remotely? Decreased set 1,2 to 75W. Turned on sets 3,4 to 250W as above. 

15:45 Sets 3, 4 now at 250W. Detector TC heating quickly at 13C/h but nowhere near dangerous temperatures yet. Decreased sets 1,2 to 50W. Leaving for 90 minutes.

17:15 Sets 3, 4 to 50W. Slightly worrying temperature difference between detectors and strain relief around 65C. Different materials and different positions could explain this.

17:40 Oddly, detector temperature now going up sharply (temperature diff. down to 40C) since the uhv1400 power was decreased. Cannot think of s logical reason,  it may just be a coincidence and heat transfer is catching up.

Leaving overnight. Will turn on the last two groups of UHV1400 tomorrow morning.

We need to connect the non-bakeable (max 80C) wire of the ion pump belonging to the vacuum division, and we also need to connect the two newly-repaired NEG cables to the D2000 modules. That requires partly opening the tent to gain access.

Set heaters temperature from 450 C -> 400 C - CARME floating TC temperature goes from ~98 C to ~88C.

Set heaters temperature to 350 C. Left to cool over lunch.

 Connected GSI vacuum division ion pump and IE514

Connected D-2000 upstream left/right with repaired cables

Flashed all ion pumps. Ion pump #4 was flashed a couple of times, but has now started arcing. The issue is not with the module - either the wire or the ion element are damaged.

Set heaters to 300 C. Left to cool a bit more overnight. Planning on NEG on tomorrow.

Repaired Neg wires for D2000 with Oemer's help.

Cable #2 is fine. 

In Cable #4, wires 1 and 2 are swapped but it's impossible this happened when the connector came apart. The pins are crimped and secured with a screw. Most likely it was sold like this and there's no difference between wire 1 and 2 (same voltage).

Edit: SAES confirmed 1&2 are interchangeable.

14/08/23

Temperature in the baking volume is ~ 42 C and the pressure is 4.6e-8 mbar. Current heaters set point is 135 C. This will now be increased to 200 C to start increasing the baking temperature more significantly.

Some issues with the interaction chamber heaters were noticed this morning. This included very large increases in temperature despite only small increases in the setpoint. When this occurred the heater channels were simply turned off. Channel 3 on the interaction chamber (main jacket on the centre of the chamber was also observed to smoke slightly). This smoking was seen when the temperature for channel 3 was ~ 70 C. 

Changing the settings of these heaters appears to have resolved most of the issues. The heaters now ramp in a much more controlled manner and remain roughly constant at the setpoint value.  The interaction chamber heaters were all increased to 70 C with no issues observed. Channel 3 also appears OK.

The CARME heaters settings have also been changed and now the heaters temperature and power output do not oscillate as observed previously. Power output per heater is approximately 15 % of total at heater temperatures of 220 C, and this value remains consistent over time. The temperatures inside the baking volume are now ~ 50 C with the pressure equal to 7e-8 mbar. Heating has not been the priority while investigating the issues observed with the heaters.

As a precaution CARME will now be maintained at the 50C level. The interaction chamber and e-target temperatures have been set to 50 C also. If the system remains stable overnight we will again begin increasing the temperature in the morning.

15/08/23

Temperatures across the whole of YR09 have remained very stable overnight. Temperature of CARME has increased slightly and is now ~ 55 C. Pressure is ~1e-7 mbar. A brief lincrease in pressure was observed yesterday due to the de-gassing of the GSI ion pump on CARME. Plan is to now increase the temperatures of YR09 to the 100 C level 

Temperature is ~ 70 C. Heating is slower than expected as I am aiming to minimise thermal gradients across the chamber, which was previously done automatically by our baking code. Current heater temperature values are ~ 320 C. I have increased the temperatures across YR09 to 70 C to match carme . Pressure in CARME is 2.1e-7 mbar.

16/08/23 

The temperatures of the CARME heaters were further increased to ~ 420 C to further increase the chamber temperature. Average chamber temperatures of ~93C achieved. Heater temperatures can be pushed to 450C which should allow us to achieve 100 C. Will allow chamber to equillibriate as temperature gradient between top and bottom is quite large.

17/08/23  

Baking will be unsupervised over the weekend. To remain safe, the berylium window and the interaction chamber will be kept at a temperature of 50C. Carme will be reduced to ~60C during this period.

Attached are plots for the bakeout period concerning this week.  

12/08/23

Baking testing. Aim to find relationship between heater temperature setpoint and the temperature of the chamber and floating temperature in the baking volume. Behaviour of the heater tempertures and ability to control the heating is also required. Ambient chamber temperature ~22 C. Interaction chamber temperatures will be raised to the level of the CARME chamber throughout, starting temperature of the interaction chamber with heating off is 28 C. P = 7.5e-9 mbar

CARME thermocouple temperatures can be read from the monitoring code which is currently running in a screen session on the baking pi. Heater temperatures and pressure in YR09 can be read from Grafana.

15:40 CARME heaters turned on, set point 50 C. Ramping is ~ 1C per minute for the heaters. 

16:00 Thermocouples inside baking volume show small increases. Setpoint increased to 60 C.

16:10 Setpoint increased to 80 C.

16:30 Setpoint increased to 100 C.

16:45 Setpoint increased to 120 C

17:00 Setpoint raised to 150 C. Average chamber temp ~ 25.2 C, bottom of the chamber is ~28C. 

17:45 Setpoint was maintained at 150 C, temperatures of the heaters continued to rise past this setpoint and were floating about 200-230 C with large variations in the temperature. At lower setpoints the heater temperature appeared to rise linearly with the 1 C per minute. I do not believe I currently have control of the heating. Average chamber temp 28.8 C with bottom of the chamber ~35-37 C. Heating has been turned off while I grab some food. P = 1e-8 mbar

Next test will be to try and maintain a consistent heater tempreture for a longer period of time, probabably at a lower set temperture. 

13/08/23

10:00 Heater setpoint to 50 C. Will maintain at this level for a while to look at increase in chamber temps and see if heater temp remains stable. P = 8.52e-9 mbar

11:47 Heater temperatures have been constant at ~ 55 C for the last hour. Little heating observed in the chamber, requires much larger temperature to heat the baking volume. Setpoint increased to 100 C.

13:10 Heater temperatures have been constant at ~103 C for the last  40 mins or so and appears stable. Temperatures are rising at a steady rate but temps are still low. Interaction chamber heating jackets activated to be a same temperature as CARME. Will require higher temperatures to heat the baking volume.

16:15 Heater temperaturtes have remained constant at ~103 C. Temps inside the baking volume remain low, Increasing heater temperature to 135 C.

18:30 Heaters temperatures remain stable. Larger fluctations are apparent than at a setpoint of 100C with temps between 140 and 150 C. Chamber temperatures increasing avg is ~ 35C, was aiming for 50C over the weekend. Increasing setpoints to 170 C.

20:50 Heater tempretures are stable. Average chamber temp is ~ 40 C. Interaction chamber heaters set to 40 C to keep constant with CARME. Pressure has increased to 2.9e-8 mbar. Heater power outputs are visible in last image, heaters currently turn on at ~80% max power about twice every 5 mins. I expect temps of 250-300 C are required to obtain significant temp increase up to 100 C on the chamber. Chamber temps appear to be fairly stable at this point, will leave for now and check back in a few hours.

23:00 Heater setpoint temp reduced to 135 C to maintain temperatures inside the baking volume overnight with no increase. Heater power has now reduced to < 50 % full power when the eaters are on. Pressure in the chamber is 4.4e-8 mbar.

I believe I now understand the heating system and plan to increase the chamber temperature throughout the day tomorrow. Aim is to get close to full bakeout temperatures.

Heater channel layouts for the interaction chamber and carme in attached documents. 

Updated thermocouple positions; (front section is closest to gas-target, rear section is moving detector end)

• 1-1 Internal TC 1
• 1-2 Internal TC 2
• 1-3 Internal TC 3
• 1-4 CARME - top of the front section
• 1-5 CARME - bottom of the front section
• 1-6 CARME - top of the mid section
• 1-7 CARME - middle of the mid section
• 1-8 CARME - bottom of the mid section
• 2-1 Float - bottom of mid section 
• 2-2 CARME - middle of rear section 
• 2-3 Float - middle of the front section
• 2-4 CARME - bottom VAT valve
• 2-5 Float - middle of the rear section 
• 2-6 CARME - Top elbow
• 2-7 Float - top of the mid section
• 2-8 CARME - rear beam pipe

Thermocouple posiitons have been updated in the monitoring code, including the averages for the CARME chamber. 

09/08/23

Attached is the current pumpdown plot taken from grafana. 

I have started to re-work the baking tent in order to close gaps from its initial installation. The heaters work and were tested to a temperature of 50C. Testing is required to gauge the heater temperature required to heat the baking volume, we suspect the heaters will be several hundred degrees. 

11/08/23

Baking tent has been fully constructed again, see photos attached.  Some thermocouples have been re-positioned, new thermocouples positions are given in elog 433. GSI ion pump cable has been connected. Roughing pump on the outside of the berylium window has been connected and berylium window is at < 10 mbar. 

10:30  IE514 1.80e-08mbar

Most of the CARME is covered by the baking tent. The tent needs to be closed and a bit covered from the front (upstream), end (downstream), and bottom side. TCs are labeled with a marker in the sequence 1-1 to 2-8.

1-1 Internal TC 1
1-2 Internal TC 2
1-3 Internal TC 3
1-4 CARME front top
1-5 CARME front bottom
1-6 Top hanging middle
1-7 CARME middle middle
1-8 CARME middle bottom

2-1 CARME middle top
2-2 CARME back middle
2-3 Elbow
2-4 Valve
2-5 Magnet pipe
2-6 CARME back top
2-7 CARME back float
2-8 CARME back bottom

Also, TC extensions would be needed to connect to TC-8 modules.

09:26  IE514 1.96e-08mbar

10:10  IE514 2.0e-08mbar

Upstream part of the CARME is covered.

11.40 Pirani 1.00e-3 mbar IE514 2.23e-08mbar

15.15 7d/24h vacuum plots

11.15 Pirani 1.0e-03mbar IE514 2.72e-08mbar

16.35 Pirani 1.0e-03mbar IE514 2.61e-08mbar

10.20 Pirani 1e-03mbar IE514 3.26e-08mbar


12.53 CARME Leybold IE514 gauge OFF


Top right DSSSD bias -20V leakage current -10.51uA


Three internal ThermoCouples (TCs) -labeled 1, 2, and 3 in attachment 1 - were attached and tested on the TC-08 boards. Only 2 showed the ambient temperature, see attachment 3. 1 and 3 showed some strange numbers, see attachments 2 and 4 respectively.


15.28 CARME Leybold IE514 gauge OFF. Top right DSSSD bias -20V leakage current -10.44A


All the TCs are attached to TC-08 modules. See attachments 5-7. Once all TCs were plugged into TC-08 modules, they were fine.


TC-08 A0067

1. Internal TC 1
2. Internal TC 2
3. Internal TC 3
4. CARME front top
5. CARME front bottom
6. Top hanging middle
7. CARME middle middle
8. CARME middle bottom



TC-08 A0066

1. CARME middle top
2. CARME back middle
3. Elbow
4. Valve
5. Magnet pipe
6. CARME back top
7. CARME back float
8. CARME back bottom

Heating Jackets placed beneath the heaters. See attachments 8-10.

09.20 Pirani 1.00e-3 mbar IE514 3.90e-8 mbar

      Today - install TCs, connect to Pico controller and test

10.00 Bakeout tent setup guide (draft) - attachment 1

14:30 Tested all TCs and are fine

05.25 Pirani 1.00e-3 mbar IE514 3.71e-8 mbar

09.28 Pirani 1.00e-3 mbar IE514 5.45e-8 mbar

12.10 Bakeout tent frame installed
      SAES UHV1400 NEG pump power cables #1-6 installed
      See attachments 1 & 2

16.35 Attachments 3-6
      upstream baseplate
      3x sets of heaters beneath CARME chamber connected to CRYRING heater/TC manifold - attachments 7-8

      
16.45 Pirani 1.00e-3 mbar IE514 4.80e-8 mbar


17.10 SAES D-2000 #1 downstream, outside   NEG power + HV ( attachment 9 - OK cable connector )
      SAES D-2000 #3 downstream, inside   NEG power + HV
      SAES D-2000 #4 upstream, inside   HV
      SAES D-2000 #2 upstream, outside   HV

2x ( of 4x ) SAES D-2000 neg power cables were broken - forward-shell detached (unscrewed ) from back-shell remaining attached to D-2000 neg power connector. 
Re-assembly is straightforward but it is unclear how the 6x female sockets are aligned with the connector keys of the forward-shell (attachments 10-11 cf. attachment 9).
Until this is understood these 2x SAES D-2000 neg power cables should not be used.

09.16 Pirani 1.00e-3 mbar IE514 8.77e-8 mbar
      Leak tester c. 5e-12 mbar.l/s (leak tester valved off)

09.48 Pirani 1.00e-3 mbar IE514 8.67e-8 mbar
      Leak tester c. 3.5e-12 mbar.l/s (leak tester valved off)

10.33 Pirani 1.00e-3 mbar IE514 8.55e-8 mbar
      Leak tester c. 4e-10 mbar.l/s (leak tester connected to CARME roughing line)

10.55 Leak tester remains at few e-10 mbar.l/s - we do not understand why
      With leak tester in differential mode the highest reading around the COF600 seal is c. 3.5e-12 mbar.l/s
      Following checks elsewhere on the chamber we observe c. 6e-12 mbar.l/s which presumably reflects accumulation of He around the chamber 

      Dispersing He with compressed air reduces diff mode reading to c. 2e-12 mbar.l/s

11.00 Pirani 9.80e-4 mbar IE514 8.47e-8 mbar

14.49 Pirani 1.00e-3 mbar IE514 7.71e-8 mbar
      Leak tester isolated from CARME backing line - CARME Edwards scroll pump now backing turbo.

09.15 Following overnight pumping of CARME chamber by Leybold turbo
      Pirani 1.00e-3 mbar IE514 2.83e-7 mbar

     The Leybold IE514 gauge currently in use is the CARME unit ( serial # 1054 ). The calibration data can be found at https://elog.ph.ed.ac.uk/CARME/56


09.44 Pirani 1.00e-3 mbar IE514 2.71e-7 mbar

10.10 Leak tester attached to backing line

      Valves to leak tester and turbo (connection to Edwards scroll pump in between) closed.

      Stop Edwards scroll pump with magnetic valve 'red' button - magnetic valves closes and Edwards scroll pump is switched off.

      Open valves to leak tester and then (slowly) turbo.

      Wait for leak tester to settle in e-12 mbar.l/s range. Leak tester pressure 3.6e-3 mbar. IE514 2.69e-7 mbar.

11.00 Pirani 1.00e-3 mbar IE514 2.42e-7 mbar
      Leak tester 1.8e-10 mbar.l/s

13.04 Pirani 1.00e-3 mbar IE514 2.06e-7 mbar

      Leak testing.
      Downstream DN100 flanges from CARME to dipole OK
      Lower NEG power connector feedhtrough (downstream chamber) OK
      Left hand side (looking downstream) bellows OK

      c. few e-9 mbar.l/s leak bottom left (lookimg downstream) of COF600 wire seal

      Additional torque to M16 bolts bottom left - c. 10 bolts -> 220Nm+

      c. 0.1-0.3e-11 mbar.l/s

      Further checks around circumference of COF600 seal reveal some small leaks c. e-11-12 mbar.l/s.? However background rate
      in leak tester too high for sensitive test - will await recovery to resume checking.

14.00 Pirani 9.8e-4 mbar IE514 1.96e-7 mbar
      Leak tester 4.9e-10 mbar.l/s


14.35 DSSSD test
      IE514 OFF

Ambient temperature 21.5 deg C

Top right
-10V -4uA
-20V 10uA
c. -30V > -20uA trip

Top left
-150V -5.15uA

Bottom left
-150V -4.97uA

Bottom right
-150V -6.33uA

Top right (repeat)
-20V -10uA
-25V -14uA

15.28 IE514 ON
      Pirani 9.8e-4 mbar IE514 1.87e-7 mbar

15.40 c. few e-9 mbar.l/s leak around magnetic valve - probably not an issue in terms of backing performance but may interfere with/confuse leak testing

16.05 Pirani 9.8e-4 mbar IE514 1.74e-7 mbar
      Leak tester 1.2e-9 mbar.l/s

Will leave CARME pumping with leak tester on roughing line and see whether leak rate recovers. If so, will continue leak tests tomorrow. 

18.50 Pirani 1.00e-3 mbar IE514 1.48e-7 mbar
      Leak tester < 1e-12 mbar.l/s (leak tester valved off)

Grosmantage are scheduled to arrive 10.00 to inspect COF600 seals and their fit to the COF600 flange. They may decide to adjust the seals so that they fit the flange 
under 
tension.

Once Grosmontage are satisfied with the seal and its fit we can close the CARME chamber and start pumping down. 

Attachment 1 - photo of bakeout tent parts including those adapted/modified by Davide.


18.40 Grosmontage were re-tasked this morning.

      Davide, Omer, Cosmas checked alignment of COF600 male and female flanges.

      4x COF600 seals were checked - for some seals we observed wave over a ~ 10cm section of the circumference and/or surface scratches/roughness.
      Also observed that COF600 seals appeared to be too large - consequently the wire seal would be seated well ( i.e. in contact with the flange surface )
      for the majority of the circumference but would lift from flange surface for part(s) of the circumference.
      The fourth COF600 seal seated well over all of the circumference and we decided to proceed with closing the chamber.

      COF600 flange bolts tightened sequentially 5, 10, 20, 30, 40, 60, 80, 100, 120, 140, 160Nm.

      ML roughed CARME chamber slowly ( < 1mbar/s ) to protect Be window. Turbo started at 1.3mbar. Pirani gauge indicated e-3 mbar range when turbo at speed.


18.53 IE514 gauge ON pressure 4.05e-5mbar.

19.10 Pirani 1.42e-3 mbar IE514 3.02e-5 mbar

09.20 CARME chamber at atmospheric pressure as expected.

      Will open CARME chamber and inspect COF600 wire seal for similar (or other) issues to the seal removed yesterday. One possibility is that the COF600 flange 
surfaces/structures are misaligned and part of seal is 'escaping' the seal zone. Given the flanges close easily the misalignment would have to be small.

13.30 Observations

      The bolts on the upstream feet of the far detector chamber are all loose and have probably been so for some time

      Based on measurements with calipers and the insertion of shims into the *radial* gap between COF600 flange surfaces the COF600 flanges may be misaligned and rotated 
transversely by a fraction of a millimeter. The challenge is that the chamber is not designed for fine transverse adjustments and it is not easy to measure the changes.

      The Pfeiffer recommended procedure for closing COF seals is to start tightening bolts *diametrically* and then to tighten sequentially

https://www.pfeiffer-vacuum.com/en/know-how/mechanical-components-in-vacuum/connections/detachable-flange-connections/cof-flanges/

     The plan

     Adjust position of far detector chamber and secure upstream feet bolts

     Close COF600 flange using Pfeiffer recommended procedure

     Today is Friday so it will probably be Monday before we can pump the CARME chamber

13.50 Another view

     Grosmontage also inspected the COF600 seal this afternoon. They said that it was important that the seal should be installed under tension to ensure that it remained 
in position as the flanges were closed and the bolts begin to be tightened. They think the seals are a little too large in circumference but that they can shorten one on 
Monday morning. 

09.30 CARME chamber status

      Edwards scroll pump ON
      Edwards Speedivac valves #1 & #2 OPEN
      VAT gate valves OPEN

      Re-connect Leybold Pirani gauge to local display/control
      CARME chamber pressure 2.4mbar

      Scroll pump should be able to achieve < 0.1mbar - indicates significant leak

      Davide will first check bolt torques for all flanges opened during most recent vacuum cycle

12.05 Additional torque did not resolve problems
      ML test COF600 seal with isopropanol and observed increased chamber pressure

      Will open COF600 seal after lunch

13.55 COF600 seal removed - one section marked by Kapton tape shows some distortion and a burr on the inner diameter - see attachments 1 & 2. 


      ML speculated that perhaps the seal was lightly too long and a section of circumference was forced outwards?
      Vacuum surfaces cleaned with isopropanol and tissues - red/brown discoloration probably indicating copper observed

14.30 New COF600 seal installed, flanges closed, bolts finger tight, bolts then systematically torqued in small increments 

      COF600 torques finger tight, 10, 20, 30, 40, 60, 80, 100, 120, 140, 160Nm x2
      DN100 flanges doenstream finger tight, 8, 16, 24Nm

16.40 Start scroll pump and slowly rough chamber

18.00 Roughing CARME chamber pressure flatlines c. 30mbar indicating a large leak.
      ML ran some isopropanol around COF600 flange and observed response in roughing pressure of c. 0.2-0.3mbar at c. 30mbar which appears to confirm COF600 is leaking.