On 15/2/22 it was noted that the leakage current increases when the motors are moved IN. This is contary to tests conducted on 12/2/22 (elog 192) where no increase was observed. 

16/2/22: Servo motor and pneumatic used to move in by different differences with leakage current observed to increase linearly with distance moved in, measured using the potentiometer. (attachment 10)

After each movement the pulser width was measured for each of aida1,2,3,4. No change observed for aida3,4 with pulser width ~220 channels. Pulser width decreased for aida1,2 when moved IN. (attachment 11)

Noise on aida 2,3,4 appeared low throughout tests. Aida1 has a large noise peak when in fully out position. When moved fully in, the noise peak disappears (attachments 1,2)

Cryring signals for ring RF, electron cooler voltage and schottky added to the motors signal and fed into mac b to be read in daq as aida correlation info #8 signal. 

cryring bnc 13  - ring rf - aida5 ~ 500,000 rate

cryring bnc 14 - e cooler voltage - aida3 ~ 80 rate

cryring bnc 15 - shottky -aida2 ~2000 rate

cryring bnc 16 motors - aida1 ~3 rate   

correlation info #8 appears to be added to the good events rate (attachments 4,5) ADC items (attachment 3)

Waveforms 1.8*W attached for 20us, 200us, 2ms, 20ms

16:30

NOTE: Pulser is ON!

Check Leakage current: (see attachment 1)

Check NewMerger tab: (see attachment 2)

Check Fee temperatures: all green (see attachment 3)

Check statistics tab: ADC data items (see attachment 4)

                                 ADC coincidence-info #8 (see attachment 5)

                                 Good events (see attachment 6)

 Check Spectrum Browser tab: (see attachment 7)                 

Check the following plots: LowEnergyXYRateDSSD0 (see attachment 8)

                                         LowEnergyXYTotalDSSD0 (see attachment 9)

                                         LowEnergyExTotal (see attachment 10)

                                         LowEnergyEyTotal (see attachment 11)

Check system wide checks tab: all okay.

All is good.

13:30

Check Leakage current: (see attachment 1)

Check NewMerger tab: (see attachment 2)

Check Fee temperatures: all green (see attachment 3)

Check statistics tab: ADC data items (see attachment 4)

                                 ADC coincidence-info #8 (see attachment 5)

                                 Good events (see attachment 6)

 Check Spectrum Browser tab: (see attachment 7)                 

Check the following plots: LowEnergyXYRateDSSD0 (see attachment 8)

                                         LowEnergyXYTotalDSSD0 (see attachment 9)

                                         LowEnergyExTotal (see attachment 10)

                                         LowEnergyEyTotal (see attachment 11)

Check system wide checks tab: all okay.

All is good.

09:15

Check Leakage current: (see attachment 1)

Check NewMerger tab: (see attachment 2)

Check Fee temperatures: all green (see attachment 3)

Check statistics tab: ADC data items (see attachment 4)

                                 ADC coincidence-info #8 (see attachment 5)

                                 Good events (see attachment 7)

 Check Spectrum Browser tab: (see attachment 6)                 

Check the following plots: LowEnergyExTotal (see attachment 8)

                                         LowEnergyEyTotal (see attachment 9)

                                         LowEnergyXYRateDSSD0 (see attachment 10)

                                         LowEnergyXYTotalDSSD0 (see attachment 11)

Check system wide checks tab: all okay.

All is good.

All fees and asics checked to find noisy channels. Channels are listed along with the noise peak height for the low energy plot at 22:00. Run started ~17:45. 

Attachment 1 shows 3*L for aida01 with x axis zoomed in on noise peak. Attachment 2 shows stat for aida01-04 in log scale. 

Aida01

• 1.0L ~3000
• 1.2L ~3500
• 1.7L ~3000
• 1.8L ~2000
• 2.11L ~2000
• 2.12L ~400
• 3.7L ~1000
• 3.10 ~60000
• 3.11L ~2000
• 3.12L ~15000
• 4.12L ~400
• 4.14L ~750 

Aida02

• 1.0L ~1000
• 1.3L ~750
• 1.7L ~600
• 1.15L ~600
• 4.15 ~125

Aida03

• 1.11L ~3000

nothing noisy of note on aida04

16:00

Check Fee temperatures: all green (see attachment 1)

Check statistics tab: ADC data items (see attachment 2)

                                 Good events (see attachment 3)

                                  ADC coincidence-info #8 (see attachment 4)

Check NewMerger tab: (see attachment 5)

Check Leakage current: (see attachment 6)

Check the following plots: LowEnergyExTotal (see attachment 7)

                                         LowEnergyEyTotal (see attachment 8)

                                         LowEnergyXYTotalDSSD0 (empty.)

Check Spectrum Browser tab: (see attachment 9)

Check system wide checks tab: all okay.

All is good.

17:00

Pulser walkthrough completed on run R38

290

280

270

250

240

230

220

210

210 / 10

230 / 10 (can't see)

210 / 5 (can't see)

190

90

150

130

70

50

10 (can't see)

30

15 (can't see)

Pulser to 70

11:30

Check Fee temperatures: all green (see attachment 1)

Check statistics tab: ADC data items (see attachment 2)

                                 Good events (see attachment 3)

                                  ADC coincidence-info #8 (see attachment 4)

Check NewMerger tab: (see attachment 5)

Check Leakage current: (see attachment 6)

Check the following plots: LowEnergyExTotal (see attachment 7)

                                         LowEnergyEyTotal (see attachment 8)

                                         LowEnergyXYTotalDSSD0 (see attachment 9)

Check Spectrum Browser tab: (see attachment 10)

Check system wide checks tab: all okay.

All is good.

EMERGENCY SHUTDOWNS

1. CARME motors: Q key on keyboard while on workspace 1 of appc218 (right of this double-monitor setup)

2. Internal target: Q on keyboard while on workspace 3 of appc218, go to server room, press "CLOSE" button on DUMP.

CARME DAQ

CARME computer has 6 workspaces (WS), two monitors for each WS.

• WS1 : left monitor - fee power

          right monitor - fee power server

• WS2 : left monitor - DAQ controls

          right monitor - CAEN bias control

Most important tabs (right hand monitor)

• Run control : shows daq status, stopped / going
• Fee temperatures : temperatures of fees labelled aida01, aida02, aida03, aida04, aida05
• Statistics : shows the statistics for each fee
• System wide checks : checks for the setup
• Spectrum browser : loads histograms for fees and fee channels from preset layouts
• Asic control : sets asic parameters

• WS3 : left monitor - Merge control, tape server

          right monitor - servers, data being written

• WS4 : leakage current plot
• WS5 : online data monitoring
• WS6 : browser, elog
• Motors are running on second computer

TO DO FOR THE SHIFTERS - CHECK EVERY HOUR AND POST TO ELOG WITH SCREENSHOTS

-->IMPORTANT NOTE: MIDAS DAQ can respond slowly. DO NOT try to change tab or give new command while the previous one is being executed. Check activity in right-hand part of tab currently in use. Do not double click on buttons.

Motors computer (Computer to the right)

• Check motors cycle is running. The code will cycle through the loaded motors instructions. When an instruction is being done it is highlighted in bold. Check the bold highlighting cycles through entire set of instructions.
• Check potentiometer is reproducing same max/min values for each cycle of instructions. 74.02 fully in, 29.5 fully out.
• Detectors are on LEFT arm. Nothing on RIGHT arm which should not move.
• No need for screenshots, but record in the ELOG you checked this.

Merger, Tape server (WS3)

• Check NewMerger tab. Hit reload button and see if 'current links with data' and 'current merged data rate' are updating. All links should go green, but not necessarily at the same time. Keep hitting refresh until you see all of them going green at least once.
• Data rate should be around 100,000. Take screenshot.
• Check tapeService tab. Hit reload button. Check data rate in Kbytes/sec box. Record data rate to elog. This is where the current run number is displayed. Take screenshot.
• On the right-hand monitor, bottom right terminal, check the current run fragment (sub-run) is being written and is slowly increasing in size. It refreshed automatically every 5 seconds.

Leakage current (WS4)

• Check leakage current plot is stable. Make sure it is updating and time is correct. Take screenshot. Check present values on WS2, CAEN window, channel 1.

DAQ (WS2)

• Check Fee temperatures tab. Hit reload, temperatures will update and should all be green. 1-2 C above safe limit (top) is OK. More than that - warn expert shifter immediately. Take screenshot.
• Check system wide checks tab. Click on 'Check Clock Status', 'Check ADC Calibration' and 'Check the White rabbit decoder status' to run checks. Copy-paste test in the ELOG the first time you do it. If there are no changes throughout the shift, just record no changes.
• Check statistics tab. Hit Update All to see if the counter and rate are updating. Change counter being displayed using drop-down menu on the right hand side of the left monitor.
  • Good events - aida 1-4 should have similar counting rates. If any rate is a lot higher than the others, warn expert shifter.
  • ADC coincidence-info #8
  • Take screenshot of each for the ELOG
• Spectrum browser tab. Check it is displaying aida 1-4 rate (top right of each histogram). IF NOT: select layout 1 from 'layout ID' then restore layout from 'Arrange functions' drop down menu. Select log scale and change xmax as 128. This displays the data rate per strip. If any strips are a lot higher than the others, or very different from previous entry, make a note in the ELOG. Take screenshot.

Online monitoring (WS 5)

• Check the following plots are updating
  • LowEnergyXYTotal (right click - col - colz). This shows the total number of events, per pixel, since the code was started / re-started.
  • LowEnergyXYRate (right click - col - colz). This shows the rate over the last minute.
  • LowEnergyExTotal (double left click. Right click on y axis - Log scale on y). This shows the total energy deposited on all X strips since code restart.
  • LowEnergyEyTotal (as above). as above, Y strips.
• Please take screenshot of each

Alpha background gathered from 17:35 12/2/22 to 13:44 13/2/22 is run32. Run time ~20 hours. Some alphas visible in aida1 1*L histograms attached.

Noticed grounding of crimps that attched to cryring ground has come loose. Those crimps not ideal for our grounding cable so were loose. Grounding cable from each fee now meet at a single grounding cable joined by an m3 nut and bolt. This grounding cable then goes to the cryring ground and is properly secured. See attachment 1. No significant change to pulser widths with new grounding method from previous elog. 

Aida5 appeared non-responsive, no good events / data items etc and would not stop when daq was stopped. Fee power cycled. Aida 3 and 5 then showed no data items / good events or any stats. When stopping daq all fees then became non responsive. Full restart initiated, fee power off, all servers incl merger and tape server closed. Restarted system, all fees now responsive, all show full stats and are writing data. 

Alpha background started 17:06 13/2/22 to 09:41 14/2/22 is run35. Run time ~16 hours 40 mins.

Ex, Ey 1D histograms for run32 and run35 attached. Bin size 80 keV.

> Initial grounding see final grounding from elog 187. ASIC settings for all - LEC slow comparator 0x64. Note on 11/02/22 1.8*L widths with same settings were aida1-4: 300,240,250,260

ADC data items <10k

Good events fluctuate 10k-20k

1.8*L pulser widths aida1-4: 433,400,375,375(double peak)

Attachments 1-4

> New grounding implemented. 3mm grounding cable connect fee body to adaptor board. Each adaptor board is then connected directly to the cryring ground. Attachements 10-11

ADC data items <10k

Good events fluctuate 10k-20k

1.8*L pulser widths aida1-4: 340,288,288,295 (reduced double peak)

Noise in the system also seems to have decreaed in zoomed out 1.8*L plots (not attached). Aida1 still shows large noise peak channel 32000 on 1.8*L. 1.8*W no significant change, perhaps slightly reduced.

Attachements 5-9

Screencaps for following movement tests not taken as 1.8*L and 1.8*W spectra appeared the very similar as 5-9 except for slightly different widths.  

> Detectors moved in and out (pneumatic). Leakage current dropped by ~0.2 uA, torque now reads +2.8 previously was ~0. 

no change to ADC data items or good events. During movement aida3 saw momentary increase in good events to ~40k before returning to normal range. Rest of fees good events stats unaffected by movement.

1.8*L pulser widths aida1-4: 413,313,295,306

> servo motor (only) moved, torque reduced back to ~0

no change in good events stats for any fees during servo movement. 

1.8*L pulser widths aida1-4: 425,335,316,288

> Detector moved to IN position (pnuematic and servo). 19.3mm to 81.8mm on potentiometer. 

No change to good events. During movements aida3 saw momentary increase in good events to ~41k, all other fees unaffected. 

1.8*L pulser widths aida1-4: 302,221,305,293 

Noise peak at channel ~32000 on aida 1 (1.8*L) significantly reduced. little noise on other fees 

>Detector moved to fully OUT position (pneumatic and servo). 

no change in good events, during movement aida3 saw momentary increase to ~35k

1.8*L pulser widths aida1-4: 450,350,316,290

Noise peak at chn 32000 on aida 1 has increased to previous size. little noise on other fees. 

> Bias increased to 120V. Leakage current stable however slightly higher ~0.1 uA

no change in good events. 

1.8*L pulser widths aida1-4: 447,345,189,301

> Bias increased to 140V, increase in leakage current ~0.2 uA, appears stable. Increase to 150V, fluctuate in leakage current of +/-0.5uA then huge increase to >20uA trip. 

> Fee power cycle. Bias 100V, leakage current 9.9 uA ~0.1-0.2 uA higher than previous but stable. 

no change to good events

1.8*L pulser widths aida1-4: 468, 350,311, 296

> Some decrease seen with new grounding, however main source of noise still apparent and any decrease seems insignificant compared to changes due to position movement. 

> Water pipes have been zipped tied so they no loger touch motors, no change in pulser widths or noise levels however.  

To test effect of the motors on the leakage current, they were cycled once per hour overnight at approximately 50 minutes past the hour. - attachment 1
Leakgage current was seen to decrease slowly overnight before flattening out.

At 11:25 This morning the FEEs were powered on. From this point on the leakage current was observed to start to increase. - Attachment 2
Note the time recorded by the pi logging the leakage current is approximately 2 minutes ahead of the FEE.

15:40 I notice again that powering on the FEEs causes a momentary increase in the leakage current. - Attachment 3

08:52 Overnight the DAQ was left running to check if there was any individual radioactivity in the Si.
      This is stored in R7_75

      It was observed that even with no beam the detectors continued to increase in leakage current overnight but slowly - Attachment 1
      We then stopped the DAQ and gave the detector a single shake.
      This caused the leakage currents to decrease but not to the same level that they did last night - Attachment 2

13:45 Bias tripped as people removed the cover on the windows on the interaction chamber
      Iset was still at 11uA for safety
      Will leave off until they are finished

      

09:00 Detector still biased to 60V and leakage current has been mostly stable overnight.
      Only minor fluctuations observed - attachment 1

      FEE temps all ok - attachment 2
      Stats at 60V - attachment 3
      System wide checks all ok

      Increasing bias to 80V
      60V 9.45uA
      70V 9.7uA
      80V 80 9.82uA Still have diode response

10:15 In prep for taking data with beam tuning have tested the merger and tapeserver.
      To enable writing to file had to implement the method in https://elog.ph.ed.ac.uk/DESPEC/36
      Probably a corrupution caused in the difference between the carme system and the aida system.
      Now can write data to file. To limit the data of FEEs 01-04 have set threshold to 0xFF

10:33 Ion source was started this morning but there was no noticable effect on the leakage currents.

10:56 Leakage current is slowly rising continuing the trend at 60V. - Attachment 4
      This could just be the temperature shift during the day or linked to the ion source being powered on.

11:18 Pulser peaks for the FEEs using layout 3 and 4.
      Layout 3 - attachment 5
      Layout 4 - attachment 6
      The pulser peaks appear to be asymetric and in FEE 7 we are observing a double peak.
      Will check the circuit when we next have access.

11:23 Have started writing data to disk in advance of them putting beam into the ring.
      Writing to R4_8 at around 1.8MB/s. N.B. TapeServer shows the same behavior as AIDA with writing nothing to files R4_0->R4_7
      FEE 01-04 slow comp 0xFF
      FEE 05-08 slow comp 0x64
      Pulser running at 50Hz or so

14:50 Having been writing to DAQ since lunch.
      Leakage currents were mostly stable but we are now seeing an increase in leakage currents at a much greater rate - attachment 7

14:58 Attachment8 Updated leakage currents.

15:00 Due to the rapidly increasing bias we asked them to stop the beam.
      Motors were then cycled but we did not notice the leakage coming down.
      Bias was turned off and then motors cycled again.
      Bias was then set to 60V - Run R5
      Bias raised to 80V - R6
      Carlo requested we stop writing data to disk

16:25 Stopped writing to disk. Will retry bias at 80V and monitor.

18:13 Motors cycled before leaving for night no beam in ring - attachment 9
      to test if the increase is beam related we will leave at 80V overnight with the beam blocked all night.
      To avoid the rapidly increasing leakage current the iSet on the CAEN module is at 11uA with a trip time of 2s.
      

09:09 Arrived at GSI for comissioning beamtime.
      CARME currently in a stopped state, with all FEEs powered off.
      Upper detector is biased with 80V and has a leakage current of 9.92uA.
      This is up on the previous recording by JM of 5.4uA recorded in elog 187.
      It is possible light is reaching the detector via the interaction chamber which shall be checked.

09:19 Cave checked:
      Water is running to the FEEs (OK for power on when required)
      There are 4 uncovered windows on the interaction chamber. Will see if we can get these covered.

14:29 Tested different bias voltages.
      Between 80V and 120V got a normal diode response with small increases in leakage current
      At 120V leakage current was stable at 9.92 for one minute.
      Checked waveforms on DAQ and looked at current again and it was up to 16uA.
      Turned off bias and let ramp down.
      Now when trying to ramp up detector in similar position to last week with a diode response between 0-30V and a rapid increase above that.
      Carlo spoke to Gleb who was beam tuning. Which around the time of our trip he opened the valves and put beam into the ring for a rotation.
      Gleb put in several faraday cups prior to the CARME chamber yet we still see the non-diode bias characteristics.
      He will enter in 30 minutes and we will check if anything looks out of place. Will also remove motor pin to give option of movement.

15:00 Gleb and Carlo enters the cave.
      Dipole and beam off.
      Pin removed.
      Diode response restored 9.4uA at 40V. Which is only slightly above where it was during the morning


      Before they got anywhere with starting again the current has gone up again..

16:49 Since around 15:30 the current has come down to its levels earlier today at 40V. We are preparing a script to monitor the current levels going forward.

17:00 Gleb stopped tuning the beam for the day.
      There is no beam in the ring.
      Magnets and everything else continue as if there was beam in the ring.

19:40 Attempted to raise bias to 50. Immediately started to trip
      Carlo cycled motors multiple times.
      Bias in out position dropped.
      To lower
      Have made it to 60V. Will leave there overnight with leakage current tracking running.

      No beam so no daq running.

JM plans to install new crimped, 3mm dia copper drain wires (a la AIDA@DESPEC) this afternoon. 

Initial Grounding - Lemo cables connecting adaptor cards ground in daisy chain. Fee bodies connected by 3mm copper drain wires in daisy chain. Two chains brought together 
at final adaptor card which is connected to cryring ground. 

Attachments 1-5

ADC data item rates   all <10k
ASIC settings         all LEC slow comparators 0x64
1.8.L spectra aida05-08 widths 556, 511, ?, 570 ch FWHM respectively
1.8.W spectra 20us FSR
Adaptor card grounding image

1.8L widths appear broader than previous day of ~300 width. Reason unknown, no change made to setup as yet. 

Second Grounding - 3mm copper drain wires connect each adaptor card to Fee body. Fee bodies daisy chained together using 3mm copper drain wire and connected to cryring 
ground. 

ADC data item rates   all <10k
ASIC settings         all LEC slow comparators 0x64

Attachments 6-8
1.8.L spectra aida05-06 widths 620, 550 ch FWHM respectively, 1.1.L aida08 width 350 ch FWHM
1.8.W spectra 20us FSR. Some extra high frequency noise
Adaptor card grounding 
 

Third Grounding - 3mm copper drain wires connect adaptor cards in daisy chains. Fee bodies daisy chained together using 3mm copper drain wire. Two chains brought 
together and connected to cryring ground. 

ADC data item rates   all <10k
ASIC settings         all LEC slow comparators 0x64
1.8.L spectra aida05-08 similar to initial grounding ~500 ch FWHM

Attachments 9-11
1.8.W spectra 20us FSR.
AIDA5-8 channel rates. Rates dominated by noisy strips. Missing strips come and go as ASIC settings checked. 
Adaptor card grounding

Some work still required to reduce noise in the system and reduce pulser widths. Pulser should also be checked on scope 

Gauge turned back ON. Bias 80V, 3.5 uA before. Bias 80V, 5.4 uA after. Bias turned off and on again. NO change to diode characteristic, current ramps high before coming 
down once bias on. Bias left ON 80V to see if stable when second gauge turned on and valves reopened. Pulser ON. Fee power OFF. Water cooling ON. 

No significant changes to waveforms, pulser width or rates with gauge ON. Other noise contributions probably outweigh effect of the gauge at this point. 

3mm copper grounding cable and attachments, crimper and Yorkshire tea left in control room. 

DAQ was left running last night from 19:30. Detector bias 80V with a leakage current ~3.4 uA. Pulser OFF.

Observed alphas in spectra overnightin the 20,000 to 28,000 channel range for the pn side and 36,000 to 44,000 range on the nn side of the detector. Zero energy noise peak is at channel 32000.

Alphas checked for AIDA5 and AIDA6 (pn) with ~150 alphas counted using a range 20,000-26,000/27,000 depending how large the noise peak tail is. Attached spectra show the spectra for the 4 ASICs of AIDA5 and 6.

9:30 Bias remained stable overnight - 60V 2.95 uA.

10:00 No grounding attached initailly. Pulser ON, lots of noise across all ~250-300 kHz across biased detector. Waveforms and histograms saved to /tmp. Pulser width for aida 6 ~300 channels. Aida 7,8 similar, aida 5 producing no ADC data. 

11:00 Grounding connected as in elog 181. Rates decreased to ~180-250 kHz. Waveforms and histograms saved to /tmp. 

12:00 Bias slowly increased too 100V giving a leakage current of ~3.5 uA which was stable for several hours. Bias varied between 60V and 100V for noise reduction perposes, 80V deemed least noisy at the time, noise still ~180-200 kHz

13:30 New grounding trialled. Bottom detector removed from pulser and grounding loops so that only the biased detector is grounded. Bias set to 80V

14:00 Rates significantly reduced for aida 6,7,8 - aida5 still producing no ADC data. Rates are in range 15-50 kHZ with noisy channels accounting for most of the rates. Waveforms and histograms saved to /tmp. Pulser peak on aida 6 ~300 channels. 

16:00 Aida 5 fee investigated to reason it is not producing data. FEE was not properly connected to ERNI connector due to incorrect installation of black adaptor card mount. Adaptor card mount reinstalled and fee remounted. Aida 5 now produces ADC with similar rates to aida6,7,8. 

During investigation of aida5, aida4 and 5 were swapped. Config files relating to MAC addresses, polarity etc of FEES has been updated. Elog 175 will be appended to include this swap.  

19:00 Bias set to 80V, DAQ running with pulser off to allow for alpha detection overnight. 

FEE mounts for 5,6,7,8 disassembled to access adaptor boards. No sign of debris which could cause short on adaptor boards. 

Adaptor boards removed and checked against spares with multimeter. Both show same characteristics. Pins on feedthrough checked using multimeter. No longer see -18.5 KOhm between bias pins, no see 140 kOhm which is similar to bias pin to non bias pin from prevous tests.

Camera on interaction chamber removed and replaced with foil, no change. 

Ion gauge (Edinburgh group) turned ON

Decided to move detectors using motors. Movement caused change in bias characteristic of the detector. Initial movement was 50 repetitions in and out over which the bias was increased from 10V to 140V. Over this range the detector exhibited diode like behaviour. The difference between the in and position and out position caused a 0.5 uA difference in leakage current. Once stationary, the current was stabel at 140V but after 20s current increased rapidly and tripped.  

Increasing bias again to 30V while stationary (out position on motors) gave 7.3 uA , going to 60V tripped again. 

Moving the detectors IN but keeping stationary gave a stable current at 30V of 4.2 uA increasing to 70V tripped again. Turning off the gauge reduced the current at 30V to 2.9 uA and 70V to be be ~15.5 uA. Opening an optical window at 70V saw no change in leakage current. 

Possibility thermocouple attached to or around the detector may have come loose within the chamber? Thermocouples outisde of chamber removed from thermocouple flange so they don't interact with outside of chamber. 

Detector biased to 60V - 10 uA. Detector position moved in and out 20 repetions, during movement current dropped to 2.85 uA. After motion stopped leakage current remained stable at 2.85 uA. Optical window opened no change. Torch light shone into optical window - increase in leakage current to 3.5 uA. Torch off - return to 2.85 uA. 

Motion started again for 50 repetitions to test stability of leakage current. Current varies between 2.84-2.88 uA during movement, settles to 2.85 uA once motion stopped. 

FEE holder reassembley. Involved turning off bias, unplugging bias cables and rebuilding assembley. FEE holders reaasembled with FEEs reconnected. Water cables, HDMI, ethernet, power connected. Bias turned off and on during reassembley multiple times to check for changes. Each time bias at 60V remained 2.85uA. Each time lower voltages appear to show diode like response in leakage current output. 

Pulser connected and FEEs turned back on. Aida05 appears not producing ADC data rates 8 kHz on statistics page -investigate further. Aida06 rate ~70 kHZ can see pulser. Aida07, 08 nn bias show rates ~300 kHZ possible not enough bias for nn side. Rate for all FEEs yesterday (unbiased) was 200-300 kHZ except for AIDA03 disconnected and AIDA02 which wasn't producing ADC data. Photo of Aida06 spectra attached (log scale for pulser). 

FEEs turned OFF. Bias left on 60V - 2.85 uA see if stable overnight, stable for approx 2.5 hours before leaving cryring. Detector moved to OUT position, pin back in. 

To do - Grounding, reduce noise, waveforms,  investigate AIDA05, how high can we bias?  

Detector is not biasing as it has in previous tests. Last bias 15/1/22 see previous elogs, max leakage current ~10 uA at 150V

Detector bias increases well above 10uA for voltages as low as 30-40V. Optical windows covered, Y09 valves closed, large ion pump OFF and both ion gauges OFF produces similar results but with a slightly lower leakage current.

Grounding and pulser cables removed from FEEs and jumper attached to nn bias adaptor card - no change. Bias module swapped out - no effect. NIM rack shows expected voltages (6,12,24) on multimeter.

AIDA03 - disconnected. Behaves as expected. Get waveforms and pulser peak. Pulser peak FWHM ~ 70 channels, noise transients observed.

AIDA02 - Not producing ADC data

AIDA01,04,05,06,07,08 - Very high rates - noisy

Steps taken:
- MIDAS_Releases, Embedded and Exports files brought across from the previously working SL6 aida-gsi install. Brought as tar balls to preserve ownership
- dhcpd.conf set up with same IP addresses as used on the SL6 install from aida-gsi
- nfs-utils package installed on centos7
- Paths to directories listed in /etc/exports checked that they point to the correct locations on the centos system
- Firewall disabled to confirm that it is not interfering
- setenforce 0 to confirm selinux is not interfering.


- Attachment 1 Screenshot of FEE console for aida04 as it stops booting.
- Attachment 2 text dump of /var/log/messages during the FEE boot sequence
- Attachment 3 text dump from tcpdump filtered to aida04
- Attachment 4 /var/lib/dhcp/dhcp.leases file as mentioned in the /var/log/messages dump
- Attachment 5 A copy of dhcpd.conf
- Attachment 6 A copy of /etc/hosts
- Attachment 7 A copy of /etc/exports    
- Attachment 8 Text dump of ifconfig -a (Note the FEEs are conencted to p4p1)

From the messages dump it seems there is an issue being encountered with dhcp leases for the static IP addresses of the FEEs. They are then getting assigned new IP addresses which are getting put in the leases file.