ASIC settings for the pn FEEs has been reset to the default aida settings. This was done to combat the observed dropping of several  channels in pn only FEE modules 
following motors movement. This effect was also observed in 2022 in several pulser walkthroughs conducted pre and post detector movement. No dropped channels were 
observed in nn FEEs. 

ASIC seetings changed;

IBias LF feedback from 0xf to 0x8
diode link threshold from 0xbf to 0xca

We also set the slow comparatort for all FEE modules back to 0x64 (1 MeV)

Settings tested using only 2 pn FEE modules (aida05 and aida16). Movement left and right. Dropped channels observed in all pn FEEs except aida05 and aida16. All pn ASIC 
settings then changed. Movement for 10 repeptitions. No dropped channels observed in pn FEEs. 

New settings saved in 2024Feb08-15.59.09

EMERGENCY SHUTDOWNS

1. CARME motors: Q key on keyboard while on workspace 1 of appc218 (right of this double-monitor setup)

2. Internal target:

ELOG

Each shift should put all information relating to the shift in ONE elog entry unless a significant change occurs during the shift i.e beam off, target off, run change, detectors out etc.

Elog entry should be named by the shift - morning/afternoon/night followed by the current run number

TO DO FOR THE SHIFTERS - CHECK EVERY HOUR AND POST TO ELOG WITH SCREENSHOTS. ELOG should be updated with current beam energy, injection beam intensity, system checks OK/not, FEE temperature OK/not, Statistics OK/not, Merger OK/not, tape server data rate, Motors OK/not and min/max motors potentiometer value. The time of each update should be noted in the elog. Screenshots of important screens (detailed below) should also be uploaded to the elog.

CARME DAQ

CARME computer has 6 workspaces (WS), two monitors for each WS.

WS1 : left monitor - fee power

          right monitor - Detector Bias control, fee power server (just a terminal, do not close)

WS2 : left monitor - DAQ controls

          right monitor - DAQ server (just a terminal, do not close)

          Most important tabs (DAQ controls)

• Run control : shows daq status, stopped / going
• Fee temperatures : temperatures of fees labelled aida01, aida02, aida03, aida04, aida05, aida06, aida07, aida08, aida09, aida10, aida11, aida12, aida13, aida14, aida15, aida16. Hit reload, temperatures will update and should all be green. 1-2 C above safe limit (top) is OK. More than that - warn expert shifter immediately.
• Statistics : shows the statistics for each fee. Make sure ADC Data Items is selected in the left hand menu. Hit update all  (SCREENSHOT once, if no changes record OK in elog)
• System wide checks : checks for the setup. Click on 'Check Clock Status', 'Check ADC Calibration' and 'Check the White rabbit decoder status' to run checks. (SCREENSHOT once, if no changes record OK in elog)
• Spectrum browser : loads histograms for fees and fee channels from preset layouts. select layout from 'layout ID' then restore layout from 'Arrange functions' drop down menu. Check layout 1, Select log scale and change xmax as 128 . This displays the data rate per strip. If any strips are a lot higher than the others, or very different from previous entry, make a note in the ELOG. (SCREENSHOT once, if no changes record OK in elog)
• Asic control : sets asic parameters

WS3 : left monitor - Merge control, Tape server

          right monitor - servers (just terminals, do not close) , data being written to file

• Check NewMerger tab. Hit reload button and see if 'current links with data' and 'current merged data rate' are updating. All links should go green, but not necessarily at the same time. Keep hitting refresh until you see all of them going green at least once.
• Data rate, hit reload should be around 100,000. Record data rate in elog
• Check tapeService tab. Hit reload button. Check data rate in Kbytes/sec box. Record data rate to elog. This is where the current run number is displayed
• On the right-hand monitor, bottom right terminal, check the current run fragment (sub-run) is being written and is slowly increasing in size. It refreshed automatically every 5 seconds.

WS4 : Leakage current plot

• Check leakage current plot is stable. Transients will occur during motors movement but baseline current should remain consistent. Make sure it is updating and time is correct. (SCREENSHOT once, record OK in elog if no changes) Check present values on WS2, CAEN window

WS5 : Online data monitoring

• Check the following plots are updating for each DSSD
  • LowEnergyXYTotal (right click - col - colz). This shows the total number of events, per pixel, since the code was started / re-started.
  • LowEnergyXYRate (right click - col - colz). This shows the rate over the last 20 seconds.
  • LowEnergyExRate (double left click. Right click on y axis - Log scale on y). Rate over the last 20 seconds
  • LowEnergyExTotal (double left click. Right click on y axis - Log scale on y). This shows the total energy deposited on all X strips since code restart.
  • LowEnergyEyTotal (as above). as above, Y strips. (SCREENSHOT of each per DSSD per shift. If significant changes are observed screenshot again)

WS6 : Browser, ELOG

-->IMPORTANT NOTE: MIDAS DAQ can respond slowly. DO NOT try to change tab or give new command while the previous one is being executed. Check activity in right-hand part of tab currently in use. Do not double click on buttons.

Motors computer (Computer to the right)

• Check motors cycle is running. The code will cycle through the loaded motors instructions. When an instruction is being done it is highlighted in bold. Check the bold highlighting cycles through entire set of instructions.
• Check potentiometer is reproducing same max/min values for each cycle of instructions.
• Detectors are on LEFT and RIGHT arm.
• No need for screenshots, but record in the ELOG you checked this.

14:50

Temperature, Statistics fine (see screenshots).

System wide checks fine (see below).

All FEEs pass clock check.

All FEEs pass White Rabbit check.

ADC Calibration check:

FEE64 module aida02 failed
FEE64 module aida16 failed
Calibration test result: Passed 14, Failed 2

If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module

FPGA timestamp error check:

         Base         Current         Difference
aida09 fault      0x1 :      0xa :      9  
FPGA Timestamp error counter test result: Passed 15, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last

16:00 Slow comp thresh lowered to 100 keV in preparation for detector moving. Will have to test systematically what is the highest threshold we can accept

17:00 Temps fine. No overnight run.

11:00

Changed slow comp threshold to 0x32 (500 keV). Checkloaded.

Temperature, Statistics fine (see screenshots).

System wide checks fine (see below).

All FEEs pass clock check.

All FEEs pass White Rabbit check.

ADC Calibration check:

FEE64 module aida02 failed
FEE64 module aida16 failed
Calibration test result: Passed 14, Failed 2

If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module

FPGA timestamp error check:

         Base         Current         Difference
aida09 fault      0x1 :      0xa :      9  
FPGA Timestamp error counter test result: Passed 15, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last

10:30 Found ADC#4 on aida 1 and 4 are suddenly noisy. See attach 1-2. Checkloaded multiple times - no effect.

Leakage current overnight was stable, but a clear transient can be observed yesterday around the times of the DSSD motion test. We did not observe this in the cave looking at the CAEN module - it might have been too fast.

Now reducing bias to 50 V and will check differences later. FEE temperatures OK.

10:50 Started DAQ. Rates are much higher in n+n strips, but p+n strips appear unaffected. See attach 4-5.

Stopped DAQ. Will leave 50V until after lunch.

11:27 Checked motor control status after yesterday's setup (see previous ELOG entry). See attach 6.

IN counters are respectively: read, increases, latch (left), latch (right)

Read is the number of pulses sent with the instruction to move. Increases is the number of times the instruction was considered. Latch on/off indicates the 1 s time interval during which commands sent to a motor arm (left or right) are considered double-firing or mistake, and ignored.

The number of increases is the same for move in/out - good!

The number of increases for move out is 1 unit less than move in - one double pulse / double command must have been received overnight and ignored.

Note torque on right arm is 2 (arbitrary units) which is unusually high and may indicate some tension in the kapton wires. No apparent effect on DSSD2,4 on that arm. Will not move for now.

13:24 FEE temperatures OK, leakage currents decreasing slighly.

13:41 Slowly increased bias to 100V. Interestingly it is now DSSD#4 (Ch#3) that displays the highest leakage current. Attach 7

13:45 Stats & Rates look rather similar to earlier this morning at 150V, but aida01 and 04 are quieter. Attach 8-9. Will leave at 100V for a while longer before moving to 150V.

15:20 Set bias to 150V

15:50 Nikos testing target. Started short run. Stats & Rates attached (10-11). Same as yesterday, aida01 and 04 ADC#4 no longer noisy.

V-I on attach 12-13. Legends unreliable!

17:00 Nikos completed target test. YR09 pressure around 3E-10 mbar with around 5E11 atoms/cm2 He target. Target not quite stable (10-20%), some optimisation required, but pressure holds at 3E-10 mbar for several minutes.

Beam on target will increase ring pressure. Can't test for now. Target tests without beam complete. Need to log all the data remotely. No logging available at present.

Leaving bias at 150V overnight. No run.

17.43 setup signals from detector movement controller.

See attachment 1
                                                         NIM               TTL                 TTL             +12V
Signal from CRYRING ------> CAEN N89 ------> LF4000 -------> CAEN N89 -------> LeCroy 222 -------> PCB amp -------> start move in
( active high )         TTL              NIM  FIFO  -------> CAEN N89 -------> LeCroy 222 -------> PCB amp -------> start move out
                                                    not(NIM)               TTL                 TTL    ^        +12V
                                                                                                      |
                                                                                                      |
CAEN N89 mode switch to OUT                                                                                                  PSU +12V 

Widths of signals from LeCroy 222
 move in = 75ms
 move out = 25ms

Note - the outputs of EG&G/ESN LF4000 FIFO could be connected directly to the LeCroy 222 as the inputs accept NIM and TTL

16.40 CB/ML install fibre optic cable from patch panel beneath dipole downstream of YR09/CARME chamber to VETAR2 front panel fibre connector - see attachments 1 & 2.

VETAR2 LCD display changes from magenta to cyan.

DAQ stop. DAQ reset. DAQ setup.

All system wide checks OK.

WR timestamp control - attachment 3 is timed 16.43.11 UTC+1

WR timestamp 0x17B101E7 7FCD1ACD -> 1707147827543349965ns -> 1707147827s -> Monday, February 5, 2024 3:43:47 PM UTC ( https://www.epochconverter.com/ )

13.41 DAQ start - file G22-88/R4

Pulser walkthrough

EGG Ortec 448 Pulser

Amplitide 90,000-10,000 @ 10,000 step
Normalise 5.0
Attenuators x5 IN
t_r 100ns
Tau_d 50us
Frequency Int Osc 50Hz
Polarity + ( - polarity via Cooknell SA1 Sum & Invert Amp )

File G22-88/R4

14.08 DAQ stop file G22-88/R4_8

06.40 DAQ continues file G22-88/R3_42

      DSSSD bias OK - attachment 1
      ambient temperature 16.8 deg C

      FEE64 temperatures OK - attachment 2

      ADC data item stats - attachments 3
       aida15 rate c. 34k ( 2x 'hot' channels, 1x channel missing, all other channels OK )
       aida01 rate c. 75k ( 2x 'hot' channels )

      per FEE64 Stat spectra - attachments 4-5

      Merger and TapeServer - attachments 6-7

      All system wide checks OK except
       aida02 & aida08 FPGA timestamp error - attachment 8

      WR timestamps OK - attachment 9

09.47 DAQ continues file G22-88/R3_46

      DSSSD bias OK - attachment 10
      ambient temperature 16.8 deg C

      FEE64 temperatures OK - attachment 11

      ADC data item stats - attachments 12
       aida15 rate c. 34k ( 2x 'hot' channels, 1x channel missing, all other channels OK )
       aida01 rate c. 75k ( 2x 'hot' channels )

      per FEE64 Stat spectra - attachments 13-14

      Merger and TapeServer - attachments 15-16

09.52 per FEE64 Rate spectra - attachments 17-18

      per FEE64 1.8.L spectra - attachments 19-20

      per FEE64 1.8.H spectra - attachments 21-22

      per p+n FEE64 1.8.W spectra - 20us, 200us, 2ms & 20ms FSR - attachments 23-26

      per n+n FEE64 1.8.W spectra - 20us, 200us, 2ms & 20ms FSR - attachments 27-30


10.11 Merger stats blocks 0-4, 14-16 - attachments 31-38

10.14 aida01-aida04, aida14-aida16 stats- attachments 39-45

10.21 Overnight V-I attach 46

10.23 Save current ASIC settings 2024Feb05-10.22.28
       changes to disc mask and LED waveform threshold for some FEE64s

10.32 Analysis of data file G22-88/R3_45 - attachments 47
       deadtime aida01 c. 0.5%, all other FEE64s deadtime c. 0%


13.07 DAQ stop file G22-88/R3_50
      Stop merger
      Stop tapeserver

Afternoon: Tested detector motion. Channels on p+n strips drop out with high threshold (1 MeV), but not with low threshold (100 keV). No issues on n+n strips. Reasons unclear.

09.30 DAQ continues file G22-88/R3_17

      DSSSD bias OK - attachment 1
      ambient temperature 16.8 deg C

      FEE64 temperatures OK - attachment 2

      ADC data item stats - attachment 3
       aida15 rate c. 25k ( 2x 'hot' channels, 1x channel missing, all other channels OK )
       aida01 rate c. 45k ( 2x 'hot' channels )

      per FEE64 Stat spectra - attachments 4-5

      per FEE64 1.8.L spectra - attachments 6-7

      per FEE64 1.8.W spectra - 20us FSR - attachments 8-9

      Merger and TapeServer - attachments 10-11

09.36 Check ASIC load
      No change to aida01 ADC data item rate

      Last access times of data files indicate that data rate changed c. 05.00 - 06.00 this am.
      No obvious vacuum/temperature transients in last six hours.

09.55 All system wide checks OK except
       FPGA timestamp errors - attachment 12
      WR timestamps OK - attachment 13

10:06 V-I plot overnight - attach 14.

13.35 DAQ continues file G22-88/R3_22

      DSSSD bias OK - attachment 15
      ambient temperature 16.8 deg C

      FEE64 temperatures OK - attachment 16

      ADC data item stats - attachment 17
       aida15 rate c. 35k ( 2x 'hot' channels, 1x channel missing, all other channels OK )
       aida01 rate c. 51k ( 2x 'hot' channels )

      per FEE64 Rate spectra - attachments 18

      per FEE64 Stat spectra - attachments 19-20

      Merger and TapeServer - attachments 21-22

1. Check Proxy server is correct

localhost : 8080

incoming connections 7070

Establish port

ssh -L 8080:proxy.gsi.de:8080 carme@atppc025

2. Check a password for remote access has been set up

3. Close anydesk and check it is *really* closed using

ps -o pid= -C anydesk

kill -9 any process (as root)

4 Restart AnyDesk

systemctl restart anydesk (as npg)

01.07 DAQ continues file G22-88/R1_46
      background alpha

      DSSSD bias OK - attachment 1
      FEE64 temps OK - attachment 2
      ADC data item stats - attachment 3
       aida15 asic #2 u/s - rate c. 266k 

      aida11 1*L spectra - attachment 4

      ADC data items/s Merger, Tape Server & Merger stats - attachments 5-7
       data rate to disk c. 2Mb/s dominated by aida15 asic 2


10.05 DAQ continues file G22-88/R1_79
      background alpha

      DSSSD bias OK - attachment 8
       ambient temperature 16.8 deg C
      FEE64 temps OK - attachment 9
      ADC data item stats - attachment 10
       aida15 asic #2 u/s - rate c. 266k 

      aida11 1*L spectra - attachment 11

      ADC data items/s Merger, Tape Server & Merger stats - attachments 12-14
       data rate to disk c. 2Mb/s dominated by aida15 asic 2

10.12 per FEE64 Stat spectra - attachment 15
      missing channels aida03, aida04, aida05 & aida15
      Check ASIC load aida03, aida04, aida05 & aida15

13:25 Overnight leakage current plot - att 16


13.38 DAQ continues file G22-88/R1_92
      background alpha

      DSSSD bias OK - attachment 17
       ambient temperature 16.8 deg C
      FEE64 temps OK - attachment 18
      ADC data item stats - attachment 19
       aida15 asic #2 u/s - rate c. 266k 

      aida11 1*L spectra - attachment 20
       ASIC check load at 10.12 should result in 1-2 hits per channel in the Stat spectra aida03, aida04, aida05 and aida15
       Channels which continue to show zero counts likely indicate a FEE64-adaptor PCB connection issue
       Channels which now show 1, or more, counts cf. attachment 15 likely correspond to channels now working following the check ASIC load

      ADC data items/s Merger, Tape Server & Merger stats - attachments 21-23
       data rate to disk c. 2Mb/s dominated by aida15 asic 2

13.48 Stopped DAQ file G22-88/R1_93

      DSSSD bias OFF
      FEE64 power OFF


16.00 Installed FEE64 power cable to aida16
      Replaced aida15 with new FEE64 MAC 41:f6:5a

      Update dhcpd.conf
      Update MIDAS config to include aida16

      FEE64 power ON
      
      All system wide checks OK

      DSSSD bias OK - attachment 24

      FEE64 temperatures OK - attachment 25

      ADC data item stats - attachment 26
       aida15 rate changes from c. 260k to c. 25k ( 2x 'hot' channels, 1x channel missing, all other channels OK )

      per FEE64 Rate spectra - attachment 27

      per FEE64 1.8.L spectra - attachments 28-29
       pulser peak width aida11 44 ch FWHM, aida14 57 ch FWHM

      per FEE64 1.8.W spectra - 20us FSR - attachments 30-31

      
     Update NewMerger startup script and Options for 16x FEE64s - see attachments 32-33


16.20 file G22-88/R2
      Pulser 50Hz

16.28 DAQ stop

      Pulser OFF

      All histograms zero'd

16.33 DAQ starts file G22-88/R3
      alpha background

      ADC data item stats OK - attachment 34

      NewMerger, TapeServer and Merger stats - attachments 35-38

16.50 Pulser peak width spreadsheet for file R2 - attachment 39

20.32 DAQ continues file G22-88/R3_9

      DSSSD bias OK - attachment 40

      FEE64 temperatures OK - attachment 41

      ADC data item stats - attachment 42
       aida15 rate c. 25k ( 2x 'hot' channels, 1x channel missing, all other channels OK )

      per FEE64 Stat spectra - attachment 43

      Merger, Merger stats and TapServer - attachments 44-46

      Disc data items - attachment 47
       aida15 disc rate c. 8k
      Disable aida15 disc ( fast comparator ) outputs for asic#3 - attachment 48 
      Disc data items- attachment 49
       all FEE64s rate << 1Hz
    
     Update NewMerger startup script and Options for 16x FEE64s - see attachments 32-33

DAQ and NewMerger appear to be cofigured correctly.

Can allocate device, create volume and open file with TapeServer but cannot write data.

See attachmnents 1-3




Noted aida09 WR timestamp issue in attachment 3 - FEE64 Reboot fixed aida09 WR timestamp issue

Restarted DAQ/Merger/TapeServer - still cannot write to disk


14.30 CU provides the following list of NetVar variables used by the Merger

"MERGE_PID"
"MERGE_ID"
"MERGE_Action"
"MERGE_Signal"
   "MERGE_Parameter"
   "MERGE_Response"
   "MERGE_State"
   "MERGE_Trace"
   "MERGE_Paused"
   "MERGE_RunOptions"
   "MERGE_HardwareVersion"
   "MERGE_LinksAvailable"
   "MERGE_TestMode"
   "MERGE_PollTicks"
   "MERGE_LinksInUse"
   "MERGE_LinksAlive"
   "MERGE_StatsTime"
   "MERGE_Command"
   "MERGE_ZeroStats"
   "MERGE_SyncStep"
   "MERGE_InitialTimeStamp"
"MERGE_LinkState"
"MERGE_LinkStatus"
"MERGE_LinkAlive"
"MERGE_LinkInUse"

15.30 Changed Merger startup and NetVars files ( attachments 6 & 7 )

      Change server command line from

./master64 -i 16 -l 16 -p 11001  &

to

./master64 -i 15 -l 15 -p 11001  &


i.e. *exactly* matches number of FEE64s currently in use ( 15x FEE64s, aida01-aida15 )

      Change NetVars from 

MERGE.LinksInUse string 1%1%1%1%1%1%1%1%1%1%1%1%1%1%1%0%

to

MERGE.LinksInUse string 1%1%1%1%1%1%1%1%1%1%1%1%1%1%1%

i.e. string has 15x 1% strings *not* 15x 1% *plus 1x 0% string 

With *both* changes we are now able to write to disk.

N.B. We did not test the above changes individually - it is possible/probable that just one of the changes above fixes the problem.

Current pulser widths excel spreadsheet attached

09.58 DSSSD bias OK - attachment 1 

Ambient temperature +16.1 deg C FEE64 temps OK - attachment 2 

ADC data item stats OK - attachment 3 
 aida15 asic #2 high rate - asic u/s

system wide tests OK *except* aida09 - attachments 4-5 
 aida09 WR status 0x20 aida09 WR timestamp incorrect cf. all other FEE64s

per FEE64 Rate spectra - attachment 6

per FEE64 1.8.L spectra - attachments 7-8 
 aida14 pulse peak witdh 58 ch FWHM
 aida11 pulse peak witdh 48 ch FWHM

per FEE64 1.8.W spetra - 20us FSR - attachments 9-10
 

12.00 

aida09 WR timestamp issue resolved by FEE64 reboot 


14.10 

Safety pins removed from detector motion system. CB performs one, complete DSSSD movement cycle. Zero all histogram and stats per FEE64 Stat spectra ( log scale ) and 
ADC 
data item stats for the complete DSSSD movement cycle - attachments 11-12 
No change observed in DSSSD leakage currents during DSSSD movement cycle. 

15.35 

Resolved issue from previous elog. Begin background run writing data to disk.
ASIC check load 
Start DAQ G22-88/R1 background alpha 
EG&G Ortec 448 Pulser OFF 
ASIC settings 2024Jan20-12.02.08 
slow comparator 0x64 ( all FEE64s *except aida15 asic #2 0xff ) 
All histograms set to zero

DSSSD bias OK - attachment 13 
FEE64 temps OK - attachment 14 

WR timestamps OK - attachment 15 

ADC data item stats - attachment 16 
aida15 asic #2 u/s - rate c. 250k 

ADC data items/s Merger, Tape Server & Merger stats - attachments 17-19 
 data rate to disk c. 2Mb/s dominated by aida15 asic 2

To include alll 16 FEE modules in the NewMerger we edited the file /MIDAS/DB/EXPERIMENTS/MERGE/Options/carme-gsi from MERGE.LinksAvailable string 5 to MERGE.LinksAvailable string 15

We also edited the file /MIDAS/Linux/startup/NewMerger from ./master64 -i 8 -l 16 -p 11001  & to ./master64 -i 16 -l 16 -p 11001  &

We have started the Merger and tape server. The merger settings were 'corrupted' and did not display xfer Links => Merger. We implemented a fix from https://elog.ph.ed.ac.uk/DESPEC/36 and this resolved the issue.

We started the tape server, merger and go on the run control. We observe the merger working and data links to the tape server. A new file is created each time but we observe no data being written to that file. After 5 minutes of no data being written the merger crashed and the fees crashed soon after. We will leave until tomorrow to resolve.

Issue subsequently resolved see elog https://elog.ph.ed.ac.uk/CARME/487

Arrived at the control room to find we had lost connection to the power relay. We rebooted the nnrpi1 and re-connected according to the steps laid out in elog 471 -> all okay

11:00 Restarted daq with no changes to the system from yesterday. Bias consistent with previous days. All pulser peaks are slightly improved (even more for the pn pulser, one goes from 91 chn to 78). No pulser observed in nn FEE modules 9, 10, 13, 14 again.

ADC data items check OK - all ~3k

Rates - all OK  - attachment 1

1.8L spectra. - attachment 2,3

1.8W waveforms - attachment 4,5

11:20 Changed shaping time from 0xf to 0x3 (2us)

ADC data items check OK - all ~3k attachment 6

Rates - all OK  - attachment 7

1.8L pn spectra. - attachment 8

1.8W waveforms - attachment 9,10

11:35 Changed shaping time from 0x3 to 0x7 (4us)

ADC data items check OK - all ~3k attachment 11

Rates - all OK  - attachment 12

1.8L pn spectra. - attachment 13

1.8W waveforms - attachment 14,15

11:45 Changed shaping time from 0x7 to 0xb (6us)

ADC data items check OK - all ~3k - attachment 16

Rates - all OK  - attachment 17

1.8L spectra. - attachment 18,19

1.8W waveforms - attachment 20,21

11:59 Changed shaping time from 0xb to 0xd (7us)

ADC data items check OK - all ~3k - attchment 22

Rates - all OK  - attachment 23

1.8L pn spectra. - attachment 24

1.8W waveforms - attachment 25, 26

12:09 Changed shaping time back to 0xf (8us)

ADC data items check OK - all ~3k - attchment 27

Rates - all OK  - attachment 28

1.8L pn spectra. - attachment 29

1.8W waveforms - attachment 30,31

13:10 Power cycled FEE modules. nn pulser for FEEs 9,10,13,14 are now active again (no changes made in the cave) Pulser widths for nn and pn  both consistent with before lunch

Removed ground from DSSD #1 to the chamber and set MACBs so that timestamp is from the NIM WR emulator not the VME VETAR2.

Root MACB mode 0x3 -> 0xd - all other MACB modes 0x3 ( unchanged ).

No timestamps observed from the emulator. Re-set to take timestamps from the VETAR" unit -> all timestamps observed.

10:30

No pulser observed in nn FEE modules 9, 10, 13, 14.

ADC data items check OK - all ~3k - attachment 2

Rates - all OK  - attachment 3

1.8L spectra. - attachment 4,5

1.8W waveforms - attachment 6,7

11:00

Checked daisy chain which goes from 8->14->13->10->9. Replaced the lemo cable from 8->14 and pulsers re-appear. Rates all OK except aida10 which has now disappeared, tried asic check and reset and startup again no change in aida10.

Now majority of pulsers are observable the widths will be noted in a spreadsheet

After replacing lemo for nn pulser chain, all nn pulser widths decreased slightly ~10% for all nn FEEs, little change in pn pulser widths but n pulser for DSSD#1,2 appears more gaussian

11:44

FEE64s power cycled. aida10 now prioducing data.

Removed internal jumper from CAEN N1419ET 

ADC data items check OK - all ~3k - attachment 8

Rates - all OK  - attachment 9

1.8L spectra - attachment 10,11 little change in pn pulser widths but nn pulser widths are slightly broader.

1.8W waveforms - attachment 12,13

15:50 Jumper re-mounted onto caen bias module

ADC data items check OK - all ~3k - attachment 14

Rates - all OK  - attachment 15

1.8L spectra - attachment 16,17

1.8W waveforms - attachment 18,19

16:45 Ground cables attached from water pipe (chamber ground) to lemo cable on aida08 (nn), aida13 (nn), aida09 (nn), aida01 (pn) so that all DSSDs are connected to chamber ground.

ADC data items check OK - all ~3k - attachment 20

Rates - all OK  - attachment 21

1.8L spectra - attachment 22,23

1.8W waveforms - attachment 24,25

This is how to update the MACB firmware using the opensource tool xc3sprog


Setup (only has to be done first time)
--------------------
Install xc3sprog and fxload (ubuntu: sudo apt install xc3sprog fxload)

To configure the USB programmer:
Download xilinx.tar.gz and put the hex files in /usr/share

Download the udev rules and put it in /etc/udev/rules.d

Run:
sudo udevadm control --reload-rules

[Taken from https://github.com/timvideos/litex-buildenv/wiki/Xilinx-Platform-Cable-USB-under-Linux]

Plug in programmer and should work

----------------------------------
Programming a MACB

Open MACB side panel and connect JTAG programmer to port 

Plug MACB into NIM bin and power NIM bin on

Run
xc3sprog -c xpc -m -j

Expected output:
XC3SPROG (c) 2004-2011 xc3sprog project $Rev$ OS: Linux
Free software: If you contribute nothing, expect nothing!
Feedback on success/failure/enhancement requests:
        http://sourceforge.net/mail/?group_id=170565 
Check Sourceforge for updates:
        http://sourceforge.net/projects/xc3sprog/develop

JTAG loc.:   0  IDCODE: 0x16d8c093  Desc:                  XC2C128_TQ144 Rev: A  IR length:  8

This confirms the JTAG is working

Then to program  run

xc3sprog -c xpc -m . macb_apr20.jed

The directory should contain the macb_apr20.jed and the xc2c128.map files

Expected output:
XC3SPROG (c) 2004-2011 xc3sprog project $Rev$ OS: Linux
Free software: If you contribute nothing, expect nothing!
Feedback on success/failure/enhancement requests:
        http://sourceforge.net/mail/?group_id=170565 
Check Sourceforge for updates:
        http://sourceforge.net/projects/xc3sprog/develop

Programming row  81                                        
Verify: Success   

Now power off the NIM bin and remove the programmer and put the cover back on

09.00 CARME IE514 5.1e-11mbar

09.15 IE514 not working - one of cables disconnected?

      DSSSD #1 bias cable changed from SHV - Lemo 00.250 to SHV - 2x Lemo 00.250

      DSSSD #1 & #2 LK1 removed from n+n Ohmic side bias FEE64s

      aida11 HDMI cabling re-seated at MACB

10.26 DSSSD bias OK - attachment 1
      Ambient temperature +16.2 deg C

10.30 ADC data item stats OK - attachment 2
      aida06 & aida15 > 100k, all others c. 3k consistent with 50Hz pulser

10.34 per FEE64 Rate spectra - attachment 3
       OK except for known issues aida05, aida07 and aida16 - all others OK

      per p+n FEE64s 1.8.L spectra - attachment 4
       pulser peak width 52 ch FWHM aida11- double peaking DSSSD #1 & #2

      per n+n FEE64s 1.8.L spectra - attachment 7
       pulser peak width 77 ch FWHM aida04 - more uniform noise across all DSSSDs

10.42 per FEE64 1.8.W spectra - 20us FSR - attachments 5-6

11:00 LK2 removed from aida01-04. Configuration for all adaptor cards/FEE modules is now the same. No other change to the setup.

12:00 FEEs power on, bias to 150 V. No change in bias observed. 

12.02 ADC data item stats OK 
      aida06 & aida15 > 100k, all others c. 3k consistent with 50Hz pulser

12.07 per FEE64 Rate spectra 
       OK except for known issues aida05, aida07 and aida16 - all others OK
      
12:09  per p+n FEE64s 1.8.L spectra 
       We now observe a large peak around chn 30,000 on all FEEs. Perhaps an effect of some change in the ring?
       pulser peak width 53 ch FWHM aida11- still observe double peaking DSSSD #1 & #2

      per n+n FEE64s 1.8.L spectra 
      Lots of events at ~chn 65,000 - double peaking now observed on aida03 and aida04
      pulser peak width 93 ch FWHM aida14 

12.20 per FEE64 1.8.W spectra - 20us FSR 

      Perhaps bias was off while acquiring and spectra was not cleared properly. Re-do tests

13:40 ADC data item stats OK - attachment 7 
      aida06 & aida15 > 100k, all others c. 3k consistent with 50Hz pulser

       per FEE64 Rate spectra - attachment 8
       OK except for known issues aida05, aida07 and aida16 - all others OK

      per p+n FEE64s 1.8.L spectra - attachment 9 
       pulser peak width 55 ch FWHM aida11- double peaking DSSSD #1 & #2

      per n+n FEE64s 1.8.L spectra - attachment 10
       pulser peak width 65 ch FWHM aida14 - more uniform noise across all DSSSDs

       per FEE64 1.8.W spectra - 20us FSR - attachments 11-12 

      With help from NH updated firmware on all MACB modules to version 4/20 - see https://elog.ph.ed.ac.uk/CARME/482

      Timestamps for all FEE modules now correct - attachment 13

      MAC address of aida07 was found to be 41-b4-16 not 41-d4-16 as was labelled on the FEE module
       re-labelled FEE64
       dhcpd.conf and MIDAS startup.tcl updated, dhcpd and MIDAS 8015 server restarted - aida07 is now working OK
 
      CAEN N1419ET internal jumper fitted - outputs are no longer floating

15:20 ADC data item stats OK 
      aida06 & aida15 > 100k, all others c. 3k consistent with 50Hz pulser - attachment 14

       per FEE64 Rate spectra
       OK except for known issues aida05, and aida16 - all others OK  - attachment 15

      per p+n FEE64s 1.8.L spectra -
       pulser peak width 50 ch FWHM aida11- double peaking DSSSD #1 & #2 - attachment 16

      per n+n FEE64s 1.8.L spectra 
       pulser peak width 67 ch FWHM aida14 - more uniform noise across all DSSSDs

       per FEE64 1.8.W spectra - 20us FSR - aida14 missing from waveforms - attachments 17,18
 
   Swapped the test inputs so that +ve is now from the inverter and -ve direct from pulser.  - attachments 19,20. Observed reduced noise for when plugged directly into the pulser than when connected from the inverter. 



   Lowered threshold from 0x64 (1 MeV) to 0x20 (320 keV) - attachment 21, 22 - Observed increase in rates, in particular for the DSSD #1,2 pn side. Conclusion -> increased pulser width for DSSD #1,2 is real and not from the pulser
 


   aida01 lemo to water pipe (chamber ground) connection installed 

   ADC data item stats OK 
      aida06 & aida15 > 100k, all others c. 3k consistent with 50Hz pulser 

       per FEE64 Rate spectra
       OK except for known issues aida16 - all others OK 

      per p+n FEE64s 1.8.L spectra -
       pulser peak width 48 ch FWHM aida11- double peaking DSSSD #1 & #2 - attachment 23

      per n+n FEE64s 1.8.L spectra 
       pulser peak width 67 ch FWHM aida14 - attachment 24

       per FEE64 1.8.W spectra - 20us FSR - slightly reduced 100 kHz on pn waveforms - attachments 25,26

   Disconnected aida06 and replaced with the spare FEE module (mac address: 42:0d:16). Power cycled and updated config files. 

   ADC data item stats OK 
      aida15 > 100k, all others including aida06 c. 3k consistent with 50Hz pulser - attachment 27

       per FEE64 Rate spectra
       OK except for known issues aida16 - all others OK - attachment 28

      per p+n FEE64s 1.8.L spectra -
       pulser peak width 48 ch FWHM aida11- no longer observe double peaking DSSSD #1 & #2 but the pulser peaks are still broader compared to DSSD #3,4 - attachment 29

      per n+n FEE64s 1.8.L spectra 
       pulser peak width 55 ch FWHM aida14 - attachment 30

       per FEE64 1.8.W spectra - 20us FSR - greatly reduced 100 kHz on pn waveforms, dodgy FEE previously aida06 must@ve had some impact on DSSD#1,2 - attachments 31,32
      

Test + and test - daisy chain cabling extended from DSSSD #1 to #2 to DSSSD #1 to DSSSD #4
 terminated by 50 Ohm at end of daisy chain

Grounding cabling extended from DSSSD #1 ( only ) to DSSSD #1 to DSSSD #4
 grounding cables daisy-chained between 4x adaptor PCBs of each DSSSD - no inter-DSSSD grounding cables

DSSSD#1 bias cable SHV-Lemo 00.250 ( to p+n bias adaptor PCB )
DSSSD#2 - DSSSD #4 bias cables SHV - 2x Lemo 00.250 ( to p+n/n+n bias adaptor PCBs )

LK1 fitted for DSSSD #1 & #2 n+n Ohmic side bias adaptor PCBs
LK2 not fitted on any adaptor PCBs? to be checked

CAEN N1419ET internal jumper fitted ( non-floating outputs )? to be checked

16.54 DSSSD bias OK - see attachment 1

      FEE64 temperatures - attachment 2
       aida06 ASIC temp low

      System wide checks and WR - attachments 3-6
       global clock status aid04 0x6, aida06 0x4
       multiple FPGA timestamp and WR decoder errors for MACBs 4 and 5 ( of 5 )

       WR timestamps OK for aida01-aida08 ( w/ MACB firmware version 04/20 )

     ADC data item stats - attachment 7
      all rates consistent with 50Hz pulser *except* 
       aida11 global clock fail
       aida07 no boot
       aida16 no power
       aida06 ? 
       aida15 faulty asic #2

     per FEE64 Rate spectra - attachment 8

     per p+n and n+n FEE64 1.8.L spectra - attachments 9-12
      Pulser peak width p+n side 52 ch FWHM ( aida12 ), n+n side 70 ch FWHM ( aida14 )

     per p+n and n+n 1.8.W spectra - 20us FSR - attachments 13-16