AIDA GELINA BRIKEN nToF CRIB ISOLDE CIRCE nTOFCapture DESPEC DTAS EDI_PSA 179Ta CARME StellarModelling DCF K40
  DESPEC, Page 10 of 36  ELOG logo
Entry  Sat Jun 8 23:00:52 2024, TD, Sunday 9 June 16x
23.58 8.6.24

      DSSSD bias & leakage current OK - attachments 1

      FEE64 temperatures OK - attachment 2

      ADC data item stats - attachment 3

      per FEE64 Rate spectra - attachment 4

03.43 slow comparator 0x64

      DSSSD bias & leakage current OK - attachments 5

      FEE64 temperatures OK - attachment 6

      ADC data item stats - attachment 7

      per FEE64 1.8.H spectra - attachments 8-9

      per FEE64 Rate spectra - attachment 10


09.11 DSSSD bias & leakage current OK - attachments 11

      FEE64 temperatures OK - attachment 12

      ADC data item stats - attachment 13

      per FEE64 Rate spectra - attachment 14


15.05 online spectra
      per DSSSD HEC E_p versus E_n and x versus y
      observe flat field of c. 500MeV at per spull rate c. 5Hz - fission fragments?
Entry  Sun May 8 15:10:45 2022, TD, Sunday 8 May 6x
DSSSD bias and FEE64 power OFF during Ge install



Photographs of AIDA (triple) + DEGAS setup as of Sunday pm.

Attachments 1-4 side/front/oblique views - note AIDA snout droops and twists

Attachments 5 & 6 - position of the AIDA stand vertical adjustment threads (brass) 

The asymmetry upstream-downstream pre-dates this setup. To lift the end of the snout clear of the of lower DEGAS detectors all 3x 1/4 turns clockwise (viewed from above) were applied to each of the four threads.
Entry  Sat Mar 6 23:37:22 2021, TD, Sunday 7 March 00.00-08:00 18x
00.30 UNILAC problem - no beam

00:39 DAQ continues R1_518
      FRS 190W setting 

      ASIC settinfs 2019Dec19-16.19.51
      DSSSD#1 slow comparator 0xa
      DSSSD#2 slow comparator 0xa
      DSSSD#3 slow comparator 0xa

      BNC PB-5 Pulser 
      Amplitude1.0V
      Attenuation x1
      Frequency 2Hz
      tau_d 1ms
      - polarity
      Delay 250ns, tail pulse

01:07 all system wide checks OK

      detector biases & leakage currents OK - see attachment 1
      FEE64 temperatures OK - see attachment 2
      good event statistics - OK - see attachment 3
      Merger OK - see attachment 4
      TapeServer OK - see attachment 5

01:20 zero'd all histograms
      ASIC check load

      rate spectra - see attachment 6

01:48 beam return

      run74 (R1_608)

03.05 all system wide checks OK

      detector biases & leakage currents OK - see attachment 7
      FEE64 temperatures OK - see attachment 8
      good event statistics - OK - see attachment 9
      Merger OK - see attachment 10
      TapeServer OK - see attachment 11

     rate spectra - see attachment 12

04.46 run74 ends (R1_841)

05.00 run75 (R1_859)

05:40 detector biases & leakage currents OK - see attachment 13
      FEE64 temperatures OK - see attachment 14
      good event statistics - OK - see attachment 15
      Merger OK - see attachment 16
      TapeServer OK - see attachment 17

     rate spectra - see attachment 18

07:29 run76 (R1_1055)
Entry  Sun Apr 7 18:13:56 2024, TD, Sunday 7 April contd. 6x
19.10 CAEN N1419ET LK fitted
      power cycle

attachments 1-6
Entry  Sun Apr 7 12:02:23 2024, TD, Sunday 7 April 46x
13.00 Cooling water temperature and flow OK - attachment 1

      Test of AIDA 2x MSL type BB18(DS)-1000 24cm x 8cm DSSSDs 'as is' ( to be defined later in this Elog )

      DSSSD bias & leakage current OK - attachment 2
      Ambient temp 24.7 deg C, d.p. 7.1 deg C, RH 32.4%
      Leakage current c. 6.6uA => 4nA/cm2/100um ( very good )

      FEE64 temps OK - attachment 3
      *except* aida02 ASIC temp which is known to be u/s

      All system wide checks OK *except* aida02 and aida04 WR decoder status - attachment 4

      WR timestamps OK - attachment 5

      WR (info code 4 & 5), correlation scaler, PAUSE, RESUME, DISC, ADC data item stats - attachments 6-12

      per FEE64 Rate spectra - attachments 13-15

      per 1.8.W spectra - 20us FSR - attachments 16-1710129


14.45 FEE64 config check - per https://elog.ph.ed.ac.uk/DESPEC/562


FEE64 # PSU cable #           MAC
1       1                     ?
2       2                     ?
3       3                     ?
4       4                     41 ee 71
5       5                     ?
6       6                     ?
7       7                     f6 5a
8       8                     41 d7 cd
9       ?                     ?
10      10                    41 d0 0e
11      11                    41 ee 0f
12      ?                     ?
13      13                    ?
14      ?                     0d 15
15      15                    ?
16      16                    f6 ed

? = no line of sight

- AIDA FEE64 PSU cabling - see attachment 18

Currently

PSU #1 1-3, 2-4, 9-5, 15-12
PSU #2 14-7, 6-8, 10-13, 11-16

Should be changed to

PSU #1 1-3, 2-4, 9-15, 5-12
PSU #2 14-7, 6-8, 10-11, 13-16


- AIDA snout mount to support frame - attachments 19 & 21
LHS ( looking downstream ) incorrect, RHS OK

- LKs
LK1 fitted aida02, aida04, aida06, aida08
LK3 fitted aida03, aida07, aida01, aida14


Significant gaps in snout ( i.e. possible light leaks ) where bPlas cabling exits snout - attachments 20 & 22

bPlas driver PCBs removed 
bPlas ribbon cables, drain wires and Lemo 00.250 cabling disconnected - some cabling/drain wires touching snout etc


15.15 Adjust bPlas ribbon cables/drain wires/Lemo cables to ensure that none are touching snout or AIDA support frame - significant improvement

      DSSSD bias & leakage current OK - attachment 23

      FEE64 temps OK - attachment 24

      ADC data item stats - attachments 25

      per FEE64 Rate spectra - attachments 26
       6x FEE64s < 20k. all FEE64s < 120k ( cf. 4x FEE64s < 20k, 5x FEE64s > 100k, 1x FEE64 320k earlier - attachment 12 )

      per 1.8.W spectra - 20us FSR - attachments 27-28

17.15

LHS top hat washers
LK1, LK3
PSU power cables
tighten gnd screws
attachments 29-34

Note that some of the newly installed FEE64s have the incorrect rails fitted - FEE64s may ground to AIDA support stand

17.55 CAEN N1419ET LK removed -> floating outputs

18.15 +LK1 restored
attachments 35-40

18.40 +pwwer cycle
attachments 41-46
Entry  Sun Jun 6 14:15:05 2021, TD, Sunday 6 June 9x
14.15 DAQ continues file S496/R32_117
      Hot HEC channels multiple FEE64s from c. 06.00 this morning
      ASIC check fixes

      System wide checks OK *except*

	 Base 		Current 	Difference
aida01 fault 	 0x8944 : 	 0x8946 : 	 2  
aida02 fault 	 0x9b36 : 	 0x9b38 : 	 2  
aida03 fault 	 0xf26b : 	 0xf26d : 	 2  
aida04 fault 	 0x57a0 : 	 0x57a2 : 	 2  
aida05 fault 	 0x4d3c : 	 0x4d3d : 	 1  
aida05 : WR status 0x10
 aida06 fault 	 0x640c : 	 0x640d : 	 1  
aida07 fault 	 0x255e : 	 0x255f : 	 1  
aida08 fault 	 0xffbd : 	 0xffbe : 	 1  
White Rabbit error counter test result: Passed 8, Failed 8

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

	
Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0  
Mem(KB) :	4	8	16	32	64	128	256	512	1k	2k	4k
aida01 :     15     12     12      3      3      4      2      2      1      4      7   : 40572
aida02 :      7      8     10      5      2      3      3      2      1      4      7   : 40604
aida03 :     27     10      4      8      1      4      1      3      2      3      7   : 39740
aida04 :     18      7      4      5      1      3      2      3      2      3      7   : 39712
aida05 :     15     13      8      4      1      3      1      3      1      4      7   : 40548
aida06 :     14      8      8      6      3      2      1      3      1      4      7   : 40568
aida07 :     11      4      5      6      2      3      3      2      1      4      7   : 40540
aida08 :     15      6      5      3      4      3      3      2      1      4      7   : 40604
aida09 :     18      4      4      3      3      2      2      3      1      4      7   : 40648
aida10 :     12      6      8      6      1      1      1      3      1      4      7   : 40288
aida11 :     16      8      6      6      4      4      3      3      2      3      7   : 40352
aida12 :     10      7      4      8      3      4      3      3      2      3      7   : 40288
aida13 :     15      7      5      4      5      4      3      3      2      3      7   : 40324
aida14 :      9      4      6      7      3      4      3      3      2      3      7   : 40260
aida15 :     12      4      8      3      2      2      2      4      1      4      7   : 41136
aida16 :     11      8      6      6      3      4      3      3      2      3      7   : 40268

	
 Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same

 FEE : aida01 =>   Options file size is 1026 	 Last changed Fri Jun 04 09:30:42 CEST 2021
 FEE : aida02 =>   Options file size is 1025 	 Last changed Sun May 23 00:19:21 CEST 2021
 FEE : aida03 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:43:50 CEST 2021
 FEE : aida04 =>   Options file size is 1025 	 Last changed Fri May 14 16:54:56 CEST 2021
 FEE : aida05 =>   Options file size is 1025 	 Last changed Mon May 17 06:25:41 CEST 2021
 FEE : aida06 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:43:59 CEST 2021
 FEE : aida07 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:02 CEST 2021
 FEE : aida08 =>   Options file size is 1025 	 Last changed Sun May 23 00:16:54 CEST 2021
 FEE : aida09 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:08 CEST 2021
 FEE : aida10 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida11 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida12 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida13 =>   Options file size is 1025 	 Last changed Fri May 07 19:40:34 CEST 2021
 FEE : aida14 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida15 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida16 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021

      Grafana - DSSSD bias & leakage current - most recent 7 days - attachment 1
      Lost activity monitor - attachment 2
      1.8.W spectra - 20us FSR - attachments 3 & 4
      ADC data items - attachment 5
      FEE64 temperatures OK - attachment 6
      DSSSD bias & leakage currents OK - attachment 7
      Merger/Tape Server/Merger statistics - attachment 8
       no merger errors reported since previous restart

16.28 analysis of file S496/R32_118 - attachment 9
       max deadtime 0.28% aida04
Entry  Sun May 5 10:22:24 2024, TD, Sunday 5 May Capture.PNG
11.22 DSSSD#1 bias -120V leakage current -17.1uA
      DSSSD#2 bias -120V leakage current -10.6uA
      S4 temperature 25 deg C

      cf. 07.00 29.4.24 S4 temperature 25 deg C

      DSSSD#1 -21.8uA
      DSSSD#2 -12.8uA

      Leakage currents currently c. 80% of 29.4.24 values at same temperature 
Entry  Sun May 30 08:57:53 2021, TD, Sunday 30 May 9x
09.55 DAQ continues - file S496/R18_588
      alpha background

      increased data rate due to hot HEC channels since c. 06.00 this am - ASIC check fixes problem

      All system wide checks OK *except*

		 Base 		Current 	Difference
aida01 fault 	 0x82b0 : 	 0x82b7 : 	 7  
aida02 fault 	 0xfe72 : 	 0xfe79 : 	 7  
aida03 fault 	 0x97f4 : 	 0x97fb : 	 7  
aida04 fault 	 0x950 : 	 0x957 : 	 7  
aida05 fault 	 0x8e93 : 	 0x8e98 : 	 5  
aida06 fault 	 0x545a : 	 0x545f : 	 5  
aida07 fault 	 0xc04d : 	 0xc052 : 	 5  
aida08 fault 	 0xa0f7 : 	 0xa0fc : 	 5  
White Rabbit error counter test result: Passed 8, Failed 8

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0  
Mem(KB) :	4	8	16	32	64	128	256	512	1k	2k	4k
aida01 :      6      9      5      1      3      2      2      2      2      4      6   : 37008
aida02 :      3      5      5      2      2      4      3      3      2      4      6   : 37956
aida03 :     22     12      7      5      3      3      2      3      3      3      6   : 36872
aida04 :     40     13      5      5      3      4      2      3      3      3      6   : 37048
aida05 :     15      5      5      4      2      2      2      2      2      4      6   : 37044
aida06 :     11      6      2      1      2      3      3      4      2      4      6   : 38300
aida07 :     17      5      4      3      1      3      3      2      2      4      6   : 37324
aida08 :      1      8      1      2      2      4      4      3      2      4      6   : 38164
aida09 :     10      8      6      1      2      4      3      3      1      4      6   : 36968
aida10 :     14      7      5      2      1      3      3      3      1      4      6   : 36800
aida11 :      7      9      3      1      2      3      3      3      1      4      6   : 36788
aida12 :      3      5      2      3      1      4      2      3      2      4      6   : 37620
aida13 :     12      4      4      2      1      3      2      2      2      4      6   : 37008
aida14 :     11     10      5      1      1      3      2      2      2      4      6   : 37036
aida15 :     11      5      1      2      1      3      3      4      2      4      6   : 38244
aida16 :     10      4      8      1      2      3      2      2      2      4      6   : 37096

 Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same

 FEE : aida01 =>   Options file size is 1026 	 Last changed Sun May 23 12:04:25 CEST 2021
 FEE : aida02 =>   Options file size is 1025 	 Last changed Sun May 23 00:19:21 CEST 2021
 FEE : aida03 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:43:50 CEST 2021
 FEE : aida04 =>   Options file size is 1025 	 Last changed Fri May 14 16:54:56 CEST 2021
 FEE : aida05 =>   Options file size is 1025 	 Last changed Mon May 17 06:25:41 CEST 2021
 FEE : aida06 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:43:59 CEST 2021
 FEE : aida07 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:02 CEST 2021
 FEE : aida08 =>   Options file size is 1025 	 Last changed Sun May 23 00:16:54 CEST 2021
 FEE : aida09 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:08 CEST 2021
 FEE : aida10 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida11 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida12 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida13 =>   Options file size is 1025 	 Last changed Fri May 07 19:40:34 CEST 2021
 FEE : aida14 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida15 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida16 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021

      Grafana - DSSSD bias & leakage current - most recent 7 days - attachment 1
      Lost activity monitor - attachment 2
      1.8.W spectra - attachments 3 & 4
      ADC data item stats - attachment 5
      FEE64 temps OK - attachment 6
      DSSSD bias & leakage current OK - attachment 7
      Merger/Tape Server/Merger stats - attachment 8
       no merger errors reported since previous restart

14.24 All histograms, statistics and merger statistics zero'd

16.16 analysis of file S496/R18_541 - attachment 9
       max deadtime 0.02% aida03

18.40 DAQ continues - file S496/R18_589
      alpha background

      increased data rate due to hot HEC channels since c. 18.00 this am - ASIC check fixes problem
Entry  Sun Mar 3 16:06:11 2024, TD, Sunday 3 March 24x
17.00 FEE64 41:a0:71 ASIC #2 u/s
      Replaced ASIC mezzanine with new ASIC mezzanine
      Installed as aida04

      Unable to boot nnrpi2 - manual/local control of BNC PB-5 and FEE64 PSU - no interlock

      Water temperature & pressure as measured outside S4 area - OK 

      Manually power FEE64s

      BNC PB-5 local control/ON
      Amplitude 1.0V
      Attenuation x10
      Polarity -
      tau_d 1ms
      Frequency 22Hz

      FEE64 temps OK - attachment 1

      All system wide checks OK

      WR timestamp OK - attachment 2

      aida04 ASIC settings - attachment 3

      ADC data item stats OK - attachment 4
   
      aida04 Rate spectrum - attachment 5

      aida04 *.*.L spectra - attachments 6-9
       1.8.L pulser peak width 15 ch FWHM

      aida04 1.8.W spectra - 20us FSR - attachment 10-11

      MIDAS configuration - attachment 12

      DHCP config - attachment 13



18.30 FEE64 41:f6:b7 ASIC #1 u/s
      Replaced ASIC mezzanine with new ASIC mezzanine
      Installed as aida02

      All system wide checks OK

      FEE64 temps - attachment 14
       aida02 ASIC temperature c. 512 deg C !
       PSU/Virtex temps OK
       Mezzanine ambient to touch - assume faulty sensor/poor connection?

      aida02 ASIC settings - attachment 15

      WR timestamp OK - attachment 16

      ADC data item stats OK - attachment 4

      aida04 Rate spectrum - attachment 18

      aida02 1.8.W spectra - 20us FSR - attachment 19-20

      aida02 *.*.L spectra - attachments 21-24
       1.8.L pulser peak width 16 ch FWHM


19.30 Power OFF



     
Entry  Sun Apr 3 10:56:11 2022, TD, Sunday 3 April 35x
all system wide checks OK *except* aida10 failed ADC calibration

Attachment 1 ADC data item stats
Attachment 2 FEE64 temps OK
Attachment 3 DSSSD bias & leakage currents OK





DSSSD bias to left (looking upstream) wafer *only*
      FEE64s 10 & 12, 14 & 16

Attachments 4 & 5 1.8.W spectra 20us FSR

Attachment 6 ADC data item stats

Attachment 7 DSSSD bias & leakage current




ASIC check all FEE64s

DSSSD bias to middle (looking upstream) wafer *only*
      FEE64s 1 & 3, 5 & 7

Attachments 8 & 9 1.8.W spectra 20us FSR

Attachment 10 ADC data item stats

Attachment 11 DSSSD bias & leakage current





ASIC check all FEE64s

DSSSD bias to right (looking upstream) wafer *only*
      FEE64s 9 & 11, 13 & 15

Attachments 12 & 13 1.8.W spectra 20us FSR

Attachment 14 ADC data item stats

Attachment 15 DSSSD bias & leakage current



Conclusions

ADC data item stats -> 300-400k

n+n 1.8.W spectra show large signal excursions

p+n 1.8.W spectra for biased Si wafers OK - unbiased Si wafers show large signal excursions


DSSSD#1 leakage currents 1.850+2.220+1.990=6.06uA    cf. 5.095uA with all 3 wafers biased 
DSSSD#2 leakage currents 1.510+1.580+2.350=5.44uA    cf. 5.235uA 


Right wafer (looking upstream)
             
Bias(V)         -20    -40    -60    -80    -100
DSSSD#1 I_L(uA) -1.135 -1.475 -1.645 -1.750 -1.805
DSSSD#2 I_l(uA) -1.625 -2.020 -2.205 -2.320 -2.360

Middle wafer (looking upstream)
          
Bias(V)         -20    -40    -60    -80    -100
DSSSD#1 I_L(uA) -1.540 -1.870 -2.035 -2.150 -2.210
DSSSD#2 I_l(uA) -0.985 -1.285 -1.465 -1.575 -1.615

Left wafer (looking upstream)
          
Bias(V)         -20    -40    -60    -80    -100
DSSSD#1 I_L(uA) -0.735 -0.915 -1.005 -1.080 -1.110
DSSSD#2 I_l(uA) -0.865 -1.205 -1.400 -1.510 -1.560

Bias versus leakage cuurent characteristic of each wafer of each DSSSD indicates diode behaviour.

Note LK1 not fitted - presumably LK3 of mid/top p+n FEE64 adaptor PCBs (aida01 and aida05) connecting n+n bias to gnd is sufficient to establish circuit.




15.50 Re-connect bias to all 3x Si wafers p+n junction side bias rings

ASIC check all FEE64s

Attachments 16 & 17 1.8.W spectra 20us FSR

Attachment 18 ADC data item stats

Attachment 19 DSSSD bias & leakage current

Conclusion - status quo ante restored



16.40 Remove aida01 & aida05 LK3, add aida04 & aida08 LK1

ASIC check all FEE64s

Attachments 20 & 21 1.8.W spectra 20us FSR

Attachment 22 ADC data item stats

Attachment 23 DSSSD bias & leakage current

Conclusion - no change



17.00 Add Thorn-Lab HV filters DSSSD#1 & #2


ASIC check all FEE64s

Attachments 24 & 25 1.8.W spectra 20us FSR

Attachment 26 ADC data item stats

Attachment 27 DSSSD bias & leakage current

Conclusion - aida09-aida12 no change, aida13-aida16 -> 300-400k
             I do not know why there's a difference between DSSSD#1 & #2 but more important point (probably) no improvement aida09-aida12
             Thorn Lab HV filters removed - status quo ante restored


18.02 Remove LK2 & LK4 all top p+n adaptor PCBs (aida13, 5, 14, 9, 1, & 10)

ASIC check all FEE64s

Attachments 28 & 29 1.8.W spectra 20us FSR

Attachment 30 ADC data item stats

Attachment 31 DSSSD bias & leakage current

Conclusion - no change cf. 16.40 test



19.10 Re-order power cabling at AIDA FEE64 PSUs from ...

Top    1-3 2-4 9-10 11-12
Bottom 5-7 6-8 13-14 15-16

to ...

Top    1-2 3-4 9-10 11-12
Bottom 5-6 7-8 13-14 15-16


ASIC check all FEE64s

Attachments 32 & 33 1.8.W spectra 20us FSR

Attachment 34 ADC data item stats

Attachment 35 DSSSD bias & leakage current

Conclusion - all p+n FEE64 rates c. 100k, or less, n+n FEE64 rates c. 400k

FEE64 adaptor PCB config

LK2-LK3-LK4 fitted for all bottom p+n FEE64s (aida11, 3, 12, 15, 7, 16)

LK1 fitted n+n FEE64s aida06 & aida02 *only*
Entry  Sun Jun 26 08:51:20 2022, OH, NH, Sunday 26 June 08:00-24:00 27x
09:51 Taken over from Tom following the night shift
      Experiment is still running smoothly
      Compression of the files is complete up to the start of R5.
      Have started compression of files in R5 which should run up to R5_499
      Currently on R5_528

      Statistics ok - attachment 1
      Temperatures ok - attachment 2
      Bias and leakage currents ok - attachment 3
      System wide checks:
          		 Base 		Current 	Difference
      aida07 fault 	 0xc53d : 	 0xc594 : 	 87  
      aida08 fault 	 0xf1be : 	 0xf271 : 	 179  
      White Rabbit error counter test result: Passed 6, Failed 2

      			 Base 		Current 	Difference
      aida07 fault 	 0x2a : 	 0x3b : 	 17  
      FPGA Timestamp error counter test result: Passed 7, Failed 1

      Current merger rate is 2E6-4E6 events per second
      Current tapeserver rate is 5800 kB/s
      Free HDD space is 1.1 TB
      At current rates will last 54 hours which should see out the experiment
      Experiment currently scheduled to finish at 6am on Tuesday morning (May get until 8am)
      
      Analysis of R5_528 - attachment 4
      Deadtime of AIDA02 currently around 15%

11:56 Statistics ok - attachment 5
      Temps ok - attachment 6
      Bias and leakage currents ok - attachment 7
      System wide checks:
      Clocks all ok 
      
         		 Base 		Current 	Difference
      aida07 fault 	 0xc53d : 	 0xc59c : 	 95  
      aida08 fault 	 0xf1be : 	 0xf27a : 	 188  
      White Rabbit error counter test result: Passed 6, Failed 2

	
			 Base 		Current 		Difference
      aida07 fault 	 0x2a : 	 0x3b : 	 17  
      FPGA Timestamp error counter test result: Passed 7, Failed 1

      Analysis of file R5_458 - attachment 8
      Deadtime in AIDA02 only 11.5% in this file
      TapeServer rate still 5.5 MB/s

14:36 Has been no beam for the last while or so.
      With no beam AIDA02 has 0.25% deadtime so the deadtime is almost entirely due to the spill on time
      Stats- attachment 9
      Temp - attachment 10
      Bias and leakage currents ok - attachment 11
      System wide checks:
      Clock ok
      	
        		 Base 		Current 	Difference
      aida07 fault 	 0xc53d : 	 0xc59d : 	 96  
      aida08 fault 	 0xf1be : 	 0xf27c : 	 190  
      White Rabbit error counter test result: Passed 6, Failed 2
      	
			 Base 		Current 		Difference
      aida07 fault 	 0x2a : 	 0x3c : 	 18  
      FPGA Timestamp error counter test result: Passed 7, Failed 1

13:30 Beam taken to change an ion source

14:57 Beam back

16:21 Statistics ok - attachment 12
      Temperatures ok - attachment 13
      Bias and leakage currents ok - attachment 14

      System wide checks - ASIC clocks ok 
	
	        	 Base 		Current 	Difference
      aida07 fault 	 0xc53d : 	 0xc5a3 : 	 102  
      aida08 fault 	 0xf1be : 	 0xf285 : 	 199  
      White Rabbit error counter test result: Passed 6, Failed 2

	
			 Base 		Current 		Difference
      aida07 fault 	 0x2a : 	 0x3e : 	 20  
      FPGA Timestamp error counter test result: Passed 7, Failed 1

      Currently on file R5_592
      Analysis of R5_591 - Attachment 15
      Deadtime of AIDA02 sitting at 17%

[NH taking over for OH so he can get home]

18:53 - FRS DAQ problems mean we weren't taking DESPEC data for the past hour or so... now all back
        Statitics ok - attachement 16
        Temps ok - attachement 17
        Bias & leakage ok - attachement 18

        System wide checks - 
        Clocks OK
        ADC Calibration N/A 
	WR Decoder -
        		 Base 		Current 	Difference
        aida07 fault 	 0xc53d : 	 0xc5ac : 	 111  
        aida08 fault 	 0xf1be : 	 0xf28c : 	 206  
        White Rabbit error counter test result: Passed 6, Failed 2
        FPGA -
         Base 		Current 		Difference
        aida07 fault 	 0x2a : 	 0x3f : 	 21  
        PLL OK

        Currently on file R5_621
        Analysis of R5_620 - Attachement 19
        aida02 deadtime only 5%, all others negligible 

20:20 Stats ok - attachment 20
      Temps ok - attachment 21
      Bias and leakage currents ok - attachment 22
      System wide checks:
      Clock ok
    	
	        	 Base 		Current 	Difference
      aida07 fault 	 0xc53d : 	 0xc5ad : 	 112  
      aida08 fault 	 0xf1be : 	 0xf293 : 	 213  
      White Rabbit error counter test result: Passed 6, Failed 2

     	
			 Base 		Current 		Difference
      aida07 fault 	 0x2a : 	 0x3f : 	 21  
      FPGA Timestamp error counter test result: Passed 7, Failed 1

      Analysis of R6_637 - attachment 23

22:13 Statistics ok - attachment 24
      Temperatures ok - attachment 25
      Bias and leakage currents ok - attachment 26
      ASIC clock check ok

	
		         Base 		Current 	Difference
      aida07 fault 	 0xc53d : 	 0xc5ba : 	 125  
      aida08 fault 	 0xf1be : 	 0xf2a2 : 	 228  
      White Rabbit error counter test result: Passed 6, Failed 2

			 Base 		Current 		Difference
      aida07 fault 	 0x2a : 	 0x40 : 	 22  
      FPGA Timestamp error counter test result: Passed 7, Failed 1

      Analysis of R5_658 - attachment 27
Entry  Sat Jun 25 22:48:57 2022, TD, Sunday 26 June 00:00-08:00 30x
23.43 Check ASIC control
      Zero stats & all histograms

ASIC settings 2021Apr29-13-16-00
 slow comparator 0x64 -> 0xa

all waveform AD9252 ADCs disabled
all fast discs disabled

BNC PB-5 settings (to p+n FEE64s only)
 amplitude 1.0V
 attenuator x1
 decay time 1ms
 polarity +
 frequency 22Hz

All system wide checks OK *except* WR & FPGA errors - attachments 1 & 2

DSSSD bias & leakage currents OK - attachment 3

adc data item stats - attachment 4

FEE64 temps OK - attachment 5

DSSSD bias & leakage currents OK - attachment 6

analysis of file S505/R5_415 - attachment 7
 zero timewarps
 max deadtime aida02 c. 17%, all other FEE64s < 1%



02:36

analysis of file S505/R5_449 - attachment 8
 zero timewarps
 max deadtime aida02 c. 15%, all other FEE64s < 1%

per FEE64 rate & stat spectra - attachments 9 & 10

per p+n FEE64 1.8.L spectra - attachment 11
 aida01 pulser peak width 99 ch FWHM

per FEE64 1.8.H spectra - attachments 12 & 13

All system wide checks OK *except* WR & FPGA errors - attachments 14 & 15

adc data item stats - attachment 16

FEE64 temps OK - attachment 17

DSSSD bias & leakage currents OK - attachment 18


05:38

analysis of file S505/R5_483 - attachment 19
 zero timewarps
 max deadtime aida02 c. 16%, all other FEE64s < 2%

All system wide checks OK *except* WR & FPGA errors - attachments 20 & 21

adc data item stats - attachment 22

FEE64 temps OK - attachment 23

DSSSD bias & leakage currents OK - attachment 24



07:27

analysis of file S505/R5_502 - attachment 25
 zero timewarps
 max deadtime aida02 c. 16%, all other FEE64s < 2%

All system wide checks OK *except* WR & FPGA errors - attachments 26 & 27

adc data item stats - attachment 28

FEE64 temps OK - attachment 29

DSSSD bias & leakage currents OK - attachment 30
Entry  Sun Apr 25 10:22:16 2021, TD, Sunday 25 April 9x
11.25 All system wide checks OK *except*

	 Base 		Current 	Difference
aida01 fault 	 0xc95f : 	 0xc960 : 	 1  
aida02 fault 	 0x8bbe : 	 0x8bbf : 	 1  
aida03 fault 	 0xae9d : 	 0xae9e : 	 1  
aida04 fault 	 0x6e2 : 	 0x6e3 : 	 1  
aida05 fault 	 0x31a2 : 	 0x31a4 : 	 2  
aida05 : WR status 0x10
 aida06 fault 	 0x22e7 : 	 0x22ea : 	 3  
aida07 fault 	 0x438d : 	 0x438f : 	 2  
aida08 fault 	 0x61eb : 	 0x61ed : 	 2  
White Rabbit error counter test result: Passed 4, Failed 8
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
	
Returned 0 0 0 0 0 0 0 0 0 0 0 0  
Mem(KB) :	4	8	16	32	64	128	256	512	1k	2k	4k
aida01 :     55     21     31      6      3      2      2      3      2      3      4   : 28148
aida02 :     28      3      5      4      0      3      1      3      3      3      6   : 36312
aida03 :     25     10      2      2      2      4      1      3      3      3      6   : 36500
aida04 :     14      7     10      1      2      2      2      3      3      3      6   : 36528
aida05 :     58     35     28      6      4      2      2      3      3      3      4   : 29312
aida06 :     16     11      6      1      0      4      1      3      3      3      6   : 36376
aida07 :     75     31     25      9      4      3      2      2      2      3      4   : 27988
aida08 :     28      7      1      1      1      4      1      3      3      3      6   : 36376
aida09 :     50     48     28     10      3      3      3      4      2      3      4   : 29320
aida10 :     26      5      4      3      1      3      2      4      2      3      6   : 36080
aida11 :     21      7      5      3      5      2      2      4      2      3      6   : 36220
aida12 :     64     29     26      7      3      1      2      3      2      3      4   : 28072

 Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same

 FEE : aida01 =>   Options file size is 1026 	 Last changed Sat Apr 24 03:57:22 CEST 2021
 FEE : aida02 =>   Options file size is 1014 	 Last changed Fri Apr 16 00:56:20 CEST 2021
 FEE : aida03 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:04 CEST 2021
 FEE : aida04 =>   Options file size is 1025 	 Last changed Sat Apr 17 06:07:36 CEST 2021
 FEE : aida05 =>   Options file size is 1025 	 Last changed Fri Apr 16 00:53:25 CEST 2021
 FEE : aida06 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:04 CEST 2021
 FEE : aida07 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:04 CEST 2021
 FEE : aida08 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:04 CEST 2021
 FEE : aida09 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:05 CEST 2021
 FEE : aida10 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:06 CEST 2021
 FEE : aida11 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:05 CEST 2021
 FEE : aida12 =>   Options file size is 1025 	 Last changed Mon Apr 19 09:05:25 CEST 2021

Most recent merger errors reported - server terminal now reset & clear'd

MERGE Data Link (4408): bad timestamp  10 3 0xc2b67d8f 0x0b50c3e2 0x0000f437cb50c3e2 0x1678f437cb50c3e2 0x1678f43a16f1e6b2
MERGE Data Link (4408): bad timestamp  10 3 0xc2877f27 0x0b50cbb2 0x0000f437cb50cbb2 0x1678f437cb50cbb2 0x1678f43a16f1e6b2
MERGE Data Link (4408): bad timestamp  10 3 0xc2a07fcd 0x0b50cbb2 0x0000f437cb50cbb2 0x1678f437cb50cbb2 0x1678f43a16f1e6b2
MERGE Data Link (4408): bad timestamp  10 3 0xc2b77c9c 0x0b50cbb2 0x0000f437cb50cbb2 0x1678f437cb50cbb2 0x1678f43a16f1e6b2
MERGE Data Link (4408): bad timestamp  10 3 0xc28b7d9c 0x0b50d382 0x0000f437cb50d382 0x1678f437cb50d382 0x1678f43a16f1e6b2
MERGE Data Link (4408): bad timestamp  10 3 0xc2937e6c 0x0b50e322 0x0000f437cb50e322 0x1678f437cb50e322 0x1678f43a16f1e6b2
MERGE Data Link (4408): bad timestamp  10 3 0xc2a17d78 0x0b50e322 0x0000f437cb50e322 0x1678f437cb50e322 0x1678f43a16f1e6b2
MERGE Data Link (4408): bad timestamp  10 3 0xc2817fff 0x0b50eaf2 0x0000f437cb50eaf2 0x1678f437cb50eaf2 0x1678f43a16f1e6b2
MERGE Data Link (4408): bad timestamp  10 3 0xc2b77ecf 0x06fb3d52 0x0000f437cb50eaf2 0x1678f437c6fb3d52 0x1678f43a16f1e6b2

11.25 Grafana DSSSD bias |& leakage currents for previous 7 days - attachment 1
      DSSSD bias |& leakage currents Ok - attachment 2
      FEE64 temps OK - attachment 3
      good event stats - attachment 4
      merger time seq errors - attachment 5

      All histograms, stats and merge stats zero'd
      WR decoder errors *new*  baseline

11.45 1.8.L spectra - attachments 6 & 7
      peak width FWHM aida05 65 ch, aida06 95 ch
      1.8.W spectra = attachments 8 & 9
Entry  Sat May 22 23:40:32 2021, TD, Sunday 23 May Screenshot_2021-05-23_AIDA_Alerting_-_Grafana.pngScreenshot_2021-05-23_Statistics_aidas-gsi.pngScreenshot_2021-05-23_Temperature_and_status_scan_aidas-gsi.png810.pngR16_33
00.40 Unable to connect to FEE64s 1-7
      DAQ power cycled OK

      ASIC settings 2021Apr29-13-16-00
      slow comparator 0x64 *all* FEE64s

      BNC PB-5 OFF

      file S496/R16

      alpha background

07.17 Some hot HEC channels
      ASIC check fixes issue
      System wide checks OK *except*

		 Base 		Current 	Difference
aida06 fault 	 0x5457 : 	 0x5457 : 	 0  
aida06 : WR status 0x10
 White Rabbit error counter test result: Passed 15, Failed 1

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR


			 Base 		Current 		Difference
aida12 fault 	 0x0 : 	 0x1cd : 	 461  
FPGA Timestamp error counter test result: Passed 15, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last


Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0  
Mem(KB) :	4	8	16	32	64	128	256	512	1k	2k	4k
aida01 :     16      9      4      3      1      4      3      2      1      4      7   : 40552
aida02 :      3      2      2      1      2      3      3      2      2      4      7   : 41308
aida03 :     14      5      0      1      1      3      2      4      2      3      7   : 40000
aida04 :     14      4      5      1      3      3      1      4      2      3      7   : 39944
aida05 :     12      4      1      3      2      3      3      2      2      4      7   : 41408
aida06 :      8      3      2      1      2      4      2      3      3      4      7   : 42744
aida07 :     10      4      3      2      1      2      2      3      2      4      7   : 41464
aida08 :      7      5      2      0      3      3      3      2      2      4      7   : 41380
aida09 :     10      5      1      1      1      2      2      3      2      4      7   : 41408
aida10 :     11      6      3      2      1      3      2      2      2      4      7   : 41100
aida11 :     12      8      3      2      2      2      3      2      1      4      7   : 40288
aida12 :      9      9      3      1      2      3      3      2      1      4      7   : 40380
aida13 :     10      4      0      1      1      2      1      3      2      4      7   : 41128
aida14 :      9      4      2      1      1      2      1      3      2      4      7   : 41156
aida15 :      5      4      0      3      3      3      3      3      2      4      7   : 41940
aida16 :      9      6      3      2      0      2      1      3      2      4      7   : 41156

Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same

 FEE : aida01 =>   Options file size is 1026 	 Last changed Sun May 23 00:29:48 CEST 2021
 FEE : aida02 =>   Options file size is 1025 	 Last changed Sun May 23 00:19:21 CEST 2021
 FEE : aida03 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:43:50 CEST 2021
 FEE : aida04 =>   Options file size is 1025 	 Last changed Fri May 14 16:54:56 CEST 2021
 FEE : aida05 =>   Options file size is 1025 	 Last changed Mon May 17 06:25:41 CEST 2021
 FEE : aida06 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:43:59 CEST 2021
 FEE : aida07 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:02 CEST 2021
 FEE : aida08 =>   Options file size is 1025 	 Last changed Sun May 23 00:16:54 CEST 2021
 FEE : aida09 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:08 CEST 2021
 FEE : aida10 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida11 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida12 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida13 =>   Options file size is 1025 	 Last changed Fri May 07 19:40:34 CEST 2021
 FEE : aida14 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida15 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida16 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021

      FEE64 temperatures OK - attachment 1
      ADC data item stats - attachment 2
      Grafana DSSSD bias & leakage currents most recent 7 days - attachment 3
      DSSSD bias & leakage currents OK - attachment 4

11.18 Analysis of file S496/R16_33 - attachment 5

[npg@aidas-gsi S496]$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_aidasgsi-lv_root
                       50G   18G   30G  37% /
tmpfs                 7.9G  685M  7.2G   9% /dev/shm
/dev/sda2             477M   42M  410M  10% /boot
/dev/sda1             200M  264K  200M   1% /boot/efi
/dev/mapper/vg_aidasgsi-lv_home
                      401G   90G  292G  24% /home
/dev/mapper/vg_aidas2-lv_home
                      7.2T  6.0T  867G  88% /media/1e121361-83d3-4825-b6ae-8700b07e0ca7
/dev/sdd1             7.2T  5.6T  1.3T  82% /media/SecondDrive
/dev/sdc1             7.2T  6.6T  259G  97% /media/ThirdDrive

867/2 x 3279s = 16.5 days to filesystem full - OK

12.11 aida15 zero data/poll stats - re-loaded DAQ program OK
      aida04 unable to connect - multiple merger starts ineffective - aida04 rebooted and DAQ re-started OK

      DAQ continues file S496/R18
Entry  Sun May 22 14:12:55 2022, TD, Sunday 22 May Screenshot_from_2022-05-22_15-10-25.pngScreenshot_from_2022-05-22_15-10-48.pngScreenshot_from_2022-05-22_15-11-35.png
15.11 DAQ stopped

Attachment 1 - DSSSD bias & leakage current

Attachment 2 - grafana DSSSD bias, leakage current - OK, temp data has ceased

Attachment 3 - FEE64 temps - OK
Entry  Sun Apr 21 00:34:33 2024, TD, Sunday 21 April 37x
01.35 DSSSD bias & leakage current OK - Grafana - attachment 1

      FEE64 temperatures OK - attachment 2
 
      ADC data item stats OK - attachment 3

      Merger OK - attachment 4

04.05 DSSSD bias & leakage current OK - Grafana - attachment 5

      FEE64 temperatures OK - attachment 6
 
      ADC data item stats OK - attachment 7

      Merger OK - attachment 8

08.20 DSSSD bias & leakage current OK - Grafana - attachment 9

      FEE64 temperatures OK - attachment 10
 
      ADC data item stats - attachment 11
       increased rates observed esp. DSSSD#2 n+n

      Merger etc - attachment 12
      rate increased x2 cf. overnight

      WR timestamp - attachment 13
       aida03 out sequence

09.25 MBS data relay restarted

      Merger data links - attachment 14
       aida03 rate zero

      per FEE64 1.8.W spectra - 20us FSR - attachments 15-16

      per FEE64 Rate spectra - attachment 17
      increased rate for DSSSD#2 n+n FEE64s in particular

      MIDASsort online analysis - attachments 18-19
      per DSSSD e_front versus e_back for LEC (20keV/channel) and HEC (20MeV/channel)
      observe light ions (LEC) c. 2MeV
      observe low rate of high energy ions (HEC) c. 600MeV
      similar energies in both DSSSDs for LEC and HEC implying both types of event have range >> 2mm Si


10.09 DAQ STOP
      aida03 does not respond - reboot aida03

      DAQ RESET/SETUP etc     

      Synchronise ASIC clocks
      All system wide checks OK *except* FPGA errors

      Data transfer disabled
      Histograms & stats zero'd
      DAQ GO

      Merger restarted 

      Data transfer enabled
      DAQ GO

      WR timestamp OK - attachments 20-21

      ADC data item stats OK - attachment 22
       rates status quo ante
  
      Merger etc OK - attachment 23


14.58 DSSSD bias & leakage current OK - Grafana - attachment 24

      FEE64 temperatures OK - attachment 25

      ADC data item stats OK - attachment 26

      Merger etc OK - attachment 27

18.13 DSSSD bias & leakage current OK - Grafana - attachment 28

      FEE64 temperatures OK - attachment 29

      ADC data item stats OK - attachment 30

      Merger etc OK - attachment 31


20.30 Primary c, few kHz 170Er beam to S4 - upstream s/steel flange removed
      Start of stopping range calibration in the implantation stack - currently ion appear to be stopping in DSSSD#1 (upstream)


      Decay and implant events as function of time showing spill structure - attachment 32

      per DSSSD e_front versus e_back (HEC, 20MeV/channel) - attachment 33

      per DSSSD x versus y hit pattern (HEC) - attachment 34
       mapping to be configured but clear ions are mostly implanted in centre of DSSSD#1 

      2D colour/contour levels - attachment 35

      per DSSSD m_front versus m_back (HEC) - attachment 36

      per DSSSD per pixel HEC-HEC time (4.096us/channel) - attachment 37

23.44 aida02 has stopped running
      DAQ STOP
      aida02 rebooted

      Data transfer disabled

      DAQ RESET/SETUP 

      Synchronise ASIC clocks
      All system wide checks OK *except* FPGA errors

      Histograms & stats zero'd
      DAQ GO

      Merger restarted 

      Data transfer enabled
      DAQ GO







        


        
Entry  Sun Jun 2 01:45:05 2024, TD, Sunday 2 June 25x
02.41 DSSSD bias & leakage current - attachment 1

      FEE64 temps OK - attachment 2

      ADC data item stats - attachment 3

      per FEE64 Stat spectra - attachment 4


06.52 DSSSD bias & leakage current - attachment 5

      FEE64 temps OK - attachment 6

      ADC data item stats - attachment 7

      per FEE64 Stat spectra - attachment 8

12.13 DSSSD bias & leakage current - attachment 9

      FEE64 temps OK - attachment 10

      ADC data item stats - attachment 11

      per FEE64 Stat spectra - attachment 12


14.51 DSSSD bias & leakage current - attachments 13-14

      FEE64 temps OK - attachment 15

      ADC data item stats - attachment 16

      per FEE64 Stat spectra - attachment 17

21.49 DSSSD bias & leakage current - attachment 18

      FEE64 temps OK - attachment 19

      ADC data item stats - attachment 20

      per FEE64 Stat spectra - attachment 21


03.55 DSSSD bias & leakage current - attachment 22

      FEE64 temps OK - attachment 23

      ADC data item stats - attachment 24

      per FEE64 Stat spectra - attachment 25
Entry  Sun Jun 19 12:36:10 2022, OH, Sunday 19th June 1.jpg2.jpg3.jpg4.jpg220618_1753_stats_no_waveform.png
Yesterday the sum/inverter was found to be a considerable source of noise in the system.

Plugging it into an oscilloscope this morning it was clear there was a Vpp 17mv 60kHz sin wave in its signal - attachment 1 yellow trace

Moved the inverter to a crate in the messheute and did not see same behaviour. This time see Vpp 5mV and a ~265Hz wave that looks much less clean. - attachment 2 yellow

Put inverter back in CAEN crate in S4 and confirm similar behavior.

Move CAEN crate to new mains port, same behaviour

Move CAEN crate to AC voltage stabiliser, same behaviour

Remove module one by one from CAEN crate.
Removing BIAS supply - No change
Remove Pulser - No change
Remove right most MACB - No change
Remove second right most MACB - Signal disappears  - attachment 3 blue trace
Remove second right most MACB from the rack entirely
Put right most MACB back - Signal comes back - attachment 4 blue trace
Test MACB with and without HDMI connectors connected signal remains
Remove rightmost MACB from the rack - Signal disappears

Put pulser and bias back - Signal does not return
Signal also doesn't return when ramping up bias or running the pulser
Power DAQ up - No change in performance from yesterday with inverter removed.

MACB and Inverter don't share any voltages on NIM:
MACB +-6V and Gnd
Inverter +-24V and Gnd

CAEN Crate does not have any easily accessible monitoring points. Anyone have any ideas?

MACBs and inverter checked on another crate - no signal

Inverter put back in CAEN crate and no noise observed on scope.
With waveforms disabled am able to get <100kHz on all FEEs with the pulser connected

Tested adding bias filters - No change

15:30 Nic realised that one of the mains boxes is labelled measurement network so we connect the AIDA PSU to this via the ac stabiliser
      We get ok stats even with waveforms on at 0xa- attachment 5

      Waveforms still have the 100kHz oscillation in them though attachments 6 and 7

      Without waveforms we get similar performance to before the switch to the new mains - attachment 8

15:56 Move CAEN NIM crate across to this mains network
      300+kHz on all FEEs with waveforms on. Can recover waveform off performance though

16:01 Place CAEN NIM crate back onto platform mains but don't recover the ealier rates with waveform. Everything still around 300kHz.

17:43 Current best settings are waveforms off and we can run at 0xa on all FEEs - attachment 9
Entry  Sun Jun 19 00:22:24 2022, TD, Sunday 19 June 14x
01.20 

ASIC settings 2021Apr29-13-16-00
 *except* slow comparator 0x64 -> 0xa

All waveform ADCs disabled

BNC PB-5 pulser
 to p+n FEE64s only

all system wide checks OK *except* ADC calibration, WR decoder & FPGA timestamp errors - see attachments 1 & 2

DSSSD bias & leakage current OK - see attachments 3 & 4

FEE64 temps OK - see attachment 5

statistics OK - ADC data, pause, resume & correlation scaler data items
                data push, data flush, aida01, aida02 - see attachments 6-13

per FEE64 1.8.L spectra - see attachment 14
 aida01 pulser peak width 73 ch FWHM
Entry  Sun Apr 18 07:30:20 2021, DJ, TD, Sunday 18th April 08:00-12:00 7x

---- 09-26
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida10 failed
Calibration test result: Passed 9, Failed 3

If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module

---


         Base         Current     Difference
aida01 fault      0x7685 :      0x7686 :      1  
aida02 fault      0x941c :      0x941d :      1  
aida03 fault      0x7cd6 :      0x7cd7 :      1  
aida04 fault      0xb86c :      0xb86d :      1  
aida05 fault      0x1a52 :      0x1a59 :      7  
aida06 fault      0x4f3e :      0x4f45 :      7  
aida07 fault      0x3bcd :      0x3bfc :      47  
aida08 fault      0xc7c7 :      0xc7ce :      7  
aida09 fault      0xb33a :      0xb33b :      1  
White Rabbit error counter test result: Passed 3, Failed 9

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

--


         Base         Current     Difference
aida01 fault      0x7685 :      0x7686 :      1  
aida02 fault      0x941c :      0x941d :      1  
aida03 fault      0x7cd6 :      0x7cd7 :      1  
aida04 fault      0xb86c :      0xb86d :      1  
aida05 fault      0x1a52 :      0x1a59 :      7  
aida06 fault      0x4f3e :      0x4f45 :      7  
aida07 fault      0x3bcd :      0x3bfc :      47  
aida08 fault      0xc7c7 :      0xc7ce :      7  
aida09 fault      0xb33a :      0xb33b :      1  
White Rabbit error counter test result: Passed 3, Failed 9

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

-


         Base         Current     Difference
aida01 fault      0x7685 :      0x7686 :      1  
aida02 fault      0x941c :      0x941d :      1  
aida03 fault      0x7cd6 :      0x7cd7 :      1  
aida04 fault      0xb86c :      0xb86d :      1  
aida05 fault      0x1a52 :      0x1a59 :      7  
aida06 fault      0x4f3e :      0x4f45 :      7  
aida07 fault      0x3bcd :      0x3bfc :      47  
aida08 fault      0xc7c7 :      0xc7ce :      7  
aida09 fault      0xb33a :      0xb33b :      1  
White Rabbit error counter test result: Passed 3, Failed 9

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

 

 

08.45 TD resets baseline for WR and FPGA errors

 

ELOG V3.1.4-unknown