AIDA GELINA BRIKEN nToF CRIB ISOLDE CIRCE nTOFCapture DESPEC DTAS EDI_PSA 179Ta CARME StellarModelling DCF K40
  DESPEC, Page 11 of 36  ELOG logo
ID Date Author Subjectdown
  239   Sun Apr 18 11:18:21 2021 TDSunday 18 April 12.00-16.00
12.18 DAQ continues file NULL/R30_490

      ASIC settings 2019Dec19-16.19.51
      slow comparator 0x64 -> 0xa

      BNC PB-5
      amplitude 2V
      attenuation 1x
      decay time 1ms
      frequency 2Hz

12.22 all histograms zero'd

c. 12.30 219Rn setting

12.35 analysis of R30_490 - see attachment 1

13.15 system wide checks

Clock status test result: Passed 12, Failed 0
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1' 
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable

FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida10 failed
Calibration test result: Passed 9, Failed 3
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module

	 Base 		Current 	Difference
aida01 fault 	 0x7686 : 	 0x768a : 	 4  
aida02 fault 	 0x941d : 	 0x9421 : 	 4  
aida03 fault 	 0x7cd7 : 	 0x7cdb : 	 4  
aida04 fault 	 0xb86d : 	 0xb871 : 	 4  
aida05 fault 	 0x1a59 : 	 0x1a5f : 	 6  
aida06 fault 	 0x4f45 : 	 0x4f4b : 	 6  
aida07 fault 	 0x3bfc : 	 0x3c04 : 	 8  
aida08 fault 	 0xc7ce : 	 0xc7d4 : 	 6  
aida09 fault 	 0xb33b : 	 0xb33c : 	 1  
White Rabbit error counter test result: Passed 3, Failed 9
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

			 Base 		Current 		Difference
aida12 fault 	 0xa : 	 0xb : 	 1  
FPGA Timestamp error counter test result: Passed 11, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last


Returned 0 0 0 0 0 0 0 0 0 0 0 0  
Mem(KB) :	4	8	16	32	64	128	256	512	1k	2k	4k
aida01 :     23     10      3      1      1      2      1      3      3      3      6   : 36156
aida02 :      2     11      7      4      2      2      2      4      2      3      6   : 36048
aida03 :     21      8      6      4      5      4      3      3      2      3      6   : 36276
aida04 :     19     11     14      3      2      3      2      4      2      3      6   : 36324
aida05 :     25      4      5      4      4      2      1      3      2      4      6   : 37460
aida06 :     13      9     15      1      3      5      1      3      3      3      6   : 36812
aida07 :     19     10      6      1      3      3      2      2      4      3      6   : 37212
aida08 :     23      6      4      2      1      1      3      4      2      3      6   : 36044
aida09 :     11      5      5      1      2      4      3      3      2      3      6   : 35908
aida10 :     14      5      6      5      3      1      3      3      2      3      6   : 35744
aida11 :     16      5      3      0      3      3      1      4      2      3      6   : 35800
aida12 :     12     10      0      2      4      4      1      3      3      3      6   : 36544

 Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same

 FEE : aida01 =>   Options file size is 1025 	 Last changed Sat Apr 17 06:14:30 CEST 2021
 FEE : aida02 =>   Options file size is 1014 	 Last changed Fri Apr 16 00:56:20 CEST 2021
 FEE : aida03 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:04 CEST 2021
 FEE : aida04 =>   Options file size is 1025 	 Last changed Sat Apr 17 06:07:36 CEST 2021
 FEE : aida05 =>   Options file size is 1025 	 Last changed Fri Apr 16 00:53:25 CEST 2021
 FEE : aida06 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:04 CEST 2021
 FEE : aida07 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:04 CEST 2021
 FEE : aida08 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:04 CEST 2021
 FEE : aida09 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:05 CEST 2021
 FEE : aida10 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:06 CEST 2021
 FEE : aida11 =>   Options file size is 1014 	 Last changed Wed Apr 14 21:52:05 CEST 2021
 FEE : aida12 =>   Options file size is 1025 	 Last changed Wed Apr 14 21:58:54 CEST 2021

*no faults detected by ~/oh/OptionsCheck.py

13.10 Detector biases & leakage currents OK - see attachment 2
      Statistics - good events, adc data, disc data, info code 4 & 5, pause, resume, correlation info - attachments 3-12
      per FEE64 rate spectra - attachments 13-14
      1.8.L spectra - attachments 15-18

       pulser peak widths
       1 80
       2 113
       3 308
       4 105
       5 50
       6 91
       7 61
       8 90
       9 200
       10 133
       11 187
       12 107
  
      1.8.H spectra - attachments 19-20
      1.8.W spectra - attachments 21-22
      ucesb - attachment 23
       implant rates DSSSD#1-3 to c. 1kHz
      grafana - AIDA DSSSD leakage currents for previous 2 days - attachment 24
      merger & merger/tape server/mbs data tranfser - attachment 25
       no recent merger warning/error messages

13.30 225At setting
Attachment 1: R30_490
 *** TDR format 3.3.0 analyser - TD - January 2019
 *** ERROR: READ I/O error:       5002
                   blocks:      32000
          ADC data format:  256516272 ( 1223332.2 Hz)
        Other data format:    5403728 (   25770.5 Hz)
 Sample trace data format:          0 (       0.0 Hz)
         Undefined format:          0 (       0.0 Hz)
   Other data format type:      PAUSE:          0 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:      32781 (     156.3 Hz)
                              WR48-63:      32781 (     156.3 Hz)
                           FEE64 disc:     828389 (    3950.6 Hz)
                             MBS info:    4509777 (   21507.2 Hz)
                           Other info:          0 (       0.0 Hz)

   ADC data range bit set:     263493 (    1256.6 Hz)

                Timewarps:        ADC:          0 (       0.0 Hz)
                                PAUSE:          0 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:          0 (       0.0 Hz)
                              WR48-63:          0 (       0.0 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:          0 (       0.0 Hz)
                            Undefined:          0 (       0.0 Hz)
                         Sample trace:          0 (       0.0 Hz)

 *** Timestamp elapsed time:      209.687 s
 FEE  elapsed dead time(s) elapsed idle time(s)
  1                0.000               76.433
  2                0.000               87.604
  3                0.000               78.795
  4                0.000              195.707
  5                0.000              158.854
  6                0.000              193.755
  7                0.000              152.656
  8                0.000               23.126
  9                0.000               23.243
 10                0.000               11.178
 11                0.000              108.457
 12                0.000                0.000
 13                0.000                0.000
 14                0.000                0.000
 15                0.000                0.000
 16                0.000                0.000
 17                0.000                0.000
 18                0.000                0.000
 19                0.000                0.000
 20                0.000                0.000
 21                0.000                0.000
 22                0.000                0.000
 23                0.000                0.000
 24                0.000                0.000
 25                0.000                0.000
 26                0.000                0.000
 27                0.000                0.000
 28                0.000                0.000
 29                0.000                0.000
 30                0.000                0.000
 31                0.000                0.000
 32                0.000                0.000

 *** Statistics
 FEE  ADC Data Other Data     Sample  Undefined      Pause     Resume    SYNC100    WR48-63       Disc        MBS      Other   HEC Data
  0    9851554      72944          0          0          0          0       1261       1261      69168       1254          0      26448
  1   21158309      48956          0          0          0          0       2701       2701      43554          0          0      16374
  2   17027137    1992411          0          0          0          0       2376       2376     156273    1831386          0      25239
  3   18790255    1894647          0          0          0          0       2567       2567      58490    1831023          0      23746
  4    2269151     296174          0          0          0          0        339        339      74660     220836          0      26646
  5    8373135     260125          0          0          0          0       1152       1152      43783     214038          0      16230
  6    4256540     286725          0          0          0          0        532        532      81589     204072          0      25586
  7    9526319     267328          0          0          0          0       1246       1246      57668     207168          0      22643
  8   54059298      76554          0          0          0          0       6619       6619      63316          0          0      22746
  9   49446691      53505          0          0          0          0       6256       6256      40993          0          0      14045
 10   45199495      92635          0          0          0          0       5711       5711      81213          0          0      23246
 11   16558388      61724          0          0          0          0       2021       2021      57682          0          0      20544
 12          0          0          0          0          0          0          0          0          0          0          0          0
 13          0          0          0          0          0          0          0          0          0          0          0          0
 14          0          0          0          0          0          0          0          0          0          0          0          0
 15          0          0          0          0          0          0          0          0          0          0          0          0
 16          0          0          0          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0          0          0          0

 *** Timewarps
 FEE       ADC      Pause     Resume    SYNC100    WR48-63       Disc        MBS  Undefined    Samples
  0          0          0          0          0          0          0          0          0          0
  1          0          0          0          0          0          0          0          0          0
  2          0          0          0          0          0          0          0          0          0
  3          0          0          0          0          0          0          0          0          0
  4          0          0          0          0          0          0          0          0          0
  5          0          0          0          0          0          0          0          0          0
  6          0          0          0          0          0          0          0          0          0
  7          0          0          0          0          0          0          0          0          0
  8          0          0          0          0          0          0          0          0          0
  9          0          0          0          0          0          0          0          0          0
 10          0          0          0          0          0          0          0          0          0
 11          0          0          0          0          0          0          0          0          0
 12          0          0          0          0          0          0          0          0          0
 13          0          0          0          0          0          0          0          0          0
 14          0          0          0          0          0          0          0          0          0
 15          0          0          0          0          0          0          0          0          0
 16          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0

 *** Program elapsed time:44645.059s (    0.717 blocks/s,   0.045 Mb/s)
Attachment 2: 50.png
50.png
Attachment 3: Screenshot_2021-04-18_Statistics_aidas-gsi(1).png
Screenshot_2021-04-18_Statistics_aidas-gsi(1).png
Attachment 4: Screenshot_2021-04-18_Temperature_and_status_scan_aidas-gsi(2).png
Screenshot_2021-04-18_Temperature_and_status_scan_aidas-gsi(2).png
Attachment 5: Screenshot_2021-04-18_Statistics_aidas-gsi(2).png
Screenshot_2021-04-18_Statistics_aidas-gsi(2).png
Attachment 6: Screenshot_2021-04-18_Statistics_aidas-gsi(3).png
Screenshot_2021-04-18_Statistics_aidas-gsi(3).png
Attachment 7: Screenshot_2021-04-18_Statistics_aidas-gsi(4).png
Screenshot_2021-04-18_Statistics_aidas-gsi(4).png
Attachment 8: Screenshot_2021-04-18_Statistics_aidas-gsi(5).png
Screenshot_2021-04-18_Statistics_aidas-gsi(5).png
Attachment 9: Screenshot_2021-04-18_Statistics_aidas-gsi(6).png
Screenshot_2021-04-18_Statistics_aidas-gsi(6).png
Attachment 10: Screenshot_2021-04-18_Statistics_aidas-gsi(7).png
Screenshot_2021-04-18_Statistics_aidas-gsi(7).png
Attachment 11: Screenshot_2021-04-18_Statistics_aidas-gsi(8).png
Screenshot_2021-04-18_Statistics_aidas-gsi(8).png
Attachment 12: Screenshot_2021-04-18_Statistics_aidas-gsi(9).png
Screenshot_2021-04-18_Statistics_aidas-gsi(9).png
Attachment 13: Screenshot_2021-04-18_Spectrum_Browser_aidas-gsi.png
Screenshot_2021-04-18_Spectrum_Browser_aidas-gsi.png
Attachment 14: Screenshot_2021-04-18_Spectrum_Browser_aidas-gsi(1).png
Screenshot_2021-04-18_Spectrum_Browser_aidas-gsi(1).png
Attachment 15: Screenshot_2021-04-18_Spectrum_Browser_aidas-gsi(2).png
Screenshot_2021-04-18_Spectrum_Browser_aidas-gsi(2).png
Attachment 16: Screenshot_2021-04-18_Spectrum_Browser_aidas-gsi(3).png
Screenshot_2021-04-18_Spectrum_Browser_aidas-gsi(3).png
Attachment 17: Screenshot_2021-04-18_Spectrum_Browser_aidas-gsi(4).png
Screenshot_2021-04-18_Spectrum_Browser_aidas-gsi(4).png
Attachment 18: Screenshot_2021-04-18_Spectrum_Browser_aidas-gsi(5).png
Screenshot_2021-04-18_Spectrum_Browser_aidas-gsi(5).png
Attachment 19: Screenshot_2021-04-18_Spectrum_Browser_aidas-gsi(6).png
Screenshot_2021-04-18_Spectrum_Browser_aidas-gsi(6).png
Attachment 20: Screenshot_2021-04-18_Spectrum_Browser_aidas-gsi(7).png
Screenshot_2021-04-18_Spectrum_Browser_aidas-gsi(7).png
Attachment 21: Screenshot_2021-04-18_Spectrum_Browser_aidas-gsi(8).png
Screenshot_2021-04-18_Spectrum_Browser_aidas-gsi(8).png
Attachment 22: Screenshot_2021-04-18_Spectrum_Browser_aidas-gsi(9).png
Screenshot_2021-04-18_Spectrum_Browser_aidas-gsi(9).png
Attachment 23: Screenshot_2021-04-18_ucesb.png
Screenshot_2021-04-18_ucesb.png
Attachment 24: Screenshot_2021-04-18_AIDA_Alerting_-_Grafana.png
Screenshot_2021-04-18_AIDA_Alerting_-_Grafana.png
Attachment 25: 51.png
51.png
  665   Sun Jun 16 03:11:27 2024 TDSunday 16 June
04.08 DSSSD bias & leakage current OK - attachment 1

      FEE64 temperatures OK - attachment 2

      ADC data item stats OK - attachment 3

      per FEE64 Rate spectra - attachment 4


07.52 DSSSD bias & leakage current OK - attachment 5

      FEE64 temperatures OK - attachment 6

      ADC data item stats OK - attachment 7

      per FEE64 Rate spectra - attachment 8

      per p+n FEE64 1.8.L spectra - attachments 9-10
       aida09 pulser peak width 59 ch FWHM


15.01 DSSSD bias & leakage current OK - attachment 11

      FEE64 temperatures OK - attachment 12

      ADC data item stats OK - attachment 13

      per p+n FEE64 1.8.L spectra - attachments 14-15
       aida09 pulser peak width 59 ch FWHM
       aida05 pulser peak width 68 ch FWHM
       aida12 pulser peak width 71 ch FWHM


      per FEE64 Rate spectra - attachment 16

      Merger, TapeSever etc - attachment 17
       no data transfer to MBS - MBS down?

15.06 analysis data file R9_747 - attachment 18
       max dead time 4.3% aida08, all other FEE64s < 2%
       ADC data rate 1.667M, HEC data rate 2.3k 


19.12 DSSSD bias & leakage current OK - attachment 19

      FEE64 temperatures OK - attachment 20

      ADC data item stats OK - attachment 21

      per FEE64 Rate spectra - attachment 22

23.34 DSSSD bias & leakage current OK - attachment 23

      FEE64 temperatures OK - attachment 24

      ADC data item stats OK - attachment 25

      per FEE64 Rate spectra - attachment 26
Attachment 1: Screenshot_from_2024-06-16_04-10-35.png
Screenshot_from_2024-06-16_04-10-35.png
Attachment 2: Screenshot_from_2024-06-16_04-09-07.png
Screenshot_from_2024-06-16_04-09-07.png
Attachment 3: Screenshot_from_2024-06-16_04-09-41.png
Screenshot_from_2024-06-16_04-09-41.png
Attachment 4: Screenshot_from_2024-06-16_04-10-21.png
Screenshot_from_2024-06-16_04-10-21.png
Attachment 5: Screenshot_from_2024-06-16_07-52-49.png
Screenshot_from_2024-06-16_07-52-49.png
Attachment 6: Screenshot_from_2024-06-16_07-53-20.png
Screenshot_from_2024-06-16_07-53-20.png
Attachment 7: Screenshot_from_2024-06-16_07-53-47.png
Screenshot_from_2024-06-16_07-53-47.png
Attachment 8: Screenshot_from_2024-06-16_07-54-42.png
Screenshot_from_2024-06-16_07-54-42.png
Attachment 9: Screenshot_from_2024-06-16_07-59-28.png
Screenshot_from_2024-06-16_07-59-28.png
Attachment 10: Screenshot_from_2024-06-16_08-01-35.png
Screenshot_from_2024-06-16_08-01-35.png
Attachment 11: Screenshot_from_2024-06-16_15-01-23.png
Screenshot_from_2024-06-16_15-01-23.png
Attachment 12: Screenshot_from_2024-06-16_15-01-46.png
Screenshot_from_2024-06-16_15-01-46.png
Attachment 13: Screenshot_from_2024-06-16_15-02-12.png
Screenshot_from_2024-06-16_15-02-12.png
Attachment 14: Screenshot_from_2024-06-16_15-03-05.png
Screenshot_from_2024-06-16_15-03-05.png
Attachment 15: Screenshot_from_2024-06-16_15-06-14.png
Screenshot_from_2024-06-16_15-06-14.png
Attachment 16: Screenshot_from_2024-06-16_15-07-06.png
Screenshot_from_2024-06-16_15-07-06.png
Attachment 17: Screenshot_from_2024-06-16_15-08-02.png
Screenshot_from_2024-06-16_15-08-02.png
Attachment 18: R9_747
 *** TDR format 3.3.0 analyser - TD - May 2021
 *** ERROR: READ I/O error:       5002
                   blocks:      32000
          ADC data format:  261839349 ( 1667447.6 Hz)
        Other data format:      80651 (     513.6 Hz)
 Sample trace data format:          0 (       0.0 Hz)
         Undefined format:          0 (       0.0 Hz)
   Other data format type:      PAUSE:        683 (       4.3 Hz)
                               RESUME:        683 (       4.3 Hz)
                              SYNC100:      32585 (     207.5 Hz)
                              WR48-63:      32585 (     207.5 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:      14115 (      89.9 Hz)
                           Other info:          0 (       0.0 Hz)

   ADC data range bit set:     357749 (    2278.2 Hz)

                Timewarps:        ADC:          0 (       0.0 Hz)
                                PAUSE:          0 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:          0 (       0.0 Hz)
                              WR48-63:          0 (       0.0 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:          0 (       0.0 Hz)
                            Undefined:          0 (       0.0 Hz)
                         Sample trace:          0 (       0.0 Hz)

 *** Timestamp elapsed time:      157.030 s
 FEE  elapsed dead time(s) elapsed idle time(s)
  0                0.474                0.000
  1                0.000                0.000
  2                0.114                0.000
  3                0.086                0.000
  4                0.007                0.000
  5                0.165                0.000
  6                0.000                0.000
  7                6.815                0.000
  8                0.000                0.000
  9                3.068                0.000
 10                0.526                0.000
 11                0.000                0.000
 12                0.000                0.000
 13                0.247                0.000
 14                0.002                0.000
 15                1.373                0.000
 16                0.000                0.000
 17                0.000                0.000
 18                0.000                0.000
 19                0.000                0.000
 20                0.000                0.000
 21                0.000                0.000
 22                0.000                0.000
 23                0.000                0.000
 24                0.000                0.000
 25                0.000                0.000
 26                0.000                0.000
 27                0.000                0.000
 28                0.000                0.000
 29                0.000                0.000
 30                0.000                0.000
 31                0.000                0.000
 32                0.000                0.000

 *** Statistics
 FEE  ADC Data Other Data     Sample  Undefined      Pause     Resume    SYNC100    WR48-63       Disc        MBS      Other   HEC Data
  0   17385482       9389          0          0         56         56       2291       2291          0       4695          0          0
  1    4359471       1110          0          0          0          0        555        555          0          0          0          0
  2   16890768       4312          0          0         14         14       2142       2142          0          0          0          0
  3   16799432       4226          0          0         17         17       2096       2096          0          0          0          0
  4    5895210       1414          0          0          2          2        705        705          0          0          0          0
  5   15605735       4074          0          0         27         27       2010       2010          0          0          0          0
  6    2976504      10096          0          0          0          0        338        338          0       9420          0          0
  7   48753528      12540          0          0        223        223       6047       6047          0          0          0     357748
  8    5419325       1414          0          0          0          0        707        707          0          0          0          0
  9   41464320      10520          0          0        151        151       5109       5109          0          0          0          0
 10   21531781       5458          0          0         66         66       2663       2663          0          0          0          1
 11    3242993        744          0          0          0          0        372        372          0          0          0          0
 12    3988909        970          0          0          0          0        485        485          0          0          0          0
 13   16192913       4074          0          0         38         38       1999       1999          0          0          0          0
 14    4620379       1122          0          0          1          1        560        560          0          0          0          0
 15   36712599       9188          0          0         88         88       4506       4506          0          0          0          0
 16          0          0          0          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0          0          0          0

 *** Timewarps
 FEE       ADC      Pause     Resume    SYNC100    WR48-63       Disc        MBS  Undefined    Samples
  0          0          0          0          0          0          0          0          0          0
  1          0          0          0          0          0          0          0          0          0
  2          0          0          0          0          0          0          0          0          0
  3          0          0          0          0          0          0          0          0          0
  4          0          0          0          0          0          0          0          0          0
  5          0          0          0          0          0          0          0          0          0
  6          0          0          0          0          0          0          0          0          0
  7          0          0          0          0          0          0          0          0          0
  8          0          0          0          0          0          0          0          0          0
  9          0          0          0          0          0          0          0          0          0
 10          0          0          0          0          0          0          0          0          0
 11          0          0          0          0          0          0          0          0          0
 12          0          0          0          0          0          0          0          0          0
 13          0          0          0          0          0          0          0          0          0
 14          0          0          0          0          0          0          0          0          0
 15          0          0          0          0          0          0          0          0          0
 16          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0

 *** Program elapsed time:   63.340s (  505.211 blocks/s,  31.576 Mb/s)
Attachment 19: Screenshot_from_2024-06-16_19-11-54.png
Screenshot_from_2024-06-16_19-11-54.png
Attachment 20: Screenshot_from_2024-06-16_19-12-22.png
Screenshot_from_2024-06-16_19-12-22.png
Attachment 21: Screenshot_from_2024-06-16_19-14-40.png
Screenshot_from_2024-06-16_19-14-40.png
Attachment 22: Screenshot_from_2024-06-16_19-15-10.png
Screenshot_from_2024-06-16_19-15-10.png
Attachment 23: Screenshot_from_2024-06-16_23-34-39.png
Screenshot_from_2024-06-16_23-34-39.png
Attachment 24: Screenshot_from_2024-06-16_23-35-06.png
Screenshot_from_2024-06-16_23-35-06.png
Attachment 25: Screenshot_from_2024-06-16_23-35-37.png
Screenshot_from_2024-06-16_23-35-37.png
Attachment 26: Screenshot_from_2024-06-16_23-36-04.png
Screenshot_from_2024-06-16_23-36-04.png
  457   Sun May 15 15:51:49 2022 MLSunday 15th May 16:00-0:00

Status at 16:45 (CET)

The expreiment continues to run smoothly.

AIDA Stats look ok - Attachment 1

AIDA Temperature ok - Attachment 2

AIDA Leakage current: ok - Attachment 3

System wide check:

Clock: 13 passed, 1 failed (aida09)

ADC Calibration: 9 passed, 5 fialed (aida2,6,9,10,13)

White rabbit decoder: 14 passed, 0 failed.

FPGA timestamp: 14 passed, 0 failed.

 

Status at 18:45 (CET):

All rates ok on ucesb.

AIDA Stats ok - Attachment 4

AIDA Temperature ok - Attachment 5

AIDA Leakage current: increase slightly - Attachment 6

System wide check: No changes, same as above

 

Status at 20:45 (CET):

All rates ok on ucesb.

AIDA Stats ok - Attachment 7

AIDA Temperature ok - Attachment 8

AIDA Leakage current: increase slightly - Attachment 9

System wide check: No changes, same as above

 

Status at 22:45 (CET):

All rates ok on ucesb.

AIDA Stats ok - Attachment 10

AIDA Temperature ok - Attachment 11

AIDA Leakage current: increase slightly - Attachment 12

System wide check: No changes, same as above

Attachment 1: Stats_Screenshot_from_2022-05-15_16-49-06.png
Stats_Screenshot_from_2022-05-15_16-49-06.png
Attachment 2: Temp_Screenshot_from_2022-05-15_16-48-31.png
Temp_Screenshot_from_2022-05-15_16-48-31.png
Attachment 3: HV_LC_Screenshot_from_2022-05-15_16-47-44.png
HV_LC_Screenshot_from_2022-05-15_16-47-44.png
Attachment 4: Stats_Screenshot_from_2022-05-15_18-43-16.png
Stats_Screenshot_from_2022-05-15_18-43-16.png
Attachment 5: Temp_Screenshot_from_2022-05-15_18-44-06.png
Temp_Screenshot_from_2022-05-15_18-44-06.png
Attachment 6: HV_LC_Screenshot_from_2022-05-15_18-42-23.png
HV_LC_Screenshot_from_2022-05-15_18-42-23.png
Attachment 7: Stats_Screenshot_from_2022-05-15_20-42-03.png
Stats_Screenshot_from_2022-05-15_20-42-03.png
Attachment 8: Temp_Screenshot_from_2022-05-15_20-42-42.png
Temp_Screenshot_from_2022-05-15_20-42-42.png
Attachment 9: HV_LC_Screenshot_from_2022-05-15_20-36-28.png
HV_LC_Screenshot_from_2022-05-15_20-36-28.png
Attachment 10: Stats_Screenshot_from_2022-05-15_22-43-37.png
Stats_Screenshot_from_2022-05-15_22-43-37.png
Attachment 11: Temp_Screenshot_from_2022-05-15_22-44-20.png
Temp_Screenshot_from_2022-05-15_22-44-20.png
Attachment 12: HV_LC_Screenshot_from_2022-05-15_22-42-36.png
HV_LC_Screenshot_from_2022-05-15_22-42-36.png
  456   Sun May 15 07:40:52 2022 OHSunday 15th May 08:00-16:00
08:40 FEE Temperatures ok - attachment 1
      FEE statistics - Attachment 2
      Bias and leakage currents ok - attachment 3
      ASIC check ok
      All system wide checks ok - Except aida09 fails clock check with bit 6

11:00 FEE statistics - attachment 4
      FEE temperatures all ok - attachment 5
      BIAS and leakage currents - attachment 6
      ASIC check ok
      All system wide checks as before
      Currently on file R4_1030
      13.1 MB/s to disk
      4.8 million items per second
      
13:13 FEE statistics - attachment 7
      FEE Temperatures all ok - attahcment 8
      Bias and leakage currents ok - attachment 9
      ASIC check ok
      All system wide checks as before
      Currently on file R4_1078
      13.1 MB/s to disk
      4.8 million items per second

15:35 Analysis of R4_1128 - attachment 10
Attachment 1: Screenshot_from_2022-05-15_08-38-04.png
Screenshot_from_2022-05-15_08-38-04.png
Attachment 2: Screenshot_from_2022-05-15_08-38-34.png
Screenshot_from_2022-05-15_08-38-34.png
Attachment 3: Screenshot_from_2022-05-15_08-39-12.png
Screenshot_from_2022-05-15_08-39-12.png
Attachment 4: Screenshot_from_2022-05-15_11-00-32.png
Screenshot_from_2022-05-15_11-00-32.png
Attachment 5: Screenshot_from_2022-05-15_11-00-50.png
Screenshot_from_2022-05-15_11-00-50.png
Attachment 6: Screenshot_from_2022-05-15_11-01-02.png
Screenshot_from_2022-05-15_11-01-02.png
Attachment 7: Screenshot_from_2022-05-15_13-13-50.png
Screenshot_from_2022-05-15_13-13-50.png
Attachment 8: Screenshot_from_2022-05-15_13-14-12.png
Screenshot_from_2022-05-15_13-14-12.png
Attachment 9: Screenshot_from_2022-05-15_13-14-31.png
Screenshot_from_2022-05-15_13-14-31.png
Attachment 10: R4_1128
 *** TDR format 3.3.0 analyser - TD - May 2021
 *** ERROR: READ I/O error:       5002
                   blocks:      32000
          ADC data format:  259195371 ( 1637875.8 Hz)
        Other data format:    2724629 (   17217.1 Hz)
 Sample trace data format:          0 (       0.0 Hz)
         Undefined format:          0 (       0.0 Hz)
   Other data format type:      PAUSE:       2455 (      15.5 Hz)
                               RESUME:       2456 (      15.5 Hz)
                              SYNC100:      32590 (     205.9 Hz)
                              WR48-63:      32590 (     205.9 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:    2654538 (   16774.2 Hz)
                           Other info:          0 (       0.0 Hz)

   ADC data range bit set:      38026 (     240.3 Hz)

                Timewarps:        ADC:          0 (       0.0 Hz)
                                PAUSE:          0 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:          0 (       0.0 Hz)
                              WR48-63:          0 (       0.0 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:          0 (       0.0 Hz)
                            Undefined:          0 (       0.0 Hz)
                         Sample trace:          0 (       0.0 Hz)

 *** Timestamp elapsed time:      158.251 s
 FEE  elapsed dead time(s) elapsed idle time(s)
  0                0.000                0.000
  1                4.735                0.000
  2                0.000                0.000
  3               16.674                0.000
  4                0.000                0.000
  5               77.034                0.000
  6                0.000                0.000
  7               33.677                0.000
  8                0.000                0.000
  9                0.000                0.000
 10                0.000               12.872
 11               17.247                0.000
 12                0.000                0.000
 13                0.000                8.200
 14                0.000                0.000
 15                0.000                0.000
 16                0.000                0.000
 17                0.000                0.000
 18                0.000                0.000
 19                0.000                0.000
 20                0.000                0.000
 21                0.000                0.000
 22                0.000                0.000
 23                0.000                0.000
 24                0.000                0.000
 25                0.000                0.000
 26                0.000                0.000
 27                0.000                0.000
 28                0.000                0.000
 29                0.000                0.000
 30                0.000                0.000
 31                0.000                0.000
 32                0.000                0.000

 *** Statistics
 FEE  ADC Data Other Data     Sample  Undefined      Pause     Resume    SYNC100    WR48-63       Disc        MBS      Other   HEC Data
  0    7464928       2802          0          0          0          0        927        927          0        948          0        661
  1   38695355      10162          0          0        254        254       4827       4827          0          0          0      32843
  2    6890959    1397581          0          0          0          0       1028       1028          0    1395525          0        633
  3   41303989    1261446          0          0        398        398       5227       5227          0    1250196          0        331
  4    4066993       1070          0          0          0          0        535        535          0          0          0        242
  5   37291979      10883          0          0        737        738       4704       4704          0          0          0        222
  6   12390385       7719          0          0          0          0       1626       1626          0       4467          0        394
  7   42901955      15088          0          0        626        626       5217       5217          0       3402          0        167
  8   10872099       2752          0          0          0          0       1376       1376          0          0          0        592
  9    4061050       1014          0          0          0          0        507        507          0          0          0        432
 10    2113468        548          0          0          0          0        274        274          0          0          0        291
 11   41561869      11172          0          0        440        440       5146       5146          0          0          0        468
 12    6798748       1722          0          0          0          0        861        861          0          0          0        436
 13    2781594        670          0          0          0          0        335        335          0          0          0        314
 14          0          0          0          0          0          0          0          0          0          0          0          0
 15          0          0          0          0          0          0          0          0          0          0          0          0
 16          0          0          0          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0          0          0          0

 *** Timewarps
 FEE       ADC      Pause     Resume    SYNC100    WR48-63       Disc        MBS  Undefined    Samples
  0          0          0          0          0          0          0          0          0          0
  1          0          0          0          0          0          0          0          0          0
  2          0          0          0          0          0          0          0          0          0
  3          0          0          0          0          0          0          0          0          0
  4          0          0          0          0          0          0          0          0          0
  5          0          0          0          0          0          0          0          0          0
  6          0          0          0          0          0          0          0          0          0
  7          0          0          0          0          0          0          0          0          0
  8          0          0          0          0          0          0          0          0          0
  9          0          0          0          0          0          0          0          0          0
 10          0          0          0          0          0          0          0          0          0
 11          0          0          0          0          0          0          0          0          0
 12          0          0          0          0          0          0          0          0          0
 13          0          0          0          0          0          0          0          0          0
 14          0          0          0          0          0          0          0          0          0
 15          0          0          0          0          0          0          0          0          0
 16          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0

 *** Program elapsed time:   33.348s (  959.588 blocks/s,  59.974 Mb/s)
  152   Sat Mar 14 23:09:28 2020 MB, SJ, MRSunday 15th March 0:00-8:00

AIDA ELog filled out by DESPEC night shift members Matthew Brunet (MB), Shaheen Jazrawi (SJ), and Matthias Rudiger (MR).

0:05-0:08 Beam operators altering settings, beam off/sporadic during this period.

0:46
    All system wide checks passed
    Leak currents lower than previous measurement - attachment #1
    Implants look as previously - attachment #2
    Stats about where they have been - attachment #3
    FEE temperatures fine - attachment #4
    Merger fine, 3 Mega items / sec
    TapeServer writing about 25 MB/s

 

2:46

      All system wide checks passed
     Leak currents similar to previous - attachment #5 
     Implants look as previously - attachment #6
     Stats about where they have been - attachment #7
     FEE temperatures fine - attachment #8
     Merger fine, 3 Mega items / sec
     TapeServer writing about 27 MB/s
 

4:49

      All system wide checks passed
     Leak currents similar to previous - attachment #9
     Implants look as previously - attachment #10
     Stats good - attachment #11
     FEE temperatures fine - attachment #12
     Merger fine, 3 Mega items / sec
     TapeServer writing about 27 MB/s

6:49

      All system wide checks passed
     Leak currents no significant change - attachment #13
     Implants look as previously - attachment #14
     Stats good - attachment #15
     FEE temperatures fine - attachment #16
     Merger fine, 3 Mega items / sec
     TapeServer writing about 25 MB/s

Attachment 1: Matt-15-BiasesA.png
Matt-15-BiasesA.png
Attachment 2: Matt-15-HistosA.png
Matt-15-HistosA.png
Attachment 3: Matt-15-StatsA.png
Matt-15-StatsA.png
Attachment 4: Matt-15-TempsA-.png
Matt-15-TempsA-.png
Attachment 5: Matt-15-BiasesB.png
Matt-15-BiasesB.png
Attachment 6: Matt-15-HistosB.png
Matt-15-HistosB.png
Attachment 7: Matt-15-StatsB.png
Matt-15-StatsB.png
Attachment 8: Matt-15-TempsB.png
Matt-15-TempsB.png
Attachment 9: Matt-15-BiasesC.png
Matt-15-BiasesC.png
Attachment 10: Matt-15-HistosC.png
Matt-15-HistosC.png
Attachment 11: Matt-15-StatsC.png
Matt-15-StatsC.png
Attachment 12: Matt-15-TempsC.png
Matt-15-TempsC.png
Attachment 13: Matt-15-BiasesD.png
Matt-15-BiasesD.png
Attachment 14: Matt-15-HistosD.png
Matt-15-HistosD.png
Attachment 15: Matt-15-StatsD.png
Matt-15-StatsD.png
Attachment 16: Matt-15-TempsD.png
Matt-15-TempsD.png
  153   Sun Mar 15 06:45:00 2020 CA, LS, CGSunday 15th March 08:00 - 24:00
ASIC settings 2019Oct31-13.24.23
     slow comparator 0xa

BNC PB-5 pulser
     amplitude 1.0V , attenuator x1
     frequency 2Hz
     decay time 1ms

07:46 all system wide checks ok

      good event statistics ok - attachment 1
  
      FEE64 temperatures ok - attachment 2

      detector bias/leakage currents ok - attachment 3

08:48 merger ok ~ 3 million data items/s

      TapeServer ok ~ 27 MB/s

      data forwarding to MBS ok

08:50 AIDA writing to file SecondDrive/TapeData/S480/R12_1524

09:56 all system wide checks ok

      good event statistics ok - attachment 4

      detector bias/leakage currents ok - attachment 5
  
      FEE64 temperatures ok - attachment 6

12:09 all system wide checks ok


12:13 all system wide checks ok

      good event statistics ok - attachment 7
  
      FEE64 temperatures ok - attachment 8

      detector bias/leakage currents ok, but ramping up - attachment 9


13:32 merger ok ~ 3 million data items/s

      TapeServer ok ~ 23 MB/s

      data forwarding to MBS ok

      Beam seems to be off

13:33 AIDA writing to file R12_1743

*update* checked DESPEC S480 Elog, issues with UNILAC, beam lost at 13:29

13:55 beam back

13:57 AIDA writing to file R12_1759

14:35 all system wide checks ok

      FEE64 temperatures ok - attachment 10

      good event statistics ok - attachment 11
  
      detector bias/leakage currents ok, but ramping up - attachment 12

15:18 merger ok ~ 2.7 million data items/s

      TapeServer ok ~ 27 MB/s

      data forwarding to MBS ok

      AIDA writing to file R12_1283

16:00 1.8.H high energy spectra - attachments 13 & 14

      hit-rate spectra - attachment 15
 
16.22 all system wide checks okay 
      FEE temperatures okay (attachment 16)
      leakage currents okay and recorded to spreadsheet (attachment 17)
      good event stats okay (attachment 18)
      merger running at 3M items/sec
      tape service running at 26MB/sec

18.00 began compressing R7 files with command: nice -n 10 gzip -v R7_* &
      
      all system wide checks okay 
      FEE temperatures okay (attachment 19)
      leakage currents okay and recorded to spreadsheet (attachment 20)
      good event stats okay (attachment 21)
      merger running at 3M items/sec
      tape service running at 27MB/sec    

20.10 all system wide checks okay 
      FEE temperatures okay (attachment 22)
      leakage currents okay and recorded to spreadsheet, detectors 1 and 2 still rising (attachment 23)
      good event stats okay (attachment 24)
      merger running at 3M items/sec
      tape service running at 25MB/sec

      rates histogram (attachment 25)
      low energy histograms (attachment 26 and 27)
      high energy histograms (attachments 28 and 29)

      
22.07 all system wide checks okay 
      FEE temperatures okay (attachment 30)
      leakage currents okay and recorded to spreadsheet (attachment 31)
      good event stats okay (attachment 32)
      merger running at 3M items/sec
      tape service running at 25MB/sec
      currently on R12_2097

23.56 all system wide checks okay 
      FEE temperatures okay (attachment 33)
      leakage currents okay and recorded to spreadsheet (attachment 34)
      good event stats okay (attachment 35)
      merger running at 3M items/sec
      tape service running at 26MB/sec
      
finishing up now. night shift group have been informed how to check up on AIDA, there is a sheet on the table with relevant information
      
Attachment 1: 200315_0749_stats.png
200315_0749_stats.png
Attachment 2: 200315_0749_temp.png
200315_0749_temp.png
Attachment 3: 200315_0750_bias.png
200315_0750_bias.png
Attachment 4: 200315_0956_stats.png
200315_0956_stats.png
Attachment 5: 200315_0957_bias.png
200315_0957_bias.png
Attachment 6: 200315_0957_temp.png
200315_0957_temp.png
Attachment 7: 200315_1210_stats.png
200315_1210_stats.png
Attachment 8: 200315_1211_temp.png
200315_1211_temp.png
Attachment 9: 200315_1212_bias.png
200315_1212_bias.png
Attachment 10: corrigan_temperature.png
corrigan_temperature.png
Attachment 11: corrigan_stats.png
corrigan_stats.png
Attachment 12: corrigan_bias.png
corrigan_bias.png
Attachment 13: corriganhe1.png
corriganhe1.png
Attachment 14: corriganhe2.png
corriganhe2.png
Attachment 15: 200315_1553_rates.png
200315_1553_rates.png
Attachment 16: 200315_1619_feetemps.png
200315_1619_feetemps.png
Attachment 17: 200315_1620_leakagecurrents.png
200315_1620_leakagecurrents.png
Attachment 18: 200315_1620_goodevents.png
200315_1620_goodevents.png
Attachment 19: 200315_1758_feetemps.png
200315_1758_feetemps.png
Attachment 20: 200315_1759_leakagecurrents.png
200315_1759_leakagecurrents.png
Attachment 21: 200315_1758_goodevents.png
200315_1758_goodevents.png
Attachment 22: 200315_2004_feetemps.png
200315_2004_feetemps.png
Attachment 23: 200315_2007_leakagecurrents#.png
200315_2007_leakagecurrents#.png
Attachment 24: 200315_2006_goodevents.png
200315_2006_goodevents.png
Attachment 25: 200315_2009_rates.png
200315_2009_rates.png
Attachment 26: 200315_2010_lowodd.png
200315_2010_lowodd.png
Attachment 27: 200315_2011_loweven.png
200315_2011_loweven.png
Attachment 28: 200315_2012_highodd.png
200315_2012_highodd.png
Attachment 29: 200315_2014_higheven.png
200315_2014_higheven.png
Attachment 30: 200315_2202_feetemps.png
200315_2202_feetemps.png
Attachment 31: 200315_2203_leakagecurrents.png
200315_2203_leakagecurrents.png
Attachment 32: 200315_2204_goodevents.png
200315_2204_goodevents.png
Attachment 33: 200315_2358_FEETEMPS.png
200315_2358_FEETEMPS.png
Attachment 34: 200315_2358LEAKAGECURRENTS.png
200315_2358LEAKAGECURRENTS.png
Attachment 35: 200315_2359_goodevents.png
200315_2359_goodevents.png
  455   Sat May 14 23:02:33 2022 TDSunday 15 May 00:00-08:00
Pb beam c. 1e+9/spill

00.02 zero all histograms
      system wide checks - baseline counters

00.08 ASIC check control - all FEE64s, all ASICs

Attachment 1 - grafana DSSSD bias, leakage current & temp - OK

Attachments 2 & 3 - ucesb - AIDA DSSSD #1 c. 10-20Hz peak rate on spill, c. 0Hz off spill

Attachment 4 - DSSSD bias & Leakage current - OK

Attachment 5 - FEE64 temps OK

Attachment 6-13 - adc, pause, resume & correlation scaler data items, push, flush, aida01, aida06

Attachments 14-18 - system wide checks - all OK *except* aida09 clock fail status 6

Attachments 19-25 - iptraf, TapeServer, NewMerger, NewMerger stats



00.44 file S450/R4_792

      ADC control register 0xff

      all disc outputs disabled

      ASIC settings 2021Apr29-13-16-00
       slow comparator aida02 & aida04 0x16, aida06 & aida08 0x23, aida12 0xd, all others 0xc

      BNC PB-5 settings
      amplitude 1.000V
      attenuation x1
      tau_d 1ms
      frequency 2Hz
      polarity +

00.51 disk space OK - c. 3' per data file

[npg@aidas-gsi S450]$ ls -l /
total 40
lrwxrwxrwx.   1 root root     7 Oct 18  2021 bin -> usr/bin
dr-xr-xr-x.   5 root root  4096 May 12 18:31 boot
drwxr-xr-x.  23 root root  4100 May 12 18:31 dev
drwxr-xr-x. 145 root root  8192 May 12 18:31 etc
drwxr-xr-x.   4 root root    56 Jan 28 14:56 home
lrwxrwxrwx.   1 root root     7 Oct 18  2021 lib -> usr/lib
lrwxrwxrwx.   1 root root     9 Oct 18  2021 lib64 -> usr/lib64
drwxr-xr-x.   5 root root   113 Apr 28 11:58 media
lrwxrwxrwx.   1 root root    45 Oct 18  2021 MIDAS -> /home/npg/MIDAS_Releases/23Jan19/MIDAS_200119
drwxr-xr-x.   2 root root     6 Apr 11  2018 mnt
drwxr-xr-x.   3 root root    16 Oct 18  2021 opt
dr-xr-xr-x. 464 root root     0 May 12 18:30 proc
dr-xr-x---.  10 root root  4096 May 13 11:47 root
drwxr-xr-x.  44 root root  1360 May 14 06:34 run
lrwxrwxrwx.   1 root root     8 Oct 18  2021 sbin -> usr/sbin
drwxr-xr-x.   2 root root     6 Apr 11  2018 srv
dr-xr-xr-x.  13 root root     0 May 12 18:31 sys
lrwxrwxrwx.   1 root root    28 Apr 28 14:19 TapeData -> /media/SecondDrive/TapeData/
drwxrwxrwt.  40 root root 12288 May 15 00:49 tmp
drwxr-xr-x.  13 root root   155 Oct 18  2021 usr
drwxr-xr-x.  21 root root  4096 Oct 18  2021 var
[npg@aidas-gsi S450]$ df -h
Filesystem               Size  Used Avail Use% Mounted on
devtmpfs                 7.8G     0  7.8G   0% /dev
tmpfs                    7.8G  373M  7.4G   5% /dev/shm
tmpfs                    7.8G   19M  7.7G   1% /run
tmpfs                    7.8G     0  7.8G   0% /sys/fs/cgroup
/dev/mapper/centos-root   50G   16G   35G  31% /
/dev/sda2               1014M  226M  789M  23% /boot
/dev/sda1                200M   12M  189M   6% /boot/efi
/dev/sde1                7.2T  3.1T  3.8T  46% /media/SecondDrive
/dev/mapper/centos-home  407G   91G  316G  23% /home
tmpfs                    1.6G   52K  1.6G   1% /run/user/1000
/dev/sdd1                7.2T  6.5T  310G  96% /run/media/npg/ThirdDrive

Attachment 26 - analysis data file R4_792

Attachments 27-28 - per FEE64 rate & stat spectra - shows distro HEC events

Attachments 29-30 - per FEE64 1.8.L spectra 
 pulser peak widths aida01 134 ch FWHM, aida04 393 ch FWHM

Attachments 31-32 - per FEE64 1.8.H spectra

Attachments 33-34 - aida02 & aida04 1.*.H spectra

04.13

Attachment 35 - DSSSD bias & Leakage current - OK

Attachment 36 - FEE64 temps OK

Attachment 37 - adc data item stats

Attachments 38 - ucesb

system wide checks - all OK *except* aida09 clock fail status 6


07.05

Attachment 39 - DSSSD bias & Leakage current - OK

Attachment 40 - FEE64 temps OK

Attachment 41 - adc data item stats

Attachments 42 - ucesb

system wide checks - all OK *except* aida09 clock fail status 6

Attachments 43-48 - aida02 & aida04 & aida06 & aida08 & 1.*.H spectra
Attachment 1: Screenshot_from_2022-05-15_00-03-33.png
Screenshot_from_2022-05-15_00-03-33.png
Attachment 2: Screenshot_from_2022-05-15_00-03-58.png
Screenshot_from_2022-05-15_00-03-58.png
Attachment 3: Screenshot_from_2022-05-15_00-04-18.png
Screenshot_from_2022-05-15_00-04-18.png
Attachment 4: Screenshot_from_2022-05-15_00-04-39.png
Screenshot_from_2022-05-15_00-04-39.png
Attachment 5: Screenshot_from_2022-05-15_00-05-10.png
Screenshot_from_2022-05-15_00-05-10.png
Attachment 6: Screenshot_from_2022-05-15_00-07-25.png
Screenshot_from_2022-05-15_00-07-25.png
Attachment 7: Screenshot_from_2022-05-15_00-16-59.png
Screenshot_from_2022-05-15_00-16-59.png
Attachment 8: Screenshot_from_2022-05-15_00-17-52.png
Screenshot_from_2022-05-15_00-17-52.png
Attachment 9: Screenshot_from_2022-05-15_00-18-55.png
Screenshot_from_2022-05-15_00-18-55.png
Attachment 10: Screenshot_from_2022-05-15_00-19-35.png
Screenshot_from_2022-05-15_00-19-35.png
Attachment 11: Screenshot_from_2022-05-15_00-20-14.png
Screenshot_from_2022-05-15_00-20-14.png
Attachment 12: Screenshot_from_2022-05-15_00-20-57.png
Screenshot_from_2022-05-15_00-20-57.png
Attachment 13: Screenshot_from_2022-05-15_00-21-32.png
Screenshot_from_2022-05-15_00-21-32.png
Attachment 14: Screenshot_from_2022-05-15_00-26-13.png
Screenshot_from_2022-05-15_00-26-13.png
Attachment 15: Screenshot_from_2022-05-15_00-26-49.png
Screenshot_from_2022-05-15_00-26-49.png
Attachment 16: Screenshot_from_2022-05-15_00-27-19.png
Screenshot_from_2022-05-15_00-27-19.png
Attachment 17: Screenshot_from_2022-05-15_00-28-00.png
Screenshot_from_2022-05-15_00-28-00.png
Attachment 18: Screenshot_from_2022-05-15_00-28-45.png
Screenshot_from_2022-05-15_00-28-45.png
Attachment 19: Screenshot_from_2022-05-15_00-33-46.png
Screenshot_from_2022-05-15_00-33-46.png
Attachment 20: Screenshot_from_2022-05-15_00-35-33.png
Screenshot_from_2022-05-15_00-35-33.png
Attachment 21: Screenshot_from_2022-05-15_00-36-00.png
Screenshot_from_2022-05-15_00-36-00.png
Attachment 22: Screenshot_from_2022-05-15_00-36-26.png
Screenshot_from_2022-05-15_00-36-26.png
Attachment 23: Screenshot_from_2022-05-15_00-36-58.png
Screenshot_from_2022-05-15_00-36-58.png
Attachment 24: Screenshot_from_2022-05-15_00-37-25.png
Screenshot_from_2022-05-15_00-37-25.png
Attachment 25: Screenshot_from_2022-05-15_00-37-46.png
Screenshot_from_2022-05-15_00-37-46.png
Attachment 26: R4_792
 *** TDR format 3.3.0 analyser - TD - May 2021
 *** ERROR: READ I/O error:       5002
                   blocks:      32000
          ADC data format:  250102849 ( 1597022.2 Hz)
        Other data format:   11817151 (   75458.0 Hz)
 Sample trace data format:          0 (       0.0 Hz)
         Undefined format:          0 (       0.0 Hz)
   Other data format type:      PAUSE:       2692 (      17.2 Hz)
                               RESUME:       2692 (      17.2 Hz)
                              SYNC100:      32584 (     208.1 Hz)
                              WR48-63:      32584 (     208.1 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:   11746599 (   75007.5 Hz)
                           Other info:          0 (       0.0 Hz)

   ADC data range bit set:      48170 (     307.6 Hz)

                Timewarps:        ADC:          0 (       0.0 Hz)
                                PAUSE:          0 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:          0 (       0.0 Hz)
                              WR48-63:          0 (       0.0 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:          0 (       0.0 Hz)
                            Undefined:          0 (       0.0 Hz)
                         Sample trace:          0 (       0.0 Hz)

 *** Timestamp elapsed time:      156.606 s
 FEE  elapsed dead time(s) elapsed idle time(s)
  0                0.000                0.000
  1                6.802                0.000
  2                0.000                0.000
  3               28.710                0.000
  4                0.000                0.000
  5               76.548                0.000
  6                0.000                0.000
  7               36.670                0.000
  8                0.002                0.000
  9                0.000                0.000
 10                0.000               14.137
 11               20.418                0.000
 12                0.000                0.000
 13                0.000                3.327
 14                0.000                0.000
 15                0.000                0.000
 16                0.000                0.000
 17                0.000                0.000
 18                0.000                0.000
 19                0.000                0.000
 20                0.000                0.000
 21                0.000                0.000
 22                0.000                0.000
 23                0.000                0.000
 24                0.000                0.000
 25                0.000                0.000
 26                0.000                0.000
 27                0.000                0.000
 28                0.000                0.000
 29                0.000                0.000
 30                0.000                0.000
 31                0.000                0.000
 32                0.000                0.000

 *** Statistics
 FEE  ADC Data Other Data     Sample  Undefined      Pause     Resume    SYNC100    WR48-63       Disc        MBS      Other   HEC Data
  0    7219269       2805          0          0          0          0        933        933          0        939          0        589
  1   37816150      10006          0          0        269        269       4734       4734          0          0          0      43094
  2    6973157    6465448          0          0          0          0       1676       1676          0    6462096          0        570
  3   37723722    5287878          0          0        579        580       5326       5326          0    5276067          0        352
  4    3982759       1044          0          0          0          0        522        522          0          0          0        233
  5   36626106      10369          0          0        747        746       4438       4438          0          0          0        211
  6   11915365       7321          0          0          0          0       1544       1544          0       4233          0        403
  7   42135728      15066          0          0        653        653       5248       5248          0       3264          0        174
  8   10616770       2532          0          0          2          2       1264       1264          0          0          0        570
  9    3975235        966          0          0          0          0        483        483          0          0          0        497
 10    2072466        538          0          0          0          0        269        269          0          0          0        289
 11   40106886      10836          0          0        442        442       4976       4976          0          0          0        472
 12    6251481       1636          0          0          0          0        818        818          0          0          0        413
 13    2687755        706          0          0          0          0        353        353          0          0          0        303
 14          0          0          0          0          0          0          0          0          0          0          0          0
 15          0          0          0          0          0          0          0          0          0          0          0          0
 16          0          0          0          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0          0          0          0

 *** Timewarps
 FEE       ADC      Pause     Resume    SYNC100    WR48-63       Disc        MBS  Undefined    Samples
  0          0          0          0          0          0          0          0          0          0
  1          0          0          0          0          0          0          0          0          0
  2          0          0          0          0          0          0          0          0          0
  3          0          0          0          0          0          0          0          0          0
  4          0          0          0          0          0          0          0          0          0
  5          0          0          0          0          0          0          0          0          0
  6          0          0          0          0          0          0          0          0          0
  7          0          0          0          0          0          0          0          0          0
  8          0          0          0          0          0          0          0          0          0
  9          0          0          0          0          0          0          0          0          0
 10          0          0          0          0          0          0          0          0          0
 11          0          0          0          0          0          0          0          0          0
 12          0          0          0          0          0          0          0          0          0
 13          0          0          0          0          0          0          0          0          0
 14          0          0          0          0          0          0          0          0          0
 15          0          0          0          0          0          0          0          0          0
 16          0          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0          0

 *** Program elapsed time:   36.205s (  883.854 blocks/s,  55.241 Mb/s)
Attachment 27: Screenshot_from_2022-05-15_00-56-27.png
Screenshot_from_2022-05-15_00-56-27.png
Attachment 28: Screenshot_from_2022-05-15_00-57-44.png
Screenshot_from_2022-05-15_00-57-44.png
Attachment 29: Screenshot_from_2022-05-15_00-59-12.png
Screenshot_from_2022-05-15_00-59-12.png
Attachment 30: Screenshot_from_2022-05-15_01-02-36.png
Screenshot_from_2022-05-15_01-02-36.png
Attachment 31: Screenshot_from_2022-05-15_01-04-44.png
Screenshot_from_2022-05-15_01-04-44.png
Attachment 32: Screenshot_from_2022-05-15_01-05-49.png
Screenshot_from_2022-05-15_01-05-49.png
Attachment 33: Screenshot_from_2022-05-15_01-13-22.png
Screenshot_from_2022-05-15_01-13-22.png
Attachment 34: Screenshot_from_2022-05-15_01-15-01.png
Screenshot_from_2022-05-15_01-15-01.png
Attachment 35: Screenshot_from_2022-05-15_04-13-03.png
Screenshot_from_2022-05-15_04-13-03.png
Attachment 36: Screenshot_from_2022-05-15_04-13-31.png
Screenshot_from_2022-05-15_04-13-31.png
Attachment 37: Screenshot_from_2022-05-15_04-13-52.png
Screenshot_from_2022-05-15_04-13-52.png
Attachment 38: Screenshot_from_2022-05-15_04-14-31.png
Screenshot_from_2022-05-15_04-14-31.png
Attachment 39: Screenshot_from_2022-05-15_07-05-19.png
Screenshot_from_2022-05-15_07-05-19.png
Attachment 40: Screenshot_from_2022-05-15_07-05-50.png
Screenshot_from_2022-05-15_07-05-50.png
Attachment 41: Screenshot_from_2022-05-15_07-06-15.png
Screenshot_from_2022-05-15_07-06-15.png
Attachment 42: Screenshot_from_2022-05-15_07-06-42.png
Screenshot_from_2022-05-15_07-06-42.png
Attachment 43: Screenshot_from_2022-05-15_07-06-42.png
Screenshot_from_2022-05-15_07-06-42.png
Attachment 44: Screenshot_from_2022-05-15_07-12-37.png
Screenshot_from_2022-05-15_07-12-37.png
Attachment 45: Screenshot_from_2022-05-15_07-14-49.png
Screenshot_from_2022-05-15_07-14-49.png
Attachment 46: Screenshot_from_2022-05-15_07-16-18.png
Screenshot_from_2022-05-15_07-16-18.png
Attachment 47: Screenshot_from_2022-05-15_07-20-47.png
Screenshot_from_2022-05-15_07-20-47.png
Attachment 48: Screenshot_from_2022-05-15_07-22-02.png
Screenshot_from_2022-05-15_07-22-02.png
  706   Sun Jun 15 10:53:08 2025 TDSunday 15 June 2025
11.45 DSSSD bias & leakage current - attachment 1
      FEE64 temps OK - attachment 2
      All system wide checks OK *except* WR decoder status aida02 - attachment 3
      WR timestamps OK - attachment 4
      ADC data item stats - attachment 5

      per FEE64 Rate spectra - attachments 6-7
       note ASICs with per channel rates of 22Hz (pulser)
       note *all* FEE64 slow comparator 0x64
       note ignore aida10 - connected to MSL type BB7 - not biased yet

      per p+n FEE64 1.8.L spectra - attachment 8-9
       pulser peak widths
        aida05, aida14, aida16 c. 15-20 ch FWHM ( c. 10-15keV FWHM ) - cable from DSSSD not connected to adaptor PCB
        aida15 c. 137 ch FWHM - electronic noise c. 2x worse than configuration used for 2023/4 tests

      per p+n FEE64 1.8.W spectra - 20us FSR - attachments 10
      per p+n FEE64 1.8.W spectra - 20us FSR - expanded y-scale = 2000 x 122uV/LSB = 0.244V FSR - attachment 11

      per n+n FEE64 1.8.W spectra - 20us FSR - attachments 12


15.20 Data suggests 2x (of 6x) Si wafers are not biased - aida11 & aida05-aida12 - J2 cable disconnected

      Switch from negative to positive bias polarity

      CAEN N1419ET channels 1 and 2 switched from negative to positive polarity
      Note internal LK fitted - HV outputs *not* floating

      Channel 1 -> DSSSD#0 from aida12 to aida04 (HV daisy chain aida12-aida03-aida15 *not* removed)
      Channel 2 -> DSSSD#1 from aida16 to aida08 (HV daisy chain aida16-aida07-aida11 *not* removed)

      LK1 removed from adaptor PCBs for aida02. aida04, aida06, aaida08

      LK1 fitted aida05, aida13, aida10, aida09, aida15, aida11

      LK3 *no* change

      DSSSD bias & leakage current - attachment 13
       note increase in leakage current for DSSSD#0 but not DSSSD#1
       - indicating DSSSD#0 has all Si wafers biased
       - all wafers of DSSSD#1 not biased => damaged adaptor PCB and/or cabling, misaligned ribbon cable/Kapton PCB connection within snout
      FEE64 temps OK - attachment 14
      ADC data item stats - attachment 15

      per FEE64 Rate spectra - attachments 16-17
       note ASICs with per channel rates of 22Hz (pulser)
       note *all* FEE64 slow comparator 0x64
       note ignore aida10 - connected to MSL type BB7 - not biased yet

      per p+n FEE64 1.8.L spectra - attachment 18-19
       pulser peak widths
        aida05, aida14, aida16 c. 15-20 ch FWHM ( c. 10-15keV FWHM ) - cable from DSSSD not connected to adaptor PCB
        aida15 c. 108 ch FWHM - electronic noise c. 2x worse than configuration used for 2023/4 tests

      per p+n FEE64 1.8.W spectra - 20us FSR - attachments 20
      per p+n FEE64 1.8.W spectra - 20us FSR - expanded y-scale = 2000 x 122uV/LSB = 0.244V FSR - attachment 21

      per n+n FEE64 1.8.W spectra - 20us FSR - attachments 22


16.40 Add LK1 to aida03, aida07

      DSSSD bias & leakage current - attachment 23
       no change

      ADC data item stats - attachment 24

      per p+n FEE64 1.8.L spectra - attachment 25-26
       pulser peak widths
        aida05, aida14, aida16 c. 15-20 ch FWHM ( c. 10-15keV FWHM ) - cable from DSSSD not connected to adaptor PCB
        aida15 c. 117 ch FWHM - electronic noise c. 2x worse than configuration used for 2023/4 tests

      per p+n FEE64 1.8.W spectra - 20us FSR - attachments 27
       no change

16.50 Increase DSSSD#0 bias from +100V to +120V

      DSSSD bias & leakage current - attachment 28

      per p+n FEE64 1.8.L spectra - attachment 29-30
       pulser peak widths
        aida05, aida14, aida16 c. 15-20 ch FWHM ( c. 10-15keV FWHM ) - cable from DSSSD not connected to adaptor PCB
        aida15 c. 70 ch FWHM - electronic noise c. 1.2-2x worse than configuration used for 2023/4 tests

      per FEE64 Rate spectra - attachments 31-32

      per FEE64 1.8.W spectra - 20us FSR - attachments 33-36
       aida02 & aida04 stabilised but still very noisy

      Switch BNC PB-5 Pulser polarity + to - and connect test- test daisy chain

      ADC data item stats - attachment 37

      per FEE64 Rate spectra - attachments 38-39
       aida02 & aida04 rates significantly lower but not 25Hz expected

      per n+n FEE64 1.8.L spectra - attachment 40
       pulser peak width aida02 c. 700 ch FWHM - very noisy!

      per n+n FEE64 1.8.W spectra 20us FSR - attachment 41 

*** Summary

    DSSSD#0
    3x Si wafers biased OK
    p+n junction side electronic noise c. 1.2-2x 2023/4 configuration
    n+n Ohmic side very noisy - increased detector HV to stabilise => increased capacitative load cf. 2023/4 configuration?

    DSSSD#1
    2x Si wafers biased OK, 1x probably not biased
    p+n junction side electronic noise > 4x 2023/4 configuration
    n+n Ohmic side very noisy/unstable - increased detector HV not attempted due to issues with biasing all 3x Si wafers

    MSL type BB7 
    not biased yet

    To Do
    Complete connection and grounding of Bplas detectors
Attachment 1: Screenshot_from_2025-06-15_11-45-34.png
Screenshot_from_2025-06-15_11-45-34.png
Attachment 2: Screenshot_from_2025-06-15_11-45-50.png
Screenshot_from_2025-06-15_11-45-50.png
Attachment 3: Screenshot_from_2025-06-15_11-46-01.png
Screenshot_from_2025-06-15_11-46-01.png
Attachment 4: Screenshot_from_2025-06-15_11-46-29.png
Screenshot_from_2025-06-15_11-46-29.png
Attachment 5: Screenshot_from_2025-06-15_11-46-45.png
Screenshot_from_2025-06-15_11-46-45.png
Attachment 6: Screenshot_from_2025-06-15_11-47-05.png
Screenshot_from_2025-06-15_11-47-05.png
Attachment 7: Screenshot_from_2025-06-15_11-47-11.png
Screenshot_from_2025-06-15_11-47-11.png
Attachment 8: Screenshot_from_2025-06-15_11-48-00.png
Screenshot_from_2025-06-15_11-48-00.png
Attachment 9: Screenshot_from_2025-06-15_11-48-06.png
Screenshot_from_2025-06-15_11-48-06.png
Attachment 10: Screenshot_from_2025-06-15_11-51-44.png
Screenshot_from_2025-06-15_11-51-44.png
Attachment 11: Screenshot_from_2025-06-15_11-52-19.png
Screenshot_from_2025-06-15_11-52-19.png
Attachment 12: Screenshot_from_2025-06-15_11-52-47.png
Screenshot_from_2025-06-15_11-52-47.png
Attachment 13: Screenshot_from_2025-06-15_15-19-56.png
Screenshot_from_2025-06-15_15-19-56.png
Attachment 14: Screenshot_from_2025-06-15_15-20-14.png
Screenshot_from_2025-06-15_15-20-14.png
Attachment 15: Screenshot_from_2025-06-15_15-30-29.png
Screenshot_from_2025-06-15_15-30-29.png
Attachment 16: Screenshot_from_2025-06-15_15-31-20.png
Screenshot_from_2025-06-15_15-31-20.png
Attachment 17: Screenshot_from_2025-06-15_15-31-37.png
Screenshot_from_2025-06-15_15-31-37.png
Attachment 18: Screenshot_from_2025-06-15_15-32-39.png
Screenshot_from_2025-06-15_15-32-39.png
Attachment 19: Screenshot_from_2025-06-15_15-33-02.png
Screenshot_from_2025-06-15_15-33-02.png
Attachment 20: Screenshot_from_2025-06-15_15-35-33.png
Screenshot_from_2025-06-15_15-35-33.png
Attachment 21: Screenshot_from_2025-06-15_15-36-17.png
Screenshot_from_2025-06-15_15-36-17.png
Attachment 22: Screenshot_from_2025-06-15_15-36-54.png
Screenshot_from_2025-06-15_15-36-54.png
Attachment 23: Screenshot_from_2025-06-15_16-41-02.png
Screenshot_from_2025-06-15_16-41-02.png
Attachment 24: Screenshot_from_2025-06-15_16-38-08.png
Screenshot_from_2025-06-15_16-38-08.png
Attachment 25: Screenshot_from_2025-06-15_16-39-34.png
Screenshot_from_2025-06-15_16-39-34.png
Attachment 26: Screenshot_from_2025-06-15_16-39-46.png
Screenshot_from_2025-06-15_16-39-46.png
Attachment 27: Screenshot_from_2025-06-15_16-41-30.png
Screenshot_from_2025-06-15_16-41-30.png
Attachment 28: Screenshot_from_2025-06-15_16-52-15.png
Screenshot_from_2025-06-15_16-52-15.png
Attachment 29: Screenshot_from_2025-06-15_16-52-47.png
Screenshot_from_2025-06-15_16-52-47.png
Attachment 30: Screenshot_from_2025-06-15_16-53-00.png
Screenshot_from_2025-06-15_16-53-00.png
Attachment 31: Screenshot_from_2025-06-15_16-54-00.png
Screenshot_from_2025-06-15_16-54-00.png
Attachment 32: Screenshot_from_2025-06-15_16-54-08.png
Screenshot_from_2025-06-15_16-54-08.png
Attachment 33: Screenshot_from_2025-06-15_16-54-40.png
Screenshot_from_2025-06-15_16-54-40.png
Attachment 34: Screenshot_from_2025-06-15_16-54-58.png
Screenshot_from_2025-06-15_16-54-58.png
Attachment 35: Screenshot_from_2025-06-15_16-55-14.png
Screenshot_from_2025-06-15_16-55-14.png
Attachment 36: Screenshot_from_2025-06-15_16-55-31.png
Screenshot_from_2025-06-15_16-55-31.png
Attachment 37: Screenshot_from_2025-06-15_17-03-49.png
Screenshot_from_2025-06-15_17-03-49.png
Attachment 38: Screenshot_from_2025-06-15_17-04-00.png
Screenshot_from_2025-06-15_17-04-00.png
Attachment 39: Screenshot_from_2025-06-15_17-04-15.png
Screenshot_from_2025-06-15_17-04-15.png
Attachment 40: Screenshot_from_2025-06-15_17-04-40.png
Screenshot_from_2025-06-15_17-04-40.png
Attachment 41: Screenshot_from_2025-06-15_17-06-24.png
Screenshot_from_2025-06-15_17-06-24.png
  198   Sun Mar 14 07:04:30 2021 OHSunday 14th March 2021
08:07 Realised the correlation scalers hadn't been enabled again. Have enabled them now. R56_151

Looking through the merger messages from the nightshift we see bad timestamps in FEE 11. This FEE does not have any scalers going into it.
It is also the least noisy FEE in the back DSSD

MERGE Data Link (20510): bad timestamp  10 3 0xc2b77eff 0x037a3b92 0x000017f7f37a3b92 0x166c17f7f37a3b92 0x166c17f7f3877262
MERGE Data Link (20510): bad timestamp  10 3 0xc2827f3a 0x037aa122 0x000017f7f37aa122 0x166c17f7f37aa122 0x166c17f7f3877262

09:01 System wide checks

		 Base 		Current 	Difference
aida07 fault 	 0x8c74 : 	 0x8c77 : 	 3  
White Rabbit error counter test result: Passed 11, Failed 1

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

			 Base 		Current 		Difference
aida07 fault 	 0x2 : 	 0x5 : 	 3  
FPGA Timestamp error counter test result: Passed 11, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last

     Statistics - attachment 1
     Temperature - attachment 2
     Bias and leakage current - attachment 3

11:18 System wide checks

		 Base 		Current 	Difference
aida07 fault 	 0x8c77 : 	 0x8c78 : 	 1  
White Rabbit error counter test result: Passed 11, Failed 1

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

     Statistics - attachment 4
     Temperatures - attachment 5
     Bias and leakage - attachment 6


13.14 DAQ continues OK - file R56_285

      ASIC settings 2019Dec19-16.19.51
      DSSSD#1 slow comparator 0xa
      DSSSD#2 slow comparator 0xa
      DSSSD#3 slow comparator 0xd

      BNC PB-5 Pulser 
      Amplitude1.0V
      Attenuation x1
      Frequency 2Hz
      tau_d 1ms
      - polarity
      Delay 250ns, tail pulse

13.18 System wide checks

FEE64 module aida09 global clocks failed, 6
 Clock status test result: Passed 11, Failed 1

Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1' 
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable

FEE64 module aida07 failed
FEE64 module aida09 failed
Calibration test result: Passed 10, Failed 2

If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module


		 Base 		Current 	Difference
aida07 fault 	 0x8c77 : 	 0x8c78 : 	 1  
White Rabbit error counter test result: Passed 11, Failed 1

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR


FPGA Timestamp error counter test result: Passed 12, Failed 0
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last

Returned 0 0 0 0 0 0 0 0 0 0 0 0  
Mem(KB) :	4	8	16	32	64	128	256	512	1k	2k	4k
aida01 :     27      8      3      4      2      2      2      2      3      4     10   : 54492
aida02 :      0      2      3      0      2      3      2      3      3      3      6   : 36416
aida03 :     18      9      1      2      0      4      1      3      3      3      6   : 36320
aida04 :      2      3      3      3      1      4      3      2      2      4     15   : 74224
aida05 :     26     72     72     54     53     49     22     18      7      4      7   : 72104
aida06 :      7      5      5     10      0      2      2      4      3      2      5   : 30932
aida07 :      2      4      2      1      1      3      2      3      2      3      7   : 39464
aida08 :     25      9      4      2      1      2      1      4      3      3      6   : 36716
aida09 :      4     11      3      5      2      4      1      2      3      3      6   : 36024
aida10 :     18      5      2      1      0      5      2      2      3      3      6   : 36144
aida11 :     25     10      4      1      1      4      2      3      2      3      6   : 35668
aida12 :     26      7      1      1      1      3      2      3      3      3      6   : 36496

13.19 FEE64 temperatures OK - attachment 7
      Good events stats OK - attachment 8
      Bias & leakage currents OK - attachment 9
      Merger 4.5M items/s
      Tape Server 14Mb/s
      No Merger warning/error messages since last report. No MBS relay warning/error/messages since last restart.

      All histograms zero'd


16.00 DAQ contiues R56_361

       System wide checks

FEE64 module aida09 global clocks failed, 6
 Clock status test result: Passed 11, Failed 1

Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1' 
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable

FEE64 module aida07 failed
FEE64 module aida09 failed
Calibration test result: Passed 10, Failed 2

If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module

	 Base 		Current 	Difference
aida07 fault 	 0x8c77 : 	 0x8c79 : 	 2  
White Rabbit error counter test result: Passed 11, Failed 1

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

FPGA Timestamp error counter test result: Passed 12, Failed 0
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last

Returned 0 0 0 0 0 0 0 0 0 0 0 0  
Mem(KB) :	4	8	16	32	64	128	256	512	1k	2k	4k
aida01 :     23      8      6      4      2      2      2      2      3      4     10   : 54524
aida02 :      1      3      4      2      1      2      1      4      3      3      6   : 36572
aida03 :      2      3      2      3      1      3      2      2      3      3      6   : 35936
aida04 :     19      7      2      3      1      4      3      3      1      4     15   : 73796
aida05 :     34     68     72     54     53     49     22     18      7      4      7   : 72104
aida06 :     11     10      4     10      0      2      2      4      3      2      5   : 30972
aida07 :      1      3      2      2      2      2      2      3      2      3      7   : 39420
aida08 :     24     10      5      2      1      3      1      4      3      3      6   : 36864
aida09 :     20      8      3      5      2      3      2      3      3      3      6   : 36704
aida10 :      2      3      2      3      1      3      3      2      3      3      6   : 36192
aida11 :     17     10      4      0      1      4      2      3      2      3      6   : 35604
aida12 :     20      6      3      2      1      3      2      2      2      4      6   : 37040


15.05 FEE64 temperatures OK - attachment 10
      Good events stats OK - attachment 11
      Bias & leakage currents OK - attachment 12
      Merger 4.6M items/s
      Tape Server 14Mb/s
      No Merger warning/error messages since last report. No MBS relay warning/error/messages since last restart.

      Rate spectra - attachments 13 & 14
      p+n junction HEC spectra - attachment 15

17.33 end of S452 run file R56_397
      No Merger warning/error messages since last report. No MBS relay warning/error/messages since last restart.

      BNC PB-5 config - attachment 16
      CAEN 8xxx NIM bin config - attachment 17
Attachment 1: 210314_0858_Stats.png
210314_0858_Stats.png
Attachment 2: 210314_0859_Temp.png
210314_0859_Temp.png
Attachment 3: 210314_0900_Bias.png
210314_0900_Bias.png
Attachment 4: 210314_1116_Stats.png
210314_1116_Stats.png
Attachment 5: 210314_1116_Temp.png
210314_1116_Temp.png
Attachment 6: 210314_1117_Bias.png
210314_1117_Bias.png
Attachment 7: Screenshot_2021-03-14_Temperature_and_status_scan_aidas-gsi.png
Screenshot_2021-03-14_Temperature_and_status_scan_aidas-gsi.png
Attachment 8: Screenshot_2021-03-14_Statistics_aidas-gsi.png
Screenshot_2021-03-14_Statistics_aidas-gsi.png
Attachment 9: 34.png
34.png
Attachment 10: Screenshot_2021-03-14_Temperature_and_status_scan_aidas-gsi(1).png
Screenshot_2021-03-14_Temperature_and_status_scan_aidas-gsi(1).png
Attachment 11: Screenshot_2021-03-14_Statistics_aidas-gsi(1).png
Screenshot_2021-03-14_Statistics_aidas-gsi(1).png
Attachment 12: 35.png
35.png
Attachment 13: Screenshot_2021-03-14_Spectrum_Browser_aidas-gsi.png
Screenshot_2021-03-14_Spectrum_Browser_aidas-gsi.png
Attachment 14: Screenshot_2021-03-14_Spectrum_Browser_aidas-gsi(1).png
Screenshot_2021-03-14_Spectrum_Browser_aidas-gsi(1).png
Attachment 15: Screenshot_2021-03-12_Spectrum_Browser_aidas-gsi(1).png
Screenshot_2021-03-12_Spectrum_Browser_aidas-gsi(1).png
Attachment 16: 40.png
40.png
Attachment 17: 41.png
41.png
Attachment 18: 51.png
51.png
Attachment 19: 52.png
52.png
Attachment 20: Screenshot_2021-03-14_Spectrum_Browser_aidas-gsi(3).png
Screenshot_2021-03-14_Spectrum_Browser_aidas-gsi(3).png
  199   Sun Mar 14 08:16:00 2021 OH PJCSSunday 14th March 2021
> 08:07 Realised the correlation scalers hadn't been enabled again. Have enabled them now. R56_151
> 
> Looking through the merger messages from the nightshift we see bad timestamps in FEE 11. This FEE does not have any scalers going into it.
> It is also the least noisy FEE in the back DSSD
> 
> MERGE Data Link (20510): bad timestamp  10 3 0xc2b77eff 0x037a3b92 0x000017f7f37a3b92 0x166c17f7f37a3b92 0x166c17f7f3877262
> MERGE Data Link (20510): bad timestamp  10 3 0xc2827f3a 0x037aa122 0x000017f7f37aa122 0x166c17f7f37aa122 0x166c17f7f3877262
> 
> 09:01 System wide checks
> 
> 		 Base 		Current 	Difference
> aida07 fault 	 0x8c74 : 	 0x8c77 : 	 3  
> White Rabbit error counter test result: Passed 11, Failed 1
> 
> Understand the status reports as follows:-
> Status bit 3 : White Rabbit decoder detected an error in the received data
> Status bit 2 : Firmware registered WR error, no reload of Timestamp
> Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
> 
> 			 Base 		Current 		Difference
> aida07 fault 	 0x2 : 	 0x5 : 	 3  
> FPGA Timestamp error counter test result: Passed 11, Failed 1
> If any of these counts are reported as in error
> The ASIC readout system has detected a timeslip.
> That is the timestamp read from the time FIFO is not younger than the last
> 
>      Statistics - attachment 1
>      Temperature - attachment 2
>      Bias and leakage current - attachment 3
It might be worth considering decreasing the time between the flushing of slower FEEs. Is the Merger waiting for the slower data and causing ‘deadtime’ in the faster ones. Perhaps consider this as a 
balancing across the system? 
  200   Sun Mar 14 17:35:05 2021 TD, OHSunday 14 March - Pulser Walkthrough
18.01 Pulser walkthrough at end of run S452 file S452/R57_0 ... R57_28

      ASIC settings 2019Dec19-16.19.51
      DSSSD#1 slow comparator 0xa
      DSSSD#2 slow comparator 0xa
      DSSSD#3 slow comparator 0xd

      BNC PB-5 Pulser 
      Amplitude1.0V
      Attenuation x1
      Frequency 2Hz
      tau_d 1ms
      - polarity
      Delay 250ns, tail pulse

      BNC PB-5 settings changed by terminal command line (see attachments 1, 2 & 3)
      using the commands

      set rep rate 25
      set attenuation 5
      set amplitude 10
      set amplitude 9
      :
      :
      set amplitude 1
      set attenuation 1
      set rep rate 2

      Example n+n LEC spectrum - see attachment 4

18.27 R57 ends OK file R57_28
Attachment 1: 52.png
52.png
Attachment 2: 51.png
51.png
Attachment 3: 40.png
40.png
Attachment 4: Screenshot_2021-03-14_Spectrum_Browser_aidas-gsi(3).png
Screenshot_2021-03-14_Spectrum_Browser_aidas-gsi(3).png
  367   Sun Jun 13 17:29:04 2021 TDSunday 13 June
18.15 DAQ continues file S496/R32_946
      Hot HEC channels multiple FEE64s from c. 06.00 this morning
      ASIC check fixes

      Out of disk space on /TapeData filesystem - switch to no storage mode

      System wide checks OK *except*

		 Base 		Current 	Difference
aida01 fault 	 0x8944 : 	 0x8949 : 	 5  
aida02 fault 	 0x9b36 : 	 0x9b3b : 	 5  
aida03 fault 	 0xf26b : 	 0xf270 : 	 5  
aida04 fault 	 0x57a0 : 	 0x57a5 : 	 5  
aida05 fault 	 0x4d3c : 	 0x4d3e : 	 2  
aida05 : WR status 0x10
 aida06 fault 	 0x640c : 	 0x640e : 	 2  
aida07 fault 	 0x255e : 	 0x2560 : 	 2  
aida08 fault 	 0xffbd : 	 0xffbf : 	 2  
aida15 fault 	 0x4e4 : 	 0x1267 : 	 3459  
White Rabbit error counter test result: Passed 7, Failed 9

Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR

			 Base 		Current 		Difference
aida12 fault 	 0x0 : 	 0x3e01 : 	 15873  
aida13 fault 	 0x0 : 	 0x32d2 : 	 13010  
FPGA Timestamp error counter test result: Passed 14, Failed 2
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last

Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0  
Mem(KB) :	4	8	16	32	64	128	256	512	1k	2k	4k
aida01 :     26     16      4      6      4      4      2      2      2      4      6   : 37608
aida02 :     29     11      9      3      3      4      3      2      2      4      6   : 37756
aida03 :     32      9      6      8      5      3      1      3      3      3      6   : 36840
aida04 :     32      8      3      4      3      4      2      3      3      3      6   : 36912
aida05 :     36      8      6      4      5      4      2      2      2      4      6   : 37616
aida06 :     25     13      4      3      2      3      2      3      4      3      6   : 37740
aida07 :     28     13      4      5      5      2      3      2      2      4      6   : 37624
aida08 :     29      5      9      4      3      4      3      2      2      4      6   : 37740
aida09 :     41      8      4      7      1      3      3      2      2      4      6   : 37572
aida10 :     45     11      7      5      5      3      3      3      3      3      6   : 37340
aida11 :     33     13      5      6      3      4      3      3      1      4      6   : 37308
aida12 :     39      8      6      6      4      4      3      3      1      4      6   : 37372
aida13 :     28     14      3      6      4      4      3      3      1      4      6   : 37328
aida14 :     30     11      3      6      4      4      3      3      3      3      6   : 37312
aida15 :      3      3      4      3      0      3      3      2      1      4      7   : 40260
aida16 :     28      9      5      6      5      3      3      3      1      4      6   : 37256

Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same

 FEE : aida01 =>   Options file size is 1026 	 Last changed Sat Jun 12 15:46:33 CEST 2021
 FEE : aida02 =>   Options file size is 1025 	 Last changed Sun May 23 00:19:21 CEST 2021
 FEE : aida03 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:43:50 CEST 2021
 FEE : aida04 =>   Options file size is 1025 	 Last changed Fri May 14 16:54:56 CEST 2021
 FEE : aida05 =>   Options file size is 1025 	 Last changed Mon May 17 06:25:41 CEST 2021
 FEE : aida06 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:43:59 CEST 2021
 FEE : aida07 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:02 CEST 2021
 FEE : aida08 =>   Options file size is 1025 	 Last changed Sun May 23 00:16:54 CEST 2021
 FEE : aida09 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:08 CEST 2021
 FEE : aida10 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida11 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida12 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida13 =>   Options file size is 1025 	 Last changed Fri May 07 19:40:34 CEST 2021
 FEE : aida14 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021
 FEE : aida15 =>   Options file size is 1025 	 Last changed Sat Jun 12 15:48:28 CEST 2021
 FEE : aida16 =>   Options file size is 1014 	 Last changed Thu Apr 29 14:44:57 CEST 2021

      Grafana - DSSSD bias & leakage current - most recent 7 days - attachment 1
      Lost activity monitor - attachment 2
      1.8.W spectra - 20us FSR - attachments 3 & 4
      ADC data items - attachment 5
      FEE64 temperatures OK - attachment 6
      DSSSD bias & leakage currents OK - attachment 7
      Merger/Tape Server/Merger statistics - attachment 8
       no merger errors reported since previous restart
Attachment 1: Screenshot_2021-06-13_Spectrum_Browser_aidas-gsi(1).png
Screenshot_2021-06-13_Spectrum_Browser_aidas-gsi(1).png
Attachment 2: Screenshot_2021-06-13_Spectrum_Browser_aidas-gsi.png
Screenshot_2021-06-13_Spectrum_Browser_aidas-gsi.png
Attachment 3: Screenshot_2021-06-13_AIDA_Alerting_-_Grafana.png
Screenshot_2021-06-13_AIDA_Alerting_-_Grafana.png
Attachment 4: Screenshot_2021-06-13_Lost_activity_monitor_aidas-gsi.png
Screenshot_2021-06-13_Lost_activity_monitor_aidas-gsi.png
Attachment 5: Screenshot_2021-06-13_Statistics_aidas-gsi.png
Screenshot_2021-06-13_Statistics_aidas-gsi.png
Attachment 6: Screenshot_2021-06-13_Temperature_and_status_scan_aidas-gsi.png
Screenshot_2021-06-13_Temperature_and_status_scan_aidas-gsi.png
Attachment 7: 1000.png
1000.png
Attachment 8: 1001.png
1001.png
  534   Tue Mar 12 16:16:49 2024 NH, TD, JB, HASummary of DSSSD Biasing 12.03
DSSD#1 undergoes a breakdown at 90V, two of the three wafers show this
- The adapter PCBs themselves have no breakdown at 100V, indicating the   issue is internal to the snout

If we are lucky it may be a loose or misaligned kapton connector inside the snout. If not we will have to remove DSSD#2 and inspect DSSD#1 for damage/lint/etc. If we see nothing it should be replaced

DSSD#2 biases perfectly fine, but the leakage current is unstable with biasing -ve to p+n and gnd (return) to n+n. Leakage current is stable biasing +ve to n+n and gnd to p+n. (https://elog.ph.ed.ac.uk/DESPEC/240311_093933/Downstream_positive_bias_vs._Current_(uA).png very nice curve)

We see the same fluctuations just daisy chaining 3 p+n PCBs togerher with the bias (no DSSSD or gnd links). The fluctuations were reduced by connecting all 8 adapter boards of the DSSD together.
When biasing just *one* PCB in this way, the current is stable
The fluctuations happened changing the HV channel, adapter board PCB and LEMO cables: it doesn't seem to be a defect with a specific thing.

The test today confirmed the HV cables (the only ones used) were correctly isolated from everything (OL = "infinite" resistance to ground)

We saw unstable current indications using the Mesytec MHV-4 to apply voltage to the PCB as well.

The behaviour is odd but doesn't seem to be related to the DSSD, which I suspect is OK. One idea is to try connecting the 3 adapter boards to FEEs and repeating the test, as this introduces another (substantial) path to gnd. Maybe this eliminates the current instability? It's about the only difference left from December.

-

For the next steps I believe the snout must be dismounted to inspect DSSD#1. It would be best to coordinate this with replacing the broken(?) bPlas. If we are lucky we may not have to remove the DSSDs but it is likely we have to remove both DSSDs to swap out #1

After replacing the DSSD(s) we should cover the snout with a clean black bin bag and bias it on the MH table to confirm both detectors work. This saves the effort of carrying it to S4 just to find another problem.
If both detectors make it to 120V we can mount it again more confidently

Once we have two biased detectors we can rearrange some FEEs to get 8 for DSSD#1 to do the noise tests. I suggest we wire them up for the numbering plan in https://elog.ph.ed.ac.uk/DESPEC/532 but in the DHCP renumber 9,10,15,16 to 5,6,7,8 temporarily to allow data to be sent to MBS for the dry run (merger limitation)

When the remaining FEEs are recovered from UK+CRYRING we can instrument DSSD#2 and renumber the FEEs to match the cables
  595   Tue Apr 23 13:08:28 2024 TDSummary of AIDA FEE64 stops
We have observed a number of AIDA FEE64s stop producing data.

Summary of dates & (approximate) times

aida03	21.4.24	08.20
aida04	22.4.24	09.14             https://elog.ph.ed.ac.uk/DESPEC/586
aida01	22.4.24	16.00
aida02	22.4.24	22.00
aida02	23.4.24	10.21
aida02	23.4.24 11.25
aida04  23.4.24 16.53?            attachment 2
aida04/aida05 25.4.24 22.28       no messages in aida04 system console
aida04  27.8.24 c.23:00
aida04  28.8.24 c. 12-13:00       power cycle
aida04  28.8.24 c. 16.00          reset per Elog 618
aida02  28.4.24 c. 22.20          reset per Elog 618, no error messages in system console
aida04  29.4.24 01.15
aida04  30.4.24 <03.30
aida02  30.4.24 <10.00


aida02 2.5.24 c. 19.30 reset


In future, it would be useful to copy system console reports to the Elog to check whether we can identify common error messages etc.


ssh pi@nnrpi1
<password>

cd logs
cat /ttyUSBx 

The current mapping of USB port # to AIDA FEE64s - attachment 1


aida02 system console c. 19.30 2.5.24

02:05:24/14:40:51|executing generic doStop
02:05:24/19:27:28|Halt (1)
02:05:24/19:27:29|Halt (2)
02:05:24/19:27:29|Halt (3)
02:05:24/19:27:30|Halt (4)
02:05:24/19:27:30|Halt (5)
02:05:24/19:27:31|Halt (6)
02:05:24/19:27:31|Halt (7)
02:05:24/19:27:32|Halt (8)
02:05:24/19:27:32|Halt (9)
02:05:24/19:27:33|Halt (10)
02:05:24/19:27:33|Halt (11)
02:05:24/19:27:34|Halt (12)
02:05:24/19:27:34|Halt (13)
02:05:24/19:27:35|Halt (14)
02:05:24/19:27:35|Halt (15)
02:05:24/19:27:36|Halt (16)
02:05:24/19:27:36|Halt (17)
02:05:24/19:27:37|Halt (18)
02:05:24/19:27:38|Halt (19)
02:05:24/19:27:38|Action has not completed
02:05:24/19:27:39|Giving up waiting
02:05:24/19:27:39|disconnect xfer stream #0 (1)
02:05:24/19:27:39|disconnect xfer stream #0 (2)
02:05:24/19:27:39|disconnect xfer stream #0 (3)
02:05:24/19:27:40|disconnect xfer stream #0 (4)
02:05:24/19:27:40|disconnect xfer stream #0 (5)
02:05:24/19:27:41|disconnect xfer stream #0 (6)
02:05:24/19:27:41|disconnect xfer stream #0 (7)
02:05:24/19:27:42|disconnect xfer stream #0 (8)
02:05:24/19:27:42|disconnect xfer stream #0 (9)
02:05:24/19:27:43|disconnect xfer stream #0 (10)
02:05:24/19:27:43|disconnect xfer stream #0 (11)
02:05:24/19:27:44|disconnect xfer stream #0 (12)
02:05:24/19:27:44|disconnect xfer stream #0 (13)
02:05:24/19:27:45|disconnect xfer stream #0 (14)
02:05:24/19:27:46|disconnect xfer stream #0 (15)
02:05:24/19:27:46|disconnect xfer stream #0 (16)
02:05:24/19:27:47|disconnect xfer stream #0 (17)
02:05:24/19:27:47|disconnect xfer stream #0 (18)
02:05:24/19:27:48|disconnect xfer stream #0 (19)
02:05:24/19:27:48|Action has not completed
02:05:24/19:27:49|Giving up waiting
02:05:24/19:27:49|executing generic Stop_Merger
02:05:24/19:27:49|executing generic Stop_DataStorageServer
02:05:24/19:27:49|completed generic doStop
02:05:24/19:27:49|do_GetState returned z=0 and 1
02:05:24/19:27:49|do_GetState returned z=0 and 1
02:05:24/19:28:01|Transfer Error - : Connection reset by peer
02:05:24/19:28:08|send() failed: 
02:05:24/19:28:08|Aida state now Stopped. AidaExecV10.0_Jul  6 2022: Build 15:10:57 
02:05:24/19:28:08|do_GetState returned z=0 and 0
02:05:24/19:28:13|executing generic doGo
02:05:24/19:28:45|Initialising communication with storage server 192.168.11.99 on port 11002
02:05:24/19:28:45|Setting Transfer Block Size 65536
02:05:24/19:28:45|Setting Transfer Mode 3
02:05:24/19:28:45|Setting Overlap Mode 0
02:05:24/19:28:45|Setting Blocking Mode 0
02:05:24/19:28:45|Setting nice 0
02:05:24/19:28:45|TCP transfer library version 3.12
02:05:24/19:28:45|TCP socket send buffer was 16384 - now 221184
02:05:24/19:28:45|TCP socket receive buffer was 87380 - now 221184
02:05:24/19:28:45|TCP socket created OK - now connecting to 192.168.11.99 port 11002
02:05:24/19:28:45|Connected to 192.168.11.99 port 11002
02:05:24/19:28:45|connect xfer stream #0 (1)
02:05:24/19:28:45|Data Acquisition Statistics counters now cleared
02:05:24/19:28:46|ASIC DMA buffer size 2097152; WAVE DMA buffer size 1048576
02:05:24/19:28:46|Aida state now Going. AidaExecV10.0: Build Jul  6 2022_15:10:57 
02:05:24/19:28:46|Go (1)
02:05:24/19:28:46|Running Go_Electronics for aida02 with TSMaster aida01
02:05:24/19:28:46|ADCs Calibrated. 
02:05:24/19:28:46|Finished Go_Electronics for AIDA
02:05:24/19:28:46|executing generic Go_Merger
02:05:24/19:28:46|executing generic Go_DataStorageServer
02:05:24/19:28:46|completed generic doGo
02:05:24/19:28:46|do_GetState returned z=0 and 1
02:05:24/19:28:46|In RDOGo_Operate: Enabled Correlation, ASIC and discriminator readout
02:05:24/19:28:51|Also in RDOGo_Operate: Enabled waveform readout
02:05:24/19:28:51|get_WAVEBlk (A) which = 0 ---- status = 0048, State machines = 0201
02:05:24/19:28:51|WV:0:Buffer info. Length = 64 : Offset = 0 
02:05:24/19:28:51|WV:1:Buffer info. Length = 62 : Offset = 2 
02:05:24/19:28:51|WV:2:Buffer info. Length = 60 : Offset = 4 
02:05:24/19:28:52|WV:3:Buffer info. Length = 58 : Offset = 6 
02:05:24/19:28:52|WV:4:Buffer info. Length = 56 : Offset = 8 
02:05:24/19:28:52|WV:5:Buffer info. Length = 54 : Offset = 10 
02:05:24/19:28:52|WV:6:Buffer info. Length = 52 : Offset = 12 
02:05:24/19:28:52|WV:7:Buffer info. Length = 50 : Offset = 14 
02:05:24/19:28:52|WV:8:Buffer info. Length = 48 : Offset = 16 
02:05:24/19:28:52|WV:9:Buffer info. Length = 46 : Offset = 18 
02:05:24/19:28:52|get_WAVEBlk (A) which = 1 ---- status = 0088, State machines = 0201
02:05:24/19:28:52|get_WAVEBlk (A) which = 0 ---- status = 0048, State machines = 0908
02:05:24/19:28:52|get_WAVEBlk (A) which = 1 ---- status = 0088, State machines = 0908
02:05:24/19:28:52|get_WAVEBlk (A) which = 0 ---- status = 0048, State machines = 0908
02:05:24/19:28:52|get_WAVEBlk (A) which = 1 ---- status = 0088, State machines = 0908
02:05:24/19:28:52|get_WAVEBlk (A) which = 0 ---- status = 0048, State machines = 0908
02:05:24/19:28:52|get_WAVEBlk (A) which = 1 ---- status = 0088, State machines = 0908
02:05:24/19:28:52|get_WAVEBlk (A) which = 0 ---- status = 1048, State machines = 0201
02:05:24/19:28:52|get_WAVEBlk (A) which = 1 ---- status = 0088, State machines = 0908
02:05:24/19:28:52|
Attachment 1: Screenshot_from_2024-04-23_14-22-04.png
Screenshot_from_2024-04-23_14-22-04.png
Attachment 2: aida04
23:04:24/11:48:07|TCP transfer library version 3.12
23:04:24/11:48:07|TCP socket send buffer was 16384 - now 221184
23:04:24/11:48:07|TCP socket receive buffer was 87380 - now 221184
23:04:24/11:48:07|TCP socket created OK - now connecting to 192.168.11.99 port 11004
23:04:24/11:48:07|Connected to 192.168.11.99 port 11004
23:04:24/11:48:07|connect xfer stream #0 (1)
23:04:24/11:48:07|Data Acquisition Statistics counters now cleared
23:04:24/11:48:07|ASIC DMA buffer size 2097152; WAVE DMA buffer size 1048576
23:04:24/11:48:07|Aida state now Going. AidaExecV10.0: Build Jul  6 2022_15:10:57 
23:04:24/11:48:07|Go (1)
23:04:24/11:48:07|Running Go_Electronics for aida04 with TSMaster aida01
23:04:24/11:48:07|ADCs Calibrated. 
23:04:24/11:48:07|Finished Go_Electronics for AIDA
23:04:24/11:48:08|executing generic Go_Merger
23:04:24/11:48:08|executing generic Go_DataStorageServer
23:04:24/11:48:08|completed generic doGo
23:04:24/11:48:08|do_GetState returned z=0 and 1
23:04:24/11:48:08|In RDOGo_Operate: Enabled Correlation, ASIC and discriminator readout
23:04:24/11:48:13|Also in RDOGo_Operate: Enabled waveform readout
23:04:24/11:48:13|get_WAVEBlk (A) which = 0 ---- status = 104A, State machines = 0201
23:04:24/11:48:13|WV:0:Buffer info. Length = 64 : Offset = 0 
23:04:24/11:48:13|WV:1:Buffer info. Length = 62 : Offset = 2 
23:04:24/11:48:13|WV:2:Buffer info. Length = 60 : Offset = 4 
23:04:24/11:48:13|WV:3:Buffer info. Length = 58 : Offset = 6 
23:04:24/11:48:13|WV:4:Buffer info. Length = 56 : Offset = 8 
23:04:24/11:48:13|WV:5:Buffer info. Length = 54 : Offset = 10 
23:04:24/11:48:13|WV:6:Buffer info. Length = 52 : Offset = 12 
23:04:24/11:48:13|WV:7:Buffer info. Length = 50 : Offset = 14 
23:04:24/11:48:13|WV:8:Buffer info. Length = 48 : Offset = 16 
23:04:24/11:48:13|WV:9:Buffer info. Length = 46 : Offset = 18 
23:04:24/11:48:13|get_WAVEBlk (A) which = 1 ---- status = 0188, State machines = 0201
23:04:24/11:48:13|get_WAVEBlk (A) which = 0 ---- status = 0048, State machines = 0908
23:04:24/11:48:13|get_WAVEBlk (A) which = 1 ---- status = 0088, State machines = 0908
23:04:24/11:48:13|get_WAVEBlk (A) which = 0 ---- status = 0048, State machines = 0908
23:04:24/11:48:14|get_WAVEBlk (A) which = 1 ---- status = 0088, State machines = 0908
23:04:24/11:48:14|get_WAVEBlk (A) which = 0 ---- status = 0048, State machines = 0908
23:04:24/11:48:14|get_WAVEBlk (A) which = 1 ---- status = 0088, State machines = 0908
23:04:24/11:48:14|get_WAVEBlk (A) which = 0 ---- status = 0048, State machines = 0908
23:04:24/11:48:14|get_WAVEBlk (A) which = 1 ---- status = 0088, State machines = 0908
23:04:24/11:48:14|executing generic doStop
23:04:24/16:58:30|Halt (1)
23:04:24/16:58:31|Halt (2)
23:04:24/16:58:32|Halt (3)
23:04:24/16:58:32|Halt (4)
23:04:24/16:58:33|Halt (5)
23:04:24/16:58:33|Halt (6)
23:04:24/16:58:34|Halt (7)
23:04:24/16:58:35|Halt (8)
23:04:24/16:58:35|Halt (9)
23:04:24/16:58:36|Halt (10)
23:04:24/16:58:36|Halt (11)
23:04:24/16:58:37|Halt (12)
23:04:24/16:58:37|Halt (13)
23:04:24/16:58:38|Halt (14)
23:04:24/16:58:39|Halt (15)
23:04:24/16:58:39|Halt (16)
23:04:24/16:58:40|Halt (17)
23:04:24/16:58:40|Halt (18)
23:04:24/16:58:41|Halt (19)
23:04:24/16:58:41|Action has not completed
23:04:24/16:58:42|Giving up waiting
23:04:24/16:58:42|disconnect xfer stream #0 (1)
23:04:24/16:58:42|disconnect xfer stream #0 (2)
23:04:24/16:58:43|disconnect xfer stream #0 (3)
23:04:24/16:58:43|disconnect xfer stream #0 (4)
23:04:24/16:58:44|disconnect xfer stream #0 (5)
23:04:24/16:58:44|disconnect xfer stream #0 (6)
23:04:24/16:58:45|disconnect xfer stream #0 (7)
23:04:24/16:58:45|disconnect xfer stream #0 (8)
23:04:24/16:58:46|disconnect xfer stream #0 (9)
23:04:24/16:58:47|disconnect xfer stream #0 (10)
23:04:24/16:58:47|disconnect xfer stream #0 (11)
23:04:24/16:58:48|disconnect xfer stream #0 (12)
23:04:24/16:58:48|disconnect xfer stream #0 (13)
23:04:24/16:58:49|disconnect xfer stream #0 (14)
23:04:24/16:58:49|disconnect xfer stream #0 (15)
23:04:24/16:58:50|disconnect xfer stream #0 (16)
23:04:24/16:58:51|disconnect xfer stream #0 (17)
23:04:24/16:58:51|disconnect xfer stream #0 (18)
23:04:24/16:58:52|disconnect xfer stream #0 (19)
23:04:24/16:58:52|Action has not completed
23:04:24/16:58:53|Giving up waiting
23:04:24/16:58:53|executing generic Stop_Merger
23:04:24/16:58:53|executing generic Stop_DataStorageServer
23:04:24/16:58:53|completed generic doStop
23:04:24/16:58:53|do_GetState returned z=0 and 1
23:04:24/16:58:53|do_GetState returned z=0 and 1
23:04:24/16:59:06|Transfer Error - : Connection reset by peer
23:04:24/16:59:15|send() failed: 
23:04:24/16:59:15|TCP transfer library version 3.12
23:04:24/16:59:15|TCP socket send buffer was 16384 - now 221184
23:04:24/16:59:15|TCP socket receive buffer was 87380 - now 221184
23:04:24/16:59:15|TCP socket created OK - now connecting to 192.168.11.99 port 11004
23:04:24/16:59:15|Transfer Error - : Connection refused
23:04:24/16:59:15|connect() failed:  
23:04:24/16:59:15|Aida state now Stopped. AidaExecV10.0_Jul  6 2022: Build 15:10:57 
23:04:24/16:59:15|do_GetState returned z=0 and 0
23:04:24/16:59:18|do_GetState returned z=0 and 0
23:04:24/17:00:09|do_GetState returned z=0 and 0
23:04:24/17:00:11|do_GetState returned z=0 and 0
23:04:24/17:00:19|do_GetState returned z=0 and 0
INIT: Switching to runlevel: 6
INStarting killall:  Unmounting NFS filesystems:  umount: /MIDAS: not mounted
23:04:24/17:00:43|[  OK  ]
23:04:24/17:00:43|Stopping rpcbind: [  OK  ]
23:04:24/17:00:45|Shutting down kernel logger: [  OK  ]
23:04:24/17:00:47|Shutting down system logger: [  OK  ]
23:04:24/17:00:47|Stopping xinetd: [  OK  ]
23:04:24/17:00:47|[  OK  ]
23:04:24/17:00:47|Sending all processes the TERM signal... xaida: release:
23:04:24/17:00:48|
23:04:24/17:00:48|AIDAMEM: aidamem_release:
23:04:24/17:00:48|xaida: release:
23:04:24/17:00:48|Sending all processes the KILL signal... 
23:04:24/17:00:53|Saving random seed:  
23:04:24/17:00:53|Syncing hardware clock to systxaida: open:
23:04:24/17:00:53|emxaida: release:
23:04:24/17:00:53| time xaida: open:
23:04:24/17:00:53|xaida: ioctl: called with command 536899587
23:04:24/17:00:53|xaida: ioctl: called with command 1076129801
23:04:24/17:00:53|RTCxaida: release:
23:04:24/17:00:53|_RD_TIME: Inappropriate ioctl for device
23:04:24/17:00:53|ioctl() to /dev/rtc to read the time failed.
23:04:24/17:00:53|
23:04:24/17:00:53|Please stand by while rebooting the system...
Sj5õ24/17:00:54|Restarting system.
23:04:24/17:00:56|U×®­Ë
                        )	$T,W¤']Z¥anuary 2017
Flash base address=FC000000
Set Flash to ASync Mode
XST_SUCCESS:00:56|
Finished copying zImage to RAM 
23:04:24/17:00:57|
Found 0 errors checking kernel image
23:04:24/17:00:58|VHDL version number 0X03350706 
Based on AIDA Bootloader version number 1.2.0 -- 16th August 2012
Starting LMK 3200 setup 
23:04:24/17:00:58|
Setting LMK03200 to standard clock settings -- External Clock 23Nov15
.... SPI Base Address=0x81400000
clk_control_reg=0x4
23:04:24/17:00:58|Next step is SPIconfig
Control 32(0x81400000)=0x180
SlaveSel(0x81400000)=0x3
Ctrl(0x81400000)=0xE6
Ctrl(0x81400000)=0x86
23:04:24/17:00:58|SPIconfig done now to set up the LMK3200 registers
23:04:24/17:00:58|LMK #0 : regInit[0]=0x80000000
23:04:24/17:00:58|LMK #0 : regInit[1]=0x10070600
23:04:24/17:00:58|LMK #0 : regInit[2]=0x60601
23:04:24/17:00:59|LMK #0 : regInit[3]=0x60602
23:04:24/17:00:59|LMK #0 : regInit[4]=0x60603
23:04:24/17:00:59|LMK #0 : regInit[5]=0x70624
23:04:24/17:00:59|LMK #0 : regInit[6]=0x70605
23:04:24/17:00:59|LMK #0 : regInit[7]=0x70606
23:04:24/17:00:59|LMK #0 : regInit[8]=0x70627
23:04:24/17:00:59|LMK #0 : regInit[9]=0x10000908
23:04:24/17:00:59|LMK #0 : regInit[10]=0xA0022A09
23:04:24/17:00:59|LMK #0 : regInit[11]=0x82800B
23:04:24/17:00:59|LMK #0 : regInit[12]=0x28C800D
23:04:24/17:00:59|LMK #0 : regInit[13]=0x830020E
23:04:24/17:00:59|LMK #0 : regInit[14]=0xC800180F
Calibrate completed at 943 counts
Setting Clock Control =0x0000000B, to set GOE and sync bit
Ctrl @ SPIstop (0x81400000)=0x186
Timeout waiting for Lock detect Stage 2 (Zero Delay), PWR_DWN=0x00000004
23:04:24/17:00:59|
Finished Clock setup LMK03200
completed LMK 3200 setup 
Loaded all four ASICs with default settings 
Setting the ADCs into calibration mode 
23:04:24/17:00:59|
Control 32(0x81400400)=0x180
SlaveSel(0x81400400)=0xFF
Ctrl(0x81400400)=0xE6
Ctrl(0x81400400)=0x86
Init : Config of AD9252 SPI ok
23:04:24/17:01:00|
Ctrl @ SPIstop (0x81400400)=0x186ADCs initialised 
Cal DCMs not locked 
ADC calibrate failed 
Jumping to kernel simpleboot...
23:04:24/17:01:00|
zImage starting: loaded at 0x00a00000 (sp: 0x00bc4eb0)
Allocating 0x3b78cc bytes for kernel ...
gunzipping (0x00000000 <- 0x00a0f000:0x00bc380e)...done 0x39604c bytes
23:04:24/17:01:03|
Linux/PowerPC load: console=ttyS0 root=/dev/nfs ip=on rw mem=112M
Finalizing device tree... flat tree at 0xbd1300
Probing IIC bus for MAC... MAC address = 0xd8 0x80 0x39 0x41 0xa0 0x71 
23:04:24/17:01:09|Using Xilinx Virtex440 machine description
23:04:24/17:01:10|Linux version 2.6.31 (nf@nnlxb.dl.ac.uk) (gcc version 4.2.2) #34 PREEMPT Tue Nov 15 15:57:04 GMT 2011
23:04:24/17:01:10|Zone PFN ranges:
23:04:24/17:01:10|  DMA      0x00000000 -> 0x00007000
23:04:24/17:01:10|  Normal   0x00007000 -> 0x00007000
23:04:24/17:01:10|Movable zone start PFN for each node
23:04:24/17:01:10|early_node_map[1] active PFN ranges
23:04:24/17:01:10|    0: 0x00000000 -> 0x00007000
23:04:24/17:01:10|MMU: Allocated 1088 bytes of context maps for 255 contexts
23:04:24/17:01:10|Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 28448
23:04:24/17:01:10|Kernel command line: console=ttyS0 root=/dev/nfs ip=on rw mem=112M
23:04:24/17:01:10|PID hash table entries: 512 (order: 9, 2048 bytes)
23:04:24/17:01:10|Dentry cache hash table entries: 16384 (order: 4, 65536 bytes)
23:04:24/17:01:11|Inode-cache hash table entries: 8192 (order: 3, 32768 bytes)
23:04:24/17:01:11|Memory: 109680k/114688k available (3500k kernel code, 4852k reserved, 144k data, 130k bss, 168k init)
23:04:24/17:01:11|Kernel virtual memory layout:
23:04:24/17:01:11|  * 0xffffe000..0xfffff000  : fixmap
23:04:24/17:01:11|  * 0xfde00000..0xfe000000  : consistent mem
23:04:24/17:01:11|  * 0xfde00000..0xfde00000  : early ioremap
23:04:24/17:01:11|  * 0xd1000000..0xfde00000  : vmalloc & ioremap
23:04:24/17:01:11|NR_IRQS:512
23:04:24/17:01:11|clocksource: timebase mult[a00000] shift[22] registered
23:04:24/17:01:11|Console: colour dummy device 80x25
23:04:24/17:01:11|Mount-cache hash table entries: 512
23:04:24/17:01:11|NET: Registered protocol family 16
23:04:24/17:01:11|PCI: Probing PCI hardware
23:04:24/17:01:11|bio: create slab <bio-0> at 0
23:04:24/17:01:11|NET: Registered protocol family 2
23:04:24/17:01:11|IP route cache hash table entries: 1024 (order: 0, 4096 bytes)
23:04:24/17:01:11|TCP established hash table entries: 4096 (order: 3, 32768 bytes)
23:04:24/17:01:11|TCP bind hash table entries: 4096 (order: 2, 16384 bytes)
23:04:24/17:01:11|TCP: Hash tables configured (established 4096 bind 4096)
23:04:24/17:01:11|TCP reno registered
23:04:24/17:01:12|NET: Registered protocol family 1
23:04:24/17:01:12|ROMFS MTD (C) 2007 Red Hat, Inc.
23:04:24/17:01:12|msgmni has been set to 214
23:04:24/17:01:12|io scheduler noop registered
23:04:24/17:01:12|io scheduler anticipatory registered
23:04:24/17:01:12|io scheduler deadline registered
23:04:24/17:01:12|io scheduler cfq registered (default)
23:04:24/17:01:12|Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
23:04:24/17:01:12|83e00000.serial: ttyS0 at MMIO 0x83e01003 (irq = 16) is a 16550
23:04:24/17:01:12|console [ttyS0] enabled
23:04:24/17:01:12|brd: module loaded
23:04:24/17:01:12|loop: module loaded
23:04:24/17:01:12|Device Tree Probing 'ethernet'
23:04:24/17:01:12|xilinx_lltemac 81c00000.ethernet: MAC address is now d8:80:39:41:a0:71
23:04:24/17:01:12|xilinx_lltemac 81c00000.ethernet: XLlTemac: using DMA mode.
23:04:24/17:01:12|XLlTemac: DCR address: 0x80
23:04:24/17:01:12|XLlTemac: buffer descriptor size: 32768 (0x8000)
23:04:24/17:01:12|XLlTemac: Allocating DMA descriptors with kmalloc
23:04:24/17:01:12|XLlTemac: (buffer_descriptor_init) phy: 0x6938000, virt: 0xc6938000, size: 0x8000
23:04:24/17:01:12|XTemac: PHY detected at address 7.
23:04:24/17:01:12|xilinx_lltemac 81c00000.ethernet: eth0: Xilinx TEMAC at 0x81C00000 mapped to 0xD1024000, irq=17
23:04:24/17:01:13|fc000000.flash: Found 1 x16 devices at 0x0 in 16-bit bank
23:04:24/17:01:13| Intel/Sharp Extended Query Table at 0x010A
23:04:24/17:01:13| Intel/Sharp Extended Query Table at 0x010A
23:04:24/17:01:13| Intel/Sharp Extended Query Table at 0x010A
23:04:24/17:01:13| Intel/Sharp Extended Query Table at 0x010A
23:04:24/17:01:13| Intel/Sharp Extended Query Table at 0x010A
23:04:24/17:01:13| Intel/Sharp Extended Query Table at 0x010A
23:04:24/17:01:13|Using buffer write method
23:04:24/17:01:13|cfi_cmdset_0001: Erase suspend on write enabled
23:04:24/17:01:13|cmdlinepart partition parsing not available
23:04:24/17:01:13|RedBoot partition parsing not available
23:04:24/17:01:13|Creating 5 MTD partitions on "fc000000.flash":
23:04:24/17:01:13|0x000000000000-0x000000500000 : "golden_firmware"
23:04:24/17:01:13|0x000000500000-0x000000800000 : "golden_kernel"
23:04:24/17:01:13|0x000000800000-0x000000d00000 : "user_firmware"
23:04:24/17:01:13|0x000000d00000-0x000000fe0000 : "user_kernel"
23:04:24/17:01:13|0x000000fe0000-0x000001000000 : "env_variables"
23:04:24/17:01:13|xilinx-xps-spi 81400400.hd-xps-spi: at 0x81400400 mapped to 0xD1028400, irq=20
23:04:24/17:01:13|SPI: XIlinx spi: bus number now 32766
23:04:24/17:01:13|xilinx-xps-spi 81400000.xps-spi: at 0x81400000 mapped to 0xD102C000, irq=21
23:04:24/17:01:14|SPI: XIlinx spi: bus number now 32765
23:04:24/17:01:14|mice: PS/2 mouse device common for all mice
23:04:24/17:01:14|Device Tree Probing 'i2c'
23:04:24/17:01:14| #0 at 0x81600000 mapped to 0xD1030000, irq=22
23:04:24/17:01:14|at24 0-0050: 1024 byte 24c08 EEPROM (writable)
23:04:24/17:01:14|TCP cubic registered
23:04:24/17:01:14|NET: Registered protocol family 17
23:04:24/17:01:14|RPC: Registered udp transport module.
23:04:24/17:01:14|RPC: Registered tcp transport module.
23:04:24/17:01:14|eth0: XLlTemac: Options: 0x3fa
23:04:24/17:01:14|eth0: XLlTemac: allocating interrupt 19 for dma mode tx.
23:04:24/17:01:14|eth0: XLlTemac: allocating interrupt 18 for dma mode rx.
23:04:24/17:01:15|eth0: XLlTemac: speed set to 1000Mb/s
23:04:24/17:01:17|eth0: XLlTemac: Send Threshold = 24, Receive Threshold = 4
23:04:24/17:01:17|eth0: XLlTemac: Send Wait bound = 254, Receive Wait bound = 254
23:04:24/17:01:17|Sending DHCP requests ., OK
23:04:24/17:01:18|IP-Config: Got DHCP answer from 192.168.11.99, my address is 192.168.11.4
23:04:24/17:01:18|IP-Config: Complete:
23:04:24/17:01:18|     device=eth0, addr=192.168.11.4, mask=255.255.255.0, gw=255.255.255.255,
23:04:24/17:01:18|     host=aida04, domain=dl.ac.uk, nis-domain=nuclear.physics,
23:04:24/17:01:18|     bootserver=192.168.11.99, rootserver=192.168.11.99, rootpath=/home/Embedded/XilinxLinux/ppc_4xx/rfs/aida04
23:04:24/17:01:18|Looking up port of RPC 100003/2 on 192.168.11.99
23:04:24/17:01:18|Looking up port of RPC 100005/1 on 192.168.11.99
23:04:24/17:01:18|VFS: Mounted root (nfs filesystem) on device 0:12.
23:04:24/17:01:18|Freeing unused kernel memory: 168k init
INIT: version 2.86 booting
23:04:24/17:01:19|Starting sysinit...
23:04:24/17:01:19|		Welcome to DENX & STFC Daresbury Embedded Linux Environment
23:04:24/17:01:20|		Press 'I' to enter interactive startup.
23:04:24/17:01:20|Setting clock  (utc): Tue Apr 23 16:01:20 BST 2024 [  OK  ]
23:04:24/17:01:20|Building the cache [  OK  ]
23:04:24/17:01:20|Setting hostname aida04:  [  OK  ]
23:04:24/17:01:21|Mounting local filesystems:  [  OK  ]
23:04:24/17:01:22|Enabling /etc/fstab swaps:  [  OK  ]
23:04:24/17:01:24|Finishing sysinit...
INIT: Entering runlevel: 3
... 257 more lines ...
  523   Mon Nov 13 11:46:54 2023 TDSummary of AIDA 24cm x 8cm 'triple' DSSSD tests
Attachment 1: Summary_of_AIDA_tests_2022-3_-_Dec_2023.pdf
Summary_of_AIDA_tests_2022-3_-_Dec_2023.pdf Summary_of_AIDA_tests_2022-3_-_Dec_2023.pdf Summary_of_AIDA_tests_2022-3_-_Dec_2023.pdf Summary_of_AIDA_tests_2022-3_-_Dec_2023.pdf Summary_of_AIDA_tests_2022-3_-_Dec_2023.pdf Summary_of_AIDA_tests_2022-3_-_Dec_2023.pdf Summary_of_AIDA_tests_2022-3_-_Dec_2023.pdf Summary_of_AIDA_tests_2022-3_-_Dec_2023.pdf
Attachment 2: Summary_of_AIDA_tests_2022-3_-_Dec_2023.pptx
  82   Fri Nov 1 15:24:40 2019 CA, TD, NHSummary - 29.10-1.11.19
3x MSL type BB18(DS)-1000 installed 

detector bias -160V, leakage current c. 1uA @ +21 deg C for all 3x DSSSDs

all FEE64 good event rates (slow comparator 0xa, LEC fast comparator 0xff) c. 120k, or less, typically 30-50k, overall merge rate c. 1.6M data item/s

See https://elog.ph.ed.ac.uk/DESPEC/70



Outstanding issues

- occasional loss of bits 48-63 of WR timestamp for aida09 - can be recovered by power cycle - replace HDMI cable?

- aida09 asic 1 not producing ADC or disc data - replace aida09? 
  ASIC Check works OK
  aida09 asic temp c. 30deg C < other asic temps

- evidence of c. 1MHz extrinsic noise for most FEE64s

- acquisition of waveform data not robust - most ASIC channels do not produce data or quickly stop producing data  - PJCS to review firmware rev for Jan/Feb 2020

- require spare DSSSDs, FEE64s, HDMI cables etc
  83   Fri Nov 1 15:45:04 2019 CA, TD, NHSummary - 29.10-1.11.19
>
> 
> - evidence of c. 1MHz extrinsic noise for most FEE64s


Is the 1MHz noise actually 1.58Mhz? as this is the frequency of observed noise at LYCCA.
  84   Fri Nov 1 18:09:03 2019 CA, TD, NHSummary - 29.10-1.11.19
> >
> > 
> > - evidence of c. 1MHz extrinsic noise for most FEE64s
> 
> 
> Is the 1MHz noise actually 1.58Mhz? as this is the frequency of observed noise at LYCCA.

Judge for yourself

https://elog.ph.ed.ac.uk/DESPEC/191031_125102/1350_18W.png

I would estimate c. 2.5 cycles in 2us => c. 1.2MHz … ?

Note that all of the waveforms are shown on an expanded scale c. 7000-9000 of 0-16383
so the amplitude is significantly less than that observed at LYCCA.

Tom
  707   Mon Jul 21 13:01:19 2025 MPStatus

Status before grounding checks

Attachment 1: Screenshot_from_2025-07-21_14-00-23.png
Screenshot_from_2025-07-21_14-00-23.png
Attachment 2: Screenshot_from_2025-07-21_14-00-39.png
Screenshot_from_2025-07-21_14-00-39.png
  708   Tue Jul 22 17:00:26 2025 MP, CCStatus

Test with shielded HV cables for both detectors.

The FWHM for ch1 is now 184 ch.

Yesterday cables for cards 5, 6, 10, 11, 12, 14, 16 were partially connected (only 1 out of 2 flat cables was in), now both are connected.

Attachment 1: Screenshot_from_2025-07-22_17-51-52.png
Screenshot_from_2025-07-22_17-51-52.png
Attachment 2: Screenshot_from_2025-07-22_17-59-20.png
Screenshot_from_2025-07-22_17-59-20.png
ELOG V3.1.4-unknown