Two further comments
1) The 'pristine' Options files produced by OH fixed *all* of the undefined spectra issues,
i.e. we now have *L, *H, Rate, Stat, Hit, HitRate spectra for all FEE64s
2) The system has undegone 1x power cycle/reset/setup and 1x reset/setup cycle since the
new Options files were installed and we currently have 2x errors reported in aida02.
There is an open Options tab and we have viewed the Options settings for aida01 and aida02
(currently viewing aida02) - *no* changes/saves/restores requested.
>
> Rename Options directory
>
> mv Options Options.BAK-130421
>
> Copy Options.PRISTINE to Options
>
> cp -r Options.PRISTINE Options
> chmod -R 777 Options
>
> Start Options directory monitor (will check directory every 5 minutes)
>
> Python ~/oh/OptionsCheck
>
>
> 17.32 Power cycle FEE64s
>
>
> FEE64 module aida09 global clocks failed, 6
> Clock status test result: Passed 11, Failed 1
>
> Understand status as follows
> Status bit 3 : firmware PLL that creates clocks from external clock not locked
> Status bit 2 : always logic '1'
> Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
> Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
> If all these bits are not set then the operation of the firmware is unreliable
>
> Calibration test result: Passed 12, Failed 0
>
> If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
>
> White Rabbit error counter test result: Passed 12, Failed 0
>
> Understand the status reports as follows:-
> Status bit 3 : White Rabbit decoder detected an error in the received data
> Status bit 2 : Firmware registered WR error, no reload of Timestamp
> Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
>
>
> FPGA Timestamp error counter test result: Passed 12, Failed 0
> If any of these counts are reported as in error
> The ASIC readout system has detected a timeslip.
> That is the timestamp read from the time FIFO is not younger than the last
>
>
>
> Returned 0 0 0 0 0 0 0 0 0 0 0 0
> Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
> aida01 : 3 5 2 2 1 3 3 2 2 3 8 : 43348
> aida02 : 2 3 0 2 1 2 2 3 2 3 8 : 43424
> aida03 : 3 4 0 1 1 2 2 3 2 3 8 : 43404
> aida04 : 1 4 2 0 3 3 3 2 2 3 8 : 43396
> aida05 : 1 4 2 2 2 3 3 2 2 3 8 : 43396
> aida06 : 2 3 2 1 1 2 2 3 2 3 8 : 43424
> aida07 : 4 1 3 3 1 3 1 3 2 3 8 : 43368
> aida08 : 1 4 2 2 2 3 3 2 2 3 8 : 43396
> aida09 : 2 3 3 3 2 4 2 2 2 3 8 : 43312
> aida10 : 1 3 1 2 2 3 2 2 2 3 8 : 43116
> aida11 : 1 3 3 1 2 3 2 2 2 3 8 : 43116
> aida12 : 3 2 1 0 1 2 1 3 2 3 8 : 43116
>
>
> Note that system wide checks identify 1x file of different size immediately after boot/reset/setup
>
> Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same
>
> FEE : aida01 => Options file size is 1014 Last changed Tue Apr 13 17:59:03 CEST 2021
> FEE : aida02 => Options file size is 1025 Last changed Tue Apr 13 17:59:04 CEST 2021
> FEE : aida03 => Options file size is 1014 Last changed Tue Apr 13 17:59:03 CEST 2021
> FEE : aida04 => Options file size is 1014 Last changed Tue Apr 13 17:59:03 CEST 2021
> FEE : aida05 => Options file size is 1014 Last changed Tue Apr 13 17:59:04 CEST 2021
> FEE : aida06 => Options file size is 1014 Last changed Tue Apr 13 17:59:03 CEST 2021
> FEE : aida07 => Options file size is 1014 Last changed Tue Apr 13 17:59:03 CEST 2021
> FEE : aida08 => Options file size is 1014 Last changed Tue Apr 13 17:59:03 CEST 2021
> FEE : aida09 => Options file size is 1014 Last changed Tue Apr 13 17:59:03 CEST 2021
> FEE : aida10 => Options file size is 1014 Last changed Tue Apr 13 17:59:04 CEST 2021
> FEE : aida11 => Options file size is 1014 Last changed Tue Apr 13 17:59:03 CEST 2021
> FEE : aida12 => Options file size is 1014 Last changed Tue Apr 13 17:59:05 CEST 2021
>
> 06:25:00 PM
> FAULTS!
>
> Unknown option in FEE2 Options database
> Unknown entry starts:
>
da_Hist_W_Enable&&TS_SYNC_PHASE&&ExtClk&&Aida.shift&&MACB_TRIG_MODE&&Aida.offset&&RunNumber&&Aida_GroupBase&&Rate.channels&&Stat.channels&&Aida.Vchannels&&ASIC.settings&&WAV
E_DMA_HWM&&Ai
> da_Hist_D_Enable&&Stat.shift&&Aida.Wchannels&&Aida_Hist_V_Enable&&Include.Aida&&Aida.channels&&DataAcqPgm&&Aida_Hist_H_Enable&&DataFormat&&ASIC_DMA_HWM&&Aida_Hist_L_Enable
>
> 06:30:00 PM
> FAULTS!
>
> Unknown option in FEE2 Options database
> Unknown entry starts:
>
da_Hist_W_Enable&&TS_SYNC_PHASE&&ExtClk&&Aida.shift&&MACB_TRIG_MODE&&Aida.offset&&RunNumber&&Aida_GroupBase&&Rate.channels&&Stat.channels&&Aida.Vchannels&&ASIC.settings&&WAV
E_DMA_HWM&&Ai
> da_Hist_D_Enable&&Stat.shift&&Aida.Wchannels&&Aida_Hist_V_Enable&&Include.Aida&&Aida.channels&&DataAcqPgm&&Aida_Hist_H_Enable&&DataFormat&&ASIC_DMA_HWM&&Aida_Hist_L_Enable
>
> Unknown option in FEE2 Options database
> Unknown entry starts:
>
et&&Aida_Hist_W_Enable&&TS_SYNC_PHASE&&ExtClk&&Aida.shift&&MACB_TRIG_MODE&&Aida.offset&&RunNumber&&Aida_GroupBase&&Rate.channels&&Stat.channels&&Aida.Vchannels&&ASIC.setting
s&&WAVE_DMA_H
>
WM&&Aida_Hist_D_Enable&&Stat.shift&&Aida.Wchannels&&Aida_Hist_V_Enable&&Include.Aida&&Aida.channels&&DataAcqPgm&&Aida_Hist_H_Enable&&DataFormat&&ASIC_DMA_HWM&&Aida_Hist_L_En
able
>
>
>
>
>
> FEE64 module aida01 => 7
> FEE64 module aida02 => 7
> FEE64 module aida03 => 7
> FEE64 module aida04 => 7
> FEE64 module aida05 => 7
> FEE64 module aida06 => 7
> FEE64 module aida07 => 7
> FEE64 module aida08 => 7
> FEE64 module aida09 => 7
> FEE64 module aida10 => 7
> FEE64 module aida11 => 7
> FEE64 module aida12 => 7
>
>
> ASIC Clock Timestamp check test result: Passed 12, Failed 0
>
>
> If any of the above values are different from the rest then check the clocks and the White Rabbit decoder status
>
>
>
>
> aida01 timestamp memory No stop found : 0x0
> 0 0 0 0 0
> 4 0 0 0 0
> 8 0 0 0 0
> 12 0 0 0 0
> 16 0 0 0 0
> 20 0 0 0 0
> 24 0 0 0 0
> 28 0 0 0 0
>
> aida02 timestamp memory No stop found : 0x0
> 0 0 0 0 0
> 4 0 0 0 0
> 8 0 0 0 0
> 12 0 0 0 0
> 16 0 0 0 0
> 20 0 0 0 0
> 24 0 0 0 0
> 28 0 0 0 0
>
> aida03 timestamp memory No stop found : 0x0
> 0 0 0 0 0
> 4 0 0 0 0
> 8 0 0 0 0
> 12 0 0 0 0
> 16 0 0 0 0
> 20 0 0 0 0
> 24 0 0 0 0
> 28 0 0 0 0
>
> aida04 timestamp memory No stop found : 0x0
> 0 0 0 0 0
> 4 0 0 0 0
> 8 0 0 0 0
> 12 0 0 0 0
> 16 0 0 0 0
> 20 0 0 0 0
> 24 0 0 0 0
> 28 0 0 0 0
>
> aida05 timestamp memory No stop found : 0x0
> 0 0 0 0 0
> 4 0 0 0 0
> 8 0 0 0 0
> 12 0 0 0 0
> 16 0 0 0 0
> 20 0 0 0 0
> 24 0 0 0 0
> 28 0 0 0 0
>
> aida06 timestamp memory No stop found : 0x0
> 0 0 0 0 0
> 4 0 0 0 0
> 8 0 0 0 0
> 12 0 0 0 0
> 16 0 0 0 0
> 20 0 0 0 0
> 24 0 0 0 0
> 28 0 0 0 0
>
> aida07 timestamp memory No stop found : 0x0
> 0 0 0 0 0
> 4 0 0 0 0
> 8 0 0 0 0
> 12 0 0 0 0
> 16 0 0 0 0
> 20 0 0 0 0
> 24 0 0 0 0
> 28 0 0 0 0
>
> aida08 timestamp memory No stop found : 0x0
> 0 0 0 0 0
> 4 0 0 0 0
> 8 0 0 0 0
> 12 0 0 0 0
> 16 0 0 0 0
> 20 0 0 0 0
> 24 0 0 0 0
> 28 0 0 0 0
>
> aida09 timestamp memory No stop found : 0x0
> 0 0 0 0 0
> 4 0 0 0 0
> 8 0 0 0 0
> 12 0 0 0 0
> 16 0 0 0 0
> 20 0 0 0 0
> 24 0 0 0 0
> 28 0 0 0 0
>
> aida10 timestamp memory No stop found : 0x0
> 0 0 0 0 0
> 4 0 0 0 0
> 8 0 0 0 0
> 12 0 0 0 0
> 16 0 0 0 0
> 20 0 0 0 0
> 24 0 0 0 0
> 28 0 0 0 0
>
> aida11 timestamp memory No stop found : 0x0
> 0 0 0 0 0
> 4 0 0 0 0
> 8 0 0 0 0
> 12 0 0 0 0
> 16 0 0 0 0
> 20 0 0 0 0
> 24 0 0 0 0
> 28 0 0 0 0
>
> aida12 timestamp memory No stop found : 0x0
> 0 0 0 0 0
> 4 0 0 0 0
> 8 0 0 0 0
> 12 0 0 0 0
> 16 0 0 0 0
> 20 0 0 0 0
> 24 0 0 0 0
> 28 0 0 0 0
>
> 17.14 rate spectra - attachment 1
>
> 1.8.L spectra - attachments 2 & 3
> pulser peak width aida06 c.75 ch FWHM, aida07 c.85 ch FWHM
>
> 1.8.W spectra - attachments 4 & 5
>
> good events statistics (slow comparator 0xa) - attachment 6
>
> FEE64 temperatures OK - attachment 7
>
> DSSSD bias & leakage currents OK - attachment 8
>
>
>
>
> 17.38 DAQ starts file NULL/R14
> slow comparator 0xa -> 0x64
> BNC PB-5 pulser OFF
> alpha background
>
> 10:40 CET 14.04.2021 - NH
> Close file NULL/R14 |