 Wed Jan 23 08:12:01 2019, CA, TD, NH, VP, January 22nd 2019 
|
Attachment 1 - 1.8.W waveform spectra with pulser signal
Attachment 2 - 1.8.L pulser peak spectra
widths: aida01 - 16.06
aida04 - 20.68
aida05 - 14.03
aida08 - 73.02
aida09 - 669.29
aida11 - 405.35
aida12 - 397.6
Attachment 3 - good event statistics
fast comparator threshold (LEC) at 0xff
FEEs 8,9,11,12 running hot
Attachment 4 - FEE64 temperatures - all ok
System wide checks ok, except aida02 fails ADC calibration
Leak found in seal on cooling pipe, switched power off |
|
Wed Jan 23 09:25:55 2019, CA, TD, NH, VP, MACB and setup pics 23.01.19 11x
|
Pictures of leaking cooling pipe
MACB switch settings
AIDA mount - needs rotated
Attachment 11 - AIDA@DESPEC MACB settings - 100MHz and WR |
|
Wed Jan 23 17:03:08 2019, CA, TD, NH, VP, January 23rd 2019 6x
|
Post White Rabbit update
Attachment 1 - 1.8.W pulser waveform spectra for all FEE64
- pulser signals observed in all FEE64
Attachment 2 - good event statistics - aida08 has significant noise
Attachment 3 - FEE64 temperatures - all ok, but firmware version not displayed
Attachment 4 - 1.8.L pulser peak spectra
- widths:
aida01 - 23.66
aida02 - 19.76
aida03 - 20.43
aida04 - 18.82
aida05 - 20.05
aida06 - 41.04
aida07 - 21.24
aida08 - 205.02 (noisy FEE64)
aida09 - 24.81
aida10 - 22.85
aida11 - 13.98
aida12 - 19.90
Attachment 5 - New Merger statistics
Attachment 6 - Raw data stream |
|
Thu Jan 24 10:21:26 2019, CA, TD, NH, VP, January 24th 2019 7x
|
10.30 AIDA setup and DAQ start complete
writing to file TapeData/NULL/R1
FEE64 Temperatures and Good event statistics ok (attachments 1 and 2)
11.00 results of analyser - shows significant number of timewarps in aida07, and a few in aida01-04
11.24 compared timestamps between mbs and AIDA, agree to 8 digits (attachment 3)
12.49 DAQ and Merger stopped and restarted
Raised slow comparator threshold from 0xa to 0x64
Raised fast comparator threshold (LEC/MEC) from 0xf to 0xff
12.51 Good event statistics: aida08 running at a normal rate
pulser peak (1.8.L) no longer has noise shoulder (attachment 4)
writing to file TapeData/NULL/R2
13.58 stopped writing to file. Pulser frequency increased to 1 kHz
writing to file TapeData/NULL/R3
15.00 file R4 - DAQ startup
17.58 trimmed and re-crimped ground wire on aida08
lowered pulser freq back to 50Hz
reduced slow comparator threshold to 0xa, fast comparator threshold (LEC) to 0xf
good event stats ok, aida08 no longer running hot (attachment 6)
1.8.L and 1.8.W spectra of aida08 (attachment 5)
pulser peak width now at 55.71
FEE64 temperatures ok (attachment 7) |
|
Thu Jan 24 17:16:32 2019, CA, TD, NH, VP, MACB time switch settings
|
AIDA@DESPEC MACB time switch settings before and after WR are as follows.
Attachment 1 - MACB 1-4 from left to right
MACB number: 1 2 3 4
100MHZ clock setting (before WR): 0 2 2 2
White Rabbit setting (after WR): 3 3 3 3 |
|
Fri Jan 25 09:54:12 2019, CA, TD, NH, VP, 25th January 2019 8x
|
09.30: AIDA still running, not writing to file
Still running ok
No water leaks
09.45: FEE64 temperatures ok (attachment 1)
Good event statistics ok (attachment 2)
10.00: Results of System Wide Checks
ADC Calibration check (attachment 3)
Clock Status (attachment 4)
Sync error counter (attachment 5)
White Rabbit Decoder status (attachment 7)
Memory Information (attachment 8)
10.30: Clock ReSync performed (attachment 6)
Start writing to file TapeData/NULL/R7_0
11.50: DAQ stopped, TapeServer stopped
TapeServer restarted, DAQ restarted
Writing to file TapeData/NULL/R9_0 with Clock ReSync at start.
Wasn't writing to disk, restarting (DAQ stop, TapeServer stop, TapeServer start TO DISK, DAQ Start)
Writing to file TapeData/NULL/R10_0
12.04: DAQ stopped, TapeServer stopped, merger stopped
FEE64 power relay switched off
Raspberry PIs switched off and disconnected
water cooler stopped
Mains IEC cables removed from relay
|
|
Wed Oct 30 08:06:49 2019, CA, TD, NH, DSSSD stack 6x
|
Ribbon cables fitted with kapton PCB couplers -> 3 isolated + 1 non-isolated for each cable length/DSSSD
Ribbon cable lengths:
without kapton PCB coupler (cm) - 40, 47, 51.5
with kapton PCB coupler (cm) - 50.5, 57.5, 62
gap between plastic and dsssd - 15 mm
length between top of snout and dsssd - 5cm |
|
Wed Oct 30 11:51:08 2019, CA, TD, NH, Detector biasing
|
12.51 - detector bias and leakage currents - detector 3 has no current
- other detectors ok
13.13 - tried changing from HV3 cables to HV4 - still open circuit
- tried different permutations of moving HV-4 braid and core cables (high voltage to ground connections)
(dssd3)
(all directions defined as if looking upstream)
- core -> bottom, braid -> left open - circuit
- core -> top, braid -> right trips at 8V
- core -> bottom, braid -> right open - circuit
- core -> top, braid-> left trips at 8V
- removed jumpers - still trips
13.36 - testing continuity of bond wires on dsssd 3208-14 & 3208-15
- observe 20 ohm resistance on p+p and p+n bond wires
19:41 - dsssd 1 and 2 not biasing, 3 ok.
- changed bias configuration -> HV core 1 & 2 now on top, braid on right
- ribbon cables for DSSSD #1 & #2 p+n junction strips down were misaligned
- all detectors biased succesfully
HV cable configuration (looking upstream): HV-1 Core -> top-inner (FEE 1)
HV-2 Core -> top-middle (FEE 5)
HV-3 Core -> bottom-outer (FEE 11)
HV-1 Braid -> right-inner (FEE 2)
HV-2 Braid -> right-middle (FEE 6)
HV-3 Braid -> left-outer (FEE 12)
Attachment 3 - rough sketch of HV cable configuration, looking upstream.
|
|
Thu Oct 31 09:00:58 2019, CA, TD, NH, Thursday 31 October 14x
|
10.05 DSSSD # 1-3 detector biases & leakage currents OK - see attachment 1
Ambient temperature +21.2 deg C, d.p. +3.4 deg C, RH 31.2%
10.15 DSSSD stack as follows
DSSSD serial thickness depletion
# (um) voltage (V)
1 3208-7 1019 125 upstream
2 3208-13 1020 120
3 3208-14 1020 130 downstream
11.30 AIDA power relays sequenced on
DAQ start
system wide checks ok
11.33 attachment 2 - fee temperatures - ok
attachment 3 - good event statistics with slow comparator threshold = 100
attachment 4 - good event statistics with slow comparator threshold = 50
attachment 5 - good event statistics with slow comparator threshold = 20
attachment 6 - good event statistics with slow comparator threshold = 10
13.51 attachment 7 - Rates
- aida09 1 ASIC not working (no data)
attachment 8 - good event statistics
attachments 9/10 - 1.8.L spectra
- no peak observed in aida09
attachments 11/12 - 1.8.W spectra
- waveform only observed in naida 2,4,7,8,12
pulser peak widths - FEE width(ch)
1 71
2 96
3 66
4 79
5 74
6 96
7 89
8 83
9 no peak
10 82
11 153
12 120
17.26 FATIMA moved upstream around AIDA
attachment 13 - good event statistics
ASIC check-load performed
attachment 14 - good event statistics following checkload |
|
Fri Nov 1 10:46:13 2019, CA, TD, NH, Friday 1st November 2019 6x
|
11.46 - attachments 1,2,3: bias/leakage currents, good event statistics and fee temperatures (respectively)
before removing aluminium foil upstream of AIDA
note - slow comparator threshold 0x64 (alpha background), LEC fast comparator threshold 0xff, pulser OFF
11.52 - attachments 4,5,6: bias/leakage currents, good event statistics and fee temperatures (respectively)
after removing aluminium foil upstream of AIDA
- base of AIDA snout has Mylar shielding |
|
Fri Nov 1 15:24:40 2019, CA, TD, NH, Summary - 29.10-1.11.19
|
3x MSL type BB18(DS)-1000 installed
detector bias -160V, leakage current c. 1uA @ +21 deg C for all 3x DSSSDs
all FEE64 good event rates (slow comparator 0xa, LEC fast comparator 0xff) c. 120k, or less, typically 30-50k, overall merge rate c. 1.6M data item/s
See https://elog.ph.ed.ac.uk/DESPEC/70
Outstanding issues
- occasional loss of bits 48-63 of WR timestamp for aida09 - can be recovered by power cycle - replace HDMI cable?
- aida09 asic 1 not producing ADC or disc data - replace aida09?
ASIC Check works OK
aida09 asic temp c. 30deg C < other asic temps
- evidence of c. 1MHz extrinsic noise for most FEE64s
- acquisition of waveform data not robust - most ASIC channels do not produce data or quickly stop producing data - PJCS to review firmware rev for Jan/Feb 2020
- require spare DSSSDs, FEE64s, HDMI cables etc |
 Fri Nov 1 15:45:04 2019, CA, TD, NH, Summary - 29.10-1.11.19
|
>
>
> - evidence of c. 1MHz extrinsic noise for most FEE64s
Is the 1MHz noise actually 1.58Mhz? as this is the frequency of observed noise at LYCCA. |
|
Fri Nov 1 18:09:03 2019, CA, TD, NH, Summary - 29.10-1.11.19
|
> >
> >
> > - evidence of c. 1MHz extrinsic noise for most FEE64s
>
>
> Is the 1MHz noise actually 1.58Mhz? as this is the frequency of observed noise at LYCCA.
Judge for yourself
https://elog.ph.ed.ac.uk/DESPEC/191031_125102/1350_18W.png
I would estimate c. 2.5 cycles in 2us => c. 1.2MHz … ?
Note that all of the waveforms are shown on an expanded scale c. 7000-9000 of 0-16383
so the amplitude is significantly less than that observed at LYCCA.
Tom |
|
Sat Mar 13 07:08:52 2021, CA, TD, LS, March 13th 08:00 - 17:00 20x
|
ASIC settings 2019Dec19-16.19.51
DSSSD#1 slow comparator 0xa
DSSSD#2 slow comparator 0xa
DSSSD#3 slow comparator 0xd
BNC PB-5 Pulser
Amplitude1.0V
Attenuation x1
Frequency 2Hz
tau_d 1ms
- polarity
Delay 250ns, tail pulse
08:09 System wide checks ok *except*
aida09 fails clock check
aida09 calibration failed
08:14 FEE64 temperatures ok - attachment 1
good event statistics ok - attachment 2
detector bias/leakage current ok - attachment 3
08:22 Merger 4.3M items/s
Tapeserver 14 MB/s
08:24 DAQ ok, writing to file R43_45
08:39 rate spectra - attachment 4
aida09 spectrum not showing, however continues to collect statistics ok
10:03 System wide checks ok *except*
aida09 fails clock check
aida09 calibration failed
FEE64 temperatures ok - attachment 5
good event statistics ok - attachment 6
detector bias/leakage current ok - attachment 7
10:10 Merger 4.5M items/s
Tapeserver 14 MB/s
data forwarding to MBS ok
10:11 writing to file R43_91
12.00 (LS)
System wide checks:
FEE64 module aida09 global clocks failed, 6
Clock status test result: Passed 11, Failed 1
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable
FEE64 module aida09 failed
Calibration test result: Passed 11, Failed 1
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun
calibration for that module
Base Current Difference
aida07 fault 0x829e : 0x829f : 1
White Rabbit error counter test result: Passed 11, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
no FGPA timestamp errors all passed
Statistics (attachment8)
Spectra rate (attachment9), aida09 still does not show histogram, but on run control are enabled for all fees. still
showing good events and other information (temperatures etc)
FEE temps (attachment10)
Leakage currents written to sheets (attachment11)
Merger~4.6M items/s
Tapeserver~14MB/s
12.36 bunch of bad timestamp errors in the new merger terminal (attachment 12) around file R43_152
this error is still occurring at 12.45 will continue to monitor, all other system checks are reporting as normal
13.50 beam down file R43_183
beam back a couple of minutes later
14.00 System wide checks:
FEE64 module aida09 global clocks failed, 6
Clock status test result: Passed 11, Failed 1
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable
FEE64 module aida09 failed
Calibration test result: Passed 11, Failed 1
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun
calibration for that module
no white rabbit or FPGA errors all passed
Statistics (attachment13)
Spectra rate (attachment14)
FEE temps (attachment15)
Leakage currents written to sheets (attachment16)
Merger~4.6M items/s
Tapeserver~14MB/s
Writing to MBS okay, still seeing bad timestamp errors in the new merger terminal continuing to monitor and update
statistics tab every 20 to 30 minutes
14.12 Analysed files R43_185,186,187 from timestamp error in ucesb, but see no timewarps
14.30 have not seen any bad timestamp errors in new merger terminal for a while, unsure if something has been changed which
has stopped them
15.25 beam down
15.53 beam back
16.00 moved back to writing to /media/SecondDrive which should last until the end of the experiment
now writing R46
16.10 System wide checks okay except:
FEE64 module aida09 global clocks failed, 6
Clock status test result: Passed 11, Failed 1
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable
FEE64 module aida09 failed
Calibration test result: Passed 11, Failed 1
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun
calibration for that module
Statistics (attachment17)
Spectra rate (attachment18)
FEE temps (attachment19)
Leakage currents written to sheets (attachment20)
Merger~4.7M items/s
Tapeserver~14MB/s
Writing to MBS okay |
|
Mon Dec 10 11:09:53 2018, CA, TD, Scintillator pictures/system checks for 07.12.18 14x
|
11.12 System wide checks ok (attachment 1)
FEE temperatures ok (attachment 2)
bias/leakage currents ok (attachment 3)
11.13 1.8.W spectra for naida 1,9,10,11,12 (attachment 4)
1.8.W spectra expanded in Y direction (attachment 5)
pulser peaks (attachment 6)
11.20 pulser peak widths:
Aida10 -> 98.82
Aida09 -> 109.17
Aida11 -> 71.72
Aida12 -> 92.82
Aida01 -> 24.05
11.23 hit rates (attachment 7)
11.41 pictures of setup with scintillator mounted (attachment 8-14) |
|
Mon Apr 1 08:50:35 2019, CA, TD, Monday April 1st 2019 - BEAM START 18x
|
09.28 System wide checks - aida06 global clock failure (check clock status)
- aida02 , aida04, aida06 fail calibration (ADC calibration)
- SYNC error counter ok
- SYNC pulse check ok
- 'collect memory information from FEE64 Linux' still returns scripting error
- WR decoder test passes
09.33 good event statistics ok, except aida09, aida10 running hot. (attachment 1)
FEE temperatures ok (attachment 2)
detector bias/leakage current ok (attachment 3)
09.38 screenshots of NewMerger tab, Merger/Tape HTTPd and NewMerger terminal (attachments 4, 5, 6)
10.05 1.8.L pulser peak spectra (attachment 7)
10.13 1.8.W waveform spectra (attachment 8)
21.13 Beam on
AIDA DAQ running - writing to file R9 in directory 290319
System wide checks - same as above
good event statistics ok - although rates on 9,10,11,12 have dropped (attachment 10)
ASIC check load performed
FEE temperatures ok (attachment 9)
detector bias leakage currents ok (attachment 11)
Scintillator 41 -> approx 1000 ions/s, 11 second super cycle (beam on 10s, beam off 1s)
c. 130-150MeV/u 107Ag ions incident on plastic scintillator + AIDA DSSSD
21.36 High energy spectra for aida09 - (attachment 12)
High energy spectra for aida11 - (attachment 13)
Low energy spectra - (attachment 14)
1D hit pattern - (attachment 15)
21.53 ASIC control settings - see for slow comparator threshold, fast comparator threshold (HEC) and fast
comparator threshold (LEC) (attachment 16)
NewMerger screenshot - working (attachment 17)
22.00 ASIC options - including directory path to saved ASIC settings (attachment 18)
22.09 beam stop
TapeServer stopped, restarted not writing to storage
one complete file produced R9_0 (2.0 GB)
plus R9_1 (154 MB)
*NOTE FROM 02/04/19 - DEGRADER in front of detector, silver beam not observed, just unidentified light ions. |
|
Wed Apr 3 15:24:06 2019, CA, TD, report - low inquire save state
|
On Save/Restore Module Settings Tab - "inquire save state" returns an error. See attachment 1. |
|
Mon Mar 8 09:45:20 2021, CA, TD, Analysis R7_20 (new version of NewMerger with min info code 4 & 5 data items)
|
An analysis of file R6_70 can be found at https://elog.ph.ed.ac.uk/DESPEC/177 attachment 19
We observe c. 80M ADC data items per 2Gb file and c. 160M info code 4 & 5 data items.
This am NH switched NewMerger for a new version which minimises the number of info code 4 & 5
data items. The analysis of one of these data files (R7_20) is appended - attachment 1. We
observe c. 2M ADC data items per 2Gb file. The number of info code 4 data items is significantly
reduced as expected but info code 5 data items are not observed.
Info code 4 & 5 data items will increment every (2^28 * 1e-9) 0.268s and (2^48 * 1e-9) 3.26d.
If we examine the verbose output (attachment 2) we observe that each data block only contains
c. 60 ADC data items - a small fraction of the capacity of each data block. |
|
Thu May 13 07:10:17 2021, CA, TD, May 13th 08:00 - 16:00 shift 34x
|
08:10 all system wide checks ok *except*
all FEE64 modules fail ADC calibration
WR decoder status:
Base Current Difference
aida01 fault 0xf932 : 0xf933 : 1
aida02 fault 0x62ec : 0x62ed : 1
aida03 fault 0x8679 : 0x867a : 1
aida04 fault 0xf0e4 : 0xf0e5 : 1
aida05 fault 0x9db8 : 0x9db9 : 1
aida06 fault 0x7f18 : 0x7f19 : 1
aida07 fault 0xdd2c : 0xdd2d : 1
aida08 fault 0x1557 : 0x1558 : 1
White Rabbit error counter test result: Passed 8, Failed 8
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
08:16 attempt to recalibrate ADCs using FADC align and control - unsuccessful on all
08:18 FEE64 temps ok - attachment 1
bias/leakage currents ok - attachment 2
statistics - attachments 3-8
10:13 all system wide checks ok *except*
all FEE64 modules fail ADC calibration
WR decoder status:
Base Current Difference
aida01 fault 0xf932 : 0xf933 : 1
aida02 fault 0x62ec : 0x62ed : 1
aida03 fault 0x8679 : 0x867a : 1
aida04 fault 0xf0e4 : 0xf0e5 : 1
aida05 fault 0x9db8 : 0x9db9 : 1
aida06 fault 0x7f18 : 0x7f19 : 1
aida07 fault 0xdd2c : 0xdd2d : 1
aida08 fault 0x1557 : 0x1558 : 1
White Rabbit error counter test result: Passed 8, Failed 8
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
10:16 FEE64 temps ok - attachment 9
bias/leakage currents ok - attachment 10
statistics - attachments 11-16
12:16 all system wide checks ok *except*
all FEE64 modules fail ADC calibration
WR decoder status:
Base Current Difference
aida01 fault 0xf932 : 0xf933 : 1
aida02 fault 0x62ec : 0x62ed : 1
aida03 fault 0x8679 : 0x867a : 1
aida04 fault 0xf0e4 : 0xf0e5 : 1
aida05 fault 0x9db8 : 0x9dba : 2
aida06 fault 0x7f18 : 0x7f19 : 1
aida07 fault 0xdd2c : 0xdd2d : 1
aida08 fault 0x1557 : 0x1558 : 1
White Rabbit error counter test result: Passed 8, Failed 8
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
12:16 FEE64 temps ok - attachment 17
bias/leakage currents ok - attachment 18
statistics - attachments 19-25
14:08 all system wide checks ok *except*
all FEE64 modules fail ADC calibration
WR decoder status:
Base Current Difference
aida01 fault 0xf932 : 0xf933 : 1
aida02 fault 0x62ec : 0x62ed : 1
aida03 fault 0x8679 : 0x867a : 1
aida04 fault 0xf0e4 : 0xf0e5 : 1
aida05 fault 0x9db8 : 0x9dba : 2
aida06 fault 0x7f18 : 0x7f19 : 1
aida07 fault 0xdd2c : 0xdd2d : 1
aida08 fault 0x1557 : 0x1558 : 1
White Rabbit error counter test result: Passed 8, Failed 8
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
FEE64 temps ok - attachment 16
bias/leakage currents ok - attachment 27
statistics - attachments 28-35 |
|
Thu Apr 11 15:28:10 2019, CA, OH, CB, TD, Offline analysis of 290319 files R13 and R16 6x
|
X projection of high energy channel ADC hits from R13 (attachment 1)
XY hit pattern from R13 (attachment 2)
-discontinuity observed
-possible timing issues or issues with Event Clustering
X projection of high energy channel ADC hits from R16 (attachment 3)
XY hit pattern from R16 (attachment 4)
-Beam was being diffused at this point, and tpcs were being calibrated.
-Gap between Y strips ~ 23 -> 30
High Energy Ey vs Ex plots from R13 and R16 (attachments 5 and 6 respectively) |
|