| ID |
Date |
Author |
Subject |
|
276
|
Sat Apr 24 10:21:06 2021 |
NH | Saturday 24.04.2021 |
I have restarted some more of the consoles that disappeared since the aidas-gsi issue last night
Grafana is working again
Run R52 CLOSED
Switched to no storage mode (NOTAPE/R1)
System Wide Checks
Clocks: OK
FADC: OK
WR:
Base Current Difference
aida05 fault 0x31a2 : 0x31a2 : 0
aida05 : WR status 0x10
aida06 fault 0x22e7 : 0x22e8 : 1
White Rabbit error counter test result: Passed 10, Failed 2
FPGA: OK
Memory: OK
aida05 ASIC maybe didn't complete setup properly (WR 0x10?)
Timestamps: OK (1)
Temps: OK
Will start gzipping S460 runs for migration to lustre & tape |
| Attachment 1: AnyDesk_2021-04-24_11-27-49.png
|
|
| Attachment 2: AnyDesk_2021-04-24_11-28-50.png
|
|
|
343
|
Fri May 21 23:19:23 2021 |
TD | Saturday 22 May 00:00-08:00 |
00.16 413Gb remaining /TapeData filesystem
estimate 413/2 x 177s = 10.15h to filesystem full - OK
00.18 All histograms, statistics and merger statistics zero'd
00.57 All system wide checks OK *except*
Base Current Difference
aida05 fault 0x500 : 0x572 : 114
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 19 9 3 4 1 5 2 2 3 3 6 : 36356
aida02 : 4 8 7 2 4 3 3 3 2 3 6 : 35968
aida03 : 12 9 6 2 5 3 2 3 3 3 6 : 36824
aida04 : 11 5 9 2 2 2 2 4 3 3 6 : 37028
aida05 : 24 9 2 1 2 3 2 3 4 3 6 : 37608
aida06 : 25 6 3 4 2 4 3 3 3 3 6 : 37060
aida07 : 3 3 3 1 3 3 2 2 2 2 7 : 37044
aida08 : 20 4 2 3 1 3 2 2 3 3 6 : 36016
aida09 : 15 8 2 5 0 3 3 2 3 3 6 : 36284
aida10 : 16 6 4 1 2 3 3 2 2 4 6 : 37328
aida11 : 20 2 2 1 1 3 2 2 3 3 6 : 35936
aida12 : 1 2 1 2 3 4 1 2 4 3 6 : 36900
aida13 : 25 10 4 1 0 3 2 3 3 3 6 : 36500
aida14 : 20 6 9 2 2 5 1 2 2 4 6 : 37200
aida15 : 31 9 5 2 3 3 1 3 3 3 6 : 36500
aida16 : 24 4 2 3 1 4 2 3 3 3 6 : 36672
Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same
FEE : aida01 => Options file size is 1026 Last changed Mon May 17 11:43:04 CEST 2021
FEE : aida02 => Options file size is 1014 Last changed Thu Apr 29 14:43:46 CEST 2021
FEE : aida03 => Options file size is 1014 Last changed Thu Apr 29 14:43:50 CEST 2021
FEE : aida04 => Options file size is 1025 Last changed Fri May 14 16:54:56 CEST 2021
FEE : aida05 => Options file size is 1025 Last changed Mon May 17 06:25:41 CEST 2021
FEE : aida06 => Options file size is 1014 Last changed Thu Apr 29 14:43:59 CEST 2021
FEE : aida07 => Options file size is 1014 Last changed Thu Apr 29 14:44:02 CEST 2021
FEE : aida08 => Options file size is 1025 Last changed Wed May 05 12:15:54 CEST 2021
FEE : aida09 => Options file size is 1014 Last changed Thu Apr 29 14:44:08 CEST 2021
FEE : aida10 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida11 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida12 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida13 => Options file size is 1025 Last changed Fri May 07 19:40:34 CEST 2021
FEE : aida14 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida15 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida16 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
01.12 Lost activity monitor - attachment 1
1.8.H spectra - attachments 2 & 3
1.8.L spectra - attachments 4-6
aida01 pulser peak width 120 ch FWHM
Rate specttra - attachments 7 & 8
01.06 Lost activity monitor - attachment 9
ucesb - attachment 10
Grafana - DSSSD bias & leakage currents past 7 days - attachment 11
statistics - adc data, disc data, WR 28-47, pause, correlation scaler, good wave events - attachments 12-17
FEE64 temperatures OK - attachment 18
DSSSD bias & leakage currents OK - attachment 19
Merger/Tape Server/Merger stats - attachment 20
no merger warning/error messages since last restart
01.26 Analysis file S496/R14_1798 - attachment 21
aida08 5.5% deadtime
01.17 Disk space remaining /TapeData filesystem
[npg@aidas-gsi S496]$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_aidasgsi-lv_root
50G 18G 30G 37% /
tmpfs 7.9G 706M 7.2G 9% /dev/shm
/dev/sda2 477M 42M 410M 10% /boot
/dev/sda1 200M 264K 200M 1% /boot/efi
/dev/mapper/vg_aidasgsi-lv_home
401G 90G 292G 24% /home
/dev/mapper/vg_aidas2-lv_home
7.2T 6.4T 395G 95% /media/1e121361-83d3-4825-b6ae-8700b07e0ca7
/dev/sdd1 7.2T 5.6T 1.3T 82% /media/SecondDrive
/dev/sdc1 7.2T 6.6T 259G 97% /media/ThirdDrive
estimate 395/2 x 177s = 9.7h to filesystem full - OK
01.30 despec shift reports no data reported by ucesb
aida stats & tape server ok - no data being forwarded to MBS by data relay
01.39 despec shift restart FATIMA DAQ - data forwarding to MBS by data relay resumes - AIDA file S496/R14_1811
04.35 System wide checks all OK *except*
Base Current Difference
aida05 fault 0x500 : 0x573 : 115
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
[npg@aidas-gsi S496]$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_aidasgsi-lv_root
50G 18G 30G 37% /
tmpfs 7.9G 661M 7.2G 9% /dev/shm
/dev/sda2 477M 42M 410M 10% /boot
/dev/sda1 200M 264K 200M 1% /boot/efi
/dev/mapper/vg_aidasgsi-lv_home
401G 90G 292G 24% /home
/dev/mapper/vg_aidas2-lv_home
7.2T 6.5T 337G 96% /media/1e121361-83d3-4825-b6ae-8700b07e0ca7
/dev/sdd1 7.2T 5.6T 1.3T 82% /media/SecondDrive
/dev/sdc1 7.2T 6.6T 259G 97% /media/ThirdDrive
estimate 337/2 x 177s = 8.3h to filesystem full - OK
FEE64 temperatures OK - attachment 22
ADC data item statistics OK - attachment 24
DSSSD bias & leakage current OK - attachment 26
analysis of file S496/R14_1920 - attachment 28
aida04 deadtime 5.0%
07.10 Ssystem wide checks all OK *except*
Base Current Difference
aida05 fault 0x500 : 0x573 : 115
White Rabbit error counter test result: Passed 15, Failed 1
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
[npg@aidas-gsi S496]$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_aidasgsi-lv_root
50G 18G 30G 37% /
tmpfs 7.9G 662M 7.2G 9% /dev/shm
/dev/sda2 477M 42M 410M 10% /boot
/dev/sda1 200M 264K 200M 1% /boot/efi
/dev/mapper/vg_aidasgsi-lv_home
401G 90G 292G 24% /home
/dev/mapper/vg_aidas2-lv_home
7.2T 6.5T 287G 96% /media/1e121361-83d3-4825-b6ae-8700b07e0ca7
/dev/sdd1 7.2T 5.6T 1.3T 82% /media/SecondDrive
/dev/sdc1 7.2T 6.6T 259G 97% /media/ThirdDrive
estimate 287/2 x 178s = 7.1h to filesystem full - OK
FEE64 temperatures OK - attachment 23
ADC data item statistics OK - attachment 25
DSSSD bias & leakage current OK - attachment 27
analysis of file S496/R14_1920 - attachment 29
aida04 deadtime 4.4%
07.35 Found MBS data monitor kill'd
Restarted MBS data relay and data monitor OK
08.00 beam off - end of S496
08.28 Tape Server -> no storage mode
DAQ continues
09.58 All slow comparators -> 0x64
BNC PB-5 OFF
file S496/R15
alpha background
Tape server 200kb/s, merger 260k data items/s |
| Attachment 1: Screenshot_2021-05-22_Lost_activity_monitor_aidas-gsi(1).png
|
.png.png "Screenshot_2021-05-22_Lost_activity_monitor_aidas-gsi(1).png")
|
| Attachment 2: Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(7).png
|
.png.png "Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(7).png")
|
| Attachment 3: Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(6).png
|
.png.png "Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(6).png")
|
| Attachment 4: Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(5).png
|
.png.png "Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(5).png")
|
| Attachment 5: Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(3).png
|
.png.png "Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(3).png")
|
| Attachment 6: Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(2).png
|
.png.png "Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(2).png")
|
| Attachment 7: Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(1).png
|
.png.png "Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(1).png")
|
| Attachment 8: Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi.png
|
|
| Attachment 9: Screenshot_2021-05-22_Lost_activity_monitor_aidas-gsi.png
|
|
| Attachment 10: Screenshot_2021-05-22_ucesb.png
|
|
| Attachment 11: Screenshot_2021-05-22_AIDA_Alerting_-_Grafana.png
|
|
| Attachment 12: Screenshot_2021-05-22_Statistics_aidas-gsi(5).png
|
.png.png "Screenshot_2021-05-22_Statistics_aidas-gsi(5).png")
|
| Attachment 13: Screenshot_2021-05-22_Statistics_aidas-gsi(4).png
|
.png.png "Screenshot_2021-05-22_Statistics_aidas-gsi(4).png")
|
| Attachment 14: Screenshot_2021-05-22_Statistics_aidas-gsi(3).png
|
.png.png "Screenshot_2021-05-22_Statistics_aidas-gsi(3).png")
|
| Attachment 15: Screenshot_2021-05-22_Statistics_aidas-gsi(2).png
|
.png.png "Screenshot_2021-05-22_Statistics_aidas-gsi(2).png")
|
| Attachment 16: Screenshot_2021-05-22_Statistics_aidas-gsi(1).png
|
.png.png "Screenshot_2021-05-22_Statistics_aidas-gsi(1).png")
|
| Attachment 17: Screenshot_2021-05-22_Statistics_aidas-gsi.png
|
|
| Attachment 18: Screenshot_2021-05-22_Temperature_and_status_scan_aidas-gsi.png
|
|
| Attachment 19: 800.png
|
|
| Attachment 20: 801.png
|
|
| Attachment 21: R14_1798
|
*** TDR format 3.3.0 analyser - TD - May 2021
*** ERROR: READ I/O error: 5002
blocks: 32000
ADC data format: 254662088 ( 1433466.5 Hz)
Other data format: 7257912 ( 40854.0 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 256 ( 1.4 Hz)
RESUME: 257 ( 1.4 Hz)
SYNC100: 32661 ( 183.8 Hz)
WR48-63: 32661 ( 183.8 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 7192077 ( 40483.5 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 365795 ( 2059.0 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
WR48-63: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
*** Timestamp elapsed time: 177.655 s
FEE elapsed dead time(s) elapsed idle time(s)
0 0.112 0.000
1 5.898 0.000
2 0.060 0.000
3 7.669 0.000
4 0.073 7.902
5 7.648 0.000
6 0.356 0.000
7 9.852 0.000
8 1.513 0.000
9 0.000 15.513
10 2.639 0.000
11 0.903 0.000
12 0.687 0.000
13 0.000 6.517
14 0.073 0.000
15 0.212 0.000
16 0.000 0.000
17 0.000 0.000
18 0.000 0.000
19 0.000 0.000
20 0.000 0.000
21 0.000 0.000
22 0.000 0.000
23 0.000 0.000
24 0.000 0.000
25 0.000 0.000
26 0.000 0.000
27 0.000 0.000
28 0.000 0.000
29 0.000 0.000
30 0.000 0.000
31 0.000 0.000
32 0.000 0.000
*** Statistics
FEE ADC Data Other Data Sample Undefined Pause Resume SYNC100 WR48-63 Disc MBS Other HEC Data
0 5664289 2546 0 0 5 5 737 737 0 1062 0 19821
1 24002581 6376 0 0 39 39 3149 3149 0 0 0 40581
2 12140787 1878978 0 0 3 3 1758 1758 0 1875456 0 32794
3 29094115 1803103 0 0 54 54 3917 3917 0 1795161 0 46229
4 3034023 1008057 0 0 6 6 480 480 0 1007085 0 8526
5 29014047 561174 0 0 54 54 3636 3636 0 553794 0 24714
6 11191428 1180757 0 0 3 3 1528 1528 0 1177695 0 17521
7 34103710 790567 0 0 51 52 4320 4320 0 781824 0 29424
8 17182670 4348 0 0 9 9 2165 2165 0 0 0 41538
9 2132311 520 0 0 0 0 260 260 0 0 0 3304
10 25432181 6308 0 0 12 12 3142 3142 0 0 0 25046
11 16063496 3832 0 0 8 8 1908 1908 0 0 0 7738
12 11072472 2756 0 0 4 4 1374 1374 0 0 0 35884
13 2789016 664 0 0 0 0 332 332 0 0 0 2683
14 11165568 2934 0 0 4 4 1463 1463 0 0 0 24254
15 20579394 4992 0 0 4 4 2492 2492 0 0 0 5738
16 0 0 0 0 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0 0 0 0
*** Timewarps
FEE ADC Pause Resume SYNC100 WR48-63 Disc MBS Undefined Samples
0 0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 0 0
2 0 0 0 0 0 0 0 0 0
3 0 0 0 0 0 0 0 0 0
4 0 0 0 0 0 0 0 0 0
5 0 0 0 0 0 0 0 0 0
6 0 0 0 0 0 0 0 0 0
7 0 0 0 0 0 0 0 0 0
8 0 0 0 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0 0
11 0 0 0 0 0 0 0 0 0
12 0 0 0 0 0 0 0 0 0
13 0 0 0 0 0 0 0 0 0
14 0 0 0 0 0 0 0 0 0
15 0 0 0 0 0 0 0 0 0
16 0 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0
*** Program elapsed time: 41.474s ( 771.565 blocks/s, 48.223 Mb/s)
|
| Attachment 22: Screenshot_2021-05-22_Statistics_aidas-gsi(7).png
|
.png.png "Screenshot_2021-05-22_Statistics_aidas-gsi(7).png")
|
| Attachment 23: Screenshot_2021-05-22_Temperature_and_status_scan_aidas-gsi(2).png
|
.png.png "Screenshot_2021-05-22_Temperature_and_status_scan_aidas-gsi(2).png")
|
| Attachment 24: Screenshot_2021-05-22_Statistics_aidas-gsi(6).png
|
.png.png "Screenshot_2021-05-22_Statistics_aidas-gsi(6).png")
|
| Attachment 25: Screenshot_2021-05-22_Temperature_and_status_scan_aidas-gsi(1).png
|
.png.png "Screenshot_2021-05-22_Temperature_and_status_scan_aidas-gsi(1).png")
|
| Attachment 26: 802.png
|
|
| Attachment 27: 804.png
|
|
| Attachment 28: R14_1867
|
*** TDR format 3.3.0 analyser - TD - May 2021
*** ERROR: READ I/O error: 5002
blocks: 32000
ADC data format: 254778017 ( 1461534.6 Hz)
Other data format: 7141983 ( 40970.0 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 228 ( 1.3 Hz)
RESUME: 228 ( 1.3 Hz)
SYNC100: 32649 ( 187.3 Hz)
WR48-63: 32649 ( 187.3 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 7076229 ( 40592.8 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 358653 ( 2057.4 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
WR48-63: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
*** Timestamp elapsed time: 174.322 s
FEE elapsed dead time(s) elapsed idle time(s)
0 0.023 0.000
1 8.071 0.000
2 0.956 0.000
3 6.628 0.000
4 0.186 12.353
5 5.668 0.000
6 0.255 0.000
7 8.669 0.000
8 1.570 0.000
9 0.000 28.420
10 2.125 0.000
11 0.164 0.000
12 0.103 0.000
13 0.000 11.688
14 0.094 0.000
15 1.340 0.000
16 0.000 0.000
17 0.000 0.000
18 0.000 0.000
19 0.000 0.000
20 0.000 0.000
21 0.000 0.000
22 0.000 0.000
23 0.000 0.000
24 0.000 0.000
25 0.000 0.000
26 0.000 0.000
27 0.000 0.000
28 0.000 0.000
29 0.000 0.000
30 0.000 0.000
31 0.000 0.000
32 0.000 0.000
*** Statistics
FEE ADC Data Other Data Sample Undefined Pause Resume SYNC100 WR48-63 Disc MBS Other HEC Data
0 5294176 2413 0 0 1 1 685 685 0 1041 0 18715
1 22878067 6136 0 0 44 44 3024 3024 0 0 0 38998
2 11959613 1840531 0 0 7 7 1681 1681 0 1837155 0 31909
3 28914032 1781740 0 0 51 51 3845 3845 0 1773948 0 44969
4 2970776 981532 0 0 7 7 454 454 0 980610 0 8409
5 29112496 551278 0 0 40 40 3688 3688 0 543822 0 24330
6 11438278 1149690 0 0 4 4 1643 1643 0 1146396 0 17361
7 34470364 802023 0 0 41 41 4342 4342 0 793257 0 30254
8 17110979 4312 0 0 6 6 2150 2150 0 0 0 40990
9 2129568 524 0 0 0 0 262 262 0 0 0 3068
10 25946722 6422 0 0 9 9 3202 3202 0 0 0 25071
11 16268948 3960 0 0 3 3 1977 1977 0 0 0 7119
12 11320269 2746 0 0 3 3 1370 1370 0 0 0 35684
13 2799924 716 0 0 0 0 358 358 0 0 0 2469
14 11389376 2772 0 0 2 2 1384 1384 0 0 0 23994
15 20774429 5188 0 0 10 10 2584 2584 0 0 0 5313
16 0 0 0 0 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0 0 0 0
*** Timewarps
FEE ADC Pause Resume SYNC100 WR48-63 Disc MBS Undefined Samples
0 0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 0 0
2 0 0 0 0 0 0 0 0 0
3 0 0 0 0 0 0 0 0 0
4 0 0 0 0 0 0 0 0 0
5 0 0 0 0 0 0 0 0 0
6 0 0 0 0 0 0 0 0 0
7 0 0 0 0 0 0 0 0 0
8 0 0 0 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0 0
11 0 0 0 0 0 0 0 0 0
12 0 0 0 0 0 0 0 0 0
13 0 0 0 0 0 0 0 0 0
14 0 0 0 0 0 0 0 0 0
15 0 0 0 0 0 0 0 0 0
16 0 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0
*** Program elapsed time: 43.000s ( 744.186 blocks/s, 46.512 Mb/s)
|
| Attachment 29: R14_1920
|
*** TDR format 3.3.0 analyser - TD - May 2021
*** ERROR: READ I/O error: 5002
blocks: 32000
ADC data format: 254781729 ( 1428365.9 Hz)
Other data format: 7138271 ( 40018.8 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 226 ( 1.3 Hz)
RESUME: 227 ( 1.3 Hz)
SYNC100: 32665 ( 183.1 Hz)
WR48-63: 32665 ( 183.1 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 7072488 ( 39650.0 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 352885 ( 1978.4 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
WR48-63: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
*** Timestamp elapsed time: 178.373 s
FEE elapsed dead time(s) elapsed idle time(s)
0 0.017 2.836
1 6.610 0.000
2 0.573 0.000
3 7.826 0.000
4 0.024 14.831
5 7.215 0.000
6 0.482 0.000
7 6.876 0.000
8 0.394 0.000
9 0.000 17.832
10 1.157 0.000
11 1.395 0.000
12 0.037 0.000
13 0.004 9.016
14 0.101 0.000
15 1.578 0.000
16 0.000 0.000
17 0.000 0.000
18 0.000 0.000
19 0.000 0.000
20 0.000 0.000
21 0.000 0.000
22 0.000 0.000
23 0.000 0.000
24 0.000 0.000
25 0.000 0.000
26 0.000 0.000
27 0.000 0.000
28 0.000 0.000
29 0.000 0.000
30 0.000 0.000
31 0.000 0.000
32 0.000 0.000
*** Statistics
FEE ADC Data Other Data Sample Undefined Pause Resume SYNC100 WR48-63 Disc MBS Other HEC Data
0 5306415 2397 0 0 1 1 665 665 0 1065 0 18998
1 22746335 6008 0 0 36 36 2968 2968 0 0 0 37812
2 11931383 1875609 0 0 5 5 1756 1756 0 1872087 0 31542
3 28584441 1795513 0 0 57 57 3773 3773 0 1787853 0 43417
4 2954259 971818 0 0 3 3 494 494 0 970824 0 8522
5 28608195 543853 0 0 51 52 3567 3567 0 536616 0 24072
6 11355091 1133078 0 0 6 6 1573 1573 0 1129920 0 16979
7 34582740 783003 0 0 34 34 4406 4406 0 774123 0 29465
8 17425265 4332 0 0 7 7 2159 2159 0 0 0 40811
9 2130016 550 0 0 0 0 275 275 0 0 0 3176
10 26903024 6616 0 0 4 4 3304 3304 0 0 0 24219
11 16400506 4036 0 0 5 5 2013 2013 0 0 0 7163
12 11085724 2934 0 0 1 1 1466 1466 0 0 0 35361
13 2664218 650 0 0 1 1 324 324 0 0 0 2574
14 11252748 2738 0 0 4 4 1365 1365 0 0 0 23421
15 20851369 5136 0 0 11 11 2557 2557 0 0 0 5353
16 0 0 0 0 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0 0 0 0
*** Timewarps
FEE ADC Pause Resume SYNC100 WR48-63 Disc MBS Undefined Samples
0 0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 0 0
2 0 0 0 0 0 0 0 0 0
3 0 0 0 0 0 0 0 0 0
4 0 0 0 0 0 0 0 0 0
5 0 0 0 0 0 0 0 0 0
6 0 0 0 0 0 0 0 0 0
7 0 0 0 0 0 0 0 0 0
8 0 0 0 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0 0
11 0 0 0 0 0 0 0 0 0
12 0 0 0 0 0 0 0 0 0
13 0 0 0 0 0 0 0 0 0
14 0 0 0 0 0 0 0 0 0
15 0 0 0 0 0 0 0 0 0
16 0 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0
*** Program elapsed time: 43.672s ( 732.737 blocks/s, 45.796 Mb/s)
|
|
344
|
Sat May 22 13:40:56 2021 |
TD | Saturday 22 May |
14.37 DAQ continues file S496/R15_14
Grafana - DSSSD bias & leakage current - attachment 1
1.8.L spectra - attachments 2 & 3
Stat spectra - attachments 4 & 5
FEE64 temperatures OK - attachment 6
stats - ADC data items, pause items OK - attachments 7 & 8
DSSSD bias & leakage current OK - attachment 9
Merger/Tape Server/Merger stats - attachment 10
analysis of file S496/R14_1920 - attachment 11
all FEE64 deadtime 0% as expected
17.30 aida01, 3, 5, 9 & 11 have hot HEC channels
ASIC check fixes
All histogram, statistics & merger statistics zero'd
DAQ continues
Note gamma-ray source tests have started with FATIMA and DEGAS
20.20 Analysis file R15_19 - attachment 12
[npg@aidas-gsi S496]$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_aidasgsi-lv_root
50G 18G 30G 37% /
tmpfs 7.9G 665M 7.2G 9% /dev/shm
/dev/sda2 477M 42M 410M 10% /boot
/dev/sda1 200M 264K 200M 1% /boot/efi
/dev/mapper/vg_aidasgsi-lv_home
401G 90G 292G 24% /home
/dev/mapper/vg_aidas2-lv_home
7.2T 6.3T 530G 93% /media/1e121361-83d3-4825-b6ae-8700b07e0ca7
/dev/sdd1 7.2T 5.6T 1.3T 82% /media/SecondDrive
/dev/sdc1 7.2T 6.6T 259G 97% /media/ThirdDrive
estimate 90/2 x 3168s = 40h to filesystem full - OK
20.22 All system wide checks OK *except*
Base Current Difference
aida01 fault 0xf82c : 0xf82f : 3
aida02 fault 0xdda6 : 0xdda9 : 3
aida03 fault 0xb13d : 0xb140 : 3
aida04 fault 0xf4cd : 0xf4d0 : 3
aida05 fault 0x500 : 0x575 : 117
aida06 fault 0xc6e7 : 0xc6e9 : 2
aida07 fault 0x4920 : 0x4922 : 2
aida08 fault 0xbd9a : 0xbd9c : 2
White Rabbit error counter test result: Passed 8, Failed 8
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Returned 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 8 4 6 6 1 4 3 3 4 3 6 : 38048
aida02 : 11 8 10 3 4 4 3 4 3 3 6 : 37740
aida03 : 2 8 3 3 2 2 1 4 2 4 6 : 37720
aida04 : 26 15 17 4 4 2 1 4 2 4 6 : 38256
aida05 : 15 6 4 1 1 2 1 3 3 4 6 : 38156
aida06 : 7 7 16 6 3 4 2 2 3 4 6 : 38612
aida07 : 11 5 12 3 3 3 2 3 1 3 7 : 38836
aida08 : 13 6 7 2 1 2 2 2 3 4 6 : 37972
aida09 : 23 3 8 5 0 5 3 3 4 3 6 : 38164
aida10 : 12 5 7 1 2 2 4 3 2 4 6 : 37992
aida11 : 19 2 7 1 2 2 1 2 3 4 6 : 37740
aida12 : 21 5 3 3 3 3 1 2 3 4 6 : 37964
aida13 : 18 7 7 1 2 4 2 3 2 4 6 : 37776
aida14 : 8 3 13 7 2 4 2 3 2 4 6 : 37992
aida15 : 11 5 14 1 3 4 2 3 2 4 6 : 37908
aida16 : 17 6 6 2 2 3 1 2 3 4 6 : 37908
Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same
FEE : aida01 => Options file size is 1026 Last changed Mon May 17 11:43:04 CEST 2021
FEE : aida02 => Options file size is 1014 Last changed Thu Apr 29 14:43:46 CEST 2021
FEE : aida03 => Options file size is 1014 Last changed Thu Apr 29 14:43:50 CEST 2021
FEE : aida04 => Options file size is 1025 Last changed Fri May 14 16:54:56 CEST 2021
FEE : aida05 => Options file size is 1025 Last changed Mon May 17 06:25:41 CEST 2021
FEE : aida06 => Options file size is 1014 Last changed Thu Apr 29 14:43:59 CEST 2021
FEE : aida07 => Options file size is 1014 Last changed Thu Apr 29 14:44:02 CEST 2021
FEE : aida08 => Options file size is 1025 Last changed Wed May 05 12:15:54 CEST 2021
FEE : aida09 => Options file size is 1014 Last changed Thu Apr 29 14:44:08 CEST 2021
FEE : aida10 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida11 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida12 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida13 => Options file size is 1025 Last changed Fri May 07 19:40:34 CEST 2021
FEE : aida14 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida15 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
FEE : aida16 => Options file size is 1014 Last changed Thu Apr 29 14:44:57 CEST 2021
19.30 Lost activity monitors - attachments 13-15
20.54 1.8.W spectra 20us FSR attachments 16 & 23
1.8.W spectra 200us FSR attachments 17 & 22
1.8.W spectra 2ms FSR attachments 18 & 21
1.8.W spectra 20ms FSR attachments 19 & 20 |
| Attachment 1: Screenshot_2021-05-22_AIDA_Alerting_-_Grafana(1).png
|
.png.png "Screenshot_2021-05-22_AIDA_Alerting_-_Grafana(1).png")
|
| Attachment 2: Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(11).png
|
.png.png "Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(11).png")
|
| Attachment 3: Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(10).png
|
.png.png "Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(10).png")
|
| Attachment 4: Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(9).png
|
.png.png "Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(9).png")
|
| Attachment 5: Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(8).png
|
.png.png "Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(8).png")
|
| Attachment 6: Screenshot_2021-05-22_Statistics_aidas-gsi(9).png
|
.png.png "Screenshot_2021-05-22_Statistics_aidas-gsi(9).png")
|
| Attachment 7: Screenshot_2021-05-22_Statistics_aidas-gsi(8).png
|
.png.png "Screenshot_2021-05-22_Statistics_aidas-gsi(8).png")
|
| Attachment 8: Screenshot_2021-05-22_Temperature_and_status_scan_aidas-gsi(3).png
|
.png.png "Screenshot_2021-05-22_Temperature_and_status_scan_aidas-gsi(3).png")
|
| Attachment 9: 805.png
|
|
| Attachment 10: 806.png
|
|
| Attachment 11: R15_13
|
*** TDR format 3.3.0 analyser - TD - May 2021
*** ERROR: READ I/O error: 5002
blocks: 32000
ADC data format: 102550190 ( 10129.0 Hz)
Other data format: 159369822 ( 15741.1 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 69714 ( 6.9 Hz)
WR48-63: 69714 ( 6.9 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 159230394 ( 15727.3 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 0 ( 0.0 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
WR48-63: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
*** Timestamp elapsed time: 10124.450 s
FEE elapsed dead time(s) elapsed idle time(s)
0 0.000 0.000
1 0.000 0.000
2 0.000 67.951
3 0.000 0.000
4 0.000 0.000
5 0.000 9971.491
6 0.000 0.000
7 0.000 10069.837
8 0.000 0.000
9 0.000 0.000
10 0.000 0.000
11 0.000 0.000
12 0.000 0.000
13 0.000 0.000
14 0.000 0.000
15 0.000 0.000
16 0.000 0.000
17 0.000 0.000
18 0.000 0.000
19 0.000 0.000
20 0.000 0.000
21 0.000 0.000
22 0.000 0.000
23 0.000 0.000
24 0.000 0.000
25 0.000 0.000
26 0.000 0.000
27 0.000 0.000
28 0.000 0.000
29 0.000 0.000
30 0.000 0.000
31 0.000 0.000
32 0.000 0.000
*** Statistics
FEE ADC Data Other Data Sample Undefined Pause Resume SYNC100 WR48-63 Disc MBS Other HEC Data
0 213 0 0 0 0 0 0 0 0 0 0 0
1 1159 2 0 0 0 0 1 1 0 0 0 0
2 279 79660137 0 0 0 0 22470 22470 0 79615197 0 0
3 101219504 79708933 0 0 0 0 46868 46868 0 79615197 0 0
4 246 0 0 0 0 0 0 0 0 0 0 0
5 588847 366 0 0 0 0 183 183 0 0 0 0
6 280 0 0 0 0 0 0 0 0 0 0 0
7 737827 382 0 0 0 0 191 191 0 0 0 0
8 262 0 0 0 0 0 0 0 0 0 0 0
9 221 0 0 0 0 0 0 0 0 0 0 0
10 242 2 0 0 0 0 1 1 0 0 0 0
11 210 0 0 0 0 0 0 0 0 0 0 0
12 273 0 0 0 0 0 0 0 0 0 0 0
13 193 0 0 0 0 0 0 0 0 0 0 0
14 250 0 0 0 0 0 0 0 0 0 0 0
15 184 0 0 0 0 0 0 0 0 0 0 0
16 0 0 0 0 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0 0 0 0
*** Timewarps
FEE ADC Pause Resume SYNC100 WR48-63 Disc MBS Undefined Samples
0 0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 0 0
2 0 0 0 0 0 0 0 0 0
3 0 0 0 0 0 0 0 0 0
4 0 0 0 0 0 0 0 0 0
5 0 0 0 0 0 0 0 0 0
6 0 0 0 0 0 0 0 0 0
7 0 0 0 0 0 0 0 0 0
8 0 0 0 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0 0
11 0 0 0 0 0 0 0 0 0
12 0 0 0 0 0 0 0 0 0
13 0 0 0 0 0 0 0 0 0
14 0 0 0 0 0 0 0 0 0
15 0 0 0 0 0 0 0 0 0
16 0 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0
*** Program elapsed time: 45.816s ( 698.440 blocks/s, 43.652 Mb/s)
|
| Attachment 12: R15_19
|
*** TDR format 3.3.0 analyser - TD - May 2021
*** ERROR: READ I/O error: 5002
blocks: 32000
ADC data format: 55871232 ( 17633.8 Hz)
Other data format: 206048770 ( 65032.0 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 56 ( 0.0 Hz)
RESUME: 56 ( 0.0 Hz)
SYNC100: 43803 ( 13.8 Hz)
WR48-63: 43803 ( 13.8 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 205961052 ( 65004.3 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 0 ( 0.0 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
WR48-63: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
*** Timestamp elapsed time: 3168.420 s
FEE elapsed dead time(s) elapsed idle time(s)
0 0.000 0.000
1 0.000 2865.072
2 0.000 0.000
3 0.419 0.000
4 0.000 0.000
5 0.000 2166.184
6 0.000 0.000
7 0.000 1949.169
8 0.000 0.000
9 0.000 0.000
10 0.000 0.000
11 0.000 0.000
12 0.000 0.000
13 0.000 0.000
14 0.000 0.000
15 0.000 0.000
16 0.000 0.000
17 0.000 0.000
18 0.000 0.000
19 0.000 0.000
20 0.000 0.000
21 0.000 0.000
22 0.000 0.000
23 0.000 0.000
24 0.000 0.000
25 0.000 0.000
26 0.000 0.000
27 0.000 0.000
28 0.000 0.000
29 0.000 0.000
30 0.000 0.000
31 0.000 0.000
32 0.000 0.000
*** Statistics
FEE ADC Data Other Data Sample Undefined Pause Resume SYNC100 WR48-63 Disc MBS Other HEC Data
0 55 0 0 0 0 0 0 0 0 0 0 0
1 133932 56 0 0 0 0 28 28 0 0 0 0
2 84 103025372 0 0 0 0 18982 18982 0 102987408 0 0
3 42389344 103017790 0 0 56 56 22017 22017 0 102973644 0 0
4 56 0 0 0 0 0 0 0 0 0 0 0
5 6040914 2574 0 0 0 0 1287 1287 0 0 0 0
6 103 0 0 0 0 0 0 0 0 0 0 0
7 7306081 2976 0 0 0 0 1488 1488 0 0 0 0
8 78 0 0 0 0 0 0 0 0 0 0 0
9 86 0 0 0 0 0 0 0 0 0 0 0
10 68 0 0 0 0 0 0 0 0 0 0 0
11 75 0 0 0 0 0 0 0 0 0 0 0
12 116 2 0 0 0 0 1 1 0 0 0 0
13 80 0 0 0 0 0 0 0 0 0 0 0
14 80 0 0 0 0 0 0 0 0 0 0 0
15 80 0 0 0 0 0 0 0 0 0 0 0
16 0 0 0 0 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0 0 0 0
*** Timewarps
FEE ADC Pause Resume SYNC100 WR48-63 Disc MBS Undefined Samples
0 0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 0 0
2 0 0 0 0 0 0 0 0 0
3 0 0 0 0 0 0 0 0 0
4 0 0 0 0 0 0 0 0 0
5 0 0 0 0 0 0 0 0 0
6 0 0 0 0 0 0 0 0 0
7 0 0 0 0 0 0 0 0 0
8 0 0 0 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0 0
11 0 0 0 0 0 0 0 0 0
12 0 0 0 0 0 0 0 0 0
13 0 0 0 0 0 0 0 0 0
14 0 0 0 0 0 0 0 0 0
15 0 0 0 0 0 0 0 0 0
16 0 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0
*** Program elapsed time: 43.406s ( 737.221 blocks/s, 46.076 Mb/s)
|
| Attachment 13: Screenshot_2021-05-22_Lost_activity_monitor_aidas-gsi(4).png
|
.png.png "Screenshot_2021-05-22_Lost_activity_monitor_aidas-gsi(4).png")
|
| Attachment 14: Screenshot_2021-05-22_Lost_activity_monitor_aidas-gsi(3).png
|
.png.png "Screenshot_2021-05-22_Lost_activity_monitor_aidas-gsi(3).png")
|
| Attachment 15: Screenshot_2021-05-22_Lost_activity_monitor_aidas-gsi(2).png
|
.png.png "Screenshot_2021-05-22_Lost_activity_monitor_aidas-gsi(2).png")
|
| Attachment 16: Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(19).png
|
.png.png "Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(19).png")
|
| Attachment 17: Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(18).png
|
.png.png "Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(18).png")
|
| Attachment 18: Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(17).png
|
.png.png "Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(17).png")
|
| Attachment 19: Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(16).png
|
.png.png "Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(16).png")
|
| Attachment 20: Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(15).png
|
.png.png "Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(15).png")
|
| Attachment 21: Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(14).png
|
.png.png "Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(14).png")
|
| Attachment 22: Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(13).png
|
.png.png "Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(13).png")
|
| Attachment 23: Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(12).png
|
.png.png "Screenshot_2021-05-22_Spectrum_Browser_aidas-gsi(12).png")
|
|
469
|
Sat May 21 15:30:45 2022 |
TD | Saturday 21 May |
16.31 DAQ stopped
Attachment 1 - DSSSD bias & leakage current
Attachment 2 - grafana DSSSD bias, leakage current - OK, temp data has ceased
Attachment 3 - FEE64 temps - OK |
| Attachment 1: Screenshot_from_2022-05-21_16-27-58.png
|
|
| Attachment 2: Screenshot_from_2022-05-21_16-28-59.png
|
|
| Attachment 3: Screenshot_from_2022-05-21_16-29-36.png
|
|
|
583
|
Sat Apr 20 17:52:27 2024 |
TD, NH | Saturday 20 April contd. |
Note 40kBq 60Co source placed on bottom surface of snout centred on bPlas detectors and DSSSDs
18.50 Primary beam 750MeV/u Er to S4
Primary beam stopped by flange (5cm s/steel in position upstream of implantation stack)
Observe c. 2MeV dE peak in AIDA DSSSD#1 and DSSSD#2 using MIDASsort online analysis - attachment 1
21.50 Primary beam 1000MeV/u Er to S4 - attachment 2
per DSSSD e_front versus e_back, 20keV/channel
DSSSD #1 x & y -axis projections
10.50 Overnight stats - attachment 3
Primary beam ?
per DSSSD e_front versus e_back, 20keV/channel
DSSSD #1 x & y -axis projections
note multiple dE peaks observed in both DSSSDs |
| Attachment 1: Screenshot_from_2024-04-20_18-51-53.png
|
|
| Attachment 2: Screenshot_from_2024-04-20_21-52-08.png
|
|
| Attachment 3: Screenshot_from_2024-04-21_10-52-19.png
|
|
|
581
|
Sat Apr 20 03:20:23 2024 |
TD | Saturday 20 April |
Note 60Co source (current activity c. 43kBq) was installed yesterday evening - assume it is positioned on the bottom side of the snout as per the other sources used this week/
04.17 DSSSD bias & leakage current OK - Grafana - attachment 1
FEE64 temperatures OK - attachment 2
aida02 ASIC temp known to be u/s
ADC data item stats - attachment 3
all FEE64 slow comparators 0x14 from yesterday's pulser walkthroughs
per FEE64 Rate spectra - attachment 4
Merger etc - attachment 5
11.24 DSSSD bias & leakage current OK - Grafana - attachment 6
FEE64 temperatures OK - attachment 7
aida02 ASIC temp known to be u/s
ADC data item stats - attachment 8
all FEE64 slow comparators 0x14 from yesterday's pulser walkthroughs
per FEE64 Rate spectra - attachment 9
Merger etc - attachment 10
17.50 DSSSD bias & leakage current OK - Grafana - attachment 11
FEE64 temperatures OK - attachment 12
aida02 ASIC temp known to be u/s
ALl syhstem wide checks OK *except* WR/FPGA errors - attachments 13-14
WR timestamps OK - attachment 15
per FEE64 Rate spectra - attachment 16
per FEE64 1.8.W spectra - 20us FSR - attachments 17-18
Merger, TapeServer, Data Links - attachments 19-20
no data storage
ADC data item stats - attachment 21
all FEE64 slow comparators 0x14 from yesterday's pulser walkthroughs
18.01 Grafana alerting shows detail of DSSSD leakage current and S4 temperature variations over time - attachment 22
DEGAS LN2 fill cycle every 6h 10.00 - 16.00 - 22.00 ... etc
https://despec-vm-01.gsi.de/grafana/d/tHRVJYfGz/aida-alerting?orgId=1 |
| Attachment 1: Screenshot_from_2024-04-20_04-17-19.png
|
|
| Attachment 2: Screenshot_from_2024-04-20_04-17-52.png
|
|
| Attachment 3: Screenshot_from_2024-04-20_04-18-26.png
|
|
| Attachment 4: Screenshot_from_2024-04-20_04-19-13.png
|
|
| Attachment 5: Screenshot_from_2024-04-20_04-19-58.png
|
|
| Attachment 6: Screenshot_from_2024-04-20_11-25-02.png
|
|
| Attachment 7: Screenshot_from_2024-04-20_11-25-54.png
|
|
| Attachment 8: Screenshot_from_2024-04-20_11-26-43.png
|
|
| Attachment 9: Screenshot_from_2024-04-20_11-27-20.png
|
|
| Attachment 10: Screenshot_from_2024-04-20_11-28-05.png
|
|
| Attachment 11: Screenshot_from_2024-04-20_17-50-12.png
|
|
| Attachment 12: Screenshot_from_2024-04-20_17-50-38.png
|
|
| Attachment 13: Screenshot_from_2024-04-20_17-51-20.png
|
|
| Attachment 14: Screenshot_from_2024-04-20_17-51-03.png
|
|
| Attachment 15: Screenshot_from_2024-04-20_17-54-41.png
|
|
| Attachment 16: Screenshot_from_2024-04-20_17-51-58.png
|
|
| Attachment 17: Screenshot_from_2024-04-20_17-52-58.png
|
|
| Attachment 18: Screenshot_from_2024-04-20_17-52-26.png
|
|
| Attachment 19: Screenshot_from_2024-04-20_17-53-32.png
|
|
| Attachment 20: Screenshot_from_2024-04-20_17-53-24.png
|
|
| Attachment 21: Screenshot_from_2024-04-20_17-53-15.png
|
|
| Attachment 22: Screenshot_from_2024-04-20_17-51-37.png
|
|
| Attachment 23: Screenshot_from_2024-04-20_18-05-01.png
|
|
|
407
|
Sat Apr 2 10:42:49 2022 |
TD | Saturday 2 April |
11.10 Found aida08 not running - merger/TapeServer stalled
Restart required power cycle of all FEE64s
*all* ADC calibrations failed during system wide checks it was necessary to use FADC align & control (act on all FEE64 modules selected) to calibrate - first pass
12/16 calibrated, successive individual re-calibrations calibrated remaining FEE64s
ASIC settings 2021Apr29-13-16-00
slow comparator 0x2f
BNC PB-5
amplitude 1.0V
attenuation x1
polarity +
tail pulse
tau_d 1ms
rate 25Hz
Attachments 1-4 1.8.W n+n spectra 20us, 200us, 2ms, 20ms FSR
Attachments 5-8 1.8.W p+n spectra 20ms, 2ms, 200us, 20us FSR
Attachment 9 per FEE64 1.8.L spectra n+n strips
aida04 pulser peak width 387 ch FWHM
Attachment 10 per FEE64 1.8.L spectra p+n strips
aida10 pulser peak width 93 ch FWHM
Attachment 11 per FEE64 Rate spectra
Attachments 12-15 all system wide checks OK
Attachment 16 ADC data item stats
Attachment 17 FEE64 temps OK
Attachment 18 DSSSD bias & leakage currents OK
15.45 check of ASIC settings cf. https://www2.ph.ed.ac.uk/~td/AIDA/Information/Default%20AIDA%20ASIC%20Parameters.txt
Differences from default AIDA ASIC parameters
---------------------------------------------
Input Polarity
Offset Title Positive Negative Notes
12 slow comparator threshold 0x2f->0xa 100keV nominal
14 fast comparator threshold HEC 0xf->0x02 200MeV nominal
15 fast comparator threshold LEC/MEC 0xf->0xff > LEC/MEC nominal FSR
18 preAmp reference 0xb2 0x20->0x30
21 Ibias LF feedback 0x08 0xb->0xf
26 diode link threshold 0xca->0xbf 0x11->0x23 decrease switch threshold
(front) p+n junction strips - positive input polarity
(back) n+n ohmic strips - negative input polarity
All looks OK - variations from default settings are those used at RIKEN
16:45 FEE64 adaptor PCB config
LK1 LK2 LK3 LK4
p+n n/c o * o
n+n o o n/c o
o = jumper fitted
n/c = jumper not fitted
* = top, middle FEE64s - n+n bias ring ground for Si wafer #2
bottom, middle FEE64s - DSSSD PCB gnd
DSSSD#1 & #2 bias via standard coaxial cable to top p+n FEE64 adaptor PCBs
17.05 aida09 or aida13 cable braid not secured to ground - cable tag dia too large - now sandwiched between correctly sized cable tags
no obvious difference to adc data item rates
17.20 LK1 removed for all n+n PCBs
attachment 19 - adc data item stats
18.55 LK2/LK4 removed for all n+n PCBs
attachment 20 - adc data item stats
19.10 power cycle FEE64s
attachment 21 - adc data item stats |
| Attachment 1: Screenshot_from_2022-04-02_11-50-09.png
|
|
| Attachment 2: Screenshot_from_2022-04-02_11-49-44.png
|
|
| Attachment 3: Screenshot_from_2022-04-02_11-49-20.png
|
|
| Attachment 4: Screenshot_from_2022-04-02_11-49-02.png
|
|
| Attachment 5: Screenshot_from_2022-04-02_11-48-38.png
|
|
| Attachment 6: Screenshot_from_2022-04-02_11-47-54.png
|
|
| Attachment 7: Screenshot_from_2022-04-02_11-46-51.png
|
|
| Attachment 8: Screenshot_from_2022-04-02_11-46-11.png
|
|
| Attachment 9: Screenshot_from_2022-04-02_11-44-14.png
|
|
| Attachment 10: Screenshot_from_2022-04-02_11-41-09.png
|
|
| Attachment 11: Screenshot_from_2022-04-02_11-39-57.png
|
|
| Attachment 12: Screenshot_from_2022-04-02_11-39-02.png
|
|
| Attachment 13: Screenshot_from_2022-04-02_11-38-55.png
|
|
| Attachment 14: Screenshot_from_2022-04-02_11-38-47.png
|
|
| Attachment 15: Screenshot_from_2022-04-02_11-38-35.png
|
|
| Attachment 16: Screenshot_from_2022-04-02_11-38-19.png
|
|
| Attachment 17: Screenshot_from_2022-04-02_11-37-51.png
|
|
| Attachment 18: Screenshot_from_2022-04-02_11-37-30.png
|
|
| Attachment 19: Screenshot_from_2022-04-02_17-20-12.png
|
|
| Attachment 20: Screenshot_from_2022-04-02_18-54-11.png
|
|
| Attachment 21: Screenshot_from_2022-04-02_19-07-42.png
|
|
|
234
|
Sat Apr 17 11:27:38 2021 |
JS, TD | Saturday 17th April 12:00-16:00 |
Base Current Difference
aida05 fault 0x1a52 : 0x1a53 : 1
aida06 fault 0x4f3e : 0x4f3f : 1
aida07 fault 0x3bcd : 0x3bce : 1
aida08 fault 0xc7c7 : 0xc7c8 : 1
White Rabbit error counter test result: Passed 8, Failed 4
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
13:55 CEST
Statistics : ok elog:234/4
Temp : ok elog:234/5
Bias : ok elog:234/6
ucesb : ok
DB: No faults found
ADC Calibration check:
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida10 failed
Calibration test result: Passed 9, Failed 3
If any modules fail calibration , check the clock status and open the FADC Align and
Control browser page to rerun calibration for that module
White Rabbit Check:
Base Current Difference
aida05 fault 0x1a52 : 0x1a53 : 1
aida06 fault 0x4f3e : 0x4f3f : 1
aida07 fault 0x3bcd : 0x3bcf : 2
aida08 fault 0xc7c7 : 0xc7c8 : 1
White Rabbit error counter test result: Passed 8, Failed 4
FPGA check:
Base Current Difference
aida09 fault 0x0 : 0x1 : 1
aida12 fault 0x0 : 0x2 : 2
FPGA Timestamp error counter test result: Passed 10, Failed 2
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
14:05 no beam
14:10 beam back
14:15 no beam, beam current being optimised, going to thicker degrader when beam returns
14:37 CEST
Statistics : ok
Temp : ok
Bias : ok (ch3 now over 6uA)
ucesb : ok
DB: No faults found
15:03 CEST
Statistics : ok
Temp : ok
Bias : ok
ucesb : ok
DB: No faults found
15:31 CEST
Statistics : ok
Temp : ok
Bias : ok
ucesb : ok
DB: No faults found
15:50
Statistics : ok elog:234/7
Temp : ok elog:234/8
Bias : ok elog:234/9
ucesb : ok elog:234/10
DB: No faults found
ADC Calibration check:
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida10 failed
Calibration test result: Passed 9, Failed 3
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
White Rabbit Check:
Base Current Difference
aida05 fault 0x1a52 : 0x1a53 : 1
aida06 fault 0x4f3e : 0x4f3f : 1
aida07 fault 0x3bcd : 0x3bcf : 2
aida08 fault 0xc7c7 : 0xc7c8 : 1
White Rabbit error counter test result: Passed 8, Failed 4
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
FPGA check:
Base Current Difference
aida09 fault 0x0 : 0x1 : 1
aida12 fault 0x0 : 0x2 : 2
FPGA Timestamp error counter test result: Passed 10, Failed 2
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last |
| Attachment 1: 30.png
|
|
| Attachment 2: Screenshot_2021-04-17_Statistics_aidas-gsi(3).png
|
.png.png "Screenshot_2021-04-17_Statistics_aidas-gsi(3).png")
|
| Attachment 3: Screenshot_2021-04-17_Temperature_and_status_scan_aidas-gsi(3).png
|
.png.png "Screenshot_2021-04-17_Temperature_and_status_scan_aidas-gsi(3).png")
|
| Attachment 4: Screenshot_2021-04-17_Statistics_aidas-gsi.png
|
|
| Attachment 5: Screenshot_2021-04-17_Temperature_and_status_scan_aidas-gsi.png
|
|
| Attachment 6: 31.png
|
|
| Attachment 7: Screenshot_2021-04-17_Statistics_aidas-gsi(1).png
|
.png.png "Screenshot_2021-04-17_Statistics_aidas-gsi(1).png")
|
| Attachment 8: Screenshot_2021-04-17_Temperature_and_status_scan_aidas-gsi(1).png
|
.png.png "Screenshot_2021-04-17_Temperature_and_status_scan_aidas-gsi(1).png")
|
| Attachment 9: 32.png
|
|
| Attachment 10: 33.png
|
|
|
236
|
Sat Apr 17 19:39:56 2021 |
DJ, TD | Saturday 17 April 20.00-00.00 |
19.37 per FEE64 rate spectra - attachments 1 & 2
1.8.L spectra - attachments 3 & 4
1.8.H spectra - attachments 5-8
1.8.W spectra - attachments 9 & 10
20.42 DAQ contrinues file NULL/R30_233
Base Current Difference
aida05 fault 0x1a52 : 0x1a55 : 3
aida06 fault 0x4f3e : 0x4f41 : 3
aida07 fault 0x3bcd : 0x3bd7 : 10
aida08 fault 0xc7c7 : 0xc7ca : 3
White Rabbit error counter test result: Passed 8, Failed 4
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Base Current Difference
aida09 fault 0x0 : 0x1 : 1
aida12 fault 0x0 : 0x4 : 4
FPGA Timestamp error counter test result: Passed 10, Failed 2
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
-------
23:48
Clock status test result: Passed 12, Failed 0
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida10 failed
Calibration test result: Passed 9, Failed 3
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
Base Current Difference
aida05 fault 0x1a52 : 0x1a55 : 3
aida06 fault 0x4f3e : 0x4f41 : 3
aida07 fault 0x3bcd : 0x3bde : 17
aida08 fault 0xc7c7 : 0xc7ca : 3
White Rabbit error counter test result: Passed 8, Failed 4
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Base Current Difference
aida09 fault 0x0 : 0x1 : 1
aida12 fault 0x0 : 0x7 : 7
FPGA Timestamp error counter test result: Passed 10, Failed 2
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
Returned 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 26 5 6 2 1 3 1 3 3 3 6 : 36336
aida02 : 7 9 10 3 2 2 2 4 2 3 6 : 36068
aida03 : 26 7 8 4 3 4 2 4 2 3 6 : 36448
aida04 : 18 8 2 1 2 2 3 4 2 3 6 : 36168
aida05 : 24 7 9 4 3 1 3 2 2 4 6 : 37352
aida06 : 17 10 3 1 3 5 1 3 3 3 6 : 36644
aida07 : 7 11 2 2 3 2 3 2 4 3 6 : 37268
aida08 : 23 4 1 5 1 1 2 3 3 3 6 : 36332
aida09 : 14 6 3 2 1 3 2 3 3 3 6 : 36504
aida10 : 2 2 2 2 1 2 2 2 3 3 6 : 35768
aida11 : 2 2 1 2 2 2 2 4 2 3 6 : 35816
aida12 : 26 8 5 5 4 3 1 3 3 3 6 : 36632
---- |
| Attachment 1: Screenshot_2021-04-17_Spectrum_Browser_aidas-gsi.png
|
|
| Attachment 2: Screenshot_2021-04-17_Spectrum_Browser_aidas-gsi(1).png
|
.png.png "Screenshot_2021-04-17_Spectrum_Browser_aidas-gsi(1).png")
|
| Attachment 3: Screenshot_2021-04-17_Spectrum_Browser_aidas-gsi(2).png
|
.png.png "Screenshot_2021-04-17_Spectrum_Browser_aidas-gsi(2).png")
|
| Attachment 4: Screenshot_2021-04-17_Spectrum_Browser_aidas-gsi(3).png
|
.png.png "Screenshot_2021-04-17_Spectrum_Browser_aidas-gsi(3).png")
|
| Attachment 5: Screenshot_2021-04-17_Spectrum_Browser_aidas-gsi(4).png
|
.png.png "Screenshot_2021-04-17_Spectrum_Browser_aidas-gsi(4).png")
|
| Attachment 6: Screenshot_2021-04-17_Spectrum_Browser_aidas-gsi(5).png
|
.png.png "Screenshot_2021-04-17_Spectrum_Browser_aidas-gsi(5).png")
|
| Attachment 7: Screenshot_2021-04-17_Spectrum_Browser_aidas-gsi(6).png
|
.png.png "Screenshot_2021-04-17_Spectrum_Browser_aidas-gsi(6).png")
|
| Attachment 8: Screenshot_2021-04-17_Spectrum_Browser_aidas-gsi(7).png
|
.png.png "Screenshot_2021-04-17_Spectrum_Browser_aidas-gsi(7).png")
|
| Attachment 9: Screenshot_2021-04-17_Spectrum_Browser_aidas-gsi(8).png
|
.png.png "Screenshot_2021-04-17_Spectrum_Browser_aidas-gsi(8).png")
|
| Attachment 10: Screenshot_2021-04-17_Statistics_aidas-gsi(4).png
|
.png.png "Screenshot_2021-04-17_Statistics_aidas-gsi(4).png")
|
| Attachment 11: Screenshot_2021-04-17_Temperature_and_status_scan_aidas-gsi(4).png
|
.png.png "Screenshot_2021-04-17_Temperature_and_status_scan_aidas-gsi(4).png")
|
| Attachment 12: 50.png
|
|
| Attachment 13: 51.png
|
|
|
235
|
Sat Apr 17 15:29:27 2021 |
MS | Saturday 17 April 16:00-20:00 2021 |
16:21
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida10 failed
Calibration test result: Passed 9, Failed 3
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
Base Current Difference
aida05 fault 0x1a52 : 0x1a53 : 1
aida06 fault 0x4f3e : 0x4f3f : 1
aida07 fault 0x3bcd : 0x3bcf : 2
aida08 fault 0xc7c7 : 0xc7c8 : 1
White Rabbit error counter test result: Passed 8, Failed 4
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Base Current Difference
aida09 fault 0x0 : 0x1 : 1
aida12 fault 0x0 : 0x2 : 2
FPGA Timestamp error counter test result: Passed 10, Failed 2
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same
FEE : aida01 => Options file size is 1025 Last changed Sat Apr 17 06:14:30 CEST 2021
FEE : aida02 => Options file size is 1014 Last changed Fri Apr 16 00:56:20 CEST 2021
FEE : aida03 => Options file size is 1014 Last changed Wed Apr 14 21:52:04 CEST 2021
FEE : aida04 => Options file size is 1025 Last changed Sat Apr 17 06:07:36 CEST 2021
FEE : aida05 => Options file size is 1025 Last changed Fri Apr 16 00:53:25 CEST 2021
FEE : aida06 => Options file size is 1014 Last changed Wed Apr 14 21:52:04 CEST 2021
FEE : aida07 => Options file size is 1014 Last changed Wed Apr 14 21:52:04 CEST 2021
FEE : aida08 => Options file size is 1014 Last changed Wed Apr 14 21:52:04 CEST 2021
FEE : aida09 => Options file size is 1014 Last changed Wed Apr 14 21:52:05 CEST 2021
FEE : aida10 => Options file size is 1014 Last changed Wed Apr 14 21:52:06 CEST 2021
FEE : aida11 => Options file size is 1014 Last changed Wed Apr 14 21:52:05 CEST 2021
FEE : aida12 => Options file size is 1025 Last changed Wed Apr 14 21:58:54 CEST 2021
18:29
Clock status test result: Passed 12, Failed 0
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida10 failed
Calibration test result: Passed 9, Failed 3
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
Base Current Difference
aida05 fault 0x1a52 : 0x1a53 : 1
aida06 fault 0x4f3e : 0x4f3f : 1
aida07 fault 0x3bcd : 0x3bd1 : 4
aida08 fault 0xc7c7 : 0xc7c8 : 1
White Rabbit error counter test result: Passed 8, Failed 4
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
Base Current Difference
aida09 fault 0x0 : 0x1 : 1
aida12 fault 0x0 : 0x2 : 2
FPGA Timestamp error counter test result: Passed 10, Failed 2
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
Returned 0 0 0 0 0 0 0 0 0 0 0 0
Mem(KB) : 4 8 16 32 64 128 256 512 1k 2k 4k
aida01 : 21 3 4 4 1 3 1 3 3 3 6 : 36332
aida02 : 4 3 13 2 2 3 2 4 2 3 6 : 36152
aida03 : 12 7 5 2 2 3 2 4 2 3 6 : 36088
aida04 : 17 6 9 5 1 1 2 3 3 3 6 : 36452
aida05 : 26 8 0 3 1 1 2 3 3 3 6 : 36296
aida06 : 22 6 9 4 3 4 2 3 3 3 6 : 36952
aida07 : 17 7 3 1 2 3 3 2 3 3 6 : 36300
aida08 : 24 5 4 4 1 1 2 3 3 3 6 : 36360
aida09 : 15 11 3 2 1 3 1 3 3 3 6 : 36292
aida10 : 2 3 5 2 1 2 2 2 3 3 6 : 35824
aida11 : 2 2 0 0 3 2 2 4 2 3 6 : 35800
aida12 : 18 15 7 5 4 3 1 3 3 3 6 : 36688
|
| Attachment 1: 40.png
|
|
| Attachment 2: Screenshot_2021-04-17_Statistics_aidas-gsi(2).png
|
.png.png "Screenshot_2021-04-17_Statistics_aidas-gsi(2).png")
|
| Attachment 3: Screenshot_2021-04-17_Temperature_and_status_scan_aidas-gsi(2).png
|
.png.png "Screenshot_2021-04-17_Temperature_and_status_scan_aidas-gsi(2).png")
|
| Attachment 4: Screenshot_2021-04-17_Statistics_aidas-gsi(3).png
|
.png.png "Screenshot_2021-04-17_Statistics_aidas-gsi(3).png")
|
| Attachment 5: Screenshot_2021-04-17_Temperature_and_status_scan_aidas-gsi(3).png
|
.png.png "Screenshot_2021-04-17_Temperature_and_status_scan_aidas-gsi(3).png")
|
| Attachment 6: 41.png
|
|
|
232
|
Fri Apr 16 23:13:57 2021 |
CA, MA, BA | Saturday 16th 00:00 - 08:00 |
00:05 DESPEC tuning beam, changing spill length
00:21 DAQ continues ok
Merger/TapeServer ok
00:25 still no beam in AIDA
00:30 statistics ok, all FEE64 showing rates
no error messages in database check
AIDA scalars on ucesb ok - attachment 1
00:52 beam in AIDA - rates spectra - attachment 2
01:05 statistics ok, all FEE64 showing rates
no error messages in database check
AIDA scalars on ucesb ok - attachment 3
beam is off again
01:30 beam back
01:32 statistics ok, all FEE64 showing rates
no error messages in database check
AIDA scalars on ucesb ok - attachment 4
01:37 beam off - FRS increasing spill length to 5s
01:48 beam back with new spill setting - 5s on spill, 2s off spill
02:07 system wide checks ok except;
WR decoder status
Base Current Difference
aida05 fault 0xc879 : 0xc87b : 2
aida06 fault 0x323c : 0x323e : 2
aida07 fault 0xfb3a : 0xfb42 : 8
aida08 fault 0xd3d6 : 0xd3d8 : 2
White Rabbit error counter test result: Passed 8, Failed 4
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
FPGA check :
Base Current Difference
aida07 fault 0xf : 0x10 : 1
FPGA Timestamp error counter test result: Passed 11, Failed 1
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
02:10 FEE64 Temps ok - attachment 5
02:20 good event statistics ok - attachment 6
02:27 detector bias / leakage currents ok - attachment 7
02:30 statistics ok, all FEE64 showing rates
no error messages in database check
AIDA scalars on ucesb ok
beam off
03:00 stats/ucesb/database checks ok
03:30 stats/ucesb/database checks ok
still no beam in AIDA
05:11 system is checked, beam is back with low intensity
05:13 FEE64 Temps ok - attachment 8
05:13 good event statistics ok - attachment 9
05:13 detector bias / leakage currents ok - attachment 10
05:33 stats/ucesb/database checks ok
06:00 AIDA crashes, lose connection to aida09
06:20 aida08 does not respond on powercycle and reset
06:30 Called TD
06:50 successfully reset AIDA. Writing to file R30
synchronise ASIC clocks:
FEE64 module aida01 => 7
FEE64 module aida02 => 7
FEE64 module aida03 => 7
FEE64 module aida04 => 7
FEE64 module aida05 => 7
FEE64 module aida06 => 7
FEE64 module aida07 => 7
FEE64 module aida08 => 7
FEE64 module aida09 => 7
FEE64 module aida10 => 7
FEE64 module aida11 => 7
FEE64 module aida12 => 7
ASIC Clock Timestamp check test result: Passed 12, Failed 0
Database options file size check:
If any of the above values are different from the rest then check the clocks and the White Rabbit decoder status
Collecting the file size of each FEE64 Options CONTENTS file to check they are all the same
FEE : aida01 => Options file size is 1025 Last changed Sat Apr 17 06:14:30 CEST 2021
FEE : aida02 => Options file size is 1014 Last changed Fri Apr 16 00:56:20 CEST 2021
FEE : aida03 => Options file size is 1014 Last changed Wed Apr 14 21:52:04 CEST 2021
FEE : aida04 => Options file size is 1025 Last changed Sat Apr 17 06:07:36 CEST 2021
FEE : aida05 => Options file size is 1025 Last changed Fri Apr 16 00:53:25 CEST 2021
FEE : aida06 => Options file size is 1014 Last changed Wed Apr 14 21:52:04 CEST 2021
FEE : aida07 => Options file size is 1014 Last changed Wed Apr 14 21:52:04 CEST 2021
FEE : aida08 => Options file size is 1014 Last changed Wed Apr 14 21:52:04 CEST 2021
FEE : aida09 => Options file size is 1014 Last changed Wed Apr 14 21:52:05 CEST 2021
FEE : aida10 => Options file size is 1014 Last changed Wed Apr 14 21:52:06 CEST 2021
FEE : aida11 => Options file size is 1014 Last changed Wed Apr 14 21:52:05 CEST 2021
FEE : aida12 => Options file size is 1025 Last changed Wed Apr 14 21:58:54 CEST 2021
System wide checks ok *except*
FEE64 module aida06 failed
FEE64 module aida07 failed
FEE64 module aida10 failed
Calibration test result: Passed 9, Failed 3
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
re-calibration of these FEE modules unsuccessful in FADC align and control.
07:05 FEE64 Temps ok - attachment 12
good event statistics ok - attachment 13
detector bias / leakage currents ok - attachment 14
07:10 stats/ucesb/database checks ok
07:30 stats/ucesb/database checks ok
|
| Attachment 1: Screenshot_from_2021-04-16_23-35-20.png
|
|
| Attachment 2: Screenshot_from_2021-04-16_23-52-09.png
|
|
| Attachment 3: Screenshot_from_2021-04-17_00-05-06.png
|
|
| Attachment 4: Screenshot_from_2021-04-17_00-31-46.png
|
|
| Attachment 5: Screenshot_from_2021-04-17_01-11-18.png
|
|
| Attachment 6: Screenshot_from_2021-04-17_01-12-26.png
|
|
| Attachment 7: Screenshot_from_2021-04-17_01-13-22.png
|
|
| Attachment 8: 20210417_0506.png
|
|
| Attachment 9: 20210417_0602_tem.png
|
|
| Attachment 10: 20210417_0601.png
|
|
| Attachment 11: 20210417_0603_Bias.png
|
|
| Attachment 12: Screenshot_from_2021-04-17_06-03-52.png
|
|
| Attachment 13: Screenshot_from_2021-04-17_06-04-14.png
|
|
| Attachment 14: Screenshot_from_2021-04-17_06-04-37.png
|
|
|
664
|
Sat Jun 15 00:28:01 2024 |
TD | Saturday 15 June |
Beam lost due to septum failure c. 18.15 Friday 14 June
Cooling of recirculating water failed c. 19.00 - water temperature increased from the normal 17-18deg C to 25-26 deg C
FEE64 temperatures increased and FEE64s powered off and on-call engineer requested.
Cooling of recirculating water restored and AIDA restarted
NH started background run 23.37 - data file S181/R9
01.21 DSSSD bias & leakage current OK - attachment 1
FEE64 temperatures OK - attachment 2
ADC data item stats OK - attachment 3
8x < 20k, max 238k aida08
per FEE64 Rate spectra - attachment 4
2x Check ASIC load
Merger, TapeServer etc - attachments 5-6
analysis data file R9_27 - attachment 7
max deadtime 1.9%, all other FEE64s < 1%
ADC data rate 951k, HEC data rate 1.6kHz
04.28 DSSSD bias & leakage current OK - attachment 8
FEE64 temperatures OK - attachment 9
ADC data item stats OK - attachment 10
8x < 20k, max 261k aida08
per FEE64 Rate spectra - attachment 11
08.30 DSSSD bias & leakage current OK - attachments 12-13
FEE64 temperatures OK - attachment 14
ADC data item stats OK - attachment 15
all rates increased - gamma source in use?
per FEE64 Rate spectra - attachments 16-17
note aida08 asic #3 hot HEC channel
per FEE64 1.8.L spectra - attachments 18-20
aida09 pulser peak width 59 ch FWHM
decay alphas aida05, aida12, aida02, aida04
per FEE64 1.8.W spectra - 20us FSR - attachments 21-23
Merger, TapeServer etc - attachments 25-25
analysis data file R9_152 - attachment 26
max deadtime 2.4%, all other FEE64s < 1%
ADC data rate 1.215M, HEC data rate 2.1kHz
12.22 DSSSD bias & leakage current OK - attachment 27
FEE64 temperatures OK - attachment 28
ADC data item stats OK - attachment 29
per FEE64 Rate spectra - attachment 30
15.27 DSSSD bias & leakage current OK - attachment 31
FEE64 temperatures OK - attachment 32
ADC data item stats OK - attachment 33
all rates increased - 370kBq 152Eu in use
per FEE64 Rate spectra - attachments 34
per FEE64 1.8.L spectra - attachments 35-36
aida09 pulser peak width 59 ch FWHM
decay alphas aida05, aida12, aida02, aida04
analysis data file R9_275 - attachment 37
max deadtime 4.4%, all other FEE64s < 2%
ADC data rate 1.520M, HEC data rate 2.3kHz
18.52 DSSSD bias & leakage current OK - attachment 38
FEE64 temperatures OK - attachment 39
ADC data item stats OK - attachment 40
per FEE64 Rate spectra - attachment 41
22.50 DSSSD bias & leakage current OK - attachment 42
FEE64 temperatures OK - attachment 43
ADC data item stats OK - attachment 44
per FEE64 Rate spectra - attachment 45 |
| Attachment 1: Screenshot_from_2024-06-15_01-21-06.png
|
|
| Attachment 2: Screenshot_from_2024-06-15_01-21-41.png
|
|
| Attachment 3: Screenshot_from_2024-06-15_01-23-59.png
|
|
| Attachment 4: Screenshot_from_2024-06-15_01-24-56.png
|
|
| Attachment 5: Screenshot_from_2024-06-15_01-27-07.png
|
|
| Attachment 6: Screenshot_from_2024-06-15_01-27-34.png
|
|
| Attachment 7: R9_27
|
*** TDR format 3.3.0 analyser - TD - May 2021
*** ERROR: READ I/O error: 5002
blocks: 32000
ADC data format: 258219194 ( 950585.8 Hz)
Other data format: 3700806 ( 13623.8 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 516 ( 1.9 Hz)
RESUME: 516 ( 1.9 Hz)
SYNC100: 33012 ( 121.5 Hz)
WR48-63: 33012 ( 121.5 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 3633750 ( 13377.0 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 437157 ( 1609.3 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
WR48-63: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
*** Timestamp elapsed time: 271.642 s
FEE elapsed dead time(s) elapsed idle time(s)
0 0.523 0.000
1 0.000 0.000
2 0.959 0.000
3 0.770 0.000
4 0.002 0.000
5 0.768 0.000
6 0.000 0.000
7 5.278 0.000
8 0.000 0.000
9 2.951 0.000
10 0.780 0.000
11 0.000 0.000
12 0.030 0.000
13 0.033 0.000
14 0.000 0.000
15 2.166 0.000
16 0.000 0.000
17 0.000 0.000
18 0.000 0.000
19 0.000 0.000
20 0.000 0.000
21 0.000 0.000
22 0.000 0.000
23 0.000 0.000
24 0.000 0.000
25 0.000 0.000
26 0.000 0.000
27 0.000 0.000
28 0.000 0.000
29 0.000 0.000
30 0.000 0.000
31 0.000 0.000
32 0.000 0.000
*** Statistics
FEE ADC Data Other Data Sample Undefined Pause Resume SYNC100 WR48-63 Disc MBS Other HEC Data
0 3263639 8999 0 0 7 7 426 426 0 8133 0 129
1 5814210 1456 0 0 0 0 728 728 0 0 0 128
2 6330359 1813504 0 0 6 6 1067 1067 0 1811358 0 79
3 25849187 6768 0 0 50 50 3334 3334 0 0 0 112
4 2729875 722 0 0 1 1 360 360 0 0 0 127
5 18600203 4844 0 0 37 37 2385 2385 0 0 0 94
6 5217587 17698 0 0 0 0 698 698 0 16302 0 124
7 65848180 17052 0 0 196 196 8330 8330 0 0 0 435453
8 7897456 2060 0 0 0 0 1030 1030 0 0 0 128
9 45422925 1809941 0 0 107 107 5885 5885 0 1797957 0 64
10 15079549 3854 0 0 29 29 1898 1898 0 0 0 111
11 1722257 384 0 0 0 0 192 192 0 0 0 128
12 4556679 1158 0 0 1 1 578 578 0 0 0 141
13 6725939 1702 0 0 2 2 849 849 0 0 0 130
14 3055032 760 0 0 0 0 380 380 0 0 0 128
15 40106117 9904 0 0 80 80 4872 4872 0 0 0 81
16 0 0 0 0 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0 0 0 0
*** Timewarps
FEE ADC Pause Resume SYNC100 WR48-63 Disc MBS Undefined Samples
0 0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 0 0
2 0 0 0 0 0 0 0 0 0
3 0 0 0 0 0 0 0 0 0
4 0 0 0 0 0 0 0 0 0
5 0 0 0 0 0 0 0 0 0
6 0 0 0 0 0 0 0 0 0
7 0 0 0 0 0 0 0 0 0
8 0 0 0 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0 0
11 0 0 0 0 0 0 0 0 0
12 0 0 0 0 0 0 0 0 0
13 0 0 0 0 0 0 0 0 0
14 0 0 0 0 0 0 0 0 0
15 0 0 0 0 0 0 0 0 0
16 0 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0
*** Program elapsed time: 39.733s ( 805.368 blocks/s, 50.335 Mb/s)
|
| Attachment 8: Screenshot_from_2024-06-15_04-28-07.png
|
|
| Attachment 9: Screenshot_from_2024-06-15_04-28-40.png
|
|
| Attachment 10: Screenshot_from_2024-06-15_04-30-40.png
|
|
| Attachment 11: Screenshot_from_2024-06-15_04-31-38.png
|
|
| Attachment 12: Screenshot_from_2024-06-15_08-33-07.png
|
|
| Attachment 13: Screenshot_from_2024-06-15_08-30-26.png
|
|
| Attachment 14: Screenshot_from_2024-06-15_08-33-35.png
|
|
| Attachment 15: Screenshot_from_2024-06-15_08-36-46.png
|
|
| Attachment 16: Screenshot_from_2024-06-15_08-35-16.png
|
|
| Attachment 17: Screenshot_from_2024-06-15_08-43-12.png
|
|
| Attachment 18: Screenshot_from_2024-06-15_08-40-42.png
|
|
| Attachment 19: Screenshot_from_2024-06-15_08-39-42.png
|
|
| Attachment 20: Screenshot_from_2024-06-15_08-38-37.png
|
|
| Attachment 21: Screenshot_from_2024-06-15_08-46-26.png
|
|
| Attachment 22: Screenshot_from_2024-06-15_08-45-35.png
|
|
| Attachment 23: Screenshot_from_2024-06-15_08-44-50.png
|
|
| Attachment 24: Screenshot_from_2024-06-15_08-48-10.png
|
|
| Attachment 25: Screenshot_from_2024-06-15_08-47-40.png
|
|
| Attachment 26: R9_152
|
*** TDR format 3.3.0 analyser - TD - May 2021
*** ERROR: READ I/O error: 5002
blocks: 32000
ADC data format: 259017680 ( 1215243.3 Hz)
Other data format: 2902320 ( 13616.9 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 515 ( 2.4 Hz)
RESUME: 515 ( 2.4 Hz)
SYNC100: 32794 ( 153.9 Hz)
WR48-63: 32794 ( 153.9 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 2835702 ( 13304.4 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 441097 ( 2069.5 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
WR48-63: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
*** Timestamp elapsed time: 213.141 s
FEE elapsed dead time(s) elapsed idle time(s)
0 0.001 0.000
1 0.000 0.000
2 0.002 0.000
3 0.140 0.000
4 0.000 0.000
5 0.047 0.000
6 0.000 0.000
7 5.152 0.000
8 0.000 0.000
9 2.136 0.000
10 0.186 0.000
11 0.000 0.000
12 0.000 0.000
13 0.032 0.000
14 0.000 0.000
15 1.354 0.000
16 0.000 0.000
17 0.000 0.000
18 0.000 0.000
19 0.000 0.000
20 0.000 0.000
21 0.000 0.000
22 0.000 0.000
23 0.000 0.000
24 0.000 0.000
25 0.000 0.000
26 0.000 0.000
27 0.000 0.000
28 0.000 0.000
29 0.000 0.000
30 0.000 0.000
31 0.000 0.000
32 0.000 0.000
*** Statistics
FEE ADC Data Other Data Sample Undefined Pause Resume SYNC100 WR48-63 Disc MBS Other HEC Data
0 8157992 8515 0 0 4 4 1057 1057 0 6393 0 0
1 4152307 1016 0 0 0 0 508 508 0 0 0 0
2 10764259 1418465 0 0 2 2 1556 1556 0 1415349 0 0
3 20158462 5380 0 0 30 30 2660 2660 0 0 0 0
4 3884729 1032 0 0 0 0 516 516 0 0 0 0
5 16603608 4178 0 0 21 21 2068 2068 0 0 0 0
6 5112596 13950 0 0 0 0 582 582 0 12786 0 0
7 59646516 15586 0 0 217 217 7576 7576 0 0 0 441097
8 6438302 1678 0 0 0 0 839 839 0 0 0 0
9 45360093 1412938 0 0 126 126 5756 5756 0 1401174 0 0
10 18847436 4734 0 0 36 36 2331 2331 0 0 0 0
11 2364016 648 0 0 0 0 324 324 0 0 0 0
12 1869299 460 0 0 0 0 230 230 0 0 0 0
13 11268040 2624 0 0 8 8 1304 1304 0 0 0 0
14 3774625 934 0 0 0 0 467 467 0 0 0 0
15 40615400 10182 0 0 71 71 5020 5020 0 0 0 0
16 0 0 0 0 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0 0 0 0
*** Timewarps
FEE ADC Pause Resume SYNC100 WR48-63 Disc MBS Undefined Samples
0 0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 0 0
2 0 0 0 0 0 0 0 0 0
3 0 0 0 0 0 0 0 0 0
4 0 0 0 0 0 0 0 0 0
5 0 0 0 0 0 0 0 0 0
6 0 0 0 0 0 0 0 0 0
7 0 0 0 0 0 0 0 0 0
8 0 0 0 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0 0
11 0 0 0 0 0 0 0 0 0
12 0 0 0 0 0 0 0 0 0
13 0 0 0 0 0 0 0 0 0
14 0 0 0 0 0 0 0 0 0
15 0 0 0 0 0 0 0 0 0
16 0 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0
*** Program elapsed time: 41.906s ( 763.609 blocks/s, 47.726 Mb/s)
|
| Attachment 27: Screenshot_from_2024-06-15_12-22-54.png
|
|
| Attachment 28: Screenshot_from_2024-06-15_12-23-19.png
|
|
| Attachment 29: Screenshot_from_2024-06-15_12-25-35.png
|
|
| Attachment 30: Screenshot_from_2024-06-15_12-26-47.png
|
|
| Attachment 31: Screenshot_from_2024-06-15_15-26-54.png
|
|
| Attachment 32: Screenshot_from_2024-06-15_15-27-27.png
|
|
| Attachment 33: Screenshot_from_2024-06-15_15-27-48.png
|
|
| Attachment 34: Screenshot_from_2024-06-15_15-28-25.png
|
|
| Attachment 35: Screenshot_from_2024-06-15_15-31-08.png
|
|
| Attachment 36: Screenshot_from_2024-06-15_15-32-10.png
|
|
| Attachment 37: R9_275
|
*** TDR format 3.3.0 analyser - TD - May 2021
*** ERROR: READ I/O error: 5002
blocks: 32000
ADC data format: 259129992 ( 1519853.9 Hz)
Other data format: 2790008 ( 16364.0 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 665 ( 3.9 Hz)
RESUME: 665 ( 3.9 Hz)
SYNC100: 32636 ( 191.4 Hz)
WR48-63: 32636 ( 191.4 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 2723406 ( 15973.4 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 395024 ( 2316.9 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
WR48-63: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
*** Timestamp elapsed time: 170.497 s
FEE elapsed dead time(s) elapsed idle time(s)
0 0.249 0.000
1 0.000 0.000
2 0.356 0.000
3 0.136 0.000
4 0.337 0.000
5 0.141 0.000
6 0.019 0.000
7 7.476 0.000
8 0.165 0.000
9 3.059 0.000
10 0.581 0.000
11 0.000 0.000
12 0.320 0.000
13 0.705 0.000
14 0.045 0.000
15 1.921 0.000
16 0.000 0.000
17 0.000 0.000
18 0.000 0.000
19 0.000 0.000
20 0.000 0.000
21 0.000 0.000
22 0.000 0.000
23 0.000 0.000
24 0.000 0.000
25 0.000 0.000
26 0.000 0.000
27 0.000 0.000
28 0.000 0.000
29 0.000 0.000
30 0.000 0.000
31 0.000 0.000
32 0.000 0.000
*** Statistics
FEE ADC Data Other Data Sample Undefined Pause Resume SYNC100 WR48-63 Disc MBS Other HEC Data
0 12819722 8373 0 0 35 35 1597 1597 0 5109 0 0
1 4488999 1128 0 0 0 0 564 564 0 0 0 0
2 13809452 1368881 0 0 15 15 1975 1975 0 1364901 0 0
3 17302926 4370 0 0 25 25 2160 2160 0 0 0 0
4 4688002 1104 0 0 11 11 541 541 0 0 0 0
5 15844696 4176 0 0 32 32 2056 2056 0 0 0 0
6 5051819 11648 0 0 1 1 708 708 0 10230 0 0
7 52726699 13824 0 0 233 233 6679 6679 0 0 0 395024
8 5506218 1384 0 0 2 2 690 690 0 0 0 0
9 42499600 1354256 0 0 142 142 5403 5403 0 1343166 0 0
10 20647689 5006 0 0 43 43 2460 2460 0 0 0 0
11 2780744 604 0 0 0 0 302 302 0 0 0 0
12 3628914 944 0 0 9 9 463 463 0 0 0 0
13 14531072 3696 0 0 25 25 1823 1823 0 0 0 0
14 4089821 1130 0 0 3 3 562 562 0 0 0 0
15 38713619 9484 0 0 89 89 4653 4653 0 0 0 0
16 0 0 0 0 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0 0 0 0
*** Timewarps
FEE ADC Pause Resume SYNC100 WR48-63 Disc MBS Undefined Samples
0 0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 0 0
2 0 0 0 0 0 0 0 0 0
3 0 0 0 0 0 0 0 0 0
4 0 0 0 0 0 0 0 0 0
5 0 0 0 0 0 0 0 0 0
6 0 0 0 0 0 0 0 0 0
7 0 0 0 0 0 0 0 0 0
8 0 0 0 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0 0
11 0 0 0 0 0 0 0 0 0
12 0 0 0 0 0 0 0 0 0
13 0 0 0 0 0 0 0 0 0
14 0 0 0 0 0 0 0 0 0
15 0 0 0 0 0 0 0 0 0
16 0 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0
*** Program elapsed time: 49.219s ( 650.159 blocks/s, 40.635 Mb/s)
|
| Attachment 38: Screenshot_from_2024-06-15_18-52-04.png
|
|
| Attachment 39: Screenshot_from_2024-06-15_18-52-35.png
|
|
| Attachment 40: Screenshot_from_2024-06-15_18-53-05.png
|
|
| Attachment 41: Screenshot_from_2024-06-15_18-53-49.png
|
|
| Attachment 42: Screenshot_from_2024-06-15_22-50-22.png
|
|
| Attachment 43: Screenshot_from_2024-06-15_22-50-03.png
|
|
| Attachment 44: Screenshot_from_2024-06-15_22-50-49.png
|
|
| Attachment 45: Screenshot_from_2024-06-15_22-51-18.png
|
|
|
151
|
Sat Mar 14 07:13:59 2020 |
DK, TD, LS, CG | Saturday 14th March 08:00 - 24:00 |
8:15
All system wide checks passed
Leak currents are continuing to creep up - attachment #1
FEE temperatures fine - attachment #2
Stats about where they have been - attachment #3
Implants look as previously (5ish counts per channel in SSD1 and SSD2) - attachment #4
Merger fine, 3 Mega items / sec
TapeServer still writing about 25 MB/s
09.37 analysis of data file S480/R12_443
deadtime aida09 c. 5%, all other FEE64s <<1%
See attachment #5
09.54.51
biases, SSD1 and 2 over 2 uA each now - attachment #6
11:20
All system-wide checks okay
Merger okay, 3 Mega items / sec
TapeServer normal, 27 MB/sec
Biases as attachment #7
Stats as #8
Temps as #9
Histos as #10
14:10
All system wide checks okay
Stats #11
Temps #12
Implants #13
Leak currents are getting a bit higher #14
15:30
All system wide checks passed
Stats attachment #15
Temps attachment #16
Implants attachment #17
Biases / leak currents attachment #18
Merger okay, 3 Mega items / sec
TapeServer okay, writing 27 MB/s
16.12 all online spectra zero'd
16.31 stat & rate spectra - attachments 19-22
1.8.L spectra - attachments 23 & 24
aida03 pulser peak 57 ch FWHM
1.8.H spectra - attachments 25 & 26
NewMerger & TapeServer - attachments 27 & 28
18.29
system wide checks all okay
FEE temperatures normal (attachment 29)
Leakage currents okay and recorded to spreadsheet (attachment 30)
Good event statistics okay (attachment 31)
Merger running at 3M items/sec
Tape server running at 25MB/sec
20.25
system wide checks all okay
FEE temperatures normal (attachment 32)
Leakage currents okay and recorded to spreadsheet (attachment 33)
Good event statistics okay (attachment 34)
Merger running at 3M items/sec
Tape server running at 26MB/sec
as of 20.39 just finished R12_961
21.36 beam is down confirmed by rates histogram (attachment 35), informed others on shift
was down for maybe 5 minutes is back on at 21.40 (attachment 36)
think between R12_1004 to R12_1008
22.10
system wide checks all okay
FEE temperatures normal (attachment 37)
Leakage currents okay and recorded to spreadsheet (attachment 38)
Good event statistics okay (attachment 39)
Merger running at 3M items/sec
Tape server running at 26MB/sec
23.03 someone comes to ask if we see an increase in implantation rate as they do their side
I checked the rates histogram and good events and see nothing that would indicate an increase (attachment 40 and 41)
FEE temperatures and leakage currents do not seem any different either
believe they are calling someone about the issue
23.30 leaving now to catch bus but people on night shift have been told how to check on AIDA and know that CG is on call if needed |
| Attachment 1: daid-14-biasA.png
|
|
| Attachment 2: daid-14-tempA.png
|
|
| Attachment 3: daid-14-statsA.png
|
|
| Attachment 4: daid-14-histosA.png
|
|
| Attachment 5: R12_443
|
*** TDR format 3.3.0 analyser - TD - January 2019
*** ERROR: READ I/O error: 5002
blocks: 32000
ADC data format: 86997080 ( 1109430.4 Hz)
Other data format: 174986804 ( 2231519.6 Hz)
Sample trace data format: 0 ( 0.0 Hz)
Undefined format: 0 ( 0.0 Hz)
Other data format type: PAUSE: 29 ( 0.4 Hz)
RESUME: 29 ( 0.4 Hz)
SYNC100: 87327942 ( 1113649.7 Hz)
WR48-63: 87327942 ( 1113649.7 Hz)
FEE64 disc: 330862 ( 4219.3 Hz)
MBS info: 0 ( 0.0 Hz)
Other info: 0 ( 0.0 Hz)
ADC data range bit set: 102097 ( 1302.0 Hz)
Timewarps: ADC: 0 ( 0.0 Hz)
PAUSE: 0 ( 0.0 Hz)
RESUME: 0 ( 0.0 Hz)
SYNC100: 0 ( 0.0 Hz)
WR48-63: 0 ( 0.0 Hz)
FEE64 disc: 0 ( 0.0 Hz)
MBS info: 0 ( 0.0 Hz)
Undefined: 0 ( 0.0 Hz)
Sample trace: 0 ( 0.0 Hz)
*** Timestamp elapsed time: 78.416 s
FEE elapsed dead time(s) elapsed idle time(s)
1 0.000 0.000
2 0.116 0.000
3 0.000 0.000
4 0.000 0.000
5 0.066 0.000
6 0.000 0.000
7 0.000 0.000
8 4.659 3.647
9 0.000 0.000
10 0.112 0.000
11 0.000 0.000
12 0.000 0.000
13 0.000 0.000
14 0.000 0.000
15 0.000 0.000
16 0.000 0.000
17 0.000 0.000
18 0.000 0.000
19 0.000 0.000
20 0.000 0.000
21 0.000 0.000
22 0.000 0.000
23 0.000 0.000
24 0.000 0.000
25 0.000 0.000
26 0.000 0.000
27 0.000 0.000
28 0.000 0.000
29 0.000 0.000
30 0.000 0.000
31 0.000 0.000
32 0.000 0.000
*** Statistics
FEE ADC Data Other Data Sample Undefined Pause Resume SYNC100 WR48-63 Disc MBS Other HEC Data
0 3143962 6596287 0 0 1 1 3246749 3246749 102787 0 0 19287
1 11441532 22943556 0 0 0 0 11461696 11461696 20164 0 0 9627
2 6435721 13042134 0 0 2 2 6492617 6492617 56896 0 0 12957
3 3633801 7382655 0 0 0 0 3672152 3672152 38351 0 0 19271
4 1100939 2276428 0 0 0 0 1125789 1125789 24850 0 0 7339
5 2632637 5296610 0 0 2 2 2643081 2643081 10444 0 0 4759
6 1846149 3780603 0 0 0 0 1875584 1875584 29435 0 0 12550
7 2374464 4801965 0 0 0 0 2392143 2392143 17679 0 0 8964
8 20263627 40557223 0 0 22 22 20273602 20273602 9975 0 0 2005
9 14853979 29722517 0 0 0 0 14858832 14858832 4853 0 0 1283
10 6417731 12865067 0 0 2 2 6427598 6427598 9867 0 0 2254
11 12852538 25721759 0 0 0 0 12858099 12858099 5561 0 0 1801
12 0 0 0 0 0 0 0 0 0 0 0 0
13 0 0 0 0 0 0 0 0 0 0 0 0
14 0 0 0 0 0 0 0 0 0 0 0 0
15 0 0 0 0 0 0 0 0 0 0 0 0
16 0 0 0 0 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0 0 0 0
*** Timewarps
FEE ADC Pause Resume SYNC100 WR48-63 Disc MBS Undefined Samples
0 0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 0 0
2 0 0 0 0 0 0 0 0 0
3 0 0 0 0 0 0 0 0 0
4 0 0 0 0 0 0 0 0 0
5 0 0 0 0 0 0 0 0 0
6 0 0 0 0 0 0 0 0 0
7 0 0 0 0 0 0 0 0 0
8 0 0 0 0 0 0 0 0 0
9 0 0 0 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0 0
11 0 0 0 0 0 0 0 0 0
12 0 0 0 0 0 0 0 0 0
13 0 0 0 0 0 0 0 0 0
14 0 0 0 0 0 0 0 0 0
15 0 0 0 0 0 0 0 0 0
16 0 0 0 0 0 0 0 0 0
17 0 0 0 0 0 0 0 0 0
18 0 0 0 0 0 0 0 0 0
19 0 0 0 0 0 0 0 0 0
20 0 0 0 0 0 0 0 0 0
21 0 0 0 0 0 0 0 0 0
22 0 0 0 0 0 0 0 0 0
23 0 0 0 0 0 0 0 0 0
24 0 0 0 0 0 0 0 0 0
25 0 0 0 0 0 0 0 0 0
26 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0 0 0 0
28 0 0 0 0 0 0 0 0 0
29 0 0 0 0 0 0 0 0 0
30 0 0 0 0 0 0 0 0 0
31 0 0 0 0 0 0 0 0 0
32 0 0 0 0 0 0 0 0 0
*** Program elapsed time:34518.602s ( 0.927 blocks/s, 0.058 Mb/s)
|
| Attachment 6: bias-daid-14B.png
|
|
| Attachment 7: daid-14-biasC.png
|
|
| Attachment 8: daid-14-tempC.png
|
|
| Attachment 9: daid-14-statsC.png
|
|
| Attachment 10: daid-14-histosC.png
|
|
| Attachment 11: daid-14-tempsD.png
|
|
| Attachment 12: daid-14-statsD.png
|
|
| Attachment 13: daid-14-histosD.png
|
|
| Attachment 14: daid-14-biasD.png
|
|
| Attachment 15: daid-14-statsE.png
|
|
| Attachment 16: daid-14-tempsE.png
|
|
| Attachment 17: daid-14-histosE.png
|
|
| Attachment 18: daid-14-biasE.png
|
|
| Attachment 19: 1.png
|
|
| Attachment 20: 2.png
|
|
| Attachment 21: 3.png
|
|
| Attachment 22: 4.png
|
|
| Attachment 23: 5.png
|
|
| Attachment 24: 6.png
|
|
| Attachment 25: 7.png
|
|
| Attachment 26: 8.png
|
|
| Attachment 27: 9.png
|
|
| Attachment 28: 10.png
|
|
| Attachment 29: 200314_1823_feetemps.png
|
|
| Attachment 30: 200314_1827_leakagecurrents.png
|
|
| Attachment 31: 200314_1824_goodevents.png
|
|
| Attachment 32: 200314_2020_feetemps.png
|
|
| Attachment 33: 200314_2023_leakagecurrents.png
|
|
| Attachment 34: 200314_2022_goodevents.png
|
|
| Attachment 35: 200314_2136_BEAMDOWN.png
|
|
| Attachment 36: 200314_2141_beamback.png
|
|
| Attachment 37: 200314_2207_feetemps.png
|
|
| Attachment 38: 200314_2210_leakagecurrents.png
|
|
| Attachment 39: 200314_2208_goodevents.png
|
|
| Attachment 40: 200314_2303_rates.png
|
|
| Attachment 41: 200314_2303_goodevents.png
|
|
|
150
|
Fri Mar 13 23:06:39 2020 |
CA, OH | Saturday 14th March 00:00 - 08:00 |
ASIC settings 2019Oct31-13.24.23
slow comparator 0xa
BNC PB-5 pulser
amplitude 1.0V , attenuator x1
frequency 2Hz
decay time 1ms
00.00 DAQ momentarily set to NoStorage
OH changes symbolic link such that AIDA is writing directly to mounted hard-drives ("SecondDrive" and "ThirdDrive")
00.02 DAQ now writing to file again - /media/SecondDrive/TapeData/S480/R12_0
00.30 all system wide checks ok
FEE64 temperatures ok - attachment 1
good event statistics ok - attachment 2
detector bias and leakage currents ok - attachment 3
00:43 merger ok - writing at ~ 3 million items/s
TapeServer ok - writing at ~ 27 Mb/s
Data forwarding to MBS ok
02:14 all system wide checks ok
FEE64 temperatures ok - attachment 4
good event statistics ok - attachment 5
detector bias and leakage currents ok - attachment 6
02:17 Merger ok ~ 2.7 million items/s
TapeServer ok ~ 27 Mb/s
Data forwarding to MBS ok
02:18 currently writing to file /media/SecondDrive/TapeData/S480/R12_110
02:57 attachments 7 & 8 - high energy channel spectra
attachment 9 - hit rate spectra
04:21 FEE64 temperatures ok - attachment 10
good event statistics ok - attachment 11
detector bias and leakage currents ok - attachment 12
system wide checks ok - except WR decoder status (see attachment 13)
04:26 check ASIC control
04:30 system functions -> select required function -> collect all WhiteRabbit status error counters for baseline
WR decoder status check now ok
04:50 Merger ok ~ 3 million items/s
TapeServer ok ~ 25 Mb/s
Data forwarding to MBS ok
04:52 currently writing to file /media/SecondDrive/TapeData/S480/R12_231
06:11 all system wide checks ok
FEE64 temperatures ok - attachment 14
good event statistics ok - attachment 15
detector bias and leakage currents ok - attachment 16
06:41 Merger ok ~ 3.3 million items/s
TapeServer ok ~ 25 Mb/s
Data forwarding to MBS ok
06:42 currently writing to file /media/SecondDrive/TapeData/S480/R12_315
07:25 hit-rate spectra - attachment 17
07:36 low energy channel spectra (1.8.L) - attachments 18 & 19
|
| Attachment 1: 200314_0020_temp.png
|
|
| Attachment 2: 200314_0030_stats.png
|
|
| Attachment 3: 200314_0031_bias.png
|
|
| Attachment 4: 200314_0212_temp.png
|
|
| Attachment 5: 200314_0213_stats.png
|
|
| Attachment 6: 200314_0214_bias.png
|
|
| Attachment 7: even_he.png
|
|
| Attachment 8: odd_he.png
|
|
| Attachment 9: 200314_0254_rates.png
|
|
| Attachment 10: 200314_0416_temp.png
|
|
| Attachment 11: 200314_0417_stats.png
|
|
| Attachment 12: 200314_0418_bias.png
|
|
| Attachment 13: 200314_WRerror.png
|
|
| Attachment 14: 200314_0608_temp.png
|
|
| Attachment 15: 200314_0609_stats.png
|
|
| Attachment 16: 200314_0610_bias.png
|
|
| Attachment 17: 200314_0724_rates.png
|
|
| Attachment 18: 200314_0733_18l_odd.png
|
|
| Attachment 19: 200314_0735_18l_even.png
|
|
|
447
|
Sat May 14 06:57:21 2022 |
MS, OH | Saturday 14 May |
08:00 Took over the shift from Philippos
In system wide checks aida09 fails ASIC clock check but it is bit 6 which is ok |
|
448
|
Sat May 14 09:00:21 2022 |
MS, OH | Saturday 14 May |
10:00 |
| Attachment 1: Screenshot_2022-05-14_at_10-01-07_Statistics_aidas-gsi.png
|
|
| Attachment 2: Screenshot_2022-05-14_at_10-01-50_Temperature_and_status_scan_aidas-gsi.png
|
|
| Attachment 3: Screenshot_from_2022-05-14_10-02-51.png
|
|
|
449
|
Sat May 14 10:59:52 2022 |
MS, OH | Saturday 14 May |
12:00 |
| Attachment 1: Screenshot_2022-05-14_at_12-00-16_Statistics_aidas-gsi.png
|
|
| Attachment 2: Screenshot_2022-05-14_at_12-00-44_Temperature_and_status_scan_aidas-gsi.png
|
|
| Attachment 3: Screenshot_from_2022-05-14_12-01-54.png
|
|
|
450
|
Sat May 14 13:01:58 2022 |
MS, OH | Saturday 14 May |
14:00 |
| Attachment 1: Screenshot_2022-05-14_at_14-02-31_Statistics_aidas-gsi.png
|
|
| Attachment 2: Screenshot_2022-05-14_at_14-02-52_Temperature_and_status_scan_aidas-gsi.png
|
|
| Attachment 3: Screenshot_from_2022-05-14_14-03-54.png
|
|
|
451
|
Sat May 14 15:35:24 2022 |
BA, AA | Saturday 14 May |
|
| Attachment 1: Stais_from_2022-05-14_16-30-53.png
|
|
| Attachment 2: Temp_2022-05-14_16-32-42.png
|
|
| Attachment 3: Leakage_2022-05-14_16-33-45.png
|
|
|
452
|
Sat May 14 17:34:40 2022 |
BA, AA | Saturday 14 May |
FEE64 module aida09 global clocks failed, 6
Clock status test result: Passed 13, Failed 1
Understand status as follows
Status bit 3 : firmware PLL that creates clocks from external clock not locked
Status bit 2 : always logic '1'
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock
If all these bits are not set then the operation of the firmware is unreliable
FEE64 module aida02 failed
FEE64 module aida06 failed
FEE64 module aida09 failed
FEE64 module aida10 failed
FEE64 module aida13 failed
Calibration test result: Passed 9, Failed 5
If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module
White Rabbit error counter test result: Passed 14, Failed 0
Understand the status reports as follows:-
Status bit 3 : White Rabbit decoder detected an error in the received data
Status bit 2 : Firmware registered WR error, no reload of Timestamp
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR
FPGA Timestamp error counter test result: Passed 14, Failed 0
If any of these counts are reported as in error
The ASIC readout system has detected a timeslip.
That is the timestamp read from the time FIFO is not younger than the last
|
| Attachment 1: Stat_2022-05-14_18-33-16.png
|
|
| Attachment 2: Temp_2022-05-14_18-32-07.png
|
|
| Attachment 3: Leakage_2022-05-14_18-30-52.png
|
|