ELOG DESPEC

AIDA GELINA BRIKEN nToF CRIB ISOLDE CIRCE nTOFCapture DESPEC DTAS EDI_PSA 179Ta CARME StellarModelling DCF K40

DESPEC

Fields marked with * are required

Entry time:

Fri Apr 23 15:13:28 2021

Author*:

Subject*:

16:00 taking over from Muneerah

16:15 - temp, ucesb, database, stats etc. look ok.

16:41 - Nic restarting MBS

17:05&nbsp;- temp, ucesb, database, stats etc. look ok.

17:43&nbsp;- temp, ucesb, database, stats etc. look ok.

&nbsp;

18.10 - system wide&nbsp;checks. &nbsp;All same as previous checks - eg elog 264 + 266

FEE64 module aida06 global clocks failed, 6 
&nbsp;Clock status test result: Passed 11, Failed 1

Understand status as follows 
Status bit 3 : firmware PLL that creates clocks from external clock not locked 
Status bit 2 : always logic &#39;1&#39;&nbsp; 
Status bit 1 : LMK3200(2) PLL and clock distribution chip not locked to external clock 
Status bit 0 : LMK3200(1) PLL and clock distribution chip not locked to external clock 
If all these bits are not set then the operation of the firmware is unreliable

&nbsp;

&nbsp;&nbsp; &nbsp; 
FEE64 module aida06 failed 
FEE64 module aida07 failed 
FEE64 module aida12 failed 
Calibration test result: Passed 9, Failed 3

If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module

&nbsp;

&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp; Base &nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;Current &nbsp;&nbsp; &nbsp;Difference 
aida06 fault &nbsp;&nbsp; &nbsp; 0x679f : &nbsp;&nbsp; &nbsp; 0x67a2 : &nbsp;&nbsp; &nbsp; 3 &nbsp; 
White Rabbit error counter test result: Passed 11, Failed 1

Understand the status reports as follows:- 
Status bit 3 : White Rabbit decoder detected an error in the received data 
Status bit 2 : Firmware registered WR error, no reload of Timestamp 
Status bit 0 : White Rabbit decoder reports uncertain of Timestamp information from WR 
&nbsp;

Check FPGA Timestamp Error - HTML errors

&nbsp;&nbsp; &nbsp; 
Returned 0 0 0 0 0 0 0 0 0 0 0 0 &nbsp; 
Mem(KB) :&nbsp;&nbsp; &nbsp;4&nbsp;&nbsp; &nbsp;8&nbsp;&nbsp; &nbsp;16&nbsp;&nbsp; &nbsp;32&nbsp;&nbsp; &nbsp;64&nbsp;&nbsp; &nbsp;128&nbsp;&nbsp; &nbsp;256&nbsp;&nbsp; &nbsp;512&nbsp;&nbsp; &nbsp;1k&nbsp;&nbsp; &nbsp;2k&nbsp;&nbsp; &nbsp;4k 
aida01 : &nbsp; &nbsp; 19 &nbsp; &nbsp; &nbsp;7 &nbsp; &nbsp; &nbsp;4 &nbsp; &nbsp; &nbsp;7 &nbsp; &nbsp; &nbsp;4 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;4 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;6 &nbsp; : 36388 
aida02 : &nbsp; &nbsp; &nbsp;1 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;5 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;0 &nbsp; &nbsp; &nbsp;4 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;4 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;4 &nbsp; &nbsp; &nbsp;3 &nbsp; : 27044 
aida03 : &nbsp; &nbsp; 28 &nbsp; &nbsp; &nbsp;4 &nbsp; &nbsp; &nbsp;4 &nbsp; &nbsp; &nbsp;4 &nbsp; &nbsp; &nbsp;4 &nbsp; &nbsp; &nbsp;4 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;6 &nbsp; : 36432 
aida04 : &nbsp; &nbsp; &nbsp;5 &nbsp; &nbsp; &nbsp;5 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;6 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;6 &nbsp; : 36892 
aida05 : &nbsp; &nbsp; 24 &nbsp; &nbsp; &nbsp;4 &nbsp; &nbsp; &nbsp;7 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;6 &nbsp; : 36240 
aida06 : &nbsp; &nbsp; 21 &nbsp; &nbsp; &nbsp;6 &nbsp; &nbsp; &nbsp;6 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;4 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;6 &nbsp; : 36356 
aida07 : &nbsp; &nbsp; 15 &nbsp; &nbsp; &nbsp;7 &nbsp; &nbsp; &nbsp;4 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;4 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;6 &nbsp; : 36308 
aida08 : &nbsp; &nbsp; 23 &nbsp; &nbsp; &nbsp;6 &nbsp; &nbsp; &nbsp;7 &nbsp; &nbsp; &nbsp;1 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;4 &nbsp; &nbsp; &nbsp;1 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;6 &nbsp; : 36572 
aida09 : &nbsp; &nbsp; &nbsp;1 &nbsp; &nbsp; &nbsp;8 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;1 &nbsp; &nbsp; &nbsp;1 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;4 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;6 &nbsp; : 36292 
aida10 : &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;4 &nbsp; &nbsp; &nbsp;6 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;6 &nbsp; : 35660 
aida11 : &nbsp; &nbsp; 15 &nbsp; &nbsp; &nbsp;7 &nbsp; &nbsp; &nbsp;4 &nbsp; &nbsp; &nbsp;1 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;4 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;6 &nbsp; : 36052 
aida12 : &nbsp; &nbsp; 25 &nbsp; &nbsp; 13 &nbsp; &nbsp; &nbsp;5 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;6 &nbsp; : 36700 
&nbsp;

18:45 - temp, ucesb, database, stats etc. look ok.

19:03&nbsp;- temp, ucesb, database, stats etc. look ok.

19:26&nbsp;&nbsp;- temp, ucesb, database, stats etc. look ok.

TapeData (df -h) - 7.2T &nbsp;3.9T &nbsp;2.9T &nbsp;58% /media/1e121361-83d3-4825-b6ae-8700b07e0ca7

&nbsp;

19:30 - beam being shared with R3B, implantation rates fluctuation by~&nbsp; 50% back to normal 19:40

&nbsp;

19:50 system checks - all as previous

&nbsp;

FEE64 module aida06 global clocks failed, 6 
&nbsp;Clock status test result: Passed 11, Failed 1

FEE64 module aida06 failed 
FEE64 module aida07 failed 
FEE64 module aida12 failed 
Calibration test result: Passed 9, Failed 3

If any modules fail calibration , check the clock status and open the FADC Align and Control browser page to rerun calibration for that module

&nbsp;&nbsp; &nbsp; 
&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp; Base &nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;Current &nbsp;&nbsp; &nbsp;Difference 
aida06 fault &nbsp;&nbsp; &nbsp; 0x679f : &nbsp;&nbsp; &nbsp; 0x67a2 : &nbsp;&nbsp; &nbsp; 3 &nbsp; 
White Rabbit error counter test result: Passed 11, Failed 1

Check FPGA Timestamp Error - HTML errors

&nbsp;

Returned 0 0 0 0 0 0 0 0 0 0 0 0 &nbsp; 
Mem(KB) :&nbsp;&nbsp; &nbsp;4&nbsp;&nbsp; &nbsp;8&nbsp;&nbsp; &nbsp;16&nbsp;&nbsp; &nbsp;32&nbsp;&nbsp; &nbsp;64&nbsp;&nbsp; &nbsp;128&nbsp;&nbsp; &nbsp;256&nbsp;&nbsp; &nbsp;512&nbsp;&nbsp; &nbsp;1k&nbsp;&nbsp; &nbsp;2k&nbsp;&nbsp; &nbsp;4k 
aida01 : &nbsp; &nbsp; 25 &nbsp; &nbsp; &nbsp;6 &nbsp; &nbsp; &nbsp;4 &nbsp; &nbsp; &nbsp;7 &nbsp; &nbsp; &nbsp;4 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;4 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;6 &nbsp; : 36404 
aida02 : &nbsp; &nbsp; 20 &nbsp; &nbsp; &nbsp;7 &nbsp; &nbsp; &nbsp;5 &nbsp; &nbsp; &nbsp;1 &nbsp; &nbsp; &nbsp;0 &nbsp; &nbsp; &nbsp;4 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;4 &nbsp; : 27384 
aida03 : &nbsp; &nbsp; 28 &nbsp; &nbsp; &nbsp;6 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;5 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;5 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;6 &nbsp; : 36528 
aida04 : &nbsp; &nbsp; 19 &nbsp; &nbsp; &nbsp;8 &nbsp; &nbsp; &nbsp;4 &nbsp; &nbsp; &nbsp;6 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;4 &nbsp; &nbsp; &nbsp;4 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;6 &nbsp; : 36876 
aida05 : &nbsp; &nbsp; &nbsp;7 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;8 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;4 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;1 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;6 &nbsp; : 36500 
aida06 : &nbsp; &nbsp; 25 &nbsp; &nbsp; &nbsp;8 &nbsp; &nbsp; &nbsp;4 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;4 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;6 &nbsp; : 36868 
aida07 : &nbsp; &nbsp; 18 &nbsp; &nbsp; &nbsp;7 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;4 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;4 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;6 &nbsp; : 36336 
aida08 : &nbsp; &nbsp; 16 &nbsp; &nbsp; 10 &nbsp; &nbsp; &nbsp;6 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;1 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;6 &nbsp; : 36464 
aida09 : &nbsp; &nbsp; &nbsp;0 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;1 &nbsp; &nbsp; &nbsp;1 &nbsp; &nbsp; &nbsp;1 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;4 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;6 &nbsp; : 36360 
aida10 : &nbsp; &nbsp; &nbsp;8 &nbsp; &nbsp; &nbsp;1 &nbsp; &nbsp; 10 &nbsp; &nbsp; &nbsp;0 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;4 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;6 &nbsp; : 35912 
aida11 : &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;4 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;6 &nbsp; : 35928 
aida12 : &nbsp; &nbsp; 12 &nbsp; &nbsp; &nbsp;7 &nbsp; &nbsp; &nbsp;4 &nbsp; &nbsp; &nbsp;1 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;2 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;3 &nbsp; &nbsp; &nbsp;6 &nbsp; : 36616 
&nbsp;

Encoding: HTML ELCode plain
Suppress Email notification Resubmit as new entry

Attachment 1: