AIDA GELINA BRIKEN nToF CRIB ISOLDE CIRCE nTOFCapture DESPEC DTAS EDI_PSA 179Ta CARME StellarModelling DCF K40
  AIDA  ELOG logo
Message ID: 98     Entry time: Mon May 4 02:32:29 2015
Author: VFEP & TD 
Subject: How to fix DAQ stalls at GO 
It has been observed that when starting the AIDA data acquisition (select 'GO' from the 'Data Acquisition Run
Control' window) that from time to time one, or possibly more, FEE64 modules fail to begin producing data.
Once running they appear to be totally reliable. 

The result is that these FEE64 modules produce no AIDA SYNC data items or ASIC Events - they may still
produce WAVE SYNC and Wave Events.

The best check to make after starting the AIDA data acquisition is to select 'Data Acquisition Statistics'
from the 'Data Acquisition Run Control' window. In the 'Data Acquisition Statistics' window check that 
'Show ALL Data Acquisition Servers?' is selected. From the 'Select counter to monitor' menu select
'AIDA SYNC' and select 'Update Once' to update the counters and rates.

Check that all FEE64 modules show a rate of ~1500/s.

If you see 0 then that module has failed to start.

The recommended procedure is:

 1) Make a note of the FEE64 module which is not producing AIDA SYNCs.

 2) Go to the 'Merge Control' window and select 'Reload'.

    The status banner should show 
        'Merge State = GOing     :    merging     :     xfer enabled   :    waiting for SYNC' 

    Select 'Toggle Merge Program Pause State'. The status banner should show
        'Merge State = GOing     :    paused     :     xfer enabled   :    waiting for SYNC' 

    In this state the Merge software will read data from all links but discard the data and make no attempt
    to time align the data streams which is impossible if one, or more, streams is not producing AIDA SYNC
    data items.

 3) Return to the 'Data Acquisition Run Control' window
    Deselect 'Act on ALL Data Acquisition Servers?'
    Select the FEE64 module which is not producing AIDA SYNC data items from the 'Acquisition Servers' menu.
    From the 'System functions (Only use if you know what they do!!!)' menu select 'Go Electronics'.

    This will reset and restart the electronics for this FEE64 module without changing any setting. 

    If you now check the 'Run Control Data Acquisition Statistics' window, hopefully you will see
    that the FEE64 module is now producing AIDA SYNC data items.

 4) If so, return to the Merge Control window.
    Select 'Toggle Merge Program Pause State' and the status should change from 'paused' to 'merging'.
    After a few second you will hopefully observe the Merge software begin to time align the data streams
    and start producing a merged, time-ordered data stream.
ELOG V3.1.4-unknown