During testing for the S452 experiment yesterday we were running through the steps of a restart.

The FEEs were not powercycled at this stage but the MIDAS server was restarted.
Initial setup went smoothly and no issues were encountered during setup.
NewMerger and TapeServer were both setup and set to going.
Upon ''Going'' the run control the data transfer for aida09 dropped out.
Going to the NewMerger it could be seen that data was making it through and data items were being merged.
The tapeserver however was seeing no data and neither was the MBS dataspy.
During this time the FEEs became unresponsive likely as their buffers filled. Trying to reset gave the attached errors.

Checking the folder manually it could be seen that the Options file was there.

At this point a powercycle was pewrformed but upong 'Going' the DAQ the same issue was again encountered.

This time we were able to recover control access to the FEEs by running multiple NewMerger sessions which helped clear the buffers.

During our search for the problem we looked at the Options from within MIDAS and could not see anything out of the ordinary.
We also tried restoring the Options from within MIDAS.
We again reached the same issue with the tapeserver.

At this point we restored the Options folder from by copying manually within terminal.
Following this we were able to start the DAQ normally.

EXPERIMENTS/AIDA/Options/aida04 TS_SYNC_PHASE NOT available: error 0x10004; value TS_SYNC_PHASE does not exist in node /MIDAS/DB/EXPERIMENTS/AIDA/Options/aida04
EXPERIMENTS/AIDA/Options/aida04 ExtClk NOT available: error 0x10004; value ExtClk does not exist in node /MIDAS/DB/EXPERIMENTS/AIDA/Options/aida04
EXPERIMENTS/AIDA/Options/aida04 Aida.shift NOT available: error 0x10004; value Aida.shift does not exist in node /MIDAS/DB/EXPERIMENTS/AIDA/Options/aida04
EXPERIMENTS/AIDA/Options/aida04 MACB_TRIG_MODE NOT available: error 0x10004; value MACB_TRIG_MODE does not exist in node /MIDAS/DB/EXPERIMENTS/AIDA/Options/aida04
EXPERIMENTS/AIDA/Options/aida04 Aida.offset NOT available: error 0x10004; value Aida.offset does not exist in node /MIDAS/DB/EXPERIMENTS/AIDA/Options/aida04
EXPERIMENTS/AIDA/Options/aida04 RunNumber NOT available: error 0x10004; value RunNumber does not exist in node /MIDAS/DB/EXPERIMENTS/AIDA/Options/aida04
EXPERIMENTS/AIDA/Options/aida04 Aida_GroupBase NOT available: error 0x10004; value Aida_GroupBase does not exist in node /MIDAS/DB/EXPERIMENTS/AIDA/Options/aida04
EXPERIMENTS/AIDA/Options/aida04 Rate.channels NOT available: error 0x10004; value Rate.channels does not exist in node /MIDAS/DB/EXPERIMENTS/AIDA/Options/aida04
EXPERIMENTS/AIDA/Options/aida04 Stat.channels NOT available: error 0x10004; value Stat.channels does not exist in node /MIDAS/DB/EXPERIMENTS/AIDA/Options/aida04
EXPERIMENTS/AIDA/Options/aida04 Aida.Vchannels NOT available: error 0x10004; value Aida.Vchannels does not exist in node /MIDAS/DB/EXPERIMENTS/AIDA/Options/aida04
EXPERIMENTS/AIDA/Options/aida04 ASIC.settings NOT available: error 0x10004; value ASIC.settings does not exist in node /MIDAS/DB/EXPERIMENTS/AIDA/Options/aida04
EXPERIMENTS/AIDA/Options/aida04 Aida_Hist_D_Enable NOT available: error 0x10004; value Aida_Hist_D_Enable does not exist in node /MIDAS/DB/EXPERIMENTS/AIDA/Options/aida04
EXPERIMENTS/AIDA/Options/aida04 WAVE_DMA_HWM NOT available: error 0x10004; value WAVE_DMA_HWM does not exist in node /MIDAS/DB/EXPERIMENTS/AIDA/Options/aida04
EXPERIMENTS/AIDA/Options/aida04 Stat.shift NOT available: error 0x10004; value Stat.shift does not exist in node /MIDAS/DB/EXPERIMENTS/AIDA/Options/aida04
EXPERIMENTS/AIDA/Options/aida04 Aida.Wchannels NOT available: error 0x10004; value Aida.Wchannels does not exist in node /MIDAS/DB/EXPERIMENTS/AIDA/Options/aida04
EXPERIMENTS/AIDA/Options/aida04 Aida_Hist_V_Enable NOT available: error 0x10004; value Aida_Hist_V_Enable does not exist in node /MIDAS/DB/EXPERIMENTS/AIDA/Options/aida04
EXPERIMENTS/AIDA/Options/aida04 Include.Aida NOT available: error 0x10004; value Include.Aida does not exist in node /MIDAS/DB/EXPERIMENTS/AIDA/Options/aida04
EXPERIMENTS/AIDA/Options/aida04 Aida.channels NOT available: error 0x10004; value Aida.channels does not exist in node /MIDAS/DB/EXPERIMENTS/AIDA/Options/aida04
EXPERIMENTS/AIDA/Options/aida04 DataAcqPgm NOT available: error 0x10004; value DataAcqPgm does not exist in node /MIDAS/DB/EXPERIMENTS/AIDA/Options/aida04
EXPERIMENTS/AIDA/Options/aida04 Aida_Hist_H_Enable NOT available: error 0x10004; value Aida_Hist_H_Enable does not exist in node /MIDAS/DB/EXPERIMENTS/AIDA/Options/aida04
EXPERIMENTS/AIDA/Options/aida04 DataFormat NOT available: error 0x10004; value DataFormat does not exist in node /MIDAS/DB/EXPERIMENTS/AIDA/Options/aida04
EXPERIMENTS/AIDA/Options/aida04 ASIC_DMA_HWM NOT available: error 0x10004; value ASIC_DMA_HWM does not exist in node /MIDAS/DB/EXPERIMENTS/AIDA/Options/aida04
EXPERIMENTS/AIDA/Options/aida04 Aida_Hist_L_Enable NOT available: error 0x10004; value Aida_Hist_L_Enable does not exist in node /MIDAS/DB/EXPERIMENTS/AIDA/Options/aida04
STATE for aida01 returned with an error
error: SOAP http transport timed out after 20000 ms
NONE
error: SOAP http transport timed out after 20000 ms
    while executing
"$transport $procVarName $url $req"
    (procedure "::SOAP::invoke" line 18)
    invoked from within
"::SOAP::invoke ::SOAP::_DataAcquisitionControlClient__GetState"
    ("eval" body line 1)
    invoked from within
"eval ::SOAP::invoke ::SOAP::_DataAcquisitionControlClient__GetState $args"
    (procedure "DataAcquisitionControlClient__GetState" line 1)
    invoked from within
"DataAcquisitionControlClient__GetState"