AIDA GELINA BRIKEN nToF CRIB ISOLDE CIRCE nTOFCapture DESPEC DTAS EDI_PSA 179Ta CARME StellarModelling DCF K40
  AIDA, Page 30 of 46  ELOG logo
ID Date Authorup Subject
  400   Sun Oct 23 10:22:36 2016 TDReport - high - merger SEGV
Merger crashed during operation - see https://elog.ph.ed.ac.uk/AIDA/399
  401   Mon Oct 24 02:46:57 2016 TDMonday 24 October
10.47 DAQ found stopped, no merger error messages
      Most recent file R78_281, last accessed 06.46

      R78_281 ADC data synchronised

      DSSSD bias & leakage currents OK - see attachments 1 & 2
      FEE64 temperatures OK - see attachment 3
      System-wide checks - see attachments 4-11 

      Restart merger OK
      DAQ STOP OK

      DAQ RESET & SETUP OK
      ReSYNC OK
      System-wide checks OK
      Disable H & data transfer #1
      Restart merger & tapeserver
      Enable H & data transfer #1

11.07 DAQ starts (R79)

18.39 DAQ OK

      System wide checks OK *except* SYNC errors & SYNC pulses - see attachments 12-15

18.47 DAQ stalls
      Most recent file R79_182, last accessed 18.46
      System wide checks OK *except* SYNC errors & SYNC pulses - see attachments 16-19

18.56 DAQ starts (R80)
Attachment 1: 1.png
1.png
Attachment 2: 2.png
2.png
Attachment 3: 3.png
3.png
Attachment 4: 4.png
4.png
Attachment 5: 5.png
5.png
Attachment 6: 6.png
6.png
Attachment 7: 7.png
7.png
Attachment 8: 8.png
8.png
Attachment 9: 9.png
9.png
Attachment 10: 10.png
10.png
Attachment 11: 11.png
11.png
Attachment 12: 20.png
20.png
Attachment 13: 21.png
21.png
Attachment 14: 22.png
22.png
Attachment 15: 23.png
23.png
Attachment 16: 24.png
24.png
Attachment 17: 25.png
25.png
Attachment 18: 26.png
26.png
Attachment 19: 27.png
27.png
  402   Tue Oct 25 02:06:34 2016 TDTuesday 25 October
10.10 DAQ found stopped
      Most recent file R81_169, last accessed 06.26

      R81_169 ADC data synchronised

aidas1> ./analyser /TapeData/Sept2016/R81_169
 *** TDR format 3.3.0 analyser - TD - October 2016
 *** ERROR: READ I/O error:       5002
                   blocks:       8369
          ADC data format:   67712004 ( 1105598.7 Hz)
        Other data format:     570667 (    9317.8 Hz)
 Sample trace data format:          0 (       0.0 Hz)
         Undefined format:          0 (       0.0 Hz)
   Other data format type:      PAUSE:         17 (       0.3 Hz)
                               RESUME:         17 (       0.3 Hz)
                              SYNC100:     560442 (    9150.9 Hz)
                           FEE64 disc:       8352 (     136.4 Hz)
                             MBS info:       1839 (      30.0 Hz)
                           Other info:          0 (       0.0 Hz)

   ADC data range bit set:         81 (       1.3 Hz)

                Timewarps:        ADC:          0 (       0.0 Hz)
                                PAUSE:          0 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:          0 (       0.0 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:          0 (       0.0 Hz)
                            Undefined:          0 (       0.0 Hz)
                         Sample trace:          0 (       0.0 Hz)

 *** Timestamp elapsed time:       61.245 s
 FEE  elapsed dead time(s) elapsed idle time(s)
  1                0.000                0.000
  2                0.000                0.000
  3                0.000                0.000
  4                0.000                0.000
  5                0.244                0.254
  6                0.181                0.191
  7                0.000                0.000
  8                0.000                0.000
  9                0.000                0.000
 10                0.000                0.000
 11                0.000                0.000
 12                0.000                0.000
 13                0.000                0.000
 14                0.000                0.000
 15                0.002                0.000
 16                0.000                0.000
 17                0.001                0.000
 18                0.002                0.000
 19                0.003                0.000
 20                0.000                0.000
 21                0.000                0.000
 22                0.000                0.000
 23                0.085                0.000
 24                0.000                0.000
 25                0.000                0.000
 26                0.000                0.000
 27                0.000                0.000
 28                0.000                0.000
 29                0.000                0.000
 30                0.000                0.000
 31                0.000                0.000
 32                0.000                0.000

 *** Statistics
 FEE  ADC Data Other Data     Sample  Undefined      Pause     Resume    SYNC100       Disc        MBS     
Other   HEC Data
  0          0          0          0          0          0          0          0          0          0         
0          0
  1     229453      23628          0          0          0          0      23359        269          0         
0         22
  2    2464367      23972          0          0          0          0      23362        610          0         
0          0
  3    1700233      23656          0          0          0          0      23358        298          0         
0          0
  4     650519      23359          0          0          0          0      23359          0          0         
0          0
  5   15435615      25773          0          0          4          4      23267       2498          0         
0          0
  6   15768889      25301          0          0          3          3      23291        165       1839         
0          0
  7     963142      23361          0          0          0          0      23361          0          0         
0          0
  8     773768      23661          0          0          0          0      23361        300          0         
0          0
  9    2244651      23523          0          0          0          0      23359        164          0         
0          0
 10    1833324      24095          0          0          0          0      23362        733          0         
0          0
 11     276357      23657          0          0          0          0      23362        295          0         
0          0
 12     542691      23359          0          0          0          0      23359          0          0         
0          0
 13    1695447      23996          0          0          0          0      23363        633          0         
0          0
 14    2181674      23530          0          0          0          0      23361        169          0         
0          0
 15    2203091      23362          0          0          1          1      23360          0          0         
0          0
 16    2179412      23361          0          0          0          0      23361          0          0         
0          0
 17    2317043      23549          0          0          1          1      23359        188          0         
0          0
 18    2579367      23969          0          0          1          1      23356        611          0         
0          0
 19    2589829      23664          0          0          3          3      23358        300          0         
0          0
 20    1296289      23357          0          0          0          0      23357          0          0         
0          0
 21    1624570      24024          0          0          0          0      23361        663          0         
0         59
 22     513131      23525          0          0          0          0      23360        165          0         
0          0
 23    3691921      23335          0          0          4          4      23327          0          0         
0          0
 24    1957221      23650          0          0          0          0      23359        291          0         
0          0
 25          0          0          0          0          0          0          0          0          0         
0          0
 26          0          0          0          0          0          0          0          0          0         
0          0
 27          0          0          0          0          0          0          0          0          0         
0          0
 28          0          0          0          0          0          0          0          0          0         
0          0
 29          0          0          0          0          0          0          0          0          0         
0          0
 30          0          0          0          0          0          0          0          0          0         
0          0
 31          0          0          0          0          0          0          0          0          0         
0          0
 32          0          0          0          0          0          0          0          0          0         
0          0

 *** Timewarps
 FEE       ADC      Pause     Resume    SYNC100       Disc        MBS  Undefined    Samples
  0          0          0          0          0          0          0          0          0
  1          0          0          0          0          0          0          0          0
  2          0          0          0          0          0          0          0          0
  3          0          0          0          0          0          0          0          0
  4          0          0          0          0          0          0          0          0
  5          0          0          0          0          0          0          0          0
  6          0          0          0          0          0          0          0          0
  7          0          0          0          0          0          0          0          0
  8          0          0          0          0          0          0          0          0
  9          0          0          0          0          0          0          0          0
 10          0          0          0          0          0          0          0          0
 11          0          0          0          0          0          0          0          0
 12          0          0          0          0          0          0          0          0
 13          0          0          0          0          0          0          0          0
 14          0          0          0          0          0          0          0          0
 15          0          0          0          0          0          0          0          0
 16          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0

 *** Program elapsed time:    0.957s ( 8744.751 blocks/s, 546.547 Mb/s)

      DSSSD bias & leakage currents OK - see attachments 1 & 2
      FEE64 temperatures OK - see attachment 3
      System-wide checks OK *except* SYNC errors & SYNC pulses - see attachments 4-7 
      Merger error messages - see attachment 8

      Restart merger OK
      DAQ STOP OK

      DAQ RESET & SETUP OK
      ReSYNC OK
      All system-wide checks OK
      Disable H & data transfer #1
      Restart merger & tapeserver
      Enable H & data transfer #1

10.22 DAQ starts (R82)

16.07 DAQ continues OK
      All system-wide checks OK

17.16 DAQ continues OK
      All system-wide checks OK

18.27 All system wide checks OK *except* SYNC pulses - see attachment 9

18.34 DAQ stalled
      Most recent file R82_125 last accessed 18.30

      All system wide checks OK *except* SYNC errors and SYNC pulses - see attachments 10-13

18.47 AIDA -> safe state
      FEE64 power OFF
      DSSSD bias OFF
Attachment 1: 1.png
1.png
Attachment 2: 2.png
2.png
Attachment 3: 3.png
3.png
Attachment 4: 4.png
4.png
Attachment 5: 5.png
5.png
Attachment 6: 6.png
6.png
Attachment 7: 7.png
7.png
Attachment 8: 8.png
8.png
Attachment 9: 10.png
10.png
Attachment 10: 11.png
11.png
Attachment 11: 12.png
12.png
Attachment 12: 13.png
13.png
Attachment 13: 14.png
14.png
  403   Thu Oct 27 10:54:14 2016 TDThursday 27 October
14.20 DSSSD bias & leakage currents OK - see attachments 1 & 2
      FEE64 temperatures OK - see attachment 3

14.35 DAQ restarted for test purposes

18.55 DAQ stalled as expected
      Most recent file R83_125 last accessed 18.30

      DAQ STOP OK

      DAQ RESET & SETUP OK
      ReSYNC OK
      All system-wide checks OK
      Disable H & data transfer #1
      Restart merger & tapeserver
      Enable H & data transfer #1

20.48 DAQ starts (R86)
      LEC/MEC fast comparator 0xff -> 0x20
Attachment 1: 1.png
1.png
Attachment 2: 2.png
2.png
Attachment 3: 3.png
3.png
  404   Thu Oct 27 10:57:49 2016 TDAIDA data readout
Developing real-time AIDA data synchronisation test using MIDASsort

To illustrate AIDA readout, ADC and discriminator data from a pulser event is plotted as a function
of the 100MHz AIDA internal timestamp (1us/channel)

Attachment 1 -    nnaida8  number of ADC and discriminator data items
Attachment 2 - nnaida1-24  number of ADC and discriminator data items
Attachment 3 - 4*(FEE64 module # - 1) + ASIC # (ADC channels 0-95, disc channels 100-195) versus timestamp
(1us/channel)

The spectra clearly illustrate

- 500kHz ADC readout cycle
- discriminator readout precedes ADC readout by ~18us (this figure will be shaping time dependent)
- after-pulse due to pulser injection via simple charge terminator (no pole zero correction) 
  ~84us after start of ADC readout (this figure will be pulser decay time and shaping time dependent)

Note events were identified (delimited) by time periods > 2us between successive ADC data items  
Attachment 1: 100.png
100.png
Attachment 2: 101.png
101.png
Attachment 3: 102.png
102.png
  405   Fri Oct 28 06:12:50 2016 TDFriday 28 October
14.12 DAQ stalled as expected
      Most recent file R86_245 last accessed 06.26

      DSSSD bias & leakage current OK - see attachments 1 & 2
      FEE64 temperatures OK - see attachment 3
     
      Some merger errors messages - all link 4 - see attachment 4
  
      All system wide checks OK *except SYNC errors & SYNC pulses - see attachments 5-8

      Restart merger
      DAQ STOP OK
      DAQ RESET & SETUP OK
      ReSYNC OK
      All system-wide checks OK
      Disable H & data transfer #1
      Restart merger & tapeserver
      Enable H & data transfer #1

14.34 DAQ started (R87)

19.50 DAQ continues (R87)

      System-wide checks OK *except* SYNC errors & SYNC pulses - see attachments 9-12
Attachment 1: 1.png
1.png
Attachment 2: 2.png
2.png
Attachment 3: 3.png
3.png
Attachment 4: 4.png
4.png
Attachment 5: 5.png
5.png
Attachment 6: 6.png
6.png
Attachment 7: 7.png
7.png
Attachment 8: 8.png
8.png
Attachment 9: 20.png
20.png
Attachment 10: 21.png
21.png
Attachment 11: 22.png
22.png
Attachment 12: 23.png
23.png
  406   Sat Oct 29 03:31:03 2016 TDSaturday 29 October
11.30 DAQ continues (R87)
      System-wide checks OK *except* SYNC errors & SYNC pulses - see attachments 1-4

11.36 DSSSD bias & leakage currents OK - see attachments 5-8
      FEE64 temperatures OK - see attachment 9

16.46 DAQ continues (R87)
      System-wide checks OK *except* SYNC errors & SYNC pulses - see attachments 5-8

18.55 DAQ stalled
      Most recent file R87_274 last accessed 18.47

      Restart merger
      DAQ STOP
      System wide checks 
      DAQ RESET & SETUP
      System-wide checks
      Disable H & data transfer #1
      Restart merger & tapeserver
      Enable H & data transfer #1

19.10 DAQ starts (R89)
Attachment 1: 30.png
30.png
Attachment 2: 31.png
31.png
Attachment 3: 32.png
32.png
Attachment 4: 33.png
33.png
Attachment 5: 1.png
1.png
Attachment 6: 2.png
2.png
Attachment 7: 3.png
3.png
Attachment 8: 10.png
10.png
Attachment 9: 11.png
11.png
Attachment 10: 12.png
12.png
Attachment 11: 14.png
14.png
  407   Sat Oct 29 07:26:59 2016 TDAIDA ADC & disc data synchronisation check
Following analysis of AIDA ADC & disc readout ( see https://elog.ph.ed.ac.uk/AIDA/404 )
changed timestamp resolution from 1us/channel to 20.48us/channel


Offline sort (from disk file)

Attachments 1-3  s1-s32  # ADC data items per FEE64, s100-132  # disc data items per FEE64 (20.48us/channel)

Attachments 4-5  s1000   4 * ( module # - 1 ) + ASIC # versus timestamp (ch 0-95 ADC data, ch 100-195 disc data,
20.48us/channel)

Attachments 6-7  MIDASsort terminal session logs for *offline* sort


Online sort (from TapeServer)

Attachment 8     s1-s32  # ADC data items per FEE64, s100-132  # disc data items per FEE64 (20.48us/channel)

Attachments 9-10 s1000   4 * ( module # - 1 ) + ASIC # versus timestamp (ch 0-95 ADC data, ch 100-195 disc data,
20.48us/channel)

Attachments 11-12 MIDASsort terminal session logs for *online* sort

Attachment 13   MIDASsort FORTRAN sort routine sync.f

Attachment 14   Spectrum titles for the above sort routine

Note

1) data is discontinuous as a function of timestamp because we are sampling buffers
from the TapeServer rather than sorting all data as above
2) a low rate of timewarps is reported which appears to be an artifact of the buffer
sampling - no timewarps reported by analyses of the corresponding data files
 
Attachment 1: 100.png
100.png
Attachment 2: 101.png
101.png
Attachment 3: 103.png
103.png
Attachment 4: 104.png
104.png
Attachment 5: 105.png
105.png
Attachment 6: 106.png
106.png
Attachment 7: 107.png
107.png
Attachment 8: 110.png
110.png
Attachment 9: 112.png
112.png
Attachment 10: 113.png
113.png
Attachment 11: 116.png
116.png
Attachment 12: 117.png
117.png
Attachment 13: sync.f
*trigger
128  
*oned
1..32       s 65536
101..132       s 65536
*twod
1000        s 4096 4096
*vars
*sort
C----67---------------------------------------------------------------72------80
      SUBROUTINE init

      IMPLICIT none

      SAVE                                 
C
C     External functions
C
      EXTERNAL dtime
C
      INTEGER and, lshift, rshift
C
      LOGICAL btest
C
      REAL float, secnds
C
C     Parameter variables
C
      INTEGER max
      PARAMETER (max = 2048)
C
C     Local variables
C
      INTEGER adc_data, asic, ch, channel, channel_ident
      INTEGER count, data(0:4095), disc
      INTEGER*8 dt
      INTEGER events, events_old
      INTEGER*8 e_time, e_time_old
      INTEGER i
      INTEGER*2 i2(2), i2_2(4)
      INTEGER i4, i4_2(2)
      INTEGER*8 i8
      INTEGER ierr, information, information_index, int
      INTEGER invalid_id, items, items_old, j, j1, j1_old, j2, j2_old, k
      INTEGER l, mbs_data(0:3), module, m_disc
      INTEGER range, time_warp, total
      INTEGER*8 ts
      INTEGER ts28
      INTEGER ts48
      INTEGER*8 ts_old
      INTEGER type, w(0:31)
C
      LOGICAL first, hit(0:4095)
C
      REAL c(0:7), delta_t, gain(0:max-1), offset(0:max-1)
      REAL t1, t2, t2_old
C
C     Namelists
C
      NAMELIST /variables/ gain, offset, c, w
C
C     Common variables
C
      INTEGER*2 gdata(0:max-1)
C
      COMMON /fdata/ gdata
C
C     Common variables
C
      INTEGER*2 gid(0:max-1)
C
      COMMON /fid/ gid
C
C     Common variables
C
      INTEGER mult
C
      COMMON /fmult/ mult
C
      EQUIVALENCE ( i8, i4_2(1) )
      EQUIVALENCE ( i4, i2(1) )
C
C----67---------------------------------------------------------------72------80

      WRITE ( 6, * ) ' *** AIDA @ RIKEN - sync check - October 2016'
      WRITE ( 6, * ) ' *** Entry init commences'

C     Initialise counters

      events = 0
      events_old = 0

      time_warp = 0
      invalid_id = 0

C     Initialise event data

      DO i = 0, 2047
       data( i ) = 0
       hit( i ) = .FALSE.
      ENDDO

      first = .true.
      ts_old = 0
      e_time_old = 0
      count = 0
      total = 0

      DO i = 0, 127
       gain( i ) = 1.0
       offset( i ) = 0.0
      ENDDO
 
      DO i = 0, 31
       w( i ) = 0
      ENDDO

      DO i = 0, 7
       c( i ) = 1.0
      ENDDO

C     Initialise time

      t1 = SECNDS(0.0)

C     Read program variables via NAMELIST I/O

      OPEN( 1, FILE = '/home/td/GREAT/variables.dat',
     +         IOSTAT = ierr )
      IF ( ierr.NE.0 ) THEN
       WRITE( 6, * ) ' *** OPEN I/O error:', ierr
       RETURN
      ENDIF
      READ( 1, NML = variables, IOSTAT = ierr )
      IF ( ierr.NE.0 ) THEN
       WRITE( 6, * ) ' *** READ I/O error:', ierr
      ENDIF
      CLOSE( 1, IOSTAT = ierr )
      IF ( ierr.NE.0 ) THEN
       WRITE( 6, * ) ' *** CLOSE I/O error:', ierr
       RETURN
      ENDIF  

C     Display program variables

      DO i = 0, 127, 8
       WRITE( 6, 9001 ) i, i+7, ( gain( j ), j = i, i+7 )
      ENDDO

      DO i = 0, 127, 8
       WRITE( 6, 9002 ) i, i+7, ( offset( j ), j = i, i+7 )
      ENDDO

      DO i = 0, 7, 8
       WRITE( 6, 9003 ) i, i+7, ( c(j), j = 1, 8 )
      ENDDO

      DO i = 0, 31, 8
       WRITE( 6, 9004 ) i, i+7, ( w(j), j = i, i+7 )
      ENDDO

      WRITE ( 6, * ) ' *** Entry init ends'

      RETURN

C----67---------------------------------------------------------------72------80

      ENTRY sortin

      items = items + 1

      IF ( MOD( items, 10000000 ).EQ.0 ) THEN
       t2 = SECNDS( t1 )
       delta_t = t2 - t2_old
       WRITE( 6, 9010 ) items, FLOAT( items - items_old ) / delta_t,
     +                  events, FLOAT( events - events_old ) / delta_t
       items_old = items
       events_old = events
       t2_old = t2
      ENDIF

C----67---------------------------------------------------------------72------80
C     Event data

C     For Linux/x64 & Windows/x86 least significant 16-bit word i2(1)
C     For Solaris/SPARC least significant 16-bit word i2(2)

C     GREAT 3.2.1 format
C
C     Other information
C      gid[0]=-1
C     gdata[0]=module number
C     gdata[1]=information type (2=Pause, 3=Resume, 4=SYNC100, 6=FEE64 disc etc)
C     gdata[2]=information field (bits 16-19)
C     gdata[3]=information field (bits 0-15)
C     gdata[4]=time stamp (bits 16-27)
C     gdata[5]=time stamp (bits 0-15) 
C
C     ADC data format
C
C       gid[0]=channel ident (FEE64 module bits 6-11, channel bits 0-5)
C     gdata[0]=fail/veto bits (AIDA: fail=0, veto=0 LEC/MEC, veto=1 HEC)
C     gdata[1]=?
C     gdata[2]=ADC data
C     gdata[3]=time stamp (bits 16-27)
C     gdata[4]=time stamp (bits 0-15) 

C----67---------------------------------------------------------------72------80
C     gid = - 1   Other information

      IF ( gid(0).EQ.-1 ) THEN

       i2(2) = 0
       i2(1) = gdata(0)
       module = AND( i4, Z'0000003f' )
       i2(1) = gdata(1)
       type = AND( i4, Z'0000000f' )
       i2(2) = gdata(2)
       i2(1) = gdata(3)
       information = AND( i4, Z'000fffff' )
       i2(2) = gdata(4)
       i2(1) = gdata(5)
       ts28 = AND( i4, Z'0fffffff' )

       IF ( type.EQ.2 ) THEN

C----67---------------------------------------------------------------72------80
C     PAUSE timestamp

        ts48 = information

        i8 = 0
        i4_2( 1 ) = ts48
        ts = LSHIFT( i8, 28 )
        i8 = 0
        i4_2(1) = ts28
        ts = ts + i8

       ELSEIF( type.EQ.3 ) THEN

C----67---------------------------------------------------------------72------80
C     RESUME timestamp

        ts48 = information

        i8 = 0
        i4_2( 1 ) = ts48
        ts = LSHIFT( i8, 28 )
        i8 = 0
        i4_2(1) = ts28
        ts = ts + i8

       ELSEIF( type.EQ.4 ) THEN

C----67---------------------------------------------------------------72------80
C     SYNC100 timestamp

        ts48 = information

        i8 = 0
        i4_2( 1 ) = ts48
        ts = LSHIFT( i8, 28 )
        i8 = 0
        i4_2(1) = ts28
        ts = ts + i8

       ELSEIF( type.EQ.6 ) THEN

C----67---------------------------------------------------------------72------80
C     FEE64 discriminator

        i8 = 0
        IF ( ts28.LE.Z'00000A0' ) THEN
         i4_2( 1 ) = ts48 + 1
        ELSE
         i4_2( 1 ) = ts48
        ENDIF
        ts = LSHIFT( i8, 28 )
        i8 = 0
        i4_2(1) = ts28
        ts = ts + i8

        disc = AND( information, Z'0000ffff' )
        asic = AND( RSHIFT( information, 16 ), Z'0000000f' )
        m_disc = 0
        DO i = 0, 15
         IF ( btest( disc, i ) ) THEN
          m_disc = m_disc + 1
         ENDIF
        ENDDO

C       Increment ts spectra per FEE64 module
        j1 = MOD( RSHIFT( ts, 11 ), 65536 )
        CALL incv1d( module + 100, j1, m_disc)

C       Increment ts spectra per FEE64 module
        j2 = MOD( RSHIFT( ts, 11 ), 65536 )
        ch = ( ( module - 1 ) * 4 ) + asic + 100
        CALL incv2d( 1000, ch, j2, m_disc )

... 186 more lines ...
Attachment 14: titles.dat
s1 nnaida1: ADC data items (20us/ch)
s2 nnaida2: ADC data items (20us/ch)
s3 nnaida3: ADC data items (20us/ch)
s4 nnaida4: ADC data items (20us/ch)
s5 nnaida5: ADC data items (20us/ch)
s6 nnaida6: ADC data items (20us/ch)
s7 nnaida7: ADC data items (20us/ch)
s8 nnaida8: ADC data items (20us/ch)
s9 nnaida9: ADC data items (20us/ch)
s10 nnaida10: ADC data items (20us/ch)
s11 nnaida11: ADC data items (20us/ch)
s12 nnaida12: ADC data items (20us/ch)
s13 nnaida13: ADC data items (20us/ch)
s14 nnaida14: ADC data items (20us/ch)
s15 nnaida15: ADC data items (20us/ch)
s16 nnaida16: ADC data items (20us/ch)
s17 nnaida17: ADC data items (20us/ch)
s18 nnaida18: ADC data items (20us/ch)
s19 nnaida19: ADC data items (20us/ch)
s20 nnaida20: ADC data items (20us/ch)
s21 nnaida21: ADC data items (20us/ch)
s22 nnaida22: ADC data items (20us/ch)
s23 nnaida23: ADC data items (20us/ch)
s24 nnaida24: ADC data items (20us/ch)
s25 nnaida25: ADC data items (20us/ch)
s26 nnaida26: ADC data items (20us/ch)
s27 nnaida27: ADC data items (20us/ch)
s28 nnaida28: ADC data items (20us/ch)
s29 nnaida29: ADC data items (20us/ch)
s30 nnaida30: ADC data items (20us/ch)
s31 nnaida31: ADC data items (20us/ch)
s32 nnaida32: ADC data items (20us/ch)

s101 nnaida1: Disc data items (20us/ch)
s102 nnaida2: Disc data items (20us/ch)
s103 nnaida3: Disc data items (20us/ch)
s104 nnaida4: Disc data items (20us/ch)
s105 nnaida5: Disc data items (20us/ch)
s106 nnaida6: Disc data items (20us/ch)
s107 nnaida7: Disc data items (20us/ch)
s108 nnaida8: Disc data items (20us/ch)
s109 nnaida9: Disc data items (20us/ch)
s110 nnaida10: Disc data items (20us/ch)
s111 nnaida11: Disc data items (20us/ch)
s112 nnaida12: Disc data items (20us/ch)
s113 nnaida13: Disc data items (20us/ch)
s114 nnaida14: Disc data items (20us/ch)
s115 nnaida15: Disc data items (20us/ch)
s116 nnaida16: Disc data items (20us/ch)
s117 nnaida17: Disc data items (20us/ch)
s118 nnaida18: Disc data items (20us/ch)
s119 nnaida19: Disc data items (20us/ch)
s120 nnaida20: Disc data items (20us/ch)
s121 nnaida21: Disc data items (20us/ch)
s122 nnaida22: Disc data items (20us/ch)
s123 nnaida23: Disc data items (20us/ch)
s124 nnaida24: Disc data items (20us/ch)
s125 nnaida25: Disc data items (20us/ch)
s126 nnaida26: Disc data items (20us/ch)
s127 nnaida27: Disc data items (20us/ch)
s128 nnaida28: Disc data items (20us/ch)
s129 nnaida29: Disc data items (20us/ch)
s130 nnaida30: Disc data items (20us/ch)
s131 nnaida31: Disc data items (20us/ch)
s132 nnaida32: Disc data items (20us/ch)

s1000 4*(module # - 1) + asic # (0-95 ADC; 100-195 Disc) versus timestamp (20us/ch)

  408   Sun Oct 30 01:53:13 2016 TDSunday 30 October
10.53 DAQ found stalled 
      Most recent files R89_102 last accessed 07.17

      DSSSD bias & leakage current OK - see attachments 1 & 2
      FEE64 temperatures OK - see attachment 3

      All system wide checks OK *except* Clock status, SYNC errors & SYNC pulses - see attachments 4-8 - unusual
messages/reports

      One merger error messages - link 18 - see attachment 9

      Restart merger
      DAQ STOP OK
      DAQ RESET & SETUP OK
      ReSYNC OK
      All system-wide checks OK
      Disable H & data transfer #1
      Restart merger & tapeserver
      Enable H & data transfer #1

      Standard soft reset did not work - one or more FEE64s dropped data transfer #1 link at DAQ GO

      Retry - one, more FEE64s indicate error state - run control tab unresponsive 

      Power cycle FEE64s 
Attachment 1: 1.png
1.png
Attachment 2: 2.png
2.png
Attachment 3: 3.png
3.png
Attachment 4: 4.png
4.png
Attachment 5: 5.png
5.png
Attachment 6: 6.png
6.png
Attachment 7: 7.png
7.png
Attachment 8: 8.png
8.png
Attachment 9: 9.png
9.png
  409   Sun Oct 30 03:26:55 2016 TDReport - medium - cannot restore layout if currently displaying >16 galleries
Cannot restore layout if currently displaying >16 galleries

Obtain the error message 'loading Spectrum.tcl > can't read "dbtagx(31,1)": no such element in array'
  410   Sun Oct 30 04:37:31 2016 TDSpectrum layouts created
The following spectrum layouts have been created

1  All Rate spectra
2  All HitRate Spectra
3  p+n junction strip 1.8.L spectra
4  n+n ohmic strip 1.8.L spectra
5  p+n junction strip 1.8.H spectra
6  n+n ohmic strip 1.8.H spectra
5  p+n junction strip 1.8.W waveforms
6  n+n ohmic strip 1.8.W waveforms

The layouts can be found in the directory ~/LayOut and should be copied to /tmp

N.B Currently you cannot restore a layout to a spectrum display with > 16 galleries.
    If spectrum display uses > 16 galleries, switch spectrum display to 16 galleries
    or less before restoring a spectrum layout.
Attachment 1: LayOut1.mlf
30
gallery=1 overlap=1 server=nnaida1 spectrum=Rate
gallery=2 overlap=1 server=nnaida2 spectrum=Rate
gallery=3 overlap=1 server=nnaida3 spectrum=Rate
gallery=4 overlap=1 server=nnaida4 spectrum=Rate
gallery=5 overlap=1 server=nnaida5 spectrum=Rate
gallery=6 overlap=1 server=nnaida6 spectrum=Rate
gallery=7 overlap=1 server=nnaida7 spectrum=Rate
gallery=8 overlap=1 server=nnaida8 spectrum=Rate
gallery=9 overlap=1 server=nnaida9 spectrum=Rate
gallery=10 overlap=1 server=nnaida10 spectrum=Rate
gallery=11 overlap=1 server=nnaida11 spectrum=Rate
gallery=12 overlap=1 server=nnaida12 spectrum=Rate
gallery=13 overlap=1 server=nnaida13 spectrum=Rate
gallery=14 overlap=1 server=nnaida14 spectrum=Rate
gallery=15 overlap=1 server=nnaida15 spectrum=Rate
gallery=16 overlap=1 server=nnaida16 spectrum=Rate
gallery=17 overlap=1 server=nnaida17 spectrum=Rate
gallery=18 overlap=1 server=nnaida18 spectrum=Rate
gallery=19 overlap=1 server=nnaida19 spectrum=Rate
gallery=20 overlap=1 server=nnaida20 spectrum=Rate
gallery=21 overlap=1 server=nnaida21 spectrum=Rate
gallery=22 overlap=1 server=nnaida22 spectrum=Rate
gallery=23 overlap=1 server=nnaida23 spectrum=Rate
gallery=24 overlap=1 server=nnaida24 spectrum=Rate
Attachment 2: LayOut2.mlf
30
gallery=1 overlap=1 server=nnaida1 spectrum=HitRate
gallery=2 overlap=1 server=nnaida2 spectrum=HitRate
gallery=3 overlap=1 server=nnaida3 spectrum=HitRate
gallery=4 overlap=1 server=nnaida4 spectrum=HitRate
gallery=5 overlap=1 server=nnaida5 spectrum=HitRate
gallery=6 overlap=1 server=nnaida6 spectrum=HitRate
gallery=7 overlap=1 server=nnaida7 spectrum=HitRate
gallery=8 overlap=1 server=nnaida8 spectrum=HitRate
gallery=9 overlap=1 server=nnaida9 spectrum=HitRate
gallery=10 overlap=1 server=nnaida10 spectrum=HitRate
gallery=11 overlap=1 server=nnaida11 spectrum=HitRate
gallery=12 overlap=1 server=nnaida12 spectrum=HitRate
gallery=13 overlap=1 server=nnaida13 spectrum=HitRate
gallery=14 overlap=1 server=nnaida14 spectrum=HitRate
gallery=15 overlap=1 server=nnaida15 spectrum=HitRate
gallery=16 overlap=1 server=nnaida16 spectrum=HitRate
gallery=17 overlap=1 server=nnaida17 spectrum=HitRate
gallery=18 overlap=1 server=nnaida18 spectrum=HitRate
gallery=19 overlap=1 server=nnaida19 spectrum=HitRate
gallery=20 overlap=1 server=nnaida20 spectrum=HitRate
gallery=21 overlap=1 server=nnaida21 spectrum=HitRate
gallery=22 overlap=1 server=nnaida22 spectrum=HitRate
gallery=23 overlap=1 server=nnaida23 spectrum=HitRate
gallery=24 overlap=1 server=nnaida24 spectrum=HitRate
Attachment 3: LayOut3.mlf
16
gallery=1 overlap=1 server=nnaida1 spectrum=1.8.L
gallery=2 overlap=1 server=nnaida2 spectrum=1.8.L
gallery=3 overlap=1 server=nnaida5 spectrum=1.8.L
gallery=4 overlap=1 server=nnaida6 spectrum=1.8.L
gallery=5 overlap=1 server=nnaida9 spectrum=1.8.L
gallery=6 overlap=1 server=nnaida10 spectrum=1.8.L
gallery=7 overlap=1 server=nnaida13 spectrum=1.8.L
gallery=8 overlap=1 server=nnaida14 spectrum=1.8.L
gallery=9 overlap=1 server=nnaida17 spectrum=1.8.L
gallery=10 overlap=1 server=nnaida18 spectrum=1.8.L
gallery=11 overlap=1 server=nnaida21 spectrum=1.8.L
gallery=12 overlap=1 server=nnaida22 spectrum=1.8.L
Attachment 4: LayOut4.mlf
16
gallery=1 overlap=1 server=nnaida3 spectrum=1.8.L
gallery=2 overlap=1 server=nnaida4 spectrum=1.8.L
gallery=3 overlap=1 server=nnaida7 spectrum=1.8.L
gallery=4 overlap=1 server=nnaida8 spectrum=1.8.L
gallery=5 overlap=1 server=nnaida11 spectrum=1.8.L
gallery=6 overlap=1 server=nnaida12 spectrum=1.8.L
gallery=7 overlap=1 server=nnaida15 spectrum=1.8.L
gallery=8 overlap=1 server=nnaida16 spectrum=1.8.L
gallery=9 overlap=1 server=nnaida19 spectrum=1.8.L
gallery=10 overlap=1 server=nnaida20 spectrum=1.8.L
gallery=11 overlap=1 server=nnaida23 spectrum=1.8.L
gallery=12 overlap=1 server=nnaida24 spectrum=1.8.L
Attachment 5: LayOut5.mlf
16
gallery=1 overlap=1 server=nnaida1 spectrum=1.8.H
gallery=2 overlap=1 server=nnaida2 spectrum=1.8.H
gallery=3 overlap=1 server=nnaida5 spectrum=1.8.H
gallery=4 overlap=1 server=nnaida6 spectrum=1.8.H
gallery=5 overlap=1 server=nnaida9 spectrum=1.8.H
gallery=6 overlap=1 server=nnaida10 spectrum=1.8.H
gallery=7 overlap=1 server=nnaida13 spectrum=1.8.H
gallery=8 overlap=1 server=nnaida14 spectrum=1.8.H
gallery=9 overlap=1 server=nnaida17 spectrum=1.8.H
gallery=10 overlap=1 server=nnaida18 spectrum=1.8.H
gallery=11 overlap=1 server=nnaida21 spectrum=1.8.H
gallery=12 overlap=1 server=nnaida22 spectrum=1.8.H
Attachment 6: LayOut5.mlf
16
gallery=1 overlap=1 server=nnaida1 spectrum=1.8.H
gallery=2 overlap=1 server=nnaida2 spectrum=1.8.H
gallery=3 overlap=1 server=nnaida5 spectrum=1.8.H
gallery=4 overlap=1 server=nnaida6 spectrum=1.8.H
gallery=5 overlap=1 server=nnaida9 spectrum=1.8.H
gallery=6 overlap=1 server=nnaida10 spectrum=1.8.H
gallery=7 overlap=1 server=nnaida13 spectrum=1.8.H
gallery=8 overlap=1 server=nnaida14 spectrum=1.8.H
gallery=9 overlap=1 server=nnaida17 spectrum=1.8.H
gallery=10 overlap=1 server=nnaida18 spectrum=1.8.H
gallery=11 overlap=1 server=nnaida21 spectrum=1.8.H
gallery=12 overlap=1 server=nnaida22 spectrum=1.8.H
Attachment 7: LayOut7.mlf
16
gallery=1 overlap=1 server=nnaida1 spectrum=1.8.W
gallery=2 overlap=1 server=nnaida2 spectrum=1.8.W
gallery=3 overlap=1 server=nnaida5 spectrum=1.8.W
gallery=4 overlap=1 server=nnaida6 spectrum=1.8.W
gallery=5 overlap=1 server=nnaida9 spectrum=1.8.W
gallery=6 overlap=1 server=nnaida10 spectrum=1.8.W
gallery=7 overlap=1 server=nnaida13 spectrum=1.8.W
gallery=8 overlap=1 server=nnaida14 spectrum=1.8.W
gallery=9 overlap=1 server=nnaida17 spectrum=1.8.W
gallery=10 overlap=1 server=nnaida18 spectrum=1.8.W
gallery=11 overlap=1 server=nnaida21 spectrum=1.8.W
gallery=12 overlap=1 server=nnaida22 spectrum=1.8.W
Attachment 8: LayOut8.mlf
16
gallery=1 overlap=1 server=nnaida3 spectrum=1.8.W
gallery=2 overlap=1 server=nnaida4 spectrum=1.8.W
gallery=3 overlap=1 server=nnaida7 spectrum=1.8.W
gallery=4 overlap=1 server=nnaida8 spectrum=1.8.W
gallery=5 overlap=1 server=nnaida11 spectrum=1.8.W
gallery=6 overlap=1 server=nnaida12 spectrum=1.8.W
gallery=7 overlap=1 server=nnaida15 spectrum=1.8.W
gallery=8 overlap=1 server=nnaida16 spectrum=1.8.W
gallery=9 overlap=1 server=nnaida19 spectrum=1.8.W
gallery=10 overlap=1 server=nnaida20 spectrum=1.8.W
gallery=11 overlap=1 server=nnaida23 spectrum=1.8.W
gallery=12 overlap=1 server=nnaida24 spectrum=1.8.W
  411   Mon Oct 31 02:00:01 2016 TDMonday 31 October
11.00 DAQ continues OK
      DAQ STOP OK
      Most recent files R91_568 last accessed 10.59

      DSSSD bias & leakage current OK - see attachments 1 & 2
      FEE64 temperatures OK - see attachment 3
      System wide checks OK *except* SYNC pulses & errors - see attachments 4-7

14.20 Per FEE64 screenshots p+n junction and n+n ohmic strip waveforms - see attachments 8-11
      Preamplifier output noise looks OK for all FEE64s *except* nnaida5 & nnaida6 (which
      have significantly higher ADC and disc rates than other FEE64s)

21.10 DAQ found stalled
      Most recent file R94_99 last accessed 18.35

      Restart merger
      DAQ STOP
      System wide checks 
      DAQ RESET & SETUP
      System-wide checks
      Disable H & data transfer #1
      Restart merger & tapeserver
      Enable H & data transfer #1

21.16 DAQ start (R96)
Attachment 1: 1.png
1.png
Attachment 2: 2.png
2.png
Attachment 3: 3.png
3.png
Attachment 4: 4.png
4.png
Attachment 5: 5.png
5.png
Attachment 6: 6.png
6.png
Attachment 7: 7.png
7.png
Attachment 8: 10.png
10.png
Attachment 9: 11.png
11.png
Attachment 10: 12.png
12.png
Attachment 11: 13.png
13.png
  412   Mon Oct 31 02:26:32 2016 TD[How To] Start Runs (October 2016)
All AIDA operations are controlled from the Dell T410 Workstation aidas1 using web browsers.

AIDA has multiple, independent data acquisition servers connected by an Ethernet network.

Response times to commands, information updates etc are ~10-60s - please be patient!



How To Start Run 

1) Workspace 1 - Firefox web browser - Run Control tab

   Select 'Update'

   Check DAQ stopped (green 'GO' button displayed)

   Check state of *all* data acquisition servers is 'stopped'
   Check histogramming and data transfer #1 is enabled for *all* data acquisition servers as indicated by 'H X' tags

   See attachment 1

2) Workspace 2 - Firefox web browser - TapeServer tab

   Select 'Reload'

   Check TapeServer stopped and disk file closed (green GO button displayed)

   See attachment 2

3) Workspace 2 - Firefox web browser - MERGE tab

   Select 'Reload'

   Check Merge Program paused - see banner 

   Merge State = GOing     :    paused     :    xfer enabled     :    output paused

   If not paused select 'Toggle Merge Program Pause State'

   See attachment 3

4) Workspace 1 - Firefox web browser - ASIC Control tab

   Select 'Reload'

   Check 'Act on ALL FEE64 modules?' selected

   Check 'Act on ALL ASICs' selected

   Select 'Select required function' -> 'Check ASIC Control'

   After ~60s a pop-up will appear showing the response from each ASIC of each data acquisition server.
   Check all ASICs are OK and status = 0x001F0000.
   Dismiss pop-up by selecting OK

   See attachments 4-5

5) Workspace 1 - Firefox web browser - Run Control tab 

   Start DAQ - select green GO button

   Wait (~60s) until state of all data acquisition servers (nnaida1, nnaida2 ... etc) updates 

   Check state of *all* data acquisition servers is 'going'
   Check histogramming and data transfer #1 is enabled for *all* data acquisition servers as indicated by 'H X' tags

   See attachment 6

6) Workspace 2 - Firefox web browser - MERGE tab

   Select 'Reload' several times and check *all* enabled data links (0-23) are active
         bright green = active, olive green = inactive, red = disabled 

   N.B. If some data links are inactive call for expert help and do NOT proceed!

   Observe active spinner in MERGER server terminal window

   See attachment 7

7) Workspace 2 - Firefox web browser - TapeServer tab

   Check 'Volume Name' and 'File Name'

   Start TapeServer and open disk file - select green 'GO' button

   See attachment 8

8) Workspace 2 - Firefox web browser - MERGE tab

   Select 'Reload' several times and check all enabled data links (0-23) are always active
         bright green = active, olive green = inactive, red = disabled 

   N.B. If some data links are *always* inactive call for expert help and do NOT proceed!

   Select 'Toggle Merge Program Pause State' to enable data merging 

   Select 'Reload' to observe merge rate (~ 9000 data items/s)

   Observe active spinners in MERGE and TapeServer server terminal windows adjacent to web browser

   See attachment 9

9) Workspace 1 - Firefox web browser - Run Control tab

   Select 'Start ASIC Data Readout'

   See attachment 10

10) Workspace 2 - Firefox web browser - MERGE tab

    Select Reload several times and check all enabled data links (0-23) are active
         bright green = active, olive green = inactive, red = disabled 

    N.B. If some data links are *never* active call for expert help and do NOT proceed!

    Select 'Reload' to observe merge rate (~ 0.5-2M data items/s)

    See attachment 11

11) Workspace 2 - Firefox web browser - TapeServer tab

    Select 'Reload' and observe number of blocks written to disk increasing

    See attachment 12

12) Workspace 1 - Firefox web browser - Control tab

    Select 'Save/Restore Settings'

    From 'Save/Restore Module Settings' tab

     select 'Reload'

     select 'Module Names' whose settings should be saved (usually all)

     select 'Generate new DB key' and make a note of the DB key generated
          the DB key will have the format YYYYMMMDD-HH.MM.SS

          The DB key will be used to name a directory to store AIDA settings
          and is created uid: npg, gid: npgstaff, perms 755. However, its
          contents are written uid: nfsnobody, gid nfsnobody. The default
          directory (DB Key) permissions do not work - change the permissions
          to 777 - before attempting to save data to this directory (DB Key).

     launch a terminal and change the permissions of the DB key directory using the command

     chmod 777 /MIDAS/DB/EXPERIMENTS/AIDA/[YYYYMMMDD-HH.MM.SS]

     select 'Save Settings' to save AIDA ASIC, LED and waveform settings to
     the directory (DB key)

11) Update BRIKEN Elog with the following information

    AIDA
    run filename (e.g. R99)
    [# data blocks]
    start date & time
    [stop date & time]
    ASIC settings DB key 
    additional comments
    your name

    Complete run sheet for written logbook
Attachment 1: 100.png
100.png
Attachment 2: 101.png
101.png
Attachment 3: 102.png
102.png
Attachment 4: 103.png
103.png
Attachment 5: 104.png
104.png
Attachment 6: 105.png
105.png
Attachment 7: 106.png
106.png
Attachment 8: 107.png
107.png
Attachment 9: 108.png
108.png
Attachment 10: 109.png
109.png
Attachment 11: 110.png
110.png
Attachment 12: 111.png
111.png
  413   Mon Oct 31 04:02:11 2016 TD[How To] Stop Runs (October 2016)
All AIDA operations are controlled from the Dell T410 Workstation aidas1 using web browsers.

AIDA has multiple, independent data acquisition servers connected by an Ethernet network.

Response times to commands, information updates etc are ~10-60s - please be patient!


How to Stop Run

1) Workspace 2 - Firefox web browser - TapeServer tab

   Select 'Reload'

   Check number of blocks written to disk is increasing

   See attachment 1

   If TapeServer data rate is zero see https://elog.ph.ed.ac.uk/AIDA/303

2) Workspace 2 - Firefox web browser - MERGE tab

   Select 'Reload'

   Check merge rate (~0.5-2M data items/s)

   See attachment 2

   If MERGER data rate is zero see https://elog.ph.ed.ac.uk/AIDA/303

3) Workspace 1 - Firefox web browser - Run Control tab

   Select 'Update'

   Check state of *all* data acquisition servers is 'going'
   Check histogramming and data transfer #1 are enabled as indicated by 'H X' tags

   Stop DAQ - select red 'STOP' button

   Wait (~60s) until state of all data acquisition servers (nnaida1, nnaida2 ... etc) updates 

   Check state of *all* data acquisition servers is 'stopped'
   Check histogramming and data transfer #1 are enabled as indicated by 'H X' tags

   See attachment 3

   N.B. Occasionally the state of all data acquisition servers does not change to 'stopped'
        and if this happens simply re-try by re-selecting 'STOP' 

4) Workspace 2 - Firefox web browser - MERGE tab

   Pause MERGE input - select 'Toggle Merge Program Pause State'

   See attachment 4

5) Workspace 2 - Firefox web browser - TapeServer tab

   Stop TapeServer and close disk file - select red 'STOP' button

   See attachment 5

6) Complete BRIKEN Elog with the following information

    AIDA
    run filename (e.g. R99)
    [# data blocks]
    start date & time
    [stop date & time]
    ASIC settings DB key
    additional comments
    your name

    Complete run sheet for written logbook
Attachment 1: 200.png
200.png
Attachment 2: 201.png
201.png
Attachment 3: 202.png
202.png
Attachment 4: 203.png
203.png
Attachment 5: 204.png
204.png
  414   Mon Oct 31 04:50:59 2016 TD[How to] Monitor AIDA (October 2016)
All AIDA operations are controlled from the Dell T410 Workstation aidas1 using web browsers.

AIDA has multiple, independent data acquisition servers connected by an Ethernet network.

Response times to commands, information updates etc are ~10-60s - please be patient!


Every 4h

1) Desktop 7 - 2x PUTTY terminal sessions

   Each PUTTY terminal session is connected to a CAEN N1419 4 ch HV Power Supply 

   Addr 00

   Ch 0 = DSSSD # 1
    :
    :
   Ch 3 = DSSSD # 4

   Addr 01

   Ch 0 = DSSSD # 5
   Ch 1 = DSSSD # 6

   Sequentially select each terminal and obtain a screenshot (Alt-PrtScn) of DSSSD #1 = #6 detector
   bias voltages and leakage currents
   Upload screenshots to BRIKEN Elog

   If B2F/F11 ambient temperature information is available please add this information to the Elog

   Compare DSSSD leakage currents (and temperatures) with previous observations 

2) Desktop 1 - Firefox web browser - Temperatures tab

   Select 'Reload'

   Check temperatures

       Virtex < 70 deg C
       PSU    < 45 deg C
       ASIC   < 70 deg C


   Note that the ASIC temperatures for some data acquisition servers sometimes read back as 0.0

   *If the temperature values above are exceeded switch off AIDA FEE64 electrical power* as follows

   Desktop 6 - Firefox web browser

   From 'Rly16 Control' select 'Switch ALL off'

   and *now* seek expert help


*If* DAQ running also


3) Desktop 1 - Firefox web browser - Run Control tab

   Select 'Update'

   Check state of *all* data acquisition servers is 'going'
   Check histogramming and data transfer #1 are enabled as indicated by 'H X' tags

5) Desktop 2 - Firefox web browser - MERGE tab

   Select 'Reload'

   Check merge rate (~0.5-2M data items/s) consistent with value observed at start of run

   Observe active spinners in MERGE and TapeServer server terminal windows adjacent to web browser

6) Desktop 2 - Firefox web browser - TapeServer tab

   Select 'Reload'

   Check data rate consistent with value observed at start of run
  415   Tue Nov 1 02:24:05 2016 TDTuesday 1 November
11.24 DAQ found stalled
      Most recent file R95_168 last accessed 06.31

      R95_168 ADC & disc data synchronised - see attachment 8
      spectrum s1000 - per ASIC data items as function of timestamp (20us/channel)

aidas1> ./analyser /TapeData/Sept2016/R95_168
 *** TDR format 3.3.0 analyser - TD - October 2016
 *** ERROR: READ I/O error:       5002
                   blocks:      14568
          ADC data format:  108169612 ( 1126585.0 Hz)
        Other data format:   10690700 (  111343.5 Hz)
 Sample trace data format:          0 (       0.0 Hz)
         Undefined format:          0 (       0.0 Hz)
   Other data format type:      PAUSE:         31 (       0.3 Hz)
                               RESUME:         30 (       0.3 Hz)
                              SYNC100:     877566 (    9139.8 Hz)
                           FEE64 disc:    9810241 (  102173.5 Hz)
                             MBS info:       2832 (      29.5 Hz)
                           Other info:          0 (       0.0 Hz)

   ADC data range bit set:       2160 (      22.5 Hz)

                Timewarps:        ADC:          0 (       0.0 Hz)
                                PAUSE:          0 (       0.0 Hz)
                               RESUME:          0 (       0.0 Hz)
                              SYNC100:          0 (       0.0 Hz)
                           FEE64 disc:          0 (       0.0 Hz)
                             MBS info:          0 (       0.0 Hz)
                            Undefined:          0 (       0.0 Hz)
                         Sample trace:          0 (       0.0 Hz)

 *** Timestamp elapsed time:       96.015 s
 FEE  elapsed dead time(s) elapsed idle time(s)
  1                0.000                0.000
  2                0.000                0.000
  3                0.000                0.000
  4                0.000                0.000
  5                1.994                2.016
  6                1.408                1.439
  7                0.000                0.000
  8                0.000                0.000
  9                0.000                0.000
 10                0.000                0.000
 11                0.000                0.000
 12                0.000                0.000
 13                0.000                0.000
 14                0.000                0.000
 15                0.015                0.000
 16                0.000                0.000
 17                0.000                0.000
 18                0.000                0.000
 19                0.045                0.000
 20                0.000                0.000
 21                0.000                0.000
 22                0.000                0.000
 23                0.099                0.000
 24                0.000                0.000
 25                0.000                0.000
 26                0.000                0.000
 27                0.000                0.000
 28                0.000                0.000
 29                0.000                0.000
 30                0.000                0.000
 31                0.000                0.000
 32                0.000                0.000

 *** Statistics
 FEE  ADC Data Other Data     Sample  Undefined      Pause     Resume    SYNC100       Disc        MBS     
Other   HEC Data
  0          0          0          0          0          0          0          0          0          0         
0          0
  1     350112      43200          0          0          0          0      36623       6577          0         
0         48
  2    1247240      43778          0          0          0          0      36622       7156          0         
0          0
  3    2547061      39987          0          0          0          0      36626       3361          0         
0          0
  4     881007      40879          0          0          0          0      36623       4256          0         
0          5
  5   23965264    2536358          0          0         13         12      35831    2500502          0         
0         38
  6   23702370    1860427          0          0         11         11      36086    1821487       2832         
0        844
  7    1895595      40743          0          0          0          0      36623       4120          0         
0          1
  8    1603094      41634          0          0          0          0      36626       5008          0         
0        325
  9    3180182    1849501          0          0          0          0      36625    1812876          0         
0          0
 10    2576178      97708          0          0          0          0      36622      61086          0         
0          0
 11     447806      39812          0          0          0          0      36625       3187          0         
0          0
 12     905803      39001          0          0          0          0      36618       2383          0         
0          0
 13    2226595     147679          0          0          0          0      36624     111055          0         
0          0
 14    3021228    2593022          0          0          0          0      36626    2556396          0         
0        513
 15    4146089      40401          0          0          2          2      36617       3780          0         
0          0
 16    4264696      39555          0          0          0          0      36619       2936          0         
0          0
 17    2934508     416504          0          0          0          0      36623     379881          0         
0          0
 18    3319005      56583          0          0          0          0      36619      19964          0         
0          0
 19    6346836      41482          0          0          2          2      36604       4874          0         
0          1
 20    2392653      39249          0          0          0          0      36623       2626          0         
0        283
 21    2056421      41512          0          0          0          0      36622       4890          0         
0         96
 22    2824822     522339          0          0          0          0      36624     485715          0         
0          5
 23    7402741      39352          0          0          3          3      36590       2756          0         
0          1
 24    3932306      39994          0          0          0          0      36625       3369          0         
0          0
 25          0          0          0          0          0          0          0          0          0         
0          0
 26          0          0          0          0          0          0          0          0          0         
0          0
 27          0          0          0          0          0          0          0          0          0         
0          0
 28          0          0          0          0          0          0          0          0          0         
0          0
 29          0          0          0          0          0          0          0          0          0         
0          0
 30          0          0          0          0          0          0          0          0          0         
0          0
 31          0          0          0          0          0          0          0          0          0         
0          0
 32          0          0          0          0          0          0          0          0          0         
0          0

 *** Timewarps
 FEE       ADC      Pause     Resume    SYNC100       Disc        MBS  Undefined    Samples
  0          0          0          0          0          0          0          0          0
  1          0          0          0          0          0          0          0          0
  2          0          0          0          0          0          0          0          0
  3          0          0          0          0          0          0          0          0
  4          0          0          0          0          0          0          0          0
  5          0          0          0          0          0          0          0          0
  6          0          0          0          0          0          0          0          0
  7          0          0          0          0          0          0          0          0
  8          0          0          0          0          0          0          0          0
  9          0          0          0          0          0          0          0          0
 10          0          0          0          0          0          0          0          0
 11          0          0          0          0          0          0          0          0
 12          0          0          0          0          0          0          0          0
 13          0          0          0          0          0          0          0          0
 14          0          0          0          0          0          0          0          0
 15          0          0          0          0          0          0          0          0
 16          0          0          0          0          0          0          0          0
 17          0          0          0          0          0          0          0          0
 18          0          0          0          0          0          0          0          0
 19          0          0          0          0          0          0          0          0
 20          0          0          0          0          0          0          0          0
 21          0          0          0          0          0          0          0          0
 22          0          0          0          0          0          0          0          0
 23          0          0          0          0          0          0          0          0
 24          0          0          0          0          0          0          0          0
 25          0          0          0          0          0          0          0          0
 26          0          0          0          0          0          0          0          0
 27          0          0          0          0          0          0          0          0
 28          0          0          0          0          0          0          0          0
 29          0          0          0          0          0          0          0          0
 30          0          0          0          0          0          0          0          0
 31          0          0          0          0          0          0          0          0
 32          0          0          0          0          0          0          0          0

 *** Program elapsed time:    2.449s ( 5948.019 blocks/s, 371.751 Mb/s)


      DSSSD bias & leakage current OK - see attachments 1 & 2
      FEE64 temperatures OK - see attachment 3
      System wide checks OK *except* SYNC pulses & errors - see attachments 4-7

      Restart merger
      DAQ STOP
      System wide checks 
      DAQ RESET & SETUP
      System-wide checks
      Disable H & data transfer #1
      Restart merger & tapeserver
      Enable H & data transfer #1

11.46 Start DAQ (R96)

16.28 Stop DAQ OK

16.30 Start DAQ (R97)
      BNC PB-4 pulser OFF (background run)

17.04 Stop DAQ

17.08 Ambient temperature +25.0 deg C, d.p. +8.4 deg C, RH 34.8%

17.11 Start DAQ (R98)
      BNC PB-4 pulser ON
      60Co cource (10-7023) placed on top of AIDA snout approximately above DSSSD #3

17.30 DAQ stalled due to excessive diak activity from data copying & analysis tasks
      Power cycle required

18.09 DAQ start (R99)

18.38 DAQ stalled 
      Most recent file R99_7 last accessed 18.38
      Power cycle rquired

19.42 DAQ start (R101)
      BNC PB-4 pulser ON
      60Co cource (10-7023) placed on top of AIDA snout approximately above DSSSD #3
(Phong's correction: 60Co source position was in front of AIDA snout toward upstream)
      ASIC settings 2016Nov01-18.02.17
Attachment 1: 1.png
1.png
Attachment 2: 2.png
2.png
Attachment 3: 3.png
3.png
Attachment 4: 4.png
4.png
Attachment 5: 5.png
5.png
Attachment 6: 6.png
6.png
Attachment 7: 7.png
7.png
Attachment 8: 8.png
8.png
  418   Thu Nov 3 02:29:45 2016 TD[How To] Startup AIDA (October 2016)
Assumes 3-phase ac power (to Julabo FL11006 recirculating chiller) and single-phase ac power
(to AIDA NIM bins, PSUs, network switches, USB-controlled ac mains relay and Raspberry Pi)
is connected and switched on.


All AIDA operations are controlled from the Dell T410 Workstation aidas1 using web browsers.

AIDA has multiple, independent data acquisition servers connected by an Ethernet network.

Response times to commands, information updates etc are ~10-60s - please be patient!


1) Start Julabo FL11006 Recirculating Chiller

   See manual (page 19)  http://www2.ph.ed.ac.uk/~td/AIDA/Information/Julabo%20FL11006.pdf 
    
   Set point +20 deg C

   Check flow of coolant by inspecting the two flow indicators which are located on top of
   the AIDA stand

   Do NOT proceed if the Julabo FL11006 Recirculating Chiller is not operating, or if there
   is little, or no, flow in one, or both, flow indicators.

2) Start interface to USB-controlled ac mains relay to AIDA data acquisition servers

   On the AIDA Dell T410 Workstation aidas1 switch to Desktop 7 and start a Firefox web browser
   
   Enter URL   http://nnrpi1:8015/AIDA/Rly16/

   See attachment 1

   Select 'Sequence ALL On' to powerup the AIDA data acquisition servers

   Check all relays Relay 1 to Relay 8 move from the 'Switch ON' to 'Switch OFF' row

   See attachment 2

   Wait 10 minutes (yes - really!) before proceeding

3) Start Httpd Data Acquisition server

   See attachment 3

   Switch to Desktop 1 and select the yellow MIDAS icon 'Httpd for DataAcquisition' which will start a terminal
   to display server messages as follows

System identified is CPU x86_64; Platform is unix; OS is Linux and Version is 2.6.32-504.8.1.el6.x86_64
Environment selected is CPU x64_64; Platform unix; OS Linux64 and Operating System Linux64
MIDASBASE = /MIDAS@aidas and MIDAS_LIBRARY = /MIDAS@aidas/TclHttpd/Linux64
PATH =
/MIDAS@aidas/bin_Linux64:/MIDAS@aidas/TclHttpd/Linux64:/homes/npg/root/bin:/homes/npg/root/bin:/homes/npg/root/lib:/MIDAS/Linux/bin64:/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin:/bin
package limit is not available: can't find package limit
Running with default file descriptor limit
package setuid is not available: can't find package setuid
Could not change to user 50 group 50: not owner
/debug user "debug" password "jhi+u8jzdbpl"
httpd started on port 8015

Custom startup from /MIDAS@aidas/config/TclHttpd/aidas1@8015/startup.tcl
DefineMessage defined with server at http://elog.ph.ed.ac.uk:8015/MessageLoggerServer
/DataBaseAccessServer
/NetVarService
/SigTaskService
Loaded MemSasAccess
/SpectrumService
loading tcl/AIDARunControl.tcl for namespace ::
RunControlClient provided
RunControlClient loaded
Completed custom startup from /MIDAS@aidas/TclHttpd/Html/AIDA/RunControl/stats.defn.tcl

   See attachment 4

4) Start web browser for interface to AIDA

   Switch to Desktop 1 and start Firefox web browser

   Enter URL     localhost:8015

   See attachment 5

5) Commence startup of AIDA interface

   Select 'AIDA Experiment Control'

   See attachment 6

6) From AIDA Experiment Control tab 

   Select 'AIDA Hardware Control'

   See attachment 7

   Select 'Data Acquisition Run Control'

7) Reset and setup all data acquisition servers

   From 'Data Acquisition Run Control' tab

   See attachment 8

   Select 'Act On ALL Data Acquisition Servers'

   The web page will update (~20s) to show the status of all of the configured AIDA FEE64
   data acquisition servers which are named nnaida1, nnaida2 ... etc

   See attachment 9

   Select 'Reset'

   Wait for update and check that the state of *all* data acquisition servers is 'reset'

   Select 'Setup'

   Wait for timestamp re-synchronisation pop-up

   See attachments 10-11

   Usually all data acquisition servers (nnaida1, nnaida2 ... etc) will ReSYNC OK and you may proceed to step 8.

   Sometimes one, or more, data acquisition servers report ReSYNC not complete.

   Re-select the 'AIDA Experiment Control' tab and select 'AIDA Hardware Control'

   See attachments 12-13

   From the 'AIDA Hardware Control' tab select 'Master Timestamp Control'

   From the 'Master Timestamp Control' tab

   Select 'Reload'

   Select FEE64 -> nnaida6

   Select 'Perform ReSYNC'

   Wait for timestamp re-synchronisation pop-up.
   All data acquisition servers 9Nnaida1, nnaoida2 .. etc) should ReSYNC OK - if they do not
   please seek expert help

8) Start ASIC controls

   Re-select the 'AIDA Experiment Control' tab and select 'AIDA Hardware Control'

   See attachments 12-13

   Select 'ASIC Control'

   See attachment 14

   From 'ASIC Control' tab
     select 'Act on ALL FEE64 modules' and await page update
     select 'Act on ALL ASICs' and await page update

   See attachment 15

9) Check FEE64 operating temperatures

   Re-select the 'AIDA Hardware Control' tab and select 'FEE Temperatures'

   See attachment 16

   Check temperatures

       Virtex < 70 deg C
       PSU    < 45 deg C
       ASIC   < 70 deg C

   Note that the ASIC temperatures for some data acquisition servers sometimes read back as 0.0
   *If the temperature values above are exceeded switch off AIDA FEE64 electrical power* as follows
   
   Desktop 6 - Firefox web browser

   From 'Rly16 Control' select 'Switch ALL off' and *now* seek expert help

10) Perform system-wide diagnostic checks

    Re-select the 'AIDA Hardware Control' tab and select 'System Wide Checks'

    See attachments 17-18

11) Start servers for data merging and output to disk

    Switch to Desktop 2 and start the 'Httpd for Merge/TapeServer', 'Merger' and 'TapeServer' servers
    by selecting the three adjacent MIDAS in sequence

    See attachment 19

    Three terminal sessions will pop-up to display server messages. These terminals should be moved to
    left of the desktop and arranged as shown 

    See attachment 20

    This will enable you to simultaneously observe the latest messages from each server and the activity
    spinners of the Merger and TapeServer servers 

12) Start web interface to Merger and TapeServer

    Start Firefox web browser

    Enter URL     localhost:8115

    See attachment 21

13) Open data file

    Select 'Tape Service'

    See attachment 22

    In the 'TapeServer Service' tab complete the Volume Name, Run Name and Run Number fields.
    Typically the volume name will the experiment ID (e.g. NP1234) and Run Name (R). 

    See attachment 23

    Select 'Go' 

    See attachment 24

    This will create an experiment directory if it does not already exist (e.g. /TapeData/NP1234)
    and will open a data file within this directory with a filename concatenating the Run Name
    and Run Number fields - if the selected file already exists a new filename (Run Number) will
    be suggested

14) Setup merger

    Re-select the 'AIDA' tab and select 'MERGE Control'

    See attachment 25

    Select 'Setup & Configure'

    See attachment 26

    Select 'Go'

    See attachment 27

    Select 'Toggle Merge Program Pause State' 

    See attachment 28

15) Enable histogramming and data transfer #1 links

    Switch to Desktop 1 and re-select 'Data Acquisition Run Control' tab

    Select Histogramming 'Enable All' and Data Transfer #1 'Enable All'

16) AIDA is now ready to commence collecting data 
    - you can proceed from step 5 of https://elog.ph.ed.ac.uk/AIDA/412
Attachment 1: 100.png
100.png
Attachment 2: 101.png
101.png
Attachment 3: 102.png
102.png
Attachment 4: 103.png
103.png
Attachment 5: 104.png
104.png
Attachment 6: 105.png
105.png
Attachment 7: 106.png
106.png
Attachment 8: 107.png
107.png
Attachment 9: 108.png
108.png
Attachment 10: 109.png
109.png
Attachment 11: 110.png
110.png
Attachment 12: 111.png
111.png
Attachment 13: 112.png
112.png
Attachment 14: 113.png
113.png
Attachment 15: 114.png
114.png
Attachment 16: 118.png
118.png
Attachment 17: 119.png
119.png
Attachment 18: 120.png
120.png
Attachment 19: 121.png
121.png
Attachment 20: 122.png
122.png
Attachment 21: 123.png
123.png
Attachment 22: 124.png
124.png
Attachment 23: 125.png
125.png
Attachment 24: 126.png
126.png
Attachment 25: 127.png
127.png
Attachment 26: 128.png
128.png
Attachment 27: 129.png
129.png
Attachment 28: 130.png
130.png
Attachment 29: 132.png
132.png
  430   Tue Nov 8 00:12:56 2016 TDReport - medium - TapeServer changes volume name following request for existing filename
When you request a filename that already exists and select OK to the prompt
suggesting a new filename, TapeServer adds an additional character to the 
volume name - see attachment 1
Attachment 1: 9.png
9.png
  446   Wed Nov 9 09:50:50 2016 TDNP1412 R20, R21
DAQ start (R20) failed - nnaida5 dropped data transfer #1 link

start 18.51
stop 19.30
AIDA filename R21
BRIKEN filename 161109_1858_767778Ni_21
BigRIPS filename 410
ASIC settings 2016Nov07_23.03.06
DSSSD bias & leakage currents OK see attachments 1 & 2
Degrader 1mm Al from start of run
F11 rate 450Hz
BigRIPS setting 76-78Ni

19.35 Offline analysis of R21_12

 *** DSSSD # 1 count:       289 old count:       141 dt:      1.01 s  HEC rate:    146.23 Hz
 *** DSSSD # 2 count:       253 old count:       127 dt:      1.01 s  HEC rate:    124.49 Hz
 *** DSSSD # 3 count:       292 old count:       143 dt:      1.01 s  HEC rate:    147.22 Hz
 *** DSSSD # 4 count:       284 old count:       138 dt:      1.01 s  HEC rate:    144.25 Hz
 *** DSSSD # 5 count:       210 old count:        98 dt:      1.01 s  HEC rate:    110.66 Hz
 *** DSSSD # 6 count:        46 old count:        25 dt:      1.01 s  HEC rate:     20.75 Hz

Scalers - see attachment 4

1 F11
2 Upstream plastic top
3 Upstream plastic bottom
4 Si top
5 Si bottom
6 AIDA plastic veto
7 F11 and (upstream plastic top or upstream plastic bottom)
8 F11 and AIDA plastic veto 

19.42 Online analysis confirms ADC data synchronised
Attachment 1: 1.png
1.png
Attachment 2: 2.png
2.png
Attachment 3: 3.png
3.png
Attachment 4: 4.png
4.png
  450   Thu Nov 10 11:57:35 2016 TDData directory NP1412 renamed as RIBF123R1
Data directory NP1412 renamed as RIBF123R1

... there are several NP1412 'experiments' in the current schedule ... 1412 may be a date
ELOG V3.1.4-unknown