2015-02-24-MEMO C+SW

Schedule & Milestones

Group Item Milestone Persons
C&M Beamline & Detectors Run Control March 2, 2015 PMH
SS+FC SM and RC March 30, 2015 PMH
LH2 SM April 30, 2015 PMH
Nagios Interfacing May 15, 2015 PMH
B-field & PRY monitoring June 1, 2015 PMH
Online Integrated DAQ March 20, 2015 YK
Integrated Trigger May 1, 2015 YK
Offline Final Track Fit April 1, 2015 AD
Global Tracks & PID June 26, 2015 AD
Infrastructure Automated Data-movement April 1, 2015 PF
  • Milestones stand
  • C&M reports separately

Progress towards automated data processing

  • Description:
    • Data movement:
      • DAQ Data from GDC gets compacted along with the online reconstruction output and moved by a script (HN) to interim storage
      • It then gets copied from micestore to the datamover computer from where it gets copied to CASTOR, and registered in the DB
      • The steps subsequent to micestore are automated -- watch for new data data and copy/replicate
      • The script to move from the GDC to micestore is currently manually triggered by HN
      • At CM41 we had a a discussion on how to automate this -- options were either through the DAQ or from Run Control
      • It was decided to trigger this script at the end of run from Run Control
      • This week, Henry and Pierrick are meeting to hand over the script for RC integration
      • Pierrick will then integrate it into Run Control and test it during MDR2 and the March 8 running
    • Data processing:
      • Once the data are moved to CASTOR, we will reconstruct on RAL's fast response Tier-1 queue
      • During MDR1 there were issues with the job submission at RAL due to certificates
      • The issues were resolved last week. Janusz has successfully tested submission on the RAL queue with "Step IV" data from MDR1
    • Data reconstruction:
      • Recent changes to DATE mean MAUS needs a new version of the unpacking library to read in StepIV data
      • YK will release a new unpacking library by the end of this week
      • AD will then release a new version of MAUS with the updated unpacker
      • AD will then deploy the new version of MAUS on the online-reconstruction machines
      • JM/PF will install the new version of MAUS on CVFMS (the GRID) so that the GRID reconstruction can be performed on the latest MAUS

Initial consideration of processing time issues

  • MAUS currently processes data at a slower than desired rate
    • We are attempting to solve this by profiling MAUS using various standard tools to find the bottle necks and see if the can be eliminated or reduced
    • Initial studies indicate that a big slow down is caused by converting to and from strings when passing between MAUS modules
    • AD and DR in consultation with Chris Rogers are looking at ways to solve this by removing the string conversion - could be large job


  • Offline:
    • DAQ unpacking currently failing to link against new DATE libraries on SL5.
      • an upgrade to SL6 is expected to solve the problem
    • Unpacker fix to spot and repair corrupt DATE data not yet in MAUS trunk (YK)
    • EMR produces additional recon events in the data structure - bad practice, EMR group will be requested to address this.
    • Work ongoing by Chris Hunt, Adam and Ken to bring tracker reconstruction up to scratch.
    • Tracker DAQ unpacking / digitisation remains broken, though sensible VLSB data is now seen, extremely slow however. Concerned that this is somewhat orphaned at present.
  • Online + Infrastructure
    • miceonrec03 being upgraded to SL6
      • SL6 required for MAUS linking against new DATE libraries