2015-02-24-MEMO C+SW¶

Schedule & Milestones¶

• Milestones stand
• C&M reports separately

Progress towards automated data processing¶

• Description:
  • Data movement:
    • DAQ Data from GDC gets compacted along with the online reconstruction output and moved by a script (HN) to interim storage
    • It then gets copied from micestore to the datamover computer from where it gets copied to CASTOR, and registered in the DB
    • The steps subsequent to micestore are automated -- watch for new data data and copy/replicate
    • The script to move from the GDC to micestore is currently manually triggered by HN
    • At CM41 we had a a discussion on how to automate this -- options were either through the DAQ or from Run Control
    • It was decided to trigger this script at the end of run from Run Control
    • This week, Henry and Pierrick are meeting to hand over the script for RC integration
    • Pierrick will then integrate it into Run Control and test it during MDR2 and the March 8 running
  • Data processing:
    • Once the data are moved to CASTOR, we will reconstruct on RAL's fast response Tier-1 queue
    • During MDR1 there were issues with the job submission at RAL due to certificates
    • The issues were resolved last week. Janusz has successfully tested submission on the RAL queue with "Step IV" data from MDR1
  • Data reconstruction:
    • Recent changes to DATE mean MAUS needs a new version of the unpacking library to read in StepIV data
    • YK will release a new unpacking library by the end of this week
    • AD will then release a new version of MAUS with the updated unpacker
    • AD will then deploy the new version of MAUS on the online-reconstruction machines
    • JM/PF will install the new version of MAUS on CVFMS (the GRID) so that the GRID reconstruction can be performed on the latest MAUS

Initial consideration of processing time issues¶

• MAUS currently processes data at a slower than desired rate
  • We are attempting to solve this by profiling MAUS using various standard tools to find the bottle necks and see if the can be eliminated or reduced
  • Initial studies indicate that a big slow down is caused by converting to and from strings when passing between MAUS modules
  • AD and DR in consultation with Chris Rogers are looking at ways to solve this by removing the string conversion - could be large job

Other¶

• Offline:
  • DAQ unpacking currently failing to link against new DATE libraries on SL5.
    • an upgrade to SL6 is expected to solve the problem
  • Unpacker fix to spot and repair corrupt DATE data not yet in MAUS trunk (YK)
  • EMR produces additional recon events in the data structure - bad practice, EMR group will be requested to address this.
  • Work ongoing by Chris Hunt, Adam and Ken to bring tracker reconstruction up to scratch.
  • Tracker DAQ unpacking / digitisation remains broken, though sensible VLSB data is now seen, extremely slow however. Concerned that this is somewhat orphaned at present.

• Online + Infrastructure
  • miceonrec03 being upgraded to SL6
    • SL6 required for MAUS linking against new DATE libraries