Notes from the Software Workshop discussion

• How do we get data quality (really reconstruction quality) flags to the DB?

• Option A:
  • Have a post-processing job run on RAL PPD machine and get it to CDB via MICENet
    • But --
      • have to retrieve data (from CASTOR or a wget)
      • need to maintain a MAUS installation on a PPD machine (could be circumvented by CVMFS)
      • need to maintain a PPD machine and monitor cron jobs

• Option B:
  • Have the MAUS job on the GRID spit out the flags?
    • can't do it with a simple return value since it might not fit into a INT
      • currently the GRID job takes the exit status from MAUS and gets it into the IC Metadata DB
    • Solution: have MAUS write out an XML file to cwd which Janusz can then read, parse, and retrieve flags and get them into the IC Metadata DB
      • The flags will still need to get into the CDB
      • Janusz's solution is to have a cron job at RAL pull information from the IC Metadata DB and update the CD

• Option B seems better long-term

• What's needed to get the flags out of MAUS
  • Mappers to write a flag for each spill

    •  
         spill:{ 
                 data, 
                 mc, 
                 recon,
                 data_quality: {
                   "flag_name": T/F
                 }
         }

  • Each detector/mapper can write out several flags if necessary or potentially useful for analyses downstream
  • Need MC flags for Digitizers and Hits
  • Need flags from Global Reco and Global PID eventually
    • the datastructure should be put in place even if the mappers are not ready to send flags
  • Q: Do we need flags from the DAQ Unpacking? I think if we're going to setup flags, we should flag whatever could potentially be useful
  • Since the flags will be written by Mappers they will be spill-level
  • Need a Reducer at the end to go through the flags (which are spill-level) and put out a summary flag for each run
    • this can be based on a tolerance defined in a datacard
      • e.g. detector_digi_quality = True if (nTrue / (nTrue + nFalse) ) > detector_digi_tolerance