Project

General

Profile

Bug #1585

Need reconstruction quality flags

Added by Rajaram, Durga over 6 years ago. Updated almost 6 years ago.

Status:
Open
Priority:
Normal
Assignee:
Category:
RealData
Target version:
Start date:
11 November 2014
Due date:
% Done:

0%

Estimated time:
Workflow:
New Issue

Description

We want to have reconstruction quality flags from MAUS to denote the status of the production reconstruction.
These, in addition to run quality flags from MLCR, will give analysis users a way of checking whether data is 'analyzable'.


Related issues

Related to Configuration Database - Feature #1689: Read Access for CDB data quality flagsOpen03 June 2015

Actions
#1

Updated by Rajaram, Durga over 6 years ago

Notes from the Software Workshop discussion

  • How do we get data quality (really reconstruction quality) flags to the DB?
  • Option A:
    • Have a post-processing job run on RAL PPD machine and get it to CDB via MICENet
      • But --
        • have to retrieve data (from CASTOR or a wget)
        • need to maintain a MAUS installation on a PPD machine (could be circumvented by CVMFS)
        • need to maintain a PPD machine and monitor cron jobs
  • Option B:
    • Have the MAUS job on the GRID spit out the flags?
      • can't do it with a simple return value since it might not fit into a INT
        • currently the GRID job takes the exit status from MAUS and gets it into the IC Metadata DB
      • Solution: have MAUS write out an XML file to cwd which Janusz can then read, parse, and retrieve flags and get them into the IC Metadata DB
        • The flags will still need to get into the CDB
        • Janusz's solution is to have a cron job at RAL pull information from the IC Metadata DB and update the CD
  • Option B seems better long-term
  • What's needed to get the flags out of MAUS
    • Mappers to write a flag for each spill
      •  
           spill:{ 
                   data, 
                   mc, 
                   recon,
                   data_quality: {
                     "flag_name": T/F
                   }
           }
    • Each detector/mapper can write out several flags if necessary or potentially useful for analyses downstream
    • Need MC flags for Digitizers and Hits
    • Need flags from Global Reco and Global PID eventually
      • the datastructure should be put in place even if the mappers are not ready to send flags
    • Q: Do we need flags from the DAQ Unpacking? I think if we're going to setup flags, we should flag whatever could potentially be useful
    • Since the flags will be written by Mappers they will be spill-level
    • Need a Reducer at the end to go through the flags (which are spill-level) and put out a summary flag for each run
      • this can be based on a tolerance defined in a datacard
        • e.g. detector_digi_quality = True if (nTrue / (nTrue + nFalse) ) > detector_digi_tolerance
#2

Updated by Dobbs, Adam almost 6 years ago

  • Category set to RealData
  • Assignee set to Dobbs, Adam

Done for KL in MAUS 0.9.6. Other detector groups have been informed...

Also available in: Atom PDF