Bug #1585
Need reconstruction quality flags
Start date:
11 November 2014
Due date:
% Done:
0%
Estimated time:
Workflow:
New Issue
Description
We want to have reconstruction quality flags from MAUS to denote the status of the production reconstruction.
These, in addition to run quality flags from MLCR, will give analysis users a way of checking whether data is 'analyzable'.
Related issues
Updated by Rajaram, Durga about 9 years ago
Notes from the Software Workshop discussion
- How do we get data quality (really reconstruction quality) flags to the DB?
- Option A:
- Have a post-processing job run on RAL PPD machine and get it to CDB via MICENet
- But --
- have to retrieve data (from CASTOR or a wget)
- need to maintain a MAUS installation on a PPD machine (could be circumvented by CVMFS)
- need to maintain a PPD machine and monitor cron jobs
- But --
- Have a post-processing job run on RAL PPD machine and get it to CDB via MICENet
- Option B:
- Have the MAUS job on the GRID spit out the flags?
- can't do it with a simple return value since it might not fit into a INT
- currently the GRID job takes the exit status from MAUS and gets it into the IC Metadata DB
- Solution: have MAUS write out an XML file to cwd which Janusz can then read, parse, and retrieve flags and get them into the IC Metadata DB
- The flags will still need to get into the CDB
- Janusz's solution is to have a cron job at RAL pull information from the IC Metadata DB and update the CD
- can't do it with a simple return value since it might not fit into a INT
- Have the MAUS job on the GRID spit out the flags?
- Option B seems better long-term
- What's needed to get the flags out of MAUS
- Mappers to write a flag for each spill
spill:{ data, mc, recon, data_quality: { "flag_name": T/F } }
- Each detector/mapper can write out several flags if necessary or potentially useful for analyses downstream
- Need MC flags for Digitizers and Hits
- Need flags from Global Reco and Global PID eventually
- the datastructure should be put in place even if the mappers are not ready to send flags
- Q: Do we need flags from the DAQ Unpacking? I think if we're going to setup flags, we should flag whatever could potentially be useful
- Since the flags will be written by Mappers they will be spill-level
- Need a Reducer at the end to go through the flags (which are spill-level) and put out a summary flag for each run
- this can be based on a tolerance defined in a datacard
- e.g. detector_digi_quality = True if (nTrue / (nTrue + nFalse) ) > detector_digi_tolerance
- this can be based on a tolerance defined in a datacard
- Mappers to write a flag for each spill
Updated by Dobbs, Adam over 8 years ago
- Category set to RealData
- Assignee set to Dobbs, Adam
Done for KL in MAUS 0.9.6. Other detector groups have been informed...