Bug #1935
"Error in <TExMap::Remove>" when running over large input files
Start date:
08 June 2017
Due date:
% Done:
0%
Estimated time:
Workflow:
New Issue
Description
When running over large input files, at some point MAUS will spit out millions of lines like this:
Error in <TExMap::Remove>: key 503057872 not found at 23565 Error in <TExMap::Remove>: key 292471040 not found at 5611 Error in <TExMap::Remove>: key 407814496 not found at 38014Steps to reproduce:
- Download a production MC (e.g. http://gfe02.grid.hep.ph.ic.ac.uk:8301/Simulation/MCproduction/000000/000000/000035/) - downloading 260+ files is sufficient
- Merge them, for example with the attached script (modify paths etc. as appropriate)
- Run MAUS over the merged file (it is not necessary to include any modules other than the inputter. Error output should start around Spill 506.
Files
Updated by Greis, Jan over 6 years ago
Some new information on this issue:
It appears to be a problem with MC only, I was able to run over merged files of real data (tested up to 30 GB) without problems. Even in MC the limiting factor is not file size, as running over MC with DS off (where the combined file size was ~1/4) the error occurred at around the same point.