Bug #1704
Memory Leak
100%
Description
Investigate possible memory leak (related to Input/Output moving off json). Valgrind log attached...
Files
Related issues
Updated by Rogers, Chris over 8 years ago
Sample event loop with some extra verbosity, printing memory src/common_cpp/DataStructure/Spill.hh memory allocations
emitting Spill Default Constructor 0x621d8e0 <--- NEVER DESTRUCTED Spill Copy Constructor 0x6a83ea0 Spill Copy Constructor 0x6a83f20 Spill Destructor 0x6a83ea0 Spill Copy Constructor 0x6a83ea0 <---- NEVER DESTRUCTED Spill Destructor 0x6a83f20 TRANSFORM/MERGE/OUTPUT: Processed 1 events so far, 1 events in buffer. emitted Spill Copy Constructor 0x6a83f20 Spill Destructor 0x6a83f20 transforming Spill Copy Constructor 0x6a83f20 Spill Destructor 0x6a83f20 Spill Default Constructor 0x6ab4450 Spill Copy Constructor 0x6aae290 Spill Copy Constructor 0x6aae9f0 Spill Destructor 0x6aae290 Spill Destructor 0x6ab4450 Spill Copy Constructor 0x6ab4450 Spill Destructor 0x6ab4450 Spill Destructor 0x6aae9f0 Spill Default Constructor 0x6ab50b0 Spill Copy Constructor 0x6ab2ca0 Spill Copy Constructor 0x6a9b780 Spill Destructor 0x6ab2ca0 Spill Destructor 0x6ab50b0 Spill Copy Constructor 0x6ab50b0 Spill Copy Constructor 0x6ab2ca0 Spill Destructor 0x6ab50b0 Spill Destructor 0x6a9b780 Spill Copy Constructor 0x6a9b780 Spill Copy Constructor 0x6ab50b0 Spill Destructor 0x6a9b780 Spill Destructor 0x6ab2ca0 transformed Spill Copy Constructor 0x6ab2ca0 Spill Destructor 0x6ab2ca0 Spill Destructor 0x6ab50b0 merging merged saving OutputCppRoot::write_event a 4 Spill Default Constructor 0x6a9ed50 OutputCppRoot::write_event b 4 OutputCppRoot::write_event c 4 OutputCppRoot::write_event d 4 OutputCppRoot::write_event e 4 OutputCppRoot::write_event f 4 OutputCppRoot::write_event h 4 OutputCppRoot::write_event i 4 (destructing) Spill Destructor 0x6a9ed50 OutputCppRoot::write_event j 4 (destructed) OutputCppRoot::write_event k 4 saved
Updated by Rogers, Chris over 8 years ago
I found one... diff
=== modified file 'src/input/InputCppDAQData/InputCppDAQData.hh' --- src/input/InputCppDAQData/InputCppDAQData.hh 2015-06-18 14:48:59 +0000 +++ src/input/InputCppDAQData/InputCppDAQData.hh 2015-07-05 05:21:13 +0000 @@ -113,12 +113,12 @@ */ MAUS::Data _emit_cpp() { if (this->readNextEvent()) { - MAUS::Data *data_cpp = new MAUS::Data; <--- THIS WAS NEVER DELETED + MAUS::Data data_cpp; MAUS::Spill *spill_cpp = new MAUS::Spill; - data_cpp->SetSpill(spill_cpp); - this->getCurEvent(data_cpp); + data_cpp.SetSpill(spill_cpp); + this->getCurEvent(&data_cpp); _eventsCount++; - return *data_cpp; + return data_cpp; } else { throw(MAUS::Exception(Exception::recoverable, "Failed to read next event",
Updated by Rogers, Chris over 8 years ago
The other one appears to be pre-existing
=== modified file 'src/map/MapPyGroup/MapPyGroup.py' --- src/map/MapPyGroup/MapPyGroup.py 2014-07-24 10:23:22 +0000 +++ src/map/MapPyGroup/MapPyGroup.py 2015-07-05 19:40:46 +0000 @@ -155,6 +155,10 @@ for worker in self._workers: if not (hasattr(worker, "can_convert") and worker.can_convert): old_spill = converter.string_repr(nu_spill) + try: + converter.del_data_repr(nu_spill) + except TypeError: + pass else: old_spill = nu_spill nu_spill = worker.process(old_spill) @@ -163,7 +167,6 @@ except TypeError: pass old_spill = None - return nu_spill def death(self):
Updated by Rogers, Chris over 8 years ago
lp:~chris-rogers/maus/1704a now in test.
Note that I added a data card data_maximum_reference_count
which controls the maximum number of MAUS::Data objects allowed to exist. Should help with catching memory leaks I hope...
Updated by Rogers, Chris over 8 years ago
- I tried reconstructing 05679; test ran extremely quickly, suspect that MAUS was not doing anything (Step I data vs Step IV unpacker)
- I tried reconstructing 07155 (1 GB file, including tracker data) and it was still producing lots of data and working extremely slowly with lots of junk in the output. Maybe a leak deeper down in the datastructure or some issues in tracker recon?
- I tried reconstructing 06800 (Step IV, pre-tracker), I got through all 117 DAQ events okay with not too much leakage. I noted lots of things like:
WARNING: Tracker RealDataDigitization: DAQ data size does not match Recon data size, aborting
- Now trying 06818...
Updated by Rogers, Chris over 8 years ago
Resource usage now looks flat on 06818...
Updated by Rajaram, Durga over 8 years ago
That's great, Chris.
I see that 6818 doesn't have tracker data in it. I see a memory leakage when running against runs with tracker data [ I tried 7150 which is probably too large a run to try a load test on]. Since it looks like the framework is plugged as seen above, those leakages would have to be from the reconstruction side.
Updated by Rogers, Chris over 8 years ago
- Status changed from Open to Closed
- % Done changed from 0 to 100
Looks okay. Tracker memory leak is a separate issue.
Updated by Rajaram, Durga over 8 years ago
- Target version changed from Future MAUS release to MAUS-v1.0.0