Bug #1456
Run 4000 job on GRID failed with memory leak
Added by Rogers, Chris over 9 years ago.
Updated over 9 years ago.
Description
Tried to reconstruct run 4000 on GRID with release MAUS-v0.8.1, the job failed with a memory leak.
Files
usage_offline.log (10.4 KB)
usage_offline.log |
usage log for analyze_data_offline.py |
Rajaram, Durga, 17 April 2014 05:10
|
|
analyze_do_nothing.py (1.23 KB)
analyze_do_nothing.py |
script - no mappers, just I/O |
Rajaram, Durga, 17 April 2014 05:10
|
|
usage_offline_donothing.log (1.81 KB)
usage_offline_donothing.log |
usage log for pure I/O without mappers |
Rajaram, Durga, 17 April 2014 05:10
|
|
analyze_data_offline_injson.py (1.24 KB)
analyze_data_offline_injson.py |
script - InputPyJSON instead of InputCppDAQData |
Rajaram, Durga, 17 April 2014 05:10
|
|
usage_offline_json-input.log (8.05 KB)
usage_offline_json-input.log |
usage log with InputPyJSON |
Rajaram, Durga, 17 April 2014 05:10
|
|
Hm this memory leak is interesting..
Some tests on my SLF 6.3, 64-bit, 4 GB memory, gcc 4.4.6, MAUS-v0.8.3
- running out-of-the-box analyze_data_offline on 04995.00, takes up ~14% of memory -- attached usage_offline.log
- the usage is the same even after turning off all mappers -- essentially just input-output which leads me to suspect something in the intput part of the data structure -- attached analyze_do_nothing.py and corresponding log usage_offline_donothing.log
Looks like this is related to the changes that were made to InputCppDAQData to take MAUS::Data as input -- #1366 -- ~v0.7.6
Durga, do you want to take this tracker (welcome to it!)
- Status changed from Open to Closed
- % Done changed from 0 to 100
Looks like InputCppDAQData was allocating a Json::Value containing the spill structure and then not deleting it. Diff below.
=== modified file 'src/input/InputCppDAQData/InputCppDAQData.cc'
--- src/input/InputCppDAQData/InputCppDAQData.cc 2014-04-15 10:52:10 +0000
+++ src/input/InputCppDAQData/InputCppDAQData.cc 2014-04-28 11:31:40 +0000
@@ -225,11 +225,15 @@
Json::Value* spill_json_out = MAUS::CppJsonSpillConverter().convert(data_cpp);
// std::cerr << *spill_json_out << std::endl;
- delete data_cpp;
_eventsCount++;
Json::FastWriter xJSONWr;
- return xJSONWr.write(*spill_json_out);
+ std::string output = xJSONWr.write(*spill_json_out);
+
+ delete spill_json_out;
+ delete data_cpp;
+
+ return output;
}
Committed as r710
- Target version changed from Future MAUS release to MAUS-v0.8.4
Also available in: Atom
PDF