Project

General

Profile

Bug #1456

Run 4000 job on GRID failed with memory leak

Added by Rogers, Chris almost 7 years ago. Updated almost 7 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Data Structure
Target version:
Start date:
16 April 2014
Due date:
% Done:

100%

Estimated time:
Workflow:
New Issue

Description

Tried to reconstruct run 4000 on GRID with release MAUS-v0.8.1, the job failed with a memory leak.


Files

usage_offline.log (10.4 KB) usage_offline.log usage log for analyze_data_offline.py Rajaram, Durga, 17 April 2014 05:10
analyze_do_nothing.py (1.23 KB) analyze_do_nothing.py script - no mappers, just I/O Rajaram, Durga, 17 April 2014 05:10
usage_offline_donothing.log (1.81 KB) usage_offline_donothing.log usage log for pure I/O without mappers Rajaram, Durga, 17 April 2014 05:10
analyze_data_offline_injson.py (1.24 KB) analyze_data_offline_injson.py script - InputPyJSON instead of InputCppDAQData Rajaram, Durga, 17 April 2014 05:10
usage_offline_json-input.log (8.05 KB) usage_offline_json-input.log usage log with InputPyJSON Rajaram, Durga, 17 April 2014 05:10

Related issues

Related to MAUS - Bug #1431: Memory leaks in MAUSClosedRajaram, Durga12 March 2014

Actions
#1

Updated by Rajaram, Durga almost 7 years ago

Hm this memory leak is interesting..

Some tests on my SLF 6.3, 64-bit, 4 GB memory, gcc 4.4.6, MAUS-v0.8.3

  • running out-of-the-box analyze_data_offline on 04995.00, takes up ~14% of memory -- attached usage_offline.log
  • the usage is the same even after turning off all mappers -- essentially just input-output which leads me to suspect something in the intput part of the data structure -- attached analyze_do_nothing.py and corresponding log usage_offline_donothing.log

Looks like this is related to the changes that were made to InputCppDAQData to take MAUS::Data as input -- #1366 -- ~v0.7.6

#2

Updated by Rogers, Chris almost 7 years ago

Durga, do you want to take this tracker (welcome to it!)

#3

Updated by Rogers, Chris almost 7 years ago

  • Status changed from Open to Closed
  • % Done changed from 0 to 100

Looks like InputCppDAQData was allocating a Json::Value containing the spill structure and then not deleting it. Diff below.

 
=== modified file 'src/input/InputCppDAQData/InputCppDAQData.cc'
--- src/input/InputCppDAQData/InputCppDAQData.cc    2014-04-15 10:52:10 +0000
+++ src/input/InputCppDAQData/InputCppDAQData.cc    2014-04-28 11:31:40 +0000
@@ -225,11 +225,15 @@

   Json::Value* spill_json_out = MAUS::CppJsonSpillConverter().convert(data_cpp);
 //   std::cerr << *spill_json_out << std::endl;
-  delete data_cpp;
   _eventsCount++;

   Json::FastWriter xJSONWr;
-  return xJSONWr.write(*spill_json_out);
+  std::string output = xJSONWr.write(*spill_json_out);
+
+  delete spill_json_out;
+  delete data_cpp;
+
+  return output;
 }

Committed as r710

#4

Updated by Rajaram, Durga almost 7 years ago

  • Target version changed from Future MAUS release to MAUS-v0.8.4

Also available in: Atom PDF