Project

General

Profile

Feature #1376

JSON Conversion Overhead

Added by Dobbs, Adam almost 8 years ago. Updated over 6 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Python API
Target version:
Start date:
20 November 2013
Due date:
% Done:

100%

Estimated time:
87.00 h
Workflow:
New Issue

Description

This is currently being worked on by Ian, but I thought it would be good to record it here anyway. I have been using to callgrind to perform code profiling on MAUS, using MC with only the tracker detectors in use, to see which bits of the code cause the bottle necks. So far, the clear answer seems to be GEANT4 and the JSON conversions. The attached is a callgrind output file taken over a 2 spill run, with 100 events per spill (associated datacard also attached). KCacheGrind is a good way too view the output. I think this shows that removing the JSON conversion between mappers should be achieved before we need to do large scale data reconstruction, or during online running for Step IV.


Files

callgrind.out.20683 (10.7 MB) callgrind.out.20683 Dobbs, Adam, 20 November 2013 12:20
datacard_mc_helical (9.1 KB) datacard_mc_helical Dobbs, Adam, 20 November 2013 12:20
callgrind.out.16026 (11 MB) callgrind.out.16026 Dobbs, Adam, 21 November 2013 23:14
datacard_mc_helical (9.18 KB) datacard_mc_helical Dobbs, Adam, 21 November 2013 23:14
itaylor-map.tar.gz (17.5 MB) itaylor-map.tar.gz Taylor, Ian, 17 April 2014 13:58
valgrind.log (1.29 MB) valgrind.log Rogers, Chris, 02 May 2014 09:33
valgrind_reduced.log (7.78 KB) valgrind_reduced.log Rogers, Chris, 02 May 2014 09:33
valgrind.log (1.2 MB) valgrind.log Rogers, Chris, 29 May 2014 12:32

Related issues

Related to MAUS - Bug #1466: Memory problem in datastructure?ClosedRogers, Chris16 May 2014

Actions
Related to MAUS - Bug #1483: Celery fails to import MapCpp modulesClosedRogers, Chris05 June 2014

Actions

Also available in: Atom PDF