Project

General

Profile

Bug #1248

Simulation time is slow

Added by Rogers, Chris about 10 years ago. Updated about 10 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Rogers, Chris
Category:
Simulation
Target version:
MAUS-v0.5.2
Start date:
11 April 2013
Due date:
% Done:

100%

Estimated time:
Workflow:
New Issue

Files

step_size.png (8.64 KB) step_size.png Rogers, Chris, 15 April 2013 13:56
n_steps.png (9.17 KB) n_steps.png Rogers, Chris, 15 April 2013 13:56
mean_bz.png (8.25 KB) mean_bz.png Rogers, Chris, 15 April 2013 13:59
transmission.png (6.95 KB) transmission.png Rogers, Chris, 15 April 2013 13:59
maus_Stage4_100_mm_step.dat.log (95.4 KB) maus_Stage4_100_mm_step.dat.log Rogers, Chris, 15 April 2013 14:36
losses_stage4.png (5.49 KB) losses_stage4.png Rogers, Chris, 24 April 2013 12:17
losses_stage4NoBeamline.png (6.48 KB) losses_stage4NoBeamline.png Rogers, Chris, 24 April 2013 12:17

Related issues

Related to MAUS - Bug #1261: Improve MC processing time in trackerOpenHeidt, Christopher26 April 2013

Actions
#1

Updated by Rogers, Chris about 10 years ago

Following are timings for 10000 muons passing in pencil beam through

  • CoilsOnly: tracker coil pack
  • TOFOnly: TOF1
  • TrackerOnly: Tracker0

configurations/CoilOnly.dat

real 1m1.679s
user 1m0.328s
sys 0m0.224s

configurations/TOFOnly.dat

real 4m55.821s
user 4m54.382s
sys 0m1.460s

configurations/TrackerOnly.dat

real 17m46.528s
user 17m44.851s
sys 0m4.128s

#2

Updated by Rogers, Chris about 10 years ago

n_particles_per_spill 10
spill_generator_number_of_spills 100
kinetic_energy_threshold 0.1

gives

configurations/CoilOnly.dat

real 0m15.190s
user 0m13.601s
sys 0m0.216s

configurations/Nothing.dat

real 0m8.247s
user 0m6.880s
sys 0m0.184s

configurations/TOFOnly.dat

real 0m33.481s
user 0m32.098s
sys 0m0.292s

configurations/TrackerOnly.dat

real 1m44.981s
user 1m43.586s
sys 0m0.552s

Stage4.dat

real 2m6.981s
user 2m5.372s
sys 0m0.244s

Physics Processes

Try setting physics_processes = "none"

configurations/Stage4.dat

real    3m35.098s
user    3m33.585s
sys    0m0.188s

Introduces some reduction in time (note we probably have much higher transmission here so deceptively large number)

Try introducing a kinetic energy threshold to prevent showering

kinetic_energy_threshold 10

gives

configurations/Stage4.dat

real    4m19.957s
user    4m18.444s
sys    0m0.704s

configurations/CoilOnly.dat

real    0m15.272s
user    0m13.753s
sys    0m0.212s

configurations/Nothing.dat

real    0m8.297s
user    0m6.876s
sys    0m0.228s

configurations/TOFOnly.dat

real    0m22.224s
user    0m20.745s
sys    0m0.260s

configurations/TrackerOnly.dat

real    1m35.876s
user    1m34.526s
sys    0m0.576s

Summary: slightly less time spent in TOF, but tracker is the main problem and that remains constant

Try just increasing step size - note that I found (see below) that there was a 1 mm step size limiter in the beamline region, which is taken out in these configurations; i.e. the step size is 100 mm everywhere including the beamline

configurations/Stage4_100_mm_step.dat

real    4m46.632s
user    4m45.170s
sys    0m1.016s

configurations/Stage4_10_mm_step.dat

real    5m8.047s
user    5m5.583s
sys    0m1.032s

configurations/Stage4_1_mm_step.dat

real    6m17.091s
user    6m15.403s
sys    0m0.784s

DATA SIZE ISSUE?

  • n_particles_per_spill 10
  • spill_generator_number_of_spills 100
  • kinetic_energy_threshold 0.1
-rw-r--r-- 1 cr67 cr67 108K 2013-04-11 12:48 maus_CoilOnly.dat.root
-rw-r--r-- 1 cr67 cr67 108K 2013-04-11 12:49 maus_Nothing.dat.root
-rw-r--r-- 1 cr67 cr67 1.3M 2013-04-11 12:49 maus_TOFOnly.dat.root
-rw-r--r-- 1 cr67 cr67 4.3M 2013-04-11 12:51 maus_TrackerOnly.dat.root

Stage4.dat

real    5m26.555s
user    5m14.636s
sys    0m1.004s

From test_simulate_mice.py:

    map_list = [
        MAUS.MapPyBeamMaker(),
        MAUS.MapCppSimulation(),
    ]

(Compared with 
    map_list = [
        MAUS.MapPyBeamMaker(),
        MAUS.MapCppSimulation(),
        MAUS.MapPyMCReconSetup(),
        MAUS.MapCppTOFMCDigitizer(),
        MAUS.MapCppTOFSlabHits(),
        MAUS.MapCppTOFSpacePoints(),
    ]

for other tests)

Slight improvement

Stage4.dat

Stage4.dat

real    6m22.627s
user    6m21.056s
sys    0m0.968s

some progress...

configurations/Stage4NoTracker.dat

real    4m6.054s
user    4m4.499s
sys    0m0.416s

This one had beamline elements except TOF1 removed (trackers were in) - looks promising...

configurations/Stage4NoBeamline.dat

real    2m9.731s
user    2m7.572s
sys    0m0.156s

This one had beamline elements and tracker removed; but TOF1 was in. Not sure this is consistent:

configurations/Stage4NoBeamlineNoTracker.dat

real    2m10.725s
user    2m4.004s
sys    0m0.188s

This one had only beamline elements

configurations/Stage4BeamlineOnly.dat

real    2m36.184s
user    2m34.642s
sys    0m0.408s

Virtual plane every 200 mm

configurations/Stage4Virtuals.dat

real    9m55.989s
user    9m54.309s
sys    0m1.016s

#3

Updated by Rogers, Chris about 10 years ago

A note on step size/stepping. So where is G4 stepping? I attach two images:

Average step size in each bin as a function of z, r (for 100 muons)

Number of steps in each bin as a function of z, r (for 100 muons)

For convenience, I also show the mean_bz (so you know where things are, roughly)

and transmission (note some particles are lost giving a bias towards low-z regions)

The run log with full geometry logging is attached as maus_Stage4_100_mm_step.dat.log

Note the following lines

Placing TOF0.dat of type Box position: (0,0,2773.19) rotationVector: (0,0,1) angle: 0 volume: 0.00800002 m^3  material: POLYSTYRENE
Placing Cherenkov.dat of type Box position: (0,0,3518.19) rotationVector: (0,0,1) angle: 0 volume: 0.44352 m^3  material: AIR
Placing DSAWestWall.dat of type Multipole position: (0,0,4476.29) rotationVector: (0,0,1) angle: 0 volume: 9.83836 m^3  material: CONCRETE
...
Solenoid Length: 198 Inner Radius: 258 Thickness: 52.8 Current Density: 1  scaleFactor: 104.612 position: (0,0,13261.7) rotationVector: (0,0,1) angle: 0 name: Field1
Solenoid Length: 197 Inner Radius: 258 Thickness: 26.4 Current Density: 1  scaleFactor: 161.3 position: (0,0,12821.2) rotationVector: (0,0,1) angle: 0 name: Field2
Solenoid Length: 110 Inner Radius: 258 Thickness: 61.6 Current Density: 1  scaleFactor: 136.8 position: (0,0,12421.2) rotationVector: (0,0,1) angle: 0 name: Field3
Solenoid Length: 1294 Inner Radius: 258 Thickness: 22 Current Density: 1  scaleFactor: 146.9 position: (0,0,11671.7) rotationVector: (0,0,1) angle: 0 name: Field4
Solenoid Length: 110 Inner Radius: 258 Thickness: 68.2 Current Density: 1  scaleFactor: 142.492 position: (0,0,10921.2) rotationVector: (0,0,1) angle: 0 name: Field5
Solenoid Length: 210 Inner Radius: 263 Thickness: 84 Current Density: 1  scaleFactor: 113.95 position: (0,0,14000.7) rotationVector: (0,0,1) angle: 0 name: Field6
Solenoid Length: 210 Inner Radius: 263 Thickness: 84 Current Density: 1  scaleFactor: -113.95 position: (0,0,14410.7) rotationVector: (0,0,1) angle: 0 name: Field7
Solenoid Length: 198 Inner Radius: 258 Thickness: 52.8 Current Density: 1  scaleFactor: 104.612 position: (-1.3336e-13,0,15042.2) rotationVector: (0,-1,0) angle: 180 name: Field8
Solenoid Length: 197 Inner Radius: 258 Thickness: 26.4 Current Density: 1  scaleFactor: 161.3 position: (-7.94157e-14,0,15482.7) rotationVector: (0,-1,0) angle: 180 name: Field9
Solenoid Length: 110 Inner Radius: 258 Thickness: 61.6 Current Density: 1  scaleFactor: 136.8 position: (-3.04315e-14,0,15882.7) rotationVector: (0,-1,0) angle: 180 name: Field10
Solenoid Length: 1294 Inner Radius: 258 Thickness: 22 Current Density: 1  scaleFactor: 146.9 position: (6.13528e-14,0,16632.2) rotationVector: (0,-1,0) angle: 180 name: Field11
Solenoid Length: 110 Inner Radius: 258 Thickness: 68.2 Current Density: 1  scaleFactor: 142.492 position: (1.53259e-13,0,17382.7) rotationVector: (0,-1,0) angle: 180 name: Field12
Multipole Order: 4 Length: 2000 Magnitude: 1e-06 BendingRadius: 0 Height: 500 Width: 500 EndField: Tanh scaleFactor: 1.293 position: (0,0,5996.29) rotationVector: (0,0,1) angle: 0 name: Field13
Multipole Order: 4 Length: 2000 Magnitude: 1e-06 BendingRadius: 0 Height: 500 Width: 500 EndField: Tanh scaleFactor: -1.293 position: (0,0,7156.29) rotationVector: (0,0,1) angle: 0 name: Field14
Multipole Order: 4 Length: 2000 Magnitude: 1e-06 BendingRadius: 0 Height: 500 Width: 500 EndField: Tanh scaleFactor: 1.293 position: (0,0,8316.29) rotationVector: (0,0,1) angle: 0 name: Field15

Step size in Beamline

Looks like there was a G4StepMax 1 mm setting in BeamLine.dat

#6

Updated by Rogers, Chris about 10 years ago

So in revision 927 I added a tool to monitor processing time; and changed the StepIV step size to 100 mm

#7

Updated by Rogers, Chris about 10 years ago

With revision 927 i.e. step size 100 mm tracker now appears to be dominant slow down:

configurations/Stage4.dat

real    7m28.072s
user    7m26.452s
sys    0m1.348s

configurations/Stage4NoBeamline.dat

real    17m27.704s
user    17m25.965s
sys    0m2.060s

configurations/Stage4NoBeamlineNoTracker.dat

real    1m58.248s
user    1m56.779s
sys    0m0.336s

configurations/Stage4NoTracker.dat

real    2m10.582s
user    2m8.716s
sys    0m0.588s

configurations/Stage4Virtuals.dat

real    7m16.706s
user    7m15.039s
sys    0m1.044s

configurations/Stage4BeamlineOnly.dat

real    1m21.157s
user    1m19.609s
sys    0m0.344s
#8

Updated by Rogers, Chris about 10 years ago

Just to confirm that I am indeed getting much more transmission when the beamline is removed - I plot the track final z positions for beamline included

and with beamline excluded

#9

Updated by Rogers, Chris about 10 years ago

  • Status changed from Open to Closed
  • % Done changed from 0 to 100

Will open as new issue - improve simulation speed

#10

Updated by Rogers, Chris about 10 years ago

  • Target version changed from Future MAUS release to MAUS-v0.5.2
#11

Updated by Rogers, Chris about 10 years ago

See summary slides

Also available in: Atom PDF