Need to figure out a way to manage configuration - the issue is that initialisation of configuration is relatively slow, but at present needs to be done once per spill. Suggestions:
- Optimise the configuration start/stop
- Singleton (perhaps lazy initialisation)
Configuration here is a somewhat hazy term, but includes at least geometry and fields, may include "data cards" and geant4 geometry bindings also.
Updated by Tunnell, Christopher over 11 years ago
How big is the data because that changes how it's done. If we just added the fields and geometry to the datacards then had a function that you could pass the datacards JSON and it returned an interpolated field, for example, would that work? I think if it's not bigger than a few MB then JSON can work (where a few MB is a number coming from CouchDB and MongoDB people when they say after that storing binary data because can become slow... at very least not human readable).
Updated by Tunnell, Christopher about 11 years ago
How big are the field maps? And do we store them as binary with some helper function in C++ with a swig interface for python?
4 bytes is a float. First question: if we run into space issues, we can ask the question of how many mantissa and exponent bits we need in the floating point representation.
Second question: can we just use a 2D map with a slowly varying error field? My naive studies of magnet measurements and wall stuff (the wall stuff repeated stuff I think you did) suggests this may be the case.
If we have to do 3D, then this:
4 bytes * 3 (15 m / cm) pi (1m / cm)^2
which samples every cm is equal to 1 GB, which means if we are smart we can do it on a single machine. But not parallelize to many cores.