MAUSOnlineReconstructionOverview » History » Revision 2

Revision 1 (Jackson, Mike, 06 March 2012 10:34) → Revision 2/14 (Jackson, Mike, 13 March 2012 11:28)

h1. Online Reconstruction Overview 

 h2. Run numbers 

 Run numbers are assumed to be as follows: 

 * -N : Monte Carlo simulation of run N 
 * 0 : pure Monte Carlo simulation 
 * +N : run N 

 h2. Transforming spills from an input stream (Input-Transform) 

 This is the algorithm used to transform spills from an input stream: 
 <pre> 
 CLEAR document store 
 run_number = None 
 WHILE an input spill is available 
   GET next spill 
   IF spill does not have a run number 
     # Assume pure MC 
     spill_run_number = 0 
   IF (spill_run_number != run_number) 
     # We've changed run. 
     IF spill is NOT a start_of_run spill 
       WARN user of missing start_of_run spill 
     WAIT for current Celery tasks to complete 
       WRITE result spills to document store 
     run_number = spill_run_number 
     CONFIGURE Celery by DEATHing current transforms and BIRTHing new transforms 
   TRANSFORM spill using Celery 
   WRITE result spill to document store 
  DEATH Celery worker transforms 
 </pre> 
 If there is no initial @start_of_run@ spill (or no @spill_num@ in the spills) in the input stream (as can occur when using @simple_histogram_example.py@ or @simulate_mice.py@) then spill_run_number will be 0, run_number will be None and a Celery configuration will be done before the first spill needs to be transformed.  

 Spills are inserted into the document store in the order of their return from Celery workers. This may not be in synch with the order in which they were originally read from the input stream. 

 h2. Merging spills and passing results to an output stream (Merge-Output) 

 This is the algorithm used to merge spills and pass the results to an output stream: 
 <pre> 
 run_number = None 
 end_of_run = None 
 is_birthed = FALSE 
 last_time = 01/01/1970 
 WHILE TRUE 
   READ spills added since last time from document store 
   IF spill IS "end_of_run" 
     end_of_run = spill 
   IF spill_run_number != run_number 
     IF is_birthed 
       IF end_of_run == None 
           end_of_run = {"daq_event_type":"end_of_run", "run_num":run_number} 
       Send end_of_run END_OF_RUN block to merger 
       DEATH merger and outputter 
     BIRTH merger and outputter 
     run_number = spill_run_number 
     end_of_run = None 
     is_birthed = TRUE 
   MERGE and OUTPUT spill 
 Send END_OF_RUN block to merger 
 DEATH merger and outputter 
 </pre> 

 The Input-Transform policy of waiting for the processing of spills from a run to complete before starting processing spills from a new run means that all spills from run N-1 are guaranteed to have a time stamp < spills from run N. 

 is_birthed is used to ensure that there is no BIRTH-DEATH-BIRTH redundancy on receipt of the first spill. 

 h2. Document store 

 Spills are stored in documents in a collection in the document store.  

 Documents are of form @{"_id":ID, "date":DATE, "doc":SPILL}@ where: 

 * ID: index of this document in the chain of those successfuly transformed. It has no significance beyond being unique in an execution of Input-Transform loop below. It is not equal to the spill_num (Python @string@) 
 * DATE: date and time to the milli-second noting when the document was added (Python @timestamp@) 
 * DOC: spill document. (Python @string@ holding a valid JSON document) 

 h3. Collection names 

 For Input-Transform, 

 * If configuration parameter @doc_collection_name@ is @None@, @""@, or @auto@ then @HOSTNAME_PID@, where @HOSTNAME@ is the machine name and @PID@ the process ID, is used. 
 * Otherwise the value of @doc_collection_name@ is used. 
 * @doc_collection_name@ has default value @spills@. 

 For Merge-Output, 

 * If configuration parameter @doc_collection_name@ is @None@, @""@, or undefined then an error is raised. 
 * Otherwise the value of @doc_collection_name@ is used. 

 h2. Miscellaneous 

 * Currently Celery timeouts are not used, transforming a spill takes as long as it takes. 
 * Celery task retries on failure option is not used. If the transformation of a spill fails first time it can't be expected to succeed on a retry. 
 * If memory leaks arise, e.g. from C++ code, look at Celery rate limitss, which allow the time or number of tasks before sub-process is killed and respawned, to be defined. Soft rate limits would allow @death@ to be run on the transforms first.