MAUSOnlineCHEP2012.txt

Jackson, Mike, 17 April 2012 09:08

Download (1.7 kB)

 
1
MAUS Online Data Quality - Chris Tunnell and Mike Jackson
2
3
Within the Muon Ionization Cooling Experiment (MICE), the MICE
4
Analysis User Software (MAUS) framework performs both online analysis
5
of live data and detailed offline data analysis, simulation, and
6
accelerator design.
7
8
The MAUS Map-Reduce API parallelizes computing in the control room,
9
ensures that code can be run both offline and online, and displays
10
plots for users in an easily extendable manner. 
11
12
The original Map-Reduce design can be advantageous for offline
13
computing but cannot be used in online settings. It expects all map
14
operations to terminate before running the reduction; however, the
15
data flow for online analysis requires the continuous updating of live
16
plots as data arrives. 
17
18
For online running, the "map" and "reduce" steps must happen
19
concurrently; therefore, new parallelization routines were developed 
20
specifically for this use.   
21
22
The "map" step is parallelized using a Python-based distributed task
23
queue called Celery, and output from these tasks is then written into
24
a NoSQL database called CouchDB. 
25
26
As the "mapper" writes output, the plotting "reducers" query the
27
database, request data from a user-specified window in time, and make
28
plots using Matplotlib or PyRoot. 
29
30
The "reducers" serialize the plots into the data stream after which
31
all the data is written to the database by the output routines. 
32
33
Finally, plots are displayed on the web using the Django platform,
34
which queries the database and displays the plots to the control room
35
and the world. 
36
37
By maintaining the API and modifying the data flow, MICE is able to
38
use identical analysis software in both offline and online scenarios,
39
thus avoiding a common issue in experimental particle physics.