1 | MAUS Online Data Quality - Chris Tunnell and Mike Jackson
|
2 |
|
3 | Within the Muon Ionization Cooling Experiment (MICE), the MICE
|
4 | Analysis User Software (MAUS) framework performs both online analysis
|
5 | of live data and detailed offline data analysis, simulation, and
|
6 | accelerator design.
|
7 |
|
8 | The MAUS Map-Reduce API parallelizes computing in the control room,
|
9 | ensures that code can be run both offline and online, and displays
|
10 | plots for users in an easily extendable manner.
|
11 |
|
12 | The original Map-Reduce design can be advantageous for offline
|
13 | computing but cannot be used in online settings. It expects all map
|
14 | operations to terminate before running the reduction; however, the
|
15 | data flow for online analysis requires the continuous updating of live
|
16 | plots as data arrives.
|
17 |
|
18 | For online running, the "map" and "reduce" steps must happen
|
19 | concurrently; therefore, new parallelization routines were developed
|
20 | specifically for this use.
|
21 |
|
22 | The "map" step is parallelized using a Python-based distributed task
|
23 | queue called Celery, and output from these tasks is then written into
|
24 | a NoSQL database called CouchDB.
|
25 |
|
26 | As the "mapper" writes output, the plotting "reducers" query the
|
27 | database, request data from a user-specified window in time, and make
|
28 | plots using Matplotlib or PyRoot.
|
29 |
|
30 | The "reducers" serialize the plots into the data stream after which
|
31 | all the data is written to the database by the output routines.
|
32 |
|
33 | Finally, plots are displayed on the web using the Django platform,
|
34 | which queries the database and displays the plots to the control room
|
35 | and the world.
|
36 |
|
37 | By maintaining the API and modifying the data flow, MICE is able to
|
38 | use identical analysis software in both offline and online scenarios,
|
39 | thus avoiding a common issue in experimental particle physics.
|