Project

General

Profile

MAUSOnlineCHEP2012.txt

Heidt, Christopher, 19 June 2012 02:20

Download (11 KB)

 
1
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
2
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
3
<head>
4
<meta http-equiv="content-type" content="text/html; charset=utf-8" />
5
<title>MICE - MAUSOnlineCHEP2012.txt - MICEmine</title>
6
<meta name="description" content="Redmine" />
7
<meta name="keywords" content="issue,bug,tracker" />
8
<meta name="csrf-param" content="authenticity_token"/>
9
<meta name="csrf-token" content="LI8yV6YwnWDPcrGl+zWaXZl1/QDXQAkrC2bzK5HDj5U="/>
10
<link rel='shortcut icon' href='/favicon.ico?1308245515' />
11
<link href="/stylesheets/application.css?1308245516" media="all" rel="stylesheet" type="text/css" />
12

    
13
<script src="/javascripts/prototype.js?1308245516" type="text/javascript"></script>
14
<script src="/javascripts/effects.js?1308245516" type="text/javascript"></script>
15
<script src="/javascripts/dragdrop.js?1308245516" type="text/javascript"></script>
16
<script src="/javascripts/controls.js?1308245516" type="text/javascript"></script>
17
<script src="/javascripts/application.js?1308245516" type="text/javascript"></script>
18
<script type="text/javascript">
19
//<![CDATA[
20
Event.observe(window, 'load', function(){ new WarnLeavingUnsaved('The current page contains unsaved text that will be lost if you leave this page.'); });
21
//]]>
22
</script>
23

    
24
<!--[if IE 6]>
25
    <style type="text/css">
26
      * html body{ width: expression( document.documentElement.clientWidth < 900 ? '900px' : '100%' ); }
27
      body {behavior: url(/stylesheets/csshover.htc?1308245516);}
28
    </style>
29
<![endif]-->
30

    
31
<!-- page specific tags -->
32
    <link href="/stylesheets/scm.css?1308245516" media="screen" rel="stylesheet" type="text/css" /></head>
33
<body class="controller-attachments action-show">
34
<div id="wrapper">
35
<div id="wrapper2">
36
<div id="top-menu">
37
    <div id="account">
38
        <ul><li><a href="/my/account" class="my-account">My account</a></li>
39
<li><a href="/logout" class="logout">Sign out</a></li></ul>    </div>
40
    <div id="loggedas">Logged in as <a href="/users/53">Orson1981</a></div>
41
    <ul><li><a href="/" class="home">Home</a></li>
42
<li><a href="/my/page" class="my-page">My page</a></li>
43
<li><a href="/projects" class="projects">Projects</a></li>
44
<li><a href="/tab/system_show" class="tab">Calendar</a></li>
45
<li><a href="http://www.redmine.org/guide" class="help">Help</a></li></ul></div>
46
      
47
<div id="header">
48
    
49
    <div id="quick-search">
50
        <form action="/search/index/mice" method="get">
51
        
52
        <a href="/search/index/mice" accesskey="4">Search</a>:
53
        <input accesskey="f" class="small" id="q" name="q" size="20" type="text" />
54
        </form>
55
        <select onchange="if (this.value != '') { window.location = this.value; }"><option value=''>Jump to a project...</option><option value="" disabled="disabled">---</option><option selected="selected" value="/projects/mice?jump=attachments">MICE</option><option value="/projects/analysis?jump=attachments">&nbsp;&nbsp;&#187; Analysis</option><option value="/projects/computing-software?jump=attachments">&nbsp;&nbsp;&#187; Computing and Software</option><option value="/projects/configdb?jump=attachments">&nbsp;&nbsp;&nbsp;&nbsp;&#187; Configuration Database</option><option value="/projects/control-monitoring?jump=attachments">&nbsp;&nbsp;&nbsp;&nbsp;&#187; Control and Monitoring</option><option value="/projects/maus?jump=attachments">&nbsp;&nbsp;&nbsp;&nbsp;&#187; MAUS</option><option value="/projects/online?jump=attachments">&nbsp;&nbsp;&nbsp;&nbsp;&#187; Online</option><option value="/projects/mice-eb?jump=attachments">&nbsp;&nbsp;&#187; Executive Board</option><option value="/projects/operations?jump=attachments">&nbsp;&nbsp;&#187; Operations</option><option value="/projects/tracker?jump=attachments">&nbsp;&nbsp;&#187; SciFi Tracker</option><option value="/projects/x-boa?jump=attachments">&nbsp;&nbsp;&#187; x-boa</option></select>
56
    </div>
57
    
58
    
59
    <h1>MICE</h1>
60
    
61
    
62
    <div id="main-menu">
63
        <ul><li><a href="/projects/mice" class="overview">Overview</a></li>
64
<li><a href="/projects/mice/activity" class="activity">Activity</a></li>
65
<li><a href="/projects/mice/news" class="news">News</a></li>
66
<li><a href="/projects/mice/documents" class="documents">Documents</a></li>
67
<li><a href="/projects/mice/wiki" class="wiki">Wiki</a></li>
68
<li><a href="/projects/mice/settings" class="settings">Settings</a></li></ul>
69
    </div>
70
    
71
</div>
72

    
73
<div class="nosidebar" id="main">
74
    <div id="sidebar">        
75
        
76
        
77
    </div>
78
    
79
    <div id="content">
80
				
81
        <h2>MAUSOnlineCHEP2012.txt</h2>
82

    
83
<div class="attachments">
84
<p>
85
   <span class="author">Jackson, Mike, 17 April 2012 09:08</span></p>
86
<p><a href="/attachments/download/900/MAUSOnlineCHEP2012.txt">Download</a>   <span class="size">(1.7 kB)</span></p>
87

    
88
</div>
89
&nbsp;
90
<div class="autoscroll">
91
<table class="filecontent syntaxhl">
92
<tbody>
93

    
94

    
95
<tr><th class="line-num" id="L1"><a href="#L1">1</a></th><td class="line-code"><pre>MAUS Online Data Quality - Chris Tunnell and Mike Jackson
96
</pre></td></tr>
97

    
98

    
99
<tr><th class="line-num" id="L2"><a href="#L2">2</a></th><td class="line-code"><pre>
100
</pre></td></tr>
101

    
102

    
103
<tr><th class="line-num" id="L3"><a href="#L3">3</a></th><td class="line-code"><pre>Within the Muon Ionization Cooling Experiment (MICE), the MICE
104
</pre></td></tr>
105

    
106

    
107
<tr><th class="line-num" id="L4"><a href="#L4">4</a></th><td class="line-code"><pre>Analysis User Software (MAUS) framework performs both online analysis
108
</pre></td></tr>
109

    
110

    
111
<tr><th class="line-num" id="L5"><a href="#L5">5</a></th><td class="line-code"><pre>of live data and detailed offline data analysis, simulation, and
112
</pre></td></tr>
113

    
114

    
115
<tr><th class="line-num" id="L6"><a href="#L6">6</a></th><td class="line-code"><pre>accelerator design.
116
</pre></td></tr>
117

    
118

    
119
<tr><th class="line-num" id="L7"><a href="#L7">7</a></th><td class="line-code"><pre>
120
</pre></td></tr>
121

    
122

    
123
<tr><th class="line-num" id="L8"><a href="#L8">8</a></th><td class="line-code"><pre>The MAUS Map-Reduce API parallelizes computing in the control room,
124
</pre></td></tr>
125

    
126

    
127
<tr><th class="line-num" id="L9"><a href="#L9">9</a></th><td class="line-code"><pre>ensures that code can be run both offline and online, and displays
128
</pre></td></tr>
129

    
130

    
131
<tr><th class="line-num" id="L10"><a href="#L10">10</a></th><td class="line-code"><pre>plots for users in an easily extendable manner. 
132
</pre></td></tr>
133

    
134

    
135
<tr><th class="line-num" id="L11"><a href="#L11">11</a></th><td class="line-code"><pre>
136
</pre></td></tr>
137

    
138

    
139
<tr><th class="line-num" id="L12"><a href="#L12">12</a></th><td class="line-code"><pre>The original Map-Reduce design can be advantageous for offline
140
</pre></td></tr>
141

    
142

    
143
<tr><th class="line-num" id="L13"><a href="#L13">13</a></th><td class="line-code"><pre>computing but cannot be used in online settings. It expects all map
144
</pre></td></tr>
145

    
146

    
147
<tr><th class="line-num" id="L14"><a href="#L14">14</a></th><td class="line-code"><pre>operations to terminate before running the reduction; however, the
148
</pre></td></tr>
149

    
150

    
151
<tr><th class="line-num" id="L15"><a href="#L15">15</a></th><td class="line-code"><pre>data flow for online analysis requires the continuous updating of live
152
</pre></td></tr>
153

    
154

    
155
<tr><th class="line-num" id="L16"><a href="#L16">16</a></th><td class="line-code"><pre>plots as data arrives. 
156
</pre></td></tr>
157

    
158

    
159
<tr><th class="line-num" id="L17"><a href="#L17">17</a></th><td class="line-code"><pre>
160
</pre></td></tr>
161

    
162

    
163
<tr><th class="line-num" id="L18"><a href="#L18">18</a></th><td class="line-code"><pre>For online running, the &quot;map&quot; and &quot;reduce&quot; steps must happen
164
</pre></td></tr>
165

    
166

    
167
<tr><th class="line-num" id="L19"><a href="#L19">19</a></th><td class="line-code"><pre>concurrently; therefore, new parallelization routines were developed 
168
</pre></td></tr>
169

    
170

    
171
<tr><th class="line-num" id="L20"><a href="#L20">20</a></th><td class="line-code"><pre>specifically for this use.   
172
</pre></td></tr>
173

    
174

    
175
<tr><th class="line-num" id="L21"><a href="#L21">21</a></th><td class="line-code"><pre>
176
</pre></td></tr>
177

    
178

    
179
<tr><th class="line-num" id="L22"><a href="#L22">22</a></th><td class="line-code"><pre>The &quot;map&quot; step is parallelized using a Python-based distributed task
180
</pre></td></tr>
181

    
182

    
183
<tr><th class="line-num" id="L23"><a href="#L23">23</a></th><td class="line-code"><pre>queue called Celery, and output from these tasks is then written into
184
</pre></td></tr>
185

    
186

    
187
<tr><th class="line-num" id="L24"><a href="#L24">24</a></th><td class="line-code"><pre>a NoSQL database called CouchDB. 
188
</pre></td></tr>
189

    
190

    
191
<tr><th class="line-num" id="L25"><a href="#L25">25</a></th><td class="line-code"><pre>
192
</pre></td></tr>
193

    
194

    
195
<tr><th class="line-num" id="L26"><a href="#L26">26</a></th><td class="line-code"><pre>As the &quot;mapper&quot; writes output, the plotting &quot;reducers&quot; query the
196
</pre></td></tr>
197

    
198

    
199
<tr><th class="line-num" id="L27"><a href="#L27">27</a></th><td class="line-code"><pre>database, request data from a user-specified window in time, and make
200
</pre></td></tr>
201

    
202

    
203
<tr><th class="line-num" id="L28"><a href="#L28">28</a></th><td class="line-code"><pre>plots using Matplotlib or PyRoot. 
204
</pre></td></tr>
205

    
206

    
207
<tr><th class="line-num" id="L29"><a href="#L29">29</a></th><td class="line-code"><pre>
208
</pre></td></tr>
209

    
210

    
211
<tr><th class="line-num" id="L30"><a href="#L30">30</a></th><td class="line-code"><pre>The &quot;reducers&quot; serialize the plots into the data stream after which
212
</pre></td></tr>
213

    
214

    
215
<tr><th class="line-num" id="L31"><a href="#L31">31</a></th><td class="line-code"><pre>all the data is written to the database by the output routines. 
216
</pre></td></tr>
217

    
218

    
219
<tr><th class="line-num" id="L32"><a href="#L32">32</a></th><td class="line-code"><pre>
220
</pre></td></tr>
221

    
222

    
223
<tr><th class="line-num" id="L33"><a href="#L33">33</a></th><td class="line-code"><pre>Finally, plots are displayed on the web using the Django platform,
224
</pre></td></tr>
225

    
226

    
227
<tr><th class="line-num" id="L34"><a href="#L34">34</a></th><td class="line-code"><pre>which queries the database and displays the plots to the control room
228
</pre></td></tr>
229

    
230

    
231
<tr><th class="line-num" id="L35"><a href="#L35">35</a></th><td class="line-code"><pre>and the world. 
232
</pre></td></tr>
233

    
234

    
235
<tr><th class="line-num" id="L36"><a href="#L36">36</a></th><td class="line-code"><pre>
236
</pre></td></tr>
237

    
238

    
239
<tr><th class="line-num" id="L37"><a href="#L37">37</a></th><td class="line-code"><pre>By maintaining the API and modifying the data flow, MICE is able to
240
</pre></td></tr>
241

    
242

    
243
<tr><th class="line-num" id="L38"><a href="#L38">38</a></th><td class="line-code"><pre>use identical analysis software in both offline and online scenarios,
244
</pre></td></tr>
245

    
246

    
247
<tr><th class="line-num" id="L39"><a href="#L39">39</a></th><td class="line-code"><pre>thus avoiding a common issue in experimental particle physics. 
248
</pre></td></tr>
249

    
250

    
251
</tbody>
252
</table>
253
</div>
254

    
255

    
256

    
257

    
258

    
259
        
260
				<div style="clear:both;"></div>
261
    </div>
262
</div>
263

    
264
<div id="ajax-indicator" style="display:none;"><span>Loading...</span></div>
265
	
266
<div id="footer">
267
  <div class="bgl"><div class="bgr">
268
    Powered by <a href="http://www.redmine.org/">Redmine</a> &copy; 2006-2011 Jean-Philippe Lang
269
  </div></div>
270
</div>
271
</div>
272
</div>
273

    
274
</body>
275
</html>