Software Sustainability Institute / MAUS collaboration¶

A collaboration between MAUS and The Software Sustainability Institute, working on online data quality (formerly called 'online reconstruction') from 04/08/11 - 05/04/12.

Documents¶

• Project initiation document (DOC) (06/12/11)
• Software sustainability evaluation (05/09/11)
• Online reconstruction - original design (14/09/11)
• Architecture and components - CM32 status report PPT (09/02/12)
• End of project report (PDF) (19/06/12)

Meetings¶

• 13/02/12 FF_130212
• 09/02/12 CM32_Agenda
• 25-27/11/11 F2F at RAL
• 14/09/11 F2F in Oxford
• 26/08/11 Telcon
• 08/08/11 Telcon
• 04/08/11 Pre-telcon notes

Work plan¶

• #691, Online Data Quality (SSI collaborative effort)

Product	Description	Who	Quality measure	Estimate
Software evaluation report	Criteria-based assessment and tutorial-based evaluation of the MAUS software and infrastructure done using the SSI software evaluation service process. Purpose: provide information on the sustainability of their software and how easy it is for a new developer to get started with MAUS.	MJ	Senior User approval	10d
Matplotlib histogram reducer (#702)	Reduce worker that uses matplotlib to create histograms, with associated tests	MJ	Senior User approval (via code review)	5d
Web-based histogram presenter (#703)	Simple web server-based component to display histograms for demo purposes. Can be just a PHP script to list image files in a directory.	MJ	Senior User approval	3d
Distributed task queue (#704)	Component to take inputs and spawn threads, or new processes, to execute chains of map workers. Based on Celery – Python distributed task queue.	MJ	Senior User approval (via code review)	20d
Data cache (#705)	Component to cache results output by chains of map workers until they’re ready for consumption by reducers. Based on CouchDB.	MJ	Senior User approval (via code review)	20d
Web interface (#706)	Web interface for live presentation of histograms and status information on worker execution.	MJ	Senior User approval (via code review)	10d
PyROOT histogram reducer (#707)	Reduce worker that uses PyROOT to create histograms.	MJ	Senior User approval (via code review)	5d
Sustainable components	Refactored versions of all components, bug fixed, polished, and made sustainable.	MJ	Senior User approval (via code review)	Time remaining

All the components can be developed stand-alone and none blocks the others, with the exception of the final task.

Tickets¶

• Open
• Not Open
• Watched Open
• Watched Not Open

Calendar¶

Proposed deliverable dates for components, based on MJ’s current availabilities, and above estimates, are:

WB	Products	Delivery Date
19/09	Software evaluation report and Matplotlib histogram reducer	23/
26/09	Matplotlib histogram reducer	30/09
03/10	Web-based histogram presenter	07/10
10/10	Web-based histogram presenter / ill
17/10	Matplotlib histogram reducer finish. Start distributed task queue
24/10	Unavailable
31/10	Unavailable
07/11	Distributed task queue
14/11	Unavailable
21/11	Distributed task queue
28/11	Data cache
05/12	PyROOT histogram reducer
12/12	Meeting
19/12	Distributed task queue monitoring and error handling	23/12
26/12	Unavailable
02/01	Unavailable
09/01	Task queue
16/01	Task queue
23/01	Task queue/DB monitoring/error handling
30/01	Task queue/DB monitoring/error handling
06/02	Web interface
13/02	Conference, RAL visit
20/02	Framework
27/02	Framework
05/03	Framework
12/03	Framework / Web interface / Documentation
19/03	SSI collaborations workshop
26/03	Web interface
02/04	Wrap-up, documentation

Risks¶

ID	Description (including control and contingency plans)	Owner	Likelihood	Impact	Exposure (L * I)
R1	SSI personnel need extra time to learn unfamiliar software tools or environment. Control: accept. Contingency: will require additional support from existing MAUS team members.	MJJ	M	M	L
R2	Project is more complex than first estimated. Control: accept. Ensure components are as modular as far as possible; address functionality incrementally. Contingency: apply triage to later, less critical functions.	MJJ	M	H	H
R3	Dependency on technology still under development. Control: accept. Insulate development of the project’s components as far as possible through agreement on interfaces and data formats. Contingency: Pause development and review impact of changes across system rather than make piecemeal adjustments.	MJJ	H	M	M
R4	Changes in MICE requirements affect MAUS. Control: accept. Combine modular design and an agile approach to requirements. Contingency: Pause development and review impact of changes across system rather than make piecemeal adjustments.	MJJ	M	M	L
R5	SSI analyst/developer becomes unavailable through illness. Control: accept. Ensure that current progress and current outputs (software, documents) are regularly committed to the repository and current status of work is regularly updated on the wiki. Contingency: SSI development manager to identify another SSI Analyst/Developer.	MJJ	L	M	L
R6	SSI analyst/developer has increased commitments from other projects. Control: reduce. Keep aware of project commitments and ensure that any increased commitments from other projects are temporary. Raise concerns with EPCC management. Contingency: Negotiate with other projects’ leaders for time. Worst-case – SSI development manager to identify another SSI Analyst/Developer.	MJJ	L	M	L