Software Sustainability Institute / MAUS collaboration

A collaboration between MAUS and The Software Sustainability Institute, working on online data quality (formerly called 'online reconstruction') from 04/08/11 - 05/04/12.

Documents

Meetings

Work plan

  • #691, Online Data Quality (SSI collaborative effort)
Product Description Who Quality measure Estimate
Software evaluation report Criteria-based assessment and tutorial-based evaluation of the MAUS software and infrastructure done using the SSI software evaluation service process. Purpose: provide information on the sustainability of their software and how easy it is for a new developer to get started with MAUS. MJ Senior User approval 10d
Matplotlib histogram reducer (#702) Reduce worker that uses matplotlib to create histograms, with associated tests MJ Senior User approval (via code review) 5d
Web-based histogram presenter (#703) Simple web server-based component to display histograms for demo purposes. Can be just a PHP script to list image files in a directory. MJ Senior User approval 3d
Distributed task queue (#704) Component to take inputs and spawn threads, or new processes, to execute chains of map workers. Based on Celery – Python distributed task queue. MJ Senior User approval (via code review) 20d
Data cache (#705) Component to cache results output by chains of map workers until they’re ready for consumption by reducers. Based on CouchDB. MJ Senior User approval (via code review) 20d
Web interface (#706) Web interface for live presentation of histograms and status information on worker execution. MJ Senior User approval (via code review) 10d
PyROOT histogram reducer (#707) Reduce worker that uses PyROOT to create histograms. MJ Senior User approval (via code review) 5d
Sustainable components Refactored versions of all components, bug fixed, polished, and made sustainable. MJ Senior User approval (via code review) Time remaining

All the components can be developed stand-alone and none blocks the others, with the exception of the final task.

Tickets

Calendar

Proposed deliverable dates for components, based on MJ’s current availabilities, and above estimates, are:

WB Products Delivery Date
19/09 Software evaluation report and Matplotlib histogram reducer 23/
26/09 Matplotlib histogram reducer 30/09
03/10 Web-based histogram presenter 07/10
10/10 Web-based histogram presenter / ill
17/10 Matplotlib histogram reducer finish. Start distributed task queue
24/10 Unavailable
31/10 Unavailable
07/11 Distributed task queue
14/11 Unavailable
21/11 Distributed task queue
28/11 Data cache
05/12 PyROOT histogram reducer
12/12 Meeting
19/12 Distributed task queue monitoring and error handling 23/12
26/12 Unavailable
02/01 Unavailable
09/01 Task queue
16/01 Task queue
23/01 Task queue/DB monitoring/error handling
30/01 Task queue/DB monitoring/error handling
06/02 Web interface
13/02 Conference, RAL visit
20/02 Framework
27/02 Framework
05/03 Framework
12/03 Framework / Web interface / Documentation
19/03 SSI collaborations workshop
26/03 Web interface
02/04 Wrap-up, documentation

Risks

ID Description (including control and contingency plans) Owner Likelihood Impact Exposure (L * I)
R1 SSI personnel need extra time to learn unfamiliar software tools or environment. Control: accept. Contingency: will require additional support from existing MAUS team members. MJJ M M L
R2 Project is more complex than first estimated. Control: accept. Ensure components are as modular as far as possible; address functionality incrementally. Contingency: apply triage to later, less critical functions. MJJ M H H
R3 Dependency on technology still under development. Control: accept. Insulate development of the project’s components as far as possible through agreement on interfaces and data formats. Contingency: Pause development and review impact of changes across system rather than make piecemeal adjustments. MJJ H M M
R4 Changes in MICE requirements affect MAUS. Control: accept. Combine modular design and an agile approach to requirements. Contingency: Pause development and review impact of changes across system rather than make piecemeal adjustments. MJJ M M L
R5 SSI analyst/developer becomes unavailable through illness. Control: accept. Ensure that current progress and current outputs (software, documents) are regularly committed to the repository and current status of work is regularly updated on the wiki. Contingency: SSI development manager to identify another SSI Analyst/Developer. MJJ L M L
R6 SSI analyst/developer has increased commitments from other projects. Control: reduce. Keep aware of project commitments and ensure that any increased commitments from other projects are temporary. Raise concerns with EPCC management. Contingency: Negotiate with other projects’ leaders for time. Worst-case – SSI development manager to identify another SSI Analyst/Developer. MJJ L M L

SSI-MAUS-pid.docx (143.4 kB) Jackson, Mike, 06 December 2011 09:04

SSI-MAUS-endproject.pdf (144.2 kB) Jackson, Mike, 19 April 2012 16:28