Project

General

Profile

Feature #707

PyROOT histogram reducer

Added by Jackson, Mike about 10 years ago. Updated over 9 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Jackson, Mike
Category:
Online reconstruction
Target version:
Start date:
20 September 2011
Due date:
% Done:

100%

Estimated time:
Workflow:
New Issue

Description

Reduce worker that uses PyROOT to create histograms.

#1

Updated by Jackson, Mike almost 10 years ago

From MAUS SSI Component Design - Online Reconstruction:

  • Histograms allow detector performance to be monitored.
  • Converts JSON document into JPG or EPS (or other image file).
  • May be 100s of plots per output.
  • Should be easy to configure/extend by non-Python programmer.
  • Needs to allow aggregation of data to incrementally update the plot (so include a test for this!)
#2

Updated by Jackson, Mike almost 10 years ago

  • Workflow set to New Issue

See #707 in which Durga started work on this. Code is at
http://bazaar.launchpad.net/~maus-mlcr/maus/control-room/files/head:/src/reduce/ReducePyTOFPlot/. This saves files itself as there were problems in getting OutputPyImage to work with it.

#3

Updated by Jackson, Mike almost 10 years ago

Commit 697 has version of Durga's "ReducePyTOFPlot.py"http://bazaar.launchpad.net/~michaelj-h/maus/devel/view/697/src/reduce/ReducePyTOFPlot/ReducePyTOFPlot.py which works with OutputPyImage. This still needs work to pull out commonality into a super-class.

A general issue, with implications on the framework (Go.py) is that this reducer outputs histograms every N spills e.g. 5, and the number of the final spill it receives may not be a factor of N e.g. 23. So, handling of the final histograms (after processing spills 21, 22, 23) is done by death(). This breaks the contract of the framework (in my view) since death() is assumed to be used for clean up. At present the framework doesn't expect any documents back from death() so these would never be passed on to output tasks (e.g. OutputPyImage).

The simplest solution is to just ignore any lingering spills in such mappers/reducers.

A non-spill-ignoring solution would be to change the framework to push a final "end-processing" document when there is no more spills left from input tasks. Any mapper or reducer that receives this "end-processing" document can just do any final processing (e.g. for many this will be nothing, but for ReducePyTOFPlot this would be to output the final histograms) then just output this "end" spill in turn. The "end-processing" document then can be viewed as flushing any remaining outputs from mappers/reducers, prior to the invocation of death() on these, by the framework, to clean-up.

#4

Updated by Jackson, Mike almost 10 years ago

Updated to pull out PyROOT commonality to ReducePyROOTHistogram base class - commit 699

#5

Updated by Jackson, Mike almost 10 years ago

Telcon to discuss: PC_091211

#6

Updated by Jackson, Mike almost 10 years ago

Commit: "701": http://bazaar.launchpad.net/~michaelj-h/maus/devel/revision/701

The directories with the new version of ReducePyTOFCount and its ReducePyROOTHistogram super-class are in:

TODO

  • Pull out support for refresh_rate configuration and handling to ReducePyROOTHistogram super-class.
  • Add similar support to ReduceMatplotlibHistogram.
  • Add support for end-of-spills block to be passed from Go.py into reducers.
  • Update reducers to handle this end-of-spills block - especially ReducePyTOFCounts.py to output its final histograms.
  • See if a super-super class common to both ReducePyROOTHistogram and ReduceMatplotlibHistogram can be derived.
#7

Updated by Jackson, Mike almost 10 years ago

Updates to my branch relating to this ticket:

  • 704 Merged in lp:~durga/maus/dev2 revision 618 ReducePyTOFPlot changes.
  • 706 ReducePyMatplotlibHistogram and ReducePyHistogramTDCADCCounts now use ErrorHandler.
  • 707 Updated _update_histogram comments in above reducers.
  • 708 ReducePyROOTHistogram and ReducePyTOFPlots now use ErrorHandler.
  • 709 Go.py now sends {"END_OF_RUN":"END_OF_RUN"} document into reducers when no more inputs. ReducePyHistogramTDCADCCounts updated to just output an empty document in response. ReducePyTOFPlot updated to output final histogram documents in response.
#8

Updated by Jackson, Mike over 9 years ago

  • Status changed from Open to Closed
  • % Done changed from 0 to 100

Added usage info to MAUSHistogramReducers

#9

Updated by Rogers, Chris over 9 years ago

  • Target version changed from Future MAUS release to MAUS-v0.1.3

Also available in: Atom PDF