Extending the Data Structure

The data structure can be extended in MAUS by adding extra classes to the existing data structure. The data classes are in src/common_cpp/DataStructure. In order to make these classes accessible to ROOT, the following steps must be taken:

• Add a new class in src/common_cpp/DataStructure.
• Ensure that default constructor, copy constructor, equality operator and destructor is present. The destructor must be virtual.
• Add #include "src/common_cpp/Utils/VersionNumber.hh" and a call to the MAUS_VERSIONED_CLASS_DEF() macro at the end of the class definition before the closing braces. MAUS_VERSIONED_CLASS_DEF calls the ROOT ClassDef() macro which generates metaclasses based on information in the class. This is put into the (dynamically generated) MausDataStructure.h,cc files.
• Add the class to the list of classes in src/common_cpp/DataStructure/LinkDef.hh. This is required for the class to be linked properly to the main library, and a linker error will result if this step is not taken.
• Add any template definitions which you used, including STL classes (e.g. std::vector<MyClass> or whatever) to linkdef. Otherwise ROOT will generate a segmentation fault whenever the user tries to call functions of the templated class (but the code will link successfully in this case).

In order to make these classes accessible to JSON, it is necessary to add a new processor in
src/common_cpp/JsonCppProcessors. There are a few default processors available.

• src/common_cpp/JsonCppProcessors/ProcessorBase.hh contains IProcessor pure interface class for all processors and ProcessorBase base class (which may contain some implementation)
• src/common_cpp/JsonCppProcessors/PrimitivesProcessors.hh contains processors for primitive types; BoolProcessor, IntProcessor, UIntProcessor, StringProcessor, DoubleProcessor
• src/common_cpp/JsonCppProcessors/ArrayProcessors.hh contains processors for array types. Two processors are available: PointerArrayProcessor which converts an STL vector of pointers to data; and ValueArrayProcessor which converts an STL vector of values to data.
• src/common_cpp/JsonCppProcessors/ObjectProcessor.hh contains a processor for object types. Most of the classes in the MAUS data structure are represented in JSON as objects (string value pairs) where each string names a branch and each value contains data, which may be another class.
• src/common_cpp/JsonCppProcessors/ObjectMapProcessors.hh contains a processor for converting from JSON objects to STL maps. This is useful for JSON objects that contain lots of branches all of the same type.

A script, bin/user/json_branch_to_data_structure_and_cpp_processor.py is available that analyses a JSON object or JSON tree of nested objects and converts to C++ classes. The script is provided "as-is" and it is expected that developers will check the output, adding comments and tests where appropriate.

Pointer Handling

MAUS can handle pointers for arrays and classes using ROOT native support (via the TRef and TRefArray classes) or the standard JSON reference syntax. JSON references are indexed by a path relative to the root value of a JSON document. JSON references are formatted like URIs, for example the JSON object {"$ref":"#spill/recon_events/1"} would index the second recon_event in the spill object (indexing from 0). MAUS can only handle paths relative to the top level of the JSON document for the same MAUS event. Absolute URIs, URIs relative to another position in the JSON document or URIs to another MAUS event are not supported.

In MAUS, it is necessary to make a distinction between data that is stored as a value in C++ and JSON (value-as-data), data that is stored as a pointer in C++ and a value in JSON (pointer-as-data) and data that is stored as a pointer in C++ and JSON to some other data in the same tree (pointer-as-reference). In the latter case, the C++ parent object does not own the memory; rather it is owned by some other object in the same tree and borrowed by the C++ object holding the pointer-as-reference. The TRef and TRefArray classes provide this functionality by default; never owning the memory but only storing a relevant pointer. All objects referenced by a TRef or TRefArray must inherit from TObject. ROOT handles all memory management while writing to and reading from ROOT files, and the order of reading is unimportant, as long as both reference and value have been read before the reference is used.

Pointers-as-data are converted between JSON arrays and C++ objects using the
ObjectProcessor<ParentType>::RegisterPointerBranch<ChildType> method. This takes a Processor for the ChildType as an argument. For C++ arrays / vectors, the Processor argument is instead a PointerArrayProcessor<ArrayContents>. Pointers-as-reference (TRef and TRefArray) are converted using the ObjectProcessor<ParentType>::RegisterTRef and
ObjectProcessor<ParentType>::RegisterTRefArray methods respectively.

Other equivalent data formats, for example YAML, use a unique identifier to reference a pointer-as-reference and store the pointer-as-data in a reserved part of the data tree. There are some consequences of storing pointers-as-reference using the path to a pointer-as-data as implemented in MAUS.

• The user must specify which data is the primary data source (pointer-as-data) and which data is a cross reference (pointer-as-reference).
• Pointers-as-reference are position dependent. If the associated pointer-as-data is moved the pointer-as-reference can no longer be resolved. For example, inserting an element into an array can cause misalignment of pointers-as-reference.
• Pointer data will always be available at the location of the pointer-as-data in the JSON tree, even when using a parser that is not pointer aware.
• A unique identifier type algorithm can be implemented as a relatively simple extension of the data format outlined here; but it is relatively hard to extend a unique identifier algorithm to reference existing parts of the data tree.

Pointer Resolution

Conversion from C++ pointers to JSON pointers is handled in a type-safe way. Values-as-data are stored in the data tree converted at run time from JSON to C++ and vice versa. Pointers-as-data are handled in the same way as Values-as-data. Pointers-as-references are stored in the C++ data tree as a TRef (or TRefArray element) in the normal way, and in JSON as an address to the position in the tree to a pointer-as-data. It is an error to store a pointer-as-reference without storing an associated pointer-as-data as the pointer-as-reference cannot be converted, unless the pointer-as-reference is set to NULL (in which case it may be an error depending on caller settings). It is an error to store multiple C++ pointers-as-data to the same memory address as the conversion from C++ to JSON and back again would yield logically different data and the resolution of associated pointers-as-reference is dependent on the resolution order of the data tree, which is ill-defined.

In order to implement the data conversion, the pointers have to be resolved in a two-stage process. In the first stage, it is necessary to collect all of the pointers-as-data and pointers-as-reference by traversing the data tree. This is performed during the standard data conversion, but pointers-as-reference are left pointing to NULL. A mapping from the pointer-as-data in the original data format to the pointer-as-data in the converted data format is stored, together with a list of pointers-as-reference in the original data format and the necessary mutators in the converted data format. In the second stage MAUS iterates over the pointers-as-reference, finds the appropriate pointer-as-data and writes the location of the pointer-as-data to the pointer-as-reference in the converted data format. The code is templated to maintain full type-safety during this process.

frame=single, basicstyle=, keywordstyle= , identifierstyle= , commentstyle= , stringstyle=,showstringspaces=false,captionpos=b