Project

General

Profile

Bug #1137

ROOT and std::vector

Added by Dobbs, Adam almost 9 years ago. Updated almost 9 years ago.

Status:
Rejected
Priority:
Normal
Assignee:
Richards, Alexander
Category:
Tracker
Target version:
Start date:
26 September 2012
Due date:
% Done:

0%

Estimated time:
Workflow:
New Issue

Description

I am attempting to add a std::vector<double> member to one of my classes and process it with the JsonCppProcessor framework. Everything seems to work, except for the final ROOT output. The data is successfully passed from a mapper to a reducer and appears when I use json as the output format. I have added a typedef to the class to disguise the fact that it is a stl container, but I'm still not seeing any data when I do a TTree::Scan. In addition, the member appears as a leaf in the TBrowser and is not browsable / split-able. I attach the output in json and root as an example (the tracker branch will be needed to read the root, to get the latest dictionary definitions). The member in question is:

Spill._recon._scifi_event._scifihelicalprtracks._phi_i

phi_i being the stl vector. I have also tried converting from json to root using the converter utility, but with no joy.

Thanks.


Files

maus_output.json (3.56 MB) maus_output.json Dobbs, Adam, 26 September 2012 18:34
maus_output.root (536 KB) maus_output.root Dobbs, Adam, 26 September 2012 18:34
SciFiHelicalPRTrack.hh (4.74 KB) SciFiHelicalPRTrack.hh Dobbs, Adam, 27 September 2012 12:22
Alex.root (5.35 KB) Alex.root Richards, Alexander, 28 September 2012 09:44
#1

Updated by Rogers, Chris almost 9 years ago

This took me ages to figure out, although it's kinduv obvious - you need to include in src/common_cpp/DataStructure/LinkDef.hh

#pragma link C++ class std::vector<MYTYPE>;

where obviously MYTYPE is filled with whatever. If you want to use iterators, you will need to include those as well, I guess something like

#pragma link C++ class std::vector<MYTYPE>::iterator;
#pragma link C++ class std::vector<MYTYPE>::const_iterator;

Note also #1066, which I haven't figured out the solution to yet.

#2

Updated by Rogers, Chris almost 9 years ago

Possibly this is dupe of #1051

#3

Updated by Richards, Alexander almost 9 years ago

That's correct, if I'd seen this ticket sooner I could have saved you the time. Essentially anything that you want to stream to ROOT needs to be known to root by building into ROOT dictionary. In real terms this amounts to doing the following. (Note for stl classes like vector you only need to do the second).
  1. Add the ' ClassDef() ' macro into your class definition
  2. Add the corresponding link line into the LinkDef header from which the ROOT dictionaries are built using rootcint

Cheers
Alex

#4

Updated by Rogers, Chris almost 9 years ago

Yes, for some reason I didn't realise that you need to specify the template type for STL vectors (or it just didn't click quite somehow). But of course template sorting happens at compile time so it's obviously needed in the linkdef... I guess I expect the linker to complain if something is missing from the LinkDef.hh, but for some reason this doesn't happen for STL stuff.

#5

Updated by Richards, Alexander almost 9 years ago

Below is a minimal example implementation:

My Class/structure to be rootable

#ifndef STR_B_H
#define STR_B_H
#include <TObject.h>

struct B {
  ClassDef(B,1)
  int a;
  int b;
};

#endif

and the contents of the LinkDef.h

#ifdef __CINT__

#pragma link C++ class B+;

#endif

Note this is sufficient to store into a root file with the streamers and to load with them. If you are planning on inspecting the root files or working with them then you will also need to make sure that the session of ROOT that you start up also understands about the ROOT able classes by loading the same ROOT dictionary created from above. To do this you do the following from the prompt
  • gROOT->ProcessLine(".L struct.h+");

This can also be put into a script called rootlogon.C and will then be called by default when you start root. Note that without this loading of the ROOT dictionary for your rootable objects they will most probably look like leaves and not be understood when you try to access them. You may of course get lucky if your structure is simple but generally they will be unintelligible to root without the dictionary loaded.

Hope that's clear
Cheers
Alex

#6

Updated by Rogers, Chris almost 9 years ago

Can you have a look at the documentation here and here and check that you are happy with it/improve where appropriate? It's in the trunk under doc/doc_src/introduction.tex and doc/doc_src/data_structure.tex respectively

#7

Updated by Richards, Alexander almost 9 years ago

will do.

#8

Updated by Dobbs, Adam almost 9 years ago

Hi Both, thanks for the advice. I have added a line to src/common_cpp/DataStructure/LinkDef.hh:

#pragma link C++ class std::vector<double>;

and done a clean and rebuild. On regenerating the ROOT output, I still see the same problem, the data being passed from the mapper to the reducer, but registering as 0 in ROOT. Doing a scan gives me:

root [3] Spill.Scan("_recon._scifi_event._scifihelicalprtracks._phi_i[0]","","colsize=50")
Warning in <TTreeFormula::DefinedVariable>: TTreeFormula support only 2 level of variables size collections.  Assuming '@' notation for the collection _phi_i.
****************************************************************************
*    Row   * Instance * _recon._scifi_event._scifihelicalprtracks._phi_i[0 *
****************************************************************************
*        0 *        0 *                                                  0 *
*        0 *        1 *                                                  0 *
*        1 *        0 *                                                  0 *
*        1 *        1 *                                                  0 *
*        2 *        0 *                                                  0 *
*        2 *        1 *                                                  0 *
*        3 *        0 *                                                  0 *
*        3 *        1 *                                                  0 *

...

Also the browser still views it as a non-splitable leaf. The dictionary is being loaded with:

.L $MAUS_ROOT_DIR/build/libMausCpp.so

The .hh file in question is attached. Any ideas?

#9

Updated by Dobbs, Adam almost 9 years ago

Also, the processor .hh for the class is:

namespace MAUS {

/** @class SciFiHelicalPRTrackProcessor processor for SciFiHelicalPRTrack */
class SciFiHelicalPRTrackProcessor : public ObjectProcessor<SciFiHelicalPRTrack> {
 public:
    /** Constructor - registers the branch structure */
    SciFiHelicalPRTrackProcessor();

 private:
    IntProcessor _int_proc;
    DoubleProcessor _double_proc;
    ValueArrayProcessor<double> _double_array_proc;
    ValueArrayProcessor<SciFiSpacePoint> _sf_spoint_array_proc;
};
} // ~namespace MAUS

#endif
#10

Updated by Rogers, Chris almost 9 years ago

Alex? Any ideas?

#11

Updated by Richards, Alexander almost 9 years ago

Hi all,

data member '_phi_i' is of type DoubleArray

DoubleArray _phi_i;

Is this type also rootable?

Cheers
Alex

#12

Updated by Richards, Alexander almost 9 years ago

I.e. does it fulfil points 1 and 2 above?

#13

Updated by Richards, Alexander almost 9 years ago

In addition you should add the '+' option to the linkdef and also when loading the dictionary as below:

#pragma link C++ class std::vector<double>+;
gROOT->ProcessLine(".L <yourclass>.h+");

This will instruct rootcint to make the streamer functions use readbuffer and writebuffer to handle the streaming.

see http://root.cern.ch/root/roottalk/roottalk01/3751.html where I quote

Unless you have good reasons to implement the Streamer function yourself,
you should always use the option "+" 

Cheers
Alex

#14

Updated by Dobbs, Adam almost 9 years ago

Thanks Alex.

DoubleArray was just a typedef for std::vector<double>, as the instructions said that it was neccessary if you were using stl containers with ROOT. Have now removed this and am using std::vector<double> directly. Have also changed the line in LinkDef.hh to

#pragma link C++ class std::vector<double>+;

Have searched for any .h+ files but there don't seem to be any to load. Still seeing the same behaviour in the ROOT output...

#15

Updated by Richards, Alexander almost 9 years ago

Hi Adam,
I just tried a very simplified example which seemed to show that vector<double> root dictionary already exists with my version of ROOT.

my LinkDef.h

#ifdef __CINT__

#pragma link C++ class std::vector<double>+;

#endif

when compiling:

[arichard@heppc242 TMP]$ make
rootcint -f DataDict.C -c struct.h LinkDef.h
Note: Link requested for already precompiled class vector<double,allocator<double> > (ignore this message) :0:
g++ -o main main.cxx DataDict.C `root-config --libs --cflags` `alex-config --libs --cflags RootStreamer`
[arichard@heppc242 TMP]$

anyway using a VERY simplified program for an example, see below, I have no problem storing vectors so If your around today I think the best way to solve this is if I come and have a look in person.

#include "RootStreamer/rstream.h" 
#include <vector>

int main(){

  std::vector<double> b;

  orstream f("Alex.root");

  f<<branchName("phi")<<b;

  b.push_back(12.4);
  b.push_back(124.);
  b.push_back(99.2);
  b.push_back(50.78);

  f<<fillEvent;
  b.clear();

  b.push_back(1.9276);

  f<<fillEvent;
  f.close();

  return 0;
}

Which gives me the root file attached which you should be able to view the correct structure (at least I can). Please check that you can as for you vector<double> may not be linked automatically and this will show up when you view the file.

Output from running program:

[arichard@heppc242 TMP]$ ./main
orstream            : INFO    : Written 2 event(s) to file.
[arichard@heppc242 TMP]$

Cheers
Alex

#16

Updated by Dobbs, Adam almost 9 years ago

Hi Alex,

Thanks for looking at this, and sorry for the slow reply, I was out of the office on Friday. I am in the office today if you are around? My suspicion is that this is not a dictionary issue, but due to the number of nested levels of stl containers we now have in the data structure. Apparently ROOT only supports 3 nested levels, so we may just have passed that limit (1 for level for the spill number, 1 level for the recon event number, 1 level for SciFiHelicalTracks). See http://root.cern.ch/root/roottalk/roottalk05/2690.html for a discussion. Be a pain if that is the case, but not sure what we can do about it.

Ad

#17

Updated by Richards, Alexander almost 9 years ago

Hi Adam,
This nested problem looks like it related to the drawing of the data structure or scanning. It should not stop you from storing said structure or retrieving it using the streamers. Have you checked that you can store a sample event and recall it using the streamers. I will be around tomorrow, will you be around then?
Cheers
Alex

#18

Updated by Richards, Alexander almost 9 years ago

I'm afraid that I have not got as firm a grasp of the MAUS data structure as yourselves. I assume the nesting levels here that your talking about amounts to there being multiple recon events per spill and multiple SciFiHelicalTracks per recon event right?

--Alex

#19

Updated by Dobbs, Adam almost 9 years ago

Yes, that's pretty much it, but the fact that we store multiple spills also counts I think.
Ad

#20

Updated by Richards, Alexander almost 9 years ago

Well assuming that a spill is synonymous with an event then you could remove one layer there as root is already geared up for storing events. i.e. you wouldn't need a separate spill object, you could just attach the branches for the recon events etc directly. You would still be able to iterate over the structure at the event/spill level using fill/readEvent.

--Alex

#21

Updated by Dobbs, Adam almost 9 years ago

Alex I sat down and looked at this earlier. It does seem to be the inherent ROOT limitation in viewing beyond three nested levels. The data is safely stored, but is not available via Scan or Draw. If converting from the ROOT file to JSON the data reappears, confirming this. It should also be possible to access the data by loading it back in to classes.

Chris, could you comment on Alex's last point above, about simplifying the implementation of the data structure in ROOT, so we have 1 less nested loop? After that I think we can close this.

#22

Updated by Rogers, Chris almost 9 years ago

Ack ROOT is a mess. No, we stick with the structure as defined, I don't want to mess with that. It took too much effort setting it up and, well, the spill is the correct thing to have at the root level.

Can you do something like

spill -> vector1 -> vector2
            \         ||       
             \        \/
              \---> vector3

where vector2 holds a pointer to vector3?

#23

Updated by Richards, Alexander almost 9 years ago

Storing the data and retrieving it again isn't the problem, so I assume that you could do this. Whether ROOT would be any better at letting you Draw/Scan this is anyone's guess, one would have to try.

I should point out that while an inconvienience it isn't the end of the world as the data is still stored and retrievable so you can still make the plots by looping over the data yourself. In fact TTree has a makeClass method which creates the code for you to do just this. Including adding any cuts or whatever you need. Back in the day we used to do a lot of our analysis in ATLAS using the makeclass classes.

Cheers
Alex

#24

Updated by Rogers, Chris almost 9 years ago

  • Status changed from Open to Rejected

Okay, so the answer is this is a feature/bug in ROOT and not fixable by us. So rejecting the issue.

Also available in: Atom PDF