OK, starting to look at this again.

At present the code runs but finds no data, not even tracker digits. Initial investigations show:

• Tracker real digitization process really needs a rewrite following the upgrade to the MAUS API. Currently we convert from a MAUS Data object, to JSON, back to a MAUS Spill

• The flaw however seems to enter in before then. The Data object seems to have empty arrays for the raw tracker data, which implies the bug is below the level of tracker digitization i.e. in the unpacking or some of the MAUS machinery that forms the Data object.

Work is ongoing...

Let me know if I can assist. Cheers...

OK, I think the problem lies in both the InputCppDAQData module and in UnpackingEventLib (both in src/input/InputCppDAQData).

Firstly, it seems InputCppDAQData, whilst having a VLSB processor member to deal with regular tracker DAQ data, has no VLSB_C processor for implementing tracker cosmic DAQ data (though the processor itself is available).

Secondly, it does not look like the unpacking libraries implements VLSB_C fully either. There is a VLSB_CDataProcessor class there, but it looks decidedly incomplete when compared with the VLSBCppDataProcessor. In particular, only the process function is implemented, without any fill_daq_data() or reset() functions, implying it cannot save any data it extracts.

Chris, thanks for the offer, would you be able to take a look at the indicated files and tell me if you agree with my conclusions? After that we need to decide who is responsible for writing the fix (one of us from tracker I would guess).

In case this is of help -- the unpacking library version changed at MAUS-v0.7.6.
To isolate the unpacker, it might be useful to see if it reconstructs with MAUS-v0.7.5?

Update: I have written new methods into UnpackingEventLib to process VLSB_C data using the data structure classes without recourse to json. I did this by imitating what had already been done for VLSB objects and by looking at the old json based methods for the VLSB_C objects. I now have VLSB_C daq data getting to the tracker digitisation routines, but the reconstruction comes out as a mess. The next step is to alter the digitisation so that it bypasses the conversion to json too, to see if that takes care of it.

Additionally I have come across a few issues along the way:

1. In different sections of code, the manner of differentiating between trackers seems to change. In UnpackingEventLib, it seems to be done by assuming the following:

LDC_ID = 0 -> tracker 1
LDC_ID = 2 -> tracker 2

In the digitisation however (RealDataDigitisation), for organising the DAQ data it seems to done as:

board = floor(bank/4) (for VLPC)

OR

board = Geo - 1 (for VLPC_C)

then

if board < 4 -> tracker 1
else -> tracker 2

It then done differently again by using the calibration files for creating the SciFiDigit, again in RealDataDigitisation.

Perhaps I do not understand, but this makes me uncomfortable. Referring to David for his view.

2. In UnpackingEventLib.cc VLSBCppDataProcessor::Process, the following logic appears:

xPartEv = xVLSBFragment->GetEventNum(xWordCount)-1;
if (xPartEv > _tracker1_spill.size()) {
  _tracker1_spill.resize(xPartEv);
  _tracker0_spill.resize(xPartEv);
  _single_st_spill.resize(xPartEv);
}

however this lead to a seg fault when applied to VLSB_C data, as xPartEv can be zero. I believe the correct implementation should replace xPartEv with xPartEv + 1 everywhere in the if block (cannot test without tracker data...), which is how I implemented it for the VLSB_C's.

3. In RealDataDigitisation.cc the VLSB processing is disabled with a few code tweaks - I am curious why someone would do this. There should be no VLSB data except for the single station test and Step IV onwards anyway.

Durga, thanks for the suggestion, if I get stuck I may give it a try.

More to follow...

Update: I have got reconstruction partially working. I think things are ok for tracker 1, but the calibration fails for tracker 2, and I think this is leading to the tracker 2 digits ending up in tracker 1.

I have made a branch to record my progress at https://code.launchpad.net/~phuccj/maus/cosmic_repair

David, could you take a look at the following and let me know what you think:

http://bazaar.launchpad.net/~phuccj/maus/cosmic_repair/view/head:/src/common_cpp/Recon/SciFi/RealDataDigitization.cc
VLSB_CCppDataProcessor::Process function (be careful of the name, there are a lot of similarly named functions.)

http://bazaar.launchpad.net/~phuccj/maus/cosmic_repair/view/head:/src/input/InputCppDAQData/UnpackEventLib.cc
RealDataDigitization::process_VLSB_c_2
RealDataDigitization::get_StatPlaneChannel

Work ongoing...

Aside: Unable to download and install maus 0.7.5 - cdb third party library cannot be downloaded when using bazaar, and tar ball links are broken from maus wiki for both source and third party libraries.

Yes, the problem is the cdb tarball source somehow disappeared/got mangled in Antony Wilson's branch
[ this is already an open issue #1471 ]
I then stuck the tarball in Micemine.
In addition, since ~0.8 the scons source disappeared from the pypi repository, and so we now get it from sourceforge.
The short answer - to get 0.7.5 to build -- is an ugly hack:
get

third_party/bash/40python_extras.bash

third_party/bash/44cdb.bash

third_party/bash

I was trying to save a MAUS copy of the old 3rd party stuff on heplnv156 in the third_party tarball - I guess some stuff got missed, python packages which don't come in the main 40python_extras.bash?

A test server job might be to disable $http_proxy, $https_proxy, then attempt to do the third party build; stuff that is available from heplnv156 should download without need for proxy; stuff from web should not download (fail to go through the proxy). This could be done, for example, at the release_candidate stage.

Is this the /redmine_repos/ area on heplnv156 you're talking about, Chris?
If so, do you mind if I use that area to stuff some TP sources in there? Or let me know if you suggest a different machine/location.

I meant e.g. /www/maus/MAUS_release_version_0.9.1/third_party_libraries_incl_python.tar.gz on heplnv156. We have a backlog of these all the way back to 0.2.0 on heplnm069 e.g. /backups/rsync-weekly/weekly/www/maus/MAUS_release_version_0.7.5/third_party_libraries_incl_python.tar.gz but needs an scp to access - we don't have so much hard drive space on the web machine that we can keep many back copies. I see Adam has a login on heplnm069.

Well, it seems I do. Out of curiosity, what is the purpose of that machine (it is missing from the list available on the computing wiki)? Pulled down the third party source, building at the moment...

Out of curiosity, what is the purpose of that machine (it is missing from the list available on the computing wiki)

Parry-riposte!

http://micewww.pp.rl.ac.uk/projects/computing-software/wiki/Computing_infrastructure#Responsibility-for-Hardware
http://micewww.pp.rl.ac.uk/projects/computing-software/wiki/Heplnm069_services

Well struck ;)

Also working in maus 0.7.5...

In the unpacking library here are two types of "data fragment" which correspond to the chunk of tracker data from each spill, one for the cosmic tests, in which the data from each VLSB board were packaged into the same "equipment" (i.e. under the same header), and one since then where the data from each VLSB were broken down into the memory banks, so under four separate headers.

In UnpackEventLib.cc there are two functions to process, both called VLSBDataProcessor::Process

The first uses the Non-cosmic MDfragmentVLSB and is intended to use the post-cosmic data.
The second (line 786) has a comment saying (actually they both do), only use with MDfragmentVLSB_C (for the cosmic data) as the argument, and seems to go and cast it as MDfragmentVLSB...

Both the cosmic and post-cosmic data types are in the equipment map from the unpacking, so the fragment should know what type it is and the processor should act accordingly.

My guess would be to remove the cast and let the function work on the MDfragmentVLSB_C type.

Update: Broken in maus v0.7.6, so now know that the break occurs between 0.7.5 and 0.7.6. Will try David's solution above and see what happens, and also examine changes between those two versions...

The third party unpacking library changed at 0.7.6
I've added Yordan to the watch list.
Yordan - do you have any idea how/why the unpacker change affects the tracker data?

Update: have managed to get the json-converter-cpp path working in RealDataDigitisation, but the output is again wrong, with nearly everything being in the upstream tracker and the z positions being clearly out. Tried switching to one of the older calibration files, but no joy either.

Running out of steam on how to solve this... David, I think this needs the eye of someone who knows the DAQ system now, can I transfer ownership of this to you?

Yordan, I'll echo Durga's comments - do you know what might have caused the break between maus 0.7.5 and 0.7.6?

A quick mind dump.. Yordan found 1. to make the on-mon work in the mock data run.

1. Modify /third_party/build/unpacking-mice/build/unpacking-mice/src/MDfragmentVLSB.cpp so _madeOfParticles = false; => _madeOfParticles = true;

2. Readout now returns: {"daq_event_type":"","errors":{"bad_data_input":"InputCppDAQOfflineData says:ERROR in MDprocessManager::ProcessSubEvent : \nwhile processing particle event in board VLSB Geo 0\nThe status is : 2 Phys. Event \n1 skipped!"},"maus_event_type":"Spill","run_number":0,"spill_number":0}

This is down to a cast error in L487 in src/input/InputCppDaqData/UnpackEventLib.cc
IE: MDfragmentVLSB is not being passed to the VSLBCppDataProcessor.

I'm on a train right now, but will have a major look soon.

Some things I notice:
The vlpc ADC distribution looks OK. Mose adcs read pedastool, with a hit in a couple of channels. A log-y plot should show a long tail upwards, which is the real light.

Theres something odd in the didgits plot. We should have an npe (number of photoelectrons) around 10. I can see a bump there, so I guess that's OK. The problem is the npe peak at zero. There should be a cut at 2 npe, which looks like its not getting applied. This could be confusing the software...

I found this problem too. It is a bug in one of the versions of the unpacking which must be fixed now. Try running with the current version of

http://dpnc.unige.ch/~yordan/unpacking-mice_2.tarz

Brilliant, thank you Yordan, now seeing unpacked daq data in MAUS datastructure, and scifi digits too!

Next issue: Nothing above digits produced, seeing an error message

TRANSFORM/MERGE/OUTPUT:  Processed 6 events so far, 1 events in buffer.
Traceback (most recent call last):
  File "/home/adobbs/maus/shared-repo/dev/src/common_py/ErrorHandler.py", line 159, in HandleCppException
    raise CppError(error_message)
ErrorHandler.CppError: Failed to find SciFi plane in _geometry_map. at SciFiClusterRec::process_cluster

Chris Heidt, this seems to be in your domain, any thoughts?