Project

General

Profile

Feature #1633

Set unpacking version through environment

Added by Rajaram, Durga over 6 years ago. Updated about 6 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Build System
Target version:
Start date:
05 March 2015
Due date:
% Done:

100%

Estimated time:
Workflow:
New Issue

Description

With the change to the DATE libraries, MAUS will need to set the unpacker version to link against.
53unpacking.bash will build both StepI and StepIV libraries.
An environment variable should then:
  • set the unpacking library to link against -- either libMDUnpack_StepI.so or libMDUnpack_StepIV.so
  • set a compiler flag -D_STEPIV_DATA to pick up the correct header IF StepIV
  • NOT build InputCppDAQOnlineData IF StepI
#1

Updated by Rajaram, Durga over 6 years ago

Tested installation successfully on onrec03 (SL6.4)
Pushed to

lp:~maus-mlcr/maus/1633

  • third_party/bash/53unpacking.bash
    • builds both StepI and StepIV unpackers
    • named libMDUnpack_StepI.so and libMDUnpack_StepIV.so
    • location: third_party/install/lib
  • third_party/bash/70daq.bash
    • copies event.h header from correct location
  • configure has an environment variable MAUS_UNPACKER_VERSION
    • should be either
      StepI
      or
      StepIV
  • src/input/InputCppDAQOnlineDATA/sconscript modified to NOT build if unpacker is StepI
    • do not want to build Online DAQ for StepI libraries anymore -- unsupported
  • src/common_py/maus_build_tools/environment_tools.py modified to:
    • link against libMDUnpack_StepI or libMDUnpack_StepIV based on environment variable $MAUS_UNPACKER_VERSION
    • set CCFLAG -D_STEPIV_DATA if using StepIV library
    • add -lrt to build online on SL6
  • third_party/install_build_test_daq.bash
    • disabled 61monitoring -- not needed anymore
  • bin/online/setMonitor.sh
    • set monitoring host to miceraid5
#2

Updated by Dobbs, Adam over 6 years ago

Hi Durga, I just tried build this branch on the IC cluster, and got the following error:

======================================================================
FAIL: Test reading the whole file
----------------------------------------------------------------------
Traceback (most recent call last):
File "/vols/mice3/adobbs/maus/maus-shared/mlcr_1633/build/test_InputCppDAQOfflineData.py", line 98, in test_multi
'f699f0d81aee1f64a2f1cec7968b9289')
AssertionError: '38db21ad6b0387b976edaba0181a66c4' != 'f699f0d81aee1f64a2f1cec7968b9289'
-------------------- >> begin captured stdout << ---------------------
12

--------------------- >> end captured stdout << ----------------------

Is this expected?

#3

Updated by Rajaram, Durga over 6 years ago

Did you see the checksum difference with MAUS_UNPACKER_VERSION = StepI? or was it StepIV?

#4

Updated by Rajaram, Durga over 6 years ago

OK, so it does show up with the StepI unpacking library.
Yordan -- can you comment?
Should the md5sum in src/input/InputCppDAQOfflineData/test_InputCppDAQOfflineData.py have changed using event_old.h and libMDUnpack_StepI?

#5

Updated by Rajaram, Durga over 6 years ago

Problem is someone's making a softlink to the wrong unpacker version.
Digging..

#6

Updated by Karadzhov, Yordan over 6 years ago

As far as I understand, this is a failure that has to be expected. MAUS is built with the StepIV flag but the test_InputCppDAQOfflineData.py still tries to unpack a binary file from StepI and it gets bananas.

#7

Updated by Dobbs, Adam over 6 years ago

OK, in that case the test needs to be updated. We should have one test for the Step IV case, one for the Step I, and the tests should check at run time which version of the unpacker is present, and only run if the correct one is set. Yordan, could you take care of this please?

#8

Updated by Dobbs, Adam over 6 years ago

Once that is done, I will set up a dedicated Jenkins job to build for Step I and run the tests, to make sure the old data unpacking is getting checked regularly.

#9

Updated by Karadzhov, Yordan over 6 years ago

OK, but the problem is that for the moment we don't have any useful StepIV data.

#10

Updated by Rajaram, Durga over 6 years ago

How about we build MAUS with StepI unpacker as a default -- until we can get some StepIV sample data files to replace the current one?
I have pushed a mod to lp:~maus-mlcr/maus/1633 -- to build StepI as a default.
The tests should now pass.
To build with StepIV, ./configure has to be modified to change MAUS_UNPACKER_VERSION and rebuild

#11

Updated by Dobbs, Adam over 6 years ago

Great idea Durga. Yordan, fair point about the Step IV data. Is there anyway way we can just put in a simple test for the moment though to check the Step IV unpacker? Something which uses empty data from the new verison of DATE?

#12

Updated by Dobbs, Adam over 6 years ago

Thanks Durga, that fix works for me. Are you happy for me to merge this into the main mlcr branch and the trunk itself?

#13

Updated by Rajaram, Durga over 6 years ago

Hi Adam -- I'm OK with it going into the trunk.
I would like to emphasize a few things in case you want to handle it differently before it goes in
  • the default unpacker is StepI -- I think it's OK until we get a sample StepIV data file into the tests
    • the default (StepI) will NOT build InputCppDAQOnlinedata (and MAUS_online test)-- online now requires the StepIV unpacker library
  • On the test server, MAUS_per_commit_third_party has to be built before the other jobs can successfully run
  • the unpacker version is set through ./configure
    • so to switch, one would have to edit ./env.sh, re-source ./env.sh, redo third_party/bash/53unpacking.bash, do a scons -c; scons -j8
    • this means on the test server, if you're using MAUS_NODE_THIRD_PARTY, that will have to be rebuilt with the different unpacker version
  • with the version set to StepIV, InputCppDAQOnlineData (and MAUS_online etc) will need to be done on SL6
    • heplnv157 is coming back up, so it should be usable for that purpose
#14

Updated by Rajaram, Durga over 6 years ago

Fixed tests that were failing and pushed to the trunk.
Some of the tests are now handled differently based on whether the unpacker is step1 or step4

Also added a step4 sample file [ 06008 ] -- but note that this is garbage data, and should be replaced with a sensible file with actual hits,
after we start the weekend running.

Tests passed OK [ with default step1 unpacking ].
[ The integration test failure on heplnv157 is due to #1642, the same test passes on other nodes ]

One issue is we don't have a real Online test with StepIV unpacking.
Should probably setup a Step4_test job on jenkins to run on heplnv157
[ but only unit tests, application tests will fail because no mongo on heplnv157, see #1643 ]

In the meanwhile, Onrec02 has now been upgraded SL6 and I suggest using onrec02/03 to build and test StepIV
I have pulled the trunk to onrec02:MAUS/.maus_trunk and started an install there with StepIV unpacking set in ./configure

#15

Updated by Dobbs, Adam about 6 years ago

  • Status changed from Open to Closed
  • % Done changed from 0 to 100

Solved since at least MAUS 0.9.5, closing the issue.

Also available in: Atom PDF