Project

General

Profile

Support #670

MAUS release 0.5 installation test failure (64bit CentOS)

Added by Dobbs, Adam almost 11 years ago. Updated almost 11 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Tunnell, Christopher
Category:
Build System
Target version:
Start date:
09 August 2011
Due date:
% Done:

100%

Estimated time:
Workflow:

Description

A test failure for 'MAUSSteppingActionTest.UserSteppingActionVirtualTest' occurs after installing maus release 0.5 on the Imperial lx cluster (64bit CentOS). Error output in the attached file. Any thoughts much appreciated.


Files

MAUSSteppingActionTestError.txt (22.2 KB) MAUSSteppingActionTestError.txt Dobbs, Adam, 09 August 2011 13:51
SconsOutput.txt (9.12 KB) SconsOutput.txt Dobbs, Adam, 09 August 2011 17:37
mausbuild.log (3.38 MB) mausbuild.log Dobbs, Adam, 10 August 2011 15:02
scons_std (24.1 KB) scons_std Dobbs, Adam, 18 August 2011 14:51
scons_err (1.42 KB) scons_err Dobbs, Adam, 18 August 2011 14:51
#1

Updated by Tunnell, Christopher almost 11 years ago

It also can't import MapCppSimulation or really anything that links to C++.

How'd you try to install it?

What's in your build directory: ls build

What gets said when you run:

python
then
import MapCppSimulation
?

#2

Updated by Dobbs, Adam almost 11 years ago

Hi Chris,

The build command I used was:

./tests/integration/install_then_build_then_test.bash

as on the website. The output from ls build and trying to import the python module is:

[adobbs@lx05 maus]$ ls build
cpplint_exceptions.py        MapPyGroup.pyc          test_always_true.py          test_InputPyEmptyDocument.py      test_MapPyValidateSpill.py
cpplint_exceptions.pyc       MapPyPrint.py           test_always_true.pyc         test_InputPyEmptyDocument.pyc     test_MapPyValidateSpill.pyc
InputPyEmptyDocument.py      MapPyPrint.pyc          test_cdb                     test_InputPyJson.py               test_numpy.py
InputPyEmptyDocument.pyc     MapPyRemoveTracks.py    test_core_configuration.py   test_InputPyJson.pyc              test_numpy.pyc
InputPyJSON.py               MapPyRemoveTracks.pyc   test_core_configuration.pyc  test_MapPyBeamMaker.py            test_OutputPyDoNothing.py
InputPyJSON.pyc              MapPyValidateSpill.py   test_core_go.py              test_MapPyBeamMaker.pyc           test_OutputPyDoNothing.pyc
libMausCpp.so                MapPyValidateSpill.pyc  test_core_go.pyc             test_MapPyCreateSpill.py          test_OutputPyJSON.py
MapPyBeamMaker.py            MAUS.py                 test_cpp_style.py            test_MapPyCreateSpill.pyc         test_OutputPyJSON.pyc
MapPyBeamMaker.pyc           MAUS.pyc                test_cpp_style.pyc           test_MapPyDoNothing.py            test_python_style.py
MapPyCreateSpill.py          OutputPyDoNothing.py    test_cpp_unit                test_MapPyDoNothing.pyc           test_python_style.pyc
MapPyCreateSpill.pyc         OutputPyDoNothing.pyc   test_cpp_unit.py             test_MapPyFakeTestSimulation.py   test_root_plot.py
MapPyDoNothing.py            OutputPyJSON.py         test_cpp_unit.pyc            test_MapPyFakeTestSimulation.pyc  test_root_plot.pyc
MapPyDoNothing.pyc           OutputPyJSON.pyc        test_error_handler.py        test_MapPyPrint.py                test_schema_schema.py
MapPyFakeTestSimulation.py   ReducePyDoNothing.py    test_error_handler.pyc       test_MapPyPrint.pyc               test_schema_schema.pyc
MapPyFakeTestSimulation.pyc  ReducePyDoNothing.pyc   test_geometry.py             test_MapPyRemoveTracks.py         test_spill_schema.py
MapPyGroup.py                suds                    test_geometry.pyc            test_MapPyRemoveTracks.pyc        test_spill_schema.pyc
[adobbs@lx05 maus]$ python
Python 2.7 (r27:82500, Aug  9 2011, 11:29:24) 
[GCC 4.1.2 20080704 (Red Hat 4.1.2-50)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import MapCppSimulation
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named MapCppSimulation
>>> 

Thanks for the swift response too.

#3

Updated by Tunnell, Christopher almost 11 years ago

Can you send the output from 'scons'? It looks like either ROOT or geant4 failed to build...

I'm testing it on a centos machine we have at RAL too.

#4

Updated by Rogers, Chris almost 11 years ago

Did you build on top of an existing copy of MAUS? Might have caused a problem...

#5

Updated by Tunnell, Christopher almost 11 years ago

So far so good for the build...

#6

Updated by Dobbs, Adam almost 11 years ago

Tunnel, please find attached the scons output. Get the same error after running the command and re-running the tests. Rogers, this was originally a clean install, but after the initial fail I have tried re-running the installation command in the same directory.

#7

Updated by Tunnell, Christopher almost 11 years ago

What version of centos? The trunk works on Centos 5.6 (what we use in the control room)

http://test.mice.rl.ac.uk/job/MAUS_trunk_batch/label=heplnm071/

Can you try the trunk instead of a release just to see if that repeats the question? Helps diagnose if something changed since the release.

I was expecting to see it not find ROOT or geant4 in the scons output... so am confused.

#8

Updated by Rogers, Chris almost 11 years ago

  • Assignee changed from Rogers, Chris to Tunnell, Christopher
  • Target version set to Future MAUS release

Assigned to Tunnell - it was passing in trunk, so presume it's an issue with build on CENT OS...

#9

Updated by Dobbs, Adam almost 11 years ago

So the centos version is also 5.6 strangely:

[adobbs@lx05 mice3]$ cat /etc/*release*
cat: /etc/lsb-release.d: Is a directory
CentOS release 5.6 (Final)
[adobbs@lx05 mice3]$ cat /proc/version
Linux version 2.6.18-238.12.1.el5 () (gcc version 4.1.2 20080704 (Red Hat 4.1.2-50)) #1 SMP Tue May 31 13:22:04 EDT 2011

Will try again from the trunk and post the results...

#10

Updated by Tunnell, Christopher almost 11 years ago

  • Subject changed from MAUS release 0.5 installation test failure to MAUS release 0.5 installation test failure (64bit CentOS)
#11

Updated by Dobbs, Adam almost 11 years ago

So, I have checked out the trunk and tried that on the imperial lx cluster. The installation was clean, and set to run with the ./tests/integration/install_then_build_then_test.bash command. The output from the whole build and test is in the attached file. Tests failed with the same error as before.

#12

Updated by Rogers, Chris almost 11 years ago

I found that

[  FAILED  ] 1 test, listed below:
[  FAILED  ] MAUSSteppingActionTest.UserSteppingActionVirtualTest

was a genuine bug. One of the tests had a dependency on the execution order (because we can't delete then reopen geant4 without running a new executable, which is difficult). Apologies and fixed in 0.0.6

Edit - the code was okay, the test was broken; but a real problem with the code.

#13

Updated by Tunnell, Christopher almost 11 years ago

  • Status changed from Open to Closed
  • % Done changed from 0 to 100

guess that's fixed then.

#14

Updated by Rogers, Chris almost 11 years ago

Still don't understand why we fail to import MapCppSimulation...

#15

Updated by Tunnell, Christopher almost 11 years ago

I can't repeat that error on our test machine and I believe it went away? The only remaining error was the SteppingAction thing that is fixed now, right?

#16

Updated by Dobbs, Adam almost 11 years ago

Glad to hear this lead to a bug being tracked down. Unfortunately Tunnel is right, the import issue is still present for me, on both the Imperial systems and one of my Ubuntu laptops (not on the other laptop though, on which maus seems to be working!). Should I start a new issue specifically for the import problem?

#17

Updated by Rogers, Chris almost 11 years ago

  • Status changed from Closed to Open
  • % Done changed from 100 to 0
#18

Updated by Dobbs, Adam almost 11 years ago

Apologies, that was actually Rogers' point...

#19

Updated by Dobbs, Adam almost 11 years ago

An example of the import problem from CentOS:

[adobbs@lx05 maus]$ ./bin/analyze_data_offline.py ~/mice2/ParticleRate/RawData/1900/ 1985.000
failed to import InputCppData
failed to import MapCppTOFDigitization
failed to import MapCppSimulation
failed to import MapCppPrint
failed to import MapCppTrackerDigitization
Data path: /home/hep/adobbs/mice2/ParticleRate/RawData/1900/
Data file 1985.000

Traceback (most recent call last):
  File "./bin/analyze_data_offline.py", line 57, in <module>
    run(data_path, data_file)
  File "./bin/analyze_data_offline.py", line 23, in run
    my_input = MAUS.InputCppData(data_path, data_file)
AttributeError: 'module' object has no attribute 'InputCppData'
[adobbs@lx05 maus]$ 

#20

Updated by Rogers, Chris almost 11 years ago

You checked that you ran source env.sh I guess? Can you do

whereis python
python -v

#21

Updated by Rogers, Chris almost 11 years ago

then maybe

pwd
echo $MAUS_ROOT_DIR
ls $MAUS_ROOT_DIR/build

#22

Updated by Dobbs, Adam almost 11 years ago

Here we go, with a 'which python' thrown in:

[adobbs@lx05 maus]$ whereis python
python: /usr/bin/python /usr/bin/python2.4 /usr/lib/python2.4 /usr/include/python2.4 /usr/share/man/man1/python.1.gz
[adobbs@lx05 maus]$ which python
~/mice3/maus/third_party/install/bin/python
[adobbs@lx05 maus]$ python -V
Python 2.7
[adobbs@lx05 maus]$ ./bin/analyze_data_offline.py ~/mice2/ParticleRate/RawData/1900/ 1985.000
failed to import InputCppData
failed to import MapCppTOFDigitization
failed to import MapCppSimulation
failed to import MapCppPrint
failed to import MapCppTrackerDigitization
Data path: /home/hep/adobbs/mice2/ParticleRate/RawData/1900/
Data file 1985.000

Traceback (most recent call last):
  File "./bin/analyze_data_offline.py", line 57, in <module>
    run(data_path, data_file)
  File "./bin/analyze_data_offline.py", line 23, in run
    my_input = MAUS.InputCppData(data_path, data_file)
AttributeError: 'module' object has no attribute 'InputCppData'
[adobbs@lx05 maus]$ pwd
/home/hep/adobbs/mice3/maus
[adobbs@lx05 maus]$ echo $MAUS_ROOT_DIR
/home/hep/adobbs/mice3/maus
[adobbs@lx05 maus]$ ls $MAUS_ROOT_DIR/build
cpplint_exceptions.py        MapPyGroup.pyc          test_always_true.py          test_InputPyEmptyDocument.py      test_MapPyValidateSpill.py
cpplint_exceptions.pyc       MapPyPrint.py           test_always_true.pyc         test_InputPyEmptyDocument.pyc     test_MapPyValidateSpill.pyc
InputPyEmptyDocument.py      MapPyPrint.pyc          test_cdb                     test_InputPyJson.py               test_numpy.py
InputPyEmptyDocument.pyc     MapPyRemoveTracks.py    test_core_configuration.py   test_InputPyJson.pyc              test_numpy.pyc
InputPyJSON.py               MapPyRemoveTracks.pyc   test_core_configuration.pyc  test_MapPyBeamMaker.py            test_OutputPyDoNothing.py
InputPyJSON.pyc              MapPyValidateSpill.py   test_core_go.py              test_MapPyBeamMaker.pyc           test_OutputPyDoNothing.pyc
libMausCpp.so                MapPyValidateSpill.pyc  test_core_go.pyc             test_MapPyCreateSpill.py          test_OutputPyJSON.py
MapPyBeamMaker.py            MAUS.py                 test_cpp_style.py            test_MapPyCreateSpill.pyc         test_OutputPyJSON.pyc
MapPyBeamMaker.pyc           MAUS.pyc                test_cpp_style.pyc           test_MapPyDoNothing.py            test_python_style.py
MapPyCreateSpill.py          OutputPyDoNothing.py    test_cpp_unit                test_MapPyDoNothing.pyc           test_python_style.pyc
MapPyCreateSpill.pyc         OutputPyDoNothing.pyc   test_cpp_unit.py             test_MapPyFakeTestSimulation.py   test_root_plot.py
MapPyDoNothing.py            OutputPyJSON.py         test_cpp_unit.pyc            test_MapPyFakeTestSimulation.pyc  test_root_plot.pyc
MapPyDoNothing.pyc           OutputPyJSON.pyc        test_error_handler.py        test_MapPyPrint.py                test_schema_schema.py
MapPyFakeTestSimulation.py   ReducePyDoNothing.py    test_error_handler.pyc       test_MapPyPrint.pyc               test_schema_schema.pyc
MapPyFakeTestSimulation.pyc  ReducePyDoNothing.pyc   test_geometry.py             test_MapPyRemoveTracks.py         test_spill_schema.py
MapPyGroup.py                suds                    test_geometry.pyc            test_MapPyRemoveTracks.pyc        test_spill_schema.pyc
[adobbs@lx05 maus]$ 

Does this shed any light on it?

#23

Updated by Tunnell, Christopher almost 11 years ago

Does your output called mausbuild.log include errors?

ie. tests/integration/install_then_build.bash 1>maus_std 2>maus_errors

Let's go step by step. It's failing to build the C++ extensions for some reason. What does which 'root' say? Then just try running ROOT to see if you get the command prompt.

Can you then edit (just add some dummy line) a line within src/map/MapCppSimulation/MapCppSimulation.cc and then send us the output from:

scons 1>scons_std 2>scons_err

Then try running:

python -m MAUS

and/or:

python

then:

import MAUS

#24

Updated by Dobbs, Adam almost 11 years ago

Hi Chris,

I believe the old mausbuild.log should have included everything. Here is the output from the above commands:

[adobbs@lx05 maus]$ source env.sh 
SUCCESS: MAUS setup
[adobbs@lx05 maus]$ which root
~/mice3/maus/third_party/build/root_v5.28.00d/bin/root
[adobbs@lx05 maus]$ root
  *******************************************
  *                                         *
  *        W E L C O M E  to  R O O T       *
  *                                         *
  *   Version  5.28/00d        7 May 2011   *
  *                                         *
  *  You are welcome to visit our Web site  *
  *          http://root.cern.ch            *
  *                                         *
  *******************************************

ROOT 5.28/00d (tags/v5-28-00d@39145, May 07 2011, 22:11:33 on linuxx8664gcc)

CINT/ROOT C/C++ Interpreter version 5.18.00, July 2, 2010
Type ? for help. Commands must be C++ statements.
Enclose multiple statements between { }.
OBJ: TStyle     style1  style1 : 0 at: 0x13f1d880
root [0] c1 = TCanvas()
(class TCanvas)337462448
root [1] .q
[adobbs@lx05 maus]$ vim src/map/MapCppSimulation/MapCppSimulation.cc
[adobbs@lx05 maus]$ scons 1>scons_std 2>scons_err
[adobbs@lx05 maus]$ vim scons_std 
^[[A[adobbs@lx05 mau
[adobbs@lx05 maus]$ vim scons_err 
[adobbs@lx05 maus]$ vim src/map/MapCppSimulation/MapCppSimulation.cc
[adobbs@lx05 maus]$ scons 1>scons_std 2>scons_err
[adobbs@lx05 maus]$ vim scons_err 
[adobbs@lx05 maus]$ vim scons_std 
[adobbs@lx05 maus]$ python -m MAUS
failed to import InputCppData
failed to import MapCppTOFDigitization
failed to import MapCppSimulation
failed to import MapCppPrint
failed to import MapCppTrackerDigitization
[adobbs@lx05 maus]$  

I added a cout to src/map/MapCppSimulation/MapCppSimulation.cc. The subsequent scons output is attached. Its a bit confusing as it seems to register the change and rebuild the module as you would expect, but python still cannot find it.

Thanks for your efforts to btw.

#25

Updated by Tunnell, Christopher almost 11 years ago

Do this:

python

Then at the prompt

from MAUS import MapCppSimulation

It should give you a bit more information. The error you see before is within a try/catch block.

#26

Updated by Tunnell, Christopher almost 11 years ago

Are you also sure that all of this:

sudo yum groupinstall "Development Tools"
sudo yum groupinstall "Development Libraries"
sudo yum install wget libX11-devel libXft-devel libXext-devel libXpm-devel libX11-devel

has been done per the installation instructions? I'm just really confused because it works on our Centos build system...

#27

Updated by Dobbs, Adam almost 11 years ago

Ok, here's the output:

[adobbs@lx05 maus]$ python
Python 2.7 (r27:82500, Aug 18 2011, 11:52:29) 
[GCC 4.1.2 20080704 (Red Hat 4.1.2-50)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from MAUS import MapCppSimulation
failed to import InputCppData
failed to import MapCppTOFDigitization
failed to import MapCppSimulation
failed to import MapCppPrint
failed to import MapCppTrackerDigitization
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name MapCppSimulation
>>> 

Unfortunately I can't do any of the yum installs as I am trying in this case to make maus work on the HEP group servers, which I do not have admin access on (boo). If this proves insoluble perhaps I could make a trip to see one of you in person and we could try to sort it out then.

#28

Updated by Tunnell, Christopher almost 11 years ago

There are some things that are required and we can't install with our build system. I know it works on our test system where those command were run and we can't reasonably install those prereqs. Could you ask the system administrator per the instructions? I'm running out of ideas is all...

#29

Updated by Tunnell, Christopher almost 11 years ago

And that last command gave something different than I expected.

And sorry, I meant:

from MapCppSimulation import MapCppSimulation

Does it give you an error like:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/tunnell/mice/maus-trunk/build/MapCppSimulation.py", line 25, in <module>
    _MapCppSimulation = swig_import_helper()
  File "/home/tunnell/mice/maus-trunk/build/MapCppSimulation.py", line 21, in swig_import_helper
    _mod = imp.load_module('_MapCppSimulation', fp, pathname, description)
ImportError: libCore.so: cannot open shared object file: No such file or directory
#30

Updated by Dobbs, Adam almost 11 years ago

Here we go:

[adobbs@lx05 maus]$ python
Python 2.7 (r27:82500, Aug 18 2011, 11:52:29) 
[GCC 4.1.2 20080704 (Red Hat 4.1.2-50)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from MapCppSimulation import MapCppSimulation
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named MapCppSimulation
>>> 

I have also added Simon to the watcher list as he administers the Imperial HEP cluster.

#31

Updated by Tunnell, Christopher almost 11 years ago

Simon: do you want me to add the Imperial machines to the Jenkins build system? All I'd need would be a private SSH key for a user named 'jenkins' and bzr installed.

#32

Updated by Fayer, Simon almost 11 years ago

We don't support SSH keys here for security reasons, so it's probably not possible to test using Jenkins... Our frontend machines here are just standard CentOS-5.6 anyway so it should just work, I'll do some investigation...

#33

Updated by Tunnell, Christopher almost 11 years ago

Do you have restricted users or something? And password login?

#34

Updated by Dobbs, Adam almost 11 years ago

  • Status changed from Open to Closed
  • % Done changed from 0 to 100

Got it. Simon did some investigating for me and it turns out scons may get confused if the a path with a sym link is in env.sh (which there was in mine for MAUS_ROOT_DIR). Once this was changed to the full path scons found and built the missing modules and maus now runs happily it seems. Thanks all.

#35

Updated by Tunnell, Christopher almost 11 years ago

Is there a way to avoid this in the future? Thanks Simon for figuring it out.

#36

Updated by Dobbs, Adam almost 11 years ago

Might be worth updating the installation instructions to add a warning about installing maus after following sym links to get to the target directory, at least until a more permanent solution can be found.

#37

Updated by Tunnell, Christopher almost 11 years ago

It's a wiki. Want to help out by adding it to:

http://micewww.pp.rl.ac.uk/projects/maus/wiki/Install

"Dependencies and building MAUS"

It's a group effort so feel free to clarify the instructions whenever possible

#38

Updated by Dobbs, Adam almost 11 years ago

Good point! Done, please amend as you see fit.

Also available in: Atom PDF