Project

General

Profile

Bug #1815

MAUS numpy fails on GRID

Added by Rajaram, Durga about 5 years ago. Updated over 4 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Build System
Target version:
Start date:
01 February 2016
Due date:
% Done:

100%

Estimated time:
Workflow:
New Issue

Description

Janusz reports that MAUS installations are failing to run on GRID nodes.

After digging, found that the problem is coming from numpy/linalg

The diagnosis:
numpy gets built with system atlas/lapack libraries if they're available.
This causes a problem when porting the MAUS libraries to the GRID because the system lapack library is no longer available on the worker node thereby causing numpy import to fail.

The numpy installer script should force the installation to build standalone numpy/linalg/lapack_lite.so [ as it is done when there is no system-wide atlas/lapack ]

#1

Updated by Rajaram, Durga about 5 years ago

Some details:

Failure message from GRID

File "/cvmfs/mice.egi.eu/sl6/MAUS-v1.4.0/bin/utilities/download_geometry.py", line 23, in <module>
    import geometry
  File "/cvmfs/mice.egi.eu/sl6/MAUS-v1.4.0/src/common_py/geometry/__init__.py", line 4, in <module>
    from geometry.CADImport import CADImport
  File "/cvmfs/mice.egi.eu/sl6/MAUS-v1.4.0/src/common_py/geometry/CADImport.py", line 25, in <module>
    import numpy
  File "/cvmfs/mice.egi.eu/sl6/MAUS-v1.4.0/third_party/install/lib/python2.7/site-packages/numpy/__init__.py", line 137, in <module>
    import add_newdocs
  File "/cvmfs/mice.egi.eu/sl6/MAUS-v1.4.0/third_party/install/lib/python2.7/site-packages/numpy/add_newdocs.py", line 9, in <module>
    from numpy.lib import add_newdoc
  File "/cvmfs/mice.egi.eu/sl6/MAUS-v1.4.0/third_party/install/lib/python2.7/site-packages/numpy/lib/__init__.py", line 13, in <module>
    from polynomial import *
  File "/cvmfs/mice.egi.eu/sl6/MAUS-v1.4.0/third_party/install/lib/python2.7/site-packages/numpy/lib/polynomial.py", line 17, in <module>
    from numpy.linalg import eigvals, lstsq
  File "/cvmfs/mice.egi.eu/sl6/MAUS-v1.4.0/third_party/install/lib/python2.7/site-packages/numpy/linalg/__init__.py", line 48, in <module>
    from linalg import *
  File "/cvmfs/mice.egi.eu/sl6/MAUS-v1.4.0/third_party/install/lib/python2.7/site-packages/numpy/linalg/linalg.py", line 23, in <module>
    from numpy.linalg import lapack_lite
ImportError: liblapack.so.3: cannot open shared object file: No such file or directory

MAUS was built on a machine at Imperial College.
On the build machine,

 ldd of third_party/install/lib/python2.7/site-packages/numpy/linalg/lapack_lite.so  shows: 
-bash-4.1$ ldd third_party/install/lib/python2.7/site-packages/numpy/linalg/lapack_lite.so 
    linux-vdso.so.1 =>  (0x00007ffd88bfc000)
    liblapack.so.3 => /usr/lib64/atlas/liblapack.so.3 (0x00007f32a8b3e000)
    libblas.so.3 => /usr/lib64/libblas.so.3 (0x00007f32a88e6000)
    libpython2.7.so.1.0 => /vols/escience01/martynia/MAUS-v1.4.0/third_party/install/lib/libpython2.7.so.1.0 (0x00007f32a84d8000)
    libg2c.so.0 => /usr/lib64/libg2c.so.0 (0x00007f32a82b7000)
    libm.so.6 => /lib64/libm.so.6 (0x00007f32a8032000)
    libgcc_s.so.1 => /vols/escience01/martynia/MAUS-v1.4.0/third_party/install/lib64/libgcc_s.so.1 (0x00007f32a7e1c000)
    libc.so.6 => /lib64/libc.so.6 (0x00007f32a7a88000)
    libgfortran.so.3 => /vols/escience01/martynia/MAUS-v1.4.0/third_party/install/lib64/libgfortran.so.3 (0x00007f32a7769000)
    libf77blas.so.3 => /usr/lib64/atlas/libf77blas.so.3 (0x00007f32a754a000)
    libcblas.so.3 => /usr/lib64/atlas/libcblas.so.3 (0x00007f32a732a000)
    libz.so.1 => /lib64/libz.so.1 (0x00007f32a7113000)
    libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f32a6ef6000)
    libdl.so.2 => /lib64/libdl.so.2 (0x00007f32a6cf2000)
    libutil.so.1 => /lib64/libutil.so.1 (0x00007f32a6aee000)
    /lib64/ld-linux-x86-64.so.2 (0x0000003e68a00000)
    libquadmath.so.0 => /vols/escience01/martynia/MAUS-v1.4.0/third_party/install/lib/../lib64/libquadmath.so.0 (0x00007f32a68b1000)
    libatlas.so.3 => /usr/lib64/atlas/libatlas.so.3 (0x00007f32a6254000)

#2

Updated by Dobbs, Adam over 4 years ago

  • Status changed from Open to Closed
  • % Done changed from 0 to 100

Now fixed, closing.

Also available in: Atom PDF