Bug #1815
MAUS numpy fails on GRID
Status:
Closed
Priority:
Normal
Assignee:
Category:
Build System
Target version:
Start date:
01 February 2016
Due date:
% Done:
100%
Estimated time:
Workflow:
New Issue
Description
Janusz reports that MAUS installations are failing to run on GRID nodes.
After digging, found that the problem is coming from numpy/linalg
The diagnosis:
numpy gets built with system atlas/lapack libraries if they're available.
This causes a problem when porting the MAUS libraries to the GRID because the system lapack library is no longer available on the worker node thereby causing numpy import to fail.
The numpy installer script should force the installation to build standalone numpy/linalg/lapack_lite.so [ as it is done when there is no system-wide atlas/lapack ]
Updated by Rajaram, Durga over 7 years ago
Some details:
Failure message from GRID
File "/cvmfs/mice.egi.eu/sl6/MAUS-v1.4.0/bin/utilities/download_geometry.py", line 23, in <module>
import geometry
File "/cvmfs/mice.egi.eu/sl6/MAUS-v1.4.0/src/common_py/geometry/__init__.py", line 4, in <module>
from geometry.CADImport import CADImport
File "/cvmfs/mice.egi.eu/sl6/MAUS-v1.4.0/src/common_py/geometry/CADImport.py", line 25, in <module>
import numpy
File "/cvmfs/mice.egi.eu/sl6/MAUS-v1.4.0/third_party/install/lib/python2.7/site-packages/numpy/__init__.py", line 137, in <module>
import add_newdocs
File "/cvmfs/mice.egi.eu/sl6/MAUS-v1.4.0/third_party/install/lib/python2.7/site-packages/numpy/add_newdocs.py", line 9, in <module>
from numpy.lib import add_newdoc
File "/cvmfs/mice.egi.eu/sl6/MAUS-v1.4.0/third_party/install/lib/python2.7/site-packages/numpy/lib/__init__.py", line 13, in <module>
from polynomial import *
File "/cvmfs/mice.egi.eu/sl6/MAUS-v1.4.0/third_party/install/lib/python2.7/site-packages/numpy/lib/polynomial.py", line 17, in <module>
from numpy.linalg import eigvals, lstsq
File "/cvmfs/mice.egi.eu/sl6/MAUS-v1.4.0/third_party/install/lib/python2.7/site-packages/numpy/linalg/__init__.py", line 48, in <module>
from linalg import *
File "/cvmfs/mice.egi.eu/sl6/MAUS-v1.4.0/third_party/install/lib/python2.7/site-packages/numpy/linalg/linalg.py", line 23, in <module>
from numpy.linalg import lapack_lite
ImportError: liblapack.so.3: cannot open shared object file: No such file or directory
MAUS was built on a machine at Imperial College.
On the build machine,
ldd of third_party/install/lib/python2.7/site-packages/numpy/linalg/lapack_lite.so shows:
-bash-4.1$ ldd third_party/install/lib/python2.7/site-packages/numpy/linalg/lapack_lite.so
linux-vdso.so.1 => (0x00007ffd88bfc000)
liblapack.so.3 => /usr/lib64/atlas/liblapack.so.3 (0x00007f32a8b3e000)
libblas.so.3 => /usr/lib64/libblas.so.3 (0x00007f32a88e6000)
libpython2.7.so.1.0 => /vols/escience01/martynia/MAUS-v1.4.0/third_party/install/lib/libpython2.7.so.1.0 (0x00007f32a84d8000)
libg2c.so.0 => /usr/lib64/libg2c.so.0 (0x00007f32a82b7000)
libm.so.6 => /lib64/libm.so.6 (0x00007f32a8032000)
libgcc_s.so.1 => /vols/escience01/martynia/MAUS-v1.4.0/third_party/install/lib64/libgcc_s.so.1 (0x00007f32a7e1c000)
libc.so.6 => /lib64/libc.so.6 (0x00007f32a7a88000)
libgfortran.so.3 => /vols/escience01/martynia/MAUS-v1.4.0/third_party/install/lib64/libgfortran.so.3 (0x00007f32a7769000)
libf77blas.so.3 => /usr/lib64/atlas/libf77blas.so.3 (0x00007f32a754a000)
libcblas.so.3 => /usr/lib64/atlas/libcblas.so.3 (0x00007f32a732a000)
libz.so.1 => /lib64/libz.so.1 (0x00007f32a7113000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f32a6ef6000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f32a6cf2000)
libutil.so.1 => /lib64/libutil.so.1 (0x00007f32a6aee000)
/lib64/ld-linux-x86-64.so.2 (0x0000003e68a00000)
libquadmath.so.0 => /vols/escience01/martynia/MAUS-v1.4.0/third_party/install/lib/../lib64/libquadmath.so.0 (0x00007f32a68b1000)
libatlas.so.3 => /usr/lib64/atlas/libatlas.so.3 (0x00007f32a6254000)
Updated by Dobbs, Adam over 6 years ago
- Status changed from Open to Closed
- % Done changed from 0 to 100
Now fixed, closing.