Project

General

Profile

Bug #1700

MAUS 0.9.6 portability issues on GRID

Added by Rajaram, Durga over 8 years ago. Updated about 8 years ago.

Status:
Closed
Priority:
Urgent
Assignee:
Dobbs, Adam
Category:
Build System
Target version:
-
Start date:
28 June 2015
Due date:
% Done:

100%

Estimated time:
Workflow:
New Issue

Description

MAUS-v0.9.6 fails on the GRID. This appears to be because the path names in the third party ROOT install has full path names which fail when ported over to CVMFS and run on the GRID.

Files

21root.bash (4.37 KB) 21root.bash Rajaram, Durga, 29 June 2015 03:41
#1

Updated by Rajaram, Durga over 8 years ago

bash/21root.bash makes a soft link with the full path name in the third_party/build/ directory.
I changed it to relative path. 

cd MAUS-v0.9.6/third_party/build
ln -sf root_v5.34.30 root

Janusz applied a hot fix and tried again.

It failed again on the GRID -- related to root.

...
Fatal in <TVirtualStreamerInfo::Factory>: Cannot find the plugin handler for TVirtualStreamerInfo! However $ROOTSYS/etc/plugins/TVirtualStreamerInfo is accessible, Check the content of this directory!
..

#2

Updated by Rajaram, Durga over 8 years ago

I believe now that the problem is from doing a "fixed location installation" of root by specifying --prefix and doing a make install.
https://root.cern.ch/drupal/content/installing-root-source

c.f the 'old' root build in 0.9.4.

Trying a build with changes to 21root.
There are also a couple of other places in install_build_test & build_all where rootsys is set explicitly, I suspect that's not robust against porting over. [ thisroot. in the build directory should be safer, ref link above ]

#3

Updated by Dobbs, Adam over 8 years ago

In MAUS 0.9.6 ROOT has undergone a number of changes designed to separate it cleanly from any existing system ROOT installations, which sometimes has lead to problems. This turned out to be surprisingly tricky. As a part of this I tried using make install to put the ROOT libraries, etc in the standard install locations within the MAUS third party directory (third_party/install/lib and so on) and added a soft link to the ROOT build directory. I am surprised this has lead to problems as it was successfully tested on three different systems prior to release. Hey ho.

In more detail, the solution I employed to keep the installations separate had a few elements:

  • using make install and --prefix and --etcdir to specify the install directory in 21root.bash (probably not necessary but seemed like good practice)
  • creating a soft link called 'root' in the third_party/build directory to point to the original directory root was built in (third_party/build/root_v5.34.30 or something - I wanted it independent of the version number as this may change). Also implemented in 21root.bash.
  • I then used this soft link to specify which thisroot.sh should be run: source ${MAUS_ROOT_DIR}/third_party/build/root/bin/thisroot.sh

My first suggestions for a solution to the GRID issue would be:

  • In install_build_test.bash and build_all.bash, change source ${MAUS_ROOT_DIR}/third_party/build/root/bin/thisroot.sh to only use a relative filenames
  • In 21root.bash remove the --etcdir --prefix commands and just use make, not make install

That may do it. More tomorrow.... Apologies for the inconvenience.

#4

Updated by Dobbs, Adam over 8 years ago

Oh, and as you already have done Durga, make the third_party/build/root soft link relative as well.

#5

Updated by Rajaram, Durga over 8 years ago

Changing the root installer to build location independent & not do full link names does it.

To test,
I modified third_party/bash/21root.bash, ran install_build_test.bash, passed OK.
Then 1) tarred up the directory as Janusz would, 2) got rid of the directory, 3) untarred into a new directory with a different name, 4) ran configure & source env.sh, 5) ran MAUS and passed ok.

Attached modified 21root.bash

Diff of changes below

=== modified file 'third_party/bash/21root.bash'
--- third_party/bash/21root.bash    2015-06-01 12:13:36 +0000
+++ third_party/bash/21root.bash    2015-06-29 02:28:41 +0000
@@ -57,15 +57,12 @@
     echo
     sleep 1

-    export ROOTSYS=${MAUS_ROOT_DIR}/third_party/build/${directory}
     x11=${MAUS_THIRD_PARTY}/third_party/install/lib/
     # hack to find third party libraries - for ubuntu et al where they have
     # weird and wonderful library locations to support multiple architectures.
     # Sticks them in ${x11} directory
     python ${MAUS_THIRD_PARTY}/third_party/install/bin/library_finder.py X11 Xext Xft
-    ./configure --prefix=${MAUS_ROOT_DIR}/third_party/install \
-              --etcdir=${MAUS_ROOT_DIR}/third_party/install/etc \
-              --disable-xrootd --enable-gsl-shared --enable-minuit2 \
+    ./configure --disable-xrootd --enable-gsl-shared --enable-minuit2 \
               --enable-builtin-freetype \
               --with-gsl-incdir=${MAUS_ROOT_DIR}/third_party/install/include \
               --with-gsl-libdir=${MAUS_ROOT_DIR}/third_party/install/lib \
@@ -84,9 +81,9 @@
     echo
     sleep 1
     make -j$MAUS_NUM_THREADS LDFLAGS="-Wl,--no-as-needed" || { echo "FAIL: Failed to configure/make"; exit 1; }
-    make install
     # Create a link to the ROOT directory not based on version number
-    ln -fs ${MAUS_ROOT_DIR}/third_party/build/${directory}/ ${MAUS_ROOT_DIR}/third_party/build/root
+    cd ${MAUS_ROOT_DIR}/third_party/build/
+    ln -fs ${directory} root
                 ##################################################
     echo
         echo "INFO: The package should be locally build now in your"

Adam should confirm if this is OK.

If it is, Janusz needs to redo the maus build with the patch
-- get rid of the MAUS-v0.9.6 directory [ there's a lot of root stuff stuck in third_party/install/.. ]
-- expand the maus 0.9.6 tarball into a new MAUS-v0.9.6,
-- replace third_party/bash/21root.bash with the attached [ pending Adam's ok & any other changes Adam wants to make]
-- ./install_build_test.bash
-- tar up, cvmfs, submit & hopefully see MAUS succeed

#6

Updated by Dobbs, Adam over 8 years ago

Thanks Durga. Could you attach the updated script to the issue please?

#7

Updated by Rajaram, Durga over 8 years ago

It is attached. Got added to the issue description at the top
And also in Comment #5

#8

Updated by Rajaram, Durga over 8 years ago

Also verified portability by tarring and copying the tarball to onrec03, untarring, configuring, and running MAUS.

#9

Updated by Martyniak, Janusz over 8 years ago

Hi,
To avoid any future confusions I would prefer to have the tarball (with the checksum) uploaded again to the right place so I could edo a full install.
best, JM

#10

Updated by Rajaram, Durga over 8 years ago

Adam, up to you how you want to handle this, whether a 0.9.6a with the fix, or a 0.9.7.
I see that the trunk hasn't changed since you merged back, should make it easier.
We've had a bit of a break with the lack of data taking the last couple of nights, but we should try to get the grid going
before data starts coming through again.

#11

Updated by Dobbs, Adam over 8 years ago

Thanks Durga, I will move straight to 0.9.7 and when it is announced I will make it plain it is a patch release for the GRID, and general users need not worry unless they are having installation issues. Is the only change need to swap in the updated ROOT script? I will get the release machinery going later today, should be done by Wednesday at latest.

Janusz, are you happy the changes fully address the issue you saw?

#12

Updated by Rajaram, Durga over 8 years ago

Thanks Adam. The root installer is the only change.

#14

Updated by Dobbs, Adam about 8 years ago

  • Status changed from Open to Closed
  • % Done changed from 0 to 100

Fixed, GRID running is fine again.

Also available in: Atom PDF