PROOF-Lite 5.26 or 5.27 trunk non-standard bin paths fails

Hi

I tried the simple instructions for starting PROOF-Lite found here: http://root.cern.ch/drupal/content/proof-multicore-desktop-laptop-proof-lite

But I got this error:

 TProof::Open("");
 +++ Starting PROOF-Lite with 8 workers +++
/usr/local/bin/proofserv: line 77: /bin/proofserv.exe: No such file or directory
/usr/local/bin/proofserv: line 77: exec: /bin/proofserv.exe: cannot execute: No such file or directory

The binary file proofserv and proofserv.exe are present in the folder /usr/local/bin/

$ ls /usr/local/bin/proofserv*
/usr/local/bin/proofserv  /usr/local/bin/proofserv.exe

Not sure why I have this error. I compiled ROOT from source (v5.26/00c) on a CentOS 5.5 Intel Xeon dual quad core and I would really really like to use all its might!.

This is what I used to compile:

Dear Andres,

This problem has been fixed in the trunk a couple of months ago.
I have just ported the fix in the 5-26-00-patches branch.
Attached is a patch for 5-26-00c: please untar the tarball in the source tree and run the full build process again (i.e. configure + make + make install). You should get the fixed version of proofserv under /usr/local/bin .

If it still does not work, please post the /usr/local/bin/proofserv file (do not worry: it is a script, not a binary).

G. Ganis
patch-33437.tar.gz (45.1 KB)

I’m afraid it didn’t work. The errors in the script are now not shown, but PROOF is not set to parallel mode:

root [0] TProof::Open("");
 +++ Starting PROOF-Lite with 8 workers +++
Opening connections to workers: OK (8 workers)                 
PROOF set to sequential mode
root [1] 

I also had a lot of trouble compiling from source. The process hanged several times in different locations when I passed “-j n” to the make command (4,6 and 8 all hanged, except 1, i.e. plain make).

If this continues I guess I’ll try the development version 5.27. I just went with 5.26 because I figured it would be more stable.

This is the output of the proofserv you requested:

#!/bin/sh

#
# $Id:$
#
# Template for the proofserv wrapper script.
# The proofserv wrapper script can be used to initialize the
# environment for proofserv as needed. It could be extended
# to select a specific ROOT version. It also allows debug
# tools like valgrind etc to be used.
#
# This example script should be sufficient for most installations
# but can be modified as needed.
#

#
# Setup notification
#
if [ -n "$ROOTPROOFLOGFILE" ]; then
# Use the standard log file
   LOGFILE="$ROOTPROOFLOGFILE"
else
# Use a temp file
   if [ -n "$TMPDIR" ]; then
      LOGFILE="$TMPDIR/proofserv.log"
   else
      LOGFILE="/tmp/proofserv.log"
   fi
fi

#
# If requested, initialize the environment.
# The PROOF_INITCMD variable contains the command to be executed to initialize
# the environment.
# For a simple variable setting just use 'echo export VAR=value', e.g.
#
# root [] TProof::AddEnvVar("PROOF_INITCMD",
#               "echo export LD_LIBRARY_PATH=/some/new/libpath:$LDLIBRARY_PATH")
#
# If the setup is defined by a script, e.g. /some/path/setup-env.sh, then the
# script should be sourced:
#
# root [] TProof::AddEnvVar("PROOF_INITCMD","source /some/path/setup-env.sh")
#
# If the script outputs the command to be executed, e.g. /some/path/getscram.sh,
# then just put the script path:
#
# root [] TProof::AddEnvVar("PROOF_INITCMD","/some/path/getscram.sh")
#

if [ -n "$PROOF_INITCMD" ]; then
   NOW=`date +%H:%M:%S`
   echo "$NOW: $1: initializing environment with: $PROOF_INITCMD" >> "$LOGFILE"
   eval `$PROOF_INITCMD`
fi

#
# Run master, workers or all with a debug command.
# E.g in the client do:
#   root [] TProof::AddEnvVar("PROOF_WRAPPERCMD","valgrind --log-file=/tmp/vg") 
#
if [ -n "$PROOF_WRAPPERCMD" ]; then
   WRAPPER="$PROOF_WRAPPERCMD "
else
   if [ -n "$PROOF_MASTER_WRAPPERCMD" -a "$1" = "proofserv" ]; then
      WRAPPER="$PROOF_MASTER_WRAPPERCMD "
   fi
   if [ -n "$PROOF_SLAVE_WRAPPERCMD" -a "$1" = "proofslave" ]; then
      WRAPPER="$PROOF_SLAVE_WRAPPERCMD "
   fi
fi

if [ "x$WRAPPER" != "x" ]; then
   NOW=`date +%H:%M:%S`
   echo "$NOW: $1: executing $WRAPPER $ROOTSYS/bin/proofserv.exe $@" >> $LOGFILE
fi

exec $WRAPPER /usr/local/bin/proofserv.exe "$@"

Well, this is very strange.
The proofserv seems correct.

If you want to stick to 5.26 you can try to build 5-26-00-patches which is basically 5.26/00c plus a couple of fixes (including this one). You have to take the source from SVN:

$ svn co http://root.cern.ch/svn/root/branches/v5-26-00-patches root 

and proceed to build as for 5-26-00c . This is actually what I used to test the patch.

G. Ganis

No luck with 5.26.00-patches either…same thing as my last post. Is there a way to debug this by setting PROOF or ROOT to verbose or something?

On the other hand the compilation with ‘make -j 8’ was carried out smoothly and very fast.

I’ll give 5.27 a try at some point.

I just tried version 5.27 from trunk not on the CentOS machine, but on the Debian Squeeze desktop (core 2 duo processor). This is what I got:

root [0] TProof::Open("");
 +++ Starting PROOF-Lite with 2 workers +++
/home/candres/.proof//session-phonon-1280527349-28712/worker-0.0.env: 1: Syntax error: "(" unexpected
Error in <TSlaveLite::Init>: an error occured while executing 'proofserv'
/home/candres/.proof//session-phonon-1280527349-28712/worker-0.1.env: 1: Syntax error: "(" unexpected
Error in <TSlaveLite::Init>: an error occured while executing 'proofserv'
Opening connections to workers: OK (2 workers)                 
PROOF set to sequential mode

The proofserv for this computer is as follows:

#!/bin/sh

#
# $Id: proofserv.in 33454 2010-05-10 20:35:07Z rdm $
#
# Template for the proofserv wrapper script.
# The proofserv wrapper script can be used to initialize the
# environment for proofserv as needed. It could be extended
# to select a specific ROOT version. It also allows debug
# tools like valgrind etc to be used.
#
# This example script should be sufficient for most installations
# but can be modified as needed.
#

#
# Setup notification
#
if [ -n "$ROOTPROOFLOGFILE" ]; then
# Use the standard log file
   LOGFILE="$ROOTPROOFLOGFILE"
else
# Use a temp file
   if [ -n "$TMPDIR" ]; then
      LOGFILE="$TMPDIR/proofserv.log"
   else
      LOGFILE="/tmp/proofserv.log"
   fi
fi

#
# If requested, initialize the environment.
# The PROOF_INITCMD variable contains the command to be executed to initialize
# the environment.
# For a simple variable setting just use 'echo export VAR=value', e.g.
#
# root [] TProof::AddEnvVar("PROOF_INITCMD",
#               "echo export LD_LIBRARY_PATH=/some/new/libpath:$LDLIBRARY_PATH")
#
# If the setup is defined by a script, e.g. /some/path/setup-env.sh, then the
# script should be sourced:
#
# root [] TProof::AddEnvVar("PROOF_INITCMD","source /some/path/setup-env.sh")
#
# If the script outputs the command to be executed, e.g. /some/path/getscram.sh,
# then just put the script path:
#
# root [] TProof::AddEnvVar("PROOF_INITCMD","/some/path/getscram.sh")
#

if [ -n "$PROOF_INITCMD" ]; then
   NOW=`date +%H:%M:%S`
   echo "$NOW: $1: initializing environment with: $PROOF_INITCMD" >> "$LOGFILE"
   eval `$PROOF_INITCMD`
fi

#
# Run master, workers or all with a debug command.
# E.g in the client do:
#   root [] TProof::AddEnvVar("PROOF_WRAPPERCMD","valgrind --log-file=/tmp/vg") 
#
if [ -n "$PROOF_WRAPPERCMD" ]; then
   WRAPPER="$PROOF_WRAPPERCMD "
else
   if [ -n "$PROOF_MASTER_WRAPPERCMD" -a "$1" = "proofserv" ]; then
      WRAPPER="$PROOF_MASTER_WRAPPERCMD "
   fi
   if [ -n "$PROOF_SLAVE_WRAPPERCMD" -a "$1" = "proofslave" ]; then
      WRAPPER="$PROOF_SLAVE_WRAPPERCMD "
   fi
fi

if [ "x$WRAPPER" != "x" ]; then
   NOW=`date +%H:%M:%S`
   echo "$NOW: $1: executing $WRAPPER $ROOTSYS/bin/proofserv.exe $@" >> $LOGFILE
fi

exec $WRAPPER /usr/local/bin/proofserv.exe "$@"

Don’t really know what to try to make this work with any computer…am I missing something important I should have when running the configuration script?:

./configure linuxx8664gcc --prefix=/usr/local --disable-builtin-afterimage --disable-builtin-freetype --disable-builtin-glew --disable-builtin-pcre --disable-builtin-zlib --disable-builtin-ftgl --enable-gsl-shared --enable-minuit2 --enable-roofit --enable-table --enable-gdml --enable-qt --enable-qtgsi --enable-soversion --enable-unuran --enable-ruby --disable-afs --with-xrootd-opts="--disable-afs"

This is described as a non-standard bin path…what is the standard bin path? /usr/bin ? but this is reserved for the OS package manager (apt or yum).

NOTE: the configure script in the CentOS machine does not have the last two options

Dear Andres,

Sorry for the misleading definition: the majority of people do not use the prefix installation of ROOT but just point ROOTSYS to a directory where the ROOT binaries, includes, etc … are. So, in this case “stardard” is $ROOTSYS/bin, but, I agree, this is not very correct.

Anyhow, CentOS and Debian Squeeze are not distributions on which we build or test. I will try to reproduce your problem in a virtual machine, but I cannot given much priority to this.

I suggest that in the meantime you drop the --prefix=/usr/local and just run

$ source <dir_where_ROOT_is_built>/bin/thisroot.sh

Then it should work.

G. Ganis

Hi,

I have tried on a Xubuntu 10.04 box (which is debian behind, right?) configuring like this:

./configure --prefix=/opt/root --etcdir=/opt/root/etc

and everything works perfectly (I did not want to touch /usr/local and /etc, so I confined everything under /opt. But this should not matter …).

As I said I will try on a CentOS dist in a virtual machine and let you know.

G. Ganis

I did what you suggested and did:

./configure linuxx8664gcc --prefix=/opt/root --etcdir=/opt/root/etc --disable-builtin-afterimage --disable-builtin-freetype --disable-builtin-glew --disable-builtin-pcre --disable-builtin-zlib --disable-builtin-ftgl --enable-gsl-shared --enable-minuit2 --enable-roofit --enable-table --enable-gdml --enable-qt --enable-qtgsi --enable-soversion --enable-unuran --enable-ruby --disable-afs --with-xrootd-opts="--disable-afs"

Unfortunately I have 2 problems. Starting root is a bit a of nightmare:

root -l
-bash: root: command not found

So I tried what the tutorial says:

. bin/thisroot.sh

But this only holds for one ssh session and depends on the directory from which I execute the script…

Also I tried:

root [0] TProof::Open("");
 +++ Starting PROOF-Lite with 2 workers +++
Opening connections to workers: OK (2 workers)                 
Setting up worker servers: OK (2 workers)                 

 *** Break *** write on a pipe with no one to read it
SysError in <TUnixSystem::UnixSend>: send (Broken pipe)

 *** Break *** write on a pipe with no one to read it
SysError in <TUnixSystem::UnixSend>: send (Broken pipe)
SysError in <TUnixSystem::DispatchOneEvent>: select: read error on 32
 (Bad file descriptor)
SysError in <TUnixSystem::DispatchOneEvent>: select: read error on 33
 (Bad file descriptor)

And root hangs…this is with the Debian machine…any clues?

Hi,
Well, I was not proposing you to use /opt, I was just explaining what I did in my test.
Of course, if you want to use that you have to:

  1. add /opt/root/bin in your PATH
       export PATH=/opt/root/bin:$PATH
   You can do that for all sessions in your $HOME/.bashrc or equivalent .
  1. add /opt/root/lib in your LD_LIBRARY_PATH or in /etc/ld.so.conf (or a dedicated entry under /etc/ld.so.conf.d). Again, you can do that for all sessions in your $HOME/.bashrc .

Btw: where you setting the LD_LIBRARY_PATH or /etc/ld.so.conf in your original attempt?

Gerri

I know. I actually wanted to use /opt initially because I feel the files are all contained in one place and easier to manage in case I want to install a new version.

I believe I did. I put a file named rootlib.conf in the /etc/ld.so.conf.d/ folder. With the following contents:

/opt/root/lib/root

And then ran ldconfig as superuser.

Thank you!. I was missing this. I used that line in a dedicated script in /etc/profiles.d/ and now I can execute root from any location.

Unfortunately PROOF-Lite still doesn’t work for me…

root [0] TProof::Open("");
 +++ Starting PROOF-Lite with 2 workers +++
/home/candres/.proof//session-phonon-1280843151-2094/worker-0.0.env: 1: Syntax error: "(" unexpected
Error in <TSlaveLite::Init>: an error occured while executing 'proofserv'
/home/candres/.proof//session-phonon-1280843151-2094/worker-0.1.env: 1: Syntax error: "(" unexpected
Error in <TSlaveLite::Init>: an error occured while executing 'proofserv'
Opening connections to workers: OK (2 workers)                 
PROOF set to sequential mode

The fact that it worked well in Ubuntu 10.04 is puzzling. Since this distro is based on precisely Debian Squeeze (testing)… I don’t know what to do…

Hi,

I think I have understood the problem.
I managed to reproduce it with CentOS on a VM.
I do not know why it did not show up on other systems.

Anyhow, the fix is in the trunk and attached is the patch including it for vanilla 5.26/00c .
You should take the source tarball for 5.26/00c (or the tag form SVN) and then apply it by untarring the patch tarball:

5-26-00c/root ~$ tar xzvf mod-patch-prefix-526c.tgz

Then you should reconfigure and rebuild.
Please, start from scratch again, even if it takes longer: just to rule out all possible side effects.

I hope this time it works better. If it does I will include the patch in 5.26/00-patches so that it will appear in 5.26/00d.

Gerri
mod-patch-prefix-526c.tgz (60 KB)

Oh wow! thank you so much! it works!

In the debian machine:

root [0] TProof::Open("");
 +++ Starting PROOF-Lite with 2 workers +++
Opening connections to workers: OK (2 workers)                 
Setting up worker servers: OK (2 workers)                 
PROOF set to parallel mode (2 workers)

In the CentOS machine:

root [0] TProof::Open("");
 +++ Starting PROOF-Lite with 8 workers +++
Opening connections to workers: OK (8 workers)                 
Setting up worker servers: OK (8 workers)                 
PROOF set to parallel mode (8 workers)

Success!

I have one naive question though. If I run my script in parallel mode, will it run faster without any modifications? or do I need to learn how to actually code for parallel processing?

Thanks again

Very good.
Thanks for the feedback.
The patch is now in 5.26/00-patches so that 5.26/00d will include it.

Gerri