PyROOT on the Grid (not with Athena)

Dear all,

I’d be very grateful if someone could help me with this. I’ve run PyROOT jobs on the Grid successfully under Athena, but I was wondering if it’s possible not to setup Athena.

At the moment, I send off a PyROOT job to the grid as follows:
prun --exec “source GridFiles/pyroot_setup.sh; python clusterAnalysis.py - %IN” --site=…

GridFiles/pyroot_setup.sh is sent with the job, and contains:

export PYTHONDIR=/afs/cern.ch/atlas/software/releases/16.3.0/sw/lcg/external/Python/2.6.5/i686-slc5-gcc43-opt
export ROOTSYS=/afs/cern.ch/atlas/software/releases/16.3.0/sw/lcg/app/releases/ROOT/5.26.00d_python2.6/i686-slc5-gcc43-opt/root
export PATH=${ROOTSYS}/bin:${PYTHONDIR}/bin:${PATH}
export LD_LIBRARY_PATH=${ROOTSYS}/lib:${PYTHONDIR}/lib:${LD_LIBRARY_PATH}
export PYTHONPATH=${ROOTSYS}/lib:${PYTHONPATH}

but the job fails when trying to find the ROOT module:

INFO: trying to wrap ROOT.py
Traceback (most recent call last):
File “clusterAnalysis.py”, line 7, in ?
from ROOT import TFile, TH1F, TH2F, TCanvas, gROOT, gPad, TChain, gStyle, TLegend, TPaveStats, TObject, TList
File “/home/pilatl02/globus-tmp.lcg-wn02-15.4087.0/condorg_MZpO4246/pilot3/Panda_Pilot_4282_1291926321/PandaJob_1160919131_1291926324/workDir/ROOT.py”, line 23, in ?
reload(ROOT)
ImportError: No module named ROOT

Many thanks in advance, if anyone can help!
Sarah


Sarah Livermore
D.Phil. student, ATLAS group
University of Oxford @ CERN


Hi Sarah,

you seem to have a mismatch in your libraries vs compiler.
When I try to execute your settings on lxplus and run root, I get:

code [139] root -l
/afs/cern.ch/atlas/software/releases/16.3.0/sw/lcg/app/releases/ROOT/5.26.00d_python2.6/i686-slc5-gcc43-opt/root/bin/root.exe: /usr/lib/libstdc++.so.6: versionGLIBCXX_3.4.9' not found (required by /afs/cern.ch/atlas/software/releases/16.3.0/sw/lcg/app/releases/ROOT/5.26.00d_python2.6/i686-slc5-gcc43-opt/root/lib/libCore.so) /afs/cern.ch/atlas/software/releases/16.3.0/sw/lcg/app/releases/ROOT/5.26.00d_python2.6/i686-slc5-gcc43-opt/root/bin/root.exe: /usr/lib/libstdc++.so.6: versionGLIBCXX_3.4.9’ not found (required by /afs/cern.ch/atlas/software/releases/16.3.0/sw/lcg/app/releases/ROOT/5.26.00d_python2.6/i686-slc5-gcc43-opt/root/lib/libCint.so)
/afs/cern.ch/atlas/software/releases/16.3.0/sw/lcg/app/releases/ROOT/5.26.00d_python2.6/i686-slc5-gcc43-opt/root/bin/root.exe: /usr/lib/libstdc++.so.6: versionGLIBCXX_3.4.9' not found (required by /afs/cern.ch/atlas/software/releases/16.3.0/sw/lcg/app/releases/ROOT/5.26.00d_python2.6/i686-slc5-gcc43-opt/root/lib/libMathCore.so) /afs/cern.ch/atlas/software/releases/16.3.0/sw/lcg/app/releases/ROOT/5.26.00d_python2.6/i686-slc5-gcc43-opt/root/bin/root.exe: /usr/lib/libstdc++.so.6: versionGLIBCXX_3.4.9’ not found (required by /afs/cern.ch/atlas/software/releases/16.3.0/sw/lcg/app/releases/ROOT/5.26.00d_python2.6/i686-slc5-gcc43-opt/root/lib/libRint.so)
[/code]

so, you should contact your Atlas colleagues for more precision.

Rene

Hi Rene
thanks very much for the message.
It seemed like the easiest thing to do was going back to using PyROOT under Athena, since I couldn’t find anyone locally to ask about this.

However, I’ve run into a problem that others have had when using 2D vectors with PyROOT under Athena.
(The whole output, when I try to run my PyROOT script, is at the end of my message, below)
A previous thread is here (Pointer to a 2D vector of integers),
In that case, to summarise, the problem there was solved by doing:
root [] gSystem->Load(“libCintex”);
Cintex::Enable();

Is there an equivalent PyROOT command for the Cintex::Enable()?
Hopefully if I put that at the top of my python script, all will be well!

Thanks again
Sarah

Full details of the current problem…
At the moment, my python script includes the line:
[size=85]gROOT.ProcessLine(’.L Loader.C+’)[/size]

and Loader.C contains:
[size=85]#include
#pragma link C++ class vector<vector >+;
#pragma link C++ class vector<vector >+;
#pragma link C++ class vector<vector >+;
#pragma link C++ class vector<vector >+; [/size]

This works fine using running not under Athena (ROOT 5.26)
But when I run the script under Athena (also ROOT 5.26), I get:
[size=85]Warning in TEnvRec::ChangeValue: duplicate entry <Library.vector=vector.dll> for level 0; ignored
Warning in TEnvRec::ChangeValue: duplicate entry <Library.vector=vector.dll> for level 0; ignored
Warning in TEnvRec::ChangeValue: duplicate entry <Library.vector=vector.dll> for level 0; ignored
Warning in TEnvRec::ChangeValue: duplicate entry <Library.vector=vector.dll> for level 0; ignored
Warning in TEnvRec::ChangeValue: duplicate entry <Library.vector=vector.dll> for level 0; ignored
Warning in TEnvRec::ChangeValue: duplicate entry <Library.vector=vector.dll> for level 0; ignored
Warning in TEnvRec::ChangeValue: duplicate entry <Library.vector=vector.dll> for level 0; ignored
Warning in TEnvRec::ChangeValue: duplicate entry <Library.vector=vector.dll> for level 0; ignored
Warning in TEnvRec::ChangeValue: duplicate entry <Library.vector=vector.dll> for level 0; ignored
Warning in TEnvRec::ChangeValue: duplicate entry <Library.vector=vector.dll> for level 0; ignored
Info in TUnixSystem::ACLiC: creating shared library /home/livermore/testareanew/16.0.2.6/ClusterAnalysis/./Loader_C.so
Note: Link requested for undefined class vector<vector > (ignore this message) :0:
Note: Link requested for undefined class vector<vector > (ignore this message) :0:
Note: Link requested for undefined class vector<vector > (ignore this message) :0:
Error: A dictionary has been requested for vector<vector > but there is no declaration!
Error: A dictionary has been requested for vector<vector > but there is no declaration!
Error: A dictionary has been requested for vector<vector > but there is no declaration!
[/size]

Hi Sarah,

I assume that Wim will follow now.

Rene

Hi,

first for the compiler, LCG uses a custom compiler which is here:

/afs/cern.ch/sw/lcg/external/gcc/4.3.2/x86_64-slc5/bin/gcc

and setting its environment will solve the linking symbol problem.

As for Cintex::Enable(), the following should work:import ROOT ROOT.gSystem.Load( "libCintex" ) ROOT.Cintex.Enable()
but more likely you simply want to do:import PyCintexeven as doing so has a few more further side-effects.

HTH,
Wim

Dear Wim,
many thanks for your reply and apologies for the late follow up.

Yes, now I just include the following line
import PyCintex

…and I submit my job with --athenaTag=AtlasProduction,16.0.2.6. I do not need to set the environment of the LCG complier.

All seems to run OK and I can access branches with 2d vectors in my PyRoot code.

Thank you very much again for your help
Sarah

Hi,
I know this post is super old, but I’m trying to also submit a set of python scripts to the grid and when i try to “import ROOT” the grid job fails.

for example, here is a job that failed bigpanda.cern.ch/task/27325906/
where the script I submitted a very minimal script that only included the following:
import ROOT
import sys
print sys.argv

When i submit the job with “import ROOT” it fails, but when i submit the job without “import ROOT” it does not fail.

When the job does fail, i’ve attached the payload.stdout file and the error is:— Running in /srv —
===== PFC from pilot =====
ERROR : Failed to collect GUIDs : [Errno 2] No such file or directory: ‘PoolFileCatalog.xml’

I was thinking maybe there is a configuration error as you mention. If this could be the reason, what do I need to do to get it working? what should I export?

Alternatively, would it be more recommended to use existing frameworks like athena or xAODAnaHelpers? again, all i want to submit to the grid are scripts and directories of python scripts to read in root files and output slimmed rootfiles.

Thanks in advance!
Cheers,
Christina

Hi,
I know this post is super old, but I’m trying to also submit a set of python scripts to the grid and when i try to “import ROOT” the grid job fails.

for example, here is a job that failed bigpanda.cern.ch/task/27325906/
where the script I submitted a very minimal script that only included the following:
import ROOT
import sys
print sys.argv

When i submit the job with “import ROOT” it fails, but when i submit the job without “import ROOT” it does not fail.

When the job does fail, the error is:
— Running in /srv —
===== PFC from pilot =====
ERROR : Failed to collect GUIDs : [Errno 2] No such file or directory: ‘PoolFileCatalog.xml’

I was thinking maybe there is a configuration error as you mention. If this could be the reason, what do I need to do to get it working? what should I export?

Alternatively, would it be more recommended to use existing frameworks like athena or xAODAnaHelpers? again, all i want to submit to the grid are scripts and directories of python scripts to read in root files and output slimmed rootfiles.

Thanks in advance!
Cheers,
Christina

Hi @nelsonc,

This error does not seem to relate to the issue you are describing when trying to import ROOT. Most likely, either the ROOT python package is not available, or any of its dependencies is failing to load.

You mentioned this appears in a file payload.stdout. Perhaps we could get more details from stderr (is there a payload.stderr file that we can look at?).

I am also inviting @etejedor to this topic, just in case he saw this before in this environment.

Cheers,
J.

Hi @jalopezg ,
Thanks for the response!
the payload.stderr file is actually empty.I took a closer look at payload.stdout and copied some of the error messages below.

I’m having problems attaching the entire stdout because it seems I can’t yet post links. But, the full file can be found here in lxplus: /afs/cern.ch/user/n/nelsonc/public/user.nelsonc.test.withROOTimport.log

/////////////////////////////////

===================
— Running in /srv —
===== PFC from pilot =====
ERROR : Failed to collect GUIDs : [Errno 2] No such file or directory: ‘PoolFileCatalog.xml’

=== stderr ===
Traceback (most recent call last):
File “purepython.py”, line 1, in
import ROOT
ImportError: No module named ROOT

==== Result ====
2021-11-11 20:44:37.430356
execute script: Running script failed : StatusCode=1

/////////////////////////////////

The only things I’m doing for set up are the following:
setupATLAS
voms-proxy-init -voms atlas
lsetup panda

I’ve tried in addition, to setup with lsetup “root 6.20.06-x86_64-centos7-gcc8-opt”, asetup AnalysisBase 21.2 latest, but nothing has worked so far.

Cheers,
Christina

Hi @jalopezg ,

I’ve found a solution to my problem. I’m able to submit to the grid and do what I want, ie. importing ROOT and using AnalysisBase functionality, with the following command:

prun --cmtConfig=x86_64-centos7-gcc8-opt --noBuild --useAthenaPackages --rootVer 6.20.06 --exec “python purepython.py %IN” --inDS user.nelsonc.
364171.STDM4_Wenu_Sherpa221_MC16a_p4239_364171-nGB4_211021_tree.root --outDS user.nelsonc.importROOT.test.SH.athenaPackages --nJobs 1 --nFiles 1

Thanks again for your help!

Hi @nelsonc; good to hear that! Could you please mark the topic as solved?

Cheers,
J.

Hi @jalopezg ,
I did not create the ticket, so I can’t seem to edit the title to mark as [SOLVED].

So, how do I mark it as solved?
Thanks!,
Cheers