Remove directory

archiron · November 23, 2021, 2:25pm

the lote.sh file with all root files.

When I type root and then TAB in order to see all the root* commands, I got :

root         rootcint     rootcling    root-config  root.exe     rootinfo     rootn.exe    rootplot     rootplotmpl  roots        roots.exe

With the cmsrel/cmsenv commands as in my initial msg, I got :

root               root2sqlite.py     rootcint           rootdrawtree       rootls             rootmv             rootplot           rootrm             rootslimtree       
root-config        root2yoda          rootcling          rooteventselector  rootmath           rootn.exe          rootplotmpl        roots              
root.exe           rootbrowse         rootcp             rootinfo           rootmkdir          rootnb.exe         rootprint          roots.exe

with the pbms as No module named cmdLineUtils

couet · November 23, 2021, 3:03pm

cmdLineUtils is part of the python root modules . I am not sure why this is not found by rootcp. May be @etejedor has an idea.

etejedor · November 23, 2021, 4:41pm

If there is an error when importing cmdLineUtils, it looks like the problem is that the executable can be found (rootcp) but the PYTHONPATH is not correctly set (rootcp is actually a Python script that imports cmdLineUtils). I would check that the environment is correctly configured. For example, can you import ROOT from Python?

archiron · November 24, 2021, 9:23am

hi, thanks.

Yes I can import ROOT (I think so) with the following example (named testROOT.py) :

from sys import argv
argv.append( '-b-' )
import ROOT
ROOT.gROOT.SetBatch(True)
ROOT.gErrorIgnoreLevel = ROOT.kFatal
argv.remove( '-b-' )

from ROOT import *

f_rel = TFile('DQM_V0001_R000000001__RelValZpToEE_m6000_14TeV__CMSSW_12_1_0_pre4-PU_121X_mcRun3_2021_realistic_v10-v1__DQMIO.root', "UPDATE")
t2 = f_rel.Get("DQMData/Run 1")
print(t2.ls())
f_rel.Close()

it works with python testROOT.py 0 or python3 testROOT.py 0

etejedor · November 24, 2021, 11:02am

And if you do import cmdLineUtils from that script, what happens? If you can import ROOT, you should be able to import cmdLineUtils too.

archiron · November 24, 2021, 11:19am

importing cmdLineUtils with : python testROOT.py 0
gives :

Traceback (most recent call last):
  File "testROOT.py", line 11, in <module>
    import cmdLineUtils
  File "/usr/lib64/python2.7/site-packages/ROOT/_facade.py", line 150, in _importhook
    return _orig_ihook(name, *args, **kwds)
ImportError: No module named cmdLineUtils

perhaps some mistake into my $PATH ?
here it is :

echo $PATH
/afs/cern.ch/cms/caf/scripts:/cvmfs/cms.cern.ch/common:/usr/sue/bin:/usr/lib64/qt-3.3/bin:/usr/condabin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin:/afs/cern.ch/user/a/archiron/bin:/afs/cern.ch/user/a/archiron/lbin:/afs/cern.ch/cms/Releases/OVAL/site:/cvmfs/cms.cern.ch/slc6_amd64_gcc700/external/bootstrap-bundle/1.7/lib/

and :

echo $PYTHONPATH

etejedor · November 24, 2021, 11:33am

I see, how did you install ROOT?

It looks like cmdLineUtils.py is not installed. Can you confirm that it is not in /usr/lib64/python2.7/site-packages/?

archiron · November 24, 2021, 11:46am

In fact I did not install ROOT. It came each time I connect me onto a lxplus machine.

when I try to find the file, I geet :

find /usr/lib64/python2.7/site-packages/ -type f -name '*cmd*.py'

/usr/lib64/python2.7/site-packages/twisted/test/process_cmdline.py
/usr/lib64/python2.7/site-packages/hgext/convert/convcmd.py
/usr/lib64/python2.7/site-packages/mercurial/cmdutil.py
/usr/lib64/python2.7/site-packages/bzrlib/cmd_test_script.py
/usr/lib64/python2.7/site-packages/bzrlib/cmdline.py
/usr/lib64/python2.7/site-packages/bzrlib/cmd_version_info.py
/usr/lib64/python2.7/site-packages/bzrlib/tests/test_cmdline.py

no cmdLineUtils and no cmdLineUtils on grep -r -i cmdLineUtils /usr/lib64/python2.7/site-packages/ command either.

etejedor · November 24, 2021, 12:34pm

Ok I see, if you are using the system installation of ROOT on lxplus, it does not have the ROOT command line tools installed (rootls etc.). What you can do is sourcing the ROOT of an LCG release, e.g. source /cvmfs/sft.cern.ch/lcg/views/LCG_100/x86_64-centos7-gcc8-opt/setup.sh. Then you’ll have access to the tools.

archiron · November 24, 2021, 1:36pm

Good !
it works fine … except for ROOT files up to CMSSW_10_6_20.
I tried with 11_3_0_pre1[pre4], 12_0_0_pre3 or 12_1_0_pre4 (see file list below) with a pbm :

Traceback (most recent call last):
  File "/cvmfs/sft.cern.ch/lcg/views/LCG_100/x86_64-centos7-gcc8-opt/bin/rootcp", line 58, in <module>
    sys.exit(execute())
  File "/cvmfs/sft.cern.ch/lcg/views/LCG_100/x86_64-centos7-gcc8-opt/bin/rootcp", line 54, in execute
    return cmdLineUtils.rootCp(sourceList, destFileName, destPathSplit, \
  File "/cvmfs/sft.cern.ch/lcg/views/LCG_100/x86_64-centos7-gcc8-opt/lib/cmdLineUtils.py", line 790, in rootCp
    retcode += _copyObjects(fileName, pathSplitList, destFile, destPathSplit, \
  File "/cvmfs/sft.cern.ch/lcg/views/LCG_100/x86_64-centos7-gcc8-opt/lib/cmdLineUtils.py", line 769, in _copyObjects
    retcode += copyRootObject(rootFile, pathSplit, destFile, destPathSplit, \
  File "/cvmfs/sft.cern.ch/lcg/views/LCG_100/x86_64-centos7-gcc8-opt/lib/cmdLineUtils.py", line 545, in copyRootObject
    retcode += copyRootObjectRecursive(sourceFile,sourcePathSplit, \
  File "/cvmfs/sft.cern.ch/lcg/views/LCG_100/x86_64-centos7-gcc8-opt/lib/cmdLineUtils.py", line 612, in copyRootObjectRecursive
    retcode +=copyRootObjectRecursive(sourceFile, \
  File "/cvmfs/sft.cern.ch/lcg/views/LCG_100/x86_64-centos7-gcc8-opt/lib/cmdLineUtils.py", line 612, in copyRootObjectRecursive
    retcode +=copyRootObjectRecursive(sourceFile, \
  File "/cvmfs/sft.cern.ch/lcg/views/LCG_100/x86_64-centos7-gcc8-opt/lib/cmdLineUtils.py", line 657, in copyRootObjectRecursive
    obj.SetName(objectName)
AttributeError: 'TObjString' object has no attribute 'SetName'

Is there a difference between those root files ? The used ROOT files are “official” files download from https://cmsweb.cern.ch/dqm/relval/data/browse/ROOT/RelVal/ and the list below are located on /afs/cern.ch/user/a/archiron/public/ROOT_SIZE

list of root files :

#SOURCE="DQM_V0001_R000000001__Global__CMSSW_X_Y_Z__RECO.root"
#SOURCE="DQM_V0001_R000000001__RelValZEE_13__CMSSW_10_6_20-106X_mc2017_realistic_v9_miniv2-v1__DQMIO.root"
#SOURCE="DQM_V0001_R000000001__RelValZEE_13__CMSSW_10_6_20-106X_mc2017_realistic_v9-v1__DQMIO.root"
#SOURCE="DQM_V0001_R000000001__RelValZEE_14__CMSSW_11_3_0_pre1-113X_mcRun4_realistic_v1_2026D49noPU-v1__DQMIO.root"
#SOURCE="DQM_V0001_R000000001__RelValZEE_14__CMSSW_11_3_0_pre4-113X_mcRun3_2021_realistic_v7-v1__DQMIO.root"
SOURCE="DQM_V0001_R000000001__RelValZEE_14__CMSSW_11_3_0_pre4-113X_mcRun4_realistic_v4_2026D76noPU-v1__DQMIO.root"
#SOURCE="DQM_V0001_R000000001__RelValZEE_14__CMSSW_12_0_0_pre3-120X_mcRun3_2021_realistic_v1-v1__DQMIO.root"
#SOURCE="DQM_V0001_R000000001__RelValZEE_14__CMSSW_12_1_0_pre4-121X_mcRun3_2021_realistic_v10-v1__DQMIO.root"

etejedor · November 24, 2021, 2:58pm

I think this might be a bug in rootcp, it seems it is trying to call SetName on a TObjString object, which does not have that method.

So to unblock you, I’d say you should do it with another tool. You can either try this in plain Python with PyROOT as you were doing or perhaps use RDataFrame’s Snapshot action, but that requires a bit of knowledge of RDataFrame: ROOT: ROOT::RDataFrame Class Reference

couet · November 25, 2021, 3:50pm

This PR fixes the problem.

archiron · November 25, 2021, 4:30pm

ok, thanks. I will test it when the PR will be done.
During time, I rewrite a script un full python to do this with a usefull use of the cmsLineUtils subroutine (changeDirectory).
It seems to work correctly but I need to do more tests.
Thanks to you.