Problem with tutorial ProofNtuple.C

ROOT
1

I want to filter an TTree to save in a new root file only events that pass a particular selection. I’m new in the manage of TFile, TProofOutputFile with Proof and I’m experiencing a problem with my code and I’m able to reproduce it with the tutorial ProofNtuple.C. Our proof cluster consist of 3 nodes: t2-wn-9, t2-wn-10, t2-wn-11. I’m logged into t2-wn-11.

.x $ROOTSYS/tutorials/proof/runProof.C+("ntuple(nevt=10000000)", "localhost")

Warning in <ACLiC>: /gpfs/storage_4/users/home/proof/root/tutorials/proof is not writeable!
Warning in <ACLiC>: Output will be written to /tmp/turra
Starting master: opening connection ...
Starting master: OK                                                 
Opening connections to workers: OK (24 workers)                 
Setting up worker servers: OK (24 workers)                 
PROOF set to parallel mode (24 workers)
getProof: WARNING: started/attached a session on external cluster (localhost): 'dir="/tmp/turra/.proof-tutorial"' ignored
runProof: ntuple: ACLiC mode: '+'

runProof: running "ntuple" with nevt= 10000000

Warning in <ACLiC>: /gpfs/storage_4/users/home/proof/root/tutorials/proof is not writeable!
Warning in <ACLiC>: Output will be written to /tmp/turra
TProofOutputFile::AddFile: error from TFileMerger::AddFile(root://t2-wn-09.mi.infn.it//proof/workingdirs/turra/session-t2-wn-11-1282728816-12862/worker-0.3-t2-wn-09-1282728819-14123//SimpleNtuple.root)
TProofOutputFile::AddFile: error from TFileMerger::AddFile(root://t2-wn-09.mi.infn.it//proof/workingdirs/turra/session-t2-wn-11-1282728816-12862/worker-0.21-t2-wn-09-1282728821-14130//SimpleNtuple.root)
TProofOutputFile::AddFile: error from TFileMerger::AddFile(root://t2-wn-09.mi.infn.it//proof/workingdirs/turra/session-t2-wn-11-1282728816-12862/worker-0.15-t2-wn-09-1282728821-14132//SimpleNtuple.root)
TProofOutputFile::AddFile: error from TFileMerger::AddFile(root://t2-wn-09.mi.infn.it//proof/workingdirs/turra/session-t2-wn-11-1282728816-12862/worker-0.12-t2-wn-09-1282728818-14118//SimpleNtuple.root)
TProofOutputFile::AddFile: error from TFileMerger::AddFile(root://t2-wn-09.mi.infn.it//proof/workingdirs/turra/session-t2-wn-11-1282728816-12862/worker-0.6-t2-wn-09-1282728820-14125//SimpleNtuple.root)
TProofOutputFile::AddFile: error from TFileMerger::AddFile(root://t2-wn-09.mi.infn.it//proof/workingdirs/turra/session-t2-wn-11-1282728816-12862/worker-0.0-t2-wn-09-1282728818-14120//SimpleNtuple.root)
TProofOutputFile::AddFile: error from TFileMerger::AddFile(root://t2-wn-09.mi.infn.it//proof/workingdirs/turra/session-t2-wn-11-1282728816-12862/worker-0.9-t2-wn-09-1282728821-14128//SimpleNtuple.root)
TProofOutputFile::AddFile: error from TFileMerger::AddFile(root://t2-wn-09.mi.infn.it//proof/workingdirs/turra/session-t2-wn-11-1282728816-12862/worker-0.18-t2-wn-09-1282728823-14134//SimpleNtuple.root)
TProofOutputFile::AddFile: error from TFileMerger::AddFile(root://t2-wn-10.mi.infn.it//proof/workingdirs/turra/session-t2-wn-11-1282728816-12862/worker-0.4-t2-wn-10-1282728820-14049//SimpleNtuple.root)
TProofOutputFile::AddFile: error from TFileMerger::AddFile(root://t2-wn-10.mi.infn.it//proof/workingdirs/turra/session-t2-wn-11-1282728816-12862/worker-0.1-t2-wn-10-1282728819-14047//SimpleNtuple.root)
TProofOutputFile::AddFile: error from TFileMerger::AddFile(root://t2-wn-10.mi.infn.it//proof/workingdirs/turra/session-t2-wn-11-1282728816-12862/worker-0.10-t2-wn-10-1282728822-14056//SimpleNtuple.root)
TProofOutputFile::AddFile: error from TFileMerger::AddFile(root://t2-wn-10.mi.infn.it//proof/workingdirs/turra/session-t2-wn-11-1282728816-12862/worker-0.19-t2-wn-10-1282728821-14054//SimpleNtuple.root)
TProofOutputFile::AddFile: error from TFileMerger::AddFile(root://t2-wn-10.mi.infn.it//proof/workingdirs/turra/session-t2-wn-11-1282728816-12862/worker-0.16-t2-wn-10-1282728819-14045//SimpleNtuple.root)
TProofOutputFile::AddFile: error from TFileMerger::AddFile(root://t2-wn-10.mi.infn.it//proof/workingdirs/turra/session-t2-wn-11-1282728816-12862/worker-0.22-t2-wn-10-1282728818-14042//SimpleNtuple.root)
TProofOutputFile::AddFile: error from TFileMerger::AddFile(root://t2-wn-10.mi.infn.it//proof/workingdirs/turra/session-t2-wn-11-1282728816-12862/worker-0.7-t2-wn-10-1282728820-14051//SimpleNtuple.root)
TProofOutputFile::AddFile: error from TFileMerger::AddFile(root://t2-wn-10.mi.infn.it//proof/workingdirs/turra/session-t2-wn-11-1282728816-12862/worker-0.13-t2-wn-10-1282728816-14040//SimpleNtuple.root)
Mst-0: merging output objects ... done                                     
Output file: /gpfs/storage_4/users/home/turra/ProofNtuple.root   
Mst-0: grand total: sent 9 objects, size: 634564 bytes       
outputFile: /gpfs/storage_4/users/home/turra/ProofNtuple.root
Managed to open file: /gpfs/storage_4/users/home/turra/ProofNtuple.root

At the end the file is merged but with only 1802988 events instead of 10000000. If you look at the warning the problem is that the client t2-wn-11 is unable to find files on the other workers.

With prooflite I don’t have the problem

2

Hi,

Is there a daemon (rootd or xrootd) serving the files from the workers?
I mean, via which protocol do you expect the master (t2-wn-11) to access files on the workers?

Gerri

3

[quote=“ganis”]Hi,

Is there a daemon (rootd or xrootd) serving the files from the workers?
I mean, via which protocol do you expect the master (t2-wn-11) to access files on the workers?

Gerri[/quote]

I’m not the administrator, and I’m only a user of the cluster, but:

[turra@t2-wn-11]~% ps -All|grep xroot
1 S 11547 19381     1  0  75   0 - 52044 stext  ?        00:00:22 xrootd
4

Hi,
The problem is that the tutorials are not general enough and assume that you can access the files via xrootd.
Uhm … I do not have a simple enough work-around to propose. I need to change the tutorials. And also probably to make TProofOutputFile more flexible.

Is it possible for you to have the proof sandboxes (working directories) under GPFS?

If yes, we can then probably workout a solution (for your final problem, not for the tutorial …) with the master getting the files via GPFS (/gpfs/…).

Gerri

5

[quote=“ganis”]Hi,
The problem is that the tutorials are not general enough and assume that you can access the files via xrootd.
Uhm … I do not have a simple enough work-around to propose. I need to change the tutorials. And also probably to make TProofOutputFile more flexible.

Is it possible for you to have the proof sandboxes (working directories) under GPFS?

If yes, we can then probably workout a solution (for your final problem, not for the tutorial …) with the master getting the files via GPFS (/gpfs/…).

Gerri[/quote]

I’ve never used sandboxes, but I think yes:

root [11] mgr->Cp("http://root.cern.ch/files/h1/dstarmb.root","~/cache/")
2010-08-26 14:20:28 (1.69 MB/s) - `/proof/workingdirs/turra/cache/dstarmb.root' saved [21330730/21330730]
root [13] mgr->Ls()
Node: t2-wn-11.mi.infn.it:1093
-----
cache				  session-t2-wn-11-1282727958-11437
data				  session-t2-wn-11-1282728001-11649
datasets			  session-t2-wn-11-1282728480-12277
last-master-session		  session-t2-wn-11-1282728648-12446
last-worker-session		  session-t2-wn-11-1282728776-12687
packages			  session-t2-wn-11-1282728816-12862
queries				  session-t2-wn-11-1282729755-13212
session-t2-wn-11-1282724611-9715  session-t2-wn-11-1282729802-13288
session-t2-wn-11-1282724747-9811  session-t2-wn-11-1282824955-28317

root [14] mgr->Ls("cache")
Node: t2-wn-11.mi.infn.it:1093
-----
bruteforce.C	      dump_simple.C	selection_C.so
bruteforce_C.d	      dump_simple_C.d	selection.h
bruteforce_C.so       dump_simple_C.so	selector_purity.C
bruteforce.h	      dump_simple.h	selector_purity_C.d
check_variables.C     Filter.C		selector_purity_C.so
check_variables_C.d   Filter.h		selector_purity.h
check_variables_C.so  ProofNtuple.C	Strip.C
check_variables.h     ProofNtuple_C.d	Strip.h
cuts.C		      ProofNtuple_C.so	TProofCollection.C
dstarmb.root	      ProofNtuple.h	TProofCollection_C.d
dump.C		      selection.C	TProofCollection_C.so
dump.h		      selection_C.d	TProofCollection.h

Why in your opinion I’m not able to access files via xrootd? Is it a configuration problem? Can you be more specific?

6

Because xrootd is not running on the machines. No error, configuration problem. Just that another way (gpfs?) is used to access the files.
But the real problem is that ProofNtuple.C assumes that accessing files via xrootd is possible. This is to retstrictive and needs to be fixed.

Gerri

7

as I said xrootd is working on the machines:

ps -ALL|grep xrootd
19381 19381 ?        00:00:00 xrootd
19381 19382 ?        00:00:00 xrootd
19381 19383 ?        00:00:00 xrootd
19381 19386 ?        00:00:00 xrootd
19381 19387 ?        00:00:00 xrootd
19381 19388 ?        00:00:00 xrootd
19381 19389 ?        00:00:00 xrootd
19381 19390 ?        00:00:00 xrootd
19381 19392 ?        00:00:00 xrootd
19381 19398 ?        00:00:00 xrootd
19381 19399 ?        00:00:28 xrootd
19381 19400 ?        00:00:00 xrootd
19381 19401 ?        00:00:00 xrootd
19381 19402 ?        00:00:00 xrootd
19381 19407 ?        00:00:00 xrootd
19381 19415 ?        00:00:00 xrootd
19381 19419 ?        00:00:00 xrootd
19381 19436 ?        00:00:00 xrootd
19381 19439 ?        00:00:00 xrootd
19381 19440 ?        00:00:00 xrootd
19381 22088 ?        00:00:00 xrootd
19381 22091 ?        00:00:00 xrootd
8

Ok, I missed that.
Well, in xrootd one has to explicitly export the paths visible to outside.
In this case you need to export the proofboxes, i.e. /proof/workingdirs/…; in this sense, this is a ‘xrootd configuration’ problem.
The xrootd directive is

all.export /proof/workingdirs r/o

Perhaps you can tell your administrator to add that to the xrootd config file?

Gerri