Problems between TPySelector and PROOF

Hi all,

I’ve created a TPySelector class, called Selec.py, to analyze a TTree file, “mytree.root”. I want to do this using PROOF (since a main file, test.py, attached below) but It doesn’t work, I suppose that the master doesn’t send the data to the workers. This is the output when you execute the main file:

[code] user$ python test.py
+++ Starting PROOF-Lite with 8 workers +++
Opening connections to workers: OK (8 workers)
Setting up worker servers: OK (8 workers)
PROOF set to parallel mode (8 workers)

Info in TProofLite::SetQueryRunning: starting query: 1
Info in TProofQueryResult::SetRunning: nwrks: 8
Begin
<ROOT.TTree* object at 0x2dadb10>
Info in TPySelector::Begin: ------------------------------
Looking up for exact location of files: OK (1 files)
Looking up for exact location of files: OK (1 files)
Info in TPacketizer::TPacketizer: Initial number of workers: 8
14:43:40 3026 Wrk-0.0 | Error in TFile::TFile: file mytree.root does not exist
14:43:40 3026 Wrk-0.0 | SysError in TDSet::GetEntries: cannot open file mytree.root (type: 0, pfx: ) (No such file or directory)
Error in TPacketizer::ValidateFiles: cannot get entries for mytree.root (
Info in TPacketizer::TPacketizer: no valid or non-empty file found: setting invalid
Error in TProofPlayerLite::InitPacketizer: instantiated packetizer object ‘TPacketizer’ is invalid
Error in TProofPlayerLite::Process: cannot init the packetizer
Info in TProofLite::UpdateDialog: processing was aborted - 0 events processed
2.1965110302 seconds
[/code]

Thanks a lot.

test.py (1.42 KB)
Selec.py (8.16 KB)

Dear aidaph,

Can you try with the full path of the file, i.e. /a/b/c/d/mytree.root ?

G Ganis

Thanks, this error fixed…

But, now, the problem is that the workers seem to not begin, only print “begin” in the master, could be that the slaves don’t get the ttree?

Begin <ROOT.TTree* object at 0x37efb20> Info in <TPySelector::Begin>: ------------------------------ Looking up for exact location of files: OK (1 files) Looking up for exact location of files: OK (1 files) Info in <TPacketizer::TPacketizer>: Initial number of workers: 8 Validating files: OK (1 files) Terminate ging output objects ... / (1 workers still sending) Lite-0: all output objects have been merged 2.74188089371 seconds

Dear aidaph,

There are two ‘unusual’ aspects in your selector:

  1. The histograms are never added to the output list (self.fOutput) ;
  2. The histograms are saved for each call of Process to a file; this means that you’ll have up to 8 files at the end, recreated for each entry with a huge overhead, and which will never get merged.

Could you please try by:

  1. Adding the histograms to the output list at latest in SlaveTerminate; you can do that in declareHisto, for example
  2. Remove the recreation and saving of the histograms in Process; you should then get them back, merged, in the output list after processing (access via TProof.GetOutputList()); you can also save them, merged, automatically to a file (see root.cern.ch/handling-outputs):
chain.Process('TPySelector', 'Selec', 'of=histos.root')

G Ganis

Thanks so much for your reply!! I add the outputlist but i’m still getting problems. Sorry, I’m newer in ROOT and proof…
I’ve tried to run with a TSelector instead of the initial TPySelector class and it works perfectly. But running with TPySelector gives me an “event Skipped” when it starts the slaves work, getting the same output than i display in the previous post(Only running begin and terminate function in the master and it finishes in ~3 seconds). If it would help, my ROOT version is 5.34/34.

Cheers!

Does it work without PROOF, i.e. just chain.Process without chain.SetProof ?
I have no experience in running in python, I would need to try …

G Ganis

Yes, without the chain.SetProof, it works fine. I don’t understand why it reads the tree entries so quickly while my read class without TPySelector runs for almost 10 minutes…

i have to keep developing the code…but thank you so much for your help!!

[quote=“ganis”]Does it work without PROOF, i.e. just chain.Process without chain.SetProof ?
I have no experience in running in python, I would need to try …

G Ganis[/quote]