PROOF Tree Merging

ROOT
1

Hello,

I’m trying to use PROOF to generate TTree’s on the slave nodes and then merge the TTrees to create one output file.

My understanding is that the best way to do this, given the large size of the Tree’s, is to create the Tree’s on disk and have PROOF merge the files.

My question – Is there any example of how this is done? It seems like this TProofOutputFile class is used, but I can’t find one example of how you use this class. Can someone point me to the documentation on this? (or even an email thread?)

I have my code working with memory resident tree’s, but it breaks very quickly when the size of the output grows, so I would really like to get this file merging method to work.

Thanks for the help.

2

I figured it out from trial and error. If anyone is curious for future reference, the way it works is:

p = TProofOutputFile(“myout.root”)
GetOutputList()->Add§
f= p->OpenFile()
… (write objects to f)
f.Close()

Really easy and intuitive, actually. It works quite nicely.

3

Dear Rob,

Since you have found a solution by yourself you’ll probably not need this anymore, but there was some documentation available on the new ROOT web site at
root.cern.ch/drupal/content/hand … root-files .

We need to add links in the proper places.

G. Ganis

4

Hello,

I experienced one issue for my analysis while running on PROOF. I have an input tree, let’s call it InputTree, and I create an output tree, let’s call it FriendOutputTree, which should be the friend to InputTree. When running without PROOF everything works as it should. When running on PROOF, it distributes the events (randomly?) of InputTree among the workers, analyze the events and fills FriendOutputTree (actually, separately on each worker with TProofOutputFile, and then merges). At this point the event order is broken and the two trees can not be used as friends anymore. Is there a possibility to preserve the order in the output tree or any advice how to do this manually in an elegant way? Maybe either of the following solutions would fix this issue:

  1. Specify the events(or event order) to be sent to each particular worker, and also the merge order in the end.
  2. Friendship with particular event correspondence?

Thanks,
Alex.

5

Hi,

PROOF does not assign events randomly but in packets. But the order of packets can not be chosen, so in the end the net effect is a somewhat random order.
Ensuring the event order in PROOF is very hard and I am not even sure that there is a way to do that without completely changing the system.
But I see your issue with the creation of a tree friend, where the same original order is mandatory.

The only possible solution that comes to my mind is that you use a special TSelector together with TPacketizerFile to steer processing by file (not by event): the entries given to Process would be each of the files for which you want to create friends, and Process would open the file, loop through the events to create the tree friend, and save the friend in some place. In this way you will have as output a set of files with the friend , with the required file-to-file relation.
If you are interested in trying this way I may try to send you an example.

G. Ganis