Root parellisation with OpenMP or : How I Learned to Start Worrying and Fear the Threads

ROOT
1

Hi Rooters,

I am probably doing this backwards but here goes.

I went on a quest to try to use OpenMP compiling with gcc version 4.9.3 with and linking the root libraries (Probably barbaric and someone will probably quickly point out that I am doing it wrong for what I want to do) trying to parallelise few nested loops containing a Tree->Draw() with a TCut, in essence I have these lines in a nested for loop.

I first start to load the MyTree in a non parallel block then try to clone MyTree in the first instance of the parallel block using a #pragma omp critical block to clone it.

MyTree->Draw("SomeVariable >> htemp_MyTree", OptiCut); TH1F *htemp_MyTree = (TH1F*)gPad->GetPrimitive("htemp_MyTree"); ResultMyTree = htemp_MyTree->GetEntries() * htemp_MyTree->GetMean();

The issue is, even if I try to create multiple “independent” copies of the original MyTree into memory by cloning it locally (I suppose here it does what I expect it to do, probably it’s where it fails miserably) I find myself with apparent non thread safeness, maybe due to the structure of the Tree itself because it somehow still knows where it comes from and also maybe eventually the pad itself is non thread safe. Though honestly on this one I am clueless.

I write the results to an output TTree but using the #pragma omp critical for the Write() seems to work decently well (each thread creates its own tree into the output TFile, probably someone would propose a merge of the independent TTrees and if so, even if that’s not the main concern, I would still be grateful for any additional information).

Here are my “simple” questions :

Is it possible to parallelise nested for loops on a Tree->Draw(“SomeVariable >> htemp_Tree”,TCut)?
If so how should I do it or where should I look for it, is there any example anywhere?
Is there a way at the end of the Multi-threading to merge the TTrees before writing them to my TFile?

I am aware of the existence of TThread but I am 100% sure I didn’t understand exactly how to deal with these and more especially how I would do to collapse a nested for loop.

Thanks a lot to anyone taking time to read this and eventually able to help.

Cheers,
Geoffrey.

2

What about using a TSelector and then PROOF (lite) for parallelisation?

root.cern.ch/using-tselector-proof
root.cern.ch/root/html/tutorial … roc.C.html

3

The ROOT code development team is active woking in providing multi-threaded access to TTrees as transparent as possible. The task is not simple as you have experienced already since several parts need to be re-worked to have thread safety. We are currently favoring a task-oriented programming model (like the one provided by TBB) to implemented it instead of OpenMP. This is still work in progress and we are open to suggestions and possible collaborations.
Meanwhile you can give a try to the old PROOF-lite or to the new multiproc package that will be basis for the new PROOF. This has the advantage that you do not need to deal with threading issues but obviously the performance is not as good because there is a some overhead involved. Examples can be found in root.cern.ch/doc/master/group__ … icore.html
Take a look at mp102_readNtuplesFillHistosAndFit.C example as good stating point.

Pere

4

Besides the good answer from Pere and ferhue. A technical point:

[quote]I am aware of the existence of TThread but I am 100% sure I didn’t understand exactly how to deal with these and more especially how I would do to collapse a nested for loop. [/quote]For any release before v6.06, it was a requirement for ROOT to be useable with multiple thread to create a TThread per thread, which you might be able to create at the same point as you clone the TTree. With v6.06 and later, all you need to do is enable the thread safety feature of ROOT by calling:

[quote]The issue is, even if I try to create multiple “independent” copies of the original MyTree into memory by cloning[/quote]The simplest/safest would actually be to clone the TFile (when cloning the TTree they still share in a non-thread safe way the same TFile instance).

Cheers,
Philippe.

5

So

1280×720

I come back around and will take a swing back at this issue, thank you all for your helpful answers and after a bit of fiddling I’ll let you know if it worked.

Thanks!