How to parallel process a macro

ROOT
1

_ROOT Version:_6.17/01
_Platform:_linuxx8664gcc
Compiler: gcc version 8.3.1

Dear Experts,

I have a macro (attached) which I execute as follows:

.x gen_SparseEDTD_CS.cxx++

and it works. But takes ages!

I have a workstation with 24 cores and I want to utilize power of parallel processing.

Can you please guide me how to do this?

Thanking you in advance.

Regards,

Ajay

gen_SparseEDTD_CS.cxx (5.1 KB)

2

Hi,
one of the simplest ways to get multi-threading “for free” is to rewrite your analysis to use RDataFrame instead of TChain directly.
It requires a bit of mental gymnastic to convert imperative code (for loops and ifs) to RDataFrame’s declarative interface, but it usually works out for the better.

There is a crash course at the link above and tutorials here with some simple example analyses (e.g. df102_NanoAODDimuonAnalysis.C)

Cheers,
Enrico

3

Thank you very much for the prompt response, much appreciated. I had earlier tried to use RDataFrame but couldn’t really understand much, and yes, also due to time constraints.

What I am looking is to achieve parallel processing using TSelector class for the attached macro.

4

I guess the easiest would be to run many independent ROOT jobs, e.g. one job per “input file”. They should then create many “output files”, which you can simply merge in the end, running yet another ROOT job (or “hadd”).

For the management of these “Local” jobs, you could try to use a Task (with “t.float = 24”, i.e. as many cores as you have available) in Ganga, for example.

Or you could simply divide the total number of your “input files” by 24 (i.e. as many cores as you have available) and then run 24 independent ROOT jobs (one job per each 1/24-th sample of files) in parallel and finally merge the 24 “output files” (no need for any additional tool which manages these jobs).

2 Likes
5

Thank you for your response. What you have suggested seems to be very simple.

In the meantime, I have succeeded in writing a macro using TSelector Class. It works fine. But crashes while processing it with PROOF. You can find details of this issue here:
PROOF Crashes while Writing Histogram

6

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.