I have a large root file, but when using pyroot, the for loop is too slow, is there any good way to deal with it?

[Esword618]

I have a large root file, but when using pyroot, the for loop is too slow, is there any good way to deal with it?

[bellenot]

RDataFrame in python loop

Dear, I heard that RDataFrame is much efficient than classic loops, but I’m not sure how to use it for a simple code as in python: chain = ROOT.TChain() chain.Add(Dataset) h = ROOT.TH1D(“h”,"", n, x_min, x_max) lvmu1=ROOT.TLorentzVector() lvmu2 =ROOT.TLorentzVector() for ev in chain: lvmu1.SetPtEtaPhiE(ev.pt_muon[0], ev.eta_muon[0], ev.phi_muon[0], ev.E_muon[0]) lvmu2.SetPtEtaPhiE(ev.pt_muon[1], ev.eta_muon[1], ev.phi_muon[1], ev.E_muon[1]) lvmu3 = lvmu1+ lvmu2 h.Fill(lvmu3.Pt())

[Esword618]

I don’t know. Who can help us? A sample code would be good.

[bellenot]

If you didn’t find anything useful on this Forum, maybe @vpadulan can help

[Esword618]

@vpadulan Can you help me?

[vpadulan]

Dear @Esword618 ,

Thanks for reaching out to the forum!

A Python for loop is slow and there’s really no way around this. Your best chance to get good performance is to move the loop (and heavy computations) to C or C++, that’s what practically all scientific Python libraries do.

Thus, I suggest you move away from running a Python for loop over the entries of your TTree as soon as possible. ROOT offers since a long while a much superior alternative, RDataFrame. You can find many usage tutorials at ROOT: Dataframe tutorials .

Feel free to post here any example code you already have with your existing loop over the entries of the TTree. We then might start working together on a path towards using RDataFrame.

Cheers,
Vincenzo

[Esword618]

@vpadulan However, I need to use the for loop to judge and filter the data in TTree, but RDataFrame cannot meet my requirements.

[vpadulan]

Dear @Esword618 ,

Generally speaking, RDataFrame can be used to judge and filter data of a TTree. Could you be more explicit as to which requirements do not fit in RDataFrame? Maybe a code example of what you are trying to do with TTree?

Cheers,
Vincenzo

[Esword618]

Dear @vpadulan
But TTree’s python loop is too slow, so there is no way, can only use C++ to handle it?

[vpadulan]

Dear @Esword618 ,

Yes, the loop over TTree entries in Python is slow, that is clear. I would like to help you speed it up, and the best way we can do that is using RDataFrame (in Python!) so that we remove the for loop.

Now, I believe you already have some code of your own that is running some kind of analysis. It would help quite a lot if you could share a snippet of code that you already have right now. From that snippet, I could show you how to move your analysis (in Python) to use RDataFrame (still in Python).

Cheers,
Vincenzo

[Esword618]

Dear @vpadulan ,
Below is the root file I’m processing with pyroot, I want to process mu, a TTree, and get the elements in the branches that satisfy my conditions through some judgment conditions, how do I facilitate the elements through the RDataFrame, because this judgment condition is more complicated and I have to use a for loop.