Machine Learning on a Large Dataset using PyTorch: RDataFrame vs. uproot?

ROOT
,
1

Is there a general opinion on whether it is more efficient to load data via RDataFrames or uproot directly from ROOT files for machine learning analysis in PyTorch? I have seen discussion on this in multiple areas but most are fairly old and I am wondering if anything has changed in the past 2/3 years.

For my purposes specifically, I believe I will not be doing much fiddling around with the data itself and would like to avoid intermediate files assuming no loss in performance (are intermediate file methods much faster?). I just simply want to get the data from the ROOT files and into a PyTorch readable format in the most efficient manner possible.

Any links to external resources and comments are appreciated.

Thanks.

Older relevant post: python - What the fastest, most memory-efficient way of opening a ROOT NTuple for machine learning? - Stack Overflow

2

Hi @Sam_Kelson,

Welcome to the ROOT forum! I’m inviting @moneta and @vpadulan to the topic, as I think they can shed some light on this.

Cheers,
J.

3

Hi Sam,

A simple way, if your dataset is not too large is to use, as pointed in the link above, RDataFrame.AsNumpy.
If this is not convenient for your data, we are developing also a batch generator using RDataFrame which will provide efficiently batches of numpy arrays that can be used for ML tools like PyTorch or Tensorflow with minimal overhead. If you are interested we can provide you a prototype implementation for this

Best regards

Lorenzo

4

Hi Lorenzo,

That batch generator sounds really interesting. A prototype implementation would be lovely if you could provide that for me.

Thanks!

Sam

6

Dear @Sam_Kelson ,

I am reviving this old conversation because now ROOT provides an interface for native data loading of ROOT data to popular ML tools, including PyTorch. This is available since ROOT 6.40 as ROOT.Experimental.ML.RDataLoader, you can find an example with PyTorch at ROOT: tutorials/machine_learning/ml_dataloader_PyTorch.py File Reference . For more details, I also encourage you to take a look at this recent public presentation 28th Conference on Computing in High Energy and Nuclear Physics (CHEP 2026) (25-29 May 2026): Optimizations and New Strategies for Native ROOT Data Loading for ML · Indico

In case this is still relevant to you, we would welcome your feedback and we would also be available to help you in integrating the interface into your workflow.

Cheers,
Vincenzo