Inconsistencies with TMVA model evaluation in pyROOT when using multithreading

TMVA
, , ,
1

Hello,

We are currently working on applying a TMVA BDT model to an RDataFrame in pyROOT (ROOT version: 6.28/00). Our goal is to use the .weights.xml file for the evaluation. However, during our analysis, we encountered some inconsistencies. Here’s a snippet of the code we used for the evaluation:

ROOT.gInterpreter.ProcessLine('''
TMVA::Experimental::RReader model("weights/TMVARegression_BDTG.weights.xml");
computeModel = TMVA::Experimental::Compute<14, float>(model);
''')

df = df.Define("predicted_val", ROOT.computeModel, list(ROOT.model.GetVariableNames()))

Initially, we verified the predicted values by plotting them alongside the theoretical values in the same graph, and they appeared to be coherent with the expected values. However, we also plotted the difference between the predicted and theoretical values to compute the error, and the resulting errors were unexpectedly high and inconsistent. This led us to suspect that the column of predicted values might have been shuffled and placed in the wrong rows of the RDataFrame.

Furthermore, we noticed that every time we ran the evaluation, the computed values, means, and standard deviations were (slightly) different.

After some investigation, we found that the issue could be related to multithreading. We had enabled multithreading using ROOT.ROOT.EnableImplicitMT(), and when we disabled it, the evaluation started to work as expected. Therefore, our conclusion at this point is that TMVA might not be multithread-safe, leading to potential race conditions during evaluation.

To address this, we would like to ask if there is a multithread-safe approach in pyROOT to use TMVA for evaluation.

Thanks and looking forward to hear from you.

Cheers,

Martí

2

Hello @pdmonte ,

with multi-threading activated the output will be shuffled w.r.t. the input entries, see ROOT: ROOT::RDataFrame Class Reference . I guess that is what you are seeing. If you want to associate a Define’s output with the entry it came from you have to output some event number indicator with it (but not rdfentry_, because that will not correspond to input event numbers either).

Cheers,
Enrico