How to evaluate XML file for MVA trained with PyTorch within RDataFrame?

TMVA
1

Hello,

Referring to this thread, we are loading .xml files generated from TMVA using RReader and RDataFrame for our analysis. While it works fine for BDT, we are having trouble loading a neural network model trained with PyTorch.

Basically, I followed this TMVA PyTorch tutorial to train our model and make the .xml file. When we try to load it with RReader, we get the following error.

root [0] TMVA::Experimental::RReader model("dataset/weights/TMVAClassification_PyTorch.weights.xml");
<FATAL>                          : Unknown method in map: PyTorch
***> abort program execution
Error in <TRint::HandleTermInput()>: std::runtime_error caught: FATAL error

What might be causing this, and how would we find a way to load the .xml file?

Thanks in advance,
Kevin

For information, the .xml file begins like this:

<?xml version="1.0"?>
<MethodSetup Method="PyTorch::PyTorch">
  <GeneralInfo>
    <Info name="TMVA Release" value="4.2.1 [262657]"/>
    <Info name="ROOT Release" value="6.26/04 [399876]"/>
    <Info name="Creator" value="kyoon"/>
    <Info name="Date" value="Wed Nov  9 03:54:58 2022"/>
    <Info name="Host" value="Linux centos7-docker 4.18.0-365.el8.x86_64 #1 SMP Thu Feb 10 16:11:23 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux"/>
2

Hi @kyoon ,

and welcome to the ROOT forum! @moneta can correct me if I’m wrong, but I think RReader covers BDTs while for NN inference you need SOFIE, see the tutorials here (in particular the TMVA_SOFIE_RDataFrame.C).

Cheers,
Enrico

3

Hi @eguiraud,

Thanks for the response. It seems like using SOFIE is the right place to start. Looking at the TMVA_SOFIE_RDataFrame.C tutorial, I see that I first need to generate a header file .hxx in order to load the data into RDataFrame with the trained model.

Since I am using PyTorch, I’ve also referred to this tutorial, TMVA_SOFIE_Pytorch.C, in order to generate the .hxx. I ran the example in SWAN, and this is the error message that I get.

SOFIE::RModel model = SOFIE::PyTorch::Parse("PyTorchModel.pt",inputShapesSequential);

Torch Version: 1.11.0a0+gitbc2c6ed
Python error message:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
TypeError: _model_to_graph() got an unexpected keyword argument 'example_outputs'

When I try to do the same with my .pt model in my environment (ROOT version 6.26/04), I still get the identical error message.

root [3] TMVA::Experimental::SOFIE::RModel model = TMVA::Experimental::SOFIE::PyTorch::Parse("trainedModelClassification.pt", inputShapesSequential);

Torch Version: 1.11.0a0+gitbc2c6ed
Python error message:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
TypeError: _model_to_graph() got an unexpected keyword argument 'example_outputs'
Error in <TRint::HandleTermInput()>: std::runtime_error caught:
Failed to run python code: graph=_model_to_graph(model,dummyInputs,example_outputs=output)
4

Hi @kyoon ,

we need @moneta 's help for the SOFIE error, let’s ping him.

Cheers,
Enrico

5

Hi,
Can you please post your PyTorch model (the .pt file) and I can have look. It is possible we need to convert to ONNX, since the support for parsing from native PyTorch is limited to few operators.

Cheers

Lorenzo

6

Hi @moneta,

Since I cannot upload a .pt file here directly, I am putting it inside a .zip file.

trainedModelClassification.zip (16.5 KB)

7

Hi,

With these simple python lines of code you can convert the model to ONNX:

import torch
model = torch.load("trainedModelClassification.pt")
xinput = torch.zeros((1,11))   # define input shape (assume batch size = 1)
torch.onnx.export(model,xinput,"trainedModelClassification.onnx",export_params=True)

Afterwards you can use generate the header file to be used with RDataFrame in C++.
You can use these lines of code to generate the header file (from tutorial TMVA_SOFIE_ONNX.C) and “.dat” file containing the weights:

{
   std::string inputName = "trainedModelClassification.onnx";
   using namespace TMVA::Experimental;
   SOFIE::RModelParser_ONNX parser;  
   std::cout << "parsing file " << inputName << std::endl;
   SOFIE::RModel model = parser.Parse(inputName);
   model.Generate();
   model.OutputGenerated();//outputName);
   std::cout << "output written " << std::endl;
}

and then integrate in RDataFrame as in the tutorial TMVA_SOFIE_RDataFrame.C.

You can also do all in Python (parsing ONNX file and compile header on the file) as shown in this tutorial, TMVA_SOFIE_RDataFrame.py.

There is only one issue, you would need to use the master version or one of the latest nightly builds, since one operator in your model (SoftMax) is not part of 6.26.

Best regards

Lorenzo

2 Likes
8

Hi Lorenzo,

That method seems to work! I can now load the PyTorch model in RDataFrame and evaluate it.

Thank you for your help,
Kevin