TMVA PyTorch

TMVA
1

Hi all,

I’m trying to train the NN using TMVA PyTorch. I’m stuck at one problem:
when I run the script it outputs such error on terminal:

You are probably missing the definition of TMVA::PyMethodBase::PyInitialize()
Maybe you need to load corresponding shared library?

It is strange because in the script there is such a piece of code: TMVA.PyMethodBase.PyInitialize(). Why even such a problem appears? For the note: I’m running the script on lxplus8 machine. Python version is 3.6.8. Root version is 6.26.00. Torch package is installed.

Am I missing something?

Cheers,
Aizat

2

Hi,
Yes this is strange, can you please post your code and if needed data file, so I can run your script and reproduce the problem

Cheers

Lorenzo

3

Hi,

Actually this is a code which presented in root’s official webpage. I’d like to get familiar with TMVA PyTorch, so I just copy pasted the entire code and run it on lxplus8 machine. Please, find in attachment the script which I’ve used.
TMVA_Pytorch.py (4.4 KB)
The data file loads automatically when the running the script.

Thank you!

Cheers,

4

Can you also please upload the data so I can run it?

5

When you run the script, it should load the data, because as I understand there is a line of code in the script which does that.

6

Hi,
Sorry for my late reply. Your script runs fine for me. I think you are maybe using a ROOT installation without PyMVA support ?
What is the output of this command from the terminal:

root-config --has-tmva-pymva

Lorenzo

7

Hi,

No need to apologize. I checked the command you’ve sent, it perfectly finds the pymva.
I run the command TMVA.PyMethodBase.PyInitialize() once again, and sending the full error output hoping that it will give more clear idea what is going wrong:

IncrementalExecutor::executeFunction: symbol ‘_ZN4TMVA12PyMethodBase12PyInitializeEv’ unresolved while linking symbol ‘__cf_3’!
You are probably missing the definition of TMVA::PyMethodBase::PyInitialize()
Maybe you need to load the corresponding shared library?

Thank you!

Cheers,

8

Very strange. Are you building ROOT from source on lxplus or using a pre-existing binary ?
In this second case I could maybe check directly

Lorenzo

9

In lxplus8 I didn’t install root, using pre-existing binary. (By pre-existing binary I understand is as root is already built). Version is 6.26.

On the machine other than lxplus8, where mainly CentOS 7, in order to train NN using TMVA PyKeras, I use the shell script /cvmfs/sft.cern.ch/lcg/views/LCG_88/x86_64-centos7-gcc62-opt/setup.sh which puts environmental variables. In this case TMVA.PyMethodBase.PyInitialize() perfectly works.

Maybe there is alternative setup scripts for CentOS 8, which I can use ??

Cheers,

10

Hi,

I can reproduce your problem with the default root installed in /usr/bin in CentOS8. That version does not contain libPyMVA, it is strange that root-config --has-tmva-pymva returns yes.
You can use another ROOT version there, for example for 6.26.02 , from
/cvmfs/sft.cern.ch/lcg/app/releases/ROOT/6.26.02/
You need to do first:
. /cvmfs/sft.cern.ch/lcg/app/releases/ROOT/6.26.02/x86_64-centos8-gcc85-opt/bin/thisroot.sh
or you can use an older version of ROOT in an LCG view as you were doing on CentoS7.
For example you can use 6.24.06 by doing:
. /cvmfs/sft.cern.ch/lcg/app/releases/ROOT/6.26.02/x86_64-centos8-gcc85-opt/bin/thisroot.sh

Cheers

Lorenzo

1 Like
11

Thank you Lorenzo,

with the setup script you sent me everything works fine!

Cheers,

12

Sorry to bother you again!

Can you please reproduce the results on your own lxplus machine? The previous problem disappeared, but unfortunately some incomprehensible errors appeared. I’m uploading my script and my data files.

PyTorch_SM.py (5.9 KB)

Here the data:
New_MiniNtuple_tHbq_SM_300K.root (682.6 KB)
New_MiniNtiple_tt_SM_3M.root (764.3 KB)

Cheers,

13

Hi,
I have this failure running your code:

<WARNING>                : Failed to run python code: model = torch.jit.load('modelbmc_generated_data.h5')
<WARNING>                : Python error message:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/moneta/Library/Python/3.8/lib/python/site-packages/torch/jit/_serialization.py", line 151, in load
    raise ValueError("The provided filename {} does not exist".format(f))  # type: ignore[str-bytes-safe]
ValueError: The provided filename modelbmc_generated_data.h5 does not exist

You need to provide also the input model file.

Lorenzo