PyMVA crashes with conda ROOT


I am using ROOT v6_24_6 installed with miniforge3 on macOS Big Sur.
I was attempting to use PyKeras method in TMVA, but PyMVA initialization fails. I paste an iPython snippet that reproduces the crash, with error trace included.

(rootenv) $ ipython
Python 3.8.10 | packaged by conda-forge | (default, May 11 2021, 06:27:18) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.27.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from ROOT import TMVA

In [2]: TMVA.Tools.Instance()
Out[2]: <cppyy.gbl.TMVA.Tools object at 0x14d608dd0>

In [3]: TMVA.PyMethodBase.PyInitialize()
Error processing line 1 of /Users/rshara01-local/miniforge3/envs/rootenv/lib/python3.8/site-packages/google_auth-1.35.0-py3.9-nspkg.pth:

Fatal Python error: init_import_size: Failed to import the site module
Python runtime state: initialized
Traceback (most recent call last):
  File "/Users/rshara01-local/miniforge3/envs/rootenv/lib/python3.8/", line 169, in addpackage
  File "<string>", line 1, in <module>
  File "/Users/rshara01-local/miniforge3/envs/rootenv/lib/python3.8/importlib/", line 14, in <module>
    from contextlib import contextmanager
  File "/Users/rshara01-local/miniforge3/envs/rootenv/lib/python3.8/", line 5, in <module>
    from collections import deque
  File "/Users/rshara01-local/miniforge3/envs/rootenv/lib/python3.8/collections/", line 24, in <module>
    import heapq as _heapq
  File "/Users/rshara01-local/miniforge3/envs/rootenv/lib/python3.8/", line 581, in <module>
    from _heapq import *
SystemError: initialization of _heapq did not return an extension module

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
SystemError: initialization of _heapq did not return an extension module

I saw what appears to be a similar issue (for PyROOT) being discussed here:

I wonder if it is indeed the same thing, and if there is a workaround?

Thank you in advance,

ROOT Version: 6.24.6 (from conda-forge)
Platform: MacOS 11.5.2
_Python Version: 3.8.10 (from conda)
Compiler: N/A

@etejedor Can you comment?

The referenced issue was solved, we now build the PyROOT libraries with undefined symbols and we don’t link to the python libs anymore.

Perhaps some problem in PyMethodBase.PyInitialize() @moneta ?

I can’t reproduce with conda ROOT 6.24.02 though.

Also tried with 6.24.06 and I can’t reproduce either (this is Ubuntu). Perhaps @moneta can try on a mac? I installed with conda create -c conda-forge --name root_624_6 root

I get similar issue on a different macOS version, so it persists for me (though the error message is different this time).

On macOS 10.15.7, I created a fresh venv:
conda create -c conda-forge --name py38root624 python=3.8 root

Then tried again:

In [1]: from ROOT import TMVA
In [2]: TMVA.Tools.Instance()
Out[2]: <cppyy.gbl.TMVA.Tools object at 0x7fcea8b6e2f0>
In [3]: TMVA.PyMethodBase.PyInitialize()
<FATAL>                         : Can't import __main__
***> abort program execution
runtime_error                             Traceback (most recent call last)
<ipython-input-3-abfda3a85476> in <module>
----> 1 TMVA.PyMethodBase.PyInitialize()

runtime_error: static void TMVA::PyMethodBase::PyInitialize() =>
    runtime_error: FATAL error

_ROOT Version: 6.24.6 (from conda-forge)
_Platform: MacOS 10.15.7 (Catalina)
Python version: 3.8.12 (from conda-forge)

@moneta was also able to reproduce with conda on MacOS but he has a different error (pointer being freed was not allocated).

I tried on a MacOS 10.15 with an installation of ROOT from source and I can’t reproduce, so it might be related to the Conda installation on MacOS. @chrisburr do you have an idea of what can be the issue here?

The resolved PyROOT issue I originally referenced suggests that similar issues can be expected for PyMVA as well in the presence of both static& dynamic libs. To quote
Note that I will discuss PyROOT, but PyMVA will need the same treatment.
I wonder if that was already addressed? Apologies if it has been, and this is something unrelated.

I see your point now, thanks. I’m still not sure that is the cause of the issue you see, I leave it up to @moneta to decide what to do.

For reference, the PR that changed the way we build the PyROOT libraries to not link against libpython is:

A similar thing could be tried with PyMVA.

We have a PR merged in master that should fix this Condo issue,