ROOT in Python3 (and in Python3-kernel jupyter notebooks)

Hi all,

I have a brand new Debian 9 installation (gcc 6.3, python=2.7 and python3=3.5), and I would like to use ROOT6 within python3 and also within jupyter notebooks running a python3 kernel.

Googling around I have the impression the this is still work in progress, though I do not know which is the most up-to-date information and the closed issue here https://sft.its.cern.ch/jira/browse/ROOT-8260 makes me hope that this is possible.

My recipe that works very well with python 2 and the last ROOT v10.00.02 is the following:

  1. I successfully compile ROOT configuring it with cmake ... -Dall=ON ..., which successfully detects python2.7 among the rest.
  2. I verify that (after sourcing thisroot.csh) I can start and operate root, and I can import ROOT and use ROOT objects in a python session
  3. To not pollute the system with the pip packages required for the jupyter notebooks, I use virtualenv to create an isolated environment. I source this environment just before sourcing root and I can start the notebook and use both the ROOT kernel and the Python 2 kernel, where I can successfully import ROOT

For Python 3, my attempts are limited so far to the interactive prompt environment, because I think that it should work there before any attempt with the notebooks

A) Trying to “import ROOT” in a python3 session results in the following error:

Python 3.5.3 (default, Jan 19 2017, 14:11:04) 
[GCC 6.3.0 20170118] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ROOT
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/ROOT6/root_v6_10_02/lib/ROOT.py", line 24, in <module>
    import cppyy
  File "/opt/ROOT6/root_v6_10_02/lib/cppyy.py", line 61, in <module>
    import libPyROOT as _backend
ImportError: dynamic module does not define module export function (PyInit_libPyROOT)

B) I compiled again ROOT with the option cmake ... -D-Dpython3=ON -DPYTHON_EXECUTABLE=/usr/bin/python3 ... -Dall=ON .... I have to specify the python3 path because it seems that cmake misses it and step back to 2.7
The result is unchanged: I can import ROOT in a python 2 session but I get the same error message in a python 3 session

Any suggestion is very welcome

Matteo

Hi Matteo,
do you have installed libpython3-dev?

Yes it is, as well as python3-dev (I think that one of the two is a dummy package that refer to the other)

libPyROOT.so is an extension module and thus needs rebuilding for each different version of Python. Note that you only need to rebuild libPyROOT.so, not all of ROOT, for p3. To work with both, you thus need two of them. The ROOT installation does not know of virtualenv: libPyROOT is simply found through PYTHONPATH. You thus need a different setting for your p2 build than your p3 one. Currently, PYTHONPATH is clearly still pointing to the p2 version of libPyROOT in all cases.

Dear wlav,

thanks for the explanation, but can I have a little bit more details? How do I rebuild libPyROOT.so using python3? Is that what I have already done compiling root with the options -Dpython3=ON -DPYTHON_EXECUTABLE=/usr/bin/python3, despite it compiled the whole root?

If I look at PYTHONPATH after sourcing thisroot.csh:

echo $PYTHONPATH
/opt/ROOT6/root_v6_10_02_python3/lib/ 

is there any way to figure out if libPyROOT.so uses the right python?

Please note that so far virtualenv has no roles. My plan was to use it afterwards to have the ROOT-flavoured kernel AND the Python-3 kernel, but this will be the next problem

Any further clarification if very welcome,
Matteo

I’m not familiar with the cmake build system and options, so I can’t help you there. But clearly in your code above, ROOT.py (and hence most likely libPyROOT.so) is NOT picked up from root_v6_10_02_python3, but from root_v6_10_02 …

Now, to verify libPyROOT.so, simply do: ldd libPyROOT.so | grep python and you’ll see the python lib it is linked with.

Dear wlav,

The ROOT directory used was right, but the library was wrong. I found out that there are 3 options to pass to cmake:

cmake [...] -Dpython3=ON -DPYTHON_EXECUTABLE=/usr/bin/python3 -DPYTHON_INCLUDE_DIR=/usr/include/python3.5m -DPYTHON_LIBRARY=/usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0 [...]

Now that I have compiled everything again (sorry for the late reply but I had no time to try out this earlier) it works as expected, and I can import ROOT also inside Python3 jupyter-notebooks.

On the contrary, the formerly-working ROOT-flavoured notebook, solution based on the virtualenv environment + the python2-linked ROOT installation, is not longer working… the ROOT kernel crashes immediately… I do not understand what I have touched of this former installation…
… any idea? Does the “root --notebook” launcher share some home-dir or elsewhere configuration file with the “jupyter-notebook” launcher?

Thanks again,
Matteo

No idea: I know neither cmake nor notebooks (all from after my time). However, I don’t see how the ROOT installation and virtualenv would play nicely together. I presume that you are now picking up the p3 libPyROOT.so in the p2 environment?

Dear wlav,

I meant that now both ROOT and python{,3} works as expected, and I can import ROOT in the respective environment provided that I do not mix up the ROOT installation that I use. In other words:

source /opt/ROOT6/root_v6_10_02/bin/thisroot.sh #allows import ROOT in python 2
source /opt/ROOT6/root_v6_10_02_python3/bin/thisroot.sh #allows import ROOT in python 3

The second part of the question concerns only jupyter notebooks.

I want to have both:
(A) IPython2/3 Jupyter notebook and
(B) the ROOT-flavoured notebook (either based on jupyter or jupyter3, I do not care at this point because I only need to run C++/ROOT code on it).

Here it seems that the problem is related to some caching/configuration files that ipython and jupyter puts in your home directory. The suspects on this Debian 9 machines are:

~/.rootnb/*
~/.jupyter/*
~/.ipython/*
~/.local/share/jupyter/*

Since (B) was running (based on jupyter2) at the beginning of the story, I tried to remove all these configuration files from my home dir, and indeed (B) works again, creating again part of the configuration files (mainly in ~/.rootnb, but also in ~/.ipython and ~/.local/share/jupyter/)

The good new is now also (A) works launching jupyter-notebook (from the Debian 9 packages) after sourcing the “python3” ROOT script in a fresh shell. In a python3-notebook I can successfully import ROOT.

This of course add more stuff in the ~/.ipython and ~/.local/share/jupyter/ directories, but for the time being it seems that the two options keeps running.

I close the matter for now (how can I close a topic in this new forum ???) and I will open a more specific thread if I have new issues

sourcing ‘thisroot.sh’ modifies PYTHONPATH to point to $ROOTSYS/lib and thus to the specific libPyROOT.so, which takes precedence over any virtualenv settings.

As for the cache files/directories, I’d hope that those do work well with virtualenv.

Don’t know how to close topics. A message at the bottom states these close automatically in 14 days.

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.