Libcppyy_backend.so: cannot open shared object file: No such file or directory

ROOT Version: 6.36.04
Platform: Ubuntu 13.3.0
Compiler: linuxx8664gcc

Dear experts,

please may I ask you for a kind help. I’ve got ROOT installed through snap for Ubuntu and try to import it from inside Python (3.12.3). I’ve got path “/snap/root-framework/954/usr/local/lib” explicitly included in the $PYTHONPATH variable, otherwise it does not find ROOT module at all, but still run into an error:

----> 1 import ROOT

File /snap/root-framework/954/usr/local/lib/ROOT/init.py:25
---> 25 import cppyy

[...]

File /snap/root-framework/954/usr/local/lib/cppyy/init.py:88
---> 88     from ._cpython_cppyy import *

[...]

File /snap/root-framework/954/usr/local/lib/cppyy/_cpython_cppyy.py:24
---> 24 c = loader.load_cpp_backend()

[...]

File /snap/root-framework/954/usr/local/lib/cppyy_backend/loader.py:100, in load_cpp_backend()
---> 100     raise RuntimeError(“could not load cppyy_backend library, details:\n%s” %

RuntimeError: could not load cppyy_backend library, details:
libcppyy_backend.so: cannot open shared object file: No such file or directory

/snap/root-framework/954/usr/local/lib/cppyy_backend/lib/libcppyy_backend.so: cannot open shared object file: No such file or directory

/snap/root-framework/954/usr/local/lib/cppyy_backend/lib/libcppyy_backend.cpython-312-x86_64-linux-gnu.so: cannot open shared object file: No such file or directory
libcppyy_backend.cpython-312-x86_64-linux-gnu.so: cannot open shared object file: No such file or directory

A file named “libcppyy_backend.so” is present in the “/snap/root-framework/954/usr/local/lib/“, but not in “/snap/root-framework/954/usr/local/lib/cppyy_backend“

Following an advice found in this topic I tried to set a variable

export CPPYY_BACKEND_LIBRARY=/snap/root-framework/954/usr/local/lib/libcppyy_backend.so

But then there’s another problem:

----> 1 import ROOT
File /snap/root-framework/954/usr/local/lib/ROOT/init.py:27—> 27 import libROOTPythonizationsImportError: libTree.so: cannot open shared object file: No such file or directory

So, probably, I’m moving in a wrong direction… Please, could anyone give a hint on how to deal with this problem?

Best regards,

Dasha.

Hello Dasha,

we have never tested snap, but your case looks like in addition to PYTHONPATH (which only helps for finding Python libraries), you should also set LD_LIBRARY_PATH, which will allow cppyy to find the ROOT libraries.

Have same on macOS and ROOT 6.36.04 compiled from sources.

$ cat test.py
#!/usr/bin/env python3

import ROOT
$ ./test.py
Traceback (most recent call last):
  File "/Users/USER/Work/mstsa/./test.py", line 3, in <module>
    import ROOT
  File "/Users/USER/soft/root_install/lib/ROOT/__init__.py", line 25, in <module>
    import cppyy
  File "/Users/USER/soft/root_install/lib/cppyy/__init__.py", line 88, in <module>
    from ._cpython_cppyy import *
  File "/Users/USER/soft/root_install/lib/cppyy/_cpython_cppyy.py", line 24, in <module>
    c = loader.load_cpp_backend()
  File "/Users/USER/soft/root_install/lib/cppyy_backend/loader.py", line 100, in load_cpp_backend
    raise RuntimeError("could not load cppyy_backend library, details:\n%s" %
        '\n'.join(['  '+x for x in err]))
RuntimeError: could not load cppyy_backend library, details:
  dlopen(libcppyy_backend.cpython-313-darwin.so, 0x000A): tried: 'libcppyy_backend.cpython-313-darwin.so' (no such file), '/System/Volumes/Preboot/Cryptexes/OSlibcppyy_backend.cpython-313-darwin.so' (no such file), '/usr/lib/libcppyy_backend.cpython-313-darwin.so' (no such file, not in dyld cache), 'libcppyy_backend.cpython-313-darwin.so' (no such file)
  dlopen(/Users/USER/soft/root_install/lib/cppyy_backend/lib/libcppyy_backend.so, 0x000A): tried: '/Users/USER/soft/root_install/lib/cppyy_backend/lib/libcppyy_backend.so' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/USER/soft/root_install/lib/cppyy_backend/lib/libcppyy_backend.so' (no such file), '/Users/USER/soft/root_install/lib/cppyy_backend/lib/libcppyy_backend.so' (no such file)
  dlopen(libcppyy_backend.so, 0x000A): tried: 'libcppyy_backend.so' (no such file), '/System/Volumes/Preboot/Cryptexes/OSlibcppyy_backend.so' (no such file), '/usr/lib/libcppyy_backend.so' (no such file, not in dyld cache), 'libcppyy_backend.so' (no such file)
  dlopen(/Users/USER/soft/root_install/lib/cppyy_backend/lib/libcppyy_backend.cpython-313-darwin.so, 0x000A): tried: '/Users/USER/soft/root_install/lib/cppyy_backend/lib/libcppyy_backend.cpython-313-darwin.so' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/USER/soft/root_install/lib/cppyy_backend/lib/libcppyy_backend.cpython-313-darwin.so' (no such file), '/Users/USER/soft/root_install/lib/cppyy_backend/lib/libcppyy_backend.cpython-313-darwin.so' (no such file)

However:

$ python3 test.py

works.

$ which root
/Users/USER/soft/root_install/bin/root
$ root-config --version
6.36.04

Loader tries to open: /Users/USER/soft/root_install/lib/cppyy_backend/lib/libcppyy_backend.so – but it does not exist.
There is library /Users/USER/soft/root_install/lib/libcppyy_backend.so, but making symlink to cppyy_backend/lib does not help.

Dear Stephan, it seems to fix the problem, thank you very much!

That’s great to hear!

For other people who find this, more details of using snap are explained here (for example, that it has its own python interpreter):

Okay, but what for macOS and why new ROOT version does not work as the previous?

Hello,

It looks like your problem has nothing to do with snap, so we need to have a separate discussion about it.

If you see a difference when running with env python3 vs python3, I wonder if your environment gets modified in an unintended way.

  • What are the values of PATH, DYLD_LIBRARY_PATH, PYTHONPATH in both cases?
  • Do you use thisroot or bashrc (or a similar setup file) to make ROOT findable?
  • Is the python3 interpreter the same in both cases?

Well, because ROOT evolves with versions. :slightly_smiling_face: One tries of course to change things for the better, make them simpler and faster, but there can of course be unintended consequences. Each change gets tested, but of course not all way of setting up ROOT of every user in the world can be tested. :sweat_smile:

  • Do you use thisroot or bashrc (or a similar setup file) to make ROOT findable?
    I use ‘thisroot.sh’ shipped with the ROOT sources (in installed in root_install/bin/)

  • Is the python3 interpreter the same in both cases?
    Yes, only one Python interpreter is installed system-wide – from the official installer (no homebrew etc.)

  • What are the values of PATH, DYLD_LIBRARY_PATH, PYTHONPATH in both cases?

$ $ cat test.py
#!/usr/bin/env python3

import os

print(os.environ['PATH'])
print(os.environ['PYTHONPATH'])
print(os.environ['DYLD_LIBRARY_PATH'])

Then:

$ python3 test.py
/Users/USER/soft/root_install/bin:...
/Users/USER/soft/root_install/lib
/Users/USER/soft/root_install/lib

But:

$ ./test.py
KeyError: 'DYLD_LIBRARY_PATH'

Seems like it is macOS SIP policy, so ROOT probably must not rely on the DYLD_LIBRARY_PATH in macOS.

OK, thanks for checking! That makes sense.

The ROOT (C++) installation generally doesn’t rely on DYLD_LIBRARY_PATH, but uses LC_RPATH. You should be able to start the root interpreter without using thisroot.sh and it will find its libraries.

The loader of cppyy backend, however, seems to try:

  1. The environment variable CPPYY_BACKEND_LIBRARY
  2. A “normal” dlopen where LD_LIBRARY_PATH or DYLD_LIBRARY_PATH are considered, and libraries in system locations are found.
  3. And finally it falls back to loading in the absolute path where loader.py is.

1 and 2 of course only work if the environment of your shell “survives”. There seems to be an error in 3., so I’ll bring this up with the cppyy maintainers.

So I guess you could use strategy 1. and 2., but you will have to change the shebang:

- #! /usr/bin/env python3
+ #! /usr/local/bin/python3

Or you can of course invoke it as python3 <script>.

#! /usr/local/bin/python3

Yep, that works, thanks!

The ROOT (C++) installation generally doesn’t rely on DYLD_LIBRARY_PATH

Yes, ‘root’ itself works after loading ‘thisroot.sh’, the problem appears only when importing ROOT module in Python.