Cppyy segfault importing c++ class

marc1uk · December 14, 2022, 5:59pm

ROOT Version: 6.24.06
Platform: Centos Stream 8
Compiler: g++ 8.5.0
Python: 3.8.13

Hello everyone.
I’m trying to add python integration into an application using cppyy, but having a hard time getting it working. Although strictly nothing I’m doing involves ROOT classes or functions, since cppyy is being provideed by ROOT the suggestion from the cppyy maintainers has been to ask here. So here goes…

Since nothing appears to be working, I’ve worked my way down to a minimal tester, which also does not work.
I have a simple c++ class, which I compile into a library, load the library in python, and then try to import the class. The result is a segmentation violation pointing to libCling.so.

the code: pydebug.zip (1.8 KB)

Within test.py there are two lines commented out that import the DataModel class. Uncommenting either produces the following stacktrace:

[root@build-01 try1]$ python3 test2.py                                                                                        
/ToolAnalysis/ToolDAQ/root-6.24.06/install/lib/libCling.so(+0x3364f8a)[0x7fbda68abf8a]
/ToolAnalysis/ToolDAQ/root-6.24.06/install/lib/libCling.so(+0x7dadc1)[0x7fbda3d21dc1]
/ToolAnalysis/ToolDAQ/root-6.24.06/install/lib/libCling.so(+0x7db76d)[0x7fbda3d2276d]
/ToolAnalysis/ToolDAQ/root-6.24.06/install/lib/libCling.so(+0xf30209)[0x7fbda4477209]
/ToolAnalysis/ToolDAQ/root-6.24.06/install/lib/libCling.so(+0xef58fb)[0x7fbda443c8fb]
/ToolAnalysis/ToolDAQ/root-6.24.06/install/lib/libCling.so(+0xf1581c)[0x7fbda445c81c]
/ToolAnalysis/ToolDAQ/root-6.24.06/install/lib/libCling.so(+0xf18059)[0x7fbda445f059]
/ToolAnalysis/ToolDAQ/root-6.24.06/install/lib/libCling.so(+0xf1e65b)[0x7fbda446565b]
/ToolAnalysis/ToolDAQ/root-6.24.06/install/lib/libCling.so(+0xf23b46)[0x7fbda446ab46]
/ToolAnalysis/ToolDAQ/root-6.24.06/install/lib/libCling.so(+0x1009915)[0x7fbda4550915]
/ToolAnalysis/ToolDAQ/root-6.24.06/install/lib/libCling.so(+0x2cfce36)[0x7fbda6243e36]
/ToolAnalysis/ToolDAQ/root-6.24.06/install/lib/libCling.so(+0x9db258)[0x7fbda3f22258]
/ToolAnalysis/ToolDAQ/root-6.24.06/install/lib/libCling.so(+0x2e315bf)[0x7fbda63785bf]
/ToolAnalysis/ToolDAQ/root-6.24.06/install/lib/libCling.so(+0x2e2e5e5)[0x7fbda63755e5]
/ToolAnalysis/ToolDAQ/root-6.24.06/install/lib/libCling.so(+0x2e316bc)[0x7fbda63786bc]
/ToolAnalysis/ToolDAQ/root-6.24.06/install/lib/libCling.so(+0x2e2e5e5)[0x7fbda63755e5]
/ToolAnalysis/ToolDAQ/root-6.24.06/install/lib/libCling.so(+0x2e3161a)[0x7fbda637861a]
/ToolAnalysis/ToolDAQ/root-6.24.06/install/lib/libCling.so(+0x2e2e5e5)[0x7fbda63755e5]
/ToolAnalysis/ToolDAQ/root-6.24.06/install/lib/libCling.so(+0x2e316bc)[0x7fbda63786bc]
/ToolAnalysis/ToolDAQ/root-6.24.06/install/lib/libCling.so(+0x2e2e5e5)[0x7fbda63755e5]
/ToolAnalysis/ToolDAQ/root-6.24.06/install/lib/libCling.so(+0x2e316bc)[0x7fbda63786bc]
/ToolAnalysis/ToolDAQ/root-6.24.06/install/lib/libCling.so(+0x2e2e5e5)[0x7fbda63755e5]
/ToolAnalysis/ToolDAQ/root-6.24.06/install/lib/libCling.so(+0x2e316bc)[0x7fbda63786bc]
/ToolAnalysis/ToolDAQ/root-6.24.06/install/lib/libCling.so(+0x2e2e5e5)[0x7fbda63755e5]
/ToolAnalysis/ToolDAQ/root-6.24.06/install/lib/libCling.so(_ZN20TClingDataMemberInfo6OffsetEv+0x65)[0x7fbda3bef5f5]
/ToolAnalysis/ToolDAQ/root-6.24.06/install/lib/libCore.so(_ZNK11TDataMember13GetOffsetCintEv+0xbe)[0x7fbdaa3a5b1e]
/ToolAnalysis/ToolDAQ/root-6.24.06/install/lib/libcppyy_backend3_8.so(_ZN5Cppyy19GetDatamemberOffsetEmm+0x236)[0x7fbdaa8558b6]/ToolAnalysis/ToolDAQ/root-6.24.06/install/lib/libcppyy3_8.so(_ZN8CPyCppyy13CPPDataMember3SetEmm+0x63)[0x7fbdaaab7d23]
/ToolAnalysis/ToolDAQ/root-6.24.06/install/lib/libcppyy3_8.so(+0x792ee)[0x7fbdaaae02ee]
/ToolAnalysis/ToolDAQ/root-6.24.06/install/lib/libcppyy3_8.so(_ZN8CPyCppyy16CreateScopeProxyERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEP7_object+0xe91)[0x7fbdaaae2d91]
/ToolAnalysis/ToolDAQ/root-6.24.06/install/lib/libcppyy3_8.so(+0x5fe4a)[0x7fbdaaac6e4a]
/lib64/libpython3.8.so.1.0(_PyObject_LookupAttr+0x8b)[0x7fbdac30238b]
/lib64/libpython3.8.so.1.0(_PyEval_EvalFrameDefault+0x48f7)[0x7fbdac3811c7]
/lib64/libpython3.8.so.1.0(_PyEval_EvalCodeWithName+0x3c6)[0x7fbdac339c46]
/lib64/libpython3.8.so.1.0(PyEval_EvalCode+0x23)[0x7fbdac33ae23]
/lib64/libpython3.8.so.1.0(+0x22af0a)[0x7fbdac3c6f0a]
/lib64/libpython3.8.so.1.0(+0x250e52)[0x7fbdac3ece52]
/lib64/libpython3.8.so.1.0(+0x102f11)[0x7fbdac29ef11]
/lib64/libpython3.8.so.1.0(PyRun_SimpleFileExFlags+0x369)[0x7fbdac2a5de8]
/lib64/libpython3.8.so.1.0(Py_RunMain+0x2ef)[0x7fbdac3eea1f]
/lib64/libpython3.8.so.1.0(Py_BytesMain+0x39)[0x7fbdac3eeba9]
/lib64/libc.so.6(__libc_start_main+0xe5)[0x7fbdab23dd85]
python3(_start+0x2e)[0x560123d3178e]
Missing transaction during deserialization!
UNREACHABLE executed at /ToolAnalysis/ToolDAQ/root-6.24.06/interpreter/cling/lib/Interpreter/DeclCollector.cpp:68!
 *** Break *** abort
 *** Break *** abort

marc1uk · December 15, 2022, 12:26am

Just in case there was any doubt (though the code is pretty simple), I’ve tried this on another system, using the latest cppyy independent of ROOT, and it works without issue, so this definitely seems to be something specifically to do with the installation, setup, or ROOT’s cppyy version.

marc1uk · December 15, 2022, 2:40pm

A little more info on this; trying things from the command prompt seems to reveal the problem specifically lies with cling’s ability to parse the class header.
As described on the cppyy user guide, it’s sufficient to load the header in order to explore the class. In a working installation, I can indeed do:

$ python3
>>> import cppyy
>>> cppyy.add_include_path('/path/to/DataModel')
>>> cppyy.include('DataModel.h')
>>> dir(cppyy.gbl.DataModel)

to see the accessible attributes of the class, even before the library is loaded.
With the ROOT installation, this same code segfaults.

moneta · December 19, 2022, 8:25am

Hi,
Thank you for your report. We will investigate the problem, but can you maybe open a ROOT GitHub issue on this so we will keep track of it ?
Thanks

Lorenzo

marc1uk · December 19, 2022, 8:49pm

Thanks for the response Lorenzo. I may hold off on that just for a bit, as I’ve managed to get the minimal reproducer working by generating a dictionary with rootcling and compiling that into the the library. working.zip (1.3 KB)

As far as I can tell this shouldn’t technically be necessary - it’s not a requirement from the cppyy side, and the ROOT page on IO of custom classes says no dictionary should be needed for interactive use either. Still, with a dictionary generated by rootcling built into the library, it works.

So far so good for the minimal reproducer.

Moving back to my actual application introduces more complexities, though, and there I’m still having trouble. Even with a dictionary included, modifying simple integer members works, but getting the size of a map<int,int> returns garbage. On the other hand, a minimal reproducer is able to modify both types without problems, so i’m slowly pulling things apart to see what’s different. I’ll update when I have more progress.

=====================

OK, well the thread closed over Christmas, but here’s the finale.

I’m afraid I can’t remember now how I solved the map size thing - perhaps some user error. However after solving that, I still found that on cleanup, after all the code had run, it would segmentation fault and/or spew out thousands of errors about corrupted double-linked list.
I eventually discovered this was a result of:

python being called from c++. Calling the python code (which uses c++ classes via cppyy) with python3 test.py always worked fine. Using the Python API to call the same test.py from c++ also worked unless …
A particular class was linked in to the calling c++ application as well. Specifically, this class had a static TF1 member. After replacing the static TF1 member with a static TF1* initialised on first use, everything now seems to be working ok (as far as i can see).

system · January 2, 2023, 8:50pm

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.