Segfault when invoking pythons help routine on a self-defined cpp macro function

phi · May 23, 2018, 3:29pm

Hello,
I am encountering the following problem in pyroot in conjunction with a custom cpp macro:
Given a simple self-defined macro, e.g.:

cat sampleRootMacro.C 

#include <iostream>
void aFunction(const char* aName) {
        std::cout << "Hello, " << aName << "!" << std::endl;
}

I can easily load the macro within an interactive pyroot session…

python
Python 2.7.13 (default, Nov 24 2017, 17:33:09) 
[GCC 6.3.0 20170516] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> 
>>> import ROOT
>>> ROOT.gROOT.LoadMacro("sampleRootMacro.C")

… and call the function:

>>> ROOT.aFunction("you")
Hello, you!

For ROOT builtin types, python’s help function works quiet beautifully:

>>> help(ROOT.TROOT)
...verbose_help_output...

However, invoking the help method leads on my custom function leads to a segfault and a total program crash.:

>>> help(ROOT.aFunction)

 *** Break *** segmentation violation



===========================================================
There was a crash.
This is the entire stack trace of all threads:
===========================================================

Thread 2 (Thread 0x7f49cb811700 (LWP 41884)):
#0  0x00007f49dbd32536 in futex_abstimed_wait_cancelable (private=0, abstime=0x0, expected=0, futex_word=0x55797c2f9730) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
#1  do_futex_wait (sem=sem
entry=0x55797c2f9730, abstime=0x0) at sem_waitcommon.c:111
#2  0x00007f49dbd325e4 in __new_sem_wait_slow (sem=0x55797c2f9730, abstime=0x0) at sem_waitcommon.c:181
#3  0x000055797b99ad34 in PyThread_acquire_lock ()
#4  0x000055797b9e6caa in PyEval_RestoreThread ()
#5  0x000055797ba92a49 in ?? ()
#6  0x000055797b9bfb9a in PyEval_EvalFrameEx ()
#7  0x000055797b9bdd45 in PyEval_EvalCodeEx ()
#8  0x000055797b9dac38 in ?? ()
#9  0x000055797b9ac413 in PyObject_Call ()
#10 0x000055797b9c6998 in PyEval_EvalFrameEx ()
#11 0x000055797b9c542f in PyEval_EvalFrameEx ()
#12 0x000055797b9c542f in PyEval_EvalFrameEx ()
#13 0x000055797b9bdd45 in PyEval_EvalCodeEx ()
#14 0x000055797b9daa7e in ?? ()
#15 0x000055797b9ac413 in PyObject_Call ()
#16 0x000055797b9f135e in ?? ()
#17 0x000055797b9ac413 in PyObject_Call ()
#18 0x000055797b9c9ec0 in PyEval_CallObjectWithKeywords ()
#19 0x000055797ba925d2 in ?? ()
#20 0x00007f49dbd2a494 in start_thread (arg=0x7f49cb811700) at pthread_create.c:333
#21 0x00007f49db147acf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97

Thread 1 (Thread 0x7f49dc130700 (LWP 41697)):
#0  0x00007f49db11704a in __GI___waitpid (pid=41981, stat_loc=stat_loc
entry=0x7ffc0636c5e0, options=options
entry=0) at ../sysdeps/unix/sysv/linux/waitpid.c:29
#1  0x00007f49db09e0ab in do_system (line=<optimized out>) at ../sysdeps/posix/system.c:148
#2  0x00007f49d87139a2 in TUnixSystem::Exec (shellcmd=<optimized out>, this=0x55797c259440) at /dsk/explora/compile_debian9/root6.12/Source/root/core/unix/src/TUnixSystem.cxx:2118
#3  TUnixSystem::StackTrace (this=0x55797c259440) at /dsk/explora/compile_debian9/root6.12/Source/root/core/unix/src/TUnixSystem.cxx:2412
#4  0x00007f49d8715e7c in TUnixSystem::DispatchSignals (this=0x55797c259440, sig=kSigSegmentationViolation) at /dsk/explora/compile_debian9/root6.12/Source/root/core/unix/src/TUnixSystem.cxx:3643
#5  <signal handler called>
#6  Cppyy::GetScopedFinalName[abi:cxx11](long) (klass=klass
entry=1) at /dsk/explora/compile_debian9/root6.12/Source/root/bindings/pyroot/src/Cppyy.cxx:574
#7  0x00007f49da603fb0 in PyROOT::CreateScopeProxy (scope=1) at /dsk/explora/compile_debian9/root6.12/Source/root/bindings/pyroot/src/RootWrapper.cxx:527
#8  0x00007f49da5ee968 in PyROOT::(anonymous namespace)::mp_meth_class (pymeth=0x7f49cd08d210) at /dsk/explora/compile_debian9/root6.12/Source/root/bindings/pyroot/src/MethodProxy.cxx:283
#9  0x000055797b9d6439 in ?? ()
#10 0x000055797b9ab8bc in _PyObject_GenericGetAttrWithDict ()
#11 0x000055797b9bf9bb in PyEval_EvalFrameEx ()
#12 0x000055797b9c542f in PyEval_EvalFrameEx ()
#13 0x000055797b9c542f in PyEval_EvalFrameEx ()
#14 0x000055797b9bdd45 in PyEval_EvalCodeEx ()
#15 0x000055797b9c5738 in PyEval_EvalFrameEx ()
#16 0x000055797b9bdd45 in PyEval_EvalCodeEx ()
#17 0x000055797b9c5738 in PyEval_EvalFrameEx ()
#18 0x000055797b9c542f in PyEval_EvalFrameEx ()
#19 0x000055797b9bdd45 in PyEval_EvalCodeEx ()
#20 0x000055797b9dac38 in ?? ()
#21 0x000055797b9ac413 in PyObject_Call ()
#22 0x000055797b9f135e in ?? ()
#23 0x000055797b9ac413 in PyObject_Call ()
#24 0x000055797ba9f428 in ?? ()
#25 0x000055797b9ac413 in PyObject_Call ()
#26 0x000055797b9c6998 in PyEval_EvalFrameEx ()
#27 0x000055797b9bdd45 in PyEval_EvalCodeEx ()
#28 0x000055797b9daa7e in ?? ()
#29 0x000055797b9ac413 in PyObject_Call ()
#30 0x000055797b9f135e in ?? ()
#31 0x000055797b9ac413 in PyObject_Call ()
#32 0x000055797ba4f937 in ?? ()
#33 0x000055797b9ac413 in PyObject_Call ()
#34 0x000055797b9c563f in PyEval_EvalFrameEx ()
#35 0x000055797b9bdd45 in PyEval_EvalCodeEx ()
#36 0x000055797b9bdb09 in PyEval_EvalCode ()
#37 0x000055797b9ee14f in ?? ()
#38 0x000055797b947278 in PyRun_InteractiveOneFlags ()
#39 0x000055797b94703c in PyRun_InteractiveLoopFlags ()
#40 0x000055797b929a9b in ?? ()
#41 0x000055797b999aa1 in Py_Main ()
#42 0x00007f49db07f2e1 in __libc_start_main (main=0x55797b9993d0 <main>, argc=1, argv=0x7ffc063705d8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffc063705c8) at ../csu/libc-start.c:291
#43 0x000055797b9992ca in _start ()
===========================================================


The lines below might hint at the cause of the crash.
You may get help by asking at the ROOT forum http://root.cern.ch/forum.
Only if you are really convinced it is a bug in ROOT then please submit a
report at http://root.cern.ch/bugs. Please post the ENTIRE stack trace
from above as an attachment in addition to anything else
that might help us fixing this issue.
===========================================================
#6  Cppyy::GetScopedFinalName[abi:cxx11](long) (klass=klass
entry=1) at /dsk/explora/compile_debian9/root6.12/Source/root/bindings/pyroot/src/Cppyy.cxx:574
#7  0x00007f49da603fb0 in PyROOT::CreateScopeProxy (scope=1) at /dsk/explora/compile_debian9/root6.12/Source/root/bindings/pyroot/src/RootWrapper.cxx:527
#8  0x00007f49da5ee968 in PyROOT::(anonymous namespace)::mp_meth_class (pymeth=0x7f49cd08d210) at /dsk/explora/compile_debian9/root6.12/Source/root/bindings/pyroot/src/MethodProxy.cxx:283
#9  0x000055797b9d6439 in ?? ()
#10 0x000055797b9ab8bc in _PyObject_GenericGetAttrWithDict ()
#11 0x000055797b9bf9bb in PyEval_EvalFrameEx ()
#12 0x000055797b9c542f in PyEval_EvalFrameEx ()
#13 0x000055797b9c542f in PyEval_EvalFrameEx ()
#14 0x000055797b9bdd45 in PyEval_EvalCodeEx ()
#15 0x000055797b9c5738 in PyEval_EvalFrameEx ()
#16 0x000055797b9bdd45 in PyEval_EvalCodeEx ()
#17 0x000055797b9c5738 in PyEval_EvalFrameEx ()
#18 0x000055797b9c542f in PyEval_EvalFrameEx ()
#19 0x000055797b9bdd45 in PyEval_EvalCodeEx ()
#20 0x000055797b9dac38 in ?? ()
#21 0x000055797b9ac413 in PyObject_Call ()
#22 0x000055797b9f135e in ?? ()
#23 0x000055797b9ac413 in PyObject_Call ()
#24 0x000055797ba9f428 in ?? ()
#25 0x000055797b9ac413 in PyObject_Call ()
#26 0x000055797b9c6998 in PyEval_EvalFrameEx ()
#27 0x000055797b9bdd45 in PyEval_EvalCodeEx ()
#28 0x000055797b9daa7e in ?? ()
#29 0x000055797b9ac413 in PyObject_Call ()
#30 0x000055797b9f135e in ?? ()
#31 0x000055797b9ac413 in PyObject_Call ()
#32 0x000055797ba4f937 in ?? ()
#33 0x000055797b9ac413 in PyObject_Call ()
#34 0x000055797b9c563f in PyEval_EvalFrameEx ()
#35 0x000055797b9bdd45 in PyEval_EvalCodeEx ()
#36 0x000055797b9bdb09 in PyEval_EvalCode ()
#37 0x000055797b9ee14f in ?? ()
#38 0x000055797b947278 in PyRun_InteractiveOneFlags ()
#39 0x000055797b94703c in PyRun_InteractiveLoopFlags ()
#40 0x000055797b929a9b in ?? ()
#41 0x000055797b999aa1 in Py_Main ()
#42 0x00007f49db07f2e1 in __libc_start_main (main=0x55797b9993d0 <main>, argc=1, argv=0x7ffc063705d8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffc063705c8) at ../csu/libc-start.c:291
#43 0x000055797b9992ca in _start ()
===========================================================

This is particularly annoying since the jupyter qtconsole (which I am normally using) seems to call this on every input sequence starting with the function name and an opening parenthesis. Probably in order to display tool-tips but this way it makes working more or less impossible.

Any ideas are welcome…

ROOT Version: 6.12/06:
Platform, compiler: Debian 9.4, gcc6.3:

wlav · May 23, 2018, 9:45pm

Is b/c the scope of that function is the global one and GetScopedFinalName in PyROOT does not check for that case. Compare in PyROOT:

std::string Cppyy::GetScopedFinalName( TCppType_t klass )
{
   // TODO: either this or GetFinalName is wrong
   TClassRef& cr = type_from_handle( klass );
   return cr->GetName();
}

v.s. cppyy master:

std::string Cppyy::GetScopedFinalName(TCppType_t klass)
{
    TClassRef& cr = type_from_handle(klass);
    if (cr.GetClass()) {
        std::string name = cr->GetName();
        if (is_missclassified_stl(name))
            return std::string("std::")+cr->GetName();
        return cr->GetName();
    }
    return "";
}

and so master works fine:

>>> help(cppyy.gbl.aFunction)
Help on method aFunction in module cppyy:

aFunction(self, const char* aName) unbound cppyy.gbl. method
    void ::aFunction(const char* aName)

phi · May 28, 2018, 1:38pm

Thank you Wim for your prompt reply and sharing these insights. I tried again with pure cppyy and everything works flawlessly here.

Is there any way how I can recycle my old cpp-macros within my current pyROOT workflow? I am thinking e.g .of an alternative import mechanism to ROOT.gROOT.LoadMacro which does some kind of scoping?
Altering all my macros to make the the functions class-scoped feels somehow wrong.

Of course I could just switch from jupyter qtconsole to jupyters plain terminal front-end (jupyter console), which does not call the help function automatically and refrain from invoking it manually. I am still wondering if I am the the only one facing that problem (In jupyter notebook you can query the doc via Shift+TAB which is causing the same behavior) and weather it is possible to fix this in pyroot’s GetScopedFinalName? An empty docstring would be far better than a segfault…

wlav · May 28, 2018, 5:59pm

I would install ROOT from source and fix this one item in-place. (Also make sure to file a JIRA bug report to see it fix; PyROOT does not follow cppyy master.)

As for scoping: you can place the functions in a namespace. For C++ compatibility, simply add a “using …” declaration for the namespace that you choose. (PyROOT does not support using, so no conflict possible.)

etejedor · May 29, 2018, 9:28am

Hi @phi,

The ROOT team is currently porting PyROOT on top of the new cppyy that @wlav mentioned, so new functionalities and bug fixes implemented by the latter will also be available in PyROOT, like the one you found here. For the moment, this new PyROOT will be added to ROOT master in experimental mode.

Cheers,
Enric

system · June 12, 2018, 9:28am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.