PyROOT and TUnfold

Newbie
1

Hello experts,
I am trying to use TUnfold within PyROOT and am having difficulties in using the ScanLcurve method with regards to the TGraph ** argument. When I create a TGraph in python and then try passing it with ROOT.AddressOf(myTGraph), the interpreter says that it cannot convert this argument for a TGraph **. When I just pass the TGraph object, it crashes. The error I get when passing the TGraph object is below:

Info in <TUnfold::SetConstraint>: fConstraint=1
TUnfold::TUnfold:0: RuntimeWarning: 18 output bins do not depend on the input data  0-1 33 36 38-51
Info in <TUnfold::TUnfold>: 100 input bins and 34 output bins
Info in <TUnfoldDensity::TUnfold>: *NOT* unfolding bin #0 (yaxis:[ufl])
Info in <TUnfoldDensity::TUnfold>: *NOT* unfolding bin #1 (yaxis:[0,16])
Info in <TUnfoldDensity::TUnfold>: *NOT* unfolding bin #33 (yaxis:[512,528])
Info in <TUnfoldDensity::TUnfold>: *NOT* unfolding bin #36 (yaxis:[560,576])
Info in <TUnfoldDensity::TUnfold>: *NOT* unfolding bin #38 (yaxis:[592,608])
Info in <TUnfoldDensity::TUnfold>: *NOT* unfolding bin #39 (yaxis:[608,624])
Info in <TUnfoldDensity::TUnfold>: *NOT* unfolding bin #40 (yaxis:[624,640])
Info in <TUnfoldDensity::TUnfold>: *NOT* unfolding bin #41 (yaxis:[640,656])
Info in <TUnfoldDensity::TUnfold>: *NOT* unfolding bin #42 (yaxis:[656,672])
Info in <TUnfoldDensity::TUnfold>: *NOT* unfolding bin #43 (yaxis:[672,688])
Info in <TUnfoldDensity::TUnfold>: *NOT* unfolding bin #44 (yaxis:[688,704])
Info in <TUnfoldDensity::TUnfold>: *NOT* unfolding bin #45 (yaxis:[704,720])
Info in <TUnfoldDensity::TUnfold>: *NOT* unfolding bin #46 (yaxis:[720,736])
Info in <TUnfoldDensity::TUnfold>: *NOT* unfolding bin #47 (yaxis:[736,752])
Info in <TUnfoldDensity::TUnfold>: *NOT* unfolding bin #48 (yaxis:[752,768])
Info in <TUnfoldDensity::TUnfold>: *NOT* unfolding bin #49 (yaxis:[768,784])
Info in <TUnfoldDensity::TUnfold>: *NOT* unfolding bin #50 (yaxis:[784,800])
Info in <TUnfoldDensity::TUnfold>: *NOT* unfolding bin #51 (yaxis:[ofl])
Info in <TUnfoldDensity::RegularizeOneDistribution>: regularizing yaxis regMode=3 densityMode=3 axisSteering=*[UOB]

 *** Break *** segmentation violation



===========================================================
There was a crash.
This is the entire stack trace of all threads:
===========================================================

Thread 2 (Thread 0x7f2a9ce38700 (LWP 24122)):
#0  0x00007f2aaca886d6 in futex_abstimed_wait_cancelable (private=0, abstime=0x0, expected=0, futex_word=0x56495f991180) at ../sysdeps/unix/sysv/linux/futex-internal.h:205
#1  do_futex_wait (sem=sem
entry=0x56495f991180, abstime=0x0) at sem_waitcommon.c:111
#2  0x00007f2aaca887c8 in __new_sem_wait_slow (sem=0x56495f991180, abstime=0x0) at sem_waitcommon.c:181
#3  0x000056495e20c634 in PyEval_RestoreThread ()
#4  0x000056495e2c984d in ?? ()
#5  0x000056495e2294ca in PyEval_EvalFrameEx ()
#6  0x000056495e226d0a in PyEval_EvalCodeEx ()
#7  0x000056495e2428bc in ?? ()
#8  0x000056495e21277e in PyObject_Call ()
#9  0x000056495e22ba21 in PyEval_EvalFrameEx ()
#10 0x000056495e22e232 in PyEval_EvalFrameEx ()
#11 0x000056495e22e232 in PyEval_EvalFrameEx ()
#12 0x000056495e226d0a in PyEval_EvalCodeEx ()
#13 0x000056495e242619 in ?? ()
#14 0x000056495e25b0de in ?? ()
#15 0x000056495e21277e in PyObject_Call ()
#16 0x000056495e232800 in PyEval_CallObjectWithKeywords ()
#17 0x000056495e2c6ba2 in ?? ()
#18 0x00007f2aaca7f6db in start_thread (arg=0x7f2a9ce38700) at pthread_create.c:463
#19 0x00007f2aacdb888f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 1 (Thread 0x7f2aad28b740 (LWP 24015)):
#0  0x00007f2aacd7b6c2 in __GI___waitpid (pid=24180, stat_loc=stat_loc
entry=0x7ffe5abee228, options=options
entry=0) at ../sysdeps/unix/sysv/linux/waitpid.c:30
#1  0x00007f2aacce6067 in do_system (line=<optimized out>) at ../sysdeps/posix/system.c:149
#2  0x00007f2aaa24d553 in TUnixSystem::Exec (shellcmd=<optimized out>, this=0x56495f87a360) at /home/aj/root-6.14.04/core/unix/src/TUnixSystem.cxx:2119
#3  TUnixSystem::StackTrace (this=0x56495f87a360) at /home/aj/root-6.14.04/core/unix/src/TUnixSystem.cxx:2413
#4  0x00007f2aaa24ff44 in TUnixSystem::DispatchSignals (this=0x56495f87a360, sig=kSigSegmentationViolation) at /home/aj/root-6.14.04/core/unix/src/TUnixSystem.cxx:3644
#5  <signal handler called>
#6  TUnfold::InvertMSparseSymmPos (this=this
entry=0x56496158f7a0, A=0x0, rankPtr=rankPtr
entry=0x7ffe5abf1144) at /home/aj/root-6.14.04/hist/unfold/src/TUnfold.cxx:1011
#7  0x00007f2a97a535e3 in TUnfold::GetInputInverseEmatrix (this=this
entry=0x56496158f7a0, out=out
entry=0x0) at /home/aj/root-6.14.04/hist/unfold/src/TUnfold.cxx:3066
#8  0x00007f2a97a55e47 in TUnfold::DoUnfold (this=0x56496158f7a0) at /home/aj/root-6.14.04/hist/unfold/src/TUnfold.cxx:296
#9  0x00007f2a97a59495 in TUnfold::DoUnfold (tau=0, this=0x56496158f7a0) at /home/aj/root-6.14.04/hist/unfold/src/TUnfold.cxx:2520
#10 TUnfold::ScanLcurve (this=0x56496158f7a0, nPoint=30, tauMin=<optimized out>, tauMax=<optimized out>, lCurve=0x7f2a9d9c5ba0, logTauX=0x0, logTauY=0x0, logTauCurvature=0x0) at /home/aj/root-6.14.04/hist/unfold/src/TUnfold.cxx:2584
#11 0x00007f2aad1a4093 in ?? ()
#12 0x00007ffe5abf17a0 in ?? ()
#13 0x00007f2aa525bceb in __gnu_cxx::new_allocator<char>::deallocate (this=0x7ffe5abf17d0, __p=<optimized out>) at /usr/include/c++/7/ext/new_allocator.h:125
#14 std::allocator_traits<std::allocator<char> >::deallocate (__a=..., __n=<optimized out>, __p=<optimized out>) at /usr/include/c++/7/bits/alloc_traits.h:462
#15 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_destroy (__size=<optimized out>, this=0x7ffe5abf17d0) at /usr/include/c++/7/bits/basic_string.h:226
#16 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_dispose (this=0x7ffe5abf17d0) at /usr/include/c++/7/bits/basic_string.h:221
#17 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string (this=0x7ffe5abf17d0, __in_chrg=<optimized out>) at /usr/include/c++/7/bits/basic_string.h:647
#18 TClingCallFunc::make_wrapper (this=<optimized out>) at /home/aj/root-6.14.04/core/metacling/src/TClingCallFunc.cxx:1088
#19 0x00007f2aab47c9fb in FastCall (method=<optimized out>, args_=<optimized out>, self=<optimized out>, result=result
entry=0x7ffe5abf1994) at /home/aj/root-6.14.04/bindings/pyroot/src/Cppyy.cxx:407
#20 0x00007f2aab47d41f in CallT<int> (args=<optimized out>, self=<optimized out>, method=<optimized out>) at /home/aj/root-6.14.04/bindings/pyroot/src/Cppyy.cxx:441
#21 Cppyy::CallI (method=<optimized out>, self=<optimized out>, args=<optimized out>) at /home/aj/root-6.14.04/bindings/pyroot/src/Cppyy.cxx:461
#22 0x00007f2aab480af0 in GILCallI (ctxt=<optimized out>, self=<optimized out>, method=<optimized out>) at /home/aj/root-6.14.04/bindings/pyroot/src/Executors.cxx:67
#23 PyROOT::TIntExecutor::Execute (this=<optimized out>, method=<optimized out>, self=<optimized out>, ctxt=<optimized out>) at /home/aj/root-6.14.04/bindings/pyroot/src/Executors.cxx:167
#24 0x00007f2aab4a5565 in PyROOT::TMethodHolder::CallFast (ctxt=0x7ffe5abf1d80, offset=0, self=0x56496158f7a0, this=0x56496216f9c0) at /home/aj/root-6.14.04/bindings/pyroot/src/TMethodHolder.cxx:69
#25 PyROOT::TMethodHolder::CallSafe (this=this
entry=0x56496216f9c0, self=self
entry=0x56496158f7a0, offset=0, ctxt=ctxt
entry=0x7ffe5abf1d80) at /home/aj/root-6.14.04/bindings/pyroot/src/TMethodHolder.cxx:121
#26 0x00007f2aab4a4cb8 in PyROOT::TMethodHolder::Execute (ctxt=0x7ffe5abf1d80, offset=<optimized out>, self=0x56496158f7a0, this=0x56496216f9c0) at /home/aj/root-6.14.04/bindings/pyroot/src/TMethodHolder.cxx:528
#27 PyROOT::TMethodHolder::Call (this=0x56496216f9c0, self=
0x7f2a9ce732a0: 0x7f2a9d9c5bf0, args=<optimized out>, kwds=<optimized out>, ctxt=0x7ffe5abf1d80) at /home/aj/root-6.14.04/bindings/pyroot/src/TMethodHolder.cxx:585
#28 0x00007f2aab487348 in PyROOT::(anonymous namespace)::mp_call (pymeth=0x7f2a9ce73290, args=0x7f2a9ce54838, kwds=0x0) at /home/aj/root-6.14.04/bindings/pyroot/src/MethodProxy.cxx:597
#29 0x000056495e22e420 in PyEval_EvalFrameEx ()
#30 0x000056495e226d0a in PyEval_EvalCodeEx ()
#31 0x000056495e226629 in PyEval_EvalCode ()
#32 0x000056495e25761f in ?? ()
#33 0x000056495e252322 in PyRun_FileExFlags ()
#34 0x000056495e25167d in PyRun_SimpleFileExFlags ()
#35 0x000056495e2001ab in Py_Main ()
#36 0x00007f2aaccb8b97 in __libc_start_main (main=0x56495e1ffb10 <main>, argc=3, argv=0x7ffe5abf22c8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffe5abf22b8) at ../csu/libc-start.c:310
#37 0x000056495e1ffa2a in _start ()
===========================================================


The lines below might hint at the cause of the crash.
You may get help by asking at the ROOT forum http://root.cern.ch/forum
Only if you are really convinced it is a bug in ROOT then please submit a
report at http://root.cern.ch/bugs Please post the ENTIRE stack trace
from above as an attachment in addition to anything else
that might help us fixing this issue.
===========================================================
#6  TUnfold::InvertMSparseSymmPos (this=this
entry=0x56496158f7a0, A=0x0, rankPtr=rankPtr
entry=0x7ffe5abf1144) at /home/aj/root-6.14.04/hist/unfold/src/TUnfold.cxx:1011
#7  0x00007f2a97a535e3 in TUnfold::GetInputInverseEmatrix (this=this
entry=0x56496158f7a0, out=out
entry=0x0) at /home/aj/root-6.14.04/hist/unfold/src/TUnfold.cxx:3066
#8  0x00007f2a97a55e47 in TUnfold::DoUnfold (this=0x56496158f7a0) at /home/aj/root-6.14.04/hist/unfold/src/TUnfold.cxx:296
#9  0x00007f2a97a59495 in TUnfold::DoUnfold (tau=0, this=0x56496158f7a0) at /home/aj/root-6.14.04/hist/unfold/src/TUnfold.cxx:2520
#10 TUnfold::ScanLcurve (this=0x56496158f7a0, nPoint=30, tauMin=<optimized out>, tauMax=<optimized out>, lCurve=0x7f2a9d9c5ba0, logTauX=0x0, logTauY=0x0, logTauCurvature=0x0) at /home/aj/root-6.14.04/hist/unfold/src/TUnfold.cxx:2584
#11 0x00007f2aad1a4093 in ?? ()
#12 0x00007ffe5abf17a0 in ?? ()
#13 0x00007f2aa525bceb in __gnu_cxx::new_allocator<char>::deallocate (this=0x7ffe5abf17d0, __p=<optimized out>) at /usr/include/c++/7/ext/new_allocator.h:125
#14 std::allocator_traits<std::allocator<char> >::deallocate (__a=..., __n=<optimized out>, __p=<optimized out>) at /usr/include/c++/7/bits/alloc_traits.h:462
#15 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_destroy (__size=<optimized out>, this=0x7ffe5abf17d0) at /usr/include/c++/7/bits/basic_string.h:226
#16 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_dispose (this=0x7ffe5abf17d0) at /usr/include/c++/7/bits/basic_string.h:221
#17 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string (this=0x7ffe5abf17d0, __in_chrg=<optimized out>) at /usr/include/c++/7/bits/basic_string.h:647
#18 TClingCallFunc::make_wrapper (this=<optimized out>) at /home/aj/root-6.14.04/core/metacling/src/TClingCallFunc.cxx:1088
#19 0x00007f2aab47c9fb in FastCall (method=<optimized out>, args_=<optimized out>, self=<optimized out>, result=result
entry=0x7ffe5abf1994) at /home/aj/root-6.14.04/bindings/pyroot/src/Cppyy.cxx:407
#20 0x00007f2aab47d41f in CallT<int> (args=<optimized out>, self=<optimized out>, method=<optimized out>) at /home/aj/root-6.14.04/bindings/pyroot/src/Cppyy.cxx:441
#21 Cppyy::CallI (method=<optimized out>, self=<optimized out>, args=<optimized out>) at /home/aj/root-6.14.04/bindings/pyroot/src/Cppyy.cxx:461
#22 0x00007f2aab480af0 in GILCallI (ctxt=<optimized out>, self=<optimized out>, method=<optimized out>) at /home/aj/root-6.14.04/bindings/pyroot/src/Executors.cxx:67
#23 PyROOT::TIntExecutor::Execute (this=<optimized out>, method=<optimized out>, self=<optimized out>, ctxt=<optimized out>) at /home/aj/root-6.14.04/bindings/pyroot/src/Executors.cxx:167
#24 0x00007f2aab4a5565 in PyROOT::TMethodHolder::CallFast (ctxt=0x7ffe5abf1d80, offset=0, self=0x56496158f7a0, this=0x56496216f9c0) at /home/aj/root-6.14.04/bindings/pyroot/src/TMethodHolder.cxx:69
#25 PyROOT::TMethodHolder::CallSafe (this=this
entry=0x56496216f9c0, self=self
entry=0x56496158f7a0, offset=0, ctxt=ctxt
entry=0x7ffe5abf1d80) at /home/aj/root-6.14.04/bindings/pyroot/src/TMethodHolder.cxx:121
#26 0x00007f2aab4a4cb8 in PyROOT::TMethodHolder::Execute (ctxt=0x7ffe5abf1d80, offset=<optimized out>, self=0x56496158f7a0, this=0x56496216f9c0) at /home/aj/root-6.14.04/bindings/pyroot/src/TMethodHolder.cxx:528
#27 PyROOT::TMethodHolder::Call (this=0x56496216f9c0, self=
0x7f2a9ce732a0: 0x7f2a9d9c5bf0, args=<optimized out>, kwds=<optimized out>, ctxt=0x7ffe5abf1d80) at /home/aj/root-6.14.04/bindings/pyroot/src/TMethodHolder.cxx:585
#28 0x00007f2aab487348 in PyROOT::(anonymous namespace)::mp_call (pymeth=0x7f2a9ce73290, args=0x7f2a9ce54838, kwds=0x0) at /home/aj/root-6.14.04/bindings/pyroot/src/MethodProxy.cxx:597
#29 0x000056495e22e420 in PyEval_EvalFrameEx ()
#30 0x000056495e226d0a in PyEval_EvalCodeEx ()
#31 0x000056495e226629 in PyEval_EvalCode ()
#32 0x000056495e25761f in ?? ()
#33 0x000056495e252322 in PyRun_FileExFlags ()
#34 0x000056495e25167d in PyRun_SimpleFileExFlags ()
#35 0x000056495e2001ab in Py_Main ()
#36 0x00007f2aaccb8b97 in __libc_start_main (main=0x56495e1ffb10 <main>, argc=3, argv=0x7ffe5abf22c8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffe5abf22b8) at ../csu/libc-start.c:310
#37 0x000056495e1ffa2a in _start ()
===========================================================


Traceback (most recent call last):
  File "test.py", line 25, in <module>
    iBest = unfold.ScanLcurve(nscan, taumin, taumax, lsCurve)
SystemError: int TUnfold::ScanLcurve(int nPoint, double tauMin, double tauMax, TGraph** lCurve, TSpline** logTauX = 0, TSpline** logTauY = 0, TSpline** logTauCurvature = 0) =>
    problem in C++; program state has been reset
2

Hi @AWildridge

You should call ScanLcurve with myTGraph, not with ROOT.AddressOf(myTGraph).

Could you share a minimal reproducer of the issue here? Have you tried to do the same in C++?

Cheers,
Enric

3

Why is that when it wants a double pointer? I actually just tried that yesterday and got the same issue…I’m guessing it has something to do with my TH2D then.

#Import ROOT File and TTree
root_file = TFile.Open(sys.argv[1])
tree = root_file.Get("tree")
histmc = TH2D("histmc", "histmc", 100, 0, 800, 50, 0, 800)

entries = tree.GetEntries()
for i in range(entries):
    tree.GetEntry(i)
    histmc.Fill(tree.gen_l_pt, tree.l_pt)

#Unfold
nscan = 30
taumin = 0
taumax = 0
lsCurve = TGraph()

unfold = ROOT.TUnfoldDensity(histmc, TUnfold.kHistMapOutputVert)
iBest = unfold.ScanLcurve(nscan, taumin, taumax, lsCurve)

If that’s the case then I probably need to just spend more time understanding how TUnfold works…

4

The TGraph object that you have in Python is actually a proxy object which points to the real C++ object. If the method you call expects a double pointer, that proxy is able to provide it, so you do not need to worry about it.

So something else is going on. I translated the code to C++ macro and I see the same crash:

{
TH2D histmc("histmc", "histmc", 100, 0, 800, 50, 0, 800);

for (int i = 0; i < 10; i++) {
    histmc.Fill(1., 2.);
}

int nscan = 30;
int taumin = 0;
int taumax = 0;
auto lsCurve = new TGraph();

TUnfoldDensity unfold(&histmc, TUnfold::kHistMapOutputVert);

auto iBest = unfold.ScanLcurve(nscan, taumin, taumax, &lsCurve);
}

Perhaps @moneta knows what the issue could be?

5

Hi
I am reviving this problem thread. Can anyone please comment on the bug or the possible solution?
@AWildridge @etejedor or @moneta

Thanks in advance

6

So I can comment on it as I figured out the problem (of my case at least). So I added correctly underflow and overflow then the bin count was correct and my bug is gone. I hope it might help someone in future :slight_smile:

7

Thanks for sharing the solution ! Pinging @moneta in case he wants to comment.

1 Like
8

Hello,
Can you please explain better what the solution was ? How did you modify the code above to avoid the crash ?
Thank you for this, it will be helpful to understand it and eventually fix any remaining bug

Best regards

Lorenzo