Updated pyROOT in current master (6.23/01)

LeWhoo · October 14, 2020, 6:29am

Hello,

This is a follow-up to ROOT 6.22.02 and pyROOT GUI to which I can’t reply any more, because it is closed.
Thanks for the update! I’ve compiled, it works, but there are still issues. Some of them are probably bugs in ROOT, some of them may be bugs in my code that were tollerated by the previous pyROOT. Sorry if I can’t always distinguish between them.

Before I needed to create a custom class in C++ to handle TGListView events in pyROOT. I am not sure what should be done now. For sure, the old way of creating the custom class does not work anymore, because:

/home/lewhoo/workspace/etos/cpp_helpers/etos_dispatcher.C:3:10: fatal error: 'TPyDispatcher.h' file not found

When I moved mouse over my toolbar, I’ve got the following error and the crash. Unfortunately it does not say where in my code it happened:

input_line_317:1:64: error: redefinition of 'is_equal'
namespace __cppyy_internal { template<class C1, class C2> bool is_equal(const C1& c1, const C2& c2) { return (bool)(c1 == c2); } }
                                                               ^
input_line_13:1:64: note: previous definition is here
namespace __cppyy_internal { template<class C1, class C2> bool is_equal(const C1& c1, const C2& c2) { return (bool)(c1 == c2); } }
                                                               ^
input_line_318:1:64: error: redefinition of 'is_not_equal'
namespace __cppyy_internal { template<class C1, class C2> bool is_not_equal(const C1& c1, const C2& c2) { return (bool)(c1 != c2); } }
                                                               ^
input_line_14:1:64: note: previous definition is here
namespace __cppyy_internal { template<class C1, class C2> bool is_not_equal(const C1& c1, const C2& c2) { return (bool)(c1 != c2); } }
                                                               ^
 *** Break *** segmentation violation

I’ve noticed that the same happens when exiting the gui_ex.py from the tutorial:

lewhoo@lwpcomp2 pyroot]$ python2 -i gui_ex.py 
>>> input_line_92:1:64: error: redefinition of 'is_equal'
namespace __cppyy_internal { template<class C1, class C2> bool is_equal(const C1& c1, const C2& c2) { return (bool)(c1 == c2); } }
                                                               ^
input_line_13:1:64: note: previous definition is here
namespace __cppyy_internal { template<class C1, class C2> bool is_equal(const C1& c1, const C2& c2) { return (bool)(c1 == c2); } }
                                                               ^
input_line_93:1:64: error: redefinition of 'is_not_equal'
namespace __cppyy_internal { template<class C1, class C2> bool is_not_equal(const C1& c1, const C2& c2) { return (bool)(c1 != c2); } }
                                                               ^
input_line_14:1:64: note: previous definition is here
namespace __cppyy_internal { template<class C1, class C2> bool is_not_equal(const C1& c1, const C2& c2) { return (bool)(c1 != c2); } }

This may not be a bug, but for some reason, a long time ago, I needed to replace the MapSubwindows() function in my class deriving form TGMainFrame with:

	# Hook over the original MapSubwindows - it shows even the hidden frames
	def MapSubwindows(self):
		super(pMainFrame,self).MapSubwindows()
		if self.cur_avg_gtus<2:
			self.MainFrame.HideFrame(self.AvgNavigationBar)

This doesn’t work anymore:

  File "/home/lewhoo/workspace/etos/etoswindows/mainwindow.py", line 2332, in MapSubwindows
    super(pMainFrame,self).MapSubwindows()
RuntimeError: void TGCompositeFrame::MapSubwindows() =>
    RuntimeError: void TGCompositeFrame::MapSubwindows() =>
    RuntimeError: void TGCompositeFrame::MapSubwindows() =>
...
    RuntimeError: void TGCompositeFrame::MapSubwindows() =>
    RuntimeError: maximum recursion depth exceeded while calling a Python object

I just commented out this replacement and my program seems to work OK, but I can’t be sure yet, especially that I am not sure what was happening a long time ago without this replacement. However, maybe it will be some hint to you

Probably a feature, not a bug, or rather a bug in my code that is not tolareted any longer. Still, it is an incompatibility, so I report.

I have my main class deriving from the TGMainFrame, and inside the init() I set:
self.data_tree = data_tree
later, I create a panel in the same init:

self.DrawPanelDock = ROOT.TGDockableFrame(self.MainHorFrame, widgetsIds.wDrawPanelDock)
self.draw_panel = pDrawPanel(self.DrawPanelDock, self.DrawPanelDock)

where

class pDrawPanel(ROOT.TGMainFrame):
	## Constructor
	def __init__(self, parent, main_window):
...
		ROOT.TGMainFrame.__init__(self, parent, width, height)
		self.main_window = main_window.GetParent().GetParent()

The last line was giving in the old pyroot object of type:
<class 'etoswindows.mainwindow.pMainFrame'>
and now is giving:
<class cppyy.gbl.TGMainFrame at 0x55a922433eb0>
And the current object does not have the attributes of my main window that I set in its constructor. Not a big problem, I just pass the main window to the init function of the dock as I should have done probably from the beginning, but I report this different behaviour.

More testing will come, I guess, after fixing the bug (2), as at the moment it prevents my app for working properly

etejedor · October 14, 2020, 10:02am

Hello @LeWhoo,

Thank you for the complete report! My answers below:

Please #include "ROOT/TPyDispatcher.h" instead, that one should be there. So I understand you are using TPyDispatcher from C++, instead of just using it to program callbacks from Python?
This indicates that libcppyyX_Y.so is being loaded twice, which should not happen. Do you have a multi-Python (2&3) ROOT build? It could be that, for some reason, libcppyy2_7 and libcppyy3_X are both being loaded. I couldn’t reproduce it in a master multi-python build when running gui_ex.py with Python2 as you did. You say this happens when you do Ctrl+D on the Python interpreter?
It looks like it is calling recursively the MapSubwindows in Python, instead of calling the method of the superclass. This doesn’t look right, I’ll investigate. Good that it does not seem to block you though.
What is the type of main_window.GetParent() ? It looks like it’s giving you a Python proxy of type TGMainFrame which points to the right C++ object, but not a pMainFrame proxy. It could be that old PyROOT found already a Python proxy for that C++ object in its cache (a pMainFrame proxy), but that does not seem to happen now, so you get a brand new TGMainFrame proxy.

LeWhoo · October 14, 2020, 2:18pm

Thanks. This fixes it, although the first time I ran it I got:

Info in <TUnixSystem::ACLiC>: creating shared library /home/lewhoo/workspace/etos/cpp_helpers/etos_dispatcher_C.so
cling::DynamicLibraryManager::loadLibrary(): libPyROOT.so: cannot open shared object file: No such file or directory

Afterwards, it disappeared. The small inconvenience is that ROOT/TPyDispatcher.h is incompatible with old pyROOT. Not terrible if we are making the switch, however, if compatibility could be maintained with small effort, this could help.

I use TPyDispatcher in C++ only when something is missing in Python. That was the case for some TGListView signals, and this solution has been suggested by wlav here:

It’s been a long time ago… has it been updated?

Yes, I have a multi-Python build now. This happend when I press the “Exit” button in the app after running “python2 -i gui_ex.py”. It exits OK when I press Ctrl+D. The sample exits ok with the button in python2 though.

Thanks. Tell me if I can help.

Both in the old and new pyROOT it is <class ‘ROOT.TGFrame’>

Thanks for your answers!

etejedor · October 14, 2020, 3:10pm

Hello,

Another round of answers

We placed the headers we export of the new PyROOT in ROOT/, which is what we are doing with new headers in ROOT. I’m sorry for the inconvenience!

1b. I see, the Dispatch overload that you mentioned in that other ticket seems to be there in the current version of TPyDispatcher:

github.com

root-project/root/blob/master/bindings/pyroot/pythonizations/inc/TPyDispatcher.h#L94


PyObject *Dispatch(ULong_t param) { return DispatchVA("k", param); }
// ULong_t also for Handle_t (and Window_t, etc. ... )

PyObject *Dispatch(Event_t *event) { return DispatchVA1("Event_t", event, 0); }
PyObject *Dispatch(Event_t *event, ULong_t wid) { return DispatchVA1("Event_t", event, "k", wid); }
PyObject *Dispatch(TEveDigitSet *qs, Int_t idx) { return DispatchVA1("TEveDigitSet", qs, "i", idx); }
PyObject *Dispatch(TEveElement *el) { return DispatchVA1("TEveElement", el, 0); }
PyObject *Dispatch(TEveTrack *et) { return DispatchVA1("TEveTrack", et, 0); }
PyObject *Dispatch(TEveWindow *window) { return DispatchVA1("TEveWindow", window, 0); }
PyObject *Dispatch(TGFrame *frame) { return DispatchVA1("TGFrame", frame, 0); }
PyObject *Dispatch(TGFrame *frame, Int_t btn) { return DispatchVA1("TGFrame", frame, "i", btn); }
PyObject *Dispatch(TGFrame *frame, Int_t btn, Int_t x, Int_t y)
{
   return DispatchVA1("TGFrame", frame, "iii", btn, x, y);
}
PyObject *Dispatch(TGFrame *frame, UInt_t keysym, UInt_t mask)
{
   return DispatchVA1("TGFrame", frame, "II", keysym, mask);
}
PyObject *Dispatch(TGListTreeItem *entry) { return DispatchVA1("TGListTreeItem", entry, 0); }
PyObject *Dispatch(TGListTreeItem *entry, UInt_t mask) { return DispatchVA1("TGListTreeItem", entry, "I", mask); }

so perhaps you don’t need to subclass it anymore?

I understand it now. When pressing the Exit button, some Python code is executed with TPython:
https://github.com/root-project/root/blob/master/tutorials/pyroot/gui_ex.py#L51

In a multi-python build, libROOTTPython is only built for the highest Python version. Bottomline: if you run TPython from Python, make sure your Python version is the one that TPython was built for. That is why Python3 works fine.

I propose to follow this in the issue I just opened for it:
https://github.com/root-project/root/issues/6637

I was able to reproduce it standalone.

This must be related to caching of the Python proxy for your pMainFrame. Do you construct the pMainFrame object and, after that, you run self.main_window = main_window.GetParent().GetParent()?

LeWhoo · October 14, 2020, 3:48pm

While I may not need it, I guess I could somehow #ifdef this include based on a ROOT version. Could you advise what ifdef I should make?

I hope so, would make my code much simpler I’ll check.

So I understand, that currently code that includes “TPython” is not universal - I can’t use it with both python2 and python3. Do you think it is fixable?

However, maybe I don’t need it anymore. I needed it to handle some of the window mouse events - I think tracking the position of the cursor, etc (but I am not completely sure at the moment what it does in the code). I was pointed to this solution somewhere on the forum or in the documentation, but I can’t find where exactly. Anyway, in the code I have:

import __main__                                                                
__main__.slicer = self.handle_mouse_canvas                                     
self.Canvas = ROOT.TRootEmbeddedCanvas('Canvas', self.MainFrame, 200, 500)
self.Canvas.GetCanvas().AddExec('dynamic', 'TPython::Exec( "slicer()" );')

Do you think it could be replaced by something that doesn’t use a TPython call?

OK, thanks.

Hmm, honestly, I am not sure, because I am not sure where in python the construction finishes. The call to the self.main_window = main_window.GetParent().GetParent() happens in an init of another object that is created in the pMainFrame init(). Thus it should be executed before the pMainFrame init() finishes.

etejedor · October 15, 2020, 12:51pm

Hello,

While I may not need it, I guess I could somehow #ifdef this include based on a ROOT version. Could you advise what ifdef I should make?

This will be available from 6.22/04 (in 6.22/00 and 02, the inheritance issues prevented the programming of GUIs from Python, as we discussed in a previous post).

So I understand, that currently code that includes “TPython” is not universal - I can’t use it with both python2 and python3. Do you think it is fixable?

It is universal, but in a multi-python build, TPython is only available for the highest Python version (3).
If you have a Python2-only build, TPython will be built and work with Python2 (since it’s the highest). So it’s not a matter of changing the scripts depending on the Python version, it’s more about being aware of what version TPython is built for. So you can still use TPython in your code, but avoid running your scripts with Python2 if you built ROOT for both Python3 and Python2. You can run with Python2 if you so wish, but then build ROOT with Python2 only. The instructions to do so are here:

OK, thanks.

About 3, I investigated more and I updated the ticket:

github.com/root-project/root

Infinite recursion in cross inheritance when invoking method of base class

opened 02:45PM - 14 Oct 20 UTC

etejedor

bug affects:master affects:6.22 in:PyROOT

### Describe the bug Reported in: https://root-forum.cern.ch/t/updated-pyroo…t-in-current-master-6-23-01 The following is a standalone reproducer. When running the code below: ```python import cppyy cppyy.cppdef(""" class MyCppClass { public: virtual void MyMethod() { cout << "In C++" << endl; } virtual ~MyCppClass() {} }; """) class MyPyClass(cppyy.gbl.MyCppClass): def MyMethod(self): super(MyPyClass, self).MyMethod() print("In Python") a = MyPyClass() a.MyMethod() ``` it goes in infinite recursion when invoking `a.MyMethod()`, in particular in `super(MyPyClass, self).MyMethod()`, which calls again the `MyMethod` defined in Python. ### Expected behavior That `super(MyPyClass, self).MyMethod()` invokes `MyCppClass::MyMethod` (behaviour of old PyROOT). ### To Reproduce Described above. ### Setup ROOT 6.23/01, Ubuntu18, compiled from sources.

It could be a TCling issue, but I need @Axel to confirm and he will be off for a few days. I’ll check with him when he is back.

Hmm, honestly, I am not sure, because I am not sure where in python the construction finishes. The call to the self.main_window = main_window.GetParent().GetParent() happens in an init of another object that is created in the pMainFrame init(). Thus it should be executed before the pMainFrame init() finishes.

If the pMainFrame object is not completely constructed when you do main_window.GetParent().GetParent(), then I would say it’s normal that you can’t get its proxy. Old PyROOT could have done some early caching of that proxy to make it work.

LeWhoo · October 15, 2020, 1:28pm

I understand. It means, that python scripts that include TPython calls can not benefit from the multi-python builds. In my case, it means I can not benefit from the multi-python build, because I am forced to use a TPython call. Thus I am asking if in future it is possible that TPython will be built against both python version and thus will be usable in my case. If not, a shame, but I understand.

etejedor · October 16, 2020, 9:18am

I understand. It means, that python scripts that include TPython calls can not benefit from the multi-python builds. In my case, it means I can not benefit from the multi-python build, because I am forced to use a TPython call. Thus I am asking if in future it is possible that TPython will be built against both python version and thus will be usable in my case. If not, a shame, but I understand.

Indeed, if your Python script calls TPython, you can only use the TPython Python.

The problem with TPython is that we generate a precompiled module (PCM) for its library. If we generated libROOTTPython2_X and libROOTPython3_Y, ROOT would load both of its PCMs and there would be a conflict since they would define the same symbols.

Since it will be soon one year after the end of life of Python2, I think it’s best to look forward and focus our efforts on being compatible with new Python3.X versions. Having libROOTTPython built only for Python3 in a multi-Python build also goes in that direction.

Coming back to your slicer() example, I was thinking that you could actually try to get rid of the TPython calls by using NumbaDeclare. I don’t know if it will work for all your TPython cases, but it is worth a try. It is only available since 6.22, and here’s a tutorial about it:

https://root.cern/doc/master/pyroot004__NumbaDeclare_8py.html

In short, if you decorate a Python function with NumbaDeclare, ROOT will generate a C++ wrapper for it under the hood with numba. So if you did:

ROOT.Numba.Declare([], 'void')
def slicer():
    ...

and then:

self.Canvas.GetCanvas().AddExec('dynamic', 'Numba::slicer()" );')

So you would be calling a C++ wrapper for your Python function, instead of asking TPython to call your Python function.

system · October 30, 2020, 9:18am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.