ROOT's python bindings (SetBranchAddress not working)

[cygnus]

Hello,

Recently I have installed ROOT 5.34.36 using python 3.5. I used this python version to make ROOT compatible with another package environment which requires python 3.5. I provide the piece of code which is now giving me a problem. When I compiled ROOT 5.34.36 with python 2.7, the same piece of code worked perfectly. Could you please see how this problem can be solved?

import ROOT
from ROOT import *
-------
gSystem.Load("$PATHFOLDER/mylibrary.so")
f = ROOT.TFile("data.root")
# Getting 'Event' Tree
t=f.Get("Events")
# Setting branch address for Event class 
t.SetBranchStatus("*", 0);
t.SetBranchStatus("MEvt.", 1)
# calling the class (MEvt is a class from mylibrary.so)
cam_event = MEvt()

t.SetBranchAddress("MEvt.",cam_event)

It was working perfectly with python2.7. Now with this python3.5 version, it is giving me following error.

Traceback (most recent call last):
  File "read_root.py", line 28, in <module>
    t.SetBranchAddress("MEvt.",cam_event)
TypeError: none of the 3 overloaded methods succeeded. Full details:
  Int_t TTree::SetBranchAddress(const char* bname, void** add, TBranch** ptr = 0) =>
    could not convert argument 1 (SetBranchAddress() argument 1 must be bytes, not str)
  Int_t TTree::SetBranchAddress(const char* bname, void* add, TClass* realClass, EDataType datatype, Bool_t isptr) =>
    takes at least 5 arguments (2 given)
  Int_t TTree::SetBranchAddress(const char* bname, void* add, TBranch** ptr, TClass* realClass, EDataType datatype, Bool_t isptr) =>
    takes at least 6 arguments (2 given)

[couet]

Side note:

To show code you should simply enclose it with ```

Like:

```
The_code_you_want_to_show
```

it will produce:

The_code_you_want_to_show

the way you posted it made your post unreadable (I have fixed it)

[amadio]

In Python 3.x, the construct from ROOT import * is not supported. Please try to import only the things you need when using Python 3.x. If that still doesn’t work, let us know. @wlav may know better what is going on.

[cygnus]

I now imported only gSystem from ROOT to load my library. It is still showing me the same problem about SetBranchAddress.

import ROOT
from ROOT import gSystem
-------
gSystem.Load("$PATHFOLDER/mylibrary.so")
f = ROOT.TFile("data.root")
# Getting 'Event' Tree
t=f.Get("Events")
# Setting branch address for Event class 
t.SetBranchStatus("*", 0);
t.SetBranchStatus("MEvt.", 1)
# calling the class (MEvt is a class from mylibrary.so)
cam_event = MEvt()

t.SetBranchAddress("MEvt.",cam_event)

[cygnus]

@couet The piece of code in python script

import ROOT
from ROOT import gSystem
-------
gSystem.Load("$PATHFOLDER/mylibrary.so")
f = ROOT.TFile("data.root")
# Getting 'Event' Tree
t=f.Get("Events")
# Setting branch address for Event class 
t.SetBranchStatus("*", 0);
t.SetBranchStatus("MEvt.", 1)
# calling the class (MEvt is a class from mylibrary.so)
cam_event = MEvt()

t.SetBranchAddress("MEvt.",cam_event)

[wlav]

Plenty of completely unrelated things here.

First, @amadio this has nothing to do with “from ROOT import *”. As explained in detail in an e-mail to the ROOT team, the only thing broken in 3.6 (cygnus is using 3.5 here) is an interface change. Fixing that is so trivial that it will take less than 1/100th of the time already spend discussing the issue. Needless to say that after having set out all the details in that e-mail (and in part in the JIRA ticket), I’m no longer going to respond to anything related to “from ROOT import *”, as that is just a total waste of my time …

Second, @cygnus: the error is “must be bytes, not str”. This means that the p3 string, which is unicode, is not properly converted by TTreeSetBranchAddress in Pythonize.cxx. Again, that is a trivial fix, so you can file a JIRA ticket to whoever is in charge of PyROOT. For your own code, however, simply adding a “b” should be a solution (haven’t tried):

t.SetBranchAddress(b"MEvt.",cam_event)

What this does, is turning the unicode string into a bytes object, which consist of 1-byte chars, just ast TTree::SetBranchAddress likes it

Third, @cygnus if you do “import ROOT”, I’d recommend not mixing it with “from ROOT import …”. The problem is two-fold, first, things that appear late (b/c of the loading of mylibrary.so, for example), always need to be reimported or looked up on module ROOT, and cam_event = ROOT.MEvt() looks cleaner to my eyes than from ROOT import MEvt; cam_event = MEvt(). Of course, tastes vary, but the latter is moderately faster, too. Second, there is actually a ROOT C++ namespace. I.e. ROOT.ROOT. So being inconsistent can lead to confusing results as something that is available on module ROOT may not be on namespace ROOT (the opposite is never a problem: full namespace ROOT is available in module ROOT at the top level).

[cygnus]

@wlav Thanks a lot for your reply. I have tried with

t.SetBranchAddress(b"MEvt.",cam_event)

but it didn’t work. I got following error this time

Error in <TTree::SetBranchAddress>: unknown branch -> (null)
TypeError: bad argument type for built-in operation

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "read_root.py", line 31, in <module>
    t.SetBranchAddress(b"MEvt.",cam_event)
SystemError: <ROOT.MethodProxy object at 0x7f006ca9c4e0> returned a result with an error set

I would like to metion here that ‘MEvt’ is a class object from ‘mylibrary.so’. In an equivalent c-code I define this as follows

MEvt *cam_event = new MEvt();

However, in my python script I am just using

cam_event = MEvt()

This same piece of code worked fine when I compiled ROOT 5.34.36 with python 2.7. Hence I assume, I have defined it correctly. I am not sure if I need to call ‘MEvt’ in a different way.

[amadio]

I know this had nothing to do with the “from ROOT import *” issue. But it was decided on that JIRA issue, for the reasons you explained in your email about the limitations involved, that ROOT will no longer support that, not only in Python 3.6, but any version 3.x. I was informing the user and asking him to change his script. Since I’m not very familiar with PyROOT, I pinged you in case you’d know the answer to his actual problem. However, from your reaction, I see that it’s better to ask to someone else next time.

[wlav]

Hum … In Pythonize.cxx, TTreeSetBranchAddres does this:

            if ( PyArg_ParseTuple( args, const_cast< char* >( "SO:SetBranchAddress" ),
                    &name, &address ) ) {

So that ‘S’ is where it wants a bytes object, as per getargs.c in python:

    case 'S': { /* PyBytes object */
        PyObject **p = va_arg(*p_va, PyObject **);
        if (PyBytes_Check(arg))
            *p = arg;
        else
            return converterr("bytes", arg, msgbuf, bufsize);
        break;
    }

which was your first error message. Now that you provided that bytes argument, however, the ParseTuple code succeeds, but the next step is this:

                  tree->SetBranchAddress( PyROOT_PyUnicode_AsString( name ), buf );

As you can see, that now wants a unicode object, which fails, so in your second case, TTree gets a “nullptr” name (hence its complaint), and there is no error checking/handling, hence the rest of the complaints from python. (Ordinarily, no error checking would be needed as that’s what the “S” in the format was for.)

The solution is probably along the lines to redeclare the name variable as const char* and have ParseTuple take a s#. But that’s got to happen in Pythonize.cxx. Anyway, I’d argue it’s worth a JIRA ticket, as it’s a clear bug.

Only workaround that may still get you somewhere is (haven’t tried):

t.SetBranchAddress("MEvt.",ROOT.addressof(cam_event), ROOT.nullptr)

The idea is to fail the ParseTuple (by providing 3 arguments) and then call the normal (non-pythonized) SetBranchAddress with a proper address.

Alternatively, you can select the specific overload with disp(), although I’m not sure whether that will do the right thing (I forget how SetBranchAddress was inserted). Or if all that fails, if the final goal is to loop over the tree, than going through getattr(t, "MEvt.") might get you there (I just worry about that trailing dot syntax).

[wlav]

@amadio: Indeed. I’m interested in genuine p3 errors as the cppyy main development supports it b/c of active p3 users and some of the p3 errors latent in PyROOT may still be present in cppyy, so I want to know about (and fix) them. But general ROOT and PyROOT specific questions really have to be answered by the ROOT team.

[cygnus]

@wlav @amadio Thanks to you all for your support on this issue.

@wlav I already submitted a JIRA ticket for this issue.

I tried with the options you suggested.

t.SetBranchAddress("MEvt.",ROOT.addressof(cam_event), ROOT.nullptr)

didn’t work, Please find below the error message.

Traceback (most recent call last):
  File "read_root.py", line 48, in <module>
    t.SetBranchAddress("MEvt.",AddressOf(cam_event),ROOT.nullptr)
  File "/users/pkgs/root/lib/ROOT.py", line 486, in __getattr2
    attr = _root.LookupRootEntity( name )
AttributeError: nullptr

However, I was succesful with your other suggestion: getattr(t, "MEvt.") I am giving the code below in case other users find it useful

import ROOT
from ROOT import gSystem
-------
gSystem.Load("$PATHFOLDER/mylibrary.so")
f = ROOT.TFile("data.root")
# Getting 'Event' Tree
t=f.Get("Events")
# Setting branch address for Event class 
t.SetBranchStatus("*", 0);
t.SetBranchStatus("MEvt.", 1)
# calling the class (MEvt is a class from mylibrary.so)

for event in t:
    cam_event=getattr(event, "MEvt.")
    val = cam_event.MyFunction()

It worked well with the trailing dot still there.
Thanks a lot for your help.

[wlav]

@cygnus

AttributeError: nullptr

Only now did it hit me what version of ROOT you are using … 5.34. The particular problem here is still an issue w/ current ROOT6 (the code I quote above is from master), and hence the JIRA ticket is justified, but overall the support of p3 in 5.34 was left at “make it compile.” With 6 at least it came to “make all roottest tests and python tutorials succeed”. (And baseline cppyy has had some more p3 fixes since.)

All that said, if I check 5.34, there is a nullptr but also some comment of trouble next to it, and an alias of NULL to 0 (integer zero). ROOT.NULL may thus work instead.

[cygnus]

@wlav Thanks.
I tried with

t.SetBranchAddress("MEvt.",ROOT.addressof(cam_event), ROOT.null)

now I am getting similar errors as I got first time

Traceback (most recent call last):
  File "test_root.py", line 48, in <module>
    t.SetBranchAddress("MEvt.",AddressOf(cam_event),ROOT.NULL)
TypeError: none of the 3 overloaded methods succeeded. Full details:
  Int_t TTree::SetBranchAddress(const char* bname, void** add, TBranch** ptr = 0) =>
    could not convert argument 3
  Int_t TTree::SetBranchAddress(const char* bname, void* add, TClass* realClass, EDataType datatype, Bool_t isptr) =>
    takes at least 5 arguments (3 given)
  Int_t TTree::SetBranchAddress(const char* bname, void* add, TBranch** ptr, TClass* realClass, EDataType datatype, Bool_t isptr) =>
    takes at least 6 arguments (3 given)

[wlav]

Well, then I’ve run clean out of ideas … (I tried disp(), but that doesn’t work b/c the pythonized method hides the other ones).