Not being able to write RVec<PtEtaPhiMVector> object into root file with RDF snapshot

yburkard · January 16, 2023, 4:49pm

Dear users,

I am trying to write the column Electrons (which is composed of objects of type ROOT::VecOps::RVec<PtEtaPhiMVector>) into a root file via RDataFrame:

df.Snapshot("tree","file_extended.root",["Electrons"])

However, I obtain the following error:

df.Snapshot("tree","file_extended.root",["Electrons"])
TypeError: Template method resolution failed:
  none of the 3 overloaded methods succeeded. Full details:
  ROOT::RDF::RResultPtr<ROOT::RDF::RInterface<ROOT::Detail::RDF::RLoopManager,void> > ROOT::RDF::RInterface<ROOT::Detail::RDF::RLoopManager,void>::Snapshot(basic_string_view<char,char_traits<char> > treename, basic_string_view<char,char_traits<char> > filename, initializer_list<string> columnList, const ROOT::RDF::RSnapshotOptions& options = ROOT::RDF::RSnapshotOptions()) =>
    TypeError: could not convert argument 3
  ROOT::RDF::RResultPtr<ROOT::RDF::RInterface<ROOT::Detail::RDF::RLoopManager,void> > ROOT::RDF::RInterface<ROOT::Detail::RDF::RLoopManager,void>::Snapshot(basic_string_view<char,char_traits<char> > treename, basic_string_view<char,char_traits<char> > filename, const vector<string>& columnList, const ROOT::RDF::RSnapshotOptions& options = ROOT::RDF::RSnapshotOptions()) =>
    logic_error: Trying to insert a null branch address.
  ROOT::RDF::RResultPtr<ROOT::RDF::RInterface<ROOT::Detail::RDF::RLoopManager,void> > ROOT::RDF::RInterface<ROOT::Detail::RDF::RLoopManager,void>::Snapshot(basic_string_view<char,char_traits<char> > treename, basic_string_view<char,char_traits<char> > filename, basic_string_view<char,char_traits<char> > columnNameRegexp = "", const ROOT::RDF::RSnapshotOptions& options = ROOT::RDF::RSnapshotOptions()) =>
    TypeError: could not convert argument 3
  ROOT::RDF::RResultPtr<ROOT::RDF::RInterface<ROOT::Detail::RDF::RLoopManager,void> > ROOT::RDF::RInterface<ROOT::Detail::RDF::RLoopManager,void>::Snapshot(basic_string_view<char,char_traits<char> > treename, basic_string_view<char,char_traits<char> > filename, initializer_list<string> columnList, const ROOT::RDF::RSnapshotOptions& options = ROOT::RDF::RSnapshotOptions()) =>
    TypeError: could not convert argument 3

Is there perhaps another way of doing this?

Any help would be much appreciated!

bellenot · January 16, 2023, 8:42pm

Maybe @vpadulan knows better, but can you try this syntax:

df.Snapshot("tree", "file_extended.root", {"Electrons"})

yburkard · January 16, 2023, 9:04pm

@bellenot thanks for the suggestion. I have tried it this way but the same error still persists.

bellenot · January 16, 2023, 9:09pm

Then let’s see if @vpadulan can help

vpadulan · January 18, 2023, 12:34pm

Hi @yburkard ,

Although it’s a bit hidden, the real error of your example is the line

logic_error: Trying to insert a null branch address.

In my case, I get a much more verbose error.

Error in <TTree::Branch>: The class requested (ROOT::VecOps::RVec<ROOT::Math::LorentzVector<ROOT::Math::PtEtaPhiM4D<double> > >) for the branch "Electrons" is an instance of an stl collection and does not have a compiled CollectionProxy. Please generate the dictionary for this collection (ROOT::VecOps::RVec<ROOT::Math::LorentzVector<ROOT::Math::PtEtaPhiM4D<double> > >) to avoid to write corrupted data.
RDataFrame::Run: event loop was interrupted
Error in <TTree::Branch>: The class requested (ROOT::VecOps::RVec<ROOT::Math::LorentzVector<ROOT::Math::PtEtaPhiM4D<double> > >) for the branch "Electrons" is an instance of an stl collection and does not have a compiled CollectionProxy. Please generate the dictionary for this collection (ROOT::VecOps::RVec<ROOT::Math::LorentzVector<ROOT::Math::PtEtaPhiM4D<double> > >) to avoid to write corrupted data.
RDataFrame::Run: event loop was interrupted
Traceback (most recent call last):
  File "/home/vpadulan/Projects/rootcode/forum-posts/53129/test.py", line 5, in <module>
    .Snapshot("tree", "file.root", ("Electrons"))
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

I’m using Python 3.11 and ROOT 6.26.10, what are your Python and ROOT versions?

Now, for an answer to your problem, this is another case of missing dictionaries for your custom collection class, as it has also happened in the past, for example here and here.

So you need to generate dictionaries for the class ROOT::VecOps::RVec<PtEtaPhiMVector>. The easies way for your original example could be the following

#include <ROOT/RDataFrame.hxx>
#include <iostream>
#include <ROOT/RVec.hxx>
#include <Math/Vector4D.h>
#include <TInterpreter.h>

int main()
{
   gInterpreter->GenerateDictionary("ROOT::RVec<ROOT::Math::PtEtaPhiMVector>", "Math/Vector4D.h;ROOT/RVec.hxx");
   ROOT::RDataFrame df{5};
   df.Define("Electrons",
             [] {
                return ROOT::RVec<ROOT::Math::PtEtaPhiMVector>{{1, 2, 3, 4}, {1, 2, 3, 4}};
             })
      .Snapshot("tree", "file.root", {"Electrons"});
}

Or the equivalent in Python:

import ROOT

if __name__ == "__main__":
    ROOT.gInterpreter.GenerateDictionary(
        "ROOT::RVec<ROOT::Math::PtEtaPhiMVector>", "Math/Vector4D.h;ROOT/RVec.hxx")
    ROOT.RDataFrame(5)\
        .Define("Electrons", "ROOT::RVec<ROOT::Math::PtEtaPhiMVector>{{1, 2, 3, 4}, {1, 2, 3, 4}}")\
        .Snapshot("tree", "file.root", ("Electrons"))

For more complicated custom classes, see the manual.
Cheers,
Vincenzo

yburkard · January 30, 2023, 2:19pm

Dear @vpadulan, many thanks for your detailed reply! Implementing your suggested changes locally worked fine (as I am using root 6.26.06 and python 3.10.7), but trying this on another server with root version 6.22.09 and python 2.7 does not work (probably because the python version is too old). Is there any change I can do so that the code is compatible with python 2.7?

vpadulan · January 30, 2023, 2:29pm

Dear @yburkard,

Of course the first question would be whether you could update both the ROOT and Python versions on the other server, as that makes things extremely easier as you could also see yourself. But I guess you already considered this possibility

In any case, I just tried the Python example I sent you above in a new environment with Python 2.7.15 and ROOT 6.22, everything works on my side. Try to give me more details about your errors.

Cheers,
Vincenzo

yburkard · January 30, 2023, 2:37pm

Dear @vpadulan, I actually have found a way to upgrade the python version to 3.7.4 on the external server, and after running the code again I still obtain the same error:

cc1plus: fatal error: /cluster/home/include/stdc-predef.h: Permission denied
compilation terminated.
c++: error: /cluster/home/yburkard/data_processing/ZH3l/AutoDict_ROOT__RVec_ROOT__Math__PtEtaPhiMVector__cxx_ACLiC_dict.o: No such file or directory
Error in <ACLiC>: Executing 'cd "/cluster/home/yburkard/data_processing/ZH3l" ; c++ -fPIC -c -O2 -DNDEBUG -std=c++11 -pipe -W -Woverloaded-virtual -fsigned-char -pthread  -I$ROOTSYS/include -I"/cluster/home/yburkard/root/etc/" -I"/cluster/home/yburkard/root/etc//cling" -I"/cluster/home/yburkard/root/include/" -I"/usr/include/python2.7" -I"/cluster/home/include"   -D__ACLIC__ "/cluster/home/yburkard/data_processing/ZH3l/AutoDict_ROOT__RVec_ROOT__Math__PtEtaPhiMVector__cxx_ACLiC_dict.cxx" ; c++ -O2 -DNDEBUG "/cluster/home/yburkard/data_processing/ZH3l/AutoDict_ROOT__RVec_ROOT__Math__PtEtaPhiMVector__cxx_ACLiC_dict.o" -shared   "/lib64/libc.so" "/lib64/libm.so" "/lib64/libdl.so" "/lib64/libpython2.7.so.1.0" "/lib64/libpthread.so.0" "/lib64/libutil.so.1" "/cluster/home/yburkard/root/lib/libcppyy2_7.so" "/cluster/home/yburkard/root/lib/libcppyy_backend2_7.so.6.22" "/cluster/home/yburkard/root/lib/libCore.so.6.22" "/lib64/libnss_files.so.2" "/lib64/libnss_sss.so.2" "/cluster/home/yburkard/root/lib/libRIO.so" "/cluster/home/yburkard/root/lib/libThread.so.6.22" "/cluster/home/yburkard/root/lib/libCling.so" "/lib64/librt.so.1" "/lib64/libtinfo.so.5" "/usr/lib64/python2.7/lib-dynload/_ctypes.so" "/lib64/libffi.so.6" "/usr/lib64/python2.7/lib-dynload/_struct.so" "/usr/lib64/python2.7/lib-dynload/timemodule.so" "/usr/lib64/python2.7/lib-dynload/selectmodule.so" "/usr/lib64/python2.7/lib-dynload/fcntlmodule.so" "/usr/lib64/python2.7/lib-dynload/binascii.so" "/usr/lib64/python2.7/lib-dynload/cStringIO.so" "/usr/lib64/python2.7/lib-dynload/stropmodule.so" "/usr/lib64/python2.7/lib-dynload/operator.so" "/usr/lib64/python2.7/lib-dynload/_collectionsmodule.so" "/usr/lib64/python2.7/lib-dynload/itertoolsmodule.so" "/usr/lib64/python2.7/lib-dynload/_heapq.so" "/cluster/home/yburkard/root/lib/libROOTPythonizations2_7.so" "/cluster/home/yburkard/root/lib/libTree.so.6.22" "/cluster/home/yburkard/root/lib/libNet.so.6.22" "/cluster/home/yburkard/root/lib/libMathCore.so.6.22" "/cluster/home/yburkard/root/lib/libImt.so.6.22" "/lib64/libssl.so.10" "/lib64/libcrypto.so.10" "/cluster/home/yburkard/root/lib/libtbb.so.2" "/lib64/libgssapi_krb5.so.2" "/lib64/libkrb5.so.3" "/lib64/libcom_err.so.2" "/lib64/libk5crypto.so.3" "/lib64/libz.so.1" "/lib64/libkrb5support.so.0" "/lib64/libkeyutils.so.1" "/lib64/libresolv.so.2" "/lib64/libselinux.so.1" "/lib64/libpcre.so.1" "/usr/lib64/python2.7/lib-dynload/_functoolsmodule.so" "/usr/lib64/ld-2.17.so" "/cluster/home/yburkard/root/lib/libtbbmalloc.so.2" "/cluster/home/yburkard/root/lib/libROOTDataFrame.so.6.22.99" "/cluster/home/yburkard/root/lib/libTreePlayer.so.6.22" "/cluster/home/yburkard/root/lib/libROOTVecOps.so.6.22" "/lib64/libsqlite3.so.0" "/cluster/home/yburkard/root/lib/libGraf3d.so.6.22" "/cluster/home/yburkard/root/lib/libGpad.so.6.22" "/cluster/home/yburkard/root/lib/libGraf.so.6.22" "/cluster/home/yburkard/root/lib/libHist.so.6.22" "/cluster/home/yburkard/root/lib/libMatrix.so.6.22" "/cluster/home/yburkard/root/lib/libMultiProc.so.6.22" "/cluster/home/yburkard/root/lib/libvdt.so" "/lib64/libfreetype.so.6" "/lib64/libbz2.so.1" "/lib64/libpng15.so.15" "/cluster/home/yburkard/root/lib/libPhysics.so.6.22.99" -o "/cluster/home/yburkard/data_processing/ZH3l/AutoDict_ROOT__RVec_ROOT__Math__PtEtaPhiMVector__cxx.so"' failed!

Seems like I am having restricted access to some packages…

vpadulan · January 30, 2023, 3:23pm

Yes, that is one issue. Another one is that I still see a lot of Python 2 in your error, e.g.:

So you should fix the permission error, then make sure you are using one single environment with the same Python version to both install ROOT and run anything with it. Currently you are mixing things up and that’s recipe for trouble.

system · February 13, 2023, 3:23pm

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.