Getting a python dict into a root macro in 6.08.06

ROOT
1

Hi,

so here are the constraints that i have to work with:

The tool i have to use at the moment uses root 6.08.6 and python 2.7.13

We have a root macro which is loaded from python via ROOT.gROOT.LoadMacro("GetPValue.cxx+")

We supply global settings to it by having the following in the macro:

namespace GetPValueOptions
{
  int NCalls = -1;
  bool ExternalConstraint = false;
...

and then setting

    if options.NCalls:
        ROOT.GetPValueOptions.NCalls = int(options.NCalls)

from within python.

However now there is a new setting that needs to be synchronized between the python and root parts which is a dictionary of the type Dict[str, list[str].

Currently the root side is maintained separately as a std::map<TString, std::vector<std::string>>

However i would like to not maintain them separately but instead set the root one from within python like the other settings.

What is the best way to do this in the given root/python version.

Cheers

2

Hi @JanEric,

I wonder whether you can just instantiate the std::map<...> in the GetPValueOptions namespace, and simply access it either via ROOT.GetPValueOptions.theMap.emplace() / ROOT.GetPValueOptions.theMap.find(). Let me know whether that works.

(also CC’ing @vpadulan)

Cheers,
J.

3

Hi,

thanks for the reply.

I have now tried to do this a couple ways but i am getting errors in each of them:

If i just do:

std::map<TString, std::vector<std::string>> NPfilter = {} and then in python

ROOT.GetPValueOptions.NPfilter.emplace("Test", ["Test"])
i get:

AttributeError: 'map<TString,vector<string> >' object has no attribute 'emplace'

if i change to

ROOT.GetPValueOptions.NPfilter.insert("Test", ["Test"])

i get

>       ROOT.GetPValueOptions.NPfilter.insert("Test", ["Test"])
E       TypeError: none of the 3 overloaded methods succeeded. Full details:
E         pair<_Rb_tree_iterator<pair<const TString,vector<string> > >,bool> map<TString,vector<string> >::insert(const pair<const TString,vector<string> >& __x) =>
E           takes at most 1 arguments (2 given)
E         void map<TString,vector<string> >::insert(initializer_list<pair<const TString,vector<string> > > __list) =>
E           takes at most 1 arguments (2 given)
E         _Rb_tree_iterator<pair<const TString,vector<string> > > map<TString,vector<string> >::insert(_Rb_tree_const_iterator<pair<const TString,vector<string> > > __position, const pair<const TString,vector<string> >& __x) =>
E           could not convert argument 1

same if i make the argument to insert one tuple.

I also dont know if the conversion list → vector just works in the given root version?

If for a test i change the map to std::map<std::string, std::string > NPfilter_test{};

and do ROOT.GetPValueOptions.NPfilter_filter.insert("Test", "Test")

i get an error: TypeError: can not resolve method template call for 'insert'

4

@jalopezg @vpadulan do you have any further ideas?

5

Dear @JanEric ,

There must be something being misused, this simple script works:

# macro.cpp
#ifndef MYMACRO
#define MYMACRO

#include <map>
#include <string>
#include <TString.h>
#include <vector>

namespace MyNamespace {
std::map<TString, std::vector<std::string>> mymap;
}

#endif // MYMACRO

# repro.py
import ROOT

ROOT.gROOT.LoadMacro("macro.cpp+")

mymap = ROOT.MyNamespace.mymap

mymap.emplace("myTString", ["a", "b", "c"])
mymap.insert((ROOT.TString("another"), ["d", "e", "f"]))

print(mymap)

# output
$: python repro.py
Info in <TUnixSystem::ACLiC>: creating shared library /home/vpadulan/Projects/rootcode/forum-posts/56025/./macro_cpp.so
{ @0x557e31da9840 => { "d", "e", "f" }, @0x557e31a41f90 => { "a", "b", "c" } }

Cheers,
Vincenzo

1 Like
6

When i run exactly that i get

Info in <TUnixSystem::ACLiC>: creating shared library /nfs/dust/atlas/user/jnitschk/CAF/StatisticTool/./macro_cpp.so
Traceback (most recent call last):
  File "repro.py", line 7, in <module>
    mymap.emplace("myTString", ["a", "b", "c"])
AttributeError: 'map<TString,vector<string> >' object has no attribute 'emplace'

For:

which python
/cvmfs/sft.cern.ch/lcg/releases/LCG_88/Python/2.7.13/x86_64-slc6-gcc49-opt/bin/python

python --version
Python 2.7.13

which root
/cvmfs/sft.cern.ch/lcg/releases/LCG_88/ROOT/6.08.06/x86_64-slc6-gcc49-opt/bin/root

 root -b -q
   ------------------------------------------------------------
  | Welcome to ROOT 6.08/06                http://root.cern.ch |
  |                               (c) 1995-2016, The ROOT Team |
  | Built for linuxx8664gcc                                    |
  | From tag v6-08-06, 2 March 2017                            |
  | Try '.help', '.demo', '.license', '.credits', '.quit'/'.q' |
   ------------------------------------------------------------

root [0]

The setup is

    source /cvmfs/sft.cern.ch/lcg/releases/LCG_88/root_numpy/4.6.0/x86_64-slc6-gcc49-opt/root_numpy-env.sh
    export CAFPATH=/nfs/dust/atlas/user/jnitschk/CAF
    export ATLAS_LOCAL_ROOT_BASE=/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase
    source ${ATLAS_LOCAL_ROOT_BASE}/user/atlasLocalSetup.sh -2
    lsetup "root 6.14.04-x86_64-slc6-gcc62-opt"
    #the line below loads root 6.08.06. The line above is only needed in order to avoid 'emacs: symbol lookup error'
    lsetup "boost boost-1.54.0-python2.7-x86_64-slc6-gcc47"
    export BOOST_INC=$ALRB_BOOST_ROOT/include
    export BOOST_LIB=$ALRB_BOOST_ROOT/lib
    export GTEST_CVMFS_LIB=/cvmfs/atlas.cern.ch/repo/sw/ASG/AnalysisBase/2.4.40/RootCore/obj/x86_64-slc6-gcc49-opt/Asg_GoogleTest
    export LD_LIBRARY_PATH=$PWD/lib:/cvmfs/atlas.cern.ch/repo/sw/ASG/AnalysisBase/2.4.40/RootCore/obj/x    86_64-slc6-gcc49-opt/Asg_GoogleTest/lib:$LD_LIBRARY_PATH
    source /cvmfs/sft.cern.ch/lcg/releases/LCG_88/root_numpy/4.6.0/x86_64-slc6-gcc49-opt/root_numpy-env.sh

    source /cvmfs/sft.cern.ch/lcg/releases/LCG_88/pytest/2.2.4/x86_64-slc6-gcc49-opt/pytest-env.sh
    source /cvmfs/sft.cern.ch/lcg/releases/LCG_88/coverage/3.5.2/x86_64-slc6-gcc49-opt/coverage-env.sh
    source /cvmfs/sft.cern.ch/lcg/releases/LCG_88/mock/0.8.0/x86_64-slc6-gcc49-opt/mock-env.sh
    cd $CAFPATH/StatisticTool
    export PATH=~/.local/bin:$PATH
    export PYTHONPATH=$PYTHONHOME:$PYTHONPATH
    unset PYTHONHOME
    for VARIABLE in ~/.cache/pre-commit/repo*/py_env-python3.8/bin/
    do
        export PATH=$PATH:$VARIABLE
    done
7

Indeed it seems that there are problem with the LCG88 environment you are using.

First off, I also see the

'map<TString,vector<string> >' object has no attribute 'emplace'

Which seems like a bug to me. Not present anymore in recent ROOT versions.

Then, when using the insert method, among the various errors that appear (and you also showed in a previous comment), the real culprit is

  pair<_Rb_tree_iterator<pair<const TString,vector<string> > >,bool> map<TString,vector<string> >::insert(const pair<const TString,vector<string> >& __x) =>
    could not convert argument 1

This tells you that the conversion of the argument mymap.insert((ROOT.TString("another"), ["d", "e", "f"])) doesn’t work. The reason why it doesn’t work is that the translation from a simple tuple (i.e. (ROOT.TString("another"), ["d", "e", "f"])) to an actual std::pair<TString, std::vector<std::string> is not supported (in that particular version of ROOT).

ROOT 6.08 is an ancient version of ROOT at this point and the Python<->C++ translation layer (cppyy) wasn’t as mature as today. Thus what you are experiencing doesn’t surprise me. I suggest you make sure to use the latest ROOT version possible.

A possible workaround, although extremely cumbersome, would be to take care of explicitly instantiating all the objects needed to satisfy the signature of map::insert, e.g.

v = ROOT.std.vector("string")()
v.push_back("el1")
v.push_back("el2")
p = ROOT.std.pair("const TString", "std::vector<std::string>")("theTString", v)
mymap.insert(p)

Cheers,
Vincenzo

8

Thanks a ton. Got it to work now. And i know that 6.08 is old, but sadly that is the setup that was passed down to me and without an absolute 100% understanding of the complex framework as well as effectively 0 test coverage it is tough to make a migration to a more recent LCG release with python3 and a modern root version. Especially while also working on other things.

Another thing to note for anyone else who might have a similar issue in the future is that i eventually switched to using std::strings als as the key and that i had to use map.find(key) instead of map[key] to get it to work.

1 Like
9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.