Calling a tag-dispatched function from PyROOT does not work with a default tag argument

tmadlener · August 11, 2022, 12:14pm

Dear ROOT Team,

I have run across the following issue with PyROOT: It looks like I cannot call a tag-dispatched function with a default tag argument from Python, although it is possible from C++. From what I can tell the underlying issue is that template type deduction is not properly working in this case. The use case in which this came up is simplified below to produce a (not so) minimal reproducer:

The following is defined in RecoParticle.h and will be used in the examples below

struct Vector3f {
  float x;
  float y;
  float z;
};

struct RecoParticle {
  Vector3f momentum;
  float E;
  float M;
};

The actual functionality looks like this (defined in kinematics.h). The goal of this is to have a utils::p4 Function that can be used to get a 4-vector from the RecoParticle either using the mass or the energy that is stored in the particle by providing a tag. For better usability we provide a default tag, so that calling the function without a tag also works.

#include "Math/Vector4D.h"
#include <type_traits>

namespace utils {

using LorentzVectorE = ROOT::Math::PxPyPzEVector;
using LorentzVectorM = ROOT::Math::PxPyPzMVector;

namespace detail {
struct UseEnergyTag { using type = LorentzVectorE; };
struct UseMassTag { using type = LorentzVectorM; };

/** 
 * The tag dispatched function doing the actual work 
 */
template <typename ParticleT, typename LorentzVectorTypeTag>
inline typename LorentzVectorTypeTag::type p4(ParticleT const &particle, LorentzVectorTypeTag *tag) {
  const auto mom = particle.momentum;
  if constexpr (std::is_same_v<typename LorentzVectorTypeTag::type, LorentzVectorM>) {
    return LorentzVectorM{mom.x, mom.y, mom.z, particle.M};
  }
  if constexpr (std::is_same_v<typename LorentzVectorTypeTag::type, LorentzVectorE>) {
    return LorentzVectorE{mom.x, mom.y, mom.z, particle.E};
  }
}
} // namespace detail

/**
 * Two static tags for less typing when using things
 */
constexpr static detail::UseMassTag UseMass;
constexpr static detail::UseEnergyTag UseEnergy;

/**
 * The user facing function
 */
template <typename ParticleT, typename LorentzVectorTypeTag = detail::UseMassTag>
inline typename LorentzVectorTypeTag::type p4(ParticleT const &part, LorentzVectorTypeTag tag = UseMass) {
  return detail::p4(part, &tag);
}

}; // namespace utils

In c++ this can be used like this:

#include "RecoParticle.h"
#include "kinematics.h"

auto p = RecoParticle();
p.M = 125.f;
p.E = 3.14f;

const auto p4_M = utils::p4(p); // By default use the mass
const auto p4_E = utils::p4(p, utils::UseEnergy); // Can also use the energy instead

Trying to reproduce this with python breaks, when we try to call the version without the default tag argument:

import ROOT
ROOT.gInterpreter.LoadFile('kinematics.h')
ROOT.gInterpreter.LoadFile('RecoParticle.h')

from ROOT import utils
from ROOT import RecoParticle

p = RecoParticle()
p.M = 125
p.E = 3.14

# This works
p4_M = utils.p4(p)

# This fails
p4_E = utils.p4(p, utils.UseEnergy)

The error message (see full log below) indicates that the argument type deduction of the non-default tag argument fails and we eventually run into a conversion error.

I would have expected for this to work, but maybe I am missing some reason for it to not work?

Cheers,
Thomas

NB: Removing the default tag argument from the user facing p4 function and explicitly passing in tag arguments seems to work in c++ and python, so it really seems to be the default argument that is causing a problem here in python.

Full error message

In file included from input_line_35:1:
/home/tmadlener/work/playground/pyroot/issues/tag_dispatch/kinematics.h:44:48: error: no viable conversion from 'const detail::UseMassTag' to 'utils::detail::UseEnergyTag'
p4(ParticleT const &part, LorentzVectorTypeTag tag = UseMass) {
                                               ^     ~~~~~~~
input_line_43:8:81: note: in instantiation of default function argument expression for 'p4<RecoParticle, utils::detail::UseEnergyTag>' required here
         new (ret) (ROOT::Math::LorentzVector<ROOT::Math::PxPyPzE4D<double> >) (utils::p4<RecoParticle, utils::detail::UseEnergyTag>((const RecoParticle&)*(const RecoParticle*)args[0]));
                                                                                ^
/home/tmadlener/work/playground/pyroot/issues/tag_dispatch/kinematics.h:14:8: note: candidate constructor (the implicit copy constructor) not viable: no known conversion from 'const detail::UseMassTag' to 'const utils::detail::UseEnergyTag &' for 1st argument
struct UseEnergyTag {
       ^
/home/tmadlener/work/playground/pyroot/issues/tag_dispatch/kinematics.h:14:8: note: candidate constructor (the implicit move constructor) not viable: no known conversion from 'const detail::UseMassTag' to 'utils::detail::UseEnergyTag &&' for 1st argument
struct UseEnergyTag {
       ^
/home/tmadlener/work/playground/pyroot/issues/tag_dispatch/kinematics.h:44:48: note: passing argument to parameter 'tag' here
p4(ParticleT const &part, LorentzVectorTypeTag tag = UseMass) {
                                               ^
/home/tmadlener/work/playground/pyroot/issues/tag_dispatch/kinematics.h:44:48: error: no viable conversion from 'const detail::UseMassTag' to 'utils::detail::UseEnergyTag'
p4(ParticleT const &part, LorentzVectorTypeTag tag = UseMass) {
                                               ^     ~~~~~~~
input_line_43:12:17: note: in instantiation of default function argument expression for 'p4<RecoParticle, utils::detail::UseEnergyTag>' required here
         (void)(utils::p4<RecoParticle, utils::detail::UseEnergyTag>((const RecoParticle&)*(const RecoParticle*)args[0]));
                ^
/home/tmadlener/work/playground/pyroot/issues/tag_dispatch/kinematics.h:14:8: note: candidate constructor (the implicit copy constructor) not viable: no known conversion from 'const detail::UseMassTag' to 'const utils::detail::UseEnergyTag &' for 1st argument
struct UseEnergyTag {
       ^
/home/tmadlener/work/playground/pyroot/issues/tag_dispatch/kinematics.h:14:8: note: candidate constructor (the implicit move constructor) not viable: no known conversion from 'const detail::UseMassTag' to 'utils::detail::UseEnergyTag &&' for 1st argument
struct UseEnergyTag {
       ^
/home/tmadlener/work/playground/pyroot/issues/tag_dispatch/kinematics.h:44:48: note: passing argument to parameter 'tag' here
p4(ParticleT const &part, LorentzVectorTypeTag tag = UseMass) {
                                               ^
/home/tmadlener/work/playground/pyroot/issues/tag_dispatch/kinematics.h:44:48: error: no viable conversion from 'const detail::UseMassTag' to 'utils::detail::UseEnergyTag'
p4(ParticleT const &part, LorentzVectorTypeTag tag = UseMass) {
                                               ^     ~~~~~~~
note: in instantiation of default function argument expression for 'p4<RecoParticle, utils::detail::UseEnergyTag>' required here
/home/tmadlener/work/playground/pyroot/issues/tag_dispatch/kinematics.h:14:8: note: candidate constructor (the implicit copy constructor) not viable: no known conversion from 'const detail::UseMassTag' to 'const utils::detail::UseEnergyTag &' for 1st argument
struct UseEnergyTag {
       ^
/home/tmadlener/work/playground/pyroot/issues/tag_dispatch/kinematics.h:14:8: note: candidate constructor (the implicit move constructor) not viable: no known conversion from 'const detail::UseMassTag' to 'utils::detail::UseEnergyTag &&' for 1st argument
struct UseEnergyTag {
       ^
/home/tmadlener/work/playground/pyroot/issues/tag_dispatch/kinematics.h:44:48: note: passing argument to parameter 'tag' here
p4(ParticleT const &part, LorentzVectorTypeTag tag = UseMass) {
                                               ^
/home/tmadlener/work/playground/pyroot/issues/tag_dispatch/kinematics.h:44:48: error: no viable conversion from 'const detail::UseMassTag' to 'utils::detail::UseEnergyTag'
p4(ParticleT const &part, LorentzVectorTypeTag tag = UseMass) {
                                               ^     ~~~~~~~
input_line_44:8:81: note: in instantiation of default function argument expression for 'p4<RecoParticle, utils::detail::UseEnergyTag>' required here
         new (ret) (ROOT::Math::LorentzVector<ROOT::Math::PxPyPzE4D<double> >) (utils::p4<RecoParticle, utils::detail::UseEnergyTag>((const RecoParticle&)*(const RecoParticle*)args[0]));
                                                                                ^
/home/tmadlener/work/playground/pyroot/issues/tag_dispatch/kinematics.h:14:8: note: candidate constructor (the implicit copy constructor) not viable: no known conversion from 'const detail::UseMassTag' to 'const utils::detail::UseEnergyTag &' for 1st argument
struct UseEnergyTag {
       ^
/home/tmadlener/work/playground/pyroot/issues/tag_dispatch/kinematics.h:14:8: note: candidate constructor (the implicit move constructor) not viable: no known conversion from 'const detail::UseMassTag' to 'utils::detail::UseEnergyTag &&' for 1st argument
struct UseEnergyTag {
       ^
/home/tmadlener/work/playground/pyroot/issues/tag_dispatch/kinematics.h:44:48: note: passing argument to parameter 'tag' here
p4(ParticleT const &part, LorentzVectorTypeTag tag = UseMass) {
                                               ^
/home/tmadlener/work/playground/pyroot/issues/tag_dispatch/kinematics.h:44:48: error: no viable conversion from 'const detail::UseMassTag' to 'utils::detail::UseEnergyTag'
p4(ParticleT const &part, LorentzVectorTypeTag tag = UseMass) {
                                               ^     ~~~~~~~
input_line_44:12:17: note: in instantiation of default function argument expression for 'p4<RecoParticle, utils::detail::UseEnergyTag>' required here
         (void)(utils::p4<RecoParticle, utils::detail::UseEnergyTag>((const RecoParticle&)*(const RecoParticle*)args[0]));
                ^
/home/tmadlener/work/playground/pyroot/issues/tag_dispatch/kinematics.h:14:8: note: candidate constructor (the implicit copy constructor) not viable: no known conversion from 'const detail::UseMassTag' to 'const utils::detail::UseEnergyTag &' for 1st argument
struct UseEnergyTag {
       ^
/home/tmadlener/work/playground/pyroot/issues/tag_dispatch/kinematics.h:14:8: note: candidate constructor (the implicit move constructor) not viable: no known conversion from 'const detail::UseMassTag' to 'utils::detail::UseEnergyTag &&' for 1st argument
struct UseEnergyTag {
       ^
/home/tmadlener/work/playground/pyroot/issues/tag_dispatch/kinematics.h:44:48: note: passing argument to parameter 'tag' here
p4(ParticleT const &part, LorentzVectorTypeTag tag = UseMass) {
                                               ^
/home/tmadlener/work/playground/pyroot/issues/tag_dispatch/kinematics.h:44:48: error: no viable conversion from 'const detail::UseMassTag' to 'utils::detail::UseEnergyTag'
p4(ParticleT const &part, LorentzVectorTypeTag tag = UseMass) {
                                               ^     ~~~~~~~
note: in instantiation of default function argument expression for 'p4<RecoParticle, utils::detail::UseEnergyTag>' required here
/home/tmadlener/work/playground/pyroot/issues/tag_dispatch/kinematics.h:14:8: note: candidate constructor (the implicit copy constructor) not viable: no known conversion from 'const detail::UseMassTag' to 'const utils::detail::UseEnergyTag &' for 1st argument
struct UseEnergyTag {
       ^
/home/tmadlener/work/playground/pyroot/issues/tag_dispatch/kinematics.h:14:8: note: candidate constructor (the implicit move constructor) not viable: no known conversion from 'const detail::UseMassTag' to 'utils::detail::UseEnergyTag &&' for 1st argument
struct UseEnergyTag {
       ^
/home/tmadlener/work/playground/pyroot/issues/tag_dispatch/kinematics.h:44:48: note: passing argument to parameter 'tag' here
p4(ParticleT const &part, LorentzVectorTypeTag tag = UseMass) {
                                               ^
Traceback (most recent call last):
  File "reproducer.py", line 17, in <module>
    print(utils.p4(p, utils.UseEnergy))
TypeError: Template method resolution failed:
  ROOT::Math::LorentzVector<ROOT::Math::PxPyPzM4D<double> > utils::p4(const RecoParticle& part, utils::detail::UseMassTag tag = UseMass) =>
    TypeError: could not convert argument 2
  ROOT::Math::LorentzVector<ROOT::Math::PxPyPzE4D<double> > utils::p4(const RecoParticle& part, utils::detail::UseEnergyTag tag = UseMass) =>
    ValueError: nullptr result where temporary expected
  Failed to instantiate "p4(RecoParticle*,utils::detail::UseEnergyTag*)"
  ROOT::Math::LorentzVector<ROOT::Math::PxPyPzE4D<double> > utils::p4(const RecoParticle& part, utils::detail::UseEnergyTag tag = UseMass) =>
    ValueError: nullptr result where temporary expected

jonas · August 15, 2022, 12:07pm

Hi @tmadlener,

I have checked that this issue is also present in the newest cppyy, which pyROOT is based on:

import cppyy

cppyy.include("kinematics.h")
cppyy.include("RecoParticle.h")

# from ROOT import utils
# from ROOT import RecoParticle

p = cppyy.gbl.RecoParticle()
p.M = 125
p.E = 3.14

# This works
p4_M = cppyy.gbl.utils.p4(p)

# This fails
p4_E = cppyy.gbl.utils.p4(p, cppyy.gbl.utils.UseEnergy)

So it would be better do report this problem upsteam:

But actually, I would suggest you to just work around this corner case problem by rewriting your code. Your function looks like this right now:

template <typename ParticleT,
          typename LorentzVectorTypeTag = detail::UseMassTag>
inline typename LorentzVectorTypeTag::type p4(ParticleT const &part,
                                              LorentzVectorTypeTag tag=UseMass)
{
  return detail::p4(part, &tag);
}

Why not have two overloads like this? It’s probably less fragile, without the default template arguments:

template <typename ParticleT>
inline auto p4(ParticleT const &part) {
  return detail::p4(part, &UseMass);
}

template <typename ParticleT, typename LorentzVectorTypeTag>
inline auto p4(ParticleT const &part, LorentzVectorTypeTag tag) {
  return detail::p4(part, &tag);
}

Cheers,
Jonas

tmadlener · August 15, 2022, 1:16pm

Hi Jonas,

Thanks for checking this on the latest version of cppyy. I have opened an issue upstream:

github.com/wlav/cppyy

Default template arguments seem to break subsequent instantiations of the same template

opened 01:14PM - 15 Aug 22 UTC

tmadlener

This is potentially related to what has already been reported in #71, but I am n…ot entirely sure. It was also initially reported in the root-forum in [this thread](https://root-forum.cern.ch/t/calling-a-tag-dispatched-function-from-pyroot-does-not-work-with-a-default-tag-argument/51156) The thread has a workaround for the issue. I have made the minimal reproducer there slightly smaller and posted that version below here. The underlying issue is unchanged (as far as I can tell). Essentially what breaks is the following: ```python import cppyy cppyy.cppdef(''' #include <type_traits> struct RecoParticle { float x; float E; float M; }; struct LorentzVM { float x; float M; }; struct LorentzVE { float x; float E; }; namespace detail { struct LorentzVMTag { using type = LorentzVM; }; struct LorentzVETag { using type = LorentzVE; }; template <typename ParticleT, typename LorentzVTag> inline typename LorentzVTag::type p4(ParticleT const& particle, LorentzVTag*) { if constexpr (std::is_same_v<typename LorentzVTag::type, LorentzVM>) { return LorentzVM{particle.x, particle.M}; } if constexpr (std::is_same_v<typename LorentzVTag::type, LorentzVE>) { return LorentzVE{particle.x, particle.E}; } } } // namespace detail constexpr static detail::LorentzVMTag UseMass; constexpr static detail::LorentzVETag UseEnergy; template<typename ParticleT, typename LorentzVTag = detail::LorentzVMTag> inline auto p4(ParticleT const& particle, LorentzVTag tag = UseMass) { return detail::p4(particle, &tag); } ''') p = cppyy.gbl.RecoParticle() p.M = 125 p.E = 3.14 p4_M = cppyy.gbl.p4(p) # this works p4_E = cppyy.gbl.p4(p, cppyy.gbl.UseEnergy) # this does not ```

Why not have two overloads like this? It’s probably less fragile, without the default template arguments:

Since we haven’t really tested it via python this didn’t come up earlier. This is indeed something that works in this case. Thanks for the suggestion, I think we will go with this for now.

Cheers,
Thomas

system · August 29, 2022, 1:16pm

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.