DataFrame roofit dataset maker with mixture of column types crashes

rquaglia · September 20, 2024, 4:37pm

Dear experts,
i am trying to create a workspace where i want to store a roodataset. Such dataset is expected to store O(40) columns or so so that i can reduce and adjust slices a posteriori as well as make sPlot directly without having to go back and forth with my original ntuple.

I am doing this with some operations on rdataframe, and a list of columns to use.

When i trigger the event loop i get the following error;

In module 'RooFitRDataFrameHelpers':
/cvmfs/lhcbdev.cern.ch/conda/envs/default/2023-08-17_17-13/linux-64/include/RooAbsDataHelper.h:134:23: error: deduced conflicting types ('double' vs 'float') for initializer list element type
    for (auto&& val : {values...}) {
                      ^~~~~~~
/cvmfs/lhcbdev.cern.ch/conda/envs/default/2023-08-17_17-13/linux-64/include/ROOT/RDF/RAction.hxx:105:15: note: in instantiation of function template specialization 'RooAbsDataHelper<RooDataSet>::Exec<double, float, float, float, float, float, float, unsigned int, unsigned int, float, float, unsigned int, float, unsigned int, float, float, double, unsigned int, unsigned int, unsigned int, float, unsigned int, float, float, float, unsigned int, unsigned int, float, float, unsigned int, unsigned int, float, int, float, float, float, unsigned int, unsigned int, float, unsigned int, float, unsigned int, float, float, float, float, bool, bool, float, float, float, int, bool, bool, bool, bool, bool, bool, float, float, int, double, float, float, float, double, double, int, int, int, int, float, int, float, float, float, float, float, float, bool, bool, float, float, float, float, float, float, float, float, float, float, float, float, bool, bool, float, float, float, float, float, float, float, float, float, bool, bool, bool, bool, bool, bool, bool, bool, bool, int, bool, bool, bool, int, float, float, float, float, float, double, double, float, float, int, unsigned long long, unsigned int, unsigned int, int, int, int, int, int, int, int, int, bool, bool>' requested here
      fHelper.Exec(slot, fValues[slot][S]->template Get<ColTypes>(entry)...);
              ^
/cvmfs/lhcbdev.cern.ch/conda/envs/default/2023-08-17_17-13/linux-64/include/ROOT/RDF/RAction.hxx:113:10: note: in instantiation of function template specialization 'ROOT::Internal::RDF::RAction<RooAbsDataHelper<RooDataSet>, ROOT::Detail::RDF::RNodeBase, ROOT::TypeTraits::TypeList<double, float, float, float, float, float, float, unsigned int, unsigned int, float, float, unsigned int, float, unsigned int, float, float, double, unsigned int, unsigned int, unsigned int, float, unsigned int, float, float, float, unsigned int, unsigned int, float, float, unsigned int, unsigned int, float, int, float, float, float, unsigned int, unsigned int, float, unsigned int, float, unsigned int, float, float, float, float, bool, bool, float, float, float, int, bool, bool, bool, bool, bool, bool, float, float, int, double, float, float, float, double, double, int, int, int, int, float, int, float, float, float, float, float, float, bool, bool, float, float, float, float, float, float, float, float, float, float, float, float, bool, bool, float, float, float, float, float, float, float, float, float, bool, bool, bool, bool, bool, bool, bool, bool, bool, int, bool, bool, bool, int, float, float, float, float, float, double, double, float, float, int, unsigned long long, unsigned int, unsigned int, int, int, int, int, int, int, int, int, bool, bool> >::CallExec<double, float, float, float, float, float, float, unsigned int, unsigned int, float, float, unsigned int, float, unsigned int, float, float, double, unsigned int, unsigned int, unsigned int, float, unsigned int, float, float, float, unsigned int, unsigned int, float, float, unsigned int, unsigned int, float, int, float, float, float, unsigned int, unsigned int, float, unsigned int, float, unsigned int, float, float, float, float, bool, bool, float, float, float, int, bool, bool, bool, bool, bool, bool, float, float, int, double, float, float, float, double, double, int, int, int, int, float, int, float, float, float, float, float, float, bool, bool, float, float, float, float, float, float, float, float, float, float, float, float, bool, bool, float, float, float, float, float, float, float, float, float, bool, bool, bool, bool, bool, bool, bool, bool, bool, int, bool, bool, bool, int, float, float, float, float, float, double, double, float, float, int, unsigned long long, unsigned int, unsigned int, int, int, int, int, int, int, int, int, bool, bool, 0UL, 1UL, 2UL, 3UL, 4UL, 5UL, 6UL, 7UL, 8UL, 9UL, 10UL, 11UL, 12UL, 13UL, 14UL, 15UL, 16UL, 17UL, 18UL, 19UL, 20UL, 21UL, 22UL, 23UL, 24UL, 25UL, 26UL, 27UL, 28UL, 29UL, 30UL, 31UL, 32UL, 33UL, 34UL, 35UL, 36UL, 37UL, 38UL, 39UL, 40UL, 41UL, 42UL, 43UL, 44UL, 45UL, 46UL, 47UL, 48UL, 49UL, 50UL, 51UL, 52UL, 53UL, 54UL, 55UL, 56UL, 57UL, 58UL, 59UL, 60UL, 61UL, 62UL, 63UL, 64UL, 65UL, 66UL, 67UL, 68UL, 69UL, 70UL, 71UL, 72UL, 73UL, 74UL, 75UL, 76UL, 77UL, 78UL, 79UL, 80UL, 81UL, 82UL, 83UL, 84UL, 85UL, 86UL, 87UL, 88UL, 89UL, 90UL, 91UL, 92UL, 93UL, 94UL, 95UL, 96UL, 97UL, 98UL, 99UL, 100UL, 101UL, 102UL, 103UL, 104UL, 105UL, 106UL, 107UL, 108UL, 109UL, 110UL, 111UL, 112UL, 113UL, 114UL, 115UL, 116UL, 117UL, 118UL, 119UL, 120UL, 121UL, 122UL, 123UL, 124UL, 125UL, 126UL, 127UL, 128UL, 129UL, 130UL, 131UL, 132UL, 133UL, 134UL, 135UL, 136UL, 137UL, 138UL, 139UL, 140UL>' requested here
         CallExec(slot, entry, ColumnTypes_t{}, TypeInd_t{});
         ^
/cvmfs/lhcbdev.cern.ch/conda/envs/default/2023-08-17_17-13/linux-64/include/ROOT/RDF/RAction.hxx:67:4: note: in instantiation of member function 'ROOT::Internal::RDF::RAction<RooAbsDataHelper<RooDataSet>, ROOT::Detail::RDF::RNodeBase, ROOT::TypeTraits::TypeList<double, float, float, float, float, float, float, unsigned int, unsigned int, float, float, unsigned int, float, unsigned int, float, float, double, unsigned int, unsigned int, unsigned int, float, unsigned int, float, float, float, unsigned int, unsigned int, float, float, unsigned int, unsigned int, float, int, float, float, float, unsigned int, unsigned int, float, unsigned int, float, unsigned int, float, float, float, float, bool, bool, float, float, float, int, bool, bool, bool, bool, bool, bool, float, float, int, double, float, float, float, double, double, int, int, int, int, float, int, float, float, float, float, float, float, bool, bool, float, float, float, float, float, float, float, float, float, float, float, float, bool, bool, float, float, float, float, float, float, float, float, float, bool, bool, bool, bool, bool, bool, bool, bool, bool, int, bool, bool, bool, int, float, float, float, float, float, double, double, float, float, int, unsigned long long, unsigned int, unsigned int, int, int, int, int, int, int, int, int, bool, bool> >::Run' requested here
   RAction(Helper &&h, const ColumnNames_t &columns, std::shared_ptr<PrevNode> pd, const RColumnRegister &colRegister)
   ^
/home/conda/feedstock_root/build_artifacts/root_base_1690542718525/_build_env/x86_64-conda-linux-gnu/include/c++/12.3.0/bits/unique_ptr.h:1065:34: note: in instantiation of member function 'ROOT::Internal::RDF::RAction<RooAbsDataHelper<RooDataSet>, ROOT::Detail::RDF::RNodeBase, ROOT::TypeTraits::TypeList<double, float, float, float, float, float, float, unsigned int, unsigned int, float, float, unsigned int, float, unsigned int, float, float, double, unsigned int, unsigned int, unsigned int, float, unsigned int, float, float, float, unsigned int, unsigned int, float, float, unsigned int, unsigned int, float, int, float, float, float, unsigned int, unsigned int, float, unsigned int, float, unsigned int, float, float, float, float, bool, bool, float, float, float, int, bool, bool, bool, bool, bool, bool, float, float, int, double, float, float, float, double, double, int, int, int, int, float, int, float, float, float, float, float, float, bool, bool, float, float, float, float, float, float, float, float, float, float, float, float, bool, bool, float, float, float, float, float, float, float, float, float, bool, bool, bool, bool, bool, bool, bool, bool, bool, int, bool, bool, bool, int, float, float, float, float, float, double, double, float, float, int, unsigned long long, unsigned int, unsigned int, int, int, int, int, int, int, int, int, bool, bool> >::RAction' requested here
    { return unique_ptr<_Tp>(new _Tp(std::forward<_Args>(__args)...)); }
                                 ^
/cvmfs/lhcbdev.cern.ch/conda/envs/default/2023-08-17_17-13/linux-64/include/ROOT/RDF/InterfaceUtils.hxx:304:16: note: in instantiation of function template specialization 'std::make_unique<ROOT::Internal::RDF::RAction<RooAbsDataHelper<RooDataSet>, ROOT::Detail::RDF::RNodeBase, ROOT::TypeTraits::TypeList<double, float, float, float, float, float, float, unsigned int, unsigned int, float, float, unsigned int, float, unsigned int, float, float, double, unsigned int, unsigned int, unsigned int, float, unsigned int, float, float, float, unsigned int, unsigned int, float, float, unsigned int, unsigned int, float, int, float, float, float, unsigned int, unsigned int, float, unsigned int, float, unsigned int, float, float, float, float, bool, bool, float, float, float, int, bool, bool, bool, bool, bool, bool, float, float, int, double, float, float, float, double, double, int, int, int, int, float, int, float, float, float, float, float, float, bool, bool, float, float, float, float, float, float, float, float, float, float, float, float, bool, bool, float, float, float, float, float, float, float, float, float, bool, bool, bool, bool, bool, bool, bool, bool, bool, int, bool, bool, bool, int, float, float, float, float, float, double, double, float, float, int, unsigned long long, unsigned int, unsigned int, int, int, int, int, int, int, int, int, bool, bool> >, RooAbsDataHelper<RooDataSet>, const std::vector<std::basic_string<char>, std::allocator<std::basic_string<char> > > &, std::shared_ptr<ROOT::Detail::RDF::RNodeBase>, const ROOT::Internal::RDF::RColumnRegister &>' requested here
   return std::make_unique<Action_t>(Helper_t(std::move(*h)), bl, std::move(prevNode), colRegister);
               ^
/cvmfs/lhcbdev.cern.ch/conda/envs/default/2023-08-17_17-13/linux-64/include/ROOT/RDF/InterfaceUtils.hxx:618:21: note: in instantiation of function template specialization 'ROOT::Internal::RDF::BuildAction<double, float, float, float, float, float, float, unsigned int, unsigned int, float, float, unsigned int, float, unsigned int, float, float, double, unsigned int, unsigned int, unsigned int, float, unsigned int, float, float, float, unsigned int, unsigned int, float, float, unsigned int, unsigned int, float, int, float, float, float, unsigned int, unsigned int, float, unsigned int, float, unsigned int, float, float, float, float, bool, bool, float, float, float, int, bool, bool, bool, bool, bool, bool, float, float, int, double, float, float, float, double, double, int, int, int, int, float, int, float, float, float, float, float, float, bool, bool, float, float, float, float, float, float, float, float, float, float, float, float, bool, bool, float, float, float, float, float, float, float, float, float, bool, bool, bool, bool, bool, bool, bool, bool, bool, int, bool, bool, bool, int, float, float, float, float, float, double, double, float, float, int, unsigned long long, unsigned int, unsigned int, int, int, int, int, int, int, int, int, bool, bool, ROOT::Detail::RDF::RNodeBase, RooAbsDataHelper<RooDataSet> >' requested here
   auto actionPtr = BuildAction<ColTypes...>(cols, std::move(*helperArgOnHeap), nSlots, std::move(prevNodePtr),
                    ^
input_line_111:11:22: note: in instantiation of function template specialization 'ROOT::Internal::RDF::CallBuildAction<ROOT::Internal::RDF::ActionTags::Book, double, float, float, float, float, float, float, unsigned int, unsigned int, float, float, unsigned int, float, unsigned int, float, float, double, unsigned int, unsigned int, unsigned int, float, unsigned int, float, float, float, unsigned int, unsigned int, float, float, unsigned int, unsigned int, float, int, float, float, float, unsigned int, unsigned int, float, unsigned int, float, unsigned int, float, float, float, float, bool, bool, float, float, float, int, bool, bool, bool, bool, bool, bool, float, float, int, double, float, float, float, double, double, int, int, int, int, float, int, float, float, float, float, float, float, bool, bool, float, float, float, float, float, float, float, float, float, float, float, float, bool, bool, float, float, float, float, float, float, float, float, float, bool, bool, bool, bool, bool, bool, bool, bool, bool, int, bool, bool, bool, int, float, float, float, float, float, double, double, float, float, int, unsigned long long, unsigned int, unsigned int, int, int, int, int, int, int, int, int, bool, bool, ROOT::Detail::RDF::RNodeBase, RooAbsDataHelper<RooDataSet> >' requested here
In module 'ROOTDataFrame':
/cvmfs/lhcbdev.cern.ch/conda/envs/default/2023-08-17_17-13/linux-64/include/ROOT/RDF/RVariedAction.hxx:119:24: error: no matching member function for call to 'Exec'
      fHelpers[varIdx].Exec(slot, fInputValues[slot][varIdx][S]->template Get<ColTypes>(entry)...);
      ~~~~~~~~~~~~~~~~~^~~~
/cvmfs/lhcbdev.cern.ch/conda/envs/default/2023-08-17_17-13/linux-64/include/ROOT/RDF/RVariedAction.hxx:127:13: note: in instantiation of function template specialization 'ROOT::Internal::RDF::RVariedAction<RooAbsDataHelper<RooDataSet>, ROOT::Detail::RDF::RNodeBase, ROOT::TypeTraits::TypeList<double, float, float, float, float, float, float, unsigned int, unsigned int, float, float, unsigned int, float, unsigned int, float, float, double, unsigned int, unsigned int, unsigned int, float, unsigned int, float, float, float, unsigned int, unsigned int, float, float, unsigned int, unsigned int, float, int, float, float, float, unsigned int, unsigned int, float, unsigned int, float, unsigned int, float, float, float, float, bool, bool, float, float, float, int, bool, bool, bool, bool, bool, bool, float, float, int, double, float, float, float, double, double, int, int, int, int, float, int, float, float, float, float, float, float, bool, bool, float, float, float, float, float, float, float, float, float, float, float, float, bool, bool, float, float, float, float, float, float, float, float, float, bool, bool, bool, bool, bool, bool, bool, bool, bool, int, bool, bool, bool, int, float, float, float, float, float, double, double, float, float, int, unsigned long long, unsigned int, unsigned int, int, int, int, int, int, int, int, int, bool, bool> >::CallExec<double, float, float, float, float, float, float, unsigned int, unsigned int, float, float, unsigned int, float, unsigned int, float, float, double, unsigned int, unsigned int, unsigned int, float, unsigned int, float, float, float, unsigned int, unsigned int, float, float, unsigned int, unsigned int, float, int, float, float, float, unsigned int, unsigned int, float, unsigned int, float, unsigned int, float, float, float, float, bool, bool, float, float, float, int, bool, bool, bool, bool, bool, bool, float, float, int, double, float, float, float, double, double, int, int, int, int, float, int, float, float, float, float, float, float, bool, bool, float, float, float, float, float, float, float, float, float, float, float, float, bool, bool, float, float, float, float, float, float, float, float, float, bool, bool, bool, bool, bool, bool, bool, bool, bool, int, bool, bool, bool, int, float, float, float, float, float, double, double, float, float, int, unsigned long long, unsigned int, unsigned int, int, int, int, int, int, int, int, int, bool, bool, 0UL, 1UL, 2UL, 3UL, 4UL, 5UL, 6UL, 7UL, 8UL, 9UL, 10UL, 11UL, 12UL, 13UL, 14UL, 15UL, 16UL, 17UL, 18UL, 19UL, 20UL, 21UL, 22UL, 23UL, 24UL, 25UL, 26UL, 27UL, 28UL, 29UL, 30UL, 31UL, 32UL, 33UL, 34UL, 35UL, 36UL, 37UL, 38UL, 39UL, 40UL, 41UL, 42UL, 43UL, 44UL, 45UL, 46UL, 47UL, 48UL, 49UL, 50UL, 51UL, 52UL, 53UL, 54UL, 55UL, 56UL, 57UL, 58UL, 59UL, 60UL, 61UL, 62UL, 63UL, 64UL, 65UL, 66UL, 67UL, 68UL, 69UL, 70UL, 71UL, 72UL, 73UL, 74UL, 75UL, 76UL, 77UL, 78UL, 79UL, 80UL, 81UL, 82UL, 83UL, 84UL, 85UL, 86UL, 87UL, 88UL, 89UL, 90UL, 91UL, 92UL, 93UL, 94UL, 95UL, 96UL, 97UL, 98UL, 99UL, 100UL, 101UL, 102UL, 103UL, 104UL, 105UL, 106UL, 107UL, 108UL, 109UL, 110UL, 111UL, 112UL, 113UL, 114UL, 115UL, 116UL, 117UL, 118UL, 119UL, 120UL, 121UL, 122UL, 123UL, 124UL, 125UL, 126UL, 127UL, 128UL, 129UL, 130UL, 131UL, 132UL, 133UL, 134UL, 135UL, 136UL, 137UL, 138UL, 139UL, 140UL>' requested here
            CallExec(slot, varIdx, entry, ColumnTypes_t{}, TypeInd_t{});
            ^
/cvmfs/lhcbdev.cern.ch/conda/envs/default/2023-08-17_17-13/linux-64/include/ROOT/RDF/RVariedAction.hxx:79:4: note: in instantiation of member function 'ROOT::Internal::RDF::RVariedAction<RooAbsDataHelper<RooDataSet>, ROOT::Detail::RDF::RNodeBase, ROOT::TypeTraits::TypeList<double, float, float, float, float, float, float, unsigned int, unsigned int, float, float, unsigned int, float, unsigned int, float, float, double, unsigned int, unsigned int, unsigned int, float, unsigned int, float, float, float, unsigned int, unsigned int, float, float, unsigned int, unsigned int, float, int, float, float, float, unsigned int, unsigned int, float, unsigned int, float, unsigned int, float, float, float, float, bool, bool, float, float, float, int, bool, bool, bool, bool, bool, bool, float, float, int, double, float, float, float, double, double, int, int, int, int, float, int, float, float, float, float, float, float, bool, bool, float, float, float, float, float, float, float, float, float, float, float, float, bool, bool, float, float, float, float, float, float, float, float, float, bool, bool, bool, bool, bool, bool, bool, bool, bool, int, bool, bool, bool, int, float, float, float, float, float, double, double, float, float, int, unsigned long long, unsigned int, unsigned int, int, int, int, int, int, int, int, int, bool, bool> >::Run' requested here
   RVariedAction(std::vector<Helper> &&helpers, const ColumnNames_t &columns, std::shared_ptr<PrevNode> prevNode,
   ^
/cvmfs/lhcbdev.cern.ch/conda/envs/default/2023-08-17_17-13/linux-64/include/ROOT/RDF/RAction.hxx:167:47: note: in instantiation of member function 'ROOT::Internal::RDF::RVariedAction<RooAbsDataHelper<RooDataSet>, ROOT::Detail::RDF::RNodeBase, ROOT::TypeTraits::TypeList<double, float, float, float, float, float, float, unsigned int, unsigned int, float, float, unsigned int, float, unsigned int, float, float, double, unsigned int, unsigned int, unsigned int, float, unsigned int, float, float, float, unsigned int, unsigned int, float, float, unsigned int, unsigned int, float, int, float, float, float, unsigned int, unsigned int, float, unsigned int, float, unsigned int, float, float, float, float, bool, bool, float, float, float, int, bool, bool, bool, bool, bool, bool, float, float, int, double, float, float, float, double, double, int, int, int, int, float, int, float, float, float, float, float, float, bool, bool, float, float, float, float, float, float, float, float, float, float, float, float, bool, bool, float, float, float, float, float, float, float, float, float, bool, bool, bool, bool, bool, bool, bool, bool, bool, int, bool, bool, bool, int, float, float, float, float, float, double, double, float, float, int, unsigned long long, unsigned int, unsigned int, int, int, int, int, int, int, int, int, bool, bool> >::RVariedAction' requested here
      return std::unique_ptr<RActionBase>(new RVariedAction<Helper, PrevNode, ColumnTypes_t>{
                                              ^
/cvmfs/lhcbdev.cern.ch/conda/envs/default/2023-08-17_17-13/linux-64/include/ROOT/RDF/RAction.hxx:67:4: note: in instantiation of member function 'ROOT::Internal::RDF::RAction<RooAbsDataHelper<RooDataSet>, ROOT::Detail::RDF::RNodeBase, ROOT::TypeTraits::TypeList<double, float, float, float, float, float, float, unsigned int, unsigned int, float, float, unsigned int, float, unsigned int, float, float, double, unsigned int, unsigned int, unsigned int, float, unsigned int, float, float, float, unsigned int, unsigned int, float, float, unsigned int, unsigned int, float, int, float, float, float, unsigned int, unsigned int, float, unsigned int, float, unsigned int, float, float, float, float, bool, bool, float, float, float, int, bool, bool, bool, bool, bool, bool, float, float, int, double, float, float, float, double, double, int, int, int, int, float, int, float, float, float, float, float, float, bool, bool, float, float, float, float, float, float, float, float, float, float, float, float, bool, bool, float, float, float, float, float, float, float, float, float, bool, bool, bool, bool, bool, bool, bool, bool, bool, int, bool, bool, bool, int, float, float, float, float, float, double, double, float, float, int, unsigned long long, unsigned int, unsigned int, int, int, int, int, int, int, int, int, bool, bool> >::MakeVariedAction' requested here
   RAction(Helper &&h, const ColumnNames_t &columns, std::shared_ptr<PrevNode> pd, const RColumnRegister &colRegister)
   ^
/home/conda/feedstock_root/build_artifacts/root_base_1690542718525/_build_env/x86_64-conda-linux-gnu/include/c++/12.3.0/bits/unique_ptr.h:1065:34: note: in instantiation of member function 'ROOT::Internal::RDF::RAction<RooAbsDataHelper<RooDataSet>, ROOT::Detail::RDF::RNodeBase, ROOT::TypeTraits::TypeList<double, float, float, float, float, float, float, unsigned int, unsigned int, float, float, unsigned int, float, unsigned int, float, float, double, unsigned int, unsigned int, unsigned int, float, unsigned int, float, float, float, unsigned int, unsigned int, float, float, unsigned int, unsigned int, float, int, float, float, float, unsigned int, unsigned int, float, unsigned int, float, unsigned int, float, float, float, float, bool, bool, float, float, float, int, bool, bool, bool, bool, bool, bool, float, float, int, double, float, float, float, double, double, int, int, int, int, float, int, float, float, float, float, float, float, bool, bool, float, float, float, float, float, float, float, float, float, float, float, float, bool, bool, float, float, float, float, float, float, float, float, float, bool, bool, bool, bool, bool, bool, bool, bool, bool, int, bool, bool, bool, int, float, float, float, float, float, double, double, float, float, int, unsigned long long, unsigned int, unsigned int, int, int, int, int, int, int, int, int, bool, bool> >::RAction' requested here
    { return unique_ptr<_Tp>(new _Tp(std::forward<_Args>(__args)...)); }
                                 ^
/cvmfs/lhcbdev.cern.ch/conda/envs/default/2023-08-17_17-13/linux-64/include/ROOT/RDF/InterfaceUtils.hxx:304:16: note: in instantiation of function template specialization 'std::make_unique<ROOT::Internal::RDF::RAction<RooAbsDataHelper<RooDataSet>, ROOT::Detail::RDF::RNodeBase, ROOT::TypeTraits::TypeList<double, float, float, float, float, float, float, unsigned int, unsigned int, float, float, unsigned int, float, unsigned int, float, float, double, unsigned int, unsigned int, unsigned int, float, unsigned int, float, float, float, unsigned int, unsigned int, float, float, unsigned int, unsigned int, float, int, float, float, float, unsigned int, unsigned int, float, unsigned int, float, unsigned int, float, float, float, float, bool, bool, float, float, float, int, bool, bool, bool, bool, bool, bool, float, float, int, double, float, float, float, double, double, int, int, int, int, float, int, float, float, float, float, float, float, bool, bool, float, float, float, float, float, float, float, float, float, float, float, float, bool, bool, float, float, float, float, float, float, float, float, float, bool, bool, bool, bool, bool, bool, bool, bool, bool, int, bool, bool, bool, int, float, float, float, float, float, double, double, float, float, int, unsigned long long, unsigned int, unsigned int, int, int, int, int, int, int, int, int, bool, bool> >, RooAbsDataHelper<RooDataSet>, const std::vector<std::basic_string<char>, std::allocator<std::basic_string<char> > > &, std::shared_ptr<ROOT::Detail::RDF::RNodeBase>, const ROOT::Internal::RDF::RColumnRegister &>' requested here
   return std::make_unique<Action_t>(Helper_t(std::move(*h)), bl, std::move(prevNode), colRegister);
               ^
/cvmfs/lhcbdev.cern.ch/conda/envs/default/2023-08-17_17-13/linux-64/include/ROOT/RDF/InterfaceUtils.hxx:618:21: note: in instantiation of function template specialization 'ROOT::Internal::RDF::BuildAction<double, float, float, float, float, float, float, unsigned int, unsigned int, float, float, unsigned int, float, unsigned int, float, float, double, unsigned int, unsigned int, unsigned int, float, unsigned int, float, float, float, unsigned int, unsigned int, float, float, unsigned int, unsigned int, float, int, float, float, float, unsigned int, unsigned int, float, unsigned int, float, unsigned int, float, float, float, float, bool, bool, float, float, float, int, bool, bool, bool, bool, bool, bool, float, float, int, double, float, float, float, double, double, int, int, int, int, float, int, float, float, float, float, float, float, bool, bool, float, float, float, float, float, float, float, float, float, float, float, float, bool, bool, float, float, float, float, float, float, float, float, float, bool, bool, bool, bool, bool, bool, bool, bool, bool, int, bool, bool, bool, int, float, float, float, float, float, double, double, float, float, int, unsigned long long, unsigned int, unsigned int, int, int, int, int, int, int, int, int, bool, bool, ROOT::Detail::RDF::RNodeBase, RooAbsDataHelper<RooDataSet> >' requested here
   auto actionPtr = BuildAction<ColTypes...>(cols, std::move(*helperArgOnHeap), nSlots, std::move(prevNodePtr),
                    ^
input_line_111:11:22: note: in instantiation of function template specialization 'ROOT::Internal::RDF::CallBuildAction<ROOT::Internal::RDF::ActionTags::Book, double, float, float, float, float, float, float, unsigned int, unsigned int, float, float, unsigned int, float, unsigned int, float, float, double, unsigned int, unsigned int, unsigned int, float, unsigned int, float, float, float, unsigned int, unsigned int, float, float, unsigned int, unsigned int, float, int, float, float, float, unsigned int, unsigned int, float, unsigned int, float, unsigned int, float, float, float, float, bool, bool, float, float, float, int, bool, bool, bool, bool, bool, bool, float, float, int, double, float, float, float, double, double, int, int, int, int, float, int, float, float, float, float, float, float, bool, bool, float, float, float, float, float, float, float, float, float, float, float, float, bool, bool, float, float, float, float, float, float, float, float, float, bool, bool, bool, bool, bool, bool, bool, bool, bool, int, bool, bool, bool, int, float, float, float, float, float, double, double, float, float, int, unsigned long long, unsigned int, unsigned int, int, int, int, int, int, int, int, int, bool, bool, ROOT::Detail::RDF::RNodeBase, RooAbsDataHelper<RooDataSet> >' requested here
/cvmfs/lhcbdev.cern.ch/conda/envs/default/2023-08-17_17-13/linux-64/include/RooAbsDataHelper.h:123:8: note: candidate template ignored: substitution failure [with ColumnTypes = <double, float, float, float, float, float, float, unsigned int, unsigned int, float, float, unsigned int, float, unsigned int, float, float, double, unsigned int, unsigned int, unsigned int, float, unsigned int, float, float, float, unsigned int, unsigned int, float, float, unsigned int, unsigned int, float, int, float, float, float, unsigned int, unsigned int, float, unsigned int, float, unsigned int, float, float, float, float, bool, bool, float, float, float, int, bool, bool, bool, bool, bool, bool, float, float, int, double, float, float, float, double, double, int, int, int, int, float, int, float, float, float, float, float, float, bool, bool, float, float, float, float, float, float, float, float, float, float, float, float, bool, bool, float, float, float, float, float, float, float, float, float, bool, bool, bool, bool, bool, bool, bool, bool, bool, int, bool, bool, bool, int, float, float, float, float, float, double, double, float, float, int, unsigned long long, unsigned int, unsigned int, int, int, int, int, int, int, int, int, bool, bool>]
  void Exec(unsigned int slot, ColumnTypes... values)

I suspect this is because when trying to create a multi-variable dataset with the Book call as in the tutorial one has to pass in ‘all-variables’ having the same type? Thanks in advance.

I can promote to a newer root in case but i will need this fix here Unable to use EOS tokens with RDataFrame since 6.32 · Issue #16475 · root-project/root · GitHub , since i am relying on this for my analysis and it’s pipeline

Here my root version

  | Welcome to ROOT 6.28/04                        https://root.cern |
  | (c) 1995-2022, The ROOT Team; conception: R. Brun, F. Rademakers |
  | Built for linuxx8664gcc on Jul 28 2023, 11:19:00                 |
  | From tag , 8 May 2023

rquaglia · September 20, 2024, 4:42pm

Note that the way i am constructing my dataset is :

def datasetmaker( node : RNode , 
                  ds_name  :str ,  
                  ds_title :str  , 
                  ds_variables : Union[list, RooRealVar]= None  , 
                  node_columns : Union[list, str]       = None  ): 
    """
    The RDF node
    Name of the DataSet to create
    Title of the DataSet to create 
    List or single RooRealVar to use for Variable assignment
    Equal list of ColumnNames. 
    
    Note: 
        ds_variables and node_columns list ordering has to be the same
    """
    if type(ds_variables) == list :  assert( len(ds_variables) == len(node_columns))
    if type(ds_variables) == list : 
        #if you passed a list of roorealvars
        ds_helper = RooDataSetHelper(ds_name, ds_title , RooArgSet( *ds_variables))
        return node.Book( ROOT.std.move(ds_helper), node_columns )        
    else : 
        #if you passed a single roorealvar
        ds_helper = RooDataSetHelper(ds_name,ds_title, RooArgSet( ds_variables))
        return node.Book( ROOT.std.move(ds_helper), [node_columns] )

Danilo · September 20, 2024, 8:20pm

Hi Renato,

Thanks for the interesting post, if I understand correctly you are mixing RDF and RooFit, using the former to build a dataset for the latter, right?

This looks like a question that requires quite some expertise: let me add in the loop @mczurylo and @vpadulan .

In the meantime, what could be of help, would be a self contained example, that reproduces the issue - no need to share details of your analysis, a limited number of columns and an RDF generating fake data would be enough.

As for the post you shared, we are already following that up.

Cheers,
Danilo

rquaglia · September 20, 2024, 8:55pm

Dear @Danilo , here an attempt to the reproducer on my mac

ROOT Version: 6.30/04
Built for macosxarm64 on Feb 05 2024, 18:52:59
From heads/latest-stable@

import ROOT
import numpy as np

# Create a ROOT RDataFrame with 100 entries
n_entries = 10
float_col  = np.random.rand(n_entries).astype(np.float32)   # Float column
double_col = np.random.rand(n_entries).astype(np.float64)  # Double column
int_col    = np.random.randint(-100, 100, n_entries)          # Int column
bool_col   = np.random.choice([True, False], n_entries)      # Bool column
uint_col   = np.random.randint(0, 100, n_entries, dtype=np.uint32)  # Unsigned int column

df = ROOT.RDataFrame(n_entries)

# Define columns in the RDataFrame
df = df.Define("float_col" ,",".join(map(str, float_col))).Define("float_col2","float(float_col)")
df = df.Define("double_col", ",".join(map(str, double_col)))
df = df.Define("int_col",    ",".join(map(str, int_col)))
df = df.Define("bool_col",   ",".join(map(str, map(int, bool_col))))  # Convert bool to int for ROOT
df = df.Define("uint_col",   ",".join(map(str, uint_col)))

for c in df.GetColumnNames():
    print( df.GetColumnType(c))
vars_list =[] 
vars_name =[]
for c in df.GetColumnNames():
    v = ROOT.RooRealVar( str(c),str(c),0)
    v.setConstant(0)
    vars_list.append(v)
    vars_name.append(str(c))

helper = ROOT.RooDataSetHelper("dataset", "Title of dataset", ROOT.RooArgSet( *vars_list))

roo_data_set_result = df.Book( ROOT.std.move(helper), vars_name)

df.Count()

roo_data_set_result.Print()

the code crash in the same exact way. However if i ensure all variables being foat or double to be double for some reasons it works.

vpadulan · September 21, 2024, 9:10am

Dear @rquaglia ,

Thanks for reaching out to the forum. From a first look at the code, it seems that the RooDatasetHelper is indeed expecting the input column types to be coherent. I will investigate this further and understand, in the meanwhile thanks for your reproducer.

Cheers,
Vincenzo

jonas · September 21, 2024, 11:25am

Hi @rquaglia!

I don’t think the way you declare the dataframe in the reproducer is valid. I changed it a bit.

As the others said, it doesn’t make sense to have all possible types in a RooDataSet. They only contain doubles for real-values variables (RooRealVar), and ints for categorical features (RooCategory). And the type is encoded by the variable type you construct the dataset with.

The RooAbsDataHelper doesn’t support mixed types, but fortunately there is a nice pythonic alternative to go over numpy arrays:

import ROOT
import numpy as np

# Create a ROOT RDataFrame with 100 entries
df = ROOT.RDataFrame(10)

# Define columns in the RDataFrame
df = df.Define("float_col", "float(6.0)")
df = df.Define("double_col", "double(2.0)")
df = df.Define("int_col", "int(3)")
df = df.Define("bool_col", "int(false)")  # Convert bool to int for ROOT
df = df.Define("uint_col", "(unsigned int)(5)")

for c in df.GetColumnNames():
    print(df.GetColumnType(c))

vars_list = []

for c in map(str, df.GetColumnNames()):

    # The data type in the RooDataSet can only be "double" for real-valued
    # variables, or "int" for categorical features represented by RooCategory.
    # However, RooFit won't care about the types in the NumPy array. You have
    # to declare the types by using the appropriate RooFit object in the
    # dataset (RooRealVar or RooCategory). Then, from_numpy will cast accordingly.

    # I assume your ints and bool are categorical features?
    is_category = "int" in c or "bool" in c
    if is_category:
        # For categorical features, you need to declare the possible states
        # including their names in the constructor. Let's assume you're
        # expecting values between 0 and 10.
        states = {f"state_{idx}": idx for idx in range(10)}
        v = ROOT.RooCategory(c, c, states)
    else:
        v = ROOT.RooRealVar(c, c, 0)

    v.setConstant(0)
    vars_list.append(v)

# NumPy arrays are the Python way to transfer data around
# see https://root.cern/doc/master/rf409__NumPyPandasToRooFit_8py.html
roo_data_set = ROOT.RooDataSet.from_numpy(df.AsNumpy(), vars_list)

roo_data_set.Print()
roo_data_set.get(0).Print("v") # output first row

Output:

int
double
float
int
unsigned int
RooDataSet::[bool_col,double_col,float_col,int_col,uint_col] = 10 entries
  1) 0x59f0e6dc9160 RooCategory::   bool_col = state_0(idx = 0)
  "bool_col"
  2) 0x59f0e3b66330 RooRealVar:: double_col = 2  L(-INF - +INF)  "double_col"
  3) 0x59f0e39e1190 RooRealVar::  float_col = 6  L(-INF - +INF)  "float_col"
  4) 0x59f0e6d6ffc0 RooCategory::    int_col = state_3(idx = 3)
  "int_col"
  5) 0x59f0e63daaf0 RooCategory::   uint_col = state_5(idx = 5)
  "uint_col"

But be careful with what you’re doing, categorical values are only useful in RooFit in rare circumstances, e.g. when managing multiple channels of a RooSimultaneous. And you need to know the possible values in advance to construct the RooCategory correctly. In many cases, e.g. when you do cuts on these columns later with RooAbsData::reduce(), you’re better off just using RooRealVar for everything.

I hope this helps!

Cheers,
Jonas

vpadulan · September 21, 2024, 11:30am

Dear @rquaglia ,

On top of the excellent reply from @jonas , I wanted to bring to your attention that if you want to create an RDataFrame from a set of numpy arrays, there is a much simpler and more Pythonic way to do so that doesn’t require the string interpolation from your example. You can use ROOT.RDF.FromNumpy as shown in this tutorial.

Best,
Vincenzo

rquaglia · September 21, 2024, 12:06pm

Dear all

Thanks a lot for the replies and suggestions. It looks great and it seems like that if i would have ONLY floats and doubles to push into the dataset it would work, right? Also from the snippet above it would suffice to make roocategory for integers and booleans. This woukd be sufficient for my use case.
For my case i want to use reduce and i guess to do so i better redefine the column as float before making the dataset and keep all as roorealvar right @jonas ? I am not interested jn roocategory.

Thanks a lot again and thanks for the suggestions on the nicer wrappers which will simplify the code i am using.
I try the suggestions in the actual case i have and let you know if all works.

Edit : i think i got the point the roodatasethelper supports only double arguments , so probably it would be more efficient and faster to avoid doing
Dataframe.AsNumpy and then forward for dataset creation , i would rather go for redefine on the dataframe and then use still the datasethelper.

jonas · September 21, 2024, 12:44pm

Okay, if you care about datasets with RooAbsReals only and want to avoid the extra copy over numpy, easy! I have opened a PR so that this is possible also without re-define:

github.com/root-project/root

[RF] Explicitly cast to double in RooAbsDataHelper

root-project:master ← guitargeek:RooAbsDataHelper

opened 12:42PM - 21 Sep 24 UTC

guitargeek

+1 -1

We support only RooRealVar columns anyway, and by doing the cast to double expli…cit we can also support filling such datasets from arbitrary column types that convert to `double`. This is a small quality-of-life improvement that addresses this forum post: https://root-forum.cern.ch/t/dataframe-roofit-dataset-maker-with-mixture-of-column-types-crashes/61734/8

I think this change is non-intrusive enough such that we can also backport it to 6.32, so you can benefit from it in the next 6.32 patch release that will come out next week.

rquaglia · September 21, 2024, 1:26pm

Thanks @jonas , tough i think that in my local tests i was able to mix bool,int and doubles for the roodatasethelper using only roorealvat but as soon as there were a mix of float and doubles there was the crash. Is this also what you observed ?

rquaglia · September 21, 2024, 7:12pm

Actually i was wrong, mixed types always gives error.
For my use case i can just do

for c in df.GetColumnNames():
    v = ROOT.RooRealVar( str(c),str(c),0)
    v.setConstant(0)    
    df = df.Redefine( c, f"double({c})")
    vars_list.append(v)
    vars_name.append(str(c))

helper = ROOT.RooDataSetHelper("dataset", "Title of dataset", ROOT.RooArgSet( *vars_list))

roo_data_set_result = df.Book( ROOT.std.move(helper), vars_name)

df.Count()

While the patch for full forward-cast to double is deployed

jonas · September 23, 2024, 7:36am

Hi, just to keep you updated: the ROOT 6.32.06 patch release was released earlier than I expected (yesterday actually), so my patch didn’t land in time, although it is merged to the development branches now. So it will work in ROOT 6.32.08 (if it will be released), or 6.34.00.