Increase template recursion depth for TDataFrame::Snapshot

ROOT
1

I was just trying this very simple script to reduce an ntuple to the branches I need (and nothing else):

#include "TFile.h"
#include "ROOT/TDataFrame.hxx"
#include <iostream>
#include <stdlib.h>

using ROOT::Experimental::TDataFrame;

std::vector<std::string> active_branches = {
   "Z0_OWNPV_X", "Z0_ETA", "Z0_OWNPV_Y", "Z0_OWNPV_Z", "Z0_P", "Z0_PT", "Z0_PX", "Z0_PY", "Z0_PZ", "Z0_MM", "Z0_MMERR",
   "Z0_PE", "muplus_ETA", "muplus_OWNPV_X", "muplus_OWNPV_Y", "muplus_OWNPV_Z", "muplus_OWNPV_XERR",
   "muplus_OWNPV_YERR", "muplus_OWNPV_ZERR", "muplus_OWNPV_CHI2", "muplus_OWNPV_NDOF", "muplus_IP_OWNPV",
   "muplus_IPCHI2_OWNPV", "muplus_P", "muplus_PT", "muplus_PE", "muplus_PX", "muplus_PY", "muplus_PZ", "muplus_PIDe",
   "muplus_PIDmu", "muplus_PIDK", "muplus_PIDp", "muplus_ProbNNe", "muplus_ProbNNk", "muplus_ProbNNp",
   "muplus_ProbNNpi", "muplus_ProbNNmu", "muplus_ProbNNghost", "muplus_ProbNNe", "muplus_TRACK_CHI2NDOF",
   "muplus_TRACK_Likelihood", "muplus_ETA", "muplus_OWNPV_X", "muplus_OWNPV_Y", "muplus_OWNPV_Z", "muplus_OWNPV_XERR",
   "muplus_OWNPV_YERR", "muplus_OWNPV_ZERR", "muplus_OWNPV_CHI2", "muplus_OWNPV_NDOF", "muplus_IP_OWNPV",
   "muplus_IPCHI2_OWNPV", "muminus_P", "muminus_PT", "muminus_PE", "muminus_PX", "muminus_PY", "muminus_PZ",
   "muminus_PIDe", "muminus_PIDmu", "muminus_PIDK", "muminus_PIDp", "muminus_ProbNNe",
   // "muminus_ProbNNk",
   // "muminus_ProbNNp",
   // "muminus_ProbNNpi",
   // "muminus_ProbNNmu",
   // "muminus_ProbNNghost",
   // "muminus_ProbNNe",
   // "muminus_TRACK_CHI2NDOF",
   // "muminus_TRACK_Likelihood",
   // "runNumber",
   // "eventNumber",
   // "nTracks",
   "nPV", "nSPDHits"};

void reduce_general(std::string file)
{
   auto f = TFile::Open(file.c_str());
   auto treeName = "DecayTree";
   TDataFrame df(treeName, f);
   df.Snapshot(treeName, "test.root", active_branches);
}

This works fine. However, if I uncomment one additional branch to copy, it runs into a recursion limit:

In file included from input_line_1:1:
In file included from /usr/include/c++/6/new:40:
In file included from /usr/include/c++/6/exception:171:
In file included from /usr/include/c++/6/bits/nested_exception.h:40:
In file included from /usr/include/c++/6/bits/move.h:57:
/usr/include/c++/6/type_traits:774:43: fatal error: recursive template instantiation exceeded maximum depth of 1024
    : public __and_<is_array<_Tp>, __not_<extent<_Tp>>>
                                          ^~~~~~~~~~~~
/usr/include/c++/6/type_traits:121:26: note: in instantiation of template class 'std::__is_array_unknown_bounds<std::_Tuple_impl<66, ROOT::Internal::TDF::TColumnValue<int> > >' requested here
    : public conditional<_B1::value, _B1, _B2>::type
                         ^
/usr/include/c++/6/type_traits:126:14: note: in instantiation of template class 'std::__or_<std::__is_array_unknown_bounds<std::_Tuple_impl<66, ROOT::Internal::TDF::TColumnValue<int> > >,
      std::is_function<std::_Tuple_impl<66, ROOT::Internal::TDF::TColumnValue<int> > > >' requested here
    : public conditional<_B1::value, _B1, __or_<_B2, _B3, _Bn...>>::type
             ^
/usr/include/c++/6/type_traits:799:19: note: in instantiation of template class 'std::__or_<std::is_void<std::_Tuple_impl<66, ROOT::Internal::TDF::TColumnValue<int> > >,
      std::__is_array_unknown_bounds<std::_Tuple_impl<66, ROOT::Internal::TDF::TColumnValue<int> > >, std::is_function<std::_Tuple_impl<66,
      ROOT::Internal::TDF::TColumnValue<int> > > >' requested here
           bool = __or_<is_void<_Tp>,
                  ^
/usr/include/c++/6/type_traits:822:14: note: in instantiation of default argument for '__is_destructible_safe<std::_Tuple_impl<66, ROOT::Internal::TDF::TColumnValue<int> > >' required here
    : public __is_destructible_safe<_Tp>::type
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/c++/6/type_traits:144:26: note: in instantiation of template class 'std::is_destructible<std::_Tuple_impl<66, ROOT::Internal::TDF::TColumnValue<int> > >' requested here
    : public conditional<_B1::value, _B2, _B1>::type
                         ^
/usr/include/c++/6/type_traits:986:14: note: (skipping 1015 contexts in backtrace; use -ftemplate-backtrace-limit=0 to see all)
    : public __and_<is_destructible<_Tp>,
             ^
/usr/include/c++/6/bits/shared_ptr.h:619:14: note: in instantiation of function template specialization 'std::shared_ptr<ROOT::Internal::TDF::TAction<ROOT::Internal::TDF::SnapshotHelper<(lambda at
      /cern/root-build/include/ROOT/TDFInterface.hxx:1108:28), (lambda at /cern/root-build/include/ROOT/TDFInterface.hxx:1098:26)>, ROOT::Detail::TDF::TLoopManager,
      ROOT::Internal::TDF::TTypeList<double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double,
      double, double, double, double, int, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double,
      double, double, double, double, double, double, double, double, double, double, double, double, double, double, int, double, double, double, double, double,
      double, double, double, double, double, double, double, double, double, int, int> >
      >::shared_ptr<std::allocator<ROOT::Internal::TDF::TAction<ROOT::Internal::TDF::SnapshotHelper<(lambda at /cern/root-build/include/ROOT/TDFInterface.hxx:1108:28),
      (lambda at /cern/root-build/include/ROOT/TDFInterface.hxx:1098:26)>, ROOT::Detail::TDF::TLoopManager, ROOT::Internal::TDF::TTypeList<double, double, double,
      double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, int, double, double,
      double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double,
      double, double, double, double, double, double, double, int, double, double, double, double, double, double, double, double, double, double, double, double,
      double, double, int, int> > >, ROOT::Internal::TDF::SnapshotHelper<(lambda at /cern/root-build/include/ROOT/TDFInterface.hxx:1108:28), (lambda at
      /cern/root-build/include/ROOT/TDFInterface.hxx:1098:26)>, const std::vector<std::__cxx11::basic_string<char>, std::allocator<std::__cxx11::basic_string<char> > >
      &, ROOT::Detail::TDF::TLoopManager &>' requested here
      return shared_ptr<_Tp>(_Sp_make_shared_tag(), __a,
             ^
/usr/include/c++/6/bits/shared_ptr.h:635:19: note: in instantiation of function template specialization 'std::allocate_shared<ROOT::Internal::TDF::TAction<ROOT::Internal::TDF::SnapshotHelper<(lambda at
      /cern/root-build/include/ROOT/TDFInterface.hxx:1108:28), (lambda at /cern/root-build/include/ROOT/TDFInterface.hxx:1098:26)>, ROOT::Detail::TDF::TLoopManager,
      ROOT::Internal::TDF::TTypeList<double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double,
      double, double, double, double, int, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double,
      double, double, double, double, double, double, double, double, double, double, double, double, double, double, int, double, double, double, double, double,
      double, double, double, double, double, double, double, double, double, int, int> >,
      std::allocator<ROOT::Internal::TDF::TAction<ROOT::Internal::TDF::SnapshotHelper<(lambda at /cern/root-build/include/ROOT/TDFInterface.hxx:1108:28), (lambda at
      /cern/root-build/include/ROOT/TDFInterface.hxx:1098:26)>, ROOT::Detail::TDF::TLoopManager, ROOT::Internal::TDF::TTypeList<double, double, double, double, double,
      double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, int, double, double, double, double,
      double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double,
      double, double, double, double, double, int, double, double, double, double, double, double, double, double, double, double, double, double, double, double, int,
      int> > >, ROOT::Internal::TDF::SnapshotHelper<(lambda at /cern/root-build/include/ROOT/TDFInterface.hxx:1108:28), (lambda at
      /cern/root-build/include/ROOT/TDFInterface.hxx:1098:26)>, const std::vector<std::__cxx11::basic_string<char>, std::allocator<std::__cxx11::basic_string<char> > >
      &, ROOT::Detail::TDF::TLoopManager &>' requested here
      return std::allocate_shared<_Tp>(std::allocator<_Tp_nc>(),
                  ^
/cern/root-build/include/ROOT/TDFInterface.hxx:1118:24: note: in instantiation of function template specialization 'std::make_shared<ROOT::Internal::TDF::TAction<ROOT::Internal::TDF::SnapshotHelper<(lambda at
      /cern/root-build/include/ROOT/TDFInterface.hxx:1108:28), (lambda at /cern/root-build/include/ROOT/TDFInterface.hxx:1098:26)>, ROOT::Detail::TDF::TLoopManager,
      ROOT::Internal::TDF::TTypeList<double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double,
      double, double, double, double, int, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double,
      double, double, double, double, double, double, double, double, double, double, double, double, double, double, int, double, double, double, double, double,
      double, double, double, double, double, double, double, double, double, int, int> >, ROOT::Internal::TDF::SnapshotHelper<(lambda at
      /cern/root-build/include/ROOT/TDFInterface.hxx:1108:28), (lambda at /cern/root-build/include/ROOT/TDFInterface.hxx:1098:26)>, const
      std::vector<std::__cxx11::basic_string<char>, std::allocator<std::__cxx11::basic_string<char> > > &, ROOT::Detail::TDF::TLoopManager &>' requested here
         df->Book(std::make_shared<DFA_t>(Op_t(std::move(initLambda), std::move(fillTree)), bnames, *fProxiedPtr));
                       ^
/cern/root-build/include/ROOT/TDFInterface.hxx:266:14: note: in instantiation of function template specialization 'ROOT::Experimental::TDF::TInterface<ROOT::Detail::TDF::TLoopManager>::SnapshotImpl<double, double, double, double,
      double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, int, double, double, double,
      double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double,
      double, double, double, double, double, double, int, double, double, double, double, double, double, double, double, double, double, double, double, double,
      double, int, int, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38,
      39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66>' requested here
      return SnapshotImpl<BranchTypes...>(treename, filename, bnames, TypeInd_t());
             ^
input_line_44:2:129: note: in instantiation of function template specialization 'ROOT::Experimental::TDF::TInterface<ROOT::Detail::TDF::TLoopManager>::Snapshot<double, double, double, double,
      double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, int, double, double, double,
      double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double, double,
      double, double, double, double, double, double, int, double, double, double, double, double, double, double, double, double, double, double, double, double,
      double, int, int>' requested here
 if (gROOTMutex) gROOTMutex->UnLock(); ((ROOT::Experimental::TDF::TInterface<ROOT::Detail::TDF::TLoopManager>*)0x7ffe68b1db00)->Snapshot<double, double, double, ...
                                                                                                                                ^
/usr/include/c++/6/type_traits:774:43: note: use -ftemplate-depth=N to increase recursive template instantiation depth
    : public __and_<is_array<_Tp>, __not_<extent<_Tp>>>
                                          ^

How can I increase the recursion depth or is there another work-around for this? The current limit seems to be 66 branches (or all, I guess).

Cheers,
Graipher

2 Likes
2

Hi,

interesting observation. The limit you are hitting comes from the jitted invocation of Snapshot, with all types made explicit. The recursion limit comes therefore from the settings of the ROOT interpreter and not from the ones of your compiler.
I think that a first workaround would consist in making explicit the types of the branches you are snapshotting. I am aware that this is less than optimal since the boilerplate code to manage on your side is not negligible.

I am working on a patch which could help in this case: I’ll share it asap and discuss it with the cling expert in our team.

Cheers,
D

3

Oh, I forgot: could you share with me / on the forum the input file so that I can test my fix?

Cheers,
D

4

Yeah, not having to make the types explicit is one of the main reasons why I want to do it with TDataFrame in the first place…

For now I got around it by leaving out enough branches I probably won’t need, so no pressure there :slight_smile:

An input file can be found at:
/afs/cern.ch/user/a/aweiden/public/for_root_forum/preselection_Data_Z0_MagDown.root

5

thanks. Trying this out now…

1 Like
6

Hi,

I tried this code* which is a minimal variation of yours in order to run as a macro and I manage successfully on osx Sierra, ROOT 6.10, to create a new dataset with all the 77 column names of your list.
I went a step further and tried to reproduce this without your file, just defining N (up to 300) dummy columns ** always on osx Sierra, both 6.10 and master: it succeeded (and was 4x faster in inferring types and compiling the necessary templates - we upgraded the clang version in ROOT).

On what platform are you running (os & compiler)? Does ** crash for you?
Meanwhile I am trying on a linux box.

Cheers and thanks again for the interesting usecase,
D

#include "TFile.h"
#include "ROOT/TDataFrame.hxx"
#include <iostream>
#include <stdlib.h>

using ROOT::Experimental::TDataFrame;

std::vector<std::string> active_branches = {
   "Z0_OWNPV_X", "Z0_ETA", "Z0_OWNPV_Y", "Z0_OWNPV_Z", "Z0_P", "Z0_PT", "Z0_PX", "Z0_PY", "Z0_PZ", "Z0_MM", "Z0_MMERR",
   "Z0_PE", "muplus_ETA", "muplus_OWNPV_X", "muplus_OWNPV_Y", "muplus_OWNPV_Z", "muplus_OWNPV_XERR",
   "muplus_OWNPV_YERR", "muplus_OWNPV_ZERR", "muplus_OWNPV_CHI2", "muplus_OWNPV_NDOF", "muplus_IP_OWNPV",
   "muplus_IPCHI2_OWNPV", "muplus_P", "muplus_PT", "muplus_PE", "muplus_PX", "muplus_PY", "muplus_PZ", "muplus_PIDe",
   "muplus_PIDmu", "muplus_PIDK", "muplus_PIDp", "muplus_ProbNNe", "muplus_ProbNNk", "muplus_ProbNNp",
   "muplus_ProbNNpi", "muplus_ProbNNmu", "muplus_ProbNNghost", "muplus_ProbNNe", "muplus_TRACK_CHI2NDOF",
   "muplus_TRACK_Likelihood", "muplus_ETA", "muplus_OWNPV_X", "muplus_OWNPV_Y", "muplus_OWNPV_Z", "muplus_OWNPV_XERR",
   "muplus_OWNPV_YERR", "muplus_OWNPV_ZERR", "muplus_OWNPV_CHI2", "muplus_OWNPV_NDOF", "muplus_IP_OWNPV",
   "muplus_IPCHI2_OWNPV", "muminus_P", "muminus_PT", "muminus_PE", "muminus_PX", "muminus_PY", "muminus_PZ",
   "muminus_PIDe", "muminus_PIDmu", "muminus_PIDK", "muminus_PIDp", "muminus_ProbNNe",
   "muminus_ProbNNk",
   "muminus_ProbNNp",
   "muminus_ProbNNpi",
   "muminus_ProbNNmu",
   "muminus_ProbNNghost",
   "muminus_ProbNNe",
   "muminus_TRACK_CHI2NDOF",
   "muminus_TRACK_Likelihood",
   "runNumber",
   "eventNumber",
   "nTracks",
   "nPV", "nSPDHits"};

void reduce_general(std::string file = "preselection_Data_Z0_MagDown.root")
{
   auto f = TFile::Open(file.c_str());
   auto treeName = "DecayTree";
   TDataFrame df(treeName, f);
   df.Snapshot(treeName, "test.root", active_branches);
}

**

using TDFDEF = ROOT::Experimental::TDF::TInterface<ROOT::Detail::TDF::TCustomColumnBase>;

TDFDEF defineRecursive(TDFDEF& d, int n)
{
	std::string name = "a_" + std::to_string(n);
	std::cout << "Defining column " << name << std::endl;
	auto d2 = d.Define(name, [](){return 1.;});
	if (n == 1) return d2;
	return defineRecursive(d2, n - 1);
}

void templateRecursion()
{
    ROOT::Experimental::TDataFrame tdf(1);
	TDFDEF d = tdf.Define("a_0", "1");

	auto d_final = defineRecursive(d, 100);

	d_final.Snapshot("t", "f.root");

}
7

I’m using ROOT 6.10/0 on Ubuntu 17.04, compiled with gcc (Ubuntu 6.3.0-12ubuntu2) 6.3.0 20170406.

I just tried running your second code, slightly modified:

file templateRecursion.cxx:

#include "TFile.h"
#include "ROOT/TDataFrame.hxx"
#include <iostream>
#include <stdlib.h>

using TDFDEF = ROOT::Experimental::TDF::TInterface<ROOT::Detail::TDF::TCustomColumnBase>;

TDFDEF defineRecursive(TDFDEF &d, int n)
{
   std::string name = "a_" + std::to_string(n);
   std::cout << "Defining column " << name << std::endl;
   auto d2 = d.Define(name, []() { return 1.; });
   if (n == 1) return d2;
   return defineRecursive(d2, n - 1);
}

void templateRecursion()
{
   ROOT::Experimental::TDataFrame tdf(1);
   TDFDEF d = tdf.Define("a_0", "1");

   auto d_final = defineRecursive(d, 100);

   d_final.Snapshot("t", "f.root");
}

And running it like this: root templateRecursion.cxx+ I get the following compiler errors:

root [0] 
Processing templateRecursion.cxx+...
Info in <TUnixSystem::ACLiC>: creating shared library /home/andreas/cernbox/work/drell-yan/test_scripts/./templateRecursion_cxx.so

 *** Break *** segmentation violation



===========================================================
There was a crash.
This is the entire stack trace of all threads:
===========================================================
#0  0x00007f7b5d47005a in __GI___waitpid (pid=8665, stat_loc=stat_loc
entry=0x7ffd28b47780, options=options
entry=0) at ../sysdeps/unix/sysv/linux/waitpid.c:29
#1  0x00007f7b5d3e82cb in do_system (line=<optimized out>) at ../sysdeps/posix/system.c:148
#2  0x00007f7b5dff2198 in TUnixSystem::Exec (shellcmd=<optimized out>, this=0x55e8c18144f0) at /cern/root-git/core/unix/src/TUnixSystem.cxx:2118
#3  TUnixSystem::StackTrace (this=0x55e8c18144f0) at /cern/root-git/core/unix/src/TUnixSystem.cxx:2412
#4  0x00007f7b5dff47fc in TUnixSystem::DispatchSignals (this=0x55e8c18144f0, sig=kSigSegmentationViolation) at /cern/root-git/core/unix/src/TUnixSystem.cxx:3643
#5  <signal handler called>
#6  0x00007f7b4d7863a6 in templateRecursion() () from /home/andreas/cernbox/work/drell-yan/test_scripts/templateRecursion_cxx.so
#7  0x00007f7b59160167 in cling::Interpreter::RunFunction(clang::FunctionDecl const*, cling::Value*) [clone .part.282] () from /cern/root-6.10.00/lib/libCling.so
#8  0x00007f7b5916385d in cling::Interpreter::EvaluateInternal(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, cling::CompilationOptions, cling::Value*, cling::Transaction**, unsigned long) () from /cern/root-6.10.00/lib/libCling.so
#9  0x00007f7b59163a62 in cling::Interpreter::process(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, cling::Value*, cling::Transaction**, bool) () from /cern/root-6.10.00/lib/libCling.so
#10 0x00007f7b591efdff in cling::MetaProcessor::process(char const*, cling::Interpreter::CompilationResult&, cling::Value*, bool) () from /cern/root-6.10.00/lib/libCling.so
#11 0x00007f7b590ea969 in HandleInterpreterException (metaProcessor=<optimized out>, input_line=<optimized out>, compRes=
0x7ffd28b4a724: cling::Interpreter::kSuccess, result=result
entry=0x7ffd28b4a730) at /cern/root-git/core/metacling/src/TCling.cxx:1887
#12 0x00007f7b590fb4b4 in TCling::ProcessLine (this=0x55e8c186ecc0, line=<optimized out>, error=0x7ffd28b4b88c) at /cern/root-git/core/metacling/src/TCling.cxx:2009
#13 0x00007f7b590ed65d in TCling::ProcessLineSynch (this=0x55e8c186ecc0, line=0x55e8c232de30 ".X  /home/andreas/cernbox/work/drell-yan/test_scripts/./templateRecursion.cxx+", error=0x7ffd28b4b88c) at /cern/root-git/core/metacling/src/TCling.cxx:2886
#14 0x00007f7b5ded2f07 in TApplication::ExecuteFile (file=<optimized out>, error=0x7ffd28b4b88c, keep=<optimized out>) at /cern/root-git/core/base/src/TApplication.cxx:1137
#15 0x00007f7b5ded4cb3 in TApplication::ProcessLine (this=0x55e8c185fb10, line=<optimized out>, sync=<optimized out>, err=0x7ffd28b4b88c) at /cern/root-git/core/base/src/TApplication.cxx:982
#16 0x00007f7b5e3107e4 in TRint::ProcessLineNr (this=this
entry=0x55e8c185fb10, filestem=filestem
entry=0x7f7b5e321f4c "ROOT_cli_", line=line
entry=0x7ffd28b4b890 ".x templateRecursion.cxx+", error=error
entry=0x7ffd28b4b88c) at /cern/root-git/core/rint/src/TRint.cxx:756
#17 0x00007f7b5e311db7 in TRint::Run (this=0x55e8c185fb10, retrn=<optimized out>) at /cern/root-git/core/rint/src/TRint.cxx:416
#18 0x000055e8c0846dbc in main (argc=<optimized out>, argv=0x7ffd28b4da18) at /cern/root-git/main/src/rmain.cxx:30
===========================================================


The lines below might hint at the cause of the crash.
You may get help by asking at the ROOT forum http://root.cern.ch/forum.
Only if you are really convinced it is a bug in ROOT then please submit a
report at http://root.cern.ch/bugs. Please post the ENTIRE stack trace
from above as an attachment in addition to anything else
that might help us fixing this issue.
===========================================================
#6  0x00007f7b4d7863a6 in templateRecursion() () from /home/andreas/cernbox/work/drell-yan/test_scripts/templateRecursion_cxx.so
#7  0x00007f7b59160167 in cling::Interpreter::RunFunction(clang::FunctionDecl const*, cling::Value*) [clone .part.282] () from /cern/root-6.10.00/lib/libCling.so
#8  0x00007f7b5916385d in cling::Interpreter::EvaluateInternal(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, cling::CompilationOptions, cling::Value*, cling::Transaction**, unsigned long) () from /cern/root-6.10.00/lib/libCling.so
#9  0x00007f7b59163a62 in cling::Interpreter::process(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, cling::Value*, cling::Transaction**, bool) () from /cern/root-6.10.00/lib/libCling.so
#10 0x00007f7b591efdff in cling::MetaProcessor::process(char const*, cling::Interpreter::CompilationResult&, cling::Value*, bool) () from /cern/root-6.10.00/lib/libCling.so
#11 0x00007f7b590ea969 in HandleInterpreterException (metaProcessor=<optimized out>, input_line=<optimized out>, compRes=
0x7ffd28b4a724: cling::Interpreter::kSuccess, result=result
entry=0x7ffd28b4a730) at /cern/root-git/core/metacling/src/TCling.cxx:1887
8

Hi,

thanks: that poses another challenge which I pick up :slightly_smiling_face:
For what concerns your original macro, I can confirm that on slc6, gcc6 root master and 6.10 the problem you noticed is there.
My proposal would be* to increase the template depth allowed to the interpreter but I need to test it and this will unfortunately be possible tomorrow morning. I’ll keep you posted.

Cheers,
D

diff --git a/core/metacling/src/TCling.cxx b/core/metacling/src/TCling.cxx
index 54560cf..c9deb49 100644
--- a/core/metacling/src/TCling.cxx
+++ b/core/metacling/src/TCling.cxx
@@ -1093,6 +1093,7 @@ TCling::TCling(const char *name, const char *title)
 
       clingArgsStorage.push_back("-Wno-undefined-inline");
       clingArgsStorage.push_back("-fsigned-char");
+      clingArgsStorage.push_back("-ftemplate-depth=8192");
    }
 
    std::vector<const char*> interpArgs;
9

Hallo,
a commit that solves the issue landed in the master branch today and will be available in 6.10-02 very soon (likely this week), together with a bunch of other TDF bugfixes.

(Related but off-topic, since you are basically a power user :smile: the only major thing that stays in master and won’t be in 6.10-02 is a dramatic speed-up in the runtime of TDF applications with many actions with branch types not explicitly listed – e.g. 50 Histo1D("b") are much slower than 50 Histo1D<int>("b") in 6.10, but not in master)

Thanks for the feedback as usual!

10

P.S. I tested Snapshotting up to 400 branches. At that scale a macro takes 6 GB of RAM to run (because of the just-in-time compilation of all those branch proxies), and about 30 seconds to execute, but it does its job. Similar considerations will hold for Foreach and other actions that take an arbitrary number of branches, as well as Filter and Define.

In a program that is going to be written+compiled “once” and that is going to be run many times, it might be worth it to explicitly specify the types of the branches as template parameters, e.g. Snapshot<double, int>("outTree", "outFile.root", {"doubleBranch", "intBranch"}) – this will speed up runtimes noticeably because the action will be compiled instead of just-in-time compiled at runtime. It might be a trade-off worth considering.

11

Thanks for the follow-up, I’ll try 6.10.02 when it is released and look forward to seeing which of my macros magically start working :smiley:

12

Hi,

the new release is out.
https://root.cern.ch/content/release-61002
Your code should “magically” (or “by construction” :slight_smile: ) work now.

Cheers and thanks again for the feedback,
D

13

Hm, when I do git tag -l, I don’t see any v6-10-02, only v6-10-00…

(Of course after first doing git fetch.

14

Ok, nevermind. Apparently I need to to git fetch --tags to get the new tag…

2 Likes
15

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.