Limit on number of variables in user-defined cpp function for TTreeFormula

Hello,

I am writing a small cpp function to define an additional variable that is not stored in the TTrees I am reading.
I got it working just fine with up to 9 arguments to the cpp function but when I tried to add a 10th one, I hit a segmentation fault (see log below).

So I have a few questions:

  1. Is there a hard limit on the number of variables I can pass? If yes, where is it documented?
  2. What type of object can I pass to my function? In the past I know that only standard types (int, float, etc) were supported but maybe there is more options now?

Thanks a lot!

Cheers,
Quentin

Blockquote
[/usr/lib/system/libsystem_platform.dylib] _sigtramp (no debug info)
[] (no debug info)
[/Applications/root_v6.16.00/lib/libCling.so] TryImplicitConversion(clang::Sema&, clang::Expr*, clang::QualType, bool, bool, bool, bool, bool, bool) (no debug info)
[/Applications/root_v6.16.00/lib/libCling.so] TryCopyInitialization(clang::Sema&, clang::Expr*, clang::QualType, bool, bool, bool, bool) (no debug info)
[/Applications/root_v6.16.00/lib/libCling.so] clang::Sema::AddOverloadCandidate(clang::FunctionDecl*, clang::DeclAccessPair, llvm::ArrayRef< clang::Expr*>, clang::OverloadCandidateSet&, bool, bool, bool, llvm::MutableArrayRef<clang:: ImplicitConversionSequence>) (no debug info)
[/Applications/root_v6.16.00/lib/libCling.so] cling::overloadFunctionSelector(clang::DeclContext*, bool, llvm::SmallVectorImpl<clang:: Expr*> const&, clang::LookupResult&, clang::DeclarationNameInfo&, clang::TemplateArgumentListInfo const*, clang::ASTContext&, clang::Parser&, clang::Sema&, cling::LookupHelper::DiagSetting) (no debug info)
[/Applications/root_v6.16.00/lib/libCling.so] clang::FunctionDecl const* cling::findFunction<clang::FunctionDecl const*>(clang::DeclContext*, llvm::StringRef, llvm::SmallVectorImplclang::Expr* const&, bool, clang::ASTContext&, cling::Interpreter*, clang::FunctionDecl const* ()(clang::DeclContext, bool, llvm::SmallVectorImpl<clang:: Expr*> const&, clang::LookupResult&, clang::DeclarationNameInfo&, clang::TemplateArgumentListInfo const*, clang::ASTContext&, clang::Parser&, clang::Sema&, cling::LookupHelper::DiagSetting), cling::LookupHelper::DiagSetting) (no debug info)
[/Applications/root_v6.16.00/lib/libCling.so] clang::FunctionDecl const* cling::execFindFunction<cling::ExprFromTypes, clang::FunctionDecl const*>(clang::Parser&, cling::Interpreter*, cling::LookupHelper&, clang::Decl const*, llvm::StringRef, cling::ExprFromTypes::ArgsInput const&, bool, clang::FunctionDecl const* ()(clang::DeclContext, bool, llvm::SmallVectorImpl<clang:: Expr*> const&, clang::LookupResult&, clang::DeclarationNameInfo&, clang::TemplateArgumentListInfo const*, clang::ASTContext&, clang::Parser&, clang::Sema&, cling::LookupHelper::DiagSetting), cling::LookupHelper::DiagSetting) (no debug info)
[/Applications/root_v6.16.00/lib/libCling.so] cling::LookupHelper::findFunctionProto(clang::Decl const*, llvm::StringRef, llvm::SmallVectorImpl<clang:: QualType> const&, cling::LookupHelper::DiagSetting, bool) const (no debug info)
[/Applications/root_v6.16.00/lib/libCling.so] TClingClassInfo::GetMethod(char const*, llvm::SmallVectorImpl<clang:: QualType> const&, bool, long*, ROOT::EFunctionMatchMode, TClingClassInfo::EInheritanceMode) const (no debug info)
[/Applications/root_v6.16.00/lib/libCling.so] TClingCallFunc::SetFuncProto(TClingClassInfo const*, char const*, llvm::SmallVectorImpl<clang:: QualType> const&, bool, long*, ROOT::EFunctionMatchMode) (no debug info)
[/Applications/root_v6.16.00/lib/libCling.so] TCling::CallFunc_SetFuncProto(CallFunc_t*, ClassInfo_t*, char const*, std::__1::vector<TypeInfo_t*, std::__1::allocator<TypeInfo_t*> > const&, bool, long*, ROOT::EFunctionMatchMode) const (no debug info)
[/Applications/root_v6.16.00/lib/libHist.6.16.00.so] ROOT::v5::TFormula::AnalyzeFunction(TString&, int&, int) (no debug info)
[/Applications/root_v6.16.00/lib/libHist.6.16.00.so] ROOT::v5::TFormula::Analyze(char const*, int&, int) (no debug info)
[/Applications/root_v6.16.00/lib/libHist.6.16.00.so] ROOT::v5::TFormula::Compile(char const*) (no debug info)
[/Applications/root_v6.16.00/lib/libTreePlayer.6.16.00.so] TTreeFormula::Init(char const*, char const*) (no debug info)
[/Applications/root_v6.16.00/lib/libTreePlayer.6.16.00.so] TTreeFormula::TTreeFormula(char const*, char const*, TTree*) (no debug info)
[/Users/quentin/software/HAPPy/ROOT/MultiHistogramFill_cxx.so] MultiHistogramFill::addTTreeFormula(std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&) (no debug info)
[/Users/quentin/software/HAPPy/ROOT/MultiHistogramFill_cxx.so] MultiHistogramFill::fill(long long const&) (no debug info)
[] (no debug info)
[/Applications/root_v6.16.00/lib/libPyROOT.so] FastCall(long, void*, void*, void*) (no debug info)
[/Applications/root_v6.16.00/lib/libPyROOT.so] Cppyy::CallB(long, void*, void*) (no debug info)
[/Applications/root_v6.16.00/lib/libPyROOT.so] PyROOT::TBoolExecutor::Execute(long, void*, PyROOT::TCallContext*) (no debug info)
[/Applications/root_v6.16.00/lib/libPyROOT.so] PyROOT::TMethodHolder::CallFast(void*, long, PyROOT::TCallContext*) (no debug info)
[/Applications/root_v6.16.00/lib/libPyROOT.so] PyROOT::TMethodHolder::CallSafe(void*, long, PyROOT::TCallContext*) (no debug info)
[/Applications/root_v6.16.00/lib/libPyROOT.so] PyROOT::TMethodHolder::Execute(void*, long, PyROOT::TCallContext*) (no debug info)
[/Applications/root_v6.16.00/lib/libPyROOT.so] PyROOT::TMethodHolder::Call(PyROOT::ObjectProxy*&, _object*, _object*, PyROOT::TCallContext*) (no debug info)
[/Applications/root_v6.16.00/lib/libPyROOT.so] PyROOT::(anonymous namespace)::mp_call(PyROOT::MethodProxy*, _object*, _object*) (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] PyObject_Call (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] PyEval_EvalFrameEx (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] PyEval_EvalCodeEx (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] PyObject_Call (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] PyEval_EvalFrameEx (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] PyEval_EvalFrameEx (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] PyEval_EvalFrameEx (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] PyEval_EvalFrameEx (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] PyEval_EvalCodeEx (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] PyObject_Call (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] PyObject_Call (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] PyObject_Call (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] PyEval_EvalFrameEx (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] PyEval_EvalFrameEx (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] PyEval_EvalCodeEx (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] PyEval_EvalFrameEx (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] PyEval_EvalCodeEx (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] PyEval_EvalFrameEx (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] PyEval_EvalCodeEx (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] PyEval_EvalFrameEx (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] PyEval_EvalCodeEx (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] PyEval_EvalCode (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] PyRun_FileExFlags (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] PyRun_SimpleFileExFlags (no debug info)
[/System/Library/Frameworks/Python.framework/Versions/2.7/Python] Py_Main (no debug info)


ROOT Version: v6.16.00
Platform: MacOS.10.14.3
Compiler: clang


1 Like

Hi Quentin,

ROOT these days provides declarative analysis with RDataFrame and perhaps this will be a solution for you given that you can add new “columns” to your dataset, analyse it and even re-dump it on disk with the new variables. All examples can be found here.

Cheers,
D

Hi Danilo,

I am aware of this new root feature and this is indeed quite exciting and I am looking forward to use it in the future.

However my question relates to an ongoing analysis and I would like to avoid changing the core of our analysis software (based on TTree::Draw) if possible.

Cheers,

Quentin

I think there is an open JIRA ticket for this: https://sft.its.cern.ch/jira/browse/ROOT-7749

Thanks!

Indeed it seems that it is the same problem I have.
I will watch the issue.

Cheers,
Quentin

Hi,

I’ve posted a message two days ago to https://sft.its.cern.ch/jira/browse/ROOT-7749 but I have not received any answer.

Since this ticket was opened in 2015 and was never addressed, I assume my chances of getting an answer (even will not fix) are very low.

Is there another channel I could use to bring this issue to the attention of the ROOT developers?

Thanks!
Quentin

Yes, through using RDataFrame.

Is there a hard limit on the number of variables I can pass? If yes, where is it documented?

There is no obvious limit. Could you send the result of running the failing case through valgrind --suppressions=$ROOTSYS/etc/valgrind-root.supp ...

Cheers,
Philippe.

Hi,

Thanks a lot for looking into it!

I have attached the Valgrind report.

Cheers,
Quentin

valgrin.txt (49.7 KB)

@vvassilev Do you know what could be going on here? The error “seems” to be internal to clang:

==17219== Invalid read of size 1
==17219==    at 0x9C28E09: TryCopyInitialization(clang::Sema&, clang::Expr*, clang::QualType, bool, bool, bool, bool) (in /cvmfs/sft.cern.ch/lcg/releases/ROOT/6.10.04-22868/x86_64-slc6-gcc62-dbg/lib/libCling.so)
==17219==    by 0x9C251AC: clang::Sema::AddOverloadCandidate(clang::FunctionDecl*, clang::DeclAccessPair, llvm::ArrayRef<clang::Expr*>, clang::OverloadCandidateSet&, bool, bool, bool) (in /cvmfs/sft.cern.ch/lcg/releases/ROOT/6.10.04-22868/x86_64-slc6-gcc62-dbg/lib/libCling.so)
==17219==    by 0x88B3B40: cling::overloadFunctionSelector(clang::DeclContext*, bool, llvm::SmallVectorImpl<clang::Expr*> const&, clang::LookupResult&, clang::DeclarationNameInfo&, clang::TemplateArgumentListInfo const*, clang::ASTContext&, clang::Parser&, clang::Sema&) (in /cvmfs/sft.cern.ch/lcg/releases/ROOT/6.10.04-22868/x86_64-slc6-gcc62-dbg/lib/libCling.so)
==17219==    by 0x88B6088: clang::FunctionDecl const* cling::findFunction<clang::FunctionDecl const*>(clang::DeclContext*, llvm::StringRef, llvm::SmallVectorImpl<clang::Expr*> const&, bool, clang::ASTContext&, cling::Interpreter*, clang::FunctionDecl const* (*)(clang::DeclContext*, bool, llvm::SmallVectorImpl<clang::Expr*> const&, clang::LookupResult&, clang::DeclarationNameInfo&, clang::TemplateArgumentListInfo const*, clang::ASTContext&, clang::Parser&, clang::Sema&), cling::LookupHelper::DiagSetting) (in /cvmfs/sft.cern.ch/lcg/releases/ROOT/6.10.04-22868/x86_64-slc6-gcc62-dbg/lib/libCling.so)
==17219==    by 0x88B64E0: clang::FunctionDecl const* cling::execFindFunction<cling::ExprFromTypes, clang::FunctionDecl const*>(clang::Parser&, cling::Interpreter*, clang::Decl const*, llvm::StringRef, cling::ExprFromTypes::ArgsInput const&, bool, clang::FunctionDecl const* (*)(clang::DeclContext*, bool, llvm::SmallVectorImpl<clang::Expr*> const&, clang::LookupResult&, clang::DeclarationNameInfo&, clang::TemplateArgumentListInfo const*, clang::ASTContext&, clang::Parser&, clang::Sema&), cling::LookupHelper::DiagSetting) (in /cvmfs/sft.cern.ch/lcg/releases/ROOT/6.10.04-22868/x86_64-slc6-gcc62-dbg/lib/libCling.so)
==17219==    by 0x88B65DE: cling::LookupHelper::findFunctionProto(clang::Decl const*, llvm::StringRef, llvm::SmallVectorImpl<clang::QualType> const&, cling::LookupHelper::DiagSetting, bool) const (in /cvmfs/sft.cern.ch/lcg/releases/ROOT/6.10.04-22868/x86_64-slc6-gcc62-dbg/lib/libCling.so)
==17219==    by 0x887B7B3: TClingClassInfo::GetMethod(char const*, llvm::SmallVectorImpl<clang::QualType> const&, bool, long*, ROOT::EFunctionMatchMode, TClingClassInfo::EInheritanceMode) const (TClingClassInfo.cxx:475)
==17219==    by 0x8870037: TClingCallFunc::SetFuncProto(TClingClassInfo const*, char const*, llvm::SmallVectorImpl<clang::QualType> const&, bool, long*, ROOT::EFunctionMatchMode) (TClingCallFunc.cxx:2382)
==17219==    by 0x87CB9AD: TCling::CallFunc_SetFuncProto(CallFunc_t*, ClassInfo_t*, char const*, std::vector<TypeInfo_t*, std::allocator<TypeInfo_t*> > const&, bool, long*, ROOT::EFunctionMatchMode) const (TCling.cxx:6852)
==17219==    by 0x1B87301A: ROOT::v5::TFormula::AnalyzeFunction(TString&, int&, int) (TFormula_v5.cxx:435)
==17219==    by 0x1B87ED16: ROOT::v5::TFormula::Analyze(char const*, int&, int) (TFormula_v5.cxx:2069)
==17219==    by 0x1B881574: ROOT::v5::TFormula::Compile(char const*) (TFormula_v5.cxx:2367)
==17219==  Address 0x1c803110 is 128 bytes inside a block of size 288 freed
==17219==    at 0x4C29BA4: realloc (vg_replace_malloc.c:785)
==17219==    by 0xA1FEA56: llvm::SmallVectorBase::grow_pod(void*, unsigned long, unsigned long) (in /cvmfs/sft.cern.ch/lcg/releases/ROOT/6.10.04-22868/x86_64-slc6-gcc62-dbg/lib/libCling.so)
==17219==    by 0x88B6566: clang::FunctionDecl const* cling::execFindFunction<cling::ExprFromTypes, clang::FunctionDecl const*>(clang::Parser&, cling::Interpreter*, clang::Decl const*, llvm::StringRef, cling::ExprFromTypes::ArgsInput const&, bool, clang::FunctionDecl const* (*)(clang::DeclContext*, bool, llvm::SmallVectorImpl<clang::Expr*> const&, clang::LookupResult&, clang::DeclarationNameInfo&, clang::TemplateArgumentListInfo const*, clang::ASTContext&, clang::Parser&, clang::Sema&), cling::LookupHelper::DiagSetting) (in /cvmfs/sft.cern.ch/lcg/releases/ROOT/6.10.04-22868/x86_64-slc6-gcc62-dbg/lib/libCling.so)
==17219==    by 0x88B65DE: cling::LookupHelper::findFunctionProto(clang::Decl const*, llvm::StringRef, llvm::SmallVectorImpl<clang::QualType> const&, cling::LookupHelper::DiagSetting, bool) const (in /cvmfs/sft.cern.ch/lcg/releases/ROOT/6.10.04-22868/x86_64-slc6-gcc62-dbg/lib/libCling.so)
==17219==    by 0x887B7B3: TClingClassInfo::GetMethod(char const*, llvm::SmallVectorImpl<clang::QualType> const&, bool, long*, ROOT::EFunctionMatchMode, TClingClassInfo::EInheritanceMode) const (TClingClassInfo.cxx:475)
==17219==    by 0x8870037: TClingCallFunc::SetFuncProto(TClingClassInfo const*, char const*, llvm::SmallVectorImpl<clang::QualType> const&, bool, long*, ROOT::EFunctionMatchMode) (TClingCallFunc.cxx:2382)
==17219==    by 0x87CB9AD: TCling::CallFunc_SetFuncProto(CallFunc_t*, ClassInfo_t*, char const*, std::vector<TypeInfo_t*, std::allocator<TypeInfo_t*> > const&, bool, long*, ROOT::EFunctionMatchMode) const (TCling.cxx:6852)
==17219==    by 0x1B87301A: ROOT::v5::TFormula::AnalyzeFunction(TString&, int&, int) (TFormula_v5.cxx:435)
==17219==    by 0x1B87ED16: ROOT::v5::TFormula::Analyze(char const*, int&, int) (TFormula_v5.cxx:2069)
==17219==    by 0x1B881574: ROOT::v5::TFormula::Compile(char const*) (TFormula_v5.cxx:2367)
==17219==    by 0x1A6BC25D: TTreeFormula::Init(char const*, char const*) (TTreeFormula.cxx:209)
==17219==    by 0x1A6BBCAE: TTreeFormula::TTreeFormula(char const*, char const*, TTree*) (TTreeFormula.cxx:162)
==17219==  Block was allocated at
==17219==    at 0x4C27B4E: malloc (vg_replace_malloc.c:299)
==17219==    by 0xA1FEA84: llvm::SmallVectorBase::grow_pod(void*, unsigned long, unsigned long) (in /cvmfs/sft.cern.ch/lcg/releases/ROOT/6.10.04-22868/x86_64-slc6-gcc62-dbg/lib/libCling.so)
==17219==    by 0x88B6566: clang::FunctionDecl const* cling::execFindFunction<cling::ExprFromTypes, clang::FunctionDecl const*>(clang::Parser&, cling::Interpreter*, clang::Decl const*, llvm::StringRef, cling::ExprFromTypes::ArgsInput const&, bool, clang::FunctionDecl const* (*)(clang::DeclContext*, bool, llvm::SmallVectorImpl<clang::Expr*> const&, clang::LookupResult&, clang::DeclarationNameInfo&, clang::TemplateArgumentListInfo const*, clang::ASTContext&, clang::Parser&, clang::Sema&), cling::LookupHelper::DiagSetting) (in /cvmfs/sft.cern.ch/lcg/releases/ROOT/6.10.04-22868/x86_64-slc6-gcc62-dbg/lib/libCling.so)
==17219==    by 0x88B65DE: cling::LookupHelper::findFunctionProto(clang::Decl const*, llvm::StringRef, llvm::SmallVectorImpl<clang::QualType> const&, cling::LookupHelper::DiagSetting, bool) const (in /cvmfs/sft.cern.ch/lcg/releases/ROOT/6.10.04-22868/x86_64-slc6-gcc62-dbg/lib/libCling.so)
==17219==    by 0x887B7B3: TClingClassInfo::GetMethod(char const*, llvm::SmallVectorImpl<clang::QualType> const&, bool, long*, ROOT::EFunctionMatchMode, TClingClassInfo::EInheritanceMode) const (TClingClassInfo.cxx:475)
==17219==    by 0x8870037: TClingCallFunc::SetFuncProto(TClingClassInfo const*, char const*, llvm::SmallVectorImpl<clang::QualType> const&, bool, long*, ROOT::EFunctionMatchMode) (TClingCallFunc.cxx:2382)
==17219==    by 0x87CB9AD: TCling::CallFunc_SetFuncProto(CallFunc_t*, ClassInfo_t*, char const*, std::vector<TypeInfo_t*, std::allocator<TypeInfo_t*> > const&, bool, long*, ROOT::EFunctionMatchMode) const (TCling.cxx:6852)
==17219==    by 0x1B87301A: ROOT::v5::TFormula::AnalyzeFunction(TString&, int&, int) (TFormula_v5.cxx:435)
==17219==    by 0x1B87ED16: ROOT::v5::TFormula::Analyze(char const*, int&, int) (TFormula_v5.cxx:2069)
==17219==    by 0x1B881574: ROOT::v5::TFormula::Compile(char const*) (TFormula_v5.cxx:2367)
==17219==    by 0x1A6BC25D: TTreeFormula::Init(char const*, char const*) (TTreeFormula.cxx:209)
==17219==    by 0x1A6BBCAE: TTreeFormula::TTreeFormula(char const*, char const*, TTree*) (TTreeFormula.cxx:162)

Thanks,
Philippe.

Hi @vvassilev,

Any chance you can have a look at this soon?
I’m happy to do some extra tests if this can help to gain insight on what’s going on.

Cheers,
Quentin

Indeed, it looks like a bug in clang. We will be upgrading our fork of clang soon, so unless it is a critical issue I would not bother fixing it. It is very likely fixed already in the newer versions.

Hi @vvassilev ,

Thanks for the update!

Do you have a time estimate for when this upgrade will happen?
This issue is directly affecting an ongoing ATLAS analysis.

Also note that it was reported already a long time ago (https://sft.its.cern.ch/jira/browse/ROOT-7749). I don’t know how many clang ‘upgrade’ you made since 2015 but I am assuming it’s not zero.

Best,
Quentin

For reference, the issue was fixed and will soon be backported to older releases (see https://sft.its.cern.ch/jira/browse/ROOT-10080)

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.