Merging TTrees on the fly with TTree.MergeTrees() in PyROOT

apetukho · January 29, 2019, 1:59pm

Dear ROOT experts,

I’ve ran into a problem trying to merge TTrees on the fly with TTree.MergeTrees() in PyROOT. I get a lot of similar trees (same tree names, same branch name and structure ) in my program and I want to merge the results. However, when I try to use TList and TTree.MergeTrees() I get the last tree I added to the list repeated as much times as there entries in the TList.

Here’s the mock-up of the code I use.
test1.root contains tree with 1 entry.
test2.root contains tree with 2 entries.

from ROOT import TFile, TTree, TList

pathList = ['test1.root', 'test2.root']
treeList = TList()
outputFile = TFile('MergeTest.root', 'recreate')
 
for path in pathList:
	inputFile = TFile(path, 'read')
	inputTree = inputFile.Get('TestTree')
	outputTree = inputTree.CloneTree() #instead of an extensive processing
	treeList.Add(inputTree)

outputFile.cd()
outputTree = TTree.MergeTrees(treeList)
outputFile.Write()
outputFile.Close()

When I run this code I get contents of test2.root repeated 2 times, instead of a merged tree from test1.root and test2.root.
What could be the problem here?

Thanks in advance.

MergeTest.root (5.3 KB)
test1.root (5.2 KB)
test2.root (5.2 KB)
MergeShort.py (445 Bytes)

ROOT Version: 6.14/04
Platform: lxplus

etejedor · January 29, 2019, 4:04pm

Hi @apetukho,

You need to do this:

from ROOT import TFile, TTree, TList

pathList = ['test1.root', 'test2.root']
treeList = TList()
outputFile = TFile('MergeTest.root', 'recreate')
pyfilelist = []
pytreelist = []

for path in pathList:
        print("Path", path)
        inputFile = TFile(path, 'read')
        pyfilelist.append(inputFile) # Make this TFile survive the loop!
        inputTree = inputFile.Get('TestTree')
        pytreelist.append(inputTree) # Make this TTree survive the loop!
        outputTree = inputTree.CloneTree() #instead of extensive processing
        treeList.Add(inputTree)

outputFile.cd()
outputTree = TTree.MergeTrees(treeList)
outputFile.Write()
outputFile.Close()

Basically, if you create the TFile and TTree inside a loop, when the corresponding iteration finishes they go out of scope in Python. When this happens, PyROOT deletes the underlying C++ object. But both the two files and the two trees need to be alive when you run MergeTrees. Keeping their Python references in a list accomplishes this.

Cheers,
Enric

apetukho · January 31, 2019, 1:05pm

Thanks for the reply!
Your solution works for this example, but I get some strange errors when I try to apply it to my code.

Here’s the reduced version of my script

from ROOT import TFile, TTree, TList
from ConvertTree import ConvertTree

pathList = ['0.root', '1.root', '2.root']

outputFile = TFile('output.root', 'recreate')

treeList = TList()
pyInputFileList = list()
pyOutputTreeList = list()
pyInputTreeList = list()

for path in pathList:
	print path
	inputFile = TFile(path, 'read')
	inputTree = inputFile.Get('output_tree')
	outputFile.cd()
	outputTree = ConvertTree(inputTree) #processing in a separate ConvertTree.py module
	treeList.Add(outputTree)
	pyInputFileList.append(inputFile)
	pyOutputTreeList.append(outputTree)
	pyInputTreeList.append(inputTree)

outputFile.cd()
outputTree = TTree.MergeTrees(treeList)
outputFile.Write()
outputFile.Close()

When I run it I get all kinds of segfaults on the outputFile.Write() step.

To make sure everything is with the processing script I processed the files in a separate script:

from ROOT import TFile, TTree, TList
from ConvertTree import ConvertTree

pathList = ['0.root', '1.root', '2.root']

treeList = TList()

for idx, path in enumerate(pathList):
	inputFile = TFile(path, 'read')
	inputTree = inputFile.Get('output_tree')
	outputFile = TFile('output{0}.root'.format(idx), 'recreate')
	outputTree = ConvertTree(inputTree)
	outputFile.Write()
	outputFile.Close

and then tried to merge the resulting trees:

from ROOT import TFile, TTree, TList

pathList = ['output0.root', 'output1.root', 'output2.root']
treeList = TList()
outputFile = TFile('MergeTest.root', 'recreate')
pyFileList = list()
pyTreeList = list()
 
for path in pathList:
	inputFile = TFile(path, 'read')
	pyFileList.append(inputFile)
	inputTree = inputFile.Get('TMVA_input')
	outputFile.cd()
	outputTree = inputTree.CloneTree()
	pyTreeList.append(outputTree)
	treeList.Add(outputTree)

outputFile.cd()
outputTree = TTree.MergeTrees(treeList) 
outputFile.Write()
outputFile.Close()

and everything works just fine.

I’m very confused to what is causing all those segfaults when I try to merge the processed on the fly. What could be the problem?

etejedor · January 31, 2019, 3:24pm

Hi @apetukho
According to the documentation:

https://root.cern.ch/doc/master/classTTree.html#a8d4212a53f96f9c3bcbd384892d3bd81

the trees you merge can be memory or disk resident, so it should be possible to do it in one go (it indeed was possible for the simpler example).

Do the segfault messages give any hint on the possible cause? What changes with respect to the simpler script that does work?

apetukho · February 1, 2019, 8:56am

Do the segfault messages give any hint on the possible cause?

They don’t make much sense to me, unfortunately. That’s what I got the last time I’ve tried to run the code with processing and merging:

 *** Break *** segmentation violation



===========================================================
There was a crash.
This is the entire stack trace of all threads:
===========================================================
#0  0x00007f374c3be89e in waitpid () from /lib64/libc.so.6
#1  0x00007f374c3504e9 in do_system () from /lib64/libc.so.6
#2  0x00007f3742468ebd in TUnixSystem::StackTrace() () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCore.so
#3  0x00007f374246b624 in TUnixSystem::DispatchSignals(ESignals) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCore.so
#4  <signal handler called>
#5  0x00007f373bf20090 in cling::IncrementalJIT::getSymbolAddressWithoutMangling(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCling.so
#6  0x00007f373bf1b7ab in cling::IncrementalExecutor::getPointerToGlobalFromJIT(llvm::GlobalValue const&) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCling.so
#7  0x00007f373be7dad0 in TClingCallFunc::make_wrapper() () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCling.so
#8  0x00007f373be7de2a in TClingCallFunc::IFacePtr() () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCling.so
#9  0x00007f373be3611c in TCling::CallFunc_IFacePtr(CallFunc_t*) const () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCling.so
#10 0x00007f3745dc76a5 in FastCall(long, void*, void*, void*) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libPyROOT.so
#11 0x00007f3745dc883f in Cppyy::CallI(long, void*, void*) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libPyROOT.so
#12 0x00007f3745dcb2ff in PyROOT::TIntExecutor::Execute(long, void*, PyROOT::TCallContext*) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libPyROOT.so
#13 0x00007f3745def782 in PyROOT::TMethodHolder::CallSafe(void*, long, PyROOT::TCallContext*) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libPyROOT.so
#14 0x00007f3745deeb24 in PyROOT::TMethodHolder::Call(PyROOT::ObjectProxy*&, _object*, _object*, PyROOT::TCallContext*) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libPyROOT.so
#15 0x00007f3745dd1117 in PyROOT::(anonymous namespace)::mp_call(PyROOT::MethodProxy*, _object*, _object*) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libPyROOT.so
#16 0x00007f374cfa2af3 in PyObject_Call () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#17 0x00007f374d05a02b in PyEval_EvalFrameEx () at /mnt/build/jenkins/workspace/lcg_release_latest/BUILDTYPE/Release/COMPILER/gcc62binutils/LABEL/slc6/build/externals/Python-2.7.15/src/Python/2.7.15/Python/ceval.c:4589
#18 0x00007f374d05e3ec in PyEval_EvalCodeEx () at /mnt/build/jenkins/workspace/lcg_release_latest/BUILDTYPE/Release/COMPILER/gcc62binutils/LABEL/slc6/build/externals/Python-2.7.15/src/Python/2.7.15/Python/ceval.c:3604
#19 0x00007f374d05e4e9 in PyEval_EvalCode () at /mnt/build/jenkins/workspace/lcg_release_latest/BUILDTYPE/Release/COMPILER/gcc62binutils/LABEL/slc6/build/externals/Python-2.7.15/src/Python/2.7.15/Python/ceval.c:669
#20 0x00007f374d0828ba in PyRun_FileExFlags () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#21 0x00007f374d083c95 in PyRun_SimpleFileExFlags () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#22 0x00007f374d09a3d1 in Py_Main () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#23 0x00007f374c330d20 in __libc_start_main () from /lib64/libc.so.6
#24 0x0000000000400641 in _start ()
===========================================================


The lines below might hint at the cause of the crash.
You may get help by asking at the ROOT forum http://root.cern.ch/forum
Only if you are really convinced it is a bug in ROOT then please submit a
report at http://root.cern.ch/bugs Please post the ENTIRE stack trace
from above as an attachment in addition to anything else
that might help us fixing this issue.
===========================================================
#5  0x00007f373bf20090 in cling::IncrementalJIT::getSymbolAddressWithoutMangling(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCling.so
#6  0x00007f373bf1b7ab in cling::IncrementalExecutor::getPointerToGlobalFromJIT(llvm::GlobalValue const&) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCling.so
#7  0x00007f373be7dad0 in TClingCallFunc::make_wrapper() () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCling.so
#8  0x00007f373be7de2a in TClingCallFunc::IFacePtr() () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCling.so
#9  0x00007f373be3611c in TCling::CallFunc_IFacePtr(CallFunc_t*) const () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCling.so
#10 0x00007f3745dc76a5 in FastCall(long, void*, void*, void*) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libPyROOT.so
#11 0x00007f3745dc883f in Cppyy::CallI(long, void*, void*) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libPyROOT.so
#12 0x00007f3745dcb2ff in PyROOT::TIntExecutor::Execute(long, void*, PyROOT::TCallContext*) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libPyROOT.so
#13 0x00007f3745def782 in PyROOT::TMethodHolder::CallSafe(void*, long, PyROOT::TCallContext*) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libPyROOT.so
#14 0x00007f3745deeb24 in PyROOT::TMethodHolder::Call(PyROOT::ObjectProxy*&, _object*, _object*, PyROOT::TCallContext*) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libPyROOT.so
#15 0x00007f3745dd1117 in PyROOT::(anonymous namespace)::mp_call(PyROOT::MethodProxy*, _object*, _object*) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libPyROOT.so
#16 0x00007f374cfa2af3 in PyObject_Call () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#17 0x00007f374d05a02b in PyEval_EvalFrameEx () at /mnt/build/jenkins/workspace/lcg_release_latest/BUILDTYPE/Release/COMPILER/gcc62binutils/LABEL/slc6/build/externals/Python-2.7.15/src/Python/2.7.15/Python/ceval.c:4589
#18 0x00007f374d05e3ec in PyEval_EvalCodeEx () at /mnt/build/jenkins/workspace/lcg_release_latest/BUILDTYPE/Release/COMPILER/gcc62binutils/LABEL/slc6/build/externals/Python-2.7.15/src/Python/2.7.15/Python/ceval.c:3604
#19 0x00007f374d05e4e9 in PyEval_EvalCode () at /mnt/build/jenkins/workspace/lcg_release_latest/BUILDTYPE/Release/COMPILER/gcc62binutils/LABEL/slc6/build/externals/Python-2.7.15/src/Python/2.7.15/Python/ceval.c:669
#20 0x00007f374d0828ba in PyRun_FileExFlags () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#21 0x00007f374d083c95 in PyRun_SimpleFileExFlags () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#22 0x00007f374d09a3d1 in Py_Main () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#23 0x00007f374c330d20 in __libc_start_main () from /lib64/libc.so.6
#24 0x0000000000400641 in _start ()
===========================================================



 *** Break *** segmentation violation



===========================================================
There was a crash.
This is the entire stack trace of all threads:
===========================================================
#0  0x00007f374c3be89e in waitpid () from /lib64/libc.so.6
#1  0x00007f374c3504e9 in do_system () from /lib64/libc.so.6
#2  0x00007f3742468ebd in TUnixSystem::StackTrace() () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCore.so
#3  0x00007f374246b624 in TUnixSystem::DispatchSignals(ESignals) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCore.so
#4  <signal handler called>
#5  0x00007f373bf20090 in cling::IncrementalJIT::getSymbolAddressWithoutMangling(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCling.so
#6  0x00007f373bf1b7ab in cling::IncrementalExecutor::getPointerToGlobalFromJIT(llvm::GlobalValue const&) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCling.so
#7  0x00007f373be7dad0 in TClingCallFunc::make_wrapper() () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCling.so
#8  0x00007f373be7de2a in TClingCallFunc::IFacePtr() () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCling.so
#9  0x00007f373be3611c in TCling::CallFunc_IFacePtr(CallFunc_t*) const () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCling.so
#10 0x00007f3745dc76a5 in FastCall(long, void*, void*, void*) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libPyROOT.so
#11 0x00007f3745dc883f in Cppyy::CallI(long, void*, void*) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libPyROOT.so
#12 0x00007f3745dcb2ff in PyROOT::TIntExecutor::Execute(long, void*, PyROOT::TCallContext*) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libPyROOT.so
#13 0x00007f3745def782 in PyROOT::TMethodHolder::CallSafe(void*, long, PyROOT::TCallContext*) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libPyROOT.so
#14 0x00007f3745deeb24 in PyROOT::TMethodHolder::Call(PyROOT::ObjectProxy*&, _object*, _object*, PyROOT::TCallContext*) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libPyROOT.so
#15 0x00007f3745dd1117 in PyROOT::(anonymous namespace)::mp_call(PyROOT::MethodProxy*, _object*, _object*) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libPyROOT.so
#16 0x00007f374cfa2af3 in PyObject_Call () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#17 0x00007f374d05a02b in PyEval_EvalFrameEx () at /mnt/build/jenkins/workspace/lcg_release_latest/BUILDTYPE/Release/COMPILER/gcc62binutils/LABEL/slc6/build/externals/Python-2.7.15/src/Python/2.7.15/Python/ceval.c:4589
#18 0x00007f374d05e3ec in PyEval_EvalCodeEx () at /mnt/build/jenkins/workspace/lcg_release_latest/BUILDTYPE/Release/COMPILER/gcc62binutils/LABEL/slc6/build/externals/Python-2.7.15/src/Python/2.7.15/Python/ceval.c:3604
#19 0x00007f374d05e4e9 in PyEval_EvalCode () at /mnt/build/jenkins/workspace/lcg_release_latest/BUILDTYPE/Release/COMPILER/gcc62binutils/LABEL/slc6/build/externals/Python-2.7.15/src/Python/2.7.15/Python/ceval.c:669
#20 0x00007f374d0828ba in PyRun_FileExFlags () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#21 0x00007f374d083c95 in PyRun_SimpleFileExFlags () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#22 0x00007f374d09a3d1 in Py_Main () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#23 0x00007f374c330d20 in __libc_start_main () from /lib64/libc.so.6
#24 0x0000000000400641 in _start ()
===========================================================


The lines below might hint at the cause of the crash.
You may get help by asking at the ROOT forum http://root.cern.ch/forum
Only if you are really convinced it is a bug in ROOT then please submit a
report at http://root.cern.ch/bugs Please post the ENTIRE stack trace
from above as an attachment in addition to anything else
that might help us fixing this issue.
===========================================================
#5  0x00007f373bf20090 in cling::IncrementalJIT::getSymbolAddressWithoutMangling(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCling.so
#6  0x00007f373bf1b7ab in cling::IncrementalExecutor::getPointerToGlobalFromJIT(llvm::GlobalValue const&) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCling.so
#7  0x00007f373be7dad0 in TClingCallFunc::make_wrapper() () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCling.so
#8  0x00007f373be7de2a in TClingCallFunc::IFacePtr() () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCling.so
#9  0x00007f373be3611c in TCling::CallFunc_IFacePtr(CallFunc_t*) const () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCling.so
#10 0x00007f3745dc76a5 in FastCall(long, void*, void*, void*) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libPyROOT.so
#11 0x00007f3745dc883f in Cppyy::CallI(long, void*, void*) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libPyROOT.so
#12 0x00007f3745dcb2ff in PyROOT::TIntExecutor::Execute(long, void*, PyROOT::TCallContext*) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libPyROOT.so
#13 0x00007f3745def782 in PyROOT::TMethodHolder::CallSafe(void*, long, PyROOT::TCallContext*) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libPyROOT.so
#14 0x00007f3745deeb24 in PyROOT::TMethodHolder::Call(PyROOT::ObjectProxy*&, _object*, _object*, PyROOT::TCallContext*) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libPyROOT.so
#15 0x00007f3745dd1117 in PyROOT::(anonymous namespace)::mp_call(PyROOT::MethodProxy*, _object*, _object*) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libPyROOT.so
#16 0x00007f374cfa2af3 in PyObject_Call () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#17 0x00007f374d05a02b in PyEval_EvalFrameEx () at /mnt/build/jenkins/workspace/lcg_release_latest/BUILDTYPE/Release/COMPILER/gcc62binutils/LABEL/slc6/build/externals/Python-2.7.15/src/Python/2.7.15/Python/ceval.c:4589
#18 0x00007f374d05e3ec in PyEval_EvalCodeEx () at /mnt/build/jenkins/workspace/lcg_release_latest/BUILDTYPE/Release/COMPILER/gcc62binutils/LABEL/slc6/build/externals/Python-2.7.15/src/Python/2.7.15/Python/ceval.c:3604
#19 0x00007f374d05e4e9 in PyEval_EvalCode () at /mnt/build/jenkins/workspace/lcg_release_latest/BUILDTYPE/Release/COMPILER/gcc62binutils/LABEL/slc6/build/externals/Python-2.7.15/src/Python/2.7.15/Python/ceval.c:669
#20 0x00007f374d0828ba in PyRun_FileExFlags () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#21 0x00007f374d083c95 in PyRun_SimpleFileExFlags () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#22 0x00007f374d09a3d1 in Py_Main () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#23 0x00007f374c330d20 in __libc_start_main () from /lib64/libc.so.6
#24 0x0000000000400641 in _start ()
===========================================================


Traceback (most recent call last):
  File "DatasetConvert.py", line 33, in <module>
    outputFile.Write()
SystemError: none of the 2 overloaded methods succeeded. Full details:
  int TFile::Write(const char* name = 0, int opt = 0, int bufsiz = 0) =>
    problem in C++; program state has been reset
  int TFile::Write(const char* name = 0, int opt = 0, int bufsiz = 0) =>
    problem in C++; program state has been reset

 *** Break *** segmentation violation



===========================================================
There was a crash.
This is the entire stack trace of all threads:
===========================================================
#0  0x00007f374c3be89e in waitpid () from /lib64/libc.so.6
#1  0x00007f374c3504e9 in do_system () from /lib64/libc.so.6
#2  0x00007f3742468ebd in TUnixSystem::StackTrace() () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCore.so
#3  0x00007f374246b624 in TUnixSystem::DispatchSignals(ESignals) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCore.so
#4  <signal handler called>
#5  0x00007f373bf20090 in cling::IncrementalJIT::getSymbolAddressWithoutMangling(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCling.so
#6  0x00007f373bf1b7ab in cling::IncrementalExecutor::getPointerToGlobalFromJIT(llvm::GlobalValue const&) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCling.so
#7  0x00007f373be7dad0 in TClingCallFunc::make_wrapper() () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCling.so
#8  0x00007f373be7de2a in TClingCallFunc::IFacePtr() () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCling.so
#9  0x00007f373be3611c in TCling::CallFunc_IFacePtr(CallFunc_t*) const () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCling.so
#10 0x00007f3745dc76a5 in FastCall(long, void*, void*, void*) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libPyROOT.so
#11 0x00007f3745dc87ac in Cppyy::CallB(long, void*, void*) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libPyROOT.so
#12 0x00007f3745dcb72f in PyROOT::TBoolExecutor::Execute(long, void*, PyROOT::TCallContext*) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libPyROOT.so
#13 0x00007f3745def782 in PyROOT::TMethodHolder::CallSafe(void*, long, PyROOT::TCallContext*) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libPyROOT.so
#14 0x00007f3745deeb24 in PyROOT::TMethodHolder::Call(PyROOT::ObjectProxy*&, _object*, _object*, PyROOT::TCallContext*) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libPyROOT.so
#15 0x00007f3745dd1117 in PyROOT::(anonymous namespace)::mp_call(PyROOT::MethodProxy*, _object*, _object*) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libPyROOT.so
#16 0x00007f374cfa2af3 in PyObject_Call () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#17 0x00007f374d05a02b in PyEval_EvalFrameEx () at /mnt/build/jenkins/workspace/lcg_release_latest/BUILDTYPE/Release/COMPILER/gcc62binutils/LABEL/slc6/build/externals/Python-2.7.15/src/Python/2.7.15/Python/ceval.c:4589
#18 0x00007f374d05e3ec in PyEval_EvalCodeEx () at /mnt/build/jenkins/workspace/lcg_release_latest/BUILDTYPE/Release/COMPILER/gcc62binutils/LABEL/slc6/build/externals/Python-2.7.15/src/Python/2.7.15/Python/ceval.c:3604
#19 0x00007f374cfd3ddd in function_call () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#20 0x00007f374cfa2af3 in PyObject_Call () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#21 0x00007f374d058f67 in PyEval_EvalFrameEx () at /mnt/build/jenkins/workspace/lcg_release_latest/BUILDTYPE/Release/COMPILER/gcc62binutils/LABEL/slc6/build/externals/Python-2.7.15/src/Python/2.7.15/Python/ceval.c:4686
#22 0x00007f374d05e3ec in PyEval_EvalCodeEx () at /mnt/build/jenkins/workspace/lcg_release_latest/BUILDTYPE/Release/COMPILER/gcc62binutils/LABEL/slc6/build/externals/Python-2.7.15/src/Python/2.7.15/Python/ceval.c:3604
#23 0x00007f374cfd3d0c in function_call () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#24 0x00007f374cfa2af3 in PyObject_Call () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#25 0x00007f374d053ea7 in PyEval_CallObjectWithKeywords () at /mnt/build/jenkins/workspace/lcg_release_latest/BUILDTYPE/Release/COMPILER/gcc62binutils/LABEL/slc6/build/externals/Python-2.7.15/src/Python/2.7.15/Python/ceval.c:4241
#26 0x00007f374d083637 in Py_Finalize () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#27 0x00007f374d099d3c in Py_Main () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#28 0x00007f374c330d20 in __libc_start_main () from /lib64/libc.so.6
#29 0x0000000000400641 in _start ()
===========================================================


The lines below might hint at the cause of the crash.
You may get help by asking at the ROOT forum http://root.cern.ch/forum
Only if you are really convinced it is a bug in ROOT then please submit a
report at http://root.cern.ch/bugs Please post the ENTIRE stack trace
from above as an attachment in addition to anything else
that might help us fixing this issue.
===========================================================
#5  0x00007f373bf20090 in cling::IncrementalJIT::getSymbolAddressWithoutMangling(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCling.so
#6  0x00007f373bf1b7ab in cling::IncrementalExecutor::getPointerToGlobalFromJIT(llvm::GlobalValue const&) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCling.so
#7  0x00007f373be7dad0 in TClingCallFunc::make_wrapper() () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCling.so
#8  0x00007f373be7de2a in TClingCallFunc::IFacePtr() () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCling.so
#9  0x00007f373be3611c in TCling::CallFunc_IFacePtr(CallFunc_t*) const () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCling.so
#10 0x00007f3745dc76a5 in FastCall(long, void*, void*, void*) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libPyROOT.so
#11 0x00007f3745dc87ac in Cppyy::CallB(long, void*, void*) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libPyROOT.so
#12 0x00007f3745dcb72f in PyROOT::TBoolExecutor::Execute(long, void*, PyROOT::TCallContext*) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libPyROOT.so
#13 0x00007f3745def782 in PyROOT::TMethodHolder::CallSafe(void*, long, PyROOT::TCallContext*) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libPyROOT.so
#14 0x00007f3745deeb24 in PyROOT::TMethodHolder::Call(PyROOT::ObjectProxy*&, _object*, _object*, PyROOT::TCallContext*) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libPyROOT.so
#15 0x00007f3745dd1117 in PyROOT::(anonymous namespace)::mp_call(PyROOT::MethodProxy*, _object*, _object*) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libPyROOT.so
#16 0x00007f374cfa2af3 in PyObject_Call () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#17 0x00007f374d05a02b in PyEval_EvalFrameEx () at /mnt/build/jenkins/workspace/lcg_release_latest/BUILDTYPE/Release/COMPILER/gcc62binutils/LABEL/slc6/build/externals/Python-2.7.15/src/Python/2.7.15/Python/ceval.c:4589
#18 0x00007f374d05e3ec in PyEval_EvalCodeEx () at /mnt/build/jenkins/workspace/lcg_release_latest/BUILDTYPE/Release/COMPILER/gcc62binutils/LABEL/slc6/build/externals/Python-2.7.15/src/Python/2.7.15/Python/ceval.c:3604
#19 0x00007f374cfd3ddd in function_call () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#20 0x00007f374cfa2af3 in PyObject_Call () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#21 0x00007f374d058f67 in PyEval_EvalFrameEx () at /mnt/build/jenkins/workspace/lcg_release_latest/BUILDTYPE/Release/COMPILER/gcc62binutils/LABEL/slc6/build/externals/Python-2.7.15/src/Python/2.7.15/Python/ceval.c:4686
#22 0x00007f374d05e3ec in PyEval_EvalCodeEx () at /mnt/build/jenkins/workspace/lcg_release_latest/BUILDTYPE/Release/COMPILER/gcc62binutils/LABEL/slc6/build/externals/Python-2.7.15/src/Python/2.7.15/Python/ceval.c:3604
#23 0x00007f374cfd3d0c in function_call () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#24 0x00007f374cfa2af3 in PyObject_Call () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#25 0x00007f374d053ea7 in PyEval_CallObjectWithKeywords () at /mnt/build/jenkins/workspace/lcg_release_latest/BUILDTYPE/Release/COMPILER/gcc62binutils/LABEL/slc6/build/externals/Python-2.7.15/src/Python/2.7.15/Python/ceval.c:4241
#26 0x00007f374d083637 in Py_Finalize () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#27 0x00007f374d099d3c in Py_Main () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#28 0x00007f374c330d20 in __libc_start_main () from /lib64/libc.so.6
#29 0x0000000000400641 in _start ()
===========================================================


Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/cvmfs/sft.cern.ch/lcg/releases/LCG_94/Python/2.7.15/x86_64-slc6-gcc62-opt/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/ROOT.py", line 768, in cleanup
    isCocoa = _root.gSystem.InheritsFrom( 'TMacOSXSystem' )
TypeError: none of the 2 overloaded methods succeeded. Full details:
  bool TObject::InheritsFrom(const char* classname) =>
    problem in C++; program state has been reset
  bool TObject::InheritsFrom(const TClass* cl) =>
    could not convert argument 1
Error in sys.exitfunc:
Traceback (most recent call last):
  File "/cvmfs/sft.cern.ch/lcg/releases/LCG_94/Python/2.7.15/x86_64-slc6-gcc62-opt/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/ROOT.py", line 768, in cleanup
    isCocoa = _root.gSystem.InheritsFrom( 'TMacOSXSystem' )
TypeError: none of the 2 overloaded methods succeeded. Full details:
  bool TObject::InheritsFrom(const char* classname) =>
    problem in C++; program state has been reset
  bool TObject::InheritsFrom(const TClass* cl) =>
    could not convert argument 1

 *** Break *** segmentation violation



===========================================================
There was a crash.
This is the entire stack trace of all threads:
===========================================================
#0  0x00007f374c3be89e in waitpid () from /lib64/libc.so.6
#1  0x00007f374c3504e9 in do_system () from /lib64/libc.so.6
#2  0x00007f3742468ebd in TUnixSystem::StackTrace() () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCore.so
#3  0x00007f374246b624 in TUnixSystem::DispatchSignals(ESignals) () from /cvmfs/sft.cern.ch/lcg/releases/LCG_94/ROOT/6.14.04/x86_64-slc6-gcc62-opt/lib/libCore.so
#4  <signal handler called>
#5  0x00007f374d09a9f0 in visit_decref () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#6  0x00007f374cfd756b in list_traverse () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#7  0x00007f374d09acaf in collect () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#8  0x00007f374d09bac8 in PyGC_Collect () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#9  0x00007f374d083679 in Py_Finalize () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#10 0x00007f374d099d3c in Py_Main () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#11 0x00007f374c330d20 in __libc_start_main () from /lib64/libc.so.6
#12 0x0000000000400641 in _start ()
===========================================================


The lines below might hint at the cause of the crash.
You may get help by asking at the ROOT forum http://root.cern.ch/forum
Only if you are really convinced it is a bug in ROOT then please submit a
report at http://root.cern.ch/bugs Please post the ENTIRE stack trace
from above as an attachment in addition to anything else
that might help us fixing this issue.
===========================================================
#5  0x00007f374d09a9f0 in visit_decref () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#6  0x00007f374cfd756b in list_traverse () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#7  0x00007f374d09acaf in collect () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#8  0x00007f374d09bac8 in PyGC_Collect () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#9  0x00007f374d083679 in Py_Finalize () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#10 0x00007f374d099d3c in Py_Main () from /cvmfs/sft.cern.ch/lcg/releases/Python/2.7.15-2cba0/x86_64-slc6-gcc62-opt/bin/../lib/libpython2.7.so.1.0
#11 0x00007f374c330d20 in __libc_start_main () from /lib64/libc.so.6
#12 0x0000000000400641 in _start ()
===========================================================

etejedor · February 1, 2019, 10:50am

Hi @apetukho

It is hard to tell from the errors, at this point you have basically two options: one is try to go step by step from the solution that works in one go until your code (that does not), and see what makes it fail, the other option is to do it in two separate steps (they can still be in the same python file, it is just that you are passing through disk) if that is not an issue.

Cheers,
Enric

system · February 15, 2019, 10:51am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.