Segfaults when using RooAbsData.Tree Storange and handling multiple TFiles

While working with RooDataSets and RooWorkspaces, I need to use a RooTreeDataStore as described in the RooDataSet docs due to the memory limitation of RooVectorDataStore.
I am using python 2.7 (it does not feel good, but that is another story) for the “high level” script and run into Segmentation Violations at the end of the script. As far as I can tell, this happens if I start opening additional TFiles, to e.g. write out a new RooWorkspace that contains a Tree-backed RooDataSet.

I tried to come up with a minimal example:

import ROOT

ws = ROOT.RooWorkspace('ws')  # workspace to store fit results, etc.

x = ROOT.RooRealVar('x', 'x', 0)  # generate some data in this dimension later

fs = ROOT.TFile('store.root', 'RECREATE')
ROOT.RooAbsData.setDefaultStorageType(ROOT.RooAbsData.Tree)  # use Tree-backed data store

data = ROOT.RooDataSet('data', 'data', ROOT.RooArgSet(x))
for v in range(1000):  # fill dataset with very meaningful data
    x.setVal(v)
    data.add(ROOT.RooArgSet(x))

getattr(ws, 'import')(data)  # add dataset to workspace
                             # `import` is a python keyword, need to use `getattr`

ws.Print('v')  # check that everything looks as expected

wf = ROOT.TFile('ws.root', 'RECREATE')  # save workspace to this file
ws.Write()
wf.Close()  # this seems to be part of the problem

# when leaving wf opened and deleting `ws` and `data`, there is no segfault
# del ws, data 

print('Done!')  # the segfault appears after this line

This is (to my eye) the relevant part of the segfault:

The lines below might hint at the cause of the crash.
You may get help by asking at the ROOT forum http://root.cern.ch/forum
Only if you are really convinced it is a bug in ROOT then please submit a
report at http://root.cern.ch/bugs Please post the ENTIRE stack trace
from above as an attachment in addition to anything else
that might help us fixing this issue.
===========================================================
#9  0x000000000278b530 in ?? ()
#10 0x00007f658dae32f3 in RooTreeDataStore::~RooTreeDataStore (this=0x27679a0, __in_chrg=<optimized out>) at /mnt/build/jenkins/workspace/lcg_release_tar/BUILDTYPE/Debug/COMPILER/gcc8testing/LABEL/centos7/build/projects/ROOT-6.18.04/src/ROOT/6.18.04/roofit/roofitcore/src/RooTreeDataStore.cxx:380
#11 0x00007f658dae338c in RooTreeDataStore::~RooTreeDataStore (this=0x27679a0, __in_chrg=<optimized out>) at /mnt/build/jenkins/workspace/lcg_release_tar/BUILDTYPE/Debug/COMPILER/gcc8testing/LABEL/centos7/build/projects/ROOT-6.18.04/src/ROOT/6.18.04/roofit/roofitcore/src/RooTreeDataStore.cxx:385
#12 0x00007f658d8f28ef in RooAbsData::~RooAbsData (this=0x38118c0, __in_chrg=<optimized out>) at /mnt/build/jenkins/workspace/lcg_release_tar/BUILDTYPE/Debug/COMPILER/gcc8testing/LABEL/centos7/build/projects/ROOT-6.18.04/src/ROOT/6.18.04/roofit/roofitcore/src/RooAbsData.cxx:261
#13 0x00007f658d9b4eab in RooDataSet::~RooDataSet (this=0x38118c0, __in_chrg=<optimized out>) at /mnt/build/jenkins/workspace/lcg_release_tar/BUILDTYPE/Debug/COMPILER/gcc8testing/LABEL/centos7/build/projects/ROOT-6.18.04/src/ROOT/6.18.04/roofit/roofitcore/src/RooDataSet.cxx:944
#14 0x00007f658d9b4edc in RooDataSet::~RooDataSet (this=0x38118c0, __in_chrg=<optimized out>) at /mnt/build/jenkins/workspace/lcg_release_tar/BUILDTYPE/Debug/COMPILER/gcc8testing/LABEL/centos7/build/projects/ROOT-6.18.04/src/ROOT/6.18.04/roofit/roofitcore/src/RooDataSet.cxx:948
#15 0x00007f658d838a23 in ROOT::delete_RooDataSet (p=0x38118c0) at /mnt/build/jenkins/workspace/lcg_release_tar/BUILDTYPE/Debug/COMPILER/gcc8testing/LABEL/centos7/build/projects/ROOT-6.18.04/src/ROOT-6.18.04-build/roofit/roofitcore/G__RooFitCore.cxx:17319
#16 0x00007f659de82093 in TClass::Destructor (this=0x37733c0, obj=0x38118c0, dtorOnly=false) at /mnt/build/jenkins/workspace/lcg_release_tar/BUILDTYPE/Debug/COMPILER/gcc8testing/LABEL/centos7/build/projects/ROOT-6.18.04/src/ROOT/6.18.04/core/meta/src/TClass.cxx:5215
#17 0x00007f65a008c811 in Cppyy::Destruct (type=33, instance=0x38118c0) at /mnt/build/jenkins/workspace/lcg_release_tar/BUILDTYPE/Debug/COMPILER/gcc8testing/LABEL/centos7/build/projects/ROOT-6.18.04/src/ROOT/6.18.04/bindings/pyroot/src/Cppyy.cxx:291
#18 0x00007f65a00a9353 in PyROOT::op_dealloc_nofree (pyobj=0x7f65a6d58410) at /mnt/build/jenkins/workspace/lcg_release_tar/BUILDTYPE/Debug/COMPILER/gcc8testing/LABEL/centos7/build/projects/ROOT-6.18.04/src/ROOT/6.18.04/bindings/pyroot/src/ObjectProxy.cxx:59
#19 0x00007f65a00a9aa9 in PyROOT::(anonymous namespace)::op_dealloc (pyobj=0x7f65a6d58410) at /mnt/build/jenkins/workspace/lcg_release_tar/BUILDTYPE/Debug/COMPILER/gcc8testing/LABEL/centos7/build/projects/ROOT-6.18.04/src/ROOT/6.18.04/bindings/pyroot/src/ObjectProxy.cxx:212

Deleting the references to ws and data manually, and don’t closing the file wf seems to solve the problem partly. At least, the script exits normally. In this case, the Tree store ends up in the ws.root file as a separate key, though.

I would have hoped that the file store.root will be used as the RooDataStore, but it seems to be unused at all.

Is there a way to point the RooTreeDataStore to a specific TFile that does not interfere with other opened files? Any help would be very appreciated :slight_smile:

ROOT Version: 6.18.04
Platform: linux-64 on Centos7
Compiler: gcc 8
Python: 2.7.16

Hi @bixel,

thanks for writing such an excellent reproducer. I came across a very similar problem recently, and I think I fixed it. Let me test if your case is also covered by that.

EDIT:
No, it wasn’t the same problem, but a similar one. I fixed it, but the fix needs to be deployed:
https://sft.its.cern.ch/jira/browse/ROOT-10475

Hi @StephanH, thanks a lot for having a look into this. Which of the problems did you fix with ROOT-10475? The initial one or your similar problem?

I came up with a workaround that I don’t totally trust myself. I’m doing something like this before I create or import any RooDataSet or RooHistPdf (which is a daughter of RooAbsData iirc):

# some global holder to the treeStore file
treeStoreFile = None

# ... quadrillion lines of beautiful analysis script code

global treeStoreFile  # if you're not in the global namespace anymore
if treeStoreFile is None:
    treeStoreFile = TFile("some/treeStore.root", "RECREATE")
else:
    treeStoreFile.cd()
RooAbsData.setDefaultStorageType(RooAbsData.Tree)  # using .convertToTreeStore() seemed not to function properly

I don’t understand how the ROOT working directory actually works but I feel like I would need to be able to explicitly specify the ROOT file which should be used for each individual RooDataSet instance that I have.

As a follow-up problem I am now seeing errors and memory leaks if I start fitting to a RooTreeDataStore-backed RooDataSet with multiple threads. I think it should be relatively easy to come up with a minimal example for this as well. I’ll do so and open another thread with that when I find the time.

I fixed the problem you observed. The tree-backed storage was attaching itself to whatever file was opened last when you write it. That’s not what you want.

About the memory leaks etc:
We have to see if that’s still an issue when the other problem is out of the way.

Ok, thanks a lot for fixing this.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.