Memory limits in TH3

ROOT
1

I run into segmentation faults when trying to write a histogram of sufficient size. I can manipulate the histogram prior to writing it and it appears to behave normally. However, I cannot write it to a TFile without producing a segmentation fault, though I can if I specify fewer bins. (I have quite a few bins.)

It seems clear I’m butting up against some sort of memory limit, and I’ll find a workaround, but I wondered if you could provide any insight.

Python 2.7.13 (default, Mar  8 2017, 15:01:16) 
Type "copyright", "credits" or "license" for more information.

IPython 5.0.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: from ROOT import TH3D, TFile

In [2]: f_out = TFile('./efficiencies.root', 'RECREATE')

In [4]: test_hist = TH3D('test', 'test', 5400, 4000, 31000, 9000, 0, 45000, 20, 1, 6)  # same result with TH3F; writes successfully with, e.g., 9000 replaced by 45

In [5]: test_hist.Fill(4500, 10, 3)
Out[5]: 437675543

In [6]: test_hist.GetEntries()
Out[6]: 1.0

In [7]: test_hist.Fill(4501, 11, 7)
Out[7]: -1

In [8]: test_hist.GetEntries()
Out[8]: 2.0

In [9]: test_hist.Write()
Segmentation fault

ROOT Version: 6.08/06
Platform: x86_64-slc6-gcc62-opt
Compiler: Not Provided

2

Thats because you are creating it on the stack which has a limited size.
Create it as a pointer on the heap instead:

TH3D* test_hist = new TH3D('test', 'test', 5400, 4000, 31000, 9000, 0, 45000, 20, 1, 6);

Now of course you have to access it with the dereferencing operator -> instead of .

You can find further information on the stack and the heap here: https://www.learncpp.com/cpp-tutorial/79-the-stack-and-the-heap/

3

Maybe: 1GB buffer limit

BTW. @Triple_S If it was exceeding the stack size limit, you would get a crash directly in the line in which the histogram was being created.

4

Is there some way of accessing the heap and stack through pyROOT of which I am unaware or are you telling me I have to use C++?

5

Thanks, the THnSparse looks like it will work, but I don’t know what kind of constructor it wants in pyROOT to substitute for const int*, etc.:

Traceback (most recent call last):

File "calculate_efficiencies.py", line 16, in <module>

fnd_hist = THnSparseD('found_values', 'Found X (m, pT, y)', 3, [5400, 9000, 20], [4000, 0, 1], [31000, 45000, 6])

TypeError: none of the 4 overloaded methods succeeded. Full details:

THnSparseT<TArrayD>::THnSparseT<TArrayD>() =>

takes at most 0 arguments (6 given)

THnSparseT<TArrayD>::THnSparseT<TArrayD>(THnSparseT<TArrayD>&&) =>

takes at most 1 arguments (6 given)

THnSparseT<TArrayD>::THnSparseT<TArrayD>(const THnSparseT<TArrayD>&) =>

takes at most 1 arguments (6 given)

THnSparseT<TArrayD>::THnSparseT<TArrayD>(const char* name, const char* title, int dim, const int* nbins, const double* xmin = 0, const double* xmax = 0, int chunksize = 1024 * 16) =>

could not convert argument 4
6

Well, there are maybe easier ways …

from array import array
from ROOT import THnSparseD, TFile
nbins = array('i', [5400, 9000, 20])
xmin = array('d', [4000., 0., 1.])
xmax = array('d', [31000., 45000., 6.])
test_hist = THnSparseD('test', 'Test;XName;YName;ZName;TName', 3, nbins, xmin, xmax)
x = array('d', [4500., 10., 3.])
test_hist.Fill(x)
test_hist.GetEntries()
x = array('d', [4501., 11., 7.])
test_hist.Fill(x)
test_hist.GetEntries()
test_hist.Print('ALL')
f_out = TFile('./efficiencies.root', 'RECREATE')
test_hist.Write()
7

Ah yes. I should have thought of the array package as I use it to fill branches. Thanks much.

For posterity, only from array import array is necessary, not from array import *.

Thanks again.

8

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.