Dear Root-onians,
I am having some problems with the TTree. I am doing some kind of adaptive binning in 5 dimensions and I am using TTrees to hold the bin contents. The binning procedure gets one TTree as input, sorts this TTree using TTree::BuildIndex(), splits the TTree into two sub-TTrees, deletes the original TTree and then calls itself recursively. Effectively, I am splitting a TTree with about 10e7 entries into many sub-TTrees with around 15 entries.
Of course, the memory usage should slowly rise with the growing number of TTrees because of the overhead. However, I see that already with 500’000 events (6 branches, all doubles), the memory use grows to 1.6 GB, which seems a little bit excessive. With 10e7 events, I run out of memory.
Do you have any suggestions what I might be doing wrong? I can attach the code of the binning function if that is requested. I also have a massif profile, if you are interested. Thanks in advance for your help!
Best regards,
e_dude
Hi again,
obviously, my question was not specific enough to get an answer. I played around with some TTrees and found something peculiar: I created a lot of TTrees with one entry and checked memory use of root, with this short script:
#include<iostream>
#include<TTree.h>
#include<TSystem.h>
void testRootTrees() {
const unsigned int TREES = 32000;
ProcInfo_t pi;
std::vector<TTree*> trees;
gSystem->GetProcInfo(&pi);
printf("Drawn: Res Mem=%ld Virt Mem=%ld\n", pi.fMemResident, pi.fMemVirtual);
std::cout<<"--------------------------"<<std::endl;
for(unsigned int i = 0; i < TREES; ++i) {
TTree* tree = new TTree();
double bla = 0.313;
tree->Branch("bla", &bla, "bla/D");
tree->Fill();
trees.push_back(tree);
if(not (i % 1000)) {
gSystem->GetProcInfo(&pi);
printf("Drawn: Res Mem=%ld Virt Mem=%ld\n", pi.fMemResident, pi.fMemVirtual);
}
}
for(unsigned int i = 0; i < TREES; ++i) {
delete trees[i];
}
gSystem->GetProcInfo(&pi);
std::cout<<"--------------------------"<<std::endl;
printf("Drawn: Res Mem=%ld Virt Mem=%ld\n", pi.fMemResident, pi.fMemVirtual);
std::cout<<"--------------------------"<<std::endl;
TTree* tree = new TTree();
double bla = 0.313;
tree->Branch("bla", &bla, "bla/D");
for(unsigned int i = 0; i < TREES; ++i) {
bla = (double)i;
tree->Fill();
if(not (i % 1000)) {
gSystem->GetProcInfo(&pi);
printf("Drawn: Res Mem=%ld Virt Mem=%ld\n", pi.fMemResident, pi.fMemVirtual);
}
}
std::cout<<tree->GetEntries()<<std::endl;
}
This produces the following output:
root testRootTrees.C+
root [0]
Processing testRootTrees.C+...
Drawn: Res Mem=18556 Virt Mem=79052
--------------------------
Drawn: Res Mem=18588 Virt Mem=79220
Drawn: Res Mem=29612 Virt Mem=144448
Drawn: Res Mem=40440 Virt Mem=209868
Drawn: Res Mem=51032 Virt Mem=275068
Drawn: Res Mem=61852 Virt Mem=340348
Drawn: Res Mem=72936 Virt Mem=405696
Drawn: Res Mem=83500 Virt Mem=470976
Drawn: Res Mem=94584 Virt Mem=536256
Drawn: Res Mem=105144 Virt Mem=601508
Drawn: Res Mem=116220 Virt Mem=666856
Drawn: Res Mem=127072 Virt Mem=732136
Drawn: Res Mem=137632 Virt Mem=797388
Drawn: Res Mem=148448 Virt Mem=862668
Drawn: Res Mem=159532 Virt Mem=927948
Drawn: Res Mem=170092 Virt Mem=993360
Drawn: Res Mem=180912 Virt Mem=1058640
Drawn: Res Mem=191728 Virt Mem=1123920
Drawn: Res Mem=202744 Virt Mem=1189300
Drawn: Res Mem=213372 Virt Mem=1254580
Drawn: Res Mem=224192 Virt Mem=1319832
Drawn: Res Mem=235012 Virt Mem=1385112
Drawn: Res Mem=245832 Virt Mem=1450392
Drawn: Res Mem=256648 Virt Mem=1515672
Drawn: Res Mem=267732 Virt Mem=1580952
Drawn: Res Mem=278296 Virt Mem=1646204
Drawn: Res Mem=289116 Virt Mem=1711484
Drawn: Res Mem=300196 Virt Mem=1776764
Drawn: Res Mem=310756 Virt Mem=1842016
Drawn: Res Mem=321572 Virt Mem=1907296
Drawn: Res Mem=332656 Virt Mem=1972576
Drawn: Res Mem=343216 Virt Mem=2037988
Drawn: Res Mem=354036 Virt Mem=2103268
--------------------------
Drawn: Res Mem=19076 Virt Mem=79548
--------------------------
Drawn: Res Mem=19076 Virt Mem=79548
Drawn: Res Mem=19076 Virt Mem=79548
Drawn: Res Mem=19084 Virt Mem=79548
Drawn: Res Mem=19092 Virt Mem=79548
Drawn: Res Mem=19100 Virt Mem=79548
Drawn: Res Mem=19116 Virt Mem=79548
Drawn: Res Mem=19124 Virt Mem=79548
Drawn: Res Mem=19132 Virt Mem=79548
Drawn: Res Mem=19140 Virt Mem=79548
Drawn: Res Mem=19144 Virt Mem=79548
Drawn: Res Mem=19148 Virt Mem=79548
Drawn: Res Mem=19152 Virt Mem=79548
Drawn: Res Mem=19160 Virt Mem=79548
Drawn: Res Mem=19172 Virt Mem=79548
Drawn: Res Mem=19176 Virt Mem=79548
Drawn: Res Mem=19184 Virt Mem=79548
Drawn: Res Mem=19192 Virt Mem=79704
Drawn: Res Mem=19200 Virt Mem=79704
Drawn: Res Mem=19208 Virt Mem=79704
Drawn: Res Mem=19216 Virt Mem=79704
Drawn: Res Mem=19224 Virt Mem=79704
Drawn: Res Mem=19232 Virt Mem=79704
Drawn: Res Mem=19240 Virt Mem=79704
Drawn: Res Mem=19248 Virt Mem=79704
Drawn: Res Mem=19256 Virt Mem=79704
Drawn: Res Mem=19264 Virt Mem=79704
Drawn: Res Mem=19272 Virt Mem=79704
Drawn: Res Mem=19280 Virt Mem=79704
Drawn: Res Mem=19288 Virt Mem=79704
Drawn: Res Mem=19296 Virt Mem=79704
Drawn: Res Mem=19304 Virt Mem=79704
Drawn: Res Mem=19312 Virt Mem=79704
32000
root [1]
It seems to me that a TTree with one Branch of double and one entry uses 11 KB of memory! Is that normal? Is there any way to reduce this footprint?
Thanks in advance!
Regards,
e_dude