Hello,
Version: ROOT 6.08.00.
I’m running a long loop, where each iteration involves opening three ROOT files, processing a tree within each file, drawing a histogram for each of the three files, scaling them, and adding them together.
My issue is that the time to complete each iteration seems to be increasing gradually as the code runs. I’ve also noticed by running a “top” command in the command line, that the memory being used by the process is increasing over time. I try to make sure that every “new” has a corresponding “delete” so that there aren’t any memory leaks. And I try to make sure that I’m properly closing TFiles, resetting TTrees, and deleting histograms once I’m done with them.
I can’t post the exact code, because it’s just one function embedded inside a larger file, and I’d have to provide data files and things that I can’t give away. But here is a simplified version of what I’m doing, and I’d like to know if there’s an explanation of the code slowdown and memory growth:
void process_tree(TTree* in, TTree* out)
{
out->Reset();
// Loop over entries in "in", do some calculations, and write the values to "out".
}
void main()
{
// Allocate files, trees, and histogram.
TFile* f1 = new TFile;
TTree* in1 = new TTree;
TTree* out1 = new TTree;
TFile* f2 = new TFile;
TTree* in2 = new TTree;
TTree* out2 = new TTree;
TFile* f3 = new TFile;
TTree* in3 = new TTree;
TTree* out3 = new TTree;
TH1D* hTemp = new TH1D;
int count = 0;
while(some_condition)
{
gDirectory->Delete("hist*");
f1 = TFile::Open(Form("some_file1_%i.root", count));
in1 = (TTree*)f1->Get("t");
process_tree(in1, out1);
f2 = TFile::Open(Form("some_file2_%i.root", count));
in2 = (TTree*)f2->Get("t");
process_tree(in2, out2);
f3 = TFile::Open(Form("some_file3_%i.root", count));
in3 = (TTree*)f3->Get("t");
process_tree(in3, out3);
out1->Draw(Form("x>>hist1_%i", count), "", "goff");
out2->Draw(Form("x>>hist2_%i", count), "", "goff");
out3->Draw(Form("x>>hist3_%i", count), "", "goff");
gROOT->ProcessLine(Form("hist1_%i->Add(hist2_%i);", count, count));
gROOT->ProcessLine(Form("hist1_%i->Add(hist3_%i);", count, count));
hTemp = (TH1D*)gDirectory->Get(Form("hist1_%i", count));
// Do a calculation.
f1->Close();
f2->Close();
f3->Close();
count++;
}
}
So the main things I’ve tried to do to minimize wasted memory is to reuse TFiles, TTrees, and TH1Ds rather than creating new ones for every iteration. I try to reset my TTrees, and delete the intermediate histograms. I don’t “delete” the few things I “new” in main(), but main() only gets executed once, so I’m not worried about that. I’m mostly worried about what happens inside the “while” loop, which may iterate ~100000 times or so, in the worst case.
Is there anything I’m missing? Why is this code using up more and more memory as time goes on? And could this be related to the slowdown I’m observing?
Thanks.
Edit: Solved. Here is the working code.
void process_tree(TTree* in, TTree* out)
{
out->Reset();
// Loop over entries in "in", do some calculations, and write the values to "out".
}
void main()
{
// Declare files, trees, and histogram.
TFile* f1 = nullptr;
TTree* in1 = nullptr;
TTree* out1 = nullptr;
TFile* f2 = nullptr;
TTree* in2 = nullptr;
TTree* out2 = nullptr;
TFile* f3 = nullptr;
TTree* in3 = nullptr;
TTree* out3 = nullptr;
TH1D* hTemp = nullptr;
int count = 0;
while(some_condition)
{
gDirectory->Delete("hist*");
f1 = new TFile;
in1 = new TTree;
out1 = new TTree;
f2 = new TFile;
in2 = new TTree;
out2 = new TTree;
f3 = new TFile;
in3 = new TTree;
out3 = new TTree;
hTemp = new TH1D;
f1 = TFile::Open(Form("some_file1_%i.root", count));
in1 = (TTree*)f1->Get("t");
process_tree(in1, out1);
f2 = TFile::Open(Form("some_file2_%i.root", count));
in2 = (TTree*)f2->Get("t");
process_tree(in2, out2);
f3 = TFile::Open(Form("some_file3_%i.root", count));
in3 = (TTree*)f3->Get("t");
process_tree(in3, out3);
out1->Draw(Form("x>>hist1_%i", count), "", "goff");
out2->Draw(Form("x>>hist2_%i", count), "", "goff");
out3->Draw(Form("x>>hist3_%i", count), "", "goff");
gROOT->ProcessLine(Form("hist1_%i->Add(hist2_%i);", count, count));
gROOT->ProcessLine(Form("hist1_%i->Add(hist3_%i);", count, count));
hTemp = (TH1D*)gDirectory->Get(Form("hist1_%i", count));
// Do a calculation.
delete hTemp;
delete in1;
delete out1;
delete f1;
delete in2;
delete out2;
delete f2;
delete in3;
delete out3;
delete f3;
count++;
}
}