[solved] Memory "leak" with TFile Get

ROOT
1

Hello,
I am having a problem with the memory consumption of one of my python scripts. Basically, running something like this:

f = ROOT.TFile("myfile.root")
for i in range(100):
    sums = f.Get("Sums")

fills up my RAM in no time even though “sums” should be out of scope at the end of the loop. Sums is a TList holding a few TNtuple and histograms. Also, closing the file does not release the memory. How can I avoid this leak?

Thanks a lot,

Christian

2

Is it something you also see with the equivalent C++ script ?

3

Thanks for your reply. Indeed, this does not seem to by python related. The same problem occurs in c++:

#include "TFile.h"
#include "TList.h"

void test() {
  TFile *f = TFile::Open("hehi_pythia.root");
  for (Int_t i = 0; i < 10; i++) {
    TList *sums = (TList*)f->Get("Sums");
    //sums->Delete();
    //delete sums;
  }
  f->Close();
}

calling sums->Delete() or delete Sums, does not make a difference. The memory occupied in the loop is not freed at the end of the execution. Is there a different way in root if I want to read “Sums” many times in my script?

Cheers,
Christian

4

Hi,

I would expect that with both the sums->Delete(); and the delete sums, all the memory would be freed.

[quote]Also, closing the file does not release the memory.[/quote]A TList is not managed by the TFile and once returned by Get, it is the responsibility of the calling code to delete the list.

In addition, TList are not ‘owner’ of their content by default and thus the content is not deleted when the list is deleted. You either need to call sums->Delete() or sums->SetOwner(true) to insure that after ‘delete sums’ all the memory is released.

Cheers,
Philippe.

5

Thanks for the reply. I tried the following scripts, and none of theme released the memory (ie. memory is constantly rising during the execution):

#include "TFile.h"
#include "TList.h"

void test() {
  TFile *f = TFile::Open("./train_out/AnalysisResults.root", "read");
  for (Int_t i = 0; i < 10; i++) {
    TList *sums = (TList*)f->Get("MultEstimators/Sums");
    sums->Delete();
    delete sums;
  }
  f->Close();
}
#include "TFile.h"
#include "TList.h"

void test() {
  for (Int_t i = 0; i < 10; i++) {
    TFile *f = TFile::Open("./train_out/AnalysisResults.root", "read");
    TList *sums = (TList*)f->Get("MultEstimators/Sums");
    sums->Delete();
    delete sums;
    f->Close();
    delete f;
  }
}
#include "TFile.h"
#include "TList.h"

void test() {
  for (Int_t i = 0; i < 10; i++) {
    TFile *f = TFile::Open("./train_out/AnalysisResults.root", "read");
    TList *sums = (TList*)f->Get("MultEstimators/Sums");
    sums->SetOwner(kTRUE);
    sums->Delete();
    delete sums;
    f->Close();
    delete f;
  }
}

Any ideas?

6

Hi,

I am not sure what the problem is, it has to be related to the content of the list.
You could run a memory profiler (valgrind’s massif or igprof or something else) to pin point the issue.
Alternatively you could send us the data file so we can take a look it.

Cheers,
Philippe.

7

Thanks for offering to look into this! I send you an email with a link to a file.

8

Hi,

The list contains lists (of histo and ntuples).root [7] l->ls() OBJ: TList Sums Doubly linked list : 0 OBJ: TList EtaLt05 Doubly linked list : 0 OBJ: TH2F feta_Nch N_{ch} vs. #eta, | #eta| #leq 0.5 : 0 at: 0x7fb513c5dce0 OBJ: TH3F fNch_pT_pid Event class vs. p_{T} vs. pid, | #eta| #leq 0.5 : 0 at: 0x7fb514499e00 OBJ: TNtuple fEventTuple N_{ch}^{est} : 0 at: 0x7fb513c55690 OBJ: TH2F fcorr_thisNch_vs_refNch corr_hist_EtaLt05_vs_EtaLt05 : 0 at: 0x7fb513a2c820 OBJ: TH2F fNch_vs_nMPI fNch_vs_nMPI : 0 at: 0x7fb513a3def0 OBJ: TH2F fNch_vs_Q2 fNch_vs_nMPI : 0 at: 0x7fb513a3e2e0 OBJ: TList EtaLt08 Doubly linked list : 0 OBJ: TH2F feta_Nch N_{ch} vs. #eta, | #eta| #leq 0.8 : 0 at: 0x7fb513a3e6d0 OBJ: TH3F fNch_pT_pid Event class vs. p_{T} vs. pid, | #eta| #leq 0.8 : 0 at: 0x7fb514943400 OBJ: TNtuple fEventTuple N_{ch}^{est} : 0 at: 0x7fb513c74100 OBJ: TH2F fcorr_thisNch_vs_refNch corr_hist_EtaLt05_vs_EtaLt08 : 0 at: 0x7fb513c747c0 OBJ: TH2F fNch_vs_nMPI fNch_vs_nMPI : 0 at: 0x7fb513c74c60 OBJ: TH2F fNch_vs_Q2 fNch_vs_nMPI : 0 at: 0x7fb513c750f0

So the outer call to Delete does delete the list themselves but does not delete the contains of the sub-list.
In order to properly delete this structure (I suspect the producing code contains equivalent code (or at least it should):TIter next(sums); TObject *obj = nullptr; while( obj = next() ) { TList *list = dynamic_cast<TList*>(obj); if (list) list->Delete(); // alternatively list->SetOwner(true); } sums->Delete(); // alternatively list->SetOwner(true); delete sums;

Cheers,
Philippe.

PS. Alternatively the list could be made owner of their content in the writing program.

9

Thanks, this did indeed fix the issue!