How to effectively manage memory with ROOT

ROOT
1

Hi experts,

my macro will be generating and saving a large number of TObject, for example TGraphErrors
so now i have two ways of writing up my macro.

#1 : make a vector that holds those stuff, and only Write() them before the macro end
#2 : generate and Write those TGraphErrors() on the fly (in a loop)

does this make any difference in the memory usage ?
and how ROOT manages this ?

ROOT Version: 6.24
Platform: CentOS 7 (conda)
Compiler: Not Provided

2

In term of reducing peak memory use:

#2 : generate and Write those TGraphErrors() on the fly (in a loop) and then delete (and/or reuse for the next iteration of the loop
3

follow up questions:

  1. When I Write(), do the TGraphs or other TObjects goes from memory to file ?
  2. after writting the object, will the object in the file being modified if i reused the object for other purpose (like clearing the object and fill with new data) ?

below is the minimal version of my code

void mem_test(){

    TFile* file = new TFile("memtest.root","RECREATE");
    file->mkdir("graphs");
    file->cd("graphs");
    vector<TGraphErrors*> graphs;
    TGraphErrors* gr;
    int N = 10000000;

    for (int evt=0; evt<N; evt++){
        gr = new TGraphErrors();
        gr->SetName(Form("gr_%d",evt));
        for(int j=0; j<10000; j++){
            gr->SetPoint(gr->GetN(),j,j*0.001);
        };
        // graphs.push_back(gr); // method 1 : push back and write before the program ends
        gr->Write(); // method 2 : write on the fly
    };

    // for(auto gr:graphs){ // write before the program ends
    //     gr->Write();
    // };
    file->Close();
}

regardless of method 1 or 2, I saw memory climbing up (above 2G) all the time… how to stop the memory from climbing up ?

4

Yes.

after writting the object, will the object in the file being modified if i reused the object for other purpose (like clearing the object and fill with new data) ?

No. After the Write there is no connection between the object and the representation on file as long as you also change the name. If you don’t change the name, the key name will be the same and the 2nd write will hide the first (and possibly delete/remove the old one).

5

You either need to move:

     gr = new TGraphErrors();

out of the loop or add

    delete gr;

at the end of the loop (right after the Write)

6

what does “move the graph out of the loop” has to do with reducing memory ? i not quite get it …

7

The existing code:

    for (int evt=0; evt<N; evt++){
        gr = new TGraphErrors();
        ....
    };

Creating/Allocating and keeping in memory N graphs.

The two alternative I proposed are:

    for (int evt=0; evt<N; evt++){
        gr = new TGraphErrors();
        ....
        delete gr.
    };

Creating/Allocating N Graphs but deleting them immediately (thus deleting it N times). There is only one graph in memory.

    gr = new TGraphErrors();
    for (int evt=0; evt<N; evt++){
        .....
    };
    delete gr

The graph is created and deleted exactly once. There is only one Graph in memory.

8

follow up
1.) i tested the code with Write() then delete. it works and the memory is way below 1G.
but i have other non-ROOT object, and i suppose the same thing (delete when no longer in use) applies to them as well.

2.) also, i have a TTree that is required to live thourghout the main program’s lifetime. but in this case i can only Write() but not delete before the program ends. (maybe using Autosave(Nevents) helps ?)

am i right ?

9

yes.

2.) also, i have a TTree that is required to live thourghout the main program’s lifetime. but in this case i can only Write() but not delete before the program ends. (maybe using Autosave(Nevents) helps ?)

Yes the TFile will delete the TTree.

10

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.