Memory Increasing while writing to a file

rahul.verma · January 4, 2024, 6:29am

Dear Experts
I want to create a large size root file storing data in form of trees. I am creating trees in event loop
while running the code in root environment. While running the code memory consumption is increasing.
I created a simpler version of code which is attached and has the similar problem. If i comment out the “tr_Final->Write()” command the memory consumption stops. If I dont comment it out its memory usage are increasing continuously by about 1MB per second. I am running parallel macros with different seeds to generate data faster. After some time memory consumption becomes so high that my system crashes.
_ROOT Version: 6.26.10
_Platform: Ubuntu 22.04
I want to know why writing tree into file using “tr_Final->Write()” is increasing the memory and how to solve the memory consumption issue.
Thanks
Below is the root code that i wrote.
//

        void rootTree(){
        TFile *outFile1 = new TFile("output1Final.root","recreate");
        TDirectory *TreeDirFinal = outFile1->mkdir("TreeFinal");

        //Begin===Variables to Store
            int ArraySize=5000;
            short int nTrack ;      //2 Bytes

            int    id[ArraySize];   //4 Bytes
            float  pT[ArraySize];   //4 Bytes
            float eta[ArraySize];
            float phi[ArraySize];
            float   y[ArraySize];

            short int  index[ArraySize];
            short int status[ArraySize];

            short int           charge[ArraySize];
            short int      strangeness[ArraySize];
            short int baryonNumberType[ArraySize];//it is 3 for a baryon. -3 for anti baryon 

            float px[ArraySize];
            float py[ArraySize];
            float pz[ArraySize];
            float  e[ArraySize];
            float  m[ArraySize];

            bool isCharged[ArraySize];
            bool isStrange[ArraySize];
            bool   isMeson[ArraySize];
            bool  isBaryon[ArraySize];

            short int   mother1[ArraySize];
            short int   mother2[ArraySize];
            short int daughter1[ArraySize];
            short int daughter2[ArraySize];

            bool           isHadron[ArraySize];
            bool           isPrompt[ArraySize];
            bool        isResonance[ArraySize];
            bool          isVisible[ArraySize];
            bool            isFinal[ArraySize];
            bool isFinalPartonLevel[ArraySize];
        //
        
        int iEvent = 0 ;

        TRandom3 *t1 = new TRandom3(123456789);

        for ( iEvent = 0 ; iEvent < 100000000; iEvent++){

            if( iEvent % 100000 == 0) {
                cout<<iEvent<<endl;        
            }

            outFile1->cd();  TreeDirFinal->cd();
                TTree *tr_Final = new TTree(Form("Event%d_Final",iEvent),Form("Event%d_Final",iEvent));
                tr_Final->Branch("nTrack",&nTrack,     "nTrack/S");
                tr_Final->Branch(    "id",     id, "id[nTrack]/I");
                tr_Final->Branch(    "pT",     pT, "pT[nTrack]/F");
                tr_Final->Branch(   "eta",    eta,"eta[nTrack]/F");
                tr_Final->Branch(   "phi",    phi,"phi[nTrack]/F");
                tr_Final->Branch(     "y",      y,  "y[nTrack]/F");
                            
                tr_Final->Branch("index",index,"index[nTrack]/S");

                tr_Final->Branch(          "charge",          charge,          "charge[nTrack]/S");
                tr_Final->Branch(     "strangeness",     strangeness,     "strangeness[nTrack]/S");
                tr_Final->Branch("baryonNumberType",baryonNumberType,"baryonNumberType[nTrack]/S");//it is 3 for a baryon. -3 for anti baryon 

                tr_Final->Branch("px",px,"px[nTrack]/F");
                tr_Final->Branch("py",py,"py[nTrack]/F");
                tr_Final->Branch("pz",pz,"pz[nTrack]/F");
                tr_Final->Branch( "e", e, "e[nTrack]/F");

            nTrack = 100 + t1->Poisson(100) ;      //2 Bytes
            // cout<<nTrack<<endl;
            for(int i = 0 ; i < nTrack ; i++ ){
                id[i]  = 22;   //4 Bytes
                pT[i]  =  5;   //4 Bytes
                eta[i] =  5;
                phi[i] =  2;
                y[i]   =  5;

                index[i]  = 6;
                status[i] = 85;

                        charge[i]   = 2;
                    strangeness[i]  = 20;
                baryonNumberType[i] = -3;//it is 3 for a baryon. -3 for anti baryon 

                px[i] = 3;
                py[i] = 4;
                pz[i] = 5;
                e[i]  = 6;
                m[i]  = 7;
            }
            tr_Final->Fill();
            tr_Final->Write();
            delete tr_Final;
        }
        outFile1->Close();
        delete outFile1;
    }

//

Danilo · January 4, 2024, 7:00am

Dear Rahul,

First of all, welcome to the ROOT community!

I am not sure I understand what you are trying to achieve. Currently, your code writes one TTree per event, i.e. one columnar dataset with one entry per event, beating the very purpose of TTree, which is made to contain many entries.
Please correct me if I am missing something.

In any case, I slightly modified your code to write one individual TTree in your file.

void a(){
        TFile *outFile1 = new TFile("output1Final.root","recreate");
        TDirectory *TreeDirFinal = outFile1->mkdir("TreeFinal");

        //Begin===Variables to Store
            int ArraySize=5000;
            short int nTrack ;      //2 Bytes

            int    id[ArraySize];   //4 Bytes
            float  pT[ArraySize];   //4 Bytes
            float eta[ArraySize];
            float phi[ArraySize];
            float   y[ArraySize];

            short int  index[ArraySize];
            short int status[ArraySize];

            short int           charge[ArraySize];
            short int      strangeness[ArraySize];
            short int baryonNumberType[ArraySize];//it is 3 for a baryon. -3 for anti baryon 

            float px[ArraySize];
            float py[ArraySize];
            float pz[ArraySize];
            float  e[ArraySize];
            float  m[ArraySize];

            bool isCharged[ArraySize];
            bool isStrange[ArraySize];
            bool   isMeson[ArraySize];
            bool  isBaryon[ArraySize];

            short int   mother1[ArraySize];
            short int   mother2[ArraySize];
            short int daughter1[ArraySize];
            short int daughter2[ArraySize];

            bool           isHadron[ArraySize];
            bool           isPrompt[ArraySize];
            bool        isResonance[ArraySize];
            bool          isVisible[ArraySize];
            bool            isFinal[ArraySize];
            bool isFinalPartonLevel[ArraySize];
        //
        
        int iEvent = 0 ;

        TRandom3 *t1 = new TRandom3(123456789);

        outFile1->cd();  TreeDirFinal->cd();
        TTree *tr_Final = new TTree(Form("Event%d_Final",iEvent),Form("Event%d_Final",iEvent));

        for ( iEvent = 0 ; iEvent < 512; iEvent++){

            if( iEvent % 10 == 0) {
                cout<<iEvent<<endl;        
            }
                tr_Final->Branch("nTrack",&nTrack,     "nTrack/S");
                tr_Final->Branch(    "id",     id, "id[nTrack]/I");
                tr_Final->Branch(    "pT",     pT, "pT[nTrack]/F");
                tr_Final->Branch(   "eta",    eta,"eta[nTrack]/F");
                tr_Final->Branch(   "phi",    phi,"phi[nTrack]/F");
                tr_Final->Branch(     "y",      y,  "y[nTrack]/F");
                            
                tr_Final->Branch("index",index,"index[nTrack]/S");

                tr_Final->Branch(          "charge",          charge,          "charge[nTrack]/S");
                tr_Final->Branch(     "strangeness",     strangeness,     "strangeness[nTrack]/S");
                tr_Final->Branch("baryonNumberType",baryonNumberType,"baryonNumberType[nTrack]/S");//it is 3 for a baryon. -3 for anti baryon 

                tr_Final->Branch("px",px,"px[nTrack]/F");
                tr_Final->Branch("py",py,"py[nTrack]/F");
                tr_Final->Branch("pz",pz,"pz[nTrack]/F");
                tr_Final->Branch( "e", e, "e[nTrack]/F");

            nTrack = 100 + t1->Poisson(100) ;      //2 Bytes
            // cout<<nTrack<<endl;
            for(int i = 0 ; i < nTrack ; i++ ){
                id[i]  = 22;   //4 Bytes
                pT[i]  =  5;   //4 Bytes
                eta[i] =  5;
                phi[i] =  2;
                y[i]   =  5;

                index[i]  = 6;
                status[i] = 85;

                        charge[i]   = 2;
                    strangeness[i]  = 20;
                baryonNumberType[i] = -3;//it is 3 for a baryon. -3 for anti baryon 

                px[i] = 3;
                py[i] = 4;
                pz[i] = 5;
                e[i]  = 6;
                m[i]  = 7;
            }
            tr_Final->Fill();
        }
        tr_Final->Write();
        outFile1->Close();
        delete outFile1;
    }

Cheers,
D

rahul.verma · January 4, 2024, 8:19am

Dear Danilo,
I want to write one tree per event, so that after each event tree the memory is freed by deleting the tree. I want to create multiple jobs with different random seeds in pythia event generator. If i define one individual tree in the beginning then it will stay in memory and i wont be able to run multiple jobs in parallel due to memory limitation.
Problem i am facing is, when i dont have “tree->Write()” command and i check it with htop, Memory consumed by job remains the same. But when i use the “tree->Write()” command the “VIRT” memory and “SHR” memory are increasing. Code works fine but after 2-3 hours the system crashes due to high memory consumption and each job( i am running 100 jobs in parallel) creating data for million event is stopped before it finishes.
I want to know why the VIRT and SHR memory are increasing and how can i free it after each 1000 events so that my system doesnt crashes.

Danilo · January 4, 2024, 9:34am

Dear Rahul,

If you are sure you want to write events in a non columnar fashion, which is extremely inefficient, TTree is not the right data structure for you.
In this case, one would need to write individual “Event objects” in the file.

I hope this help.

Cheers,
D

rahul.verma · January 4, 2024, 10:43am

Thanks Danilo

Earlier i was using a single TNtuple for one event, now i switched to trees to reduce memory. And can you ( or anyone else ) answer why “tree->Write()” command increases the memory used ? How is VIRT and RES memory increasing because of this command “tree->Write()” .

pcanal · January 4, 2024, 6:29pm

What is your actual memory budget?

Note creating one TTree (or one TNTuple) per entry is what is causing your (current) memory usage crisis. The design of TTree is to store a very large number of events in the same TTree is to amortize the memory needed over the entries. The TTree will first grow in memory and then stay stable (the same memory will used and reused for all sub-sequent entry). The amount of memory used by a given TTree is customizable to fit most memory budget.

system · January 18, 2024, 6:30pm

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.