Too many open files

Dear Rooters,

I am writing a macro where I wish to read histograms from 1000 files and add them together. I open each file at the time in a loop using:

TFile *myFile = new TFile();
myFile->Open(fileName,“read”);

From the file I copy two histograms (I don’t think this is relevant but) using the ROOT feature that whenever a file is opened you automatically enter this file:

h_Ptmp = new TH1F(*h_primaryElectronFluenceSpectrum);
h_Stmp = new TH1F(*h_scatterElectronFluenceSpectrum);

Finally I close the file and delete the myFile object (at the end of the loop)

myFile->Close();
delete myFile;

However, after some 938 files ROOT crashes telling me that there are too many open files. How can this be? I only open one file at the time and I also close it afterwards? I did some manual testing with cint and also there experienced that even if a TFile is close and the pointer deleted the command “.ls” will still print what is inside the file.

Help would be most appreciated, my Ph.D depends on this merge :wink:

Cheers,
Karin

Hi,

you should simply call TFile::Open(…), not new TFile(). And even better yet would be to use hadd from $ROOTSYS/bin. You call it as “hadd out.root in1.root in2.root …”

Cheers, Axel.

Hello!

I have also tried this, but unfortunately it doesn’t work either. I have slimmed my macro down to a minimum (see below). This crashed after 994 files ( I want 1000). I do not wish to use hadd since I am manipulating the histos.

/Karin

PS
I use ROOT 5.18 on Ubuntu 9.10

void runSpectrumBAM(){
  int iM,iR;
  int nR = 500;
  char material[30];
  char folder[1000];
  char suffix[400];
  char fluence[1000];

  TH1F *h_Ptmp;
  TH1F *h_Stmp;

  TFile fluenceFile;
  
  gROOT->Reset();
  
  sprintf(folder,"/media/FREECOM\ HDD/DATA/Emax8MeV/WithoutPSF");
  
  //Loop over material
  for(iM=0; iM<2; iM++){
    //Loop over runs
    for(iR=0 ; iR<nR ; iR++){
      if(iM==0){
	sprintf(material,"Water");
	sprintf(suffix,"%u%03d",1,iR);
      }
      else{
	sprintf(material,"EFD_10cm");
	sprintf(suffix,"%u%03d",2,iR);
      }
      //print file-name
      sprintf(fluence,"%s/plots/spectra/histos/histos_Spectrum_%s_xCoord_0cm_yCoord_0cm_depth_10.05cm_%s.root",
	      folder,material,suffix);

      fluenceFile.Open(fluence,"read");
      cout<<"*   Loading fluence file: "<<fluence<<endl;

      //Copy histograms of interest depending on input parameter (either electron or photon)
      h_Ptmp = new TH1F(*h_primaryElectronFluenceSpectrum);
      h_Stmp = new TH1F(*h_scatterElectronFluenceSpectrum);
      
      fluenceFile.Close();
      
      
    }//iR	        
  }//iM
}//end

Hi Karin,

never ever call gROOT->Reset() inside a named macro!

TFile::Open() is a static function; you should not call it on an object (…->Open()) but simply call TFile::Open() and use the returned TFile*, which you delete in the end, e.g.

TFile* myFile = TFile::Open("myFile.root"); ... delete myFile;
No need to call TFile::Close() etc. So I’d use:

void runSpectrumBAM(){
  int iM,iR;
  int nR = 500;
  char material[30];
  char folder[1000];
  char suffix[400];
  char fluence[1000];

  TH1F *h_Ptmp;
  TH1F *h_Stmp;

  
  // NO! gROOT->Reset();
  
  sprintf(folder,"/media/FREECOM\ HDD/DATA/Emax8MeV/WithoutPSF");
  
  //Loop over material
  for(iM=0; iM<2; iM++){
    //Loop over runs
    for(iR=0 ; iR<nR ; iR++){
      if(iM==0){
	sprintf(material,"Water");
	sprintf(suffix,"%u%03d",1,iR);
      }
      else{
	sprintf(material,"EFD_10cm");
	sprintf(suffix,"%u%03d",2,iR);
      }
      //print file-name
      sprintf(fluence,"%s/plots/spectra/histos/histos_Spectrum_%s_xCoord_0cm_yCoord_0cm_depth_10.05cm_%s.root",
	      folder,material,suffix);

        TFile* fluenceFile = TFile::Open(fluence,"read");
      cout<<"*   Loading fluence file: "<<fluence<<endl;

      //Copy histograms of interest depending on input parameter (either electron or photon)
      h_Ptmp = new TH1F(*h_primaryElectronFluenceSpectrum);
      h_Stmp = new TH1F(*h_scatterElectronFluenceSpectrum);
      
      delete fluenceFile;
      
      
    }//iR	        
  }//iM
}//end

Let me know how that works for you!

Cheers, Axel.

Hello Axel!I

Thanks for your help, but unfortunately it still crashes. I used the code you gave me and it crashed at the second file with the message cited below. Removing the delete-statement makes the program run exactly the same way it did before, crashing after 994 files.

Other hints are most welcomed! :slight_smile:

cheers,
Karin

Hi,

Which version of ROOT are you using?
Did you try to run with valgrind and/or gdb to pinpoint the origin of the segmentation?

Cheers,
Philippe.

I do not understand what you are trying to do in your loops.
In the code below I have added a few statements that may help you.

Rene

[code]void runSpectrumBAM(){
int iM,iR;
int nR = 500;
char material[30];
char folder[1000];
char suffix[400];
char fluence[1000];
TH1::AddDirectory(kFALSE); //<============

TH1F *h_Ptmp;
TH1F *h_Stmp;

// NO! gROOT->Reset();

sprintf(folder,"/media/FREECOM\ HDD/DATA/Emax8MeV/WithoutPSF");

//Loop over material
for(iM=0; iM<2; iM++){
//Loop over runs
for(iR=0 ; iR<nR ; iR++){
if(iM==0){
sprintf(material,“Water”);
sprintf(suffix,"%u%03d",1,iR);
}
else{
sprintf(material,“EFD_10cm”);
sprintf(suffix,"%u%03d",2,iR);
}
//print file-name
sprintf(fluence,"%s/plots/spectra/histos/histos_Spectrum_%s_xCoord_0cm_yCoord_0cm_depth_10.05cm_%s.root",
folder,material,suffix);

    TFile* fluenceFile = TFile::Open(fluence,"read"); 
  cout<<"*   Loading fluence file: "<<fluence<<endl; 

  //Copy histograms of interest depending on input parameter (either electron or photon) 
  // h_Ptmp = new TH1F(*h_primaryElectronFluenceSpectrum); 
  //h_Stmp = new TH1F(*h_scatterElectronFluenceSpectrum); 
   h_Ptmp = (TH1F*)fluenceFile->Get("h_primaryElectronFluenceSpectrum"); 
  h_Stmp = (TH1F*)fluenceFile->Get("h_scatterElectronFluenceSpectrum"); 
  
  delete fluenceFile; 
  
  
}//iR           

}//iM
}//end [/code]

Thank you Axel and Rene!

The TFile::Open together withTH1::AddDirectory(kFALSE) fixed it and the whole loop now runs without problem.

/Karin