OpenMP with TFile and TTree?

Hello,

I was hoping to apply my new found knowledge of OpenMP to some bottleneck code which must read many TTree objects from many TFile files. I am trying to compile with g++, and to get it going, I have a very simple setup, merely defining 4 threads and then calling

'#pragma omp parallel ’

within a block of code to parallelize it. I then enter a function and attempt to open a TFile rootfile, then read the TTrees inside it. [url=https://root-forum.cern.ch/t/openmp-tfile-problem/11799/7 this post[/url], I protect the attempts to open/close the TFile object by and to read from the TTree object by the directive

#pragma omp critical’

I have checked that my parallelization works perfectly when I remove all the calls to TFile and TTree (in other words each thread does get a different file and there are no race conditions on dividing up the tasks to different threads), but when I try to actually do the tasks with ROOT, I get many many errors.

Critical info: I am using the latest version of ROOT as of Feb 27, 2012, Version 5.34, and my compiler is gcc 4.1.2 on Scientific Linux.

I have searched both online and within these forums to try to get an answer on this or to get some tutorial on OpenMP with ROOT and I wasn’t able to find an answer to this problem. So my question is essentially: Does ROOT allow parallelization with OpenMP in the way that I am trying to implement it? Specifically, is OpenMP compatible with the use of TTree and TFile objects? I’d be happy to post the (very lengthy) stack trace or error messages or any other information needed.

The relevant parts of my code are listed below.

// All other include files
#include <omp.h>

using namespace std;

void ProcessFile(string& filename);

int main(void)
{
 
  vector<string> rootfiles_vec;
  // Load rootfiles rootfiles_vec; 

  omp_set_num_threads(4);  
  int num_rootfiles = rootfiles_vec.size();

  int fileinitial = 0;
  int* ifile = &fileinitial;
#pragma omp parallel
  {    
    while (true) {
      string rootfile;
      int filenum;
#pragma omp critical
      {	
	if( (*ifile) < num_rootfiles) {
	  rootfile = rootfiles_vec[*ifile];
	  filenum = *ifile;
	  (*ifile)++;
	}
      }
      
      // THIS IS WHERE THE ROOT STUFF HAPPENS
      ProcessFile(rootfile);
      
      if ( (*ifile) >= num_rootfiles) break;
    }
  
  }  // end #omp parallel
    
  return 0;

}


void ProcessFile(string& filename)
{
  
  TFile* rootfile;
#pragma omp critical
  {
    rootfile = new TFile(filename.c_str(), "READ");
    cout<<"Processing file: "<<filename<<endl;
  }
  
  TIter next(rootfile->GetListOfKeys());
  TKey* key;
  while( (key=(TKey*)next()) ) {
    string keyName = key->GetName();
    if( keyName.find("electron") != std::string::npos ) {

      TTree* electronTree;
#pragma omp critical
      {
	cout<<"  Processing electron: "<<keyName<<endl;
	electronTree = (TTree*)gDirectory->Get(keyName.c_str());
      }
      
      Double_t px,py,pz,rz;
      electronTree->SetBranchAddress("px",&px);
      electronTree->SetBranchAddress("py",&py);
      electronTree->SetBranchAddress("pz",&pz);
      electronTree->SetBranchAddress("rz",&rz);
      
      Int_t nentries = (Int_t) electronTree->GetEntries();
      Int_t ientry = 0;
      for(ientry = 0; ientry<nentries; ientry++) {
	electronTree->GetEntry(ientry);
      }
#pragma omp critical
      delete electronTree;
    }

  }

#pragma omp critical
  {
    rootfile->Close();
    delete rootfile;
  }
  
}

Note that I have tried it both with and without the ‘#pragma omp critical’ when I open and close the TFile object as well as when I access and delete the TTree object.

Thank you very much for your time.