OpenMP TFile problem

ROOT
1

Hello,

I am using a parallel build of ROOT-MINUIT2 with OpenMP to minimize a function. The chi^2 function to be minimized is constructed from some data read from a ROOT file. So my problem is related with reading a TTree in parallel by various threads. Since the whole code is complicated, I am posting only posting a summary of where the error is occurring.

TFile* inputFile = new TFile("data.root","READ");
// This function is called simultaneously by various threads
double get_Prediction(double* param)
{
    TTree* tt = (TTree*) inputfile->Get("h");
    tt->SetBranchAddress("p_neutrino",&pn);
    // read this tree, carry out some analysis
    double prediction = xxxx;
   return prediction;
}

The above function is invoked by OpenMP threads and crashes with messages about error in reading the file by different threads. The actual error messages are attached here.
It however works fine with a single OpenMP thread which can be set as an environment variable.
export OMP_NUM_THREADS=1

How do I make this code work with multiple threads? I am simply reading data from a single file, a single tree, which does not need to be modified. Any help is appreciated.

Thank You,
Tarak.

Note : I am using ROOT v5.28 compiled with gcc 4.4. A parallel MINUIT2 version with OpenMP is built.
error_messages.txt (13.2 KB)

2

Hi,

The TTree and TFile class are not thread safe and should not be shared (unless you always ‘lock’ their access) between threads. However you can create several TFile object (and thus TTree object) looking at the same physical file.

Cheers,
Philippe.

3

Thank you, Philippe.

I have tried that too, i.e. creating a different TFile object opening the same physical file.

// This function is called simultaneously by various threads invoked by OPENMP
double get_Prediction(double* param)
{
    TFile* inputFile = new TFile("data.root","READ");
    TTree* tt = (TTree*) inputfile->Get("h");
    tt->SetBranchAddress("p_neutrino",&pn);
    // read this tree, carry out some analysis
    double prediction = xxxx;
   return prediction;
}

Now the different threads run for a while and arbitrarily crashes with a segmentation fault message.

Tarak.

4

Hi,

How many ‘entries’ from the TTree do you read in get_Prediction. If you read many, I would ‘lock’ the part creating the TFile and loading the TTree. If you read only one, I would move the TFile and TTree creation and the SetBranchAddress out of the function (and better yet in the main thread) and only do the GetEntry in get_Prediction.

Cheers,
Philippe.

5

Hi Philippe,

The suggestion to lock TFile creation and loading of TTree worked along with declaring some variables as private to the thread. I am reading all the entries in the TTree. Can you explain to me why this locking works? I would think that since each TFile object is local to the thread, all threads would run independently and there would be no interference. Thanks a lot. :slight_smile:

Cheers,
Tarak.

6

Hi Tarak,

TFile still manipulate some global data and not all this data is properly/completely protected for thread safety. Also reading data out of the file also require access to some global data (the StreamerInfo).

Humm, by the way, did you create a TThread object? … if you do not create such an object, ROOT does not know you are using multiple thread and does [b]not{/b] use any locking mechanism internally.

Cheers,
Philippe

7

Hi Philippe,

I am not using TThreads explicitly but instead I use OPENMP. With the “pragma omp critical” statement, it works fine. If I remove it then the code will crash. I do not know if OPENMP makes use of TThreads internally in ROOT. If it doesn’t, then the code is still working without ROOT knowing about different threads. As I read in the OPENMP documentation, the critical pragma keeps all the other threads from executing the following structured block until the current thread finishes its execution. So, I think it’s the same effect as TThread would give rise to.

// This function is called simultaneously by various threads invoked by OPENMP
double get_Prediction(double* param)
{
    TFile* inputFile;
    TTree* tt;
    #pragma omp critical
    {    
        inputFile = new TFile("data.root","READ");
        tt = (TTree*) inputfile->Get("h");
    }
    tt->SetBranchAddress("p_neutrino",&pn);
    // read this tree, carry out some analysis
    double prediction = xxxx;
    return prediction;
}

Thanks for an explanation. Well, with the last remark about not using TThread, I should check my code again to make sure that it’s working okay. 8)

Cheers,
Tarak.

1 Like