Hello ROOTers,
I’m trying to parallelize a section of my code using OpenMP. Even though it works fine on a single thread, it keeps crushing when multi-threading. The error is usually double free or corruption (!prev). I managed to trace the error back to the FillRandom() function.
Here is the section causing troubles:
// Get the PDF to generate the data
TH2D *pdf_p = pdf.getPDF(effRun2, ch, rs, aStep, pStep, a, p, "pdf_p");
TH2D *pdf_n = pdf.getPDF(effRun2, ch, rs, aStep, pStep,-a,-p, "pdf_n");
// Generate the distribution
TRandom3 *rand = new TRandom3();
omp_set_num_threads(numThreads);
#pragma omp parallel for
for(Int_t k=0; k<kMax; k++){
// Generate Dalitz plots based on the PDFs (copy the axis limits from the PDFs)
// The number of events in the Dalitz plots is extracted from a Binomial distribution
TH2D *randHisto_p = ToolsROOT::histCopyAxis(pdf_p, "randHisto_p_"+std::to_string(omp_get_thread_num()));
TH2D *randHisto_n = ToolsROOT::histCopyAxis(pdf_n, "randHisto_n_"+std::to_string(omp_get_thread_num()));
Int_t Np = rand->Binomial(nEvents,(1+gAsym)/2.0);
Int_t Nn = nEvents-Np;
randHisto_p->FillRandom(pdf_p, Np);
randHisto_n->FillRandom(pdf_n, Nn);
// Evaluate the vector {t_a, t_p}
//// First get the coefficient matrix A
std::vector<TH2D *> gFuncVec = {gFunc_a, gFunc_p};
TMatrixD *Ainv_Mx = ToolsStat::A_Mx_calculator(pdf_f, gFuncVec);
//// Then calculate t
std::vector<Double_t> tVec = ToolsStat::t_calculator(randHisto_p, randHisto_n, pdf_f, gFuncVec, nEvents, Ainv_Mx);
// Save the t value in the TTree branch
// Use omp critical to avoid race condition when filling the TTree
#pragma omp critical
{
t_a = tVec[0];
t_p = tVec[1];
tDistr->Fill();
}
// Close
Ainv_Mx->Delete();
randHisto_p->Delete();
randHisto_n->Delete();
}
// Wait for all threads to reach this point before proceeding
#pragma omp barrier
rand->Delete();
pdf_p->Delete();
pdf_n->Delete();
}
The error seems to be caused by the fact that the same pointer to pdf_p (or pdf_n) is passed to FillRandom() on all the threads. In fact, the error disappears when a different clone of pdf_p (or pdf_n) is passed to each thread. This can be achieved by doing something like
std::vector<TH2D*> pdfVec_p;
std::vector<TH2D*> pdfVec_n;
for(Int_t i=0; i<numThreads; i++){
std::string name_p = std::string(pdf_p->GetName())+"_"+std::to_string(numThreads);
TH2D *pdf_p_clone = (TH2D*)pdf_p->Clone();
pdf_p_clone->SetDirectory(0);
pdf_p_clone->SetName((name_p).c_str());
pdfVec_p.push_back(pdf_p_clone);
std::string name_n = std::string(pdf_n->GetName())+"_"+std::to_string(numThreads);
TH2D *pdf_n_clone = (TH2D*)pdf_n->Clone();
pdf_n_clone->SetDirectory(0);
pdf_n_clone->SetName((name_n).c_str());
pdfVec_n.push_back(pdf_n_clone);
}
// And then
TRandom3 *rand = new TRandom3();
omp_set_num_threads(numThreads);
#pragma omp parallel for
for(Int_t k=0; k<kMax; k++){
// ...
randHisto_p->FillRandom(pdfVec_p[omp_get_thread_num()], Np);
randHisto_n->FillRandom(pdfVec_n[omp_get_thread_num()], Nn);
// ...
}
I could use this workaround and be done with it. However, I would like to understand what I’m missing here since it may help me avoid some future pitfalls.
The thing that most confuses me is that I’ve successfully passed the same pointer to different threads other times, and I’ve never experienced this problem. As a matter of fact, pdf_f or gFuncVec are pointers or vector of pointers, and they do not create any trouble. Moreover, the error does not appear consistently at the first iteration but appears randomly, and it is not related to any specific k value.
Here is the full code (it cannot be run since it’s part of a larger project)
macroError.cpp (9.0 KB)
Instead, here is a minimal working example I wrote to try to reproduce the error. This usually works, but sometimes it gives a free(): unaligned chunk detected in tcache 2. I don’t know if the two errors are related in some way.
tryMacro.cpp (1.8 KB)
Cheers,
Francesco
ROOT Version: 6.26/10
Platform: Ubuntu 22.04.3 LTS