Parallelization inside a ROOT macro

ROOT
5

Hi @Dario_Ramirez ,

the behavior you see is unexpected and I cannot reproduce it. Map should only return when all tasks are done.

Indeed that’s what happens when I run this version of your example where I added a couple of printouts to check the order of operations:

// my_macro.C
void save_objs() { std::cerr << "saving objs\n"; }

void my_parallel_func() {
  gROOT->SetBatch();

  const UInt_t n_thread = 16;

  // This function contains what was before inside the for loop
  auto work_item = [](size_t thread_id) {
    /// heavy code
    // Creating TObjects and save them to some containers
    std::cerr << "task for " << thread_id << " done.\n";
    return 0;
  };

  // Create the pool of workers
  ROOT::TProcessExecutor workers(n_thread);

  // Fill the pool with work
  workers.Map(work_item, ROOT::TSeqI(n_thread));
  // This replaces the for loop and parallelized
}

void my_macro() {
  my_parallel_func();
  save_objs();
}

Here is the output of 3 example executions, you can see saving objs is always printed after the tasks are done (while the tasks running concurrently sometimes messes up their printouts):

/tmp root -l -b -q my_macro.C

Processing my_macro.C...
task for 0task for  done.
2 done.
task for 1 done.
task for 3 done.
task for task for 5 done.
task for 6 done.
4 done.
task for 13 done.
task for 8 done.
task for 9 done.
task for 11 done.
task for 10 done.
task for 7 done.
task for 12 done.
task for 14 done.
task for 15 done.
saving objs
/tmp root -l -b -q my_macro.C

Processing my_macro.C...
task for 0 done.
task for 2 done.
task for 1 done.
task for 5 done.
task for 3 done.
task for 4 done.
task for 6 done.
task for 8 done.
task for 7 done.
task for 9 done.
task for 13 done.
task for 12 done.
task for 10 done.
task for 11 done.
task for 14 done.
task for 15 done.
saving objs
/tmp root -l -b -q my_macro.C

Processing my_macro.C...
task for 0 done.
task for 3 done.
task for 1task for  done.
2 done.
task for 4 done.
task for 5 done.
task for task for task for task for 13task for 6task for  done.
109 done.
 done.
11 done.
task for 8 done.
7 done.
 done.
task for 12 done.
task for 14 done.
task for 15 done.
saving objs

Feel free to provide a self-contained reproducer for your issue that we can use to debug what is happening.

Cheers,
Enrico