Multithreading RooFit - Memory Corruption

In general is RooFit multi-thread safe? Currently, I am spawning threads in a similar way as shown in https://root.cern/doc/master/mt201__parallelHistoFill_8C.html but I am looking to use RooFit to do unbinned maximum likelihood fits. I can supply an example if needed but currently everything works well when the pool size is 1 but does not work with larger sizes. I get memory corruption error when using the fitTo method. I gave each Roo variable a unique name that depends on the thread index so I believe the function each thread executes should be independent as possible. It seems like simultaneous calls to fitTo is not safe. Alternatively I could always go back spawning multiple root processes instead of using threads…

I had a working version doing binned fits using TF1’s Fit function before I switched over to implementing unbinned fits.
I am using root 6.19 with minuit2. Here is the backtrace if that is of any use.
error.txt (8.4 KB)

I think @moneta may have an answer.

RooFit is being modernised at the moment, but it is not thread safe. That will be a longer project.

That is what I was beginning to think. Maybe I could ask for some advice then.
I have a large dataset that I can make lots of subsets of. I want to run unbinned fits on each of these subsets. Number of subsets on the order of 1M with sizes on the order of hundreds. What is a good way of doing this? I could always spawn independent root processes in a shell script and divy up the work into batches of subsets.

Thanks for the replies!

Do you need subsets? It sounds like you want to do some kind of bootstrapping, is that correct?
You might just start multiple processes, and randomise the weights in each process.

It is sort of bootstrappy but not related to bootstrapping. It is a side band subtraction technique that determines the signal probability of an event by looking at its k neighbors, which are the subsets I am talking about ( Q-Factors: https://arxiv.org/abs/0809.2548 ). Do this for all events and we have ourselves a signal and bkg separation.

Ok, in that case, you should indeed decide on a strategy how to draw the events for each subset, and e.g. initialise the algorithm that draws events based on the job number or similar.

Depending on what kind of information you need, it could be sufficient to draw events, fit, and save the RooFitResult from each run.

RooFit has a simple multiprocessing framework that could be used to parallelise the fitting part NumCPU(), but I would advise not to use it, because you better parallelise by fitting different subsets at the same time. One thing that could be helpful, though, is the BatchMode (since ROOT 6.20). It evaluates the PDFs much faster in unbinned fits.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.