I am working with a Barlow-Beeston style likelihood model with ~1,600 nuisance parameters.
When I try to run ProfileLikelihoodCalculator::GetInterval with Type/Algorithm = Minuit/Migrad or Minuit2/Migrad, the global fit (*) takes ~4.5 hours and ~10 GB of memory. On the other hand, if I use Type = OldMinuit (in which case the Algorithm passed to fitTo isn’t used), the fit only takes 16 minutes and 1 GB of memory.
The problem with using OldMinuit is that LikelihoodInterval::UpperLimit requires that the minimizer is set to Minuit or Minuit2. If I use ROOT::Math::MinimizerOptions::SetDefaultMinimizer to set the minimizer everywhere, the fit in ProfileLikelihoodCalculator::GetInterval goes quickly, but then LikelihoodInterval::UpperLimit fails. I tried using OldMinuit for ProfileLikelihoodCalculator::GetInterval and then Minuit for LikelihoodInterval::UpperLimit. This completed without errors, but I’m worried that switching minimizers like this isn’t correct. I’m actually surprised LikelihoodInterval::UpperLimit was able to find the MINOS errors from a minimizer that was not the one used in ProfileLikelihoodCalculator::GetInterval.
Any help on understanding why Minuit and Minuit2 take so much more time and memory than OldMinuit would be appreciated. I’m also interested in understanding if ProfileLikelihoodCalculator::GetInterval and LikelihoodInterval::UpperLimit must use the same minimizer.
(*) root.cern.ch/root/html/src/RooSt … x.html#142
Just an update:
I moved from ROOT 5.33.02 to 5.34.04, and the big difference in time and memory usage between OldMinuit and Minuit disappeared. This is also true for Minuit2 mostly – the memory usage is still a bit high at about 2.5 GB.
For my own education, I would still be interested in understanding if ProfileLikelihoodCalculator::GetInterval and LikelihoodInterval::UpperLimit must use the same minimizer.
OldMinuit uses the RooMinuit interface instead of RooMinimizer, which is used when running with the option
"Minuit". There were some optimization between 5.33 and 5.34 applied only on RooMinuit only. Now they should be the same. You might get some differences when using Minuit2, because the algorithm is similar but not exactly the same. If you find big differences in 5.34.04 between Minuit and Minuit2, please let me know.
You can use different minimizer for ProfileLikelihoodCalculator::GetInterval and for LikelihoodInterval::UpperLimit. In the first function just a global fit for the minimum is done, while in the second one you run Minos, starting from the previously found minimum.
It will work if you run Minos using a different minimizer, because the class will automatically re-minimize the function, but starting from the values found before. So if you are already at the minimum, it will basically re-compute a new minimum state in the new minimizer.
Thank you for the helpful and speedy response.
To follow up on one point: You said that only RooMinuit was optimized. However, when I changed from ROOT 5.33 to 5.34, the “Minuit” global fit went from taking 4.5 hours and 10 GB of memory to taking about 16 minutes and 1 GB of memory. Are you sure that RooMinimizer was not optimized in this update?
At some point during 5.33 only RooMinuit was optimized and not RooMinimizer. Now both should be optimized and taking approximately the same time and memory