Loss of performance using RooFit::Range in ROOT 6.30

sblancof · January 16, 2024, 10:52am

Dear ROOT experts,

We have found large performance differences when we run our code with ROOT 6.26 and ROOT 6.30. We perform a fit to the Z boson resonance in which we simultaneously fit a signal and a background template using a RooWorkspace. We fit the model into three different ranges of invariant mass:

[60, 120] GeV
[80, 100] GeV
[75, 105] GeV

To do so, we adjust the fit range with the following commands in pyROOT:

self._w.var(self._fitVar).setRange(self._fitRangeMin, self._fitRangeMax)
self._w.var(self._fitVar).setRange('fitRange', self._fitRangeMin, self._fitRangeMax)

resPass = pdfPass.fitTo(self._w.data(hPassName),
                                ROOT.RooFit.Minimizer("Minuit", "minimize"),
                                ROOT.RooFit.Optimize(1),
                                ROOT.RooFit.Save(),
                                ROOT.RooFit.Range("fitRange"),
                                ROOT.RooFit.Minos(True),
                                )

Where pdfPass is the convoluted PDF of our Monte Carlo template with a Gaussian function plus an exponential background model. Recently, we migrated from ROOT 6.26 to ROOT 6.30, but the code didn’t change. Our experience is that the fit converges similarly, but the time spent to run it has increased exponentially in ROOT 6.30.

These are the times that we get running the same fit in two versions of ROOT:

ROOT 6.26: 43 seconds
ROOT 6.30: More than 30 minutes

We are not running any complex model so we expected a fast minimization. After some debugging, we noticed that the responsible for the time increase is the usage of a determined Range. When we remove the restrictions on the Range parameter we recover the performance from ROOT 6.26.

We would appreciate the help of the ROOT experts on this issue.

Thanks you in advance for the help,
Sergio Blanco

vpadulan · January 16, 2024, 4:19pm

Dear @sblancof,

Thank you for reaching out to the forum! I believe @jonas will be able to help you understand the cause of the performance degradation.

Cheers,
Vincenzo

jonas · January 21, 2024, 1:20pm

Thank you @sblancof, thanks a lot for your report!

This is a known regression that I didn’t fix yet. There is a GitHub issue about it, where I increased now the priority:

github.com/root-project/root

[RF] Unneeded normalization integrals when fitting a RooAddPdf in a range

opened 03:24PM - 12 Apr 23 UTC

elusian

bug priority:high in:RooFit/RooStats

- [x] Checked for duplicates  ### Describe the bug  In a `RooAddPdf`, the coefficients are tied to the normalization range. When passing a range name to the `RooAbsPdf::fitTo` call, this is visible because new integrals will be created over the fit range, to fix the definition of the coefficients. I will refer to these integrals as coefnorm integrals. In ROOT 6.28, coefnorm integrals for ranged fits of `RooAddPdf`s were changed. This, however, added several integrals that with the old method would not be evaluated. In particular: - the new implementation is unable to elide coefnorm integrals over ranges equivalents to the full range - in the new implementation, to every coefnorm integral over the fit range correspond an integral over an empty named range. This is separate to the integral with no range, which is also done - calling `addPdf.fixAddCoefRange(fit_range)` before the fit (a common solution if you wanted to avoid coefnorm integrals) removes the unnamed range coefnorm integrals, but not the named range ones Note, however, that in all the test I performed these integral do not seem to affect fit results, which are the same before and after 6.28. Instead, the problem is that those integrals may not be easy to compute: this issue was noticed because among the extra integrals was a complex numeric integral that had to be evaluated at each event in the dataset, was extremely slow, and in practice caused the fit to never converge. ### Expected behavior  No extra integrals over unnamed ranges, passing the full range to fitTo does not add integrals, calling `fixAddCoefRange` removes all coefnorm integrals. ### To Reproduce  ```Python import ROOT from ROOT.RooFit import RooConst # in this simple example the symptoms are only in the logs, in the names of the integrals evaluated ROOT.RooMsgService.instance().addStream(ROOT.RooFit.INFO, Topic = ROOT.RooFit.Integration) x = ROOT.RooRealVar('x', '', 0, 1) g = ROOT.RooGaussian('g', '', x, RooConst(0.5), RooConst(0.2)) u = ROOT.RooUniform('u', '', x) f = ROOT.RooRealVar('f', '', 0.5, 0, 1) a = ROOT.RooAddPdf('a', '', [g, u], [f]) dt = a.generate({x}, NumEvents = 1000) x.setRange('limited', 0.2, 0.8) x.setRange('full', 0, 1) a.fitTo(dt, Range = 'full', PrintLevel = -1) a.fitTo(dt, Range = 'limited', PrintLevel = -1) a.fixAddCoefRange('full') a.fitTo(dt, Range = 'full', PrintLevel = -1) a.fixAddCoefRange('limited') a.fitTo(dt, Range = 'limited', PrintLevel = -1) ``` Compare 6.26 with master. ### Setup  ROOT 6.26 from LCG 102b ROOT master from LCG dev3 ### Additional context

I will try to fix this in the next days.

Cheers,
Jonas

sblancof · January 24, 2024, 5:34pm

Dear Jonas,

Thank you so much for the prompt reply! I see the problem. In the meantime, is there any known way to bypass these normalization integrals?

Bests,
Sergio

system · February 7, 2024, 5:34pm

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.