Plot normalization after RooAddPdf::fixAddCoefRange

Hello
I’m trying to plot my model, but I’m incurring in problems with normalization integrals.
The model is a RooAddPdf of RooProdPdf with a Conditional term, and is fitted over a limited range in the conditional variable (because in this dataset in particular there is no data points outside the range).
To avoid additional normalization integrals (which would be horribly slow) the RooAddPdfs are normalized over the fit range with RooAddPdf::fixAddCoefRange (this is done in ROOT 6.26, as later versions cannot elide the integrals). The fit converges and gives good results, but the plot seems to ignore that and always tries to correct the normalization.

This is easy to observe in the plots of the non conditional variables: if the range “matched” then projecting the model over the variable would be equivalent to a RooAddPdf of only the relevant product factors. In the actual plots, no matter the combination of Range or NormRange, the latter works (meaning that the fit is ok), while the former tries to correct the normalization with additional integrals over other variables and is always far off, not even normalized correctly.

In the plots of the conditional variable instead, the normalization makes sense and the plot follows the data. The problem here is that the extra normalization integrals are extremely slow (estimated plot time is > 1week).

Is there any way to tell the plot that everything is normalized in the limited range and there is no need for correction?

This is a macro showing the issue

import ROOT
from ROOT.RooFit import RooConst

x = ROOT.RooRealVar('x', '', 0, 1)
y = ROOT.RooRealVar('y', '', 0.001, 0.1)
z = ROOT.RooRealVar('z', '', 0, 1)


tm = ROOT.RooGaussModel('gm', '', x, RooConst(0), y)
tau1 = ROOT.RooRealVar('t1', '', 0.2, 0.0001, 1)
d1 = ROOT.RooDecay('d1', '', x, tau1, tm, ROOT.RooDecay.SingleSided)
tau2 = ROOT.RooRealVar('t2', '', 0.05, 0.0001, 1)
d2 = ROOT.RooDecay('d2', '', x, tau2, tm, ROOT.RooDecay.SingleSided)

g1 = ROOT.RooRealVar('g1', '', 4, 3, 5)
b1 = ROOT.RooRealVar('b1', '', 0.005, 1e-4, 0.01)
gy1 = ROOT.RooGamma('gy1', '', y, g1, b1, RooConst(0.001))
g2 = ROOT.RooRealVar('g2', '', 12, 10, 14)
b2 = ROOT.RooRealVar('b2', '', 0.003, 1e-4, 0.01)
gy2 = ROOT.RooGamma('gy2', '', y, g2, b2, RooConst(0.001))

gz = ROOT.RooGaussian('gz', '', z, RooConst(0.5), RooConst(0.1))
uz = ROOT.RooUniform('uz', '', z)

prod1 = ROOT.RooProdPdf('prod1', '', {gy1, gz}, Conditional = (d1, x))
prod2 = ROOT.RooProdPdf('prod2', '', {gy2, uz}, Conditional = (d2, x))

f = ROOT.RooRealVar('f', '', 0.8, 0, 1)
s = ROOT.RooAddPdf('s', '', [prod1, prod2], [f])
sy = ROOT.RooAddPdf('s', '', [gy1, gy2], [f])
sz = ROOT.RooAddPdf('s', '', [gz, uz], [f])

x.setRange('limited', 0.05, x.getMax())
# just in case, but it doesn't change anything
y.setRange('limited', y.getMin(), y.getMax())
z.setRange('limited', z.getMin(), z.getMax())

dt = s.generate({x,y, z}, NumEvents = 100000)

# cut, because that's how my data looks like
dt = dt.reduce(CutRange = 'limited')

s.fixAddCoefRange('limited')
sy.fixAddCoefRange('limited')
sz.fixAddCoefRange('limited')

print('Second fit', flush = True)
s.fitTo(dt, Range = 'limited', PrintLevel = -1, Save = True).Print("V")

frame = x.frame(Range = 'limited')
dt.plotOn(frame)
s.plotOn(frame)
s.plotOn(frame, Range = 'limited', LineColor = 'r', LineStyle = 9)
s.plotOn(frame, NormRange = 'limited', LineColor = 'g', LineStyle = 7)
s.plotOn(frame, NormRange = 'limited', Range = 'limited', LineColor = 'y', LineStyle = 2)
frame.Draw()

ROOT.gPad.SetLogy()
ROOT.gPad.SaveAs('testx.png')
ROOT.gPad.SetLogy(False)

frame = y.frame()
dt.plotOn(frame)
s.plotOn(frame)
s.plotOn(frame, Range = 'limited', LineColor = 'r', LineStyle = 9)
s.plotOn(frame, NormRange = 'limited', LineColor = 'g', LineStyle = 7)
s.plotOn(frame, NormRange = 'limited', Range = 'limited', LineColor = 'y', LineStyle = 2)
sy.plotOn(frame, LineColor = 'm', LineWidth = 1)
frame.Draw()

ROOT.gPad.SaveAs('testy.png')

frame = z.frame()
dt.plotOn(frame)
s.plotOn(frame)
s.plotOn(frame, Range = 'limited', LineColor = 'r', LineStyle = 9)
s.plotOn(frame, NormRange = 'limited', LineColor = 'g', LineStyle = 7)
s.plotOn(frame, NormRange = 'limited', Range = 'limited', LineColor = 'y', LineStyle = 2)
sz.plotOn(frame, LineColor = 'm', LineWidth = 1)
frame.Draw()

ROOT.gPad.SaveAs('testz.png')

Cheers and thank you in advance,
Enrico

P.S. This example does not work in ROOT >= 6.28, as the line
s.fitTo(dt, Range = 'limited', PrintLevel = -1, Save = True).Print("V")
gives
ERROR:ObjectHandling -- RooArgSet::term::addOwned: can only add to an owned list
That looks like a bug.

Maybe @jonas can take a look

Hello @elusian!

For your actual question, I need more time to investigate and I’ll come back to it next week.

As for the bug in 6.28, I have opened a PR to fix it that will be backported to the next 6.28.04 patch release:

For debugging, I translated your code to C++, which I will just add here in case it gets useful again.

plotNormalizationReproducer.C (3.3 KB)

Thank you!

By the way, rereading your C++ macro I noticed that I may have left some printout from when I was experimenting (“Second fit” for example, there is no first anymore and it was just to separate the log)…

Hello @jonas, sorry for the ping.
Did you have time to investigate? Are there any news?

Hello, just a ping to keep the thread from closing.

Hello, sorry but I’m still in need of an answer.