Normalization range seemingly ignored

[aaronsw]

Hi all,

I would like to perform extended fits over a particular range, called fitRange. Even when I specify the range, however, I still get warnings about the PDF being evaluated outside of the range. My question is: how do I make sure the normalization/evaluation of the PDF is constrained to the range I provide?

As a demonstration, I have the following:

• A PDF = 1/x, so it’s undefined at x=0
• x[0,3], so 0 is within the range of the variable x
• I call PDF->expectedEvents() to trigger this warning, however I also see the warning come up inside fits

The standalone code to reproduce this:

from __future__ import division
import sys
sys.path = ["/home/prime/Downloads/buildRoot/lib/"] + sys.path
from ROOT import *
from ROOT.RooStats import *


# This is the range I want to operate over:
fitMin=1
fitMax = 3

# Set up workspace
w = RooWorkspace("w","w")
w.factory("x[0,3]")
w.factory("EXPR::pdf('1/x',x)")
w.var("x").setRange("fitRange",fitMin,fitMax)

# make PDF
pdfNorm = RooRealVar("pdfNorm","pdfNorm",1);
pdfExpr = w.pdf("pdf")
pdfExtExpr = RooExtendPdf("pdfExt","pdfExt",pdfExpr,pdfNorm,"fitRange")

print pdfExtExpr.expectedEvents(RooArgSet(w.var("x")))
# This does not work:
#   pdfExtExpr.expectedEvents(RooArgSet(w.var("x")),RooFit.Range("fitRange"))

The warning I am concerned about:

[#0] WARNING:Integration -- RooIntegrator1D::integral: integral of pdf over range (0,3) did not converge after 20 steps
   [1] h = 1 , s = 0.5
   [2] h = 0.25 , s = 1.25
   [3] h = 0.0625 , s = 1.95833
   [4] h = 0.015625 , s = 2.65536
   [5] h = 0.00390625 , s = 3.34948
   [6] h = 0.000976562 , s = 4.04287
   [7] h = 0.000244141 , s = 4.73608

[StephanH]

Hi @aaronsw

• The range you give to the RooExtendPdf only determines how coefficients are interpreted.
  That is, you have a variable with a range [0, 1], but you define the coefficient for the extension only in the signal region, let’s say [0.5, 0.7]. Giving a range name will mean that the coefficient is calculated such that it is equal to the number of events observed in [0.5, 0.7], although it may have a different value when evaluated over the full range.
• When you fit, the argument ROOT.RooFit.Range("name") will determine what’s fitted (and prevent the fit from going into the undefined region).
  • Or: You set a default range (no name) from the beginning:
    w.var("x").setRange(fitMin,fitMax)
  • Or: You set it directly in the factory:
    w.factory("x[1,1,3]")

[aaronsw]

Hi @StephanH,

Thanks very much for your reply. So in my example, when I call pdfExtExpr.expectedEvents(RooArgSet(x)) this returns the number of expected events between x.getMin() and x.getMax()? The RooAbsPdf documentation says this function “returns expected number of events from this p.d.f for use in extended likelihood calculations.” So do I have to do something to make sure the number of events in the fit is actually calculated in the fit range?

Thanks again for your help.

[StephanH]

@aaronsw

this returns the number of expected events between x.getMin() and x.getMax()

Yes, almost. For a RooExtendPdf, it does a bit more:
https://root.cern.ch/doc/master/classRooExtendPdf.html#ace086475af42d01f4056ed062164ecc4

It calculates the ratio of events in the restricted range (the one you gave in the constructor, if you gave one) to the number of events over the full range ([0, 3]) in your case. This is where things go wrong.
The outcome is the number of events over the full range of x, regardless of what the range in the constructor was. In this way, you can measure the number of e.g. signal events in a specific range, that is, the norm parameter in the constructor will show you the number of signal events, but the fit will do the right thing because it has to fit the full range, and therefore it needs to know the total number of events.

Normally, this is what you would expect it to do. Only here, the problem is that integrating the full range will get you in trouble.

[aaronsw]

Hi @StephanH, okay, this explains it. Thanks!