Questions about RooStats::FeldmanCousins

Roofit and RooStats
1

Hello,

I wish to place upper limits on an exotic physics process. My input data is a 1-D histogram of the mass of my candidates. I parametrize this histogram by the sum of a polynomial PDF representing the background, plus a gaussian PDF (at a given mass) representing my signal.

I calculate the limits using the RooStats::FeldmanCousins. However, after reading the documentation root.cern.ch/root/html/RooStats_ … usins.html , I still have several questions:

  1. The documentation does not explain what the option FeldmanCousins::FluctuateNumDataEntries(bool) does. Am I correct that if the number of candidates would fluctuate if I repeated the experiment, then I should set this to true, but if it would always be the same (not true in my example), then it should be false ?

  2. I declare my “parameter of interest” (which is the process’s cross-section) as follows:

xsec = r.RooRealVar(“xsec”,“xsec”,1.,0.,100.)

I find that the upper limit calculated by FeldmanCousins depends strongly upon the allowed range I specify for this parameter (0. – 100. here). If I specify a very large range (0. - 100000.), then the precision of the upper limit calculation is poor. On the other hand, if I specify a very small range (0. – 0.001), then the returned upper limit starts to go down, apparently because I am truncating the distribution.

I therefore find that the upper limits calculated by FeldmanCousins can easily go wrong. To make this worse, if I do specify too small a value for the upper range of xsec, FeldmanCousins does not issue any warning.

  1. I wish the normalization of the background PDF to float free, so it can be constrained by data, without any apriori assumptions. I do this by declaring the normalisation of the background PDF to be a “nuisance parameter”, and assign it extremely large apriori uncertainties. Are the IntervalCalculators provided in RootStats all robust in this case where a nuisance parameter have very large apriori uncertainties ?

Thanks,
Ian Tomalin

2

Hi Ian,

  1. FeldmanCousins::FluctuateNumDataEntries is set by default to true, in this case the number of events per toy will be generated with Poisson fluctuation according to the expected events from the pdf.
    If the model does not provide an expected number of events, (for example in the case of number counting),
    you need to set this to false.

  2. In principle you should declare a sensible range which does not truncate the distributions (i.e. min,max are few sigma away from the maximum likelihood estimate). The problem of declaring a too large range, is that the scan will be done in non-interesting points. You need maybe in that case to use the FeldmanCousins::SetParameterPointToTest function

  3. What is the model for this extra nuisance parameter ? If you don’t want to put any priori assumption, keep it floating in its allowed range and it will be profiled by fitting the data

Cheers, Lorenzo

3

Thanks for your reply. Regarding item (2), I find it bad that RooStats::FeldmanCousins will give unreliable answers unless one gives it a “sensible range” for the POI.

Inexperienced users will not realise this, and will simply call RooStats::FeldmanCousins and may get incorrect results. Experienced users will realise it, and will be forced to write lots of code themselves to determine the “sensible range” of the POI before calling RooStats::FeldmanCousins.

I would therefore suggest that inside RooStats::FeldmanCousins, you add some code to determine the sensible range of the POI. (e.g. By calling a fast limit calculation method, such as profile likelihood). This could either override the range on the POI given by the user, or issue warning messages if the user has set a “non-sensible range”.

Thanks,
Ian