FeldmanCousins

Hi

I have a few questions concerning the RooStats::FeldmanCousins Calculator.

First, in the introduction it says, that you can choose either profiling of nuisances, or varying them over their whole range. I wonder, where this can be set.

Second, as far as I have figured out, the Parameter of interest (POI) is scanned in its predefined range, but the scan does not include the borders of this range.
If my POI is signal for example and I set the range to (0,100) and let him test 100 points, first point to be tested will be 1, not 0. Is there a way to change this behavior?

And third, I noticed, that my resulting interval depends on the range of my POI more than is expected from the fact, that my test points are discrete.
I wonder if this is reasonable, actually I think, that the decision, if for a specific value for the POI the observation is in the corresponding interval, should not depend on other values of the POI.

I hope, I could make my points clear enough.

Cheers Markus

Hi

First, in the introduction it says, that you can choose either profiling of nuisances, or varying them over their whole range. I wonder, where this can be set.

When you SetParameters, only give it the parameters you want it to scan explicitly.
fc.SetParameters(*paramsOfInterest);
If you have 10 parameters, it will do a 10-d grid (and take forever). If you only give it a sub-set of the parameters, it will scan those parameters. The remaining parameters will be fixed to their “profile” values (eg. the conditional MLE with the POI fixed).

In the underlying NeymanConstruction class, you can give it a RooDataSet with the specific points that you want it to scan, so in that way you have complete flexibility. This is useful if a grid is not appropriate for your problem. This is done via the method SetParameterPointsToTest.

Second, as far as I have figured out, the Parameter of interest (POI) is scanned in its predefined range, but the scan does not include the borders of this range.
If my POI is signal for example and I set the range to (0,100) and let him test 100 points, first point to be tested will be 1, not 0. Is there a way to change this behavior?

That’s a good use-case. This would be possible with the SetParameterPointsToTest method, but I see that it is not exposed with the FeldmanCousins calculator. I can add that, but to use it you will need to use the RooStats branch.

And third, I noticed, that my resulting interval depends on the range of my POI more than is expected from the fact, that my test points are discrete. I wonder if this is reasonable, actually I think, that the decision, if for a specific value for the POI the observation is in the corresponding interval, should not depend on other values of the POI.

You are right, it shouldn’t matter in principle. There are a few things that can change due to random numbers etc., but I would guess the more likely possibility is related to the range of the variables allowed in the fit. If you constrain the range of the variables in a way that MINUIT runs into boundaries when minimizing, it can change the results. This is because the profile likelihood ratio will not be able to take on the proper value.

For this reason, I would like to improve the way the parameters are scanned and the range of the variables. This was anticipated in the design, but the implementation is not all there. If you can outline your use-case it will help.

Thanks for the good question. Your feedback is very important, and please be patient since the project is still evolving.

Kyle

Hi, sorry that I reply so late.
Thank you very much for your answers, they helped a lot.
I managed to reproduce the results of Feldman Cousins with my own test points and my own test statistic, which I feed to the Neyman construction.

As you asked for the use case, I will try to explain:

I consider a numbercounting experiment with Gaussian uncertainties on the background expectation.
The parameter of interest is a unknown signal contribution, on which I want to calculate confidence intervals

Thank you very much again

Markus