Hello RooStats experts. I’ve implemented a class to calculate asymptotic limits using the RooStats::HypoTestInverter class, almost precisely configured as suggested here .
This seems to work fairly well, however I occasionally get strange behavior where the ±sigma values don’t make sense in the context when returning htresult->GetExpectedUpperLimit(+1) (and other indices.) Below is such an example showing the different expected value results of a limit calculation:
As you can see, the +1 and +2 sigma values are virtually identical, but the +2 value is lower than the +1. Also, the -2 sigma value is higher than the median value. How can this make sense? I don’t see anything in the output that points to why this is happening. Is this explainable?
I agree the results you obtain do not make much sense. In that case you should look at the scan plot obtained to compute the limit to understand exactly what happened. It could be that some of the fit failed at some scanned points and their results do not make sense.
If you cannot fix this issue for example by using more scanned points or more stable fits, could you please upload your RooFit workspace and your macro computing the limit, so we can investigate this in detail.
I already use quite a coarse grid, 200 scan points across a range of 0 to 2.0 for the results above, is this not actually that coarse? I’ve used the same input workspaces in a different “CalculateLimits” script (which is a giant 2k line black box that I wanted to avoid using for this reason), that I’ve validated for some other mass points of my signal production that don’t exhibit this behavior.
As for our macro, we have it included into a class in our Statistical Framework but it follows verbatim the originally linked RooStats directions.
Hi, Sorry again for not understanding exactly what this plot object is. I see it is just an inherited TObject. If I want to save this somehow, the best way would be to save the canvas with c1->SaveAs()?
If you want just to save the plots, yes it is maybe easier to save the canvas. Otherwise if you want to change slightly the plots later on, the best is to save the HypoTestInverterResult object, that you can read afterwards and re-do the plotting.
And here is the scan plot. In this example, you can see the p1 and p2 points are all lower than the medium (even if just slightly) and the m1 and m2 points are much much lower (and match the values that I get from the alternative fitting script). TestScan.pdf (20.2 KB)
It is clear from the scanned plot that for values of your parameter of interest larger than 0.5 there is a problem in getting the correct CL(s+b) and CL(b). This is due to the fits which are performed for that parameter values. I would try to understand why these fits fail. Maybe it is a problem on how your model is defined or you might need to adjust some parameters , like starting points of the fit, to make the fit working.
If I have access to the workspace, I could try helping you with the fits