Feldman Causing and large signal uncertainty


(I use root 6.03/04)
I’m learning roostats and I was trying to do something simple: evaluate the importance of uncertainties on limits. I took rs101_limitexample.C, set observed = background on l 86, and then started increasing uncertainties. If I increase the background uncertainty on l 72 things go south pretty quickly:
at 10% (as in rs101) I get

Profile upper limit on s = 28.5301
FC upper limit on s = 35.4
MCMC upper limit on s = 33.3135

at 20% the Feldman-Cousins test starts going south:
Profile upper limit on s = 44.309
FC upper limit on s = 90.6
MCMC upper limit on s = 52.0216

at 30% FC hits its scan limit
Profile upper limit on s = 62.4388
FC upper limit on s = 119.4
MCMC upper limit on s = 72.4625

Is there something special I need to do to make FC “behave” in the presence of larger uncertainties? No change I’ve tried in number of samples etc. has made things better.




Thank you for pointing to this problem. Actually it is recommended for running Feldman-Cousins to use the FrequentistCalculator and the HypoTestInverter class as it is done for limit settings using CLs.

I see also the way of doing the model is not 100% right. The model does not contain the global observables needed to correctly include the systematics in FC.
I attached a new corrected version of the macro. I addition the macro saves in a file the workspace.
After running the rs101 you can run also the roostats/StandardHypoTestInvDemo.C macro to use the recommended way of doing FeldmanCousins. For example do at the root prompt

.x rs101_limitexample.C 
.L  $ROOTSYS/tutorials/roostats/StandardHypoTestInvDemo.C
root [2] StandardHypoTestInvDemo("rs101_ws.root","w","ModelConfig","","exampleData",0,2,0,10,0,150)


rs101_limitexample.C (9.29 KB)

Thanks, this helps a lot.