I am attempting to use RooStats to set an upper limit on a parameter, R (which is a ratio of efficiency corrected yields).
I have built a fit model using RooFit, and the fit is parametrized in terms of R.
I have expressed my systematic uncertainties in the form of Gaussian constraints on parameters. For example, an efficiency that has been evaluated from MC gets a Gaussian constraint.
I am very confused as to which calculator I should use to find the upper limit. There are so many options like the Asymptotic Calculator, the Feldman Cousins calculator, Profile Likelihood Calculator etc. Further, different calculators give me very different answers.
There are also lots of choices for the test statistic and I am not sure which one I should use to set an upper limit. I find myself not knowing much about the statistics underlying these choices.
I am concerned about whether these calculators properly take into account the systematic uncertainties in my analysis. Naively, I would expect that the upper limit result would increase if I increase the systematics by hand, but I find that to not be the case when I try to use the asymptotic calculator.
I have tried to use the source code of these different calculators to work out what is happening behind the hood, but I always end up getting lost.
Here are my questions:
How does one choose a calculator and test statistic?
What is the proper way to inform the calculator of my systematic uncertainties?
I’m afraid that I am currently unable to provide a reproducible form of my code because it is very long and messy, needing input from multiple large ROOT files.
I would appreciate any advice regarding upper limit calculation using RooStats, even if it isn’t directly related to what I have asked. I have looked through a lot of tutorials and slides to find my answers, but I am still confused.