Implementation of profile likelihood in HybridCalculator

Roofit and RooStats
1

Hi,

I have a question concerning the implementation of the profiled likelihood ratio as test statistics (test statistics 3) in the HybridCalculator.

If I read the code and interpret the output correctly, no constrained likelihood fit is performed, but only a standard fit of all non-constant nuisance parameters (in contrast to the ProfileLikelihoodCalculator). Are there any reasons for this?

Thanks,
Jan

2

Hi,

sorry for my late reply, I missed your post.
Yes, the test statistics used in option 3 in the Hybrid calculator is not really a profile likelihood ratio. It is the ratio of the Likelihood_S+B over the Likelihood_B evaluated both at the best parameter values (i.e. the fitted ones). Instead, when using option 1 the likelihood’s are evaluated at the given parameter values.

Probably we could to implement in one of the next release as test statistics also the profile likelihood ratio, defined for example in the class ProfileLikelihoodTestStat.h used by the NeymanConstruction or in the ProfileLikelihoodCalculator.

Best Regards

Lorenzo

3

Hi,
after having looked better at the code, the test statistics in option 3 is a profile likelihood as in the profile likelihood calculator. So ignore please my previous post.
WHat it is calculated it is the ratio of the Likelihood_S+B over the Likelihood_B evaluated both at the best parameter values (i.e. the fitted ones), but this is equivalent to the profile likelihood of S+B evaluated at S= 0.
This is exactly what it is used in the Profile likelihood calculator to obtain the significance. The difference is that in the Hybrid calculator the distribution of the profile likelihood is obtained running toys while in the Profile likelihood calculator the asymptotic form (from the Wilk theorem) is used

Best Regards

Lorenzo

4

Hi Lorenzo,

thanks for the answers. I agree in the absence of nuisance parameters, then exactly the same things are done. However, when nuisance parameters are present, the ProfileLikelihoodCalculator calls ‘fitTo’ with ‘Constrain’ (*), whereas this is not the case for the HybridCalculator (**). I hope this explains my original question. (When I multiply the model pdf with the pdfs for thje nuisance parameters, there are also according warnings during the fits.)

Thanks again,
Jan

PS: A bit more technically, for the HybridCalculator to work correctly, I have to manually reset the parameters to be fitted in HybridCalculator::Calculate(RooAbsData& data, unsigned int nToys, bool usePriors).

(*) root.cern.ch/root/html/src/RooSt … x.html#142
(**) root.cern.ch/root/html/src/RooSt … x.html#265 (and other occurences)

5

Hi,

what you say is correct because the nuisance parameters in the hybrid calculators are varied when the toys are generated. This is equivalent to integrate them using a Bayesian method (for this reason the calculator is called hybrid).
I am not sure that makes sense to perform both the integration and the constraint fitting at the same time.

Concerning your second remark, I realized that last Friday (thanks to your post) and I have submitted a fix in the development branch of roostats
(see root.cern.ch/viewcvs?view=rev&revision=33061 )

Best Regards

Lorenzo