Shape of p value plot for frequentist approach

Dear @moneta

I have extracted upper limit on Number of signal counts for Model = (Gaussian signal + poly Bkg) by using both the frequentist and Asymptotic profile likelihood method. The upper limit and experimental sensitivity values from both methods are quite reasonable and matching with each other but i don’t understand why p value plot for frequentist approach is not so smooth as asymptotic one. Is it fine, if its like this, i mean not so smooth.

I tried frequentist approach for 500, 1000, and 5000 MC toys. The p value plot shown below is for 5000 toys. Should i generate it for more toys or is it fine like this?

z

c

Thank you!

Hi,

Yes you would need to generate much more toys to have it smoother. Especially when computing small p-values, you can have large fluctuations from the toys, since very few of them will have a larger test statistic values than the one of the data. So, what you have, it looks fine to me !

Cheers

Lorenzo

Thank you very much for your help.

You have been really really helpful to me this whole time. I appreciate that.

Have a nice day :slight_smile:

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.