I am using RooMCStudy to generate pseudo-data and then fit three p.d.f’s using binned extended max likelihood fitting. After 1000 runs I fit a gaussian to the resultant parameter distribution to get the fit result and its error. Two of the p.d.f’s have a similar shape so I apply a gaussian constraint to one using the ExternalConstraints() option in RooMCStudy.
I now have a problem interpreting the error and pull distributions that are output for the constrained parameter. The mean of the error distribution is much larger than the width of the parameter distribution. Also the width of the pull distribution is much less than 1.
How do I interpret these results?
Which value should I take as the error on the fit? The error distribution peaks at a value 4 times the width of the parameter distribution. The p.d.f with a similar shape also shows this difference, though it is not as pronounced. What is causing the difference?
I’m using ROOT 5.26, RooFit 3.12.
Thanks for any help,
To get proper pull distributions with constraints the generated value of the constrained parameters should be varied according to the constraint distribution when one does toy MC studies (so each toy has a different
’true’ value for the constrained parameters). RooMCStudy take cares of this when the constraint is part of the original pdf, but not when you
specify ExternalConstraint() in the FitOptions() as that only affects the fitting part and not the generation part. In that case you expect a pull
distribution that is too narrow (as you also observe)
Can you try to do the following: Given your pdf P and your constraint
pdf C, construct a product of the two
RooProdPdf PC("PC","PC",RooArgList(P,C)) ;
Then when you use this pdf you can specify a Constrain() argument that
lists the parameters that are constrained (rather than the pdf with the constraints) and generating, fitting and toy-MC study code should automatically parse the pdf-constraint product in the correct way.
For RooMCStudy you supply list the Constrain() argument directly
to the constructor of RooMCStudy and not nested in a FitOptions.
For a complete example of RooMCStudy with a pdf with constraints
root.cern.ch/root/html/tutorials … str.C.html
Thanks for the explanation. The method you described worked and the pull value is now reasonable. I now have some more questions about the constraint. The MCStudy is defined as follows:
In the example you linked to you mention the value for st is sampled from the constraint pdf before each toy generation. Will a poisson distribution then be generated from this st, due to the Extended() option, or will the value from the constraint pdf be taken as the number of generated events?
I also still don’t understand the error distribution.
I plot the error distribution using:
RooPlot* frame_est = mgr.plotError(st);
I plot the parameter distribution using:
RooPlot* frame_nst = st.frame(Name(“NEvents”),Title(“Number of events”));
I then fit a gaussian to the parameter distribution. For the parameters which are not constrained, the sigma of the fitted gaussian is close or equal to the mean of the error distribution. However for the constrained parameter the results are very different. I have attached the plots mentioned above for comparison.
FitError_st.pdf (14 KB)
Gaussian_nevts_st.pdf (12.3 KB)