I am doing a 2D extended maximum likelihood fit in RooFit (primarily binned but I am also summarizing the results from the unbinned version of it).
Data-Set = Signal (peak in both the variables ) and background (flat in both the variables)
(the reason of 2D fit is I have other backgrounds which are peaking, but for now, to isolate the problem I just mention flat background).
Below are the results from binned and unbinned extended maximum likelihood fits. All the pdf shape parameters are fixed except the mean, all the yields are floated. I see a consistent bias in the yield returned from fitter, a big fraction of Signal events are always assigned to background.
|All yields fixed to expected value
||-2.45235 X 107
||-2.45266 X 107
||-2.45236 X 107
||-2.45267 X 107
Between the first and second row the chi2/ndf changes by a large amount whereas the log likelihood value change doesn’t look very significant ?.
Does anyone know what might be the reason for this bias?
I also try to do the chi-square fit and couldn’t make sense of the results either
Below table summarizes the result from each method:
When the yields are floated, fitter converges to a higher value for RooChi2Var, while by definition it should converge to the least chi-square value.
Also what is the difference between # returned by FCN and the one returned by RooChi2Var::getVal()
I think we are missing a crucial piece to get the full picture: What did you do to get the
I remember that one (of probably multiple) ways to obtain
chi^2 does the
ndf correction, but
ndf has a different meaning than what you might think. I would need to look up what the function you used is doing to get the details right.
If the problem is in the determination of
ndf, it’s not surprising that you get different results when you change
ndf by floating more fit parameters.
To obtain the Mdz chi2/ndf and Delm chi2/ndf , I have used the following code:
mdz_frame->chiSquare() , deltam_frame->chiSquare()
deltam_frame are the frames used to draw 1d projections of my 2d fit pdf.
To obtain the values in 5th row, 4th and 8th column I am using:
RooChi2Var chi2_val("chi2_val","chi2",mdzdel1,ds1) ; chi2_val.getVal() ;
mdzdel1 is 2d fit pdf and ds1 is binned data-set on which chi-squared fit is performed.
Ok, so what you are seeing are chi^2 referring to different things.
- Check the documentation for RooPlot::chiSquare. It compares a curve that has been plotted with the data distribution. Here’s the catch: It doesn’t have a way to know how many fit parameters were used to arrive at the curve.
That explains the difference you are seeing when you use more or less floating parameters. Without the correction, it will always assume that no parameters were floating.
You therefore must apply a correction. The number of floating parameters can be retrieved from the fit result, or you do it manually.
- The RooChi2Var doesn’t do any NDF correction according to its documentation here. I guess you have to apply it.
Further note that in order to get properly normalised PDFs, you have to say what the observables are. Check if
chi2_val.getVal(RooArgSet(x, y)) are different (with the respective observables in your setup).
Hi @StephanH. Thanks for the reply. (Half problem solved)
Here are the new set of results for chi-square fit:
1> To get # of floating parameters I have used:
RooArgSet *flparams = mdzdel1.getParameters(observables);
int nparam = (flparams->selectByAttrib("Constant",kFALSE))->getSize();
So the new chi-square/ndf values are little higher than previous ones since ndf get reduced by the # of floating parameters.
2> To get the raw chi-square value I am doing:
RooChi2Var chi2_lowstat("chi2_lowstat","chi2",mdzdel1,ds1,Extended(kTRUE)) ; chi2_lowstat.getVal(observables);
so I added the option Extended(kTRUE) and this chi-square value matches with FCN now.
Also, adding observables in chi2_lowstat.getVal(observables); doesn’t make a difference.
3> Now, when the yield parameters are floated the fitter converges to a lower chi-square value than the one when the yields are fixed, since by definition the MINUIT is trying to find the least chi-square FCN.
All clear until here
But the chi-square values from 1d projections are much higher (8.71,8.94) in this case (yields floated) vs (1.26,1.56), do you know what might be the reason for the disagreement between 1d and 2d chi-square values ??
Thanks a lot for your time @StephanH
nice work working out the NDF correction!
For the difference between 1D and 2D, I have two ideas:
- NDF needs to be computed properly. Remember that in the 2D case, you have to compute NDF yourself because it’s only returning raw chi2 values. So you need the number of bins in 2D, and subtract the number of parameters.
- Bin volume correction. The
Chi2Var computes as follows:
Chi2 = Sum_bins (<pdf probability> * <nData_tot> * <binVolume> - nData_bin)^2 / error^2
The chi2 after plotting computes:
Chi2 = Sum_<curvePoints> (nData_bin - <integrate curve over bin> / binWidth)^2 / binError^2
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.