Parameter Uncertainties for Unbinned ML Fit to Weighted Data

[jeff]

Hello,

I am trying to perform an unbinned maximum likelihood fit (extended) to a weighted MC dataset, but the resulting fit parameter uncertainties are much larger than they should be. When no limits are placed on the parameter of interest during the fit, the uncertainty can be several times the value of the parameter itself. The command used is:

fit_result = pdf.fitTo(data, RooFit.Save(), RooFit.Range("fit_region"), RooFit.Extended(),RooFit.SumW2Error(True),RooFit.Minimizer("Minuit2", "migrad"))

The weight values in question are all below 1 and peak around 0.002. See the attachment for an example of the event weight distribution.

When I perform the same fit to the unweighted data (with SumW2Error set to False), the resulting uncertainties are substantially smaller, despite a negligible change in the shape of the distribution.

A very similar issue has been reported here:

Wrong fit errors when fitting weighted data set

Hello, I am performing an unbinned ML fit of weighted MC, where a RooAbsPdf is fitted to a weighted RooDataSet and the average weight is small with a value of about 1/333. The command looks like this: pdf.fitTo(data, RooFit::Minos(kTRUE)); The resulting errors on the fitted pdf parameters are much too large. When fitting the unweighted data set the parameter errors drop more than an order of magnitude. This problem also occurs when adding the option RooFit::SumW2Error(kTRUE) or without using…

There is a brief suggestion of using the event weight distribution itself, but it’s not very clear to me how this would be done.

I understand that an approximation is used to estimate the uncertainties for ML fits to weighted data, but is there a regime in which it is not valid? Is MINOS still not possible to use for these types of weighted fits?

For reference, I am using ROOT version 6.04.14.

Many thanks in advance!

Best,
Jeff

[moneta]

Hi,

This is very strange, because if your weights are all significantly less than one, the number of effective entries will be larger than the simple sum of the weights. Therefore the correction applied when you use RooFit.SumW2Error(True), would make the error much smaller than when not applying it.
Are you sure the correction is applied ?

If this is the case, I would need a macro and the ROoWorkspace showing this problem

Best

Lorenzo

[jeff]

Hi Lorenzo,

Many thanks for the reply. I think I’ve tracked down the issue. Indeed, you are right - the correction was not actually being applied. Here’s what I somehow missed in the output of the fit that says as much:

It turns out that I am keeping several of the parameters in my fit constant via some lines like this:

If I instead explicitly set the variables constant with:

param = RooRealVar('param',"param", 1., 0., 10.) param.setVal(1.) param.setConstant(True)

then the fitting procedure doesn’t take these parameters into account when determining the correlation matrix and I don’t get the error mentioned above. Now the SumW2Error correction is being applied properly and the uncertainties on the remaining free parameters look great. Sorry this came down to such a silly issue on my end. Thanks again!

Best,
Jeff