Hello,
I am currently trying to use the TFractionFitter to extract signal and background components from a pseudo data histogram. But somehow, the pull distributions have a sigma, which is less than one, thus showing that the errors are bigger than expected.
I tried to make the fit easier by using two Gaus functions, but still the same problem remains.
So in my script I produce two Gaus functions to create signal and background histograms, where I also draw pseudo data from the histograms for the fit.fit.cpp (2.8 KB)
What am I doing wrong?
Thanks a lot for your help.
Hi,
It is maybe related to this note about biased uncertainties which is linked in the TFractionFitter documentation,
see https://root.cern.ch/doc/master/classTFractionFitter.html
and Nappi’s note about biased uncertainties: https://arxiv.org/abs/0803.2711
it needs to be investigate further.
Lorenzo
Hi,
thanks for your fast reply. I will have a look at the paper.
Cheers
Hi,
After more investigation, I can confirm you that what you observe its exactly what described in the Nappi’s paper. If your look at figure 1 , you see that for a fraction around 0.5 you overestimate the error by around 30%. If you increase the fraction, for example to 0.9 in your script, you will see that one fraction (the one around 0.1) will have the correct error, while the other one will over-estimate it by 70%. This can early be showed in your example, bu changing the number of MC events for one source .
Furthermore, the correlation between the two parameters is wrong .TFactionFitter returns almost zero correlation, while it should be instead 1, since in the case of two fractions p1+p2 = 1 by definition.
I think TFractionFitter errors are a good approximation when you have a large number of fractions and all of them are small, so you don;t have a dominant contributions. I would need to update this in the documentation.
Best regards
Lorenzo
Hi Lorenzo,
thank you so much for your investigation.
Do you have a recommendation for me, which fitter would correctly propagate the errors in my case?
Cheers
Hi,
You will have to write the likelihood yourself as suggested in Nappi’s paper or you can use RooFit, and/or the HistFactory to help you building the model.
In the first case it will be complicated to add and that in the fit all statistical uncertainties on each MC sample. The problem simplify a lot if you consider only a global MC uncertainty in the sum.
Otherwise, another possibility is to correct the errors using error propagation and the covariance matrix, as it is suggested at the end of Nappi’s paper. I have not tried that and in case it works we could eventually add it directly in TFractionFitter.
Best regards
Lorenzo
Hi Lorenzo,
I guess I will have a go and try to correct the errors using error propagation and the covariance matrix myself then.
Thanks for all the help!
Hi Lorenzo,
just to give you an update, it seems to work by correcting the errors using error propagation and the covariance matrix.
Cheers
Hi,
This is good, then we can add this fix in TFractionFitter directly
Cheers
Lorenzo
Hi Lorenzo,
I just wanted to let you know that I tried varying the fractions of my MC compounds and I see again that the error is not correct even using the error propagation proposed in the paper.
Especially, when the fractions are close to unity or zero.
Cheers
Hi,
I think it is expected that the correction will not work very well when the fractions are close to zero or 1, since error propagation is a first order approximation, and at the borders non-linearity are larger.
In addition at the border errors returned by Minuit using Hessian approximations are not correct anymore.
I expected however that the correction is still an improvement with respect to not applying it. Is this not the case ?
Cheers,
Lorenzo
Hi Lorenzo,
yes, that is true. I guess I will use RooFit instead then.
It is also true that there is an improvement for the error when using error propagation.
Cheers
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.