EDM and weighted fits with RooFit

Hi,
I have one quick question on the EDM and weighted fits with RooFit.
I paste here the final output of my fit.

As you can see there is a step where the EDM is:
EDM=8.76222e-06
which is totally fine.

The next step is: "[#1] INFO:Fitting – RooAbsPdf::fitTo(TotalPDFq2Bin_7) Calculating sum-of-weights-squared correction matrix for covariance matrix"
After this step, the values of the parameters are same as the previous step, but the EDM become:
EDM=483.198

Is there an obvious reason for that ?
Do you have any suggestion to improve or debug this problem ?
By the way, the fit look really good by eye.

Thank you very much.

  • Mauro.

** 13 **MIGRAD 2000 1


FIRST CALL TO USER FUNCTION AT NEW START POINT, WITH IFLAG=4.
START MIGRAD MINIMIZATION. STRATEGY 1. CONVERGENCE WHEN EDM .LT. 1.00e-03
FCN=-81661.9 FROM MIGRAD STATUS=INITIATE 16 CALLS 17 TOTAL
EDM= unknown STRATEGY= 1 NO ERROR MATRIX
EXT PARAMETER CURRENT GUESS STEP FIRST
NO. NAME VALUE ERROR SIZE DERIVATIVE
1 AfbS 0.00000e+00 2.00000e-01 2.01358e-01 -4.65480e+03
2 FlS 5.00000e-01 1.00000e-01 2.01358e-01 1.56026e+03
3 nBkgComb 3.00000e+02 1.00000e+00 1.00000e+00 -1.24390e-01
4 nSig 8.00000e+03 1.00000e+00 1.00000e+00 -1.66585e-01
ERR DEF= 0.5
MIGRAD MINIMIZATION HAS CONVERGED.
MIGRAD WILL VERIFY CONVERGENCE AND ERROR MATRIX.
COVARIANCE MATRIX CALCULATED SUCCESSFULLY
FCN=-82773.2 FROM MIGRAD STATUS=CONVERGED 95 CALLS 96 TOTAL
EDM=8.77042e-06 STRATEGY= 1 ERROR MATRIX ACCURATE
EXT PARAMETER STEP FIRST
NO. NAME VALUE ERROR SIZE DERIVATIVE
1 AfbS 3.40431e-01 7.80899e-03 1.60258e-03 2.20180e-01
2 FlS 3.40332e-01 6.94907e-03 2.84704e-03 4.87964e-03
3 nBkgComb 2.55290e+02 3.32030e+01 6.36195e+00 -7.86336e-05
4 nSig 9.41465e+03 1.01268e+02 1.94753e+01 -3.98627e-06
ERR DEF= 0.5
EXTERNAL ERROR MATRIX. NDIM= 25 NPAR= 4 ERR DEF=0.5
6.098e-05 -1.151e-05 2.787e-02 -2.779e-02
-1.151e-05 4.829e-05 -1.518e-04 1.477e-04
2.787e-02 -1.518e-04 1.102e+03 -8.429e+02
-2.779e-02 1.477e-04 -8.429e+02 1.026e+04
PARAMETER CORRELATION COEFFICIENTS
NO. GLOBAL 1 2 3 4
1 0.23780 1.000 -0.212 0.107 -0.035
2 0.21320 -0.212 1.000 -0.001 0.000
3 0.27023 0.107 -0.001 1.000 -0.251
4 0.25082 -0.035 0.000 -0.251 1.000


** 18 **HESSE 2000


COVARIANCE MATRIX CALCULATED SUCCESSFULLY
FCN=-82773.2 FROM HESSE STATUS=OK 23 CALLS 119 TOTAL
EDM=8.76222e-06 STRATEGY= 1 ERROR MATRIX ACCURATE
EXT PARAMETER INTERNAL INTERNAL
NO. NAME VALUE ERROR STEP SIZE VALUE
1 AfbS 3.40431e-01 7.80353e-03 6.41034e-05 3.47375e-01
2 FlS 3.40332e-01 6.94421e-03 1.13882e-04 -3.25029e-01
3 nBkgComb 2.55290e+02 3.32161e+01 2.54478e-01 2.55290e+02
4 nSig 9.41465e+03 1.01304e+02 7.79013e-01 9.41465e+03
ERR DEF= 0.5
EXTERNAL ERROR MATRIX. NDIM= 25 NPAR= 4 ERR DEF=0.5
6.090e-05 -1.132e-05 2.785e-02 -2.785e-02
-1.132e-05 4.823e-05 -1.552e-04 1.554e-04
2.785e-02 -1.552e-04 1.103e+03 -8.479e+02
-2.785e-02 1.554e-04 -8.479e+02 1.026e+04
PARAMETER CORRELATION COEFFICIENTS
NO. GLOBAL 1 2 3 4
1 0.23499 1.000 -0.209 0.107 -0.035
2 0.21004 -0.209 1.000 -0.001 0.000
3 0.27138 0.107 -0.001 1.000 -0.252
4 0.25213 -0.035 0.000 -0.252 1.000
[#1] INFO:Fitting – RooAbsPdf::fitTo(TotalPDFq2Bin_7) Calculating sum-of-weights-squared correction matrix for covariance matrix


** 23 **HESSE 2000


COVARIANCE MATRIX CALCULATED SUCCESSFULLY
FCN=-92191.8 FROM HESSE STATUS=OK 29 CALLS 148 TOTAL
EDM=483.198 STRATEGY= 1 ERROR MATRIX ACCURATE
EXT PARAMETER INTERNAL INTERNAL
NO. NAME VALUE ERROR STEP SIZE VALUE
1 AfbS 3.40431e-01 7.77936e-03 1.28207e-05 3.47375e-01
2 FlS 3.40332e-01 6.69576e-03 2.32319e-03 -3.25029e-01
3 nBkgComb 2.55290e+02 3.23374e+01 5.19135e+00 2.55290e+02
4 nSig 9.41465e+03 9.65032e+01 1.58919e+01 9.41465e+03
ERR DEF= 0.5
EXTERNAL ERROR MATRIX. NDIM= 25 NPAR= 4 ERR DEF=0.5
6.052e-05 -1.010e-05 2.892e-02 -2.839e-02
-1.010e-05 4.484e-05 -1.071e-02 1.051e-02
2.892e-02 -1.071e-02 1.046e+03 -7.945e+02
-2.839e-02 1.051e-02 -7.945e+02 9.313e+03
PARAMETER CORRELATION COEFFICIENTS
NO. GLOBAL 1 2 3 4
1 0.22082 1.000 -0.194 0.115 -0.038
2 0.19576 -0.194 1.000 -0.049 0.016
3 0.27671 0.115 -0.049 1.000 -0.255
4 0.25474 -0.038 0.016 -0.255 1.000

Hi,

I think this EDM is not correctly computed when the sum of square of the weight correction is done. What is computed in this case is an Hessian matrix using the square of the weights at the minimum found with the normal likelihood.
I guess, when computing the EDM ( EDM = 1/2 * g H g, where g is the gradient and H is the
hessian matrix), the wrong g is used. You can probably ignore that value

Best Regards

Lorenzo