Unbinned likelihood fit with negative weights: question about fit status with MIGRAD and HESSE

Hello everyone,

I am trying to perform an unbinned likelihood fit on multiple weighted (with negative weights) dataset using a double-sided crystal ball function. I would just like to ask something regarding the output of roofit. When I perform the fit, the minimization used are MIGRAD and then HESSE (and when using SumW2Error(kTRUE), another from HESSE). There are times when I fit,

  1. the result from MIGRAD gives STATUS=FAILED with an error matrix that is NOT POS-DEF, and HESSE will give STATUS=NOT POS-DEF as well as the error matrix (and still STATUS=NOT POS-DEF with SumW2Error(kTRUE)); or
  2. when I tried using SumW2Error(kTRUE) when fitting, sometimes the results are (MIGRAD: STATUS=FAILED, ERR MATRIX NOT POS-DEF), (HESSE: STATUS=NOT POS-DEF, ERR MATRIX NOT POS-DEF), (HESSE: STATUS=OK, ERR MATRIX ACCURATE).

For these 2 cases, the summary will give (Status: MIGRAD=4, HESSE=0, HESSE=0). Am I misunderstanding the meaning of these numbers. Also, is it enough to just get a STATUS=OK with the last step (case 2)? The plots doesn’t look bad, but I wanted to make sure everything is fine. Also, I believe these NOT POS-DEF status is dues to having large correlations between the parameters. Can changing the range or the initial value fix this issue? Thanks in advance!

Thanks,
John

PS: Here are the ouputs:

CASE 1:

**********
 **   13 **MIGRAD        3000           1
 **********
 FIRST CALL TO USER FUNCTION AT NEW START POINT, WITH IFLAG=4.
 START MIGRAD MINIMIZATION.  STRATEGY  1.  CONVERGENCE WHEN EDM .LT. 1.00e-03
 FCN=0.225062 FROM MIGRAD    STATUS=INITIATE       22 CALLS          23 TOTAL
                     EDM= unknown      STRATEGY= 1      NO ERROR MATRIX
  EXT PARAMETER               CURRENT GUESS       STEP         FIRST
  NO.   NAME      VALUE            ERROR          SIZE      DERIVATIVE
   1  alpha1       1.00000e+00   5.00000e-01   1.38147e-01  -5.04127e-02
   2  alpha2       2.00000e+00   1.00000e+00   1.72186e-01  -2.64675e-02
   3  mean         1.20000e+02   7.00000e-01   2.01358e-01   9.26646e-03
   4  n1           5.00000e+00   1.00000e+00   2.01358e-01  -1.93529e-03
   5  n2           3.00000e+00   1.00000e+00   2.21072e-01  -2.57137e-03
   6  sigma        2.00000e+00   4.90000e-01   2.06953e-01   4.84354e-02
                               ERR DEF= 0.5
 MIGRAD MINIMIZATION HAS CONVERGED.
 MIGRAD WILL VERIFY CONVERGENCE AND ERROR MATRIX.
 EIGENVALUES OF SECOND-DERIVATIVE MATRIX:
        -1.0085e-01  8.9084e-02  4.8141e-01  9.3052e-01  2.1701e+00  2.4297e+00
 MINUIT WARNING IN HESSE
 ============== MATRIX FORCED POS-DEF BY ADDING 0.103279 TO DIAGONAL.
 FCN=0.214838 FROM HESSE     STATUS=NOT POSDEF     40 CALLS         109 TOTAL
                     EDM=0.000677571    STRATEGY= 1      ERR MATRIX NOT POS-DEF
  EXT PARAMETER                APPROXIMATE        STEP         FIRST
  NO.   NAME      VALUE            ERROR          SIZE      DERIVATIVE
   1  alpha1       1.10541e+00   1.08974e+01   8.24301e-04  -1.55009e-03
   2  alpha2       2.14122e+00   1.08998e+01   1.36928e-03  -1.71863e-03
   3  mean         1.19997e+02   5.22488e+00   8.74004e-04   1.13886e-03
   4  n1           5.71040e+00   6.08209e+00   1.16791e-02  -3.49558e-06
   5  n2           3.54571e+00   5.56272e+00   7.01843e-03  -5.79916e-04
   6  sigma        1.34387e+00   3.06913e+00   1.22710e-03  -1.36066e-03
                               ERR DEF= 0.5
 MINUIT WARNING IN MIGRAD
 ============== Negative diagonal element 1 in Error Matrix
 MINUIT WARNING IN MIGRAD
 ============== 7.11845 added to diagonal of error matrix
 EIGENVALUES OF SECOND-DERIVATIVE MATRIX:
        -3.7417e+00  1.3129e-01  2.6002e-01  8.5668e-01  2.7918e+00  5.7019e+00
 MINUIT WARNING IN MIGRAD
 ============== MATRIX FORCED POS-DEF BY ADDING 3.747401 TO DIAGONAL.
 MIGRAD MINIMIZATION HAS CONVERGED.
 MIGRAD WILL VERIFY CONVERGENCE AND ERROR MATRIX.
 EIGENVALUES OF SECOND-DERIVATIVE MATRIX:
        -6.7841e-02  3.6911e-01  9.2080e-01  1.0670e+00  1.2504e+00  2.4605e+00
 MINUIT WARNING IN HESSE
 ============== MATRIX FORCED POS-DEF BY ADDING 0.070302 TO DIAGONAL.
 MIGRAD TERMINATED WITHOUT CONVERGENCE.
 FCN=0.213212 FROM MIGRAD    STATUS=FAILED        312 CALLS         313 TOTAL
                     EDM=0.00616415    STRATEGY= 1      ERR MATRIX NOT POS-DEF
  EXT PARAMETER                APPROXIMATE        STEP         FIRST
  NO.   NAME      VALUE            ERROR          SIZE      DERIVATIVE
   1  alpha1       1.06188e+00   8.32772e+00   7.86063e-04  -5.73791e-04
   2  alpha2       5.88615e+00   1.24313e+01   6.93316e-03   1.64151e-03
   3  mean         1.20031e+02   4.57556e+00   7.91834e-04   1.40678e-03
   4  n1           7.93949e+00   8.86974e+00   2.94668e-02   6.21134e-05
   5  n2           2.92793e-02   8.61737e+00   1.43119e-02   2.68225e-04
   6  sigma        1.37152e+00   3.54979e+00   8.90093e-04  -2.43375e-03
                               ERR DEF= 0.5
 EXTERNAL ERROR MATRIX.    NDIM=  25    NPAR=  6    ERR DEF=0.5
  6.080e+03 -1.130e+01 -9.616e+02 -2.809e+05 -4.793e+01  4.036e+02
 -1.130e+01  2.135e+02  1.966e+00  4.961e+02 -1.171e+01 -1.431e+00
 -9.616e+02  1.966e+00  1.791e+02  4.378e+04  8.297e+00 -7.201e+01
 -2.809e+05  4.961e+02  4.378e+04  1.308e+07  2.099e+03 -1.796e+04
 -4.793e+01 -1.171e+01  8.297e+00  2.099e+03  1.287e+02 -6.240e+00
  4.036e+02 -1.431e+00 -7.201e+01 -1.796e+04 -6.240e+00  4.347e+01
ERR MATRIX NOT POS-DEF
 PARAMETER  CORRELATION COEFFICIENTS
       NO.  GLOBAL      1      2      3      4      5      6
        1  0.99787   1.000 -0.010 -0.922 -0.996 -0.054  0.785
        2  0.07383  -0.010  1.000  0.010  0.009 -0.071 -0.015
        3  0.93745  -0.922  0.010  1.000  0.905  0.055 -0.816
        4  0.99739  -0.996  0.009  0.905  1.000  0.051 -0.753
        5  0.11259  -0.054 -0.071  0.055  0.051  1.000 -0.083
        6  0.86022   0.785 -0.015 -0.816 -0.753 -0.083  1.000
 ERR MATRIX NOT POS-DEF
 **********
 **   18 **HESSE        3000
 **********
 EIGENVALUES OF SECOND-DERIVATIVE MATRIX:
        -8.6846e-02  2.0290e-01  7.3813e-01  1.0120e+00  1.6239e+00  2.5099e+00
 MINUIT WARNING IN HESSE
 ============== MATRIX FORCED POS-DEF BY ADDING 0.089356 TO DIAGONAL.
 FCN=0.213212 FROM HESSE     STATUS=NOT POSDEF     46 CALLS         359 TOTAL
                     EDM=0.00632081    STRATEGY= 1      ERR MATRIX NOT POS-DEF
  EXT PARAMETER                APPROXIMATE     INTERNAL      INTERNAL
  NO.   NAME      VALUE            ERROR       STEP SIZE       VALUE
   1  alpha1       1.06188e+00   7.76105e+00   1.57213e-04  -1.03217e+00
   2  alpha2       5.88615e+00   1.43088e+01   3.66691e-02  -4.23973e-01
   3  mean         1.20031e+02   4.23793e+00   1.58367e-04   8.82780e-03
   4  n1           7.93949e+00   8.76132e+00   5.89337e-03   6.28458e-01
   5  n2           2.92793e-02   8.80694e+00   2.86238e-03  -1.46252e+00
   6  sigma        1.37152e+00   3.86742e+00   1.78019e-04  -5.01808e-01
                               ERR DEF= 0.5
 EXTERNAL ERROR MATRIX.    NDIM=  25    NPAR=  6    ERR DEF=0.5
  5.950e+03 -8.869e+02 -8.844e+02 -2.758e+05  3.560e+01  4.140e+02
 -8.869e+02  8.426e+02  1.496e+02  3.831e+04 -5.988e+01 -1.353e+02
 -8.844e+02  1.496e+02  1.583e+02  4.029e+04 -6.019e+00 -7.133e+01
 -2.758e+05  3.831e+04  4.029e+04  1.289e+07 -1.541e+03 -1.822e+04
  3.560e+01 -5.988e+01 -6.019e+00 -1.541e+03  1.298e+02  5.327e+00
  4.140e+02 -1.353e+02 -7.133e+01 -1.822e+04  5.327e+00  5.278e+01
ERR MATRIX NOT POS-DEF
 PARAMETER  CORRELATION COEFFICIENTS
       NO.  GLOBAL      1      2      3      4      5      6
        1  0.99786   1.000 -0.396 -0.911 -0.996  0.041  0.739
        2  0.67241  -0.396  1.000  0.410  0.368 -0.181 -0.642
        3  0.93025  -0.911  0.410  1.000  0.892 -0.042 -0.780
        4  0.99739  -0.996  0.368  0.892  1.000 -0.038 -0.699
        5  0.19450   0.041 -0.181 -0.042 -0.038  1.000  0.064
        6  0.88857   0.739 -0.642 -0.780 -0.699  0.064  1.000
 ERR MATRIX NOT POS-DEF
[#1] INFO:Fitting -- RooAbsPdf::fitTo(SigDBCB) Calculating sum-of-weights-squared correction matrix for covariance matrix
 **********
 **   23 **HESSE        3000
 **********
 EIGENVALUES OF SECOND-DERIVATIVE MATRIX:
        -1.7569e-01  6.9899e-01  9.7838e-01  1.0141e+00  1.0599e+00  2.4243e+00
 MINUIT WARNING IN HESSE
 ============== MATRIX FORCED POS-DEF BY ADDING 0.178111 TO DIAGONAL.
 FCN=6.02809e-05 FROM HESSE     STATUS=NOT POSDEF     64 CALLS         423 TOTAL
                     EDM=4.24395e-06    STRATEGY= 1      ERR MATRIX NOT POS-DEF
  EXT PARAMETER                APPROXIMATE     INTERNAL      INTERNAL
  NO.   NAME      VALUE            ERROR       STEP SIZE       VALUE
   1  alpha1       1.06188e+00   7.97707e+00   3.20714e-03  -1.03217e+00
   2  alpha2       5.88615e+00   1.47026e+01   5.00000e-01  -4.23973e-01
   3  mean         1.20031e+02   5.16420e+00   3.23068e-03   8.82780e-03
   4  n1           7.93949e+00   7.87641e+00   5.00000e-01   6.28458e-01
   5  n2           2.92793e-02   7.91140e+00   5.83926e-02  -1.46252e+00
   6  sigma        1.37152e+00   2.75070e+00   3.70421e-01  -5.01808e-01
                               ERR DEF= 0.5
 EXTERNAL ERROR MATRIX.    NDIM=  25    NPAR=  6    ERR DEF=0.5
  2.292e+07  1.256e+04 -2.686e+06 -1.708e+09  3.531e+02  7.921e+05
  1.256e+04  7.901e+05 -4.529e+02 -9.207e+05  8.531e+03  2.144e+03
 -2.686e+06 -4.529e+02  3.985e+05  1.981e+08  1.357e+03 -9.725e+04
 -1.708e+09 -9.207e+05  1.981e+08  1.280e+11 -1.988e+04 -5.835e+07
  3.531e+02  8.531e+03  1.357e+03 -1.988e+04  6.166e+05  1.697e+03
  7.921e+05  2.144e+03 -9.725e+04 -5.835e+07  1.697e+03  5.295e+04
ERR MATRIX NOT POS-DEF
 PARAMETER  CORRELATION COEFFICIENTS
       NO.  GLOBAL      1      2      3      4      5      6
        1  0.99803   1.000  0.003 -0.889 -0.998  0.000  0.719
        2  0.01801   0.003  1.000 -0.001 -0.003  0.012  0.010
        3  0.89826  -0.889 -0.001  1.000  0.877  0.003 -0.670
        4  0.99780  -0.998 -0.003  0.877  1.000 -0.000 -0.709
        5  0.01951   0.000  0.012  0.003 -0.000  1.000  0.009
        6  0.72988   0.719  0.010 -0.670 -0.709  0.009  1.000
 ERR MATRIX NOT POS-DEF
[#1] INFO:Minization -- RooMinuit::optimizeConst: deactivating const optimization
[#1] INFO:InputArguments -- RooAbsData::plotOn(signal) INFO: dataset has non-integer weights, auto-selecting SumW2 errors instead of Poisson errors
[#1] INFO:Plotting -- RooAbsPdf::plotOn(SigDBCB) p.d.f was fitted in range and no explicit plot,norm range was specified, using fit range as default
[#1] INFO:Plotting -- RooAbsPdf::plotOn(SigDBCB) only plotting range 'fit_nll_SigDBCB_signal'
[#1] INFO:Plotting -- RooAbsPdf::plotOn(SigDBCB) p.d.f. curve is normalized using explicit choice of ranges 'fit_nll_SigDBCB_signal'
[#0] ERROR:InputArguments -- RooAbsReal::plotOn(SigDBCB) ERROR: unrecognized command: BinningSpec
  RooFitResult: minimized FCN value: 6.02809e-05, estimated distance to minimum: 4.24395e-06
                covariance matrix quality: Unknown, matrix was externally provided
                Status : MIGRAD=4 HESSE=0 HESSE=0

    Floating Parameter    FinalValue +/-  Error
  --------------------  --------------------------
                alpha1    1.0619e+00 +/-  1.86e+01
                alpha2    5.8861e+00 +/-  2.14e+01
                  mean    1.2003e+02 +/-  8.94e+00
                    n1    7.9395e+00 +/-  2.17e+01
                    n2    2.9279e-02 +/-  9.99e+00
                 sigma    1.3715e+00 +/-  7.55e+00

CASE 2:

**********
 **   13 **MIGRAD        3000           1
 **********
 FIRST CALL TO USER FUNCTION AT NEW START POINT, WITH IFLAG=4.
ERROR evaluating range...
 START MIGRAD MINIMIZATION.  STRATEGY  1.  CONVERGENCE WHEN EDM .LT. 1.00e-03
 FCN=0.112755 FROM MIGRAD    STATUS=INITIATE       24 CALLS          25 TOTAL
                     EDM= unknown      STRATEGY= 1      NO ERROR MATRIX
  EXT PARAMETER               CURRENT GUESS       STEP         FIRST
  NO.   NAME      VALUE            ERROR          SIZE      DERIVATIVE
   1  alpha1       1.00000e+00   5.00000e-01   1.38147e-01   4.90891e-03
   2  alpha2       2.00000e+00   1.00000e+00   1.72186e-01  -2.82151e-03
   3  mean         1.20000e+02   7.00000e-01   2.01358e-01  -2.93563e-03
   4  n1           5.00000e+00   1.00000e+00   2.01358e-01   1.87818e-04
   5  n2           3.00000e+00   1.00000e+00   2.21072e-01  -7.98305e-04
   6  sigma        2.00000e+00   4.90000e-01   2.06953e-01  -4.07225e-03
                               ERR DEF= 0.5
 MIGRAD FAILS TO FIND IMPROVEMENT
 EIGENVALUES OF SECOND-DERIVATIVE MATRIX:
        -7.0874e-01  1.8460e-01  4.4155e-01  1.1050e+00  1.7992e+00  3.1784e+00
 MINUIT WARNING IN HESSE
 ============== MATRIX FORCED POS-DEF BY ADDING 0.711922 TO DIAGONAL.
 FCN=0.113173 FROM HESSE     STATUS=NOT POSDEF     52 CALLS          78 TOTAL
                     EDM=0.00104615    STRATEGY= 1      ERR MATRIX NOT POS-DEF
  EXT PARAMETER                APPROXIMATE        STEP         FIRST
  NO.   NAME      VALUE            ERROR          SIZE      DERIVATIVE
   1  alpha1       1.00000e+00   7.69358e+00   1.40909e-02   4.90900e-03
   2  alpha2       2.00000e+00   1.64459e+01   1.75630e-02  -2.82187e-03
   3  mean         1.20000e+02   3.96195e+00   2.05385e-02  -2.93571e-03
   4  n1           5.00000e+00   7.37959e+00   2.05385e-02   1.87813e-04
   5  n2           3.00000e+00   6.45618e+00   2.25493e-02  -7.98372e-04
   6  sigma        2.00000e+00   3.76048e+00   2.11092e-02  -4.07232e-03
                               ERR DEF= 0.5
 MIGRAD MINIMIZATION HAS CONVERGED.
 MIGRAD WILL VERIFY CONVERGENCE AND ERROR MATRIX.
 EIGENVALUES OF SECOND-DERIVATIVE MATRIX:
        -4.4278e-01  1.9288e-01  5.1112e-01  1.0542e+00  2.0721e+00  2.6125e+00
 MINUIT WARNING IN HESSE
 ============== MATRIX FORCED POS-DEF BY ADDING 0.445397 TO DIAGONAL.
 MIGRAD TERMINATED WITHOUT CONVERGENCE.
 FCN=0.112709 FROM MIGRAD    STATUS=FAILED        221 CALLS         222 TOTAL
                     EDM=0.0428003    STRATEGY= 1      ERR MATRIX NOT POS-DEF
  EXT PARAMETER                APPROXIMATE        STEP         FIRST
  NO.   NAME      VALUE            ERROR          SIZE      DERIVATIVE
   1  alpha1       9.04337e-01   9.95792e+00   1.20944e-03  -3.43442e-04
   2  alpha2       2.45945e+00   1.76351e+01   4.16449e-03  -1.17314e-03
   3  mean         1.20154e+02   4.38016e+00   1.94664e-03   7.02828e-04
   4  n1           7.95811e+00   6.83628e+00   6.99276e-02  -1.18922e-05
   5  n2           6.86998e+00   5.62303e+00   4.92089e-02  -5.69306e-05
   6  sigma        2.24086e+00   3.45516e+00   2.22079e-03   9.89826e-04
                               ERR DEF= 0.5
 EXTERNAL ERROR MATRIX.    NDIM=  25    NPAR=  6    ERR DEF=0.5
  8.414e+02  4.519e+04  4.159e+02  1.235e+04 -8.256e+04  3.343e+03
  4.519e+04  2.658e+06  2.652e+04  8.312e+05 -4.857e+06  1.950e+05
  4.159e+02  2.652e+04  4.040e+02  8.062e+03 -4.844e+04  1.907e+03
  1.235e+04  8.312e+05  8.062e+03  5.074e+05 -1.519e+06  6.138e+04
 -8.256e+04 -4.857e+06 -4.844e+04 -1.519e+06  9.013e+06 -3.563e+05
  3.343e+03  1.950e+05  1.907e+03  6.138e+04 -3.563e+05  1.440e+04
ERR MATRIX NOT POS-DEF
 PARAMETER  CORRELATION COEFFICIENTS
       NO.  GLOBAL      1      2      3      4      5      6
        1  0.97264   1.000  0.956  0.713  0.598 -0.948  0.960
        2  0.99813   0.956  1.000  0.809  0.716 -0.992  0.997
        3  0.85092   0.713  0.809  1.000  0.563 -0.803  0.791
        4  0.79704   0.598  0.716  0.563  1.000 -0.710  0.718
        5  0.99237  -0.948 -0.992 -0.803 -0.710  1.000 -0.989
        6  0.99751   0.960  0.997  0.791  0.718 -0.989  1.000
 ERR MATRIX NOT POS-DEF
 **********
 **   18 **HESSE        3000
 **********
 EIGENVALUES OF SECOND-DERIVATIVE MATRIX:
        -4.3314e-01  1.7137e-01  4.8782e-01  1.0625e+00  2.1101e+00  2.6014e+00
 MINUIT WARNING IN HESSE
 ============== MATRIX FORCED POS-DEF BY ADDING 0.435745 TO DIAGONAL.
 FCN=0.112709 FROM HESSE     STATUS=NOT POSDEF     40 CALLS         262 TOTAL
                     EDM=0.0405129    STRATEGY= 1      ERR MATRIX NOT POS-DEF
  EXT PARAMETER                APPROXIMATE     INTERNAL      INTERNAL
  NO.   NAME      VALUE            ERROR       STEP SIZE       VALUE
   1  alpha1       9.04337e-01   1.03015e+01   2.41889e-04  -1.07465e+00
   2  alpha2       2.45945e+00   1.32684e+01   8.32897e-04  -8.54214e-01
   3  mean         1.20154e+02   4.94125e+00   3.89328e-04   4.40645e-02
   4  n1           7.95811e+00   5.14661e+00   1.39855e-02   6.33070e-01
   5  n2           6.86998e+00   7.39659e+00   9.84177e-03   3.83313e-01
   6  sigma        2.24086e+00   3.82376e+00   4.44159e-04  -1.26518e-01
                               ERR DEF= 0.5
 EXTERNAL ERROR MATRIX.    NDIM=  25    NPAR=  6    ERR DEF=0.5
  8.608e+02  4.494e+04  3.585e+02  1.282e+04 -8.844e+04  3.330e+03
  4.494e+04  2.575e+06  2.262e+04  8.478e+05 -5.068e+06  1.892e+05
  3.585e+02  2.262e+04  3.404e+02  7.210e+03 -4.450e+04  1.622e+03
  1.282e+04  8.478e+05  7.210e+03  5.391e+05 -1.669e+06  6.271e+04
 -8.844e+04 -5.068e+06 -4.450e+04 -1.669e+06  1.012e+07 -3.723e+05
  3.330e+03  1.892e+05  1.622e+03  6.271e+04 -3.723e+05  1.399e+04
ERR MATRIX NOT POS-DEF
 PARAMETER  CORRELATION COEFFICIENTS
       NO.  GLOBAL      1      2      3      4      5      6
        1  0.97302   1.000  0.955  0.662  0.595 -0.948  0.960
        2  0.99812   0.955  1.000  0.764  0.720 -0.993  0.997
        3  0.81855   0.662  0.764  1.000  0.532 -0.758  0.743
        4  0.80793   0.595  0.720  0.532  1.000 -0.715  0.722
        5  0.99315  -0.948 -0.993 -0.758 -0.715  1.000 -0.990
        6  0.99742   0.960  0.997  0.743  0.722 -0.990  1.000
 ERR MATRIX NOT POS-DEF
[#1] INFO:Fitting -- RooAbsPdf::fitTo(SigDBCB) Calculating sum-of-weights-squared correction matrix for covariance matrix
 **********
 **   23 **HESSE        3000
 **********
 MINUIT WARNING IN HESSE
 ============== Negative diagonal element 5 in Error Matrix
 MINUIT WARNING IN HESSE
 ============== 1 added to diagonal of error matrix
 COVARIANCE MATRIX CALCULATED SUCCESSFULLY
 FCN=3.31071e-05 FROM HESSE     STATUS=OK             56 CALLS         318 TOTAL
                     EDM=1.32205e-11    STRATEGY= 1      ERROR MATRIX ACCURATE
  EXT PARAMETER                                INTERNAL      INTERNAL
  NO.   NAME      VALUE            ERROR       STEP SIZE       VALUE
   1  alpha1       9.04337e-01   3.03186e+00   4.93453e-03  -1.07465e+00
   2  alpha2       2.45945e+00   5.52682e+00   5.00000e-01  -8.54214e-01
   3  mean         1.20154e+02   2.94227e+00   7.94229e-03   4.40645e-02
   4  n1           7.95811e+00   3.39203e+00   2.85305e-01   6.33070e-01
   5  n2           6.86998e+00   3.90203e+00   2.00772e-01   3.83313e-01
   6  sigma        2.24086e+00   2.04511e+00   9.06084e-03  -1.26518e-01
                               ERR DEF= 0.5
 EXTERNAL ERROR MATRIX.    NDIM=  25    NPAR=  6    ERR DEF=0.5
  1.275e+01  1.182e-06 -1.922e-04 -6.300e-06  1.613e-07  9.732e-05
  1.182e-06  4.314e+01  1.715e-05  1.123e-08  8.085e-07  2.869e-05
 -1.922e-04  1.715e-05  1.223e+01 -1.218e-06  1.062e-06 -6.906e-05
 -6.300e-06  1.123e-08 -1.218e-06  1.625e+01  1.533e-09  4.747e-07
  1.613e-07  8.085e-07  1.062e-06  1.533e-09  2.150e+01  1.695e-06
  9.732e-05  2.869e-05 -6.906e-05  4.747e-07  1.695e-06  5.907e+00
 PARAMETER  CORRELATION COEFFICIENTS
       NO.  GLOBAL      1      2      3      4      5      6
        1  0.00002   1.000  0.000 -0.000 -0.000  0.000  0.000
        2  0.00000   0.000  1.000  0.000  0.000  0.000  0.000
        3  0.00002  -0.000  0.000  1.000 -0.000  0.000 -0.000
        4  0.00000  -0.000  0.000 -0.000  1.000  0.000  0.000
        5  0.00000   0.000  0.000  0.000  0.000  1.000  0.000
        6  0.00001   0.000  0.000 -0.000  0.000  0.000  1.000
[#1] INFO:Minization -- RooMinuit::optimizeConst: deactivating const optimization
[#1] INFO:InputArguments -- RooAbsData::plotOn(signal) INFO: dataset has non-integer weights, auto-selecting SumW2 errors instead of Poisson errors
[#1] INFO:Plotting -- RooAbsPdf::plotOn(SigDBCB) p.d.f was fitted in range and no explicit plot,norm range was specified, using fit range as default
[#1] INFO:Plotting -- RooAbsPdf::plotOn(SigDBCB) only plotting range 'fit_nll_SigDBCB_signal'
[#1] INFO:Plotting -- RooAbsPdf::plotOn(SigDBCB) p.d.f. curve is normalized using explicit choice of ranges 'fit_nll_SigDBCB_signal'
[#0] ERROR:InputArguments -- RooAbsReal::plotOn(SigDBCB) ERROR: unrecognized command: BinningSpec

  RooFitResult: minimized FCN value: 3.31071e-05, estimated distance to minimum: 1.32205e-11
                covariance matrix quality: Unknown, matrix was externally provided
                Status : MIGRAD=4 HESSE=0 HESSE=0

    Floating Parameter    FinalValue +/-  Error
  --------------------  --------------------------
                alpha1    9.0434e-01 +/-  5.18e+01
                alpha2    2.4595e+00 +/-  6.78e+01
                  mean    1.2015e+02 +/-  1.97e+01
                    n1    7.9581e+00 +/-  1.91e+01
                    n2    6.8700e+00 +/-  3.76e+01
                 sigma    2.2409e+00 +/-  1.95e+01

Hello @jvillamo,

I took the liberty to edit your post to make the outputs more easily readable. I believe that you have two problems:

  • Large correlations: With large correlations, you can easily get fit failures and estimating the error matrix may be impossible. To disentangle this from the second case, I suggest to reparametrise the fit model.
  • Negative event weights: I am not sure that a likelihood fit with negative weights will work reliably. Is the sum of weights very positive? I.e., do the events with negative weight have a negligible contribution?
    You should also note that with SumW2, the weights of all events are treated as positive weights. This may distort the distributions if the negative weights are needed to get the shapes right.

For the correlation you have to check what’s causing it, but you can usually solve it similar to this example:
Let’s say you have two parameters, and they express almost the same thing, a scale factor for the total number of events:
nEvent = a * n_predicted + b * m_predicted
Let’s say that m/n_predicted is obtained from some external models, and a, b are parameters that express a normalisation. These two will be fully anticorrelated, and you will never get reasonable errors for them, because each one can be used to change the number of predicted events to match nEvent measured in data.
You can reparametrise in the following way:
nEvent = a * n_predicted + a * b * m_predicted
Now, a is a total scale factor for both samples, and b is an uncertainty parameter that lets a * m_predicted wiggle a bit. To make sure that it doesn’t wiggle too much, you constrain it using a Gaussian(b | 1, <sigma>), i.e., you constrain it to 1 with a sigma of your choice. This will make the correlation go away.

1 Like

Hello @StephanH,

Thanks for taking time to edit my post and reply. What I really do is a signal modeling with very few negative weights. But the signal yield (sum of weights) I am able to get is positive. I believe events with negative weights are negligible in my fits, but when I checked the chi-squared over d.o.f. (using frame->chiSquare(nParams)), some of the values I get are extremely large, even reaching as high as 9000. I am not so sure, but I do suspect that these extreme chi2 values may be due to negative weights. For the case of the error, can I turn off the SumW2Error even if I am fitting a weighted dataset?

Also, I found from this presentation that roofit considers the negative weights in an unbinned likelihood fit.

Thanks again!

Ok, if the negative weights are small in comparison to the positive, try to reparametrise such that the correlations go away.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.