FCN value with Fit

marlon.brade · June 9, 2024, 12:05pm

I try to fit the data and found that FCN is so large even though I use NLL. The PDF I used is the RooGeneric bkg_pdf define to be,

RooBreitWigner bw_z("bw_z", "Breit-Wigner PDF z", t, mean_z, width_z);

const char* formula_bkg = " bw_z * pow(1- (t/13000), b_param) * pow( (t/13000), p_Zero*log(pow( t/13000,0)) + p_One*log(pow(t/13000,1)) + p_Two*log(pow(t/13000,2)) + p_Three*log(pow(t/13000,3)) ) " ;

RooGenericPdf bkg_pdf("bkg_pdf", "my custom PDF2", formula_bkg, RooArgSet(t, bw_z, b_param, p_Zero, p_One, p_Two, p_Three));

Below is the complete result.

root [0] 
Processing hepfile.cpp...

 PARAMETER DEFINITIONS:
    NO.   NAME         VALUE      STEP SIZE      LIMITS
     1 b_param      5.00000e+01  1.00000e+01    0.00000e+00  1.00000e+02
     2 p_One       -1.00000e+00  1.00000e+00   -5.00000e+00  5.00000e+00
     3 p_Three      0.00000e+00  3.00000e+00   -1.50000e+01  1.50000e+01
     4 p_Two        2.50000e+00  2.50000e+00   -1.00000e+01  1.50000e+01
 **********
 **    3 **SET ERR         0.5
 **********
 **********
 **    4 **SET PRINT           1
 **********
 **********
 **    5 **SET STR           1
 **********
 NOW USING STRATEGY  1: TRY TO BALANCE SPEED AGAINST RELIABILITY
 **********
 **    6 **MIGRAD        2000           1
 **********
 FIRST CALL TO USER FUNCTION AT NEW START POINT, WITH IFLAG=4.
 START MIGRAD MINIMIZATION.  STRATEGY  1.  CONVERGENCE WHEN EDM .LT. 1.00e-03
 FCN=610037 FROM MIGRAD    STATUS=INITIATE       18 CALLS          19 TOTAL
                     EDM= unknown      STRATEGY= 1      NO ERROR MATRIX       
  EXT PARAMETER               CURRENT GUESS       STEP         FIRST   
  NO.   NAME      VALUE            ERROR          SIZE      DERIVATIVE 
   1  b_param      5.00000e+01   1.00000e+01   2.01358e-01   1.90577e+04
   2  p_One       -1.00000e+00   1.00000e+00   2.05758e-01   4.34136e+05
   3  p_Three      0.00000e+00   3.00000e+00   2.01358e-01   3.98784e+06
   4  p_Two        2.50000e+00   2.50000e+00   2.01358e-01   2.21522e+06
                               ERR DEF= 0.5
 MIGRAD MINIMIZATION HAS CONVERGED.
 MIGRAD WILL VERIFY CONVERGENCE AND ERROR MATRIX.
 COVARIANCE MATRIX CALCULATED SUCCESSFULLY
 FCN=342976 FROM MIGRAD    STATUS=CONVERGED     174 CALLS         175 TOTAL
                     EDM=1.21593e-05    STRATEGY= 1      ERROR MATRIX ACCURATE 
  EXT PARAMETER                                   STEP         FIRST   
  NO.   NAME      VALUE            ERROR          SIZE      DERIVATIVE 
   1  b_param      4.10015e+01   1.78196e+00   3.95474e-03   9.87311e-02
   2  p_One       -1.77602e+00   4.62730e-02   2.32315e-04  -3.18934e+00
   3  p_Three     -5.05085e-01   1.52758e-02   2.41434e-05  -3.06895e+01
   4  p_Two        1.74131e+00   2.29184e-02   4.35137e-05  -1.70279e+01
                               ERR DEF= 0.5
 EXTERNAL ERROR MATRIX.    NDIM=  25    NPAR=  4    ERR DEF=0.5
  3.177e+00  5.844e-03  1.874e-03  2.813e-03 
  5.844e-03  2.141e-03 -3.511e-04 -5.271e-04 
  1.874e-03 -3.511e-04  2.333e-04 -1.690e-04 
  2.813e-03 -5.271e-04 -1.690e-04  5.253e-04 
 PARAMETER  CORRELATION COEFFICIENTS  
       NO.  GLOBAL      1      2      3      4
        1  0.96292   1.000  0.071  0.069  0.069
        2  0.99832   0.071  1.000 -0.497 -0.497
        3  0.99828   0.069 -0.497  1.000 -0.483
        4  0.99828   0.069 -0.497 -0.483  1.000
 **********
 **    7 **SET ERR         0.5
 **********
 **********
 **    8 **SET PRINT           1
 **********
 **********
 **    9 **HESSE        2000
 **********
 COVARIANCE MATRIX CALCULATED SUCCESSFULLY
 FCN=342976 FROM HESSE     STATUS=OK             23 CALLS         198 TOTAL
                     EDM=1.19134e-05    STRATEGY= 1      ERROR MATRIX ACCURATE 
  EXT PARAMETER                                INTERNAL      INTERNAL  
  NO.   NAME      VALUE            ERROR       STEP SIZE       VALUE   
   1  b_param      4.10015e+01   1.76336e+00   1.58189e-04  -2.96064e+00
   2  p_One       -1.77602e+00   2.32050e-01   9.29261e-06  -3.63131e-01
   3  p_Three     -5.05085e-01   7.63489e-02   9.65737e-07  -3.36787e-02
   4  p_Two        1.74131e+00   1.14665e-01   1.74055e-06  -6.07322e-02
                               ERR DEF= 0.5
 EXTERNAL ERROR MATRIX.    NDIM=  25    NPAR=  4    ERR DEF=0.5
  3.111e+00  5.679e-03  1.808e-03  2.805e-03 
  5.679e-03  5.389e-02 -8.955e-03 -1.350e-02 
  1.808e-03 -8.955e-03  5.829e-03 -4.261e-03 
  2.805e-03 -1.350e-02 -4.261e-03  1.315e-02 
 PARAMETER  CORRELATION COEFFICIENTS  
       NO.  GLOBAL      1      2      3      4
        1  0.96212   1.000  0.014  0.013  0.014
        2  0.99993   0.014  1.000 -0.505 -0.507
        3  0.99993   0.013 -0.505  1.000 -0.487
        4  0.99993   0.014 -0.507 -0.487  1.000
[#1] INFO:Minimization -- RooAbsMinimizerFcn::setOptimizeConst: deactivating const optimization
RooRealVar::b_param = 41.0015 +/- 1.76336  L(0 - 100) 
RooRealVar::p_Zero = 0 C  L(-INF - +INF) 
RooRealVar::p_One = -1.77602 +/- 0.23205  L(-5 - 5) 
RooRealVar::p_Two = 1.74131 +/- 0.114665  L(-10 - 15) 
RooRealVar::p_Three = -0.505085 +/- 0.0763489  L(-15 - 15) 


////////////////////////////////////////////////////////////////////////////////////////////////////////////////

Meanwhile when I extend the PDF (bkg) say,

RooRealVar nbkg("nbkg","nbkg", 0, 200000) ;

RooExtendPdf bkg("bkg","bkg", bkg_pdf, nbkg) ;

bkg.fitTo(data_atlas, SumW2Error(false));

The FCN become negatively large (I expect since I used NLL), The Result is given below,

 FIRST CALL TO USER FUNCTION AT NEW START POINT, WITH IFLAG=4.
 START MIGRAD MINIMIZATION.  STRATEGY  1.  CONVERGENCE WHEN EDM .LT. 1.00e-03
 FCN=-17407 FROM MIGRAD    STATUS=INITIATE       20 CALLS          21 TOTAL
                     EDM= unknown      STRATEGY= 1      NO ERROR MATRIX       
  EXT PARAMETER               CURRENT GUESS       STEP         FIRST   
  NO.   NAME      VALUE            ERROR          SIZE      DERIVATIVE 
   1  b_param      5.00000e+01   1.00000e+01   2.01358e-01   1.90678e+04
   2  nbkg         1.00000e+05   2.00000e+04   2.01358e-01   3.68149e+04
   3  p_One       -1.00000e+00   1.00000e+00   2.05758e-01   4.34142e+05
   4  p_Three      0.00000e+00   3.00000e+00   2.01358e-01   3.98781e+06
   5  p_Two        2.50000e+00   2.50000e+00   2.01358e-01   2.21546e+06
                               ERR DEF= 0.5
 MIGRAD MINIMIZATION HAS CONVERGED.
 MIGRAD WILL VERIFY CONVERGENCE AND ERROR MATRIX.
 COVARIANCE MATRIX CALCULATED SUCCESSFULLY
 FCN=-292275 FROM MIGRAD    STATUS=CONVERGED     210 CALLS         211 TOTAL
                     EDM=2.65782e-06    STRATEGY= 1      ERROR MATRIX ACCURATE 
  EXT PARAMETER                                   STEP         FIRST   
  NO.   NAME      VALUE            ERROR          SIZE      DERIVATIVE 
   1  b_param      4.09972e+01   1.78034e+00   3.65013e-03  -1.46869e-01
   2  nbkg         6.31848e+04   2.51365e+02   1.00934e-03  -2.83685e-01
   3  p_One       -1.77646e+00   4.81451e-02   2.14447e-04   2.36361e+00
   4  p_Three     -5.04608e-01   1.58952e-02   2.22855e-05   2.27440e+01
   5  p_Two        1.74084e+00   2.38477e-02   4.01653e-05   1.26194e+01
                               ERR DEF= 0.5
 EXTERNAL ERROR MATRIX.    NDIM=  25    NPAR=  5    ERR DEF=0.5
  3.171e+00  4.726e-07  5.831e-03  1.871e-03  2.808e-03 
  4.726e-07  6.318e+04  4.595e-08  1.474e-08 -4.386e-08 
  5.831e-03  4.595e-08  2.318e-03 -3.806e-04 -5.713e-04 
  1.871e-03  1.474e-08 -3.806e-04  2.527e-04 -1.833e-04 
  2.808e-03 -4.386e-08 -5.713e-04 -1.833e-04  5.687e-04 
 PARAMETER  CORRELATION COEFFICIENTS  
       NO.  GLOBAL      1      2      3      4      5
        1  0.96285   1.000  0.000  0.068  0.066  0.066
        2  0.00000   0.000  1.000  0.000  0.000 -0.000
        3  0.99844   0.068  0.000  1.000 -0.497 -0.498
        4  0.99841   0.066  0.000 -0.497  1.000 -0.483
        5  0.99841   0.066 -0.000 -0.498 -0.483  1.000
 **********
 **    7 **SET ERR         0.5
 **********
 **********
 **    8 **SET PRINT           1
 **********
 **********
 **    9 **HESSE        2500
 **********
 COVARIANCE MATRIX CALCULATED SUCCESSFULLY
 FCN=-292275 FROM HESSE     STATUS=OK             31 CALLS         242 TOTAL
                     EDM=2.65812e-06    STRATEGY= 1      ERROR MATRIX ACCURATE 
  EXT PARAMETER                                INTERNAL      INTERNAL  
  NO.   NAME      VALUE            ERROR       STEP SIZE       VALUE   
   1  b_param      4.09972e+01   1.76321e+00   1.46005e-04  -2.96055e+00
   2  nbkg         6.31848e+04   2.51365e+02   4.03737e-05  -3.77020e-01
   3  p_One       -1.77646e+00   2.39023e-01   8.57786e-06  -3.63227e-01
   4  p_Three     -5.04608e-01   7.91024e-02   8.91418e-07  -3.36469e-02
   5  p_Two        1.74084e+00   1.18287e-01   1.60661e-06  -6.07703e-02
                               ERR DEF= 0.5
 EXTERNAL ERROR MATRIX.    NDIM=  25    NPAR=  5    ERR DEF=0.5
  3.110e+00  4.244e-04  5.704e-03  1.846e-03  2.734e-03 
  4.244e-04  6.318e+04  7.782e-07  2.519e-07  3.730e-07 
  5.704e-03  7.782e-07  5.718e-02 -9.583e-03 -1.420e-02 
  1.846e-03  2.519e-07 -9.583e-03  6.257e-03 -4.589e-03 
  2.734e-03  3.730e-07 -1.420e-02 -4.589e-03  1.399e-02 
 PARAMETER  CORRELATION COEFFICIENTS  
       NO.  GLOBAL      1      2      3      4      5
        1  0.96211   1.000  0.000  0.014  0.013  0.013
        2  0.00000   0.000  1.000  0.000  0.000  0.000
        3  0.99994   0.014  0.000  1.000 -0.507 -0.502
        4  0.99994   0.013  0.000 -0.507  1.000 -0.490
        5  0.99994   0.013  0.000 -0.502 -0.490  1.000
[#1] INFO:Minimization -- RooAbsMinimizerFcn::setOptimizeConst: deactivating const optimization
RooRealVar::b_param = 40.9972 +/- 1.76321  L(0 - 100) 
RooRealVar::p_Zero = 0 C  L(-INF - +INF) 
RooRealVar::p_One = -1.77646 +/- 0.239023  L(-5 - 5) 
RooRealVar::p_Two = 1.74084 +/- 0.118287  L(-10 - 15) 
RooRealVar::p_Three = -0.504608 +/- 0.0791024  L(-15 - 15)

May I know what is happening here ? Why did the FCN become negative when I extended the PDF?

Danilo · June 10, 2024, 7:10am

Hi,

I am sorry you are experiencing this issue.
Your fit is not converging. Perhaps it’s worth simplifying the fit function and increment complexity step by step to understand what you need to change. It does not really look like a problem of ROOT.
Let us know how this goes.

Best,
D

marlon.brade · June 10, 2024, 8:08am

Hello sir thanks for your response and I will try to simplify the fit function. By the way, is your statement that it do not converge is because of the FCN values (probably there are more)? I am actually confused because from Migrad says it converged, EDM is very small, Hesse says status is OK and the parameters do not strongly correlate.

system · June 24, 2024, 8:08am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.