No idea.
Could you check the output, specifically the fit status? Maybe the fit fails and doesn’t event start calculating the covariance matrix.
Hi, @StephanH
I checked the status. It returns to me a single digit 4.
gr = new TGraphErrors(data_counter,x,y,ex,ey);
TFitResultPtr fit_result=gr->Fit(yield,"S");
Int_t fitStatus = fit_result;
cout<<"fitStatus="<<fitStatus<<endl;
And the document here https://root.cern.ch/doc/master/classTMinuit.html#ab48dd4b48800edc090396e35cb465fb9
tells me that: * 4: abnormal termination (e.g., MIGRAD not converged).
So, in your opinion, what should I do? (BTW, the fact that the fit curve and fit parameters are fine is very suspicious to me.)
I would say make the fit verbose by adding “V” to the options. Then, start reading the output of Minuit. Watch out for things like
- Parameters at limit
- High correlations
- Wrong step sizes
Sometimes, the shape of the likelihood function is just not well behaved because of e.g. highly correlated parameters or wrong data errors. Could it be that the errors of the data points are somehow unreasonable?
Dear StephanH @StephanH :
Thanks. Good news: after using the verbose mode “V”, it returns to me some EXTERNAL ERROR MATRIX, which seems to be the covariance matrix that I always want (see the terminal report below).
But still the MIGRAD thing didn’t converge, I’m not sure how that happened when everything “seems” to be fine (the fit result seems to agree with my data; and it actually get some reasonable covariance matrix, and just doesn’t want to share it with me, when I print it.).
Now I’m not sure if it’s legitimate to use the EXTERNAL MATRIX as my covariance matrix after all its error-report on MIGRAD divergence.
The following is the terminal report, starting from the second last fit status report:
FCN=38.2639 FROM MIGRAD STATUS=PROGRESS 1604 CALLS 1605 TOTAL
EDM=0.00628792 STRATEGY= 1 ERROR MATRIX UNCERTAINTY 20.7 per cent
EXT PARAMETER APPROXIMATE STEP FIRST
NO. NAME VALUE ERROR SIZE DERIVATIVE
1 p0 2.96631e-01 9.40791e-02 3.91372e-03 -1.12405e+02
2 p1 3.70289e-01 2.53365e-02 -1.98576e-03 -2.65629e+01
3 p2 8.30575e-03 1.00820e-02 -1.95435e-02 -3.06363e+00
4 p3 1.23265e+00 3.75070e-02 -3.36459e-03 -1.39326e+02
5 p4 1.43869e+01 1.01233e-01 -1.43233e-03 9.98044e+01
CALL LIMIT EXCEEDED IN MIGRAD.
MIGRAD TERMINATED WITHOUT CONVERGENCE.
FCN=38.2478 FROM MIGRAD STATUS=CALL LIMIT 1630 CALLS 1631 TOTAL
EDM=0.00988881 STRATEGY= 1 ERROR MATRIX UNCERTAINTY 14.5 per cent
EXT PARAMETER APPROXIMATE STEP FIRST
NO. NAME VALUE ERROR SIZE DERIVATIVE
1 p0 3.02999e-01 1.08434e-01 1.31306e-03 -2.14674e+02
2 p1 3.72073e-01 3.38375e-02 1.20516e-03 -3.40276e+01
3 p2 8.84789e-03 1.12145e-02 -5.94815e-03 3.18160e+00
4 p3 1.22696e+00 6.50898e-02 -1.64670e-03 -3.91608e+02
5 p4 1.43640e+01 2.63856e-01 -8.94469e-04 3.39474e+02
EXTERNAL ERROR MATRIX. NDIM= 25 NPAR= 5 ERR DEF=1
1.176e-02 -1.636e-03 -1.213e-03 -6.063e-03 -1.414e-02
-1.636e-03 1.145e-03 2.077e-04 -1.446e-04 -4.736e-03
-1.213e-03 2.077e-04 1.270e-04 5.821e-04 1.162e-03
-6.063e-03 -1.446e-04 5.821e-04 4.238e-03 1.493e-02
-1.414e-02 -4.736e-03 1.162e-03 1.493e-02 6.963e-02
ERR MATRIX APPROXIMATE
PARAMETER CORRELATION COEFFICIENTS
NO. GLOBAL 1 2 3 4 5
1 1.00000 1.000 -0.446 -0.993 -0.859 -0.494
2 0.99984 -0.446 1.000 0.545 -0.066 -0.530
3 0.99989 -0.993 0.545 1.000 0.793 0.391
4 1.00000 -0.859 -0.066 0.793 1.000 0.869
5 0.99999 -0.494 -0.530 0.391 0.869 1.000
ERR MATRIX APPROXIMATE
Info in <TMinuitMinimizer::Minimize>: Finished to run MIGRAD - status 4
Warning in <Fit>: Abnormal termination of minimization.
FCN=38.2478 FROM MIGRAD STATUS=CALL LIMIT 1630 CALLS 1631 TOTAL
EDM=0.00988881 STRATEGY= 1 ERROR MATRIX UNCERTAINTY 14.5 per cent
EXT PARAMETER PARABOLIC MINOS ERRORS
NO. NAME VALUE ERROR NEGATIVE POSITIVE
1 p0 3.02999e-01 1.08434e-01
2 p1 3.72073e-01 3.38375e-02
3 p2 8.84789e-03 1.12145e-02
4 p3 1.22696e+00 6.50898e-02
5 p4 1.43640e+01 2.63856e-01
i=0 (ERR, ERR^2)=(0.108434,0.011758).
i=1 (ERR, ERR^2)=(0.0338375,0.00114498).
i=2 (ERR, ERR^2)=(0.0112145,0.000125764).
i=3 (ERR, ERR^2)=(0.0650898,0.00423668).
i=4 (ERR, ERR^2)=(0.263856,0.0696199).
covariance matrix is
5x5 matrix is as follows
| 0 | 1 | 2 | 3 | 4 |
----------------------------------------------------------------------
0 | 0 0 0 0 0
1 | 0 0 0 0 0
2 | 0 0 0 0 0
3 | 0 0 0 0 0
4 | 0 0 0 0 0
correlaton matrix is:
5x5 matrix is as follows
| 0 | 1 | 2 | 3 | 4 |
----------------------------------------------------------------------
0 | 0 0 0 0 0
1 | 0 0 0 0 0
2 | 0 0 0 0 0
3 | 0 0 0 0 0
4 | 0 0 0 0 0
Hello,
MINUIT terminates the minimisation because the number of function calls has been limited to 1630. There’s two possibilities:
- Allowing for more calls / reducing the required precision / choosing fit strategy 2 (robustness) will let the minimisation finish.
- The likelihood function has a weird shape in the minimum (weird data events, outliers, correlations between parameters, fit model that cannot describe the data), and it will infinitely err around the minimum without ever converging. This is the bad case, where I cannot help easily.
The first case can be tested, though:
EDIT I realise that the first answer was only for RooFit. I will look up how to do this in root shortly.
Own fitter ROOT style:
Foption_t fitOption;
ROOT::Fit::FitOptionsMake(ROOT::Fit::kGraph, "<options>", fitOption);
// create range and minimizer options with default values
ROOT::Fit::DataRange range(minRange, maxRange);
ROOT::Math::MinimizerOptions minOption;
minOption.SetMaxIterations(xxx);
minOption.SetMaxFunctionCalls(xxx);
minOption.SetPrecision(xxx);
minOption.SetStrategy(xxx);
auto fitResult = ROOT::Fit::FitObject(<theGraph>, <fitFunction>, fitOption , minOption, "", range);
You can check the documentation for more on minimiserOptions.
The fit options "<option>", which I left empty, are the same as the options in TGraph::Fit().
Own minimiser RooFit style:
RooAbsReal *nll = model.createNLL(*data);
// Minimize likelihood w.r.t all parameters before making plots
RooMinimizer minimiser(*nll);
// Run gradient descent
minimiser.migrad();
// Estimate erros:
minimiser.hesse();
- and before you start migrad(), use these functions to allow for more calls:
void setMaxIterations(Int_t n) ;
void setMaxFunctionCalls(Int_t n) ;
You can also change the precision you want to reach, and the strategy (i.e. more aggressive or conservative):
https://root.cern.ch/doc/master/classRooMinimizer.html#a4fad943696aa3bca85e8d834b5372771
Thanks. Haha!! I have done it successfully with your guidance. This fit is an important in my research. ^. ^
Very good!
Would you maybe say what the solution was? Maybe somebody will have a similar problem in the future.
