Hi all,
I am using iminuit to minimize a negative log likelihood on a toy problem.
I have several question about how to be sure that I am using migrad() correctly.
migrad() outputs explains that the estimated distance to minimum (EDM) is too large in the end, leading to an invalid minimum.
here is the output of migrad :
┌──────────────────────────────────┬──────────────────────────────────────┐
│ FCN = -7532 │ Nfcn = 84 │
│ EDM = 2.23 (Goal: 0.0001) │ │
├───────────────┬──────────────────┼──────────────────────────────────────┤
│INVALID Minimum│ Valid Parameters │ No Parameters at limit │
├───────────────┴──────────────────┼──────────────────────────────────────┤
│ ABOVE EDM threshold (goal x 10) │ Below call limit │
├───────────────┬──────────────────┼───────────┬─────────────┬────────────┤
│ Covariance │ Hesse ok │ Accurate │ Pos. def. │ Not forced │
└───────────────┴──────────────────┴───────────┴─────────────┴────────────┘
┌───┬─────────┬───────────┬───────────┬────────────┬────────────┬─────────┬─────────┬───────┐
│ │ Name │ Value │ Hesse Err │ Minos Err- │ Minos Err+ │ Limit- │ Limit+ │ Fixed │
├───┼─────────┼───────────┼───────────┼────────────┼────────────┼─────────┼─────────┼───────┤
│ 0 │ rescale │ 0.809 │ 0.004 │ │ │ 0.001 │ │ │
│ 1 │ mu │ 0.514 │ 0.028 │ │ │ 0.001 │ │ │
└───┴─────────┴───────────┴───────────┴────────────┴────────────┴─────────┴─────────┴───────┘
┌─────────┬─────────────────────┐
│ │ rescale mu │
├─────────┼─────────────────────┤
│ rescale │ 1.69e-05 -2.04e-05 │
│ mu │ -2.04e-05 0.000768 │
└─────────┴─────────────────────┘
What puzzles me is that the found parameters are very close to the minimum it should find.
So I guess that the minimization reached the minimum but the EDM near the minimum is simply too large.
I join here a contour plot of the negative log likelihood landscape.
According to what I read simply increasing the tolerance should fix it.
But I did not found prescription on how the new tolerance value should be chosen.
If it is too large the migrad() algorithm may stop before reaching the minimum, right ?
So how should I set the tolerance ?
Before this issue I also faced one major difficulty which may be related : due to use of GPU in my process I am limited to float32 precision.
The negative log likelihood (NLL) remains constant if the input of the function differs only by 1e-7.
1e-6 change in the input parameters (rescale or mu) does change the NLL.
According to the documentation setting the the precision
parameter of Minuit to 1e-6 should fix it.
Moreover I set a limit on my parameters to 0.001 because they should be strictly positive.
Auxiliary question : what are the stopping criterion of migrad ?
If the EDM is lower than the tolerance then it stops.
But what are the other criterions ? In my case it stops before reaching a small enough EDM.
I remember reading something in a 1970s paper about if the gradient of the next step is larger than the one of the current step then something happens to fix it.
But I did not understood the details.
I read that Migrad is an implementation of BFGS.
But courses and other tutorials I found about this algorithm stick with one stopping criteria (norm of the gradient lower that predefined tolerance value).
Thanks for reading !
ROOT Version: ROOT-v6-25-01-973-ga470d483db
Platform: Ubuntu
Compiler: Not Provided