How the derivatives are obtained

Javier · May 31, 2017, 9:34am

Dear experts,

I have a quite basic question: when I use a minimizer to minimize a likelihood, how to know how the derivatives are computed (inside the minimize ) ?

Many thanks!

Javier

moneta · May 31, 2017, 12:39pm

Hi,

Hen using a minimiser like Minuit to find the minimum of the negative log-likelihood functions, the derivatives are computed numerically using finite differences by the Minuit program. The step sizes used in t he derivative calculations are adapted automatically depending on the function.
The algorithm computing the derivative is robust and working quite well for a large variety of functions.
It has been used for more than 40 years in a large variety of problems,.

Lorenzo

Javier · May 31, 2017, 12:43pm

Dear Lorenzo.

Many thanks for the reply! Regarding " The step sizes used in the derivative calculations are adapted automatically depending on the function", could you say a bit more ? How the steps are chosen for different functions ?

Thanks,
Javier

moneta · May 31, 2017, 1:41pm

Hi,

It is an iterative algorithm trying to decrease the steps, within the allowed numerical error, until the computed derivative is stable.
You can find the implementation here

github.com

root-project/root/blob/master/math/minuit2/src/Numerical2PGradientCalculator.cxx#L72


double fval = Fcn()(par);


MinimumParameters minpars = MinimumParameters(par, fval);


return (*this)(minpars);


}






FunctionGradient Numerical2PGradientCalculator::operator()(const MinimumParameters& par, const FunctionGradient& Gradient) const {
// calculate numerical gradient from MinimumParameters object
// the algorithm takes correctly care when the gradient is approximatly zero


//    std::cout<<"########### Numerical2PDerivative"<<std::endl;
//    std::cout<<"initial grd: "<<Gradient.Grad()<<std::endl;
//    std::cout<<"position: "<<par.Vec()<<std::endl;


assert(par.IsValid());

Lorenzo