Difference of convergence for Minuit2 between a portable and a server

Wile_E_Coyote · February 11, 2022, 2:53pm

You could try to compile it using “-O2 -Wall -Wextra” and closely inspect reported problems (note: usually newer compiler will spot more, and “-O2” or “-O3” is needed here).

BTW. Another thing to check … can it be that you somewhere use “float” instead of “double”? You could also attach your source code for “inspection”.

cmercier · February 11, 2022, 5:46pm

Hello Wile_E_Coyote,

(working with ThierryA)
Thanks, -Wextra allowed me to remove unused stuff, I already had -Wall
-O0, -g, -O2, -O3 gives the exact same result, if on the same computer.
No I did not use float instead of double.

I will try remove everything and clone our git again on both, it is getting crazy.
I need to make some cleaning before attaching the source.
Minuit is taking parameters from another peace of code, which is not stable either.

Thanks

Wile_E_Coyote · February 11, 2022, 5:51pm

BTW. Make sure you are using the “latest” ROOT on both machines (i.e., with the “latest” Minuit2, as I remember, there were bug fixes applied).

cmercier · February 11, 2022, 6:36pm

Hello,
Arg, this is not the case, I downloaded isolated minuit2, Minuit2-5.34.14.tar.gz, and compiled it locally on both computers, could not find standalone minuit 2 more recent, for both debian and fedora.
Any advice ?

Wile_E_Coyote · February 11, 2022, 6:39pm

That’s a question to @Axel and @moneta … I do not know if any fixes are applied to the standalone Minuit2 (but I guess you should be able to get the latest ROOT on both systems).

moneta · February 11, 2022, 7:00pm

Hi,
That tar file is a quite old version. You can take a new version of Minuit2 directly from the ROOT github repository and build it standalone with cmake. See

github.com

root-project/root/blob/master/math/minuit2/README.md

This is the Minuit2 fitter standalone edition, from the [ROOT] toolkit. It uses [CMake] 3.1+ to build.
For information about the Minuit2 fitter, please see the [documentation in ROOT][minuitdoc].

## Source

There are two ways to get Minuit2; you can checkout the [ROOT] source, then just build or use `add_subdirectory` with `<ROOT_SOURCE>/math/minuit2`, or you can get a Minuit2 source distribution which contains all the needed files to build with [CMake]. See [DEVELOP.md] for more information about extracting the source files from [ROOT].


## Building

To build, use the standard [CMake] procedure; on most systems, this looks like:

```bash
mkdir PATH_TO_MINIUT2_BUILD
cd PATH_TO_MINUIT2_BUILD
cmake PATH_TO_MINUIT2_SOURCE
cmake --build .
```

Of course, GUIs, IDEs, etc. that work with [CMake] will work with this package. The standard method of CMake building, with a build directory inside the Minuit2 source directory and using the makefile generator, would look like:

This file has been truncated. show original

If you have any issue building it please let me know

Lorenzo

tappourc · February 13, 2022, 7:23am

Thans Lorenzo and Wild_E_Coyote. We are still investigating.
We also encountered a strange behaviour with edm.
We had the convergence stopping while the criterion was not met, how come? (See below)

Cheers,

ThierryA

Global fit minimizer: Minuit2
MnSeedGenerator: for initial parameters FCN = -524556
MnSeedGenerator: Initial state: - FCN = -524556.4700478 Edm = 1744.1 NCalls = 799
MnSeedGenerator: Negative G2 found - new state: - FCN = -524640.6092162 Edm = 1703.33 NCalls = 4153
VariableMetric: start iterating until Edm is < 0.1
VariableMetric: Initial state - FCN = -524640.6092162 Edm = 1703.33 NCalls = 4153
VariableMetric: Iteration # 1 - FCN = -525582.0514543 Edm = 561.556 NCalls = 4644
VariableMetric: Iteration # 2 - FCN = -525784.2195521 Edm = 54.3211 NCalls = 5092
…
VariableMetric: Iteration # 136 - FCN = -525963.8532168 Edm = 1.2574 NCalls = 62028
VariableMetric: Iteration # 137 - FCN = -525963.8741607 Edm = 0.163729 NCalls = 62445
MnUserParameterState .CovarianceStatus()
3
Fonction minimum

Minuit did successfully converge.

of function calls: 62456

minimum function Value: -525963.8741607
minimum edm: 0.1637290464275

moneta · February 14, 2022, 9:03am

Hi,

Are you still using the old version 5.34 or the new one ?
This is strange, but the logic for the end is rather complex, it is also corrected using some information computed from the covariance matrix. I would need the full print-out, possibly with a more verbose option, to understand exactly what is happening.

Cheers
Lorenzo

tappourc · February 14, 2022, 9:33am

Hi Lorenzo,

We are in the process of using root-minuit2-6.24.06-2.fc34.x86_64.

As for the verbose option, just let me know what to code and the output you may wish to see (I do not code, I check the outputs )

ThierryA

tappourc · February 14, 2022, 10:31am

Hi Lorenzo,

Same output with the new disrib with edm > 0.1. We need to know how verbose we need to be.

Cheers,

ThierryA

moneta · February 14, 2022, 10:57am

Can you please attach the full log you have now, and to get more verbose increase the print level to level 2 or to 3. Are you using the MInuit2Minimizer interface ? If yes, you need to just call Minuit2Minimizer::SetPrintLevel(3)

Lorenzo

tappourc · February 14, 2022, 12:48pm

This is what we use in the *.cpp

MnPrint print;
print.SetLevel(5);

Is that suffiicient?

ThierryA

tappourc · February 14, 2022, 12:56pm

And the file of the output with SetLevel(5)

ThierryA

output_file.txt (1.3 MB)

cmercier · February 14, 2022, 1:14pm

Hi all,
So using the fedora build is ok, and giving identical result than the compiled old one.
On the server, the build looks ok, but then executing seg fault on MnPrint print;
print.SetLevel(3);

Program received signal SIGSEGV, Segmentation fault.
0x00005555555758c3 in std::enable_if<std::_and<std::_not<std::__is_tuple_like >, std::is_move_constructible, std::is_move_assignable >::value, void>::type std::swap(int&, int&) ()
(gdb) where
#0 0x00005555555758c3 in std::enable_if<std::_and<std::_not<std::__is_tuple_like >, std::is_move_constructible, std::is_move_assignable >::value, void>::type std::swap(int&, int&) ()
#1 0x00005555555731c4 in ROOT::Minuit2::MnPrint::SetLevel(int) ()
#2 0x00005555555646b0 in minimize_minimize () at /home/cmercier/progfit_sol1/progfit/src/minimize.cpp:112
…

Am i not using this properly ?
(EDIT: probably becuse any calls segfault, if I comment the print the next call seg fault)
#0 0x0000000000000000 in ?? ()
#1 0x00005555555710e8 in ROOT::Minuit2::MnApplication::operator()(unsigned int, double) ()
#2 0x00005555555646d4 in minimize_minimize () at /home/cmercier/progfit_sol1/progfit/src/minimize.cpp:114

moneta · February 14, 2022, 2:02pm

Hi,
There is probably something wrong in using the right library. Which version are you using it ?
The log seems to indicate you are using an older version.

In the latest version, for setting a debug level in all Minuit2 you should do:

MnPrint::SetGlobalLevel(3);

see ROOT: ROOT::Minuit2::MnPrint Class Reference

Lorenzo

tappourc · February 14, 2022, 8:56pm

Hi Lorenzo,

Here is the full output of the code with the latest Minuit2 version. We still have edm>0.1 but I guess it also use the fractional change of the covariance for stopping.

The version is clone last-stable: 6.24.6

Nextcloud (12.3 Mb)

Just let us know. Thanks,

ThierryA

moneta · February 15, 2022, 8:32am

Hi,
Thank you for the log file. It is true the elm is corrected using the fractional change in the covariance matrix, but this correction seems to be small.
Unfortunatly, I realise now there is a small issue in the printing logic, that makes the message that I was looking (from the VariableMetricBuilder class for not printed.
Can you please just shows me the code on how are you calling the minimization ? Are you using the MnApplication class ?

Thanks

Lorenzo

tappourc · February 15, 2022, 9:17am

Hi Lorenzo,

Here is the code. Just let us know (and flame us at will…)

ThierryA

#include <fstream>
#include <sstream>
#include <iostream>
#include <vector>
#include <string>
#include <cstring>

#include <Minuit2/FunctionMinimum.h>
#include <Minuit2/MnUserParameterState.h>
#include <Minuit2/MnPrint.h>
#ifdef MINIMIZE
#include <Minuit2/MnMinimize.h> // use this one for migrad followed by simplex
#else
#include <Minuit2/MnMigrad.h>  // use this one for migrad
#endif
#include "MMinzer.h"
#include "LikeliAllFcn.h"
#include "LikeliBgFcn.h"
#include "FittedParams.h"

/** namespace ROOT For ROOT software. */
/** namespace ROOT::Minuit2 For the Minuit2 minimizer. */
using namespace ROOT::Minuit2;

static MN2::FCNBase* fcn;

static double* xopt;
static double* err;
static double fopt;
static size_t dim;
static MnUserParameters upar;
static const Fixedness* ffix;
static char** names;
static size_t nvars;

static void minimize_alloc(size_t n)
{
  dim = n;
  xopt = new double[n];
  err = new double[n];
  
  names = new char*[n];
  for (size_t i = 0; i < n; i++)
  {
    names[i] = new char[32]; // 32 to prevent compiler warnings
    
    if(i < NBG_MEMBERS) sprintf(names[i],"BG%ld",i);
    else if (i >= OFF_SPLITTINGS && i < (OFF_SPLITTINGS + NSPLITTINGS) ) sprintf(names[i],"SP%ld",i);
    else if (i == OFF_ANGLES && i <(OFF_ANGLES + NANGLES) ) sprintf(names[i],"AN%ld",i);
    else {
      int freq = (i-OFFSET)/12;
      sprintf(names[i],"%d_%ld", freq, i);
    }
  }
}

static void minimize_free()
{
  for (size_t i = 0; i < dim; i++)
    delete [] names[i];
  delete [] names;

  delete [] err;
  delete [] xopt;
  delete fcn;
}

static void minimize_set(MMinzerMode mode,
		       const double* x0,
		       const Lklhd_Parms* other,
		       const Fixedness* fx,
		       double epsrel)
{
  if (mode == MM_BACKGROUND)
  {
    fcn = new LikeliBgFcn(other);
  }
  else
  {
    fcn = new LikeliAllFcn(other);
  }
  
  for (size_t i = 0; i < dim; i++) {
    upar.Add(names[i], x0[i], epsrel * x0[i]);
  }
  
  ffix = fx;
}

static bool minimize_minimize() 
{
#ifdef MINIMIZE
  MnMinimize minimize(*fcn, upar);
#else
  MnMigrad minimize(*fcn, upar);
#endif

  const int* posfixed = ffix->posfixed;
  int nfixed = ffix->fixed;

  for (int i = 0; i < nfixed; i++) {
    minimize.Fix(posfixed[i]);
  }

  // show all parameter free , making TA believe wrong things
  //  std::cerr << "Parameters before minimizing: " << upar << std::endl;
  
  // keep this comments - if needed to free or fix one param
  // minimize.Fix(10);
  // minimize.Release(10);


  MnPrint::SetGlobalLevel(3);
  
  FunctionMinimum min = minimize(100000, 100); // (max calls, tolerance)

  const MnUserParameterState& params = min.UserState();

  nvars = params.Parameters().VariableParameters();
  for (size_t i = 0; i < dim; i++)
  {
    xopt[i] = params.Value(i);
    err[i] = params.Error(i);
  }
  fopt = min.Fval(); // minimum value

  std::cerr << "MnUserParameterState .CovarianceStatus() " << std::endl;
  std::cerr << params.CovarianceStatus() <<  std::endl;
  std::cerr << "Fonction minimum " << std::endl;
  std::cerr << min << std::endl;
  
  std::cerr << "-----------------" << std::endl;
  // check MnPrint.cxx to know what may be printed 
  std::cerr <<  "Valid         : " << (min.IsValid() ? "yes" : "NO") <<  std::endl;
  std::cerr <<  "Function calls: " << min.NFcn() <<  std::endl;
  std::cerr <<  "Minimum value : " << min.Fval() <<  std::endl;
  std::cerr <<  "Edm           : " << min.Edm()  << std::endl;
  std::cerr <<  "Internal parameters: " << min.Parameters().Vec()  << std::endl;

  if (min.HasValidCovariance())
     std::cerr << "\n  Internal covariance matrix: " << min.Error().Matrix();

  std::cerr << "External parameters: " << min.UserParameters() << std::endl;
  std::cerr << "GlocalCC: " << min.UserState().GlobalCC() << std::endl;

  if (!min.IsValid()) {
    std::cerr << "\n  FunctionMinimum is invalid:"  << std::endl; 
    if (!min.State().IsValid()) std::cerr << "State is invalid"  << std::endl;;
    if (min.IsAboveMaxEdm())    std::cerr << "Edm is above max"  << std::endl;;
    if (min.HasReachedCallLimit())  std::cerr << "Reached call limit"  << std::endl;
   }
  
  if (min.HasMadePosDefCovar() ) std::cerr << "Covar was made pos def" << std::endl;
  if (min.HasAccurateCovar()) std::cerr << "HasAccurateCovar" << std::endl;
  
  return (min.IsValid());

}

double* minimize_x_ary()
{
  return xopt;
}

double* minimize_errors()
{
  return err;
}

static size_t minimize_num_vars()
{
  return nvars;
}

static char** minimize_parmnames()
{
  return names;
}

static double minimize_fmin()
{
  return fopt;
}

static const MMinzer_t MMinzer_type = {
#ifdef MINIMIZE
  "minimize",
#else
  "migrad",
#endif
  minimize_alloc,
  minimize_free,
  minimize_set,
  minimize_minimize,
  minimize_num_vars,
  minimize_x_ary,
  minimize_errors,
  minimize_fmin,
  minimize_parmnames,
};

const MMinzer_t* MMinzer = &MMinzer_type;

tappourc · February 15, 2022, 9:47am

Hi Lorenzo,

We also sent you an e-mail with a private access to our Git.

Thanking you in advance,

ThierryA

moneta · February 15, 2022, 10:16am

Hi,
Thanks for sharing the code. It is clear now to me, the actual tolerance used in the VariableMetricMinimizer (Migrad) is scaled by 0.002. The value you pass in MnMigrad is 100 so the actual tolerance used is 0.2, and what you get then makes sense.
You can decrease then this value if you prefer a smaller tolerance.

You can get a better printout by enable debug only for the VariableMatricBuilder

MnPrint::SetGlobalLevel(1);
minimize.Minimizer().SetPrintLevel(3);
FunctionMinimum min = minimize(100000, 100); // (max calls, tolerance)

Cheers

Lorenzo