Setting fixed parameter step sizes for ROOT::Fit::Fitter

Roofit and RooStats
1

Hi all,

I was attempting to use a ROOT::Fit::Fitter to automatically optimize 8 cut parameter. This is supposed to be done by maximizing the significance of my signal or minimizing the inverse significance.

Therefore I wrote a function “GetInverseSignificance” which calculates the significance of my signal for a given set of cut parameters. Unfortunatly one function call takes a few seconds to calculate the result, so with too many function calls the whole proceedure will take forever.

Since my measured cut parameters have a limited resolution it is not necessary to vary them up to a precision of 10^-something. In fact for most of the parameters a precision of ±1 is more than sufficient.

Unfortunatly I don’t know how to tell the minimizer to restrict to this. I thought this would be possible using the step size of the parameters, but somehow the fitter seems to be ignoring them (See my example). Furthermore, I was not able to find a good documentation of what all the parameters of the different classes stand for since the documentation of the Class References often consists of only a few words, so I was wondering wether there is a more detailed documentation somewhere.

Here is my minimal example (I restrict to 2 parameters to keep it small, actually I use 8.):

Int_t cnt = 0;
Double_t GetInverseSignificance(const Double_t* pars) {
    // **************************************************
    // Extract the signal and calculate the significance for the parameters in pars
    // **************************************************

    cout << Form("Function call %d with result %.2f", ++cnt, Significance) << endl;
    cout << Form("Values: %.0f, %.2f", pars[0], pars[1]) << endl << endl;

    return 1. / Significance;
}

std::auto_ptr<ROOT::Fit::Fitter> fitter(new ROOT::Fit::Fitter());
ROOT::Fit::FitConfig& fitConfig = fitter->Config();
fitConfig.SetParamsSettings(2, new Double_t[2]);

std::vector<ROOT::Fit::ParameterSettings>& parSettings = fitConfig.ParamsSettings();
parSettings[0].SetName("Par1");
parSettings[0].SetValue(400.);
parSettings[0].SetLimits(50., 1000.);
parSettings[0].SetStepSize(50.);

parSettings[1].SetName("Par2");
parSettings[1].SetValue(3.);
parSettings[1].SetLimits(0.1, 10.);
parSettings[1].SetStepSize(0.1);

ROOT::Math::MinimizerOptions minOpt;
minOpt.SetMaxIterations(1);
minOpt.SetPrecision(1.);
minOpt.SetPrintLevel(3);
fitConfig.SetMinimizerOptions(minOpt);

fitter->FitFCN(2, GetInverseSignificance);

From this I get a result like this:

**********
 **    1 **SET PRINT           2
 **********
 **********
 **    2 **SET NOGRAD
 **********
 PARAMETER DEFINITIONS:
    NO.   NAME         VALUE      STEP SIZE      LIMITS
     1 Par1         4.00000e+02  5.00000e+01    5.00000e+01  1.00000e+03
     2 Par2         3.00000e+00  1.00000e-01    1.00000e-01  1.00000e+01
**********
 **    3 **SET ERR           1
 **********
 **********
 **    4 **SET PRINT           2
 **********
 **********
 **    5 **SET EPS           1
 **********
 FLOATING-POINT NUMBERS ASSUMED ACCURATE TO 8.88178e-16
 **********
 **    6 **SET STR           1
 **********
 NOW USING STRATEGY  1: TRY TO BALANCE SPEED AGAINST RELIABILITY
 **********
 **    7 **MIGRAD        2120        0.01
 **********
 FIRST CALL TO USER FUNCTION AT NEW START POINT, WITH IFLAG=4.
Function call 1 with result some value
Values: 400, 3.00

 START MIGRAD MINIMIZATION.  STRATEGY  1.  CONVERGENCE WHEN EDM .LT. 1.00e-05
Function call 2 with result some value
Values: 401, 3.00

Function call 3 with result some value
Values: 399, 3.00

Function call 4 with result some value
Values: 403, 3.00

Function call 5 with result some value
Values: 397, 3.00

Function call 6 with result some value
Values: 418, 3.00

Function call 7 with result some value
Values: 382, 3.00

Function call 8 with result some value
Values: 400, 3.00

Function call 9 with result some value
Values: 400, 3.00

Function call 10 with result some value
Values: 400, 3.01
...

So all the settings are applied correctly but although the step size of parameter 1 is set to 50, the minimizer first varys it by 1, then by 3 and then by 18. The step size of parameter 2 is set to 0.1, but the minimizer starts varying it by 0.01 and so on. Furthermore the values are not being changed at all between call 8 and 9 so I would assume the are being changed below the printing precision.

I 'm not sure wether I have a massive misconception with the step size value, but to me the behaviour seems strange.

I would be really happy if you can help me.

2

I guess @moneta can help you.

3

Hi @Triple_S,

the step sizes you set are only starting values. They are adaptive, and will change over time. Further, the significance can be pretty ugly, as it can oscillate wildly. In that case, the fitter will have a hard time to find the optimal value. Do you have a regularisation term, e.g. B’ = B +10?

4

Hi @StephanH,

thanks for your answer. I already read that by default the step sizes are just estimates and not fixed, but I assumed there would be some setting I don’t know to force the minimizer to restrict to the set step sizes. I assume there is no such setting.

About the fluctuations: I know that this is a huge problem. I have so to say three cases (Two with very clear siagnals (Significance ~ 1000) and one with an unclear signal (Significance ~ 5)) and my idea was to test the method with the clear signals and then apply it to the unclear case. Anyhow, the statistical fluctuation will still be a problem and this was one of the reasons I wanted to restrict the step sizes, cause the more cases you try out the more likely it becomes to have a high statistic fluctuation. Furthermore, it does’nt make sense to scan a parameter with a systematic error of ~1 with a step size of 0.1.

About the regularisation term I don’t know anything. I developed my method purely by going through the source code of the Fitting algorithm of TH1s and TGraphs, cause as I already wrote I could not find any useful documentation! In the meantime I switched from the Fitter to a Minimizer, cause I found this example (https://root.cern.ch/numerical-minimization) which I think is nice, but unfortunatly only scratches the surface of whats possible. For example there is still no explanation what the “Precision” and the “Tolerance” are. From the source code, I think the “Tolerance” is some sort of convergence criteria, but about the “Precision” I still know nothing.

Thanks and all the best,
Triple_S

5

That’s not really true. If you try fewer cases, you are just looking at fewer points, but these will also have high fluctuations, and are therefore not really trustworthy.
That’s why I proposed a regularisation term. It is meant to regularise the significance. I don’t know which formula you are using, be it AMS or the simple S/sqrt(B), but you can easily regularise by using something like (S)/sqrt(B+10) (or the equivalent for AMS). If you run into the regime of a low number of B, the +10 will keep the significance stable, but you still converge to the optimal point.

For an explanation of tolerance and precision, we can ask @moneta.

6

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.