Unsure about RooStats Output

[ngrieser]

Hello experts,

I am trying to use some different RooStats classes to (in the end) calculate Asymptotic CLs limits using the HypoInverter calculation. I am having some interesting results, and I’m not sure if I should trust the limits I receive.

To try and trouble shoot, I am using my workspace to simply do a significance calculation, and the beginning of the minimization of my Nll I get this message (which i also see in my HypoTest):

SFramework/TSSignificanceCalculator: constructed Nll with 115 floating parameters and starting value 589214
Minuit2Minimizer: Minimize with max-calls 57500 convergence for edm < 1 strategy 2
MnSeedGenerator: for initial parameters FCN = 589214.1669918
MnSeedGenerator: Initial state: - FCN = 589214.1669918 Edm = 2.48191 NCalls = 335
MnSeedGenerator: Negative G2 found - new state: - FCN = 589213.4226409 Edm = 0.345464 NCalls = 571
MnSeedGenerator: run Hesse - new state: - FCN = 589213.4226409 Edm = 0.434942 NCalls = 7737
VariableMetric: start iterating until Edm is < 0.001
VariableMetric: Initial state - FCN = 589213.4226409 Edm = 0.434942 NCalls = 7737
VariableMetric: Iteration # 0 - FCN = 589213.4226409 Edm = 0.434942 NCalls = 7737
VariableMetric: Iteration # 1 - FCN = 589213.0898308 Edm = 0.813166 NCalls = 7997
VariableMetric: Iteration # 2 - FCN = 589213.0112225 Edm = 0.0216214 NCalls = 8237
VariableMetric: Iteration # 3 - FCN = 589213.0086578 Edm = 0.0119888 NCalls = 8470
VariableMetric: Iteration # 4 - FCN = 589213.0086348 Edm = 0.0209316 NCalls = 8705
Info: VariableMetricBuilder: no improvement in line search
VariableMetric: Iteration # 5 - FCN = 589213.0086348 Edm = 0.0209316 NCalls = 8716
Info: VariableMetricBuilder: iterations finish without convergence.
Info in VariableMetricBuilder : edm = 0.0402582
Info in requested : edmval = 0.001
VariableMetric: After Hessian - FCN = 589213.0086348 Edm = 0.00367341 NCalls = 16004
VariableMetric: Iteration # 6 - FCN = 589213.0086348 Edm = 0.00367341 NCalls = 16004
Info: VariableMetricBuilder: Stop the minimization - reached machine accuracy limit
Info in Edm is smaller than machine accuracy : machineLimit = 0.0351198
Info in Current Edm is : edm = 0.00367341
Info in Required Edm is : edmval = 0.001
Info: VariableMetricBuilder: Edm has been re-computed after Hesse
Info in new value is now smaller than the required tolerance, : edm = 0.00367341
Minuit2Minimizer : Valid minimum - status = 0
FVAL = 589213.008634781814
Edm = 0.0036734132047962849
Nfcn = 16004

I’m not sure what to take of this. Initially I see some issue, but then minuit tells me I’ve obtained a valid minimum? Is this then OK?

However, when I run my unconditional fit I see many messages along these lines:

Info: VariableMetricBuilder: no improvement in line search
VariableMetric: Iteration # 1 - FCN = 589212.7774061 Edm = 0.0236309 NCalls = 262
Info: VariableMetricBuilder: machine accuracy limits further improvement.
Info in VariableMetricBuilder: INVALID function minimum - edm is above tolerance, : edm = 0.0236309
Info in VariableMetricBuilder: Required tolerance is 10 x edmval : edmval = 0.0005
Info in MnMinos could not find Upper Value for Parameter : par_name = alpha_ATLAS_JET_EffectiveNP_Statistical6
Minos: Invalid lower error for parameter alpha_ATLAS_JET_EffectiveNP_Statistical6

I’ve tried to adjust the tolerance but this hasn’t seemed to change anything. As this is a quite delicate process, is there some other way I can try and troubleshoot this process?

Thanks!
Nate

[moneta]

Hi,

It looks like the fits are not converging well. I guess one of the main cause is the limited precision in computing the full model likelihood. If you can improve the precision, it will be helpful.
Why are you using strategy 2 ? Is strategy 1 giving you problems in convergence ?
You might also try to start the the fits with different initial parameters.

Lorenzo

[ngrieser]

Hello Lorenzo,
Thanks for the look! So the fit strategy here is 2, but it is because I was getting similar issues with the strategy at 1. I will move it back to 1 for continued troubleshooting.

I get these issues when using HypoTestInverter (which is my end goal of the fit) as well, which scans over multiple fixed values of the POI and still returns this value. Should I actually adjust my initiali value of my NPs?

As for improving the precision, I’m not sure but do you mean adjusting RooFit::Precision? In that case, I can’t seem to find out how to adjust the usual roofit arguments on the roostats object. Maybe you could point me to this, as following this down the rabbit hole I couldn’t quite find where the FitTo/Minimizer function is actually called in the HypoTestInvert.

Thanks,
Nate