MINOS error problems, negative error is positive

Hello.

I am using ROOT::Math::Minimizer with Minuit2 to perform some fits, and I need to calculate asymmetric errors using Minos. My understanding is that the errors reported by Minos should be negative for the lower error (errLow in the documentation) and positive for the higher error (errUp in the documentation), the idea being that these are values added to the central estimate from the fit. Is that right?

If so, I am encountering a problem where Minos sometimes reports a value of errLow that is positive, and sometimes reports a value of errHigh that is negative, but never issues a warning saying that something went wrong. What might cause a problem like this?

For example, one instance of such an error saw Migrad produce a fit result to a parameter of 5241. The errLow value given by Minos was 12178, and the errHigh value was 28956. Is there a general situation which might cause Minos to act this way?

Thank you.

Hello @eggsAndCoffee; I am inviting @moneta to this thread. I am sure he can help with this issue.

Cheers,
J.

Hi

The returned error should be always positive, either if it is upper or lower error.

Lorenzo

Thank you for your reply, Lorenzo. It would appear that I have an even bigger problem then, as most of the errLow values returned by Minos are in fact negative. The few that are returned as positive are the exception.

For example, one parameter is returned from Migrad with a fit value of 11286900, and Minos gives errLow = -452965 and errHigh = 465637. No problem in finding these errors is reported by Minos. My understanding was that the lower limit within uncertainty would be 11286900 + errLow, because errLow should be negative. If errLow should in fact be positive, do you have any idea as to what might cause Minos to report the majority of errLow values as negative?

Thank you.

Hi,
Sorry, I got confused. The lower error returned by ROOT::Math::Minimizer::GetMinosError should be negative, while the user error should be positive.
If this is not the case for you, then I would need to investigate your case, please post the running code, if possible, otherwise the full log obtained with maximum print level (Minimizer.SetPrintLevel(3))

Lorenzo

Minuit2_output.txt.zip (1.5 MB)

Thank you Lorenzo, I appreciate this so far.

I have attached the output of the minimizer at print level 3, as you’ve asked. I am working on a 100 parameter fit, so the file is ~5 Mb when not compressed and contains ~29000 lines. I will try to give some helpful landmarks below to let you navigate this file a little more easily. Also of note is that each of the 100 parameters is lower limited to 0, using the SetLowerLimitedVariable() initialization.

I have added my own output of the format “MINOS: Par XX, low = YY high = ZZ” whenever a Minos error calculation is finished, where XX ranges from 0 to 99, and YY and ZZ are whatever Minos reports. You can navigate to the end of a Minos search by searching for a string with this pattern.

As an example of a parameter that seems to have errLow and errHigh both reported as positive by Minos, see parameter 94 at line 28333. For convenience, I have quoted the output from Minos and my own user-output below:

Minos: Lower error for parameter template_94 : 12178
Minos: Upper error for parameter template_94 : 28956.5
MINOS: Par 94, low = 12178.0385713052 high = 28956.4865644512

As far as I can tell, there was no error reported by Minos when working with parameter 94. It may also be helpful context to know that Migrad assigned parameter 94 a value of 5241.16 in the fit, see line 11644 of the output (or search for template_94, which was the name given to parameter 94 in my code).

As an example of a “successful” Minos error calculation, see parameter 56 (template_56) at line 21975, quoted below for convenience. The fit value of parameter 56 was 1.11438e+07. I also don’t see any error message given by Minos, so I’m not sure why parameter 56 gets Minos errors with the expected signs whereas parameter 94 has both Minos errors returned positive.

Minos: Lower error for parameter template_56 : -791566
Minos: Upper error for parameter template_56 : 802111
MINOS: Par 56, low = -791565.9898666848 high = 802110.7188721560

If I can supply anything else please let me know. Thank you again, I appreciate your time.

Hello,
Thank you for the output. I see that only template_94 is having this positive error.
It is maybe an issue with numerical calculations, given the special condition of your fit. The minimizing function depends weakly on that parameter.
To debug this case, I would really need to run your code. Will it be possible for you to share ?

Thank you

Lorenzo

Hello Lorenzo,

I scanned through the results and found that template_59, template_88, and template_94 all have this positive error. I’ve quoted the relevant lines below.

Minos: Lower error for parameter template_59 : 50031.5
Minos: Upper error for parameter template_59 : 200974
MINOS: Par 59, low = 50031.5107191034 high = 200973.6792186879

Minos: Lower error for parameter template_88 : 29320.7
Minos: Upper error for parameter template_88 : 481948
MINOS: Par 88, low = 29320.7317811690 high = 481948.3961627408

Minos: Lower error for parameter template_94 : 12178
Minos: Upper error for parameter template_94 : 28956.5
MINOS: Par 94, low = 12178.0385713052 high = 28956.4865644512

In order for you to run this code, I would have to send you about 2.5 Gb of data and I would need to send you the library of code that I wrote. Does that sound like something which is possible, or would it be easier for me to continue to provide you with context and output?

I noticed your comment about the minimizing function having a weak dependence on these particular parameters. I hope this information is helpful: of the 100 parameters, 16 have global correlation coefficients exceeding 0.9. The three “trouble parameters” do not have the three highest global correlations, though they are part of that set of 16 parameters with high global correlation. The other 13 parameters with high global correlation do not get erroneous Minos errors (i.e. the errors calculated by Minos have the expected signs).

It is also worth mentioning that this is not a problem that could have some parameters re-parameterized in terms of others, as far as I’m aware. At the fundamental level, each parameter corresponds to a unique physical entity in the problem that I am working on.

Hi,
Before sending me the code, can you try running on the ROOT master version. Which version are you using ?
There some more debugging can be enabled and also a small bug has been fixed in Minos, although I am not sure it affects this.
When running on the master, since the debug message will be much more, it would just enough to run Minos on one of these problematic parameter

Lorenzo

HI,
I have checked the code in Minos, and I can confirm a bug related to this was fixed last year. I think it should work fine in the ROOT master version or the 6.24 candidate release.
Can you please try it with the latest version ? Binary versions exists for example on cvmfs , see

Cheers

Lorenzo

Hello Lorenzo,

Apologies for the slow reply. I am using version 6.16 on Mac OS 10.14.6.

I will try updating to the latest version of root to see if that helps, and I will let you know what happens.

Thanks!

Hello Lorenzo,

I’ve upgraded to ROOT 6.22.06 and re-ran the code. The problem is only partially resolved. One parameter is still being assigned a positive value of errLow by Minos, template_59. I’ve quoted the snippet of relevant output below. There does not seem to be any error reported by MINOS. I have also attached the full output for template_59, and can send output for any other parameter as requested. template_59.txt (72.6 KB)

117505 ------ end Minos scan for up interval for parameter template_59
117506 Minos: Lower error for parameter template_59 : 460014
117507 Minos: Upper error for parameter template_59 : 200974
117508 MINOS: Par 59, low = 460013.8030822125 high = 200973.6792186879

A secondary question for you: many of the parameters are given an invalid upper error by Minos, but they still end up with a numerical value somehow. When Minos decides that an error estimate is invalid, does it instead assign a parabolic error estimate from the Hessian?

Thank you.

Hello,

Thank you for updating ROOT. I see some improvements, meaning that in 6.22 some fixes work for your case, although there could be still some issues remaining.
I would need some extra information to understand what is happening for parameter 59.
If you update to the ROOT master versior, thanks to a new logging, we can get more messages from the MnFunctionCross function. It would be great if you could do that.
In attrition of what you sent before, I would need only the final output of the fit to understand what are the minimum values and parabolic errors.

Concerning your second question, when the error estimate is invalid it should return the parabolic error estimate. But this if the error is not because the parameter is at the limit. In that case it will return as error the central parameter value (the one at the minimum) - the limit.

Cheers

Lorenzo
Otherwise

Hello Lorenzo,

Thank you.

Unfortunately, I do not know how to update to the master version, and I can’t seem to find any reference to a “master version” on the installation page. All of the versions on the installation page are for 6.22. Could you please help me figure out where I can find the master version that you’re talking about?

Thanks!

Hello @eggsAndCoffee,

Lorenzo refers to the “Building from source” procedure. Just be sure to get the current contents of the master branch. You can do that by specifying --branch master in the git clone command, i.e.

$ git clone --branch master https://github.com/root-project/root.git root_src

Then, follow the rest of the procedure.

Cheers,
J.

Hi,
Do you have access to cvmfs ? (https://root.cern/install/#lcg-releases-on-cvmfs)
Otherwise, could you build ROOT from source ? On which architecture are you running ROOT ?

Cheers

Lorenzo

Hi Lorenzo,

To be honest, I do not know if I have access to cvmfs because I had never heard of it previously. I might have access, as I’m running ROOT through a personal computer and so I see no reason why I wouldn’t be able to download whatever is needed. Otherwise yes, I should be able to figure out how to build ROOT from the source I think. I will try to do this in the next day or two, once I’ve backed up what I have now just in case I make a mistake while upgrading ROOT.

I am running ROOT on macOS Mojave version 10.14.6.

Hi,
I have created a tar file for the master, root_v6.23.01.macos-10.14-x86_64-clang100.tar.gz or a Mac package, root_v6.23.01.macos-10.14-x86_64-clang100.pkg, for a recent ROOT master version for 10.14.6.
Please try to use this version and let me know if you have any issues

Cheers
Lorenzo

Thank you Lorenzo. I think that I’ve been able to upgrade ROOT properly using the package that you linked, and I am now re-fitting with the updated version of ROOT. I will hopefully be able to give you an update within the day.

Hi Lorenzo,

I re-ran the code using ROOT 6.23.01 and the problem persists, with three parameters being reported with a positive value of errLow by Minos.

Unfortunately, I wasn’t able to capture the output of the code and I have no log to upload here. I tried to use the tee command, but it appears that the output from much of the ROOT code is not captured by tee. Do you know how to get around that problem, so that I can provide you with the logs that you need? There is an enormous amount of output to the terminal, so much so that I can’t manually copy and paste it into a file (five full minutes of scrolling up through the terminal output only got me to template_98!).

I will add that I was incorrect on March 5th when I told you that only one parameter was being assigned a positive value of errLow by Minos. Upon closer inspection, it appears that there were still three parameters being assigned a negative value, the same three as was originally reported (template_59, 88, and 94). This remains the case even with ROOT 6.23.01.