Extract asymmetric confidence interval on fit

[dudouet]

Hi all,

@moneta I am trying to obtain asymmetric confidence interval on a fit result. The standard method gives for each point an uncertainty, so it assume that this is symmetric. But there is no reason than in a fit, the error propagation gives the same maximum and minimum error.

Is there a way to do it ?

Thanks

Jérémie

---

ROOT Version: Not Provided
Platform: Not Provided
Compiler: Not Provided

---

[moneta]

Hi,

The standard procedure using error propagation will provide symmetric interval. One could propagate also asymmetric uncertainties in principle, but it is not in my opinion a correct things to do.
Asymmetric uncertainties indicate that there is a deviation from the standard normal uncertainties and therefore one should use a different procedure than standard error propagation.
I would use something like parametric boostrap, using pseudo experiments, i.e .generate many different dataset from your data and fit each one of them and look at the different results. From the functions you obtain you can build the desired confidence intervals

Lorenzo

[dudouet]

Hi,

Thanks for you answer. Actually, I am playing with COVID data to fit models on it. My problem is that using the standard errors propagation (and consequently with symmetric errors), I obtain error bands that have to sense because it allows a reduction of the total number of deaths:

So what I am doing, is for each point, looping on all the parameters errors, to get the minimum and maximum error. And heare I obtain more reasonables error bands. But this is not completely correct because I am here supposing that the parameters are independant, which is not true:

I know that ROOT has not been designed to fit epidemiologist data, but I think that this kind of approach can also been found in particle physics.

Do you see my point ?

Thanks in advance

Jérémie

[moneta]

Hi,

What you are doing it is not really correct because as you said you neglect the correlations. What you see is better, probably because the confidence level computed in ROOT are inflated by the obtained chi2 value of the fit. It is a kind of renormalization of the obtained parameter errors.
Now if your model is not very good, and also because the data are bad, the obtained chi2 is very large.

I think also a problem with these data is that the data points errors are not really Poisson. For example every day contains counts which could be assigned day before, because there are delays in reporting the results, so there are some kind of correlations between the points, and assuming an error of sqrt(n) is not really appropriate.
Also the model you are fitting is maybe not the correct one and this will inflate the chi2 and spoil your predictions

As I said, for the confidence level, you could try boostrapping , this is the approach we do normally in particle physics. You could also try a Bayesian analysis, which would avoid getting results which are non-sense (e.g. a predicted number of deaths which are less than the observed one)

Lorenzo

[dudouet]

Hi,

I understand, thanks. I don’t want to spend too much time tin this, so I will not go in bootsraps or Bayesian methods.

Just one more question. I am now fitting the Total and daily curves at the same time with the same parameters in a combined fit:

Is there a way to access the ConfidenceIntervall of each curve by doing this ? I didn’t find a way to do it, so I keep my “wrong” method for the moment

[moneta]

Hi,

At the moment the confidence level is obtained using the fit function and its covariance matrix from the FitResult. The FitResult contains these objects for the total fit.
So it is not presently possible to compute the confidence level for a subset of the fit functions.
I think this can be improved by implementing a method which uses an external fit function and covariance matrix.
How is you use case ? Are you fitting simultaneously two data sets with same function or two different function sharing some of the parameters ?
Are you using a macro similar to the combinedFit.C tutorial (i.e. using the ROOT::Fit::Fitter class directly) ?

Lorenzo

[dudouet]

Hi,

In my case I am fitting the two datasets with two functions that have the same parameters. Actually I am not sure this is a good way to do because I am fitting at the same time the total death, and the daily deaths, which is the derivative of the first function, so theoretically both should give the same parameters. But this is not the case, that’s why I fit both to better contrains the fit.

I am using something similar to the combinedFit.C yes

Jérémie