Question about the scaling factor in ROOT’s weighted Poisson log-likelihood (WLS fit)

Hi ROOT experts,

I have been studying the implementation of the weighted log-likelihood fit (“WL” option) in ROOT (v6.36.04), and I would like to understand the statistical motivation behind one part of the implementation.

From the source code, ROOT defines the effective scaling factor as

s_i = sigma_i^2 / y_i

and

n_eff = y_i / s_i = y_i^2 / sigma_i^2
mu_eff = mu_i / s_i

which gives

l_i = s_i * [ n_eff * log(n_eff / mu_eff) + mu_eff - n_eff ]

I have independently implemented this expression and verified numerically that it reproduces ROOT’s MinFCN exactly for my weighted histogram fits.

While studying this implementation, I became curious about the role of the final scaling factor.

Once the effective quantities are defined as

n_eff = y_i / s_i
mu_eff = mu_i / s_i

it seems natural to consider the effective Poisson deviance

l_i = n_eff * log(n_eff / mu_eff) + mu_eff - n_eff by itself.

However, ROOT instead minimizes

l_i = s_i * [ n_eff * log(n_eff / mu_eff) + mu_eff - n_eff ]

While studying this implementation, I became curious about the role of the final scaling factor.

The effective count formalism is also discussed in the paper G. Böhm and G. Zech, Statistics of weighted Poisson events and its applications.

In a section on the Scaled Poisson Distribution (SPD), it is introduced for parameter estimation, where the effective quantities are defined in essentially the same way as in ROOT.

However, the likelihood in the paper does not appear to include the additional scaling factor s_i that multiplies the effective Poisson deviance.

So my question is:

Why does ROOT include the final scaling factor s_i?

  1. If the effective quantities (n_eff and mu_eff) are already introduced, why is the effective Poisson deviance by itself not sufficient?

  2. Would omitting the factor s_i lead to an incorrect likelihood or estimator? If so, could you explain what statistical property would be lost?

If there is a derivation, technical note, or publication explaining the origin of this factor, I would greatly appreciate a reference.

tagging @jonas @StephanH

I saw it, but I haven’t found the source code yet. :sweat_smile:

@Byungchul_Yu which code are you talking about specifically? There’s at least three places that deal with Poisson likelihoods.

Thanks! @StephanH I’m referring to the same implementation discussed in this earlier ROOT Forum thread:

https://root-forum.cern.ch/t/weighted-log-likelihood-estimation-for-low-statistics-binned-histogram/58191

@moneta Lorenzo’s reply there includes the relevant source code and a detailed explanation of how the weighted likelihood is implemented. My current question is a follow-up to that discussion: I understand how ROOT evaluates the expression numerically, but I’m trying to understand the statistical motivation for the scaling factor.

Hello @Byungchul_Yu ,

@jonas and me had a look, but for the moment, we can’t explain the factor. Jonas will check more in the coming days.