Problems with HistFactory and Beeston-Barlow light statistical uncertainty

Roofit and RooStats
1

Dear experts,

I use HistFactory with ROOT v.6.24 to perform a 2D fit with many templates, however, it seems like this ROOT version (and all versions after v6.16) do not perform an analytical minimisation of the Beeston Barlow (BB) parameters. Instead, the fit seems to be trying to numerically minimise the BB parameters, and in general the fit is very slow and fails if the relative uncertainty in bins are above the threshold set by:

channel.SetStatErrorConfig(x, “poisson”)"

I have made a simple toy model, which should illustrate my problem. It can be found on lxplus:

/afs/cern.ch/work/v/vekirseb/public/HistFactory/toyModel.C

(for some reason I could not attach it here). It consists of one signal and one background template with 25 bins in 1D fitted to data. To run this on lxplus I do :

(1) make fitting templates:

lb-run -c x86_64-centos7-gcc9-opt Moore/v52r1 root ‘toyModel.C(true)’

(2) run template fit

lb-run -c x86_64-centos7-gcc9-opt Moore/v52r1 root ‘toyModel.C(false)’

If I set the threshold of the channel to:

B2D0MuNu.SetStatErrorConfig(0.03,“Poisson”);

And the statistical uncertainty is activated for each template with:

sample.ActivateStatError();

Then the fit gets STATUS=FAILED and the MINUIT table looks like this:

Floating Parameter  InitialValue    FinalValue +/-  Error     GblCorr.

gamma_stat_BMCorr_bin_1 1.0000e+00 3.6080e-01 +/- 3.16e-01
gamma_stat_BMCorr_bin_15 1.0000e+00 9.9680e-01 +/- 3.41e-01
gamma_stat_BMCorr_bin_17 1.0000e+00 9.9643e-01 +/- 3.60e-01
gamma_stat_BMCorr_bin_19 1.0000e+00 9.9622e-01 +/- 3.70e-01
gamma_stat_BMCorr_bin_20 1.0000e+00 9.9616e-01 +/- 3.72e-01
gamma_stat_BMCorr_bin_22 1.0000e+00 9.9613e-01 +/- 3.74e-01
gamma_stat_BMCorr_bin_24 1.0000e+00 9.9617e-01 +/- 3.72e-01
gamma_stat_BMCorr_bin_4 1.0000e+00 9.9458e-01 +/- 4.43e-01
gamma_stat_BMCorr_bin_6 1.0000e+00 9.9504e-01 +/- 4.24e-01
norm_histoBG 1.0000e-04 1.0000e-04 +/- 0.00e+00
norm_histoSig 1.0000e-04 1.0000e-04 +/- 0.00e+00
num_histoBG 1.0000e+04 1.0226e+04 +/- 1.72e+02
num_histoSig 1.0000e+04 9.7612e+03 +/- 1.36e+02

If I set the threshold to:

B2D0MuNu.SetStatErrorConfig(0.5,“Poisson”);

which is above the relative uncertainty of all bins, then the fit gets STATUS=OK, and the MINUIT table looks like this:

Floating Parameter  InitialValue    FinalValue +/-  Error     GblCorr.

      norm_histoBG    1.0000e-04    1.0000e-04 +/-  0.00e+00  <none>
     norm_histoSig    1.0000e-04    1.0000e-04 +/-  0.00e+00  <none>
       num_histoBG    1.0000e+04    1.0204e+04 +/-  1.61e+02  <none>
      num_histoSig    1.0000e+04    9.7974e+03 +/-  1.60e+02  <none>

However, I tried to do the same with ROOT v6.16, and here the fit does not fail when B2D0MuNu.SetStatErrorConfig(0.03,“Poisson”), and the BB parameters do not enter the MINUIT result table.

So I am wondering if I am missing something in my code, or if there is a problem with the treatment of BB statistical uncertainties in root versions after v6.16? Let me know if more information is needed to understand the issue.

Cheers,

Veronica

2

Hello,

I think @moneta or @jonas can give a hand, thanks!

3

Just to clarify: when I run the toy model on lxplus with the command “lb-run -c x86_64-centos7-gcc9-opt Moore/v52r1 root” it just means that I run the code with ROOT version 6.24.

4

I did some investigations in this.

Looking at RooBarlowBeestonLL.cxx (ROOT 6.26/04) into the function

bool RooStats::HistFactory::RooBarlowBeestonLL::getParameters(const RooArgSet* depList,
                                                              RooArgSet& outputSet,
                                                              bool stripDisconnected)

and then printing the content of outputSet with:

for (auto const& arg : outputSet) std::cout << "args: " << arg->GetName() << std::endl;

I get a list like:

args: gamma_stat_BMCorr_bin_0
args: gamma_stat_BMCorr_bin_10
args: gamma_stat_BMCorr_bin_12
args: gamma_stat_BMCorr_bin_14
args: gamma_stat_BMCorr_bin_16
args: gamma_stat_BMCorr_bin_18
args: gamma_stat_BMCorr_bin_2
args: gamma_stat_BMCorr_bin_21
args: gamma_stat_BMCorr_bin_23
args: gamma_stat_BMCorr_bin_3
args: gamma_stat_BMCorr_bin_5
args: gamma_stat_BMCorr_bin_7
args: gamma_stat_BMCorr_bin_9

However, using something like:

auto args = outputSet.selectByName("*gamma*");
args->Print();

I get:

gamma_stat_BMCorr_bin_0,
gamma_stat_BMCorr_bin_1,
gamma_stat_BMCorr_bin_10,
gamma_stat_BMCorr_bin_11,
gamma_stat_BMCorr_bin_12,
...
gamma_stat_BMCorr_bin_2,
gamma_stat_BMCorr_bin_20,
...
gamma_stat_BMCorr_bin_24,
gamma_stat_BMCorr_bin_3,
...
gamma_stat_BMCorr_bin_4,
gamma_stat_BMCorr_bin_9

i.e. the full list.
So it seems that the loop omits certain entries, although they would be needed for the removal in the analytical BarlowBeeston.

@moneta @jonas Any inside how this actually works would be greatly appreciated!

5

It turns out it is a “simple” C++ mistake.

Before 6.18, RooArgSet (RooAbsCollection) was a linked list, where deleting elements while accessing a container was fine. Since 6.18, it is a std::vector and it invalidates the iterator.

I’ll prepare a fix.

1 Like
6

Hi all,

thank you very much @decianm for your investigation and preparing the fix!

His fix is merged not to ROOT master:

I will also backport it to the 6.26 branch such that it is available in the next 6.26 patch release.

Cheers,
Jonas

7

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.