BayesianCalculator and Simultaneous Fit

Hello,

I’m trying to use the numerical integration option for the BayesianCalculator with a simultaneous fit. I have two data sets and two fits; they have the signal branching ratio in common. This signal branching ratio is related to the signal yields of the fit. The branching ratios that are returned from the fit seem reasonable, so I think I’ve set that up properly. The problems occur when I attempt to calculate the upper limit of the branching ratio.

When using the BayesianCalculator, I receive error messages saying that the interval isn’t “fully correct” and it will return an interval of [0,0]. This seems to happen regardless of the interval over which I allow the branching ratio to vary unless I allow the branching ratio to fluctuate negative; in that case I see an error about a negative number of events. The program will either skip calculating the interval or produce the [0,0] interval again.

Usual Error Output

[code][#1] INFO:Minization – p.d.f. provides expected number of events, including extended term in likelihood.
[#1] INFO:Fitting – RooAbsTestStatistic::initSimMode: creating slave calculator #0 for state emu (1451 dataset entries)
[#1] INFO:Eval – BayesianCalculator::GetPosteriorFunction : nll value -13640.1 poi value = 0.867799
[#1] INFO:Eval – BayesianCalculator::GetPosteriorFunction : minimum of NLL vs POI for POI = 0.868213 min NLL = -13640.1
[#1] INFO:Minization – p.d.f. provides expected number of events, including extended term in likelihood.
[#1] INFO:Minization – Including the following contraint terms in minimization: (flat_prior)
[#1] INFO:Eval – BayesianCalculator: Compute interval using RooFit: posteriorPdf + createCdf + RooBrentRootFinder
[#1] INFO:NumericIntegration – RooRealIntegral::init(_posteriorPdf_likelihood_times_prior_product_simPdf_flat_prior_cdf_Int[BR_A0_prime]) using numeric integrator RooIntegrator1D to calculate Int(BR_A0_prime)
[#1] INFO:NumericIntegration – RooRealIntegral::init(_posteriorPdf_likelihood_times_prior_product_simPdf_flat_prior_cdf_Int[BR_A0_prime|CDF]_Norm[BR_A0_prime]) using numeric integrator RooIntegrator1D to calculate Int(BR_A0_prime)
[#0] ERROR:Eval – BayesianCalculator::GetInterval Error returned from Root finder, estimated interval is not fully correct
[#0] WARNING:Eval – BayesianCalculator::GetInterval - computing integral from cdf failed - do a scan in 100 nbins
[#1] INFO:Eval – BayesianCalculator - scan posterior function in nbins = 100
Error in TF1::GetQuantiles: Integral of function is zero
[#0] WARNING:Eval – BayesianCalculator::GetInterval : 8594447 errors reported in evaluating log-likelihood function
[#1] INFO:Eval – BayesianCalculator::GetInterval - found a valid interval : [0 , 0 ]

90% CL interval: [ 0 - 0 ]
[/code]
Negative Fluctuation Error Output

[code][#1] INFO:Minization – p.d.f. provides expected number of events, including extended term in likelihood.
[#1] INFO:Fitting – RooAbsTestStatistic::initSimMode: creating slave calculator #0 for state emu (1451 dataset entries)
[#1] INFO:Eval – BayesianCalculator::GetPosteriorFunction : nll value -13640.1 poi value = 0.867799
[#0] ERROR:InputArguments – model_emu: calculated negative expected events: -141.058
[#0] ERROR:InputArguments – model_emu: calculated negative expected events: -109.172
[#0] ERROR:InputArguments – model_emu: calculated negative expected events: -77.287
[#0] ERROR:InputArguments – model_emu: calculated negative expected events: -45.4017
[#0] ERROR:InputArguments – model_emu: calculated negative expected events: -13.5164
[#1] INFO:Eval – BayesianCalculator::GetPosteriorFunction : minimum of NLL vs POI for POI = 0.867837 min NLL = -13640.1
[#1] INFO:Minization – p.d.f. provides expected number of events, including extended term in likelihood.
[#1] INFO:Minization – Including the following contraint terms in minimization: (flat_prior)
[#1] INFO:Eval – BayesianCalculator: Compute interval using RooFit: posteriorPdf + createCdf + RooBrentRootFinder
[#1] INFO:NumericIntegration – RooRealIntegral::init(_posteriorPdf_likelihood_times_prior_product_simPdf_flat_prior_cdf_Int[BR_A0_prime]) using numeric integrator RooIntegrator1D to calculate Int(BR_A0_prime)
[#1] INFO:NumericIntegration – RooRealIntegral::init(_posteriorPdf_likelihood_times_prior_product_simPdf_flat_prior_cdf_Int[BR_A0_prime|CDF]_Norm[BR_A0_prime]) using numeric integrator RooIntegrator1D to calculate Int(BR_A0_prime)
[#0] ERROR:Eval – BayesianCalculator::GetInterval Error returned from Root finder, estimated interval is not fully correct
[#0] WARNING:Eval – BayesianCalculator::GetInterval - computing integral from cdf failed - do a scan in 100 nbins
[#1] INFO:Eval – BayesianCalculator - scan posterior function in nbins = 100
Error in TF1::GetQuantiles: Integral of function is zero
[#0] WARNING:Eval – BayesianCalculator::GetInterval : 8965747 errors reported in evaluating log-likelihood function
[#1] INFO:Eval – BayesianCalculator::GetInterval - found a valid interval : [0 , 0 ]

90% CL interval: [ 0 - 0 ][/code]

I’m a bit stumped, to be honest. I suspect that since I have two data sets being fitted and I’m not calling a calculator for each data set, I might have set up the BayesianCalculator incorrectly. I have included the code for that section:

[code] ////////////////////////////
//UPPER LIMIT CALCULATIONS//
////////////////////////////

//BR_A0.setRange(0, 10);		//RESETTING OF RANGE OF BR_A0! Must limit to physical values!
//BR_A0.setRange(-10, 10);		//debugging
//BR_A0.setRange(-100, 100);	//debugging
//BR_A0.setRange(0, 1);			//debugging
//BR_A0.setRange(0, 100);		//debugging
BR_A0.setRange(0, 1000);		//debugging


//--Bayesean Calculator--//
RooArgSet POI_Bayes(BR_A0);												//Set BR as POI, option for other POIs
RooUniform flat_prior("flat_prior","Flat Prior", RooArgSet(BR_A0));		//Make flat prior in BR
BayesianCalculator bcalc(combData,simPdf,POI_Bayes, flat_prior);		//Call BayesianCalculator, create object
bcalc.SetConfidenceLevel(.90);											//Set 90% CL
bcalc.SetLeftSideTailFraction(0);										//Make one-sided upper limit
SimpleInterval* interval = bcalc.GetInterval();							//Get the interval
cout<<endl;
std::cout << "90% CL interval: [ " << interval->LowerLimit() << " - " << interval->UpperLimit() << " ]\n"; //Read out bounds of interval
cout<<endl;
UL_Bayes = interval->UpperLimit();										//Read out upper limit
RooPlot * plot = bcalc.GetPosteriorPlot();								//Make plot of likelihood and integral region.
TCanvas* canvas4 = new TCanvas("canvas4", "Fourth canvas", 1200, 1000);
canvas4->cd(1);
plot->Draw();
canvas4->SaveAs(strip+TString("ecm_")+strip2+TString("A0mass_BayesPlot.png"));[/code]

A larget section of code, including the setup of the models, has been attached. Any advice is appreciated!

Thank you,

J. Rorie
Larger_Section.C (6.35 KB)

Hello all,

In the process of trying to figure out what is wrong with my code, I went back and repeated the macro from the combined fit tutorial (link) and the BayesianCalculator tutorial link. While the combined fit tutorial executes as expected, using the BayesianCalculator gives me a similar [0 -0] interval. This is quite strange as I thought I recalled this giving a reasonable upper and lower limit before.

The output of the second tutorial macro is as follows:

[code]root [0] .x ReadComibinedModel_NumericalBayesian.c

RooFit v3.54 – Developed by Wouter Verkerke and David Kirkby
Copyright © 2000-2012 NIKHEF, University of California & Stanford University
All rights reserved, please read http://roofit.sourceforge.net/license.txt

[#1] INFO:Minization – p.d.f. provides expected number of events, including extended term in likelihood.
[#1] INFO:Fitting – RooAbsTestStatistic::initSimMode: creating slave calculator #0 for state channel1 (50 dataset entries)
[#1] INFO:Fitting – RooAbsTestStatistic::initSimMode: creating slave calculator #1 for state channel2 (50 dataset entries)
[#1] INFO:Eval – BayesianCalculator::GetPosteriorFunction : nll value -3858.53 poi value = 1.13364
[#1] INFO:Eval – BayesianCalculator::GetPosteriorFunction : minimum of NLL vs POI for POI = 1.13253 min NLL = -3858.53
[#1] INFO:Eval – BayesianCalculator - scan posterior function in nbins = 100
Error in TF1::GetQuantiles: Integral of function is zero
[#1] INFO:Eval – BayesianCalculator::GetInterval - found a valid interval : [0 , 0 ]

68% interval on mu is : [0, 0]
Info in TCanvas::MakeDefCanvas: created default TCanvas with name c1[/code]

So, a few questions:

  1. Is there something obvious about this tutorial that I am missing? Perhaps something in the model configuration?
  2. Is there a possibility this could indicate a problem with my ROOT/RooFit/RooStats configuration?
  3. In this macro, two slave calculators are called. Is that because of the line mc.SetObservables(RooArgSet(*w.var(“x”),*w.cat(“index”))); ? Can the category variable be passed to BayesianCalculator without using a model configuration object?

Thank you
ReadComibinedModel_NumericalBayesian.c (2.42 KB)
CombinedModel.root (15 KB)
MakeCobinedModel.c (3.71 KB)

Hi,

Sorry first for my late reply.
The problem is that the range for the nuisance parameters , which is used for integrating the likelihood is too large. The phase space where they are not zero is too small and the numerical integration algorithm fails.
A solution is to use a range in these parameter given by +/n sigma around your best fit value.
By doing this, the calculator will work. I attached your modified macro working.
Alternatively, you can also use the MCMCCalculator, which works fine for such kind of models

Best Regards

Lorenzo
ReadComibinedModel_NumericalBayesian.c (3.04 KB)

Hello,

No apology necessary; the reply was quite timely.

Thank you; I was able to execute the updated macro on my system. I’ve also taken your advice and trying out the MCMC calculator; I am finding peculiar answers for one-sided 90% CL upper limits but I want to make sure I’m not doing something silly before brining this issue to the forums.

Again, thank you

J. Rorie