Adjust binning of Asimov data when using Asymptotic Calculator

[bibsession]

Hello experts,

I am using the Asymptotic Calculator to get a CLs plot for my exclusion analysis. I have tried adjusting the binning of my observable x to find out which binning yields the best result and how my analysis is affected overall by the binning. However, the median CLs did barely change when I changed the binning, only the observed limit changed. This leads me to the assumption that my Asimov data set is not binned correctly. Here is my setup for the Asymptotic Calculator and the HypotestInverterPlot:

	ModelConfig sbmc("sbmc","Signal + Background ModelConfig",&w);
	sbmc.SetPdf(*w.pdf("constr_tot_model"));
	w.defineSet("POI","inv_half_life");
	w.defineSet("nuisP","a0,a1,nbkg,eff,..."); //
	sbmc.SetParametersOfInterest(*w.set("POI"));
	sbmc.SetNuisanceParameters(*w.set("nuisP"));
	sbmc.SetObservables("x");
	sbmc.SetConstraintParameters(*constrained_parameters);
	RooRealVar* poi_sb = ((RooRealVar*)sbmc.GetParametersOfInterest()->first());
	sbmc.SetSnapshot(RooArgSet(*poi_sb));
	w.import(sbmc);
	
	RooStats::ModelConfig bmc = sbmc;
	bmc.SetName("bmc");
	bmc.SetWorkspace(w);
	RooRealVar* poi_b = ((RooRealVar*)bmc.GetParametersOfInterest()->first());
	poi_b->setVal(0);
	cout << poi_b->getVal() << endl;
	bmc.SetSnapshot(RooArgSet(*poi_b));
	w.import(bmc);
	w.Print();

	AsymptoticCalculator ac(binnedData, bmc, sbmc);
	ac.SetOneSided(true);
	ac.SetPrintLevel(-1);

	HypoTestInverter calc(ac);
	calc.SetConfidenceLevel(0.9);
	calc.UseCLs(true);
	calc.SetVerbose(false);


	ToyMCSampler *toymcs = (ToyMCSampler*)ac.GetTestStatSampler();//toymcs = calc.GetHypoTestCalculator()->GetTestStatSampler();

	ProfileLikelihoodTestStat profll(*sbmc.GetPdf());
	profll.SetOneSided(true);
	toymcs->SetTestStatistic(&profll);

	if (!sbmc.GetPdf()->canBeExtended()){
		toymcs->SetNEventsPerToy(1);
		cout << "can not be extended" << endl;
	}

	calc.SetFixedScan(30,0,0.2);
	HypoTestInverterResult* r = calc.GetInterval();
	upperLimit = r->UpperLimit();
	expectedLimit = r->GetExpectedUpperLimit(0);

	TCanvas* canvas = new TCanvas();
	plot = HypoTestInverterPlot("HTI_Result_Plot","HypoTest Scan Result",r);
	//canv = TCanvas("HypoTestInverter Scan");
	//canv.SetLogy(false);
	plot.Draw();//"CLb 2CL"
	canvas->Draw();

Is my assumption correct that the binning of x does not transfer to the Asimov dataset correctly? If so, how do I fix the problem?

[moneta]

Hi,
I woud not expect that the binning has a strong effect on the Asimov data set, because by definition it has no statistical fluctuations. While a slightly different binning can change significantly the data bin contents due to migrations of data between bins and then will have some effect on the observed limit calculation.

Cheers

Lorenzo