Dear experts,
I am having some problems with the observed upper limits calculated with roostats. In attachment, you find the code of a toy, that well represents my problem.
With “do_workspace_pdf.C” I am building the model, and the limit_calc_pdf.C calculates the expected and observed upper limit.
The model is built with
w->factory("SUM::total_model(N_data*Bkg_Model_Full_pdf,n_signal*signal_pdf)");
Bkg_Model_Full_pdf is the background pdf, whose shape is obtained by fitting real data with a polynomial function.
Data are in this toy generated by sampling the Bkg_Model_Full_pdf
RooDataHist *data=Bkg_Model_Full_pdf.generateBinned(y,2036834);
If I calculate the observed and expected upper limits, I see that they almost coincide, as I expect since I generated data from the same pdf I used to calculate the expected U.L.
However, in the do_workspace_pdf.C you can see that there are other two pdf for the background model, HighHistForSyst4_pdf and LowHistForSyst4_pdf. If you plot them you will see a very small difference between them and Bkg_Model_Full_pdf, since they have been obtained by varying the polynomial coefficient of the background of +1 sigma or -1 sigma. Indeed, by calculating the expected upper limit I see very few differences between the different models.
My question is the following: I tried to see what happens if I generate the data with
RooDataHist *data=HighHistForSyst4_pdf.generateBinned(y,2036834);
while keeping the same “total model” built with the Bkg_Model_Full_pdf and I see a very strange shape in the observed upper limit, I put the picture in attachment.
When I generate data with LowHistForSyst4_pdf, I see that the obs limit is a bit different from Bkg_Model_Full_pdf, but it seems ok.
I would like to understand why the UL with the HighHistForSyst4_pdf becomes like that. I saw that very often, in the HighHistForSyst4_pdf, the fit to data seems to fail, I report here the output:
AsymptoticCalculator::EvaluateNLL - value = -1.95963e+07 for poi fixed at = 1616.16
[#0] PROGRESS:Eval – OBSERVED DATA : qmu = -2.82388 condNLL = -1.95963e+07 uncond -1.95963e+07
[#0] WARNING:Minimization – AsymptoticCalculator: Found a negative value of the qmu - retry to do the unconditional fit
AsymptoticCalculator::EvaluateNLL - value = -1.95963e+07
[#0] WARNING:Minimization – AsymptoticCalculator: Found a better unconditional minimum old NLL = -1.95963e+07 old muHat 1515.15
[#0] WARNING:Minimization – AsymptoticCalculator: New minimum found for NLL = -1.95963e+07 muHat 1616.16
[#0] PROGRESS:Eval – After unconditional refit, new qmu value is 0
[#0] PROGRESS:Eval – AsymptoticCalculator::GetHypoTest – Find best conditional NLL on ASIMOV data set …
AsymptoticCalculator::EvaluateNLL - value = -1.96094e+07 for poi fixed at = 1616.16
[#0] PROGRESS:Eval – ASIMOV data qmu_A = 7.314 condNLL = -1.96094e+07 uncond -1.96094e+07
[#0] PROGRESS:Eval – poi = 1616.16 qmu = 0 qmu_A = 7.314 sigma = 597.595 CLsplusb = 0.5 CLb = 0.996579 CLs = 1.99316
< CLs = 0.501716 +/- 0
< CLb = 0.996579 +/- 0
< CLsplusb = 0.5 +/- 0
Can you do please some tests to verify if I am doing something wrong?
I don’t expect so much difference between the models…
limit_calc.zip (8.0 KB)
Or have you any suggestion?
Thank you,
Laura
