Dear statistics experts,
I am working with HistFactory to set upper limits on a process, currently I am trying to understand what it is doing and hence the example is simplified. I want to combine two single bin channels, each with a very small signal, so that the expected limit on the signal strength is of the order 10. Channels and uncertainties are:
channel1:
signal: 33 \pm 0.2 (stat)
bg: 17400 \pm 270 (stat)
channel2:
signal: 5.6 \pm 0.1 (stat)
bg: 3200 \pm 40 (stat)
Approximate median expected limits are:
channel1 only, no stat. unc.: 8
channel2 only, no stat. unc.: 20
combined, no stat. unc.: 7.4
channel1 only, stat. unc. included: 13
channel2 only, stat. unc. included: 24
combined, stat. unc. included: 14
My question is the following: When including the statistical uncertainties, why does the addition of channel 2 degrade the expected limit? Is this considered normal, or am I doing something wrong?
I tried resarching the issue but all I could find were these slides by Luca Lista:
indico.in2p3.fr/event/6315/cont … ides/0.pdf
where it says: “When adding channels that have low signal sensitivity may produce upper limits that are severely worse than without adding those channels”. The slides suggest that using CLs fixes this issue.
I am using asymptotic limit calculator (2 in HistFactory), and the test statistic is one sided profile likelihood (3 in HistFactory). Please find attached the root files with histograms and the xml configuration files I use to create RooWorkspace, in case they offer any hints. (.xml has been changed to .txt so they could be uploaded)
Thank you for the help,
Miha
Hmumu_channel2.txt (1.32 KB)
Hmumu_channel1.txt (1.32 KB)
Hmumu.txt (883 Bytes)
Hmumu_7TeV.root (4.79 KB)
Hmumu_8TeV.root (4.8 KB)