For a while, I have been trying to troubleshoot a bias I see in my fits with HistFactory (with ROOT 6.22.06).
For the purpose of this post, I set up toys with a very simple model of a Gaussian signal (mean = 0, sigma = 1) over a uniform background, in two channels. The model for Channel 1 has signal + bkg, and Channel 2 has only background.
The toy data is generated to be background only with a yield of 10k in each channel per toy.
The fit is done between -5 and 5, with 40 bins.
The background shape in the fit is data driven using
HistFactory::Sample::AddShapeFactor(). The ShapeFactor is shared between the two channels.
The signal shape is kept same from toy to toy. The generated bkg only data and background template are varied from toy to toy (using
When I run the fit over 1000 toys, I see a bias in the distribution of the fitted POI value. The numerical mean of the distribution is -3.9 +/- 3.3.
I see this bias in many instances of this fit, with different signal widths, different number of bins, different generated yields, and I am not sure of why it is there.
I would appreciate any advice on what I’m doing wrong. I have attached a script that should reproduce what I am doing.
PS: When I run this script, I also see the following warning for each toy
WARNING: Can't find parameter of interest: nSig in Workspace. Not setting in ModelConfig.
I think this come from the fact that my POI (nSig) is in only one channel and not the other, so HistFactory complains a bit.
troubleShootHF_2Channel.C (8.8 KB)