This is a follow-up to Chi2Test using TProfile , which ended up with [hist] Enable chi2 tests for TProfile by hageboeck · Pull Request #19930 · root-project/root · GitHub enabling computation of chi2 for TProfile.
I think there is another issue. The computation uses the TH1 implementation with WW option. This means that the statistic is exactly invariant under rescaling one of the two profiles.
For two profiles, I believe this is not what users expect.
Example with ROOT 6.40.02:
import ROOT
p1 = ROOT.TProfile("p1", "p1", 3, 0, 3)
p2 = ROOT.TProfile("p2", "p2", 3, 0, 3)
for i, mean in enumerate([10.0, 20.0, 30.0]):
for _ in range(50):
p1.Fill(i + 0.5, mean - 1.0) # bin i of p1: mean, spread 1
p1.Fill(i + 0.5, mean + 1.0)
p2.Fill(i + 0.5, 2 * (mean - 1.0)) # bin i of p2: exactly twice, spread 2
p2.Fill(i + 0.5, 2 * (mean + 1.0))
print("bin p1 p2")
for i in range(1, 4):
print(" %d %6.2f +- %.4f %6.2f +- %.4f"
% (i, p1.GetBinContent(i), p1.GetBinError(i),
p2.GetBinContent(i), p2.GetBinError(i)))
print()
print("Chi2Test(p2, 'CHI2') = %.6g" % p1.Chi2Test(p2, "CHI2"))
print("Chi2Test(p2) = %.6g" % p1.Chi2Test(p2))
chi2 = sum((p1.GetBinContent(i) - p2.GetBinContent(i)) ** 2
/ (p1.GetBinError(i) ** 2 + p2.GetBinError(i) ** 2)
for i in range(1, 4))
print()
print("bin by bin, sum (y1-y2)^2/(e1^2+e2^2) = %.6g for 3 d.o.f. -> p = %.3g"
% (chi2, ROOT.TMath.Prob(chi2, 3)))
Output
bin p1 p2
1 10.00 +- 0.1000 20.00 +- 0.2000
2 20.00 +- 0.1000 40.00 +- 0.2000
3 30.00 +- 0.1000 60.00 +- 0.2000
Chi2Test(p2, 'CHI2') = 0
Chi2Test(p2) = 1
bin by bin, sum (y1-y2)^2/(e1^2+e2^2) = 28000 for 3 d.o.f. -> p = 0