It seems TProfile::Chi2Test is using TH1::Chi2Test.
I can obtain reasonable results only if I use “WW” option as second argument of the function, which is not the default.
TH1::Chi2Test appropriate for a TProfile?ROOT Version: ROOT 6.36.02
Hello,
That’s correct, since TProfile inherits the function from TH1, so they are really the same function. This also explains why the default arguments are unchanged.
Is the Chi2 test appropriate for TProfile?
I’m not sure. The test assumes that the bin bin contents are normally-distributed variables, so certainly, the profiles need to have been constructed such that the bin error actually returns the error of the mean (which seems to be the default). In that case, given that the bin contents represent the means with the corresponding error of the mean, one should be able to construct squared residuals from those and test that they follow the Chi2 distribution.
Looking at the code, I can neither confirm nor deny that it successfully manages to do that with a profile. It uses squares and differences, but the devil might be in the details.
I would recommend testing this empirically, either by doing toys to test that the function called with “CHI2” does return a chi2-distributed value, or, if you want to go one level deeper by passing an array in the third argument to analyse the residuals. They should be normally distributed, which can be checked running the toys.
shouldn’t be WW the default for TProfiles?
Yes, if the test works at all, it can only work with WW. Whether it is worth providing an override that calls the function with WW can only be answered if question 1 is answered.
yes, I have already tested it, and it works (only with WW): chi2 is distributed as a chi2. p-value is uniformly distributed (more or less).
Please consider overwriting TProfile::Chi2Test
The PR is up. Thanks for noticing and testing!
I confirmed with a colleague that it’s not by accident that they follow a chi2 distribution. It is indeed the fact that the means computed by the profiles should follow the normal distribution, so if we square them, they should be chi2.
