I am comparing the shapes of two TH2F histograms, and wanted to use both Chi2Test and KolmogorovTest, and see if they are consistent. I am comparing experimental points with Monte-Carlo expectation, so it is a comparison between Unweighted and Weighted distributions.
One thing is that KolmogorovTest always gives a warning:
In my understanding it is safe to use KS prob in this case, if the errors are correctly propogated, but there must be a reason for this warning. Could someone please shed a light on it?
Also, when runningn Chi2Test, I sometime run into a problem of having bins with 0 entries, and this seems to upset Chi2 calculator, so it gives 0 probability. Is it not supposed to ignore bins with 0 entries?
currently the 2D Kolmogorov test does not support the comparison with weighted histograms. If the number of effective entries of your weighted histogram is very different than the total bin content the result will not be correct.
In the latest release (5.19.04) the Chi2 has been improved to consider the case with zero bin entries and not return a zero probability. However, keep in mind, that if you have bins with few bin entries the results might also not be very reliable.
Thank you Lorenzo for the answer,
I will give the newest version a try.
For KSTest, naively I would think that if anything it will underestimate the probability, if the weighted distribution has more entries than the unweighted, hence more statistical power.
I have a follow up question. Is there any method in root that provides the the correlation matrix between TH2F histogram axis? That would be a way to sort of “project” TH2 into TH1, to reduce the number of empty bins.
Yes, I am using those to check the global correlations. I actually constructed the event-by-event correlation myself (Pearson’s correlation coefficients) which I make from my ntuple, but I was thinking if there was a way to do it automaticaly from some method in TTree or some other. It would have been very nice to have a method like this in TTree…