Question about KS test

ROOT
1

I want to apply the KS test to two histograms. But both have long tails, because of this I want to make wider bins at the tail, but the size will vary in both the histograms in exactly the same manner. Under this situation, is the KS test result reliable ?

Also if the KS test returns a probability of 0.05, does this mean that the probability of the two histograms to be similar is 5% or 95% ?
-thanks Debdatta.

2

Hello,

using histograms with variable bin sizes should be fine as long as the bin sizes are the same in the two histograms.
However, in general the KS test is not very much reliable on bin data, if the bin are not small enough. You should, if you can, apply the test on the original un-binned data.
See for example NOTE3 in the online doc:
root.cern.ch/root/htmldoc/TH1.ht … ogorovTest

A small return value of PROB will indicate that the two histograms are not compatible.

Best Regards

Lorenzo

3

HI Lorenzo,
thanks. Maybe because of merging the bins in the tail, I see a prob. of 1 in some cases. But I dont think I can apply this to unbinned data b.c. I am comparing the distribution of momentum from two histograms.
Is there anyway to access the error on the probability that is returned by the KS test.

                                                            -thanks Debdatta.
4

Seeing too often a probability of 1 is maybe an indication that the test is biased since you are using binned data.
What do you mean with the error on the probability ?
If the histogram are compatible the probability should be uniformly distributed between 0 and 1.

Regards, Lorenzo

5

HI Lorenzo,
since my data is binned in some cases the bins have very low statistics. Will this affect the outcome ? thats why i was wondering if there is some error associated with the probability that comes from statistical fluctuations.
Also, suppose I have two histograms that have a similar shape but the no. of events is different in the two cases. That is to say, the integral of the two histograms are different. In that case, should I normalize them first before putting them to the KS test?

6

As I mentioned before the KS test should be done on un-binned data. More bins your histogram has, the test will be less biased.
However, if the overall statistics is low it will be more difficult to distinguish two non-compatible histograms.

If you don’t use option “N” the two histogram, do not need to be normalized.

Regards

Lorenzo

1 Like