Kolmogorov-Smirnov Test Mean

Hi again!

I have a question about KS-Test, it is related to a previous post answer:

I couldn’t find any info about KS-test on the guide, and I still don’t understand why it’s optimal value is 0.5? how is it calculated exactly?

Thanks in advance.

Maybe @moneta can give you some hints


A Kolmogorov Smirnov test is applied between the training and the test output distribution. In case the two distributions are compatible a random value between 0 and 1 should be obtained, if they are not you will observe a difference and this might be an indication of overtraining. So a very small value is not good. A value which is not small is good, while a value too close to 1 is also no so good, because indicates that the statistical fluctuations are too small, and this can be also an indication of some potential problems.

See https://root.cern.ch/doc/master/classTH1.html#aeadcf087afe6ba203bcde124cfabbee4


1 Like

Thanks for the answer!

Then, what would mean that the value is close to 0?


Are you asking about the KS score in the TMVA output?

For the TMVA plot several KS tests are performed and their average reported. As Lorenzo explained, for a single test, if the two distributions are equal, the result will be a value between 0 and 1 (uniform dist). The expectation for the average is then 0.5.

A value close to 0 (in this case with repeated experiments) mean that the two distributions are dissimilar.


1 Like