Kolmogorov-Smirnov Test Mean

[anprodri]

Hi again!

I have a question about KS-Test, it is related to a previous post answer:

Kolmogorov-Smirnov test values

Dear TMVA users, I have a very quick question regarding the KS test used to check for overtraining. I have been using the BDT method with the following number of events: Signal – training events : 21889 Signal – testing events : 21889 Signal – training and testing events: 43778 Background – training events : 53955 Background – testing events : 53955 Background – training and testing events: 107910 And when checking the BDT score d…

I couldn’t find any info about KS-test on the guide, and I still don’t understand why it’s optimal value is 0.5? how is it calculated exactly?

Thanks in advance.

[bellenot]

Maybe @moneta can give you some hints

[moneta]

Hi,

A Kolmogorov Smirnov test is applied between the training and the test output distribution. In case the two distributions are compatible a random value between 0 and 1 should be obtained, if they are not you will observe a difference and this might be an indication of overtraining. So a very small value is not good. A value which is not small is good, while a value too close to 1 is also no so good, because indicates that the statistical fluctuations are too small, and this can be also an indication of some potential problems.

See https://root.cern.ch/doc/master/classTH1.html#aeadcf087afe6ba203bcde124cfabbee4

Lorenzo

[anprodri]

Thanks for the answer!

Then, what would mean that the value is close to 0?

[kialbert]

Hi,

Are you asking about the KS score in the TMVA output?

For the TMVA plot several KS tests are performed and their average reported. As Lorenzo explained, for a single test, if the two distributions are equal, the result will be a value between 0 and 1 (uniform dist). The expectation for the average is then 0.5.

A value close to 0 (in this case with repeated experiments) mean that the two distributions are dissimilar.

Cheers,
Kim