Problem with the ROC integral in the TMVA::CrossValidation

TMVA
1

Hello everyone.

I came across an issue when I was trying to use the ROC-Integral in the CrossValidation method. The GetROCAverage() and GetROCStandardDeviation() functions in the 6.08, 6.10 and 6.12 ROOT releases seem to return significantly different values provided the same code and input data.

For example, here is the plot of the N-1 variable optimization for the BDTG algorithm (all of the parameters are default except for the number of trees that is 1500) done in those releases.

RootReleases
RootReleases.png1418×1080 29.1 KB

What is the cause of those discrepancies and what release should I use?

Thanks in advance.

2

Hi,

There have been some bug fixes and improvements in the ROC curve and integral calculation in 6.12. These changes should explain the difference. The result obtained from 6.12 should be the accurate and correct one, so please ignore the old ones.
Maybe @kialbert could confirm this.

Lorenzo