ROC Integral from Test and Training sample

TMVA
1

Dear Experts,

I want to obtain the ROC integral for the train and test sample. So far I used

factory->GetROCIntegral("dataset","BDT");

but I couldn’t find out if this corresponds to the train or to the test sample. And in any case, I am interested in the comparison of both of them.

The methods I found (TMVA Plot Roc Curve From Test-Points , Questions about ROC curves (beginner with TMVA)) seem overly complicated for an apparently standard task.

The goal is to plot the evolution of both AUC as a function of the BDT complexity (#trees, depth) to try to spot where over-training begins.

Thank you in advance,
Joan

2

Hi jruizvid. I think @moneta can help you with this issue.

Cheers,
J.

3

Hi,
The function factory->GetROCIntegral(dataset, methodName) returns the ROC integral only for the testing dataset. I will investigate if the interface can be extended for the training dataset.
The only workaround is to compute the ROC integral either using the results from the outputfile.
This is similar to what provided in the link above. If you find it too difficult, I can provide you an example for this

Lorenzo