Dear ROOT expert:
In ROOT Tutorial, section ‘Neural Network’, I see the following words:
Two datasets must be defined before learning the network; a training dataset that is used when minimizing the error, and a test dataset that will avoid bias. Those two datasets can be build aside and then given to the network, or can be build from a standard expression. By default, half of the events are put in both datasets.
According to my experience, to check the performance of the network, we can not simply use the test dataset because it’s already involved in the training. We need a validation dataset to check the performance.
In TMVAClassification.C, I see the following lines:
// If no numbers of events are given, half of the events in the tree are used
// for training, and the other half for testing:
// factory->PrepareTrainingAndTestTree( mycut, “SplitMode=random:!V” );
// To also specify the number of testing events, use:
// factory->PrepareTrainingAndTestTree( mycut,
// “NSigTrain=3000:NBkgTrain=3000:NSigTest=3000:NBkgTest=3000:SplitMode=Random:!V” );
factory->PrepareTrainingAndTestTree( mycuts, mycutb, “nTrain_Signal=0:nTrain_Background=0:SplitMode=Random:NormMode=NumEvents:!V” );
My question is: is the testing dataset here also used to avoid bias during the training? If yes, then why don’t we use a validation dataset to check the performance? If I want to apply a third dataset to check the performance, how can I apply this in TMVA? Do we have such an example?
Or actually the testing dataset here is not used during the training? E.g. the testing dataset is acting as a validation dataset , and the training dataset is actually split into 2 parts during the training, first part to minimize the error, second part to avoid bias?
Cheers,Gang