Info about two different training

TMVA
1

Hi, I try TMVAClassification with two Prepare…:

  1. sprintf(Stringa,“nTrain_Signal=%d:nTrain_Background=%d:nTest_Signal=0:nTest_Background=0:SplitMode=Random:NormMode=NumEvents:!V”, nTrainSVar, nTrainBVar);
    dataloader->PrepareTrainingAndTestTree( mycuts, mycutb, Stringa);
  2. dataloader->PrepareTrainingAndTestTree( mycutb, nTrainSVar, nTrainBVar, 0, 0, “SplitMode=Random:NormMode=NumEvents:!V”);
    And I got different results, is it normal ?
2

Hi,

That looks like it should be expected. You are using two different cut setups. In 1) you place a cut on both signal and background while in 2) you only cut on background.

Cheers,
Kim

3

I don’t use cut at all

4

Hi,

So do I understand correctly then that your variables mycuts and mycutb are both equal to "" (or equivalently TCut(""))?

Other than that difference the two calls are equivalent. In fact, in 2) and internal option string in constructed that is exactly the one in 1). In this case, no, different behaviour is not expected.

Cheers,
Kim

5

Exact but I find differences : this is the problem …tmva3.pdf (839.6 KB)

6

Hi,

If I understand your presentation correctly you find differences in the output of 3 classifiers (consistent over both runs), namely DNN_CPU, DNN_GPU and TMlpANN.

It might be the case (can’t remember the details from the top of my head) that these methods have inherent randomness (I think e.g. the DNN uses “hogwild” parallelisation i.e. the gradients is updated non-deterministically). Could you verify that you get identical output when you run either option 1) or 2) two times in a row?

Cheers,
Kim

7

Hi,

When using the DNN you will not get the same results, because of different weight initialisation and different batch ordering when training.
If you want reproducible result, this is possible In the coming release 6.16 when using the new MethodDL,
TMVA::Types::kDL , by typing TCpu<Double_t>::SetRandomSeed(seed)

Cheers

Lorenzo

8

I would like to share my experience with randomness in TMVA training output (using TMlpANN algorithm), Lorenzo pointed out.
What I found interesting was to run a handful of training and then compute for each event the distribution of the classifiers normalized by the cut value of each training.
In that case, normalized classifiers above(below) 1 are cut(not cut) or vice versa depending which side is cut…
One realizes that a given event can be cut for some trainings and not cut for others due to this random initialization of the weights.
By doing so, I think one is able to study the robustness of the TMVA algorithm, which is particularly interesting for noisy data.

Yoann