Loss functions for BDTG

TMVA
1

Good afternoon,

I have two questions concerning the loss functions used for the Gradient Boosted Decision Tree TMVA method.

  1. What is the default loss function used?
    This post here pretty clearly states it is cross-entropy. However, page 68 of the TMVA User guide states the default for all TMVA implimentations of GradientBoost is the binomial log-likelihood loss. Thank you in advance for the clarification!

  2. Is it possible to define a custom loss function for use in BDTG training? this presentation suggests on slide 20 that it is possible for regression problems. If so, pointing me towards a practical example of how to do so would be greatly appreciated. For context, i require the use of a modified cross-entropy loss function that would still be differentiable.

Thanks again,

Matt

2

Hi,

  • The loss used for classification is the cross-entropy, which is a binomial log-likelihood loss. So what stated in the documentation is correct.
  • For regression you have the choice of several loss functions, teh default is the Huber loss function, (see here) and you can optionally uses also a Least Square function or AbsoluteDeviation. I think is not possible to provide a user defined function. It is maybe some functionality that we could add if requested

Best regards

Lorenzo