Hi,
I’m trying to use the TMVA DNN method as classificator. I have no problem using the old good TANH activation function but nearly everytime I try using the RELU activator I get ‘nan’ both as Test Err. and Train Err. after a few epochs.
When using CROSSENTROPY as error strategy I presume this is caused by RELU returning 0 or 1, both values that cannot be used as argument for the logarithms inside the cross entropy calculation.
But I tried using SUMOFSQUARES and the problem remains. In fact, I cannot train any configuration DNN with RELU without getting nan as error.
Someone know a stable workaround for using RELU in TMVA?
I am sorry for my later reply. Th problem is that you can’t use a RELU for the last, output layer. You need a tan or a sigmoid function.
This is the problem causing the NaN. We need to add a warning in TMVA for this.
If you still have problems after changing this, please let me know
