Hi !
I’m doing some TMVA Boosted Decision Tree exercise and I’d like to save the Cut Value as a variable ? Basically the 0.019 variable. I couldn’t find the method to call.
— Factory : --------------------
— Factory : Method: Cut value:
— Factory : --------------------
— Factory : BDT: +0.019
— Factory : --------------------
Thanks !!
Hi,
What is it that you want to achieve with this cut value one you have it and what is your configuration to produce the given output?
Cheers,
Kim
I’m looping over a different choice of train and test subsamples, and would like to compute the accuracy on the results by counting the true events I get. For that I would need the cut values I guess.
The factory configurations is: factory.BookMethod(TMVA.Types.kBDT, “BDT”, "BoostType=AdaBoost:"nCuts=20:"SeparationType=GiniIndex:“Ntrees=500:MaxDepth=3”).
Thanks !
I’m not sure I still understand what it is you want to do. Is the cut value presented as the cut value that optimises signal efficiency * purity?
Maybe this piece of code can help you calculate the accuracy of your classifier. Basically, you can loop over the results and calculate it for a given cut value manually.
UInt_t numEvents = ...;
TMVA::Factory factory = new Factory(...);
TMVA::IMethod * imethod = factory->GetMethod("datasetName", "methodName");
TMVA::MethodBase * method = dynamic_cast<MethodBase *>(imethod);
TMVA::DataSet * dataset = method->Data();
TMVA::DataSetInfo * dsi = method->DataInfo();
TMVA::Results * results = dataset->GetResults("methodName", TMVA::Types::kTesting, TMVA::Types::kClassification);
Double_t cutValue = 0.019;
UInt_t true_positives = 0;
UInt_t false_positives = 0;
UInt_t true_negatives = 0;
UInt_t false_negatives = 0;
for (UInt_t iEvent = 0; iEvent < numEvents; ++iEvent) {
Double_t valueEvent = results[iEvent][0];
TMVA::Event ev = dataset->GetEvent(iEvent, TMVA::Types::kTesting);
if (valueEvent > cutValue) {
// Event passed cut, consider it signal
true_positives += dsi->IsSignal(ev); // True signal
true_positives += not (dsi->IsSignal(ev)); // True background
} else {
// Event failed cut, consider it background
false_negative += dsi->IsSignal(ev); // True signal
true_negative += not (dsi->IsSignal(ev)); // True background
}
}
Hope this helps!
Cheers,
Kim
Yes, that is what I want to achieve but I need to extract the cutValue (0.019) automatically, through some
.GetCutValue -like method.
cheers, maffe.
Hi,
I think this is what you are looking for
TMVA::IMethod * imethod = factory->GetMethod("datasetName", "methodName");
TMVA::MethodBase * method = dynamic_cast<MethodBase *>(imethod);
Double_t cut = method->GetSignalReferenceCut();
This is what is printed when EvaluateAllMethods is run.
Cheers,
Kim
Marvellous Kim.
cut = GetMethod(“BDT”).GetSignalReferenceCut()
was what I was looking for.
Thanks,
maffe.
Hi, this seems to no longer work. Is there a better way to get the confusion matrix of a dataset? I want the accuracy and F1 score etc. Further, is there a way to get the signal at B=0.01 and the AUC of ROC from the factory/dataset?
For example:
In file included from input_line_8:1:
macro.C:591:46: error: unknown type name 'MethodBase'
TMVA::MethodBase * methodbase = dynamic_cast<MethodBase *>(imethod);
^
macro.C:591:20: error: cannot initialize a variable of type 'TMVA::MethodBase *' with an lvalue of type 'TMVA::IMethod *'
TMVA::MethodBase * methodbase = dynamic_cast<MethodBase *>(imethod);
^ ~~~~~~~
macro.C:594:21: error: no viable conversion from 'DataSetInfo' to 'TMVA::DataSetInfo *'
TMVA::DataSetInfo * dsi = methodbase->DataInfo();
^ ~~~~~~~~~~~~~~~~~~~~~~
macro.C:602:34: error: use of undeclared identifier 'numEvents'
for (UInt_t iEvent = 0; iEvent < numEvents; ++iEvent) {
^
macro.C:603:14: error: no viable conversion from 'const std::vector<Float_t>' (aka 'const vector<float>') to 'Double_t' (aka 'double')
Double_t valueEvent = results[iEvent][0];
^ ~~~~~~~~~~~~~~~~~~
macro.C:604:17: error: no viable conversion from 'const Event *' to 'TMVA::Event'
TMVA::Event ev = dataset->GetEvent(iEvent, TMVA::Types::kTesting);
^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/alpha/packages/root_gpu/install/include/TMVA/Event.h:59:7: note: candidate constructor not viable: no known conversion from 'const Event *' to 'const Event &' for 1st argument; dereference the argument with *
Event( const Event& );
^
In file included from input_line_8:1:
macro.C:607:41: error: no viable conversion from 'TMVA::Event' to 'const Event *'
true_positives += dsi->IsSignal(ev); // True signal
^~
/home/alpha/packages/root_gpu/install/include/TMVA/DataSetInfo.h:156:49: note: passing argument to parameter 'ev' here
Bool_t IsSignal( const Event* ev ) const;
^
In file included from input_line_8:1:
macro.C:608:46: error: no viable conversion from 'TMVA::Event' to 'const Event *'
true_positives += not (dsi->IsSignal(ev)); // True background
^~
/home/alpha/packages/root_gpu/install/include/TMVA/DataSetInfo.h:156:49: note: passing argument to parameter 'ev' here
Bool_t IsSignal( const Event* ev ) const;
^
In file included from input_line_8:1:
macro.C:611:9: error: use of undeclared identifier 'false_negative'
false_negative += dsi->IsSignal(ev); // True signal
^
macro.C:611:41: error: no viable conversion from 'TMVA::Event' to 'const Event *'
false_negative += dsi->IsSignal(ev); // True signal
^~
/home/alpha/packages/root_gpu/install/include/TMVA/DataSetInfo.h:156:49: note: passing argument to parameter 'ev' here
Bool_t IsSignal( const Event* ev ) const;
^
In file included from input_line_8:1:
macro.C:612:9: error: use of undeclared identifier 'true_negative'
true_negative += not (dsi->IsSignal(ev)); // True background
^
macro.C:612:45: error: no viable conversion from 'TMVA::Event' to 'const Event *'
true_negative += not (dsi->IsSignal(ev)); // True background
^~
/home/alpha/packages/root_gpu/install/include/TMVA/DataSetInfo.h:156:49: note: passing argument to parameter 'ev' here
Bool_t IsSignal( const Event* ev ) const;
fixed like this:
TMVA::IMethod * imethod = factory->GetMethod("dataset", myMethodList);
TMVA::MethodBase * methodbase = dynamic_cast<TMVA::MethodBase *>(imethod);
TMVA::DataSet * dataset = methodbase->Data();
//TMVA::DataSetInfo * dsi = methodbase->DataInfo();
TMVA::DataSetInfo& dsi = methodbase->DataInfo();
TMVA::Results * results = dataset->GetResults(myMethodList, TMVA::Types::kTesting, TMVA::Types::kClassification);
Double_t cutValue = methodbase->GetSignalReferenceCut();
UInt_t numEvents = methodbase->GetNEvents();
UInt_t true_positives = 0;
UInt_t false_positives = 0;
UInt_t true_negatives = 0;
UInt_t false_negatives = 0;
for (UInt_t iEvent = 0; iEvent < numEvents; ++iEvent) {
double valueEvent = results[0][iEvent][0];
const TMVA::Event* ev = dataset->GetEvent(iEvent, TMVA::Types::kTesting);
if (valueEvent > cutValue) {
// Event passed cut, consider it signal
true_positives += dsi.IsSignal(ev); // True signal
true_positives += not (dsi.IsSignal(ev)); // True background
} else {
// Event failed cut, consider it background
false_negatives += dsi.IsSignal(ev); // True signal
true_negatives += not (dsi.IsSignal(ev)); // True background
}
}