TMVA: Programmatically getting the significance

Hi,
I’m not sure if this belongs in the support forum.
I’m running a TMVA Fisher classifier on some data using pyroot. I can train, test and evaluate the method. I can then use the TMVAGui.C to look at the significance of the discriminant by clicking on “ClassifierCutEfficiencies”.
I would like to get that significance in my program to do something else with it. I can see that TMVA::MethodBase has a method called GetSignificance() which, according to the documentation, gives me what I need. My question is how I get at the actual MethodFisher so that I can call this method. I have tried factory.GetMethod(“FisherIso”) (that’s the corect title). I seem to get an object that has the correct name and title, but if I call GetSignificance(), my pyroot crashes and I am told that I called a purely virtual method. I could of course rewrite everything as a root-script, but I am probably doing something else wrong, too.
Please help.
Cheers,
Simon

Simon,

do you have an (as short as possible) script that shows the problem so that it can be reproduced and we can see what’s going on?

Thanks,
Wim

Data.txt contains a list of 500 events where a 0 designates background, 1 designates signal and the x value comes from a gaussian centered on 0 and 1 respectively.
Here’s my code:
simple.C:

{
	TTree* tree = new TTree("tree","tree");
	tree->ReadFile("data.txt","x/D:s/I");

	TFile* output = TFile::Open("TMVA.root","RECREATE");
	TMVA::Factory* factory = new TMVA::Factory( "MVAnalysis", output,"!V");

	factory->AddVariable("x",'D');

	factory->AddTree(tree,"Signal",1,"s==1");
	factory->AddTree(tree,"Background",1,"s==0");

	factory->PrepareTrainingAndTestTree( "s==1","s==0", "nTrain_Signal=100:nTrain_Background=100:nTest_Signal=100:nTest_Background=100:NormMode=NumEvents:!V");
	factory->BookMethod( TMVA::Types::kFisher, "FisherIso", "H:!V:Fisher:VarTransform=None:CreateMVAPdfs:PDFInterpolMVAPdf=Spline2:NbinsMVAPdf=50:NsmoothMVAPdf=10" );

	factory->TrainAllMethods();
	factory->TestAllMethods();
	factory->EvaluateAllMethods();
	TMVA::IMethod* meth = factory.GetMethod("FisherIso");
	cout<< "Type: "<<meth->IsA()->GetName()<<endl;
	// Here I can see that meth is an instance of TMVA::MethodFisher,
	// so I cast the pointer to that.
	
	TMVA::MethodFisher* fisher = meth;

	fisher->GetSignificance()
}

This crashes for me with the following message:

pure virtual method called
terminate called without an active exception

Somehow I cannot call this method. What should I do to get the significance? Do I have to find it myself?
data.txt (8.23 KB)

Simon,

I’m not following … You mentioned earlier that you were running pyroot, yet pyroot would automatically do the cast to the actual underlying object, so the final step should have worked. Do you have a pyroot script that does not work?

In the case here (C++), the problem is that the cast is a straight function-style cast, whereas MethodBase derives virtually from IMethod and MethodBase contains data, meaning an offset needs to be calculated to get to the actual MethodFisher object.

That said, after adjusting the offset, I’m getting a crash here:

TMVA::MethodBase::OptimizeTuningParameters(TString, TString)

meaning that the virtual inheritance is still messing up the cast when only using an offset calculation.

So, then I tried a dynamic_cast from compiled code, by creating a file mydyncast.C:[code]include “TMVA/MethodFisher.h”

TMVA::MethodFisher* dyncast( TMVA::IMethod* meth ) {
return dynamic_cast< TMVA::MethodFisher* >( meth );
}[/code]
And then using it with ACLiC from within your script: gROOT->LoadMacro( "mydyncast.C+" ); TMVA::MethodBase* fisher = dyncast( meth );
That worked … I get 0.873 as significance.

HTH,
Wim

Interesting. And good to know that it works.
I rewrote my pyroot script as a root script because I wanted to make sure it wan’t a python error.
Anyways, here it is as a pyroot script:

import ROOT

tree=ROOT.TTree("tree","tree")
tree.ReadFile("data.txt","x/D:s/I")

sigcut=ROOT.TCut("s==1;")
bkgcut=ROOT.TCut("s==0;")

output=ROOT.TFile.Open("TMVA.root","RECREATE")
factory = ROOT.TMVA.Factory( "MVAnalysis", output,"!V")

factory.AddVariable("x","D")

factory.AddTree(tree,"Signal",1,sigcut)
factory.AddTree(tree,"Background",1,bkgcut)

factory.PrepareTrainingAndTestTree( sigcut,bkgcut, "nTrain_Signal=100:nTrain_Background=100:nTest_Signal=100:nTest_Background=100:NormMode=NumEvents:!V")
factory.BookMethod( ROOT.TMVA.Types.kFisher, "FisherIso", "H:!V:Fisher:VarTransform=None:CreateMVAPdfs:PDFInterpolMVAPdf=Spline2:NbinsMVAPdf=50:NsmoothMVAPdf=10" )

factory.TrainAllMethods()
factory.TestAllMethods()
factory.EvaluateAllMethods()
fisher = factory.GetMethod("FisherIso")
fisher.GetSigninficance()

This also fails with the following message:

pure virtual method called
terminate called without an active exception
Abort trap

So it seems that pyroot is somehow making the same error that I was making.

Simon,

not completely the same. :slight_smile: At least it attempts to calculate the offset. Problem with virtual inheritance is that an actual instance is needed, since a path needs to be selected to walk the internal pointer rather than calculating a fixed offset between two types. Those internal pointers are compiler specific. With dynamic_cast, it’s all up to the compiler which knows the layout, but dynamic_cast can’t be used with dynamic types (although one could compile in instance and type, base_type combinations, like done in the mydyncast.C, that wouldn’t scale and is hard to extend with new derived classes). Some attempt is made in pyroot, but it comes with a “user beware”, I’m afraid …

Cheers,
Wim