TMVAGUI ROC curve range

mesmeriza · March 29, 2018, 9:42am

Hi guys!

I am new to TMVA and I played around a bit with the TMVA GUI and I noticed that the y-axis (background rejection) is not plotted from 0 to 1, but from 0.2 to 1 … How can I change this range? Is it just a matter of plotting or is it only calculated in the range of [0.2, 1]?

Cheers

moneta · March 29, 2018, 2:08pm

Hi,

It is just a matter of plotting it. But to change it you would need to modify this file :

github.com

root-project/root/blob/master/tmva/tmvagui/src/efficiencies.cxx#L22


//                 = 2 --> plot rejection (B) versus efficiency (S)
//                 = 3 --> plot 1/eff(B) versus efficiency (S)


Bool_t __PLOT_LOGO__  = kTRUE;
Bool_t __SAVE_IMAGE__ = kTRUE;


// the coordinates
Float_t x1 = 0;
Float_t x2 = 1;
Float_t y1 = 0;
Float_t y2 = 0.8;


// reverse order if "rejection"
if (type == 2) {
   Float_t z = y1;
   y1 = 1 - y2;
   y2 = 1 - z;
   //      cout << "--- type==2: plot background rejection versus signal efficiency" << endl;
} else if (type == 3) {
   y1 = 0;
   y2 = -1; // will be set to the max found in the histograms

and change line 22 from 0.8 to 1.0
and rebuild ROOT

Lorenzo

mesmeriza · March 29, 2018, 2:15pm

Thanks for the quick answer!
What other way is there to get a ROC curve, without using the TMVAGUI?
I found this function: TGraph * GetROCCurve (const UInt_t points=100) in the TMVA::ROCCurve class.
Does it the same as the TMVAGUI?
Also, since I am using TMVA via python, I am not sure how to use the function porperly and haven’t found an example online.

Thanks a lot!

moneta · March 29, 2018, 2:24pm

HI,

Yes , you should be able to use this function, also from Python. In case of problems, please let me know

Lorenzo

mesmeriza · March 30, 2018, 10:16am

Hi Lorenzo,

Thanks again. I tried it and I get an ''TypeError: none of the 4 overloaded methods succeeded." error. What am I doing wrong?
I added the relevant code below.

...

output = TFile('tmva_output.root', 'recreate')

factory = TMVA.Factory('classifier', output,
                   ":".join([
                            "!V",
                            "!Silent",
                            "Color",
                            "DrawProgressBar",
                            "AnalysisType=Classification"]
                            ))
         

data = TMVA.DataLoader(".")

    
for n in range(n_vars):      # adding the variables
    name = predictors[n]    # the array preditors contains the column names of my variables
    data.AddVariable(name, 'F')
   
# Call root_numpy's utility functions to add events from the arrays
add_classification_events(data, X_train, y_train) 

add_classification_events(data, X_test, y_test, test=True)  

data.PrepareTrainingAndTestTree(TCut('1'), 'NormMode=EqualNumEvents')

BookMethod = factory.BookMethod

# add methods that you want to test on the data
BookMethod(data, 'Cuts', 'testCuts') # simple rectangular cuts

factory.TrainAllMethods()
factory.TestAllMethods()
factory.EvaluateAllMethods()

roc = factory.GetROCCurve(data)   # this doesn't work

Cheers and happy easter!

moneta · March 30, 2018, 12:08pm

Hi

This looks to.me to be a PyRoot problem in getting the right overload.

Maybe @etejedor can help you with this

Lorenzo

-------- Messaggio originale --------

etejedor · April 3, 2018, 10:02am

Hi @mesmeriza,

What overload of GetROCCurve do you want to use?

https://root.cern.ch/doc/v612/classTMVA_1_1Factory.html

I tried the one with this signature, which seems to be the one that you are trying to use in your code:

TCanvas * GetROCCurve (DataLoader *loader, UInt_t iClass=0)

and it does find the overload from PyROOT.

What version of ROOT are you using? Could you share a reproducer of your issue or alternatively try with the method with signature:

TCanvas * GetROCCurve (TString datasetname, UInt_t iClass=0)

Cheers,

Enric

mesmeriza · April 3, 2018, 10:37am

Hi Enric,

Thanks a lot for your reply! I dont have a preference of which overload to use.

Maybe there is something wrong with my dataloader? is this correct: data = TMVA.DataLoader(".") ?

My ROOT version is: 6.08/02

I wanted to try TCanvas * GetROCCurve (TString datasetname, UInt_t iClass=0), but I don’t know what is meant by datasetname - I tried “.” but that doesn’t work.

(Apologies if these are stupid questions/mistakes)

Cheers!

etejedor · April 3, 2018, 11:13am

Hi @mesmeriza,

Can you paste the error message that you get for each overload that you try?

Regarding the meaning of the parameters, I will let @moneta comment on it.

Cheers,
Enric

mesmeriza · April 3, 2018, 11:36am

Hi Enric,

Sure, I attached a text file with the complete console output of the error. The most important part at the end:

Traceback (most recent call last):
File “tmva_opt_test.py”, line 243, in
roc = factory.GetROCCurve(data)
TypeError: none of the 4 overloaded methods succeeded. Full details:
TGraph* TMVA::GetROCCurve(TMVA::DataLoader* loader, TString theMethodName, bool fLegend = kTRUE) =>
takes at least 2 arguments (1 given)
TGraph* TMVA::GetROCCurve(TString datasetname, TString theMethodName, bool fLegend = kTRUE) =>
takes at least 2 arguments (1 given)
TCanvas* TMVA::GetROCCurve(TMVA::DataLoader* loader) =>
problem in C++; program state has been reset
TCanvas* TMVA::GetROCCurve(TString datasetname) =>
could not convert argument 1

Cheers

error_msg_TMVA.txt (9.5 KB)

etejedor · April 3, 2018, 12:15pm

Hi @mesmeriza,

Thank you, can you also share a minimal reproducer along with its input data (if any).

Another thing you could try in the meantime is to upgrade to ROOT 6.12, like I mentioned in a previous post I do not have problems with these overloads in 6.12.

Cheers,
Enric

moneta · April 4, 2018, 12:53pm

Hi,

The dataset name is the name of the DataLoader class.
I would however , as suggested by Eric, use the latest ROOT version.

Lorenzo

NgJunKai · July 24, 2022, 3:28am

I am using ROOT installed in the conda environment. How to rebuild in this kind of scenario ?