Minuit2Minimizer with ROOT.Math.IMultiGenFunction

ROOT
,
1

Hello,
I have some old code written in python 2.7 using ROOT < 6.18 and I am working on converting it to python 3.10 with the newest ROOT 6.30.
I saw that there were some changes considering the TPyMultiGenFunction and I tried to replace it with the Math.IMultiGenFunction but I ran into a cppyy.ll.SegmentationViolation.

Here the old version:

minimizer = ROOT.Minuit2.Minuit2Minimizer(0)               
errorfct = chi2fct3d_angles() 
errorfct.SetVals(samplesize, points, errors)
minimizer.SetFunction(errorfct)

(...)

class chi2fct3d_angles( ROOT.TPyMultiGenFunction ):
    def __init__( self ):
        ROOT.TPyMultiGenFunction.__init__( self, self )
        self.samplesize = 0
        self.points = np.zeros(12)
        self.errors = np.zeros(12)

    def NDim( self ): # total number of variables
        return 4.0

    def SetVals(self, samplesizet, pointst, errorst):
        self.samplesize = samplesizet
        self.points = pointst
        self.errors = errorst
    
    def DoEval( self, pars):
        chisq = []   
        
        for i in range(0,self.samplesize):
            errsx = self.errors[i][0]**2
            errsy = self.errors[i][1]**2
            errsz = self.errors[i][2]**2

            xvalue = self.points[i][0]
            yvalue = self.points[i][1]
            zvalue = self.points[i][2]
            distx, disty, distz = dist_p_line_angles(xvalue, yvalue, zvalue, pars) # distance squared of line to point i
            
            distx = distx**2
            disty = disty**2
            distz = distz**2       
            chisq.append((distx + disty + distz)/(errsx + errsy + errsz)) # add chisq of point i

        chisq = np.sum(np.array(chisq))

        return chisq

Here is what I tried looking at the example at Python interface: PyROOT - ROOT

minimizer = ROOT.Minuit2.Minuit2Minimizer(0)               
errorfct = chi2fct3d_angles() 
errorfct.SetVals(samplesize, points, errors)
minimizer.SetFunction(errorfct)

(...)

class chi2fct3d_angles( ROOT.Math.IMultiGenFunction ):
   def __init__( self ):
       #ROOT.Math.IMultiGenFunction.__init__( self, self )
       self.samplesize = 0
       self.points = np.zeros(12)
       self.errors = np.zeros(12)

   def NDim( self ): # total number of variables
       return 4.0

   def SetVals(self, samplesizet, pointst, errorst):
       self.samplesize = samplesizet
       self.points = pointst
       self.errors = errorst
   
   def DoEval( self, pars):
       chisq = []   
       
       for i in range(0,self.samplesize):
           errsx = self.errors[i][0]**2
           errsy = self.errors[i][1]**2
           errsz = self.errors[i][2]**2

           xvalue = self.points[i][0]
           yvalue = self.points[i][1]
           zvalue = self.points[i][2]
           distx, disty, distz = dist_p_line_angles(xvalue, yvalue, zvalue, pars) # distance squared of line to point i
           distx = distx**2
           disty = disty**2
           distz = distz**2       
           chisq.append((distx + disty + distz)/(errsx + errsy + errsz)) # add chisq of point i

       chisq = np.sum(np.array(chisq))
       
       return chisq 
   
   def Clone( self ):
       x = chi2fct3d_angles()
       ROOT.SetOwnership(x, False )
       return x

But I get the following error:


minimizer.SetFunction(errorfct)
cppyy.ll.SegmentationViolation: void ROOT::Minuit2::Minuit2Minimizer::SetFunction(ROOT::Math::IBaseFunctionMultiDimTempl<double>& func) =>
    SegmentationViolation: segfault in C++; program state was reset

What change am I still missing?

Thank you!
Viktoria

ROOT Version: 6.30/06
Built for linuxx8664gcc on Apr 22 2024, 13:50:50
From tags/v6.30.06-0-g4f4e716372@v6.30.06-0-g4f4e716372
With c++ (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0

2

Hi @viktoriak ,

I am adding PyROOT and math experts in the loop: @moneta @jonas @vpadulan

Best,
D

3

Hi @viktoriak!

I don’t know if this code is still supposed or work or not. PyROOT had big updates twice since ROOT 6.18, which a big cppyy upgrade in 6.22.

Here is a modified version of the example that works with the current ROOT but with a different (hopefully more robust) approach, using the ROOT::Math::Functor:

import ROOT

from array import array

class MyMultiGenFCN:

    def DoEval( self, x ):
        return (x[0] - 42) * (x[0] - 42)

    def to_root_math_functor(self):
        # keep the callable alive for the functor:
        self._do_eval_callable = self.DoEval
        functor = ROOT.Math.Functor(self._do_eval_callable, 1)
        return functor

def main():
    fitter = ROOT.Fit.Fitter()
    myMultiGenFCN = MyMultiGenFCN()
    functor = myMultiGenFCN.to_root_math_functor()

    params = array('d', [1.])
    fitter.FitFCN(functor, params)
    fitter.Result().Print(ROOT.std.cout, True)

if __name__ == '__main__':
    main()

Can you adapt this to your usecase?

Cheers,
Jonas

4

Hi @jonas ,
thanks for the new example! I could adapt it to my case but now python is stuck on the DoEval return line (no errors, it just stays at this line without going further and doesn’t even respond to ctrl+C). I’m not sure if this is related or a different issue.

minimizer = ROOT.Minuit2.Minuit2Minimizer(0) 
errorfct = chi2fct3d_angles() # define the function to minimise
errorfct.SetVals(samplesize, points, errors)
errorfunctor = errorfct.to_root_math_functor()
minimizer.SetFunction(errorfunctor)

(...)

class chi2fct3d_angles:
    def __init__( self ):
        self.samplesize = 0
        self.points = np.zeros(12)
        self.errors = np.zeros(12)

    def NDim( self ): # total number of variables
        return 4

    def SetVals(self, samplesizet, pointst, errorst):
        self.samplesize = samplesizet
        self.points = pointst
        self.errors = errorst
    
    def DoEval( self, pars):
        chisq = []   
      
        for i in range(0,self.samplesize):
            errsx = self.errors[i][0]**2
            errsy = self.errors[i][1]**2
            errsz = self.errors[i][2]**2
            
            xvalue = self.points[i][0]
            yvalue = self.points[i][1]
            zvalue = self.points[i][2]

            distx, disty, distz = dist_p_line_angles(xvalue, yvalue, zvalue, pars) # distance squared of line to point i
            distx = distx**2
            disty = disty**2
            distz = distz**2       

            chisq.append((distx + disty + distz)/(errsx + errsy + errsz)) # add chisq of point i

        chisq = np.sum(np.array(chisq))
        
        return chisq 
    
    def to_root_math_functor( self ):
        self._do_eval_callable = self.DoEval
        functor = ROOT.Math.Functor(self._do_eval_callable, self.NDim())
        return functor

I ran my script with python -m trace --trace and there it shows as very last line only “return chisq” before freezing.

Cheers,
Viktoria

5

I now went through it more thoroughly and the freezing seems to come from Minuit, as the return chisq does work a few times and then it gets stuck.

I added minimizer.SetPrintLevel(3) and print(minimizer.Status()):

minimizer = ROOT.Minuit2.Minuit2Minimizer(0) 

# print errors etc during the minimisation
minimizer.SetPrintLevel(3) 
       
errorfct = chi2fct3d_angles() # define the function to minimise
errorfct.SetVals(samplesize, points, errors)
errorfunctor = errorfct.to_root_math_functor()
minimizer.SetFunction(errorfunctor)
        
minimizer.SetMaxFunctionCalls(500)
minimizer.SetMaxIterations(500)
minimizer.SetTolerance(0.001)
                
#adding variables without limited range:
minimizer.SetVariable(0,"px", pars[0], step[0])
minimizer.SetVariable(1,"pz", pars[1], step[1])
minimizer.SetVariable(2,"phi", pars[2], step[2])
minimizer.SetVariable(3,"theta", pars[3], step[3]) 
                 
retval_min = minimizer.Minimize() # start the minimiser
        
minval = minimizer.MinValue() # minvalue of the function to minimise (chi2fct3d_angles() in this case)
        
stat = minimizer.Status()
print(stat)
        
covar_stat = minimizer.CovMatrixStatus()

temp_cov = []

ret_params = minimizer.X() # parameters of the fit
ret_errors = minimizer.Errors() # errors of the fit
final_params = [] # list to save all the parameters of the track
               
for i in range(0,pars.size):
        final_params.append(ret_params[i])
            for j in range(0,pars.size):
                #print minimizer.CovMatrix(i,j)
                temp_cov.append(minimizer.CovMatrix(i,j))
                
cov_matrices.append(temp_cov)
covar_mat_status.append(covar_stat)
        
final_lines_params.append(final_params)                   
chisq.append(minval)
conv_status.append(stat)

where I get an output a few times:

Minuit2Minimizer: Minimize with max-calls 500 convergence for edm < 0.001 strategy 1
Info in <Minuit2>: MnSeedGenerator Computing seed using NumericalGradient calculator
Info in <Minuit2>: MnSeedGenerator 4 free parameters, FCN pointer 0x7fff1d714240
Info in <Minuit2>: InitialGradientCalculator Calculating initial gradient at point 	[                0                0     -1.334513594      2.143902497]	
Info in <Minuit2>: InitialGradientCalculator Computed initial gradient for parameter px value 0 [ -0.001 , 0.001 ] dirin 0.001 grd 2000 g2 2e+06
Info in <Minuit2>: InitialGradientCalculator Computed initial gradient for parameter pz value 0 [ -0.001 , 0.001 ] dirin 0.001 grd 2000 g2 2e+06
Info in <Minuit2>: InitialGradientCalculator Computed initial gradient for parameter phi value -1.33451 [ -0.001 , 0.001 ] dirin 0.001 grd 2000 g2 2e+06
Info in <Minuit2>: InitialGradientCalculator Computed initial gradient for parameter theta value 2.1439 [ -0.001 , 0.001 ] dirin 0.001 grd 2000 g2 2e+06
Info in <Minuit2>: Numerical2PGradientCalculator Calculating gradient around function value 175.084 
	 at point 	[                0                0     -1.334513594      2.143902497]

and then it freezes.

Cheers,
Viktoria

6

If the behavior is so random, then I would bet on object lifetime issues here.

Can you maybe share your full script so I can reproduce the crash? Then it should be very easy to figure it out.

Cheers,
Jonas

7

Hi @jonas,

I realised my ROOT was built with a Minuit multiprocessing flag that was added unintentionally.
I just ran it again after rebuilding properly and it works just fine!

Thanks for your help,
Viktoria