I have 4 parameters to discribe a 3D line (rho, phi, alpha, theta) where rho and phi is the point that closest to z axis on the XY plane, alpha is angle between that point and z axis.
Since there are three angles to be fitted, I wonder if TMinuit2 will consider the periodic boundary condition? Like after the step chosen phi = 0.0001, next step chosen phi = 3.1414 (0, pi)

If you know that the result of your parameter is likely to be in the area around 0 radians (2pi), then replace your parameter z in radians with z = arcsin(x), and fix the limits of x accordingly. This way the boundary will be somewhere else

Likewise if you know that the likely result will be at pi/2, you can change to arccos(x)