Pre-compiled binaries conflict with scikit-learn

[riley-x]

I’ve discovered that including sklearn imports in my pyRoot scripts occasionally causes a segfault. For example having these icludes

from sklearn.gaussian_process import GaussianProcessRegressor
from sklearn.gaussian_process.kernels import RationalQuadratic, RBF, Matern, WhiteKernel, ConstantKernel
from sklearn import preprocessing

causes a variable rebinning to segfault (notably, not always, about 1/10 times)

bins = np.array(..., dtype = np.float64)
hist.Rebin(len(bins) - 1, "newname", bins)

Note the sklearn imports do NOT need to be used. Just having the imports causes occasional segfaults. The stacktrace was not illuminating, it just points to the hist.Rebin() line.

I suspect this has to do with conflicts between my system version of numpy/python and the one used to compile the ROOT binary. The versions are listed below.

1. Is there a way to check the version of numpy used to compile ROOT with?
2. If the numpy version isn’t the issue, does anyone have a better idea?

ROOT Version: 6.26/06
Built for macosxarm64 on Jul 28 2022, 18:08:51
From tags/v6-26-06@v6-26-06

Python 3.8.10
numpy: 1.23.2
scikit-learn: 1.2.2

[riley-x]

Here’s the full stacktrace in case that helps:

[Macbook2022 18:38] VVSemilep$ ./python/mj_fits.py 
Wrote reco.fit_mJ.hp.one-exp.fatjet_pt0,3000.May23-ANN.png
 *** Break *** segmentation violation
[/Applications/root_v6.26.06/lib/libCore.6.26.06.so] TUnixSystem::DispatchSignals(ESignals) (no debug info)
[/usr/lib/system/libsystem_platform.dylib] _sigtramp (no debug info)
[/Applications/root_v6.26.06/lib/libcppyy_backend3_8.6.26.06.so] WrapperCall(long, unsigned long, void*, void*, void*) (no debug info)
[/Applications/root_v6.26.06/lib/libcppyy_backend3_8.6.26.06.so] WrapperCall(long, unsigned long, void*, void*, void*) (no debug info)
[/Applications/root_v6.26.06/lib/libcppyy_backend3_8.6.26.06.so] Cppyy::CallR(long, void*, unsigned long, void*) (no debug info)
[/Applications/root_v6.26.06/lib/libcppyy3_8.6.26.06.so] CPyCppyy::(anonymous namespace)::InstancePtrExecutor::Execute(long, void*, CPyCppyy::CallContext*) (no debug info)
[/Applications/root_v6.26.06/lib/libcppyy3_8.6.26.06.so] CPyCppyy::CPPMethod::ExecuteFast(void*, long, CPyCppyy::CallContext*) (no debug info)
[/Applications/root_v6.26.06/lib/libcppyy3_8.6.26.06.so] CPyCppyy::CPPMethod::ExecuteProtected(void*, long, CPyCppyy::CallContext*) (no debug info)
[/Applications/root_v6.26.06/lib/libcppyy3_8.6.26.06.so] CPyCppyy::CPPMethod::Call(CPyCppyy::CPPInstance*&, _object*, _object*, CPyCppyy::CallContext*) (no debug info)
[/Applications/root_v6.26.06/lib/libcppyy3_8.6.26.06.so] CPyCppyy::(anonymous namespace)::mp_call(CPyCppyy::CPPOverload*, _object*, _object*) (no debug info)
[/Library/Frameworks/Python.framework/Versions/3.8/Python] _PyObject_MakeTpCall (no debug info)
[/Library/Frameworks/Python.framework/Versions/3.8/Python] call_function (no debug info)
[/Library/Frameworks/Python.framework/Versions/3.8/Python] _PyEval_EvalFrameDefault (no debug info)
[/Library/Frameworks/Python.framework/Versions/3.8/Python] _PyEval_EvalCodeWithName (no debug info)
[/Library/Frameworks/Python.framework/Versions/3.8/Python] _PyFunction_Vectorcall (no debug info)
[/Library/Frameworks/Python.framework/Versions/3.8/Python] call_function (no debug info)
[/Library/Frameworks/Python.framework/Versions/3.8/Python] _PyEval_EvalFrameDefault (no debug info)
[/Library/Frameworks/Python.framework/Versions/3.8/Python] function_code_fastcall (no debug info)
[/Library/Frameworks/Python.framework/Versions/3.8/Python] call_function (no debug info)
[/Library/Frameworks/Python.framework/Versions/3.8/Python] _PyEval_EvalFrameDefault (no debug info)
[/Library/Frameworks/Python.framework/Versions/3.8/Python] _PyEval_EvalCodeWithName (no debug info)
[/Library/Frameworks/Python.framework/Versions/3.8/Python] _PyFunction_Vectorcall (no debug info)
[/Library/Frameworks/Python.framework/Versions/3.8/Python] call_function (no debug info)
[/Library/Frameworks/Python.framework/Versions/3.8/Python] _PyEval_EvalFrameDefault (no debug info)
[/Library/Frameworks/Python.framework/Versions/3.8/Python] _PyEval_EvalCodeWithName (no debug info)
[/Library/Frameworks/Python.framework/Versions/3.8/Python] _PyFunction_Vectorcall (no debug info)
[/Library/Frameworks/Python.framework/Versions/3.8/Python] call_function (no debug info)
[/Library/Frameworks/Python.framework/Versions/3.8/Python] _PyEval_EvalFrameDefault (no debug info)
[/Library/Frameworks/Python.framework/Versions/3.8/Python] _PyEval_EvalCodeWithName (no debug info)
[/Library/Frameworks/Python.framework/Versions/3.8/Python] _PyFunction_Vectorcall (no debug info)
[/Library/Frameworks/Python.framework/Versions/3.8/Python] PyVectorcall_Call (no debug info)
[/Library/Frameworks/Python.framework/Versions/3.8/Python] _PyEval_EvalFrameDefault (no debug info)
[/Library/Frameworks/Python.framework/Versions/3.8/Python] _PyEval_EvalCodeWithName (no debug info)
[/Library/Frameworks/Python.framework/Versions/3.8/Python] _PyFunction_Vectorcall (no debug info)
[/Library/Frameworks/Python.framework/Versions/3.8/Python] call_function (no debug info)
[/Library/Frameworks/Python.framework/Versions/3.8/Python] _PyEval_EvalFrameDefault (no debug info)
[/Library/Frameworks/Python.framework/Versions/3.8/Python] function_code_fastcall (no debug info)
[/Library/Frameworks/Python.framework/Versions/3.8/Python] call_function (no debug info)
[/Library/Frameworks/Python.framework/Versions/3.8/Python] _PyEval_EvalFrameDefault (no debug info)
[/Library/Frameworks/Python.framework/Versions/3.8/Python] _PyEval_EvalCodeWithName (no debug info)
[/Library/Frameworks/Python.framework/Versions/3.8/Python] PyEval_EvalCode (no debug info)
[/Library/Frameworks/Python.framework/Versions/3.8/Python] PyRun_SimpleFileExFlags (no debug info)
[/Library/Frameworks/Python.framework/Versions/3.8/Python] Py_RunMain (no debug info)
[/Library/Frameworks/Python.framework/Versions/3.8/Python] pymain_main (no debug info)
[/Library/Frameworks/Python.framework/Versions/3.8/Python] Py_BytesMain (no debug info)
[/usr/lib/dyld] start (no debug info)
Traceback (most recent call last):
  File "./python/mj_fits.py", line 585, in <module>
    v()
  File "./python/mj_fits.py", line 546, in single_func
    graphs.hist_2d_binned(args, bins, callback=callback)
  File "/Users/riley/Documents/CERN/VVSemilep/python/graphs.py", line 561, in hist_2d_binned
    _hist_1d(hists, info, opts_bin, **new_kwargs)
  File "/Users/riley/Documents/CERN/VVSemilep/python/graphs.py", line 344, in _hist_1d
    hists, opts = callback(hists, opts)
  File "/Users/riley/Documents/CERN/VVSemilep/python/graphs.py", line 554, in new_callback
    hists, opts = user_callback(hists, opts)
  File "./python/mj_fits.py", line 486, in callback
    h_mc, opts = _common_callback(hists, opts)
  File "./python/mj_fits.py", line 372, in _common_callback
    h = graphs.rebin(h, bins)
  File "/Users/riley/Documents/CERN/VVSemilep/python/graphs.py", line 162, in rebin
    return h.Rebin(len(val) - 1, h.GetName() + '_rebin', val)
cppyy.ll.SegmentationViolation: TH1* TH1::Rebin(int ngroup = 2, const char* newname = "", const double* xbins = 0) =>
    SegmentationViolation: segfault in C++; program state was reset

[Wile_E_Coyote]

Python: No module named 'root'

You MUST use EXACTLY the same Python version (and the same compiler) that your ROOT binaries were built for (otherwise, expect mysterious crashes anywhere, anytime): root-config --python-version # C/C++ calling python ; root-config --python2-version # python 2 calling C/C++ ; root-config --python3-version # python 3 calling C/C++ ; BTW. Remember to “source /the/location/of/ROOT/bin/thisroot.sh” to add your ROOT executables and libraries to your environment (when ROOT is not installed in some “…

[riley-x]

Is there a specific numpy or scikit-learn version that is required too? Or is it just python?

So I’m using python 3.8.10 but root-config --python-version returns 3.8.9. Is that the problem?

[Wile_E_Coyote]

Try the “latest release”.