ROOTVecOps enables -ffast-math

In the context of generating python bindings using boost::python to access routines that make use of ROOT C++ libraries, it was spotted that the behaviour of NumPy was changing depending on the order of the imports. To reproduce it one can simply write the following in a C++ file (namely “module.cpp”):

#include <boost/python.hpp>
BOOST_PYTHON_MODULE( module ) {
  Py_Initialize();
}

and then compile it into a shared library

source /cvmfs/sft.cern.ch/lcg/views/LCG_102b/x86_64-centos7-gcc11-opt/setup.sh
PYTHON_OPTS="-I/cvmfs/sft.cern.ch/lcg/views/LCG_102b/x86_64-centos7-gcc11-opt/include/python3.9 -L/cvmfs/sft.cern.ch/lcg/views/LCG_102b/x86_64-centos7-gcc11-opt/lib -lpython3.9"
BOOST_OPTS="-L/cvmfs/sft.cern.ch/lcg/views/LCG_102b/x86_64-centos7-gcc11-opt/lib -lboost_python39"
ROOT_OPTS="-I/cvmfs/sft.cern.ch/lcg/releases/ROOT/6.26.08-34ede/x86_64-centos7-gcc11-opt/include -L/cvmfs/sft.cern.ch/lcg/releases/ROOT/6.26.08-34ede/x86_64-centos7-gcc11-opt/lib -lCore -lImt -lRIO -lNet -lHist -lGraf -lGraf3d -lGpad -lROOTVecOps -lTree -lTreePlayer -lRint -lPostscript -lMatrix -lPhysics -lMathCore -lThread -lMultiProc -Wl,-rpath,/cvmfs/sft.cern.ch/lcg/releases/ROOT/6.26.08-34ede/x86_64-centos7-gcc11-opt/lib -pthread -lm -ldl -rdynamic"
g++ -lstdc++ -pthread -std=c++17 -m64 -I/cvmfs/sft.cern.ch/lcg/views/LCG_102b/x86_64-centos7-gcc11-opt/include $PYTHON_OPTS $BOOST_OPTS  $ROOT_OPTS -fPIC --shared module.cpp -o module.so
python -c "import numpy; print(numpy.finfo(numpy.float64)); import module"
python -c "import module; import numpy; print(numpy.finfo(numpy.float64))"
python -c "import numpy; import module; print(numpy.finfo(numpy.float64))"

In the latter the warning will appear only in the first python call but as you can see in the third call the behaviour changes as soon as “module” is imported. This seems to be caused by the library ROOTVecOps (removing it from the list makes the warning disappear). Looking into the CMakeLists.txt file in the source repository

the library is enabling always -ffast-math, which in the case of being true, it is extremely dangerous since it compromises any code that is linked against it with an unexpected behaviour. So far the solution in my case is to explicitly avoid that library implementing a workaround, although it becomes relatively difficult because it is provided by default in the environment variable ROOT_LIBRARIES when using find_package in CMake. In addition, the problem propagates to any ROOT library that depends on ROOTVecOps, like ROOTDataFrame.

ROOT Version: v6-26-08
Platform: x86_64-centos7-gcc11-opt
Compiler: g++

An easier way of getting the error is by comparing this output

source /cvmfs/sft.cern.ch/lcg/views/LCG_102b/x86_64-centos7-gcc11-opt/setup.sh
python -c "import ROOT; import numpy; print(numpy.finfo(numpy.float64))"

which does not produce any warning. Compared to this:

source /cvmfs/sft.cern.ch/lcg/views/LCG_102b/x86_64-centos7-gcc11-opt/setup.sh
python -c "import ROOT; ROOT.RDataFrame; import numpy; print(numpy.finfo(numpy.float64))"

which produces the warning since ROOT.RDataFrame is being accessed.

Hi @mrprt ,

thank you for the report!

The -ffast-math compile option is listed as PRIVATE so it should not propagate outside of the library. Do you understand why that changes the behavior of numpy?

I can reproduce the behavior locally but rebuilding ROOT without the -ffast-math compile option for libVecOps does not solve the problem, there seems to be something else going on.

Possibly relevant: Someone’s Been Messing With My Subnormals!

I performed similar checks to Someone’s Been Messing With My Subnormals! and as far as I can tell libvdt is the culprit:

libROOTVecOps makes libvdt a PUBLIC link dependency if it comes from the system, propagating it to RNTuple, RDataFrame and TMVA

Recompiling libROOTVecOps without the libvdt dependency removes the issue. Recompiling all of ROOT with -Dbuiltin_vdt=OFF -Dvdt=OFF has the same effect (at least as far as the reproducer above goes).

This is a vdt-only reproducer:

import sys
from ctypes import CDLL
CDLL('/usr/lib/libvdt.so') # change this path to the correct location for you
import numpy as np
print(np.finfo(np.float32))

and indeed vdt’s CMakeLists sets -Ofast as part of its CMAKE_CXX_FLAGS (which implies -ffast-math).

Thanks a lot for promptly digging into this @eguiraud. Indeed I hadn’t realized that the PRIVATE keyword is being used, and that should not propagate the links as part of the ROOTVecOps library interface. Given that the problem seems to indirectly come from libvdt, I am wondering what could be the solution for this. Could be possible to specify also PRIVATE for vdt if it comes from the system? My concern is that this behaviour is present in the LCG releases, and from my perspective it should be consistent independently of where the vdt library comes from (although there might be some argument for this). From what I understood about -ffast-math, if not handled carefully its effects can be widely propagated, and I wonder how many people are affected by this who haven’t noticed yet.

For VDT in the LCG releases this is now [SPI-2290] VDT "-Ofast" (-ffast-math) is causing floating point issues when linked against - SFTJIRA , I pinged the VDT maintainers as well and for libVecOps, out of an excess of caution, I just merged [VecOps] Avoid -ffast-math by eguiraud · Pull Request #12214 · root-project/root · GitHub .

I think that’s the best we can do on our side :slight_smile:

Cheers,
Enrico

Thanks for the update :slight_smile:. I’ll wait for a new ROOT release to see these changes in-place then. In the meantime I’ll just fiddle around with the libraries to avoid the problem.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.