Compiling ROOT 6.32.04 with CUDA TMVA

touchard · September 17, 2024, 10:34am

I face some troubles to compile ROOT 6.32.04 with CUDA and TMVA_GPU options

The build command below seems to work well

cmake -Dcuda=ON -Dcudnn=ON -Dtmva-gpu=ON -Dtmva-cpu=ON -DCMAKE_CUDA_ARCHITECTURES=86 -Droofit=OFF -DCMAKE_INSTALL_PREFIX=/root/root-cuda/root-6.32.04/install /root/root-cuda/root-6.32.04/root-6.32.04/

– Enabled support for: asimage builtin_afterimage builtin_clang builtin_cling builtin_davix builtin_gl2ps builtin_llvm builtin_lz4 builtin_openui5 builtin_vdt builtin_xrootd builtin_xxhash clad cuda cudnn dataframe davix fitsio gdml http imt mysql opengl pyroot roofit webgui root7 rpath runtime_cxxmodules shared sqlite ssl tmva tmva-cpu tmva-gpu tmva-pymva spectrum vdt x11 xml xrootd

But I can not compile. The following error seems to indicate that the build is not correct. Any help will be appreciated. I tried many builds which don’t work. I’am a little bit loosed. Thanks for your help

ROOT Version: 6.32.04
Platform: Ubuntu 22.04
Compiler: GCC 11.4.0 and CUDA 11.5.119

[ 96%] Building CUDA object tmva/tmva/CMakeFiles/TMVA.dir/src/DNN/Architectures/Cuda/CudaTensor.cu.o
[ 96%] Building CXX object tmva/tmva/CMakeFiles/TMVA.dir/src/DNN/Architectures/Cudnn/TensorDataLoader.cxx.o
[ 96%] Building CUDA object tmva/tmva/CMakeFiles/TMVA.dir/src/DNN/Architectures/Cudnn.cu.o
[ 96%] Built target G__Gviz3d
[ 96%] Building CXX object graf3d/gviz3d/CMakeFiles/Gviz3d.dir/src/TStructNodeEditor.cxx.o
[ 96%] Building CXX object graf3d/gviz3d/CMakeFiles/Gviz3d.dir/src/TStructNode.cxx.o
[ 96%] Building CXX object graf3d/gviz3d/CMakeFiles/Gviz3d.dir/src/TStructViewer.cxx.o
[ 96%] Building CXX object graf3d/gviz3d/CMakeFiles/Gviz3d.dir/src/TStructNodeProperty.cxx.o
[ 96%] Building CXX object graf3d/gviz3d/CMakeFiles/Gviz3d.dir/src/TStructViewerGUI.cxx.o
Scanning dependencies of target G__Eve
[ 96%] Building CXX object graf3d/eve/CMakeFiles/G__Eve.dir/G__Eve.cxx.o
[ 96%] Linking CXX shared library ../../lib/libGviz3d.so
[ 96%] Built target Gviz3d
/usr/include/c++/11/bits/std_function.h:435:145: error: parameter packs not expanded with ‘...’:
  435 |         function(_Functor&& __f)
      |                                                                                                                                                 ^
/usr/include/c++/11/bits/std_function.h:435:145: note:         ‘_ArgTypes’
/usr/include/c++/11/bits/std_function.h:530:146: error: parameter packs not expanded with ‘...’:
  530 |         operator=(_Functor&& __f)
      |                                                                                                                                                  ^
/usr/include/c++/11/bits/std_function.h:530:146: note:         ‘_ArgTypes’
/root/root-cuda/root-6.32.04/root-6.32.04/core/base/inc/TStorage.h:123:37: warning: declaration of ‘static void TStorage::UpdateIsOnHeap(const volatile UInt_t&, volatile UInt_t&)’ with attribute ‘noinline’ follows inline declaration [-Wa
ttributes]
  123 | R__NEVER_INLINE void TStorage::UpdateIsOnHeap(volatile const UInt_t &uniqueID, volatile UInt_t &bits) {
      |                                     ^~~~~~~~
/root/root-cuda/root-6.32.04/root-6.32.04/core/base/inc/TStorage.h:83:20: note: previous declaration of ‘static void TStorage::UpdateIsOnHeap(const volatile UInt_t&, volatile UInt_t&)’ here
   83 |    static void UpdateIsOnHeap(volatile const UInt_t &uniqueID, volatile UInt_t &bits);
      |                    ^~~~~~~~~~~~~~
/usr/include/c++/11/bits/std_function.h:435:145: error: parameter packs not expanded with ‘...’:
  435 |         function(_Functor&& __f)
      |                                                                                                                                                 ^
/usr/include/c++/11/bits/std_function.h:435:145: note:         ‘_ArgTypes’
/usr/include/c++/11/bits/std_function.h:530:146: error: parameter packs not expanded with ‘...’:
  530 |         operator=(_Functor&& __f)
      |                                                                                                                                                  ^
/usr/include/c++/11/bits/std_function.h:530:146: note:         ‘_ArgTypes’
gmake[2]: *** [tmva/tmva/CMakeFiles/TMVA.dir/build.make:2414 : tmva/tmva/CMakeFiles/TMVA.dir/src/DNN/Architectures/Cuda/CudaTensor.cu.o] Erreur 1
gmake[2]: *** Attente des tâches non terminées....
/root/root-cuda/root-6.32.04/root-6.32.04/core/base/inc/TStorage.h:123:37: warning: declaration of ‘static void TStorage::UpdateIsOnHeap(const volatile UInt_t&, volatile UInt_t&)’ with attribute ‘noinline’ follows inline declaration [-Wa
ttributes]
  123 | R__NEVER_INLINE void TStorage::UpdateIsOnHeap(volatile const UInt_t &uniqueID, volatile UInt_t &bits) {
      |                                     ^~~~~~~~
/root/root-cuda/root-6.32.04/root-6.32.04/core/base/inc/TStorage.h:83:20: note: previous declaration of ‘static void TStorage::UpdateIsOnHeap(const volatile UInt_t&, volatile UInt_t&)’ here
   83 |    static void UpdateIsOnHeap(volatile const UInt_t &uniqueID, volatile UInt_t &bits);
      |                    ^~~~~~~~~~~~~~
gmake[2]: *** [tmva/tmva/CMakeFiles/TMVA.dir/build.make:2400 : tmva/tmva/CMakeFiles/TMVA.dir/src/DNN/Architectures/Cuda/CudaMatrix.cu.o] Erreur 1
/root/root-cuda/root-6.32.04/root-6.32.04/core/base/inc/TStorage.h:123:37: warning: declaration of ‘static void TStorage::UpdateIsOnHeap(const volatile UInt_t&, volatile UInt_t&)’ with attribute ‘noinline’ follows inline declaration [-Wattributes]
  123 | R__NEVER_INLINE void TStorage::UpdateIsOnHeap(volatile const UInt_t &uniqueID, volatile UInt_t &bits) {
      |                                     ^~~~~~~~
/root/root-cuda/root-6.32.04/root-6.32.04/core/base/inc/TStorage.h:83:20: note: previous declaration of ‘static void TStorage::UpdateIsOnHeap(const volatile UInt_t&, volatile UInt_t&)’ here
   83 |    static void UpdateIsOnHeap(volatile const UInt_t &uniqueID, volatile UInt_t &bits);
      |                    ^~~~~~~~~~~~~~
/usr/include/c++/11/bits/std_function.h:435:145: error: parameter packs not expanded with ‘...’:
  435 |         function(_Functor&& __f)
      |                                                                                                                                                 ^
/usr/include/c++/11/bits/std_function.h:435:145: note:         ‘_ArgTypes’
/usr/include/c++/11/bits/std_function.h:530:146: error: parameter packs not expanded with ‘...’:
  530 |         operator=(_Functor&& __f)
      |                                                                                                                                                  ^
/usr/include/c++/11/bits/std_function.h:530:146: note:         ‘_ArgTypes’
/root/root-cuda/root-6.32.04/root-6.32.04/core/base/inc/TStorage.h:123:37: warning: declaration of ‘static void TStorage::UpdateIsOnHeap(const volatile UInt_t&, volatile UInt_t&)’ with attribute ‘noinline’ follows inline declaration [-Wattributes]
  123 | R__NEVER_INLINE void TStorage::UpdateIsOnHeap(volatile const UInt_t &uniqueID, volatile UInt_t &bits) {
      |                                     ^~~~~~~~
/root/root-cuda/root-6.32.04/root-6.32.04/core/base/inc/TStorage.h:83:20: note: previous declaration of ‘static void TStorage::UpdateIsOnHeap(const volatile UInt_t&, volatile UInt_t&)’ here
   83 |    static void UpdateIsOnHeap(volatile const UInt_t &uniqueID, volatile UInt_t &bits);
      |                    ^~~~~~~~~~~~~~
/usr/include/c++/11/bits/std_function.h:435:145: error: parameter packs not expanded with ‘...’:
  435 |         function(_Functor&& __f)
      |                                                                                                                                                 ^
/usr/include/c++/11/bits/std_function.h:435:145: note:         ‘_ArgTypes’
/usr/include/c++/11/bits/std_function.h:530:146: error: parameter packs not expanded with ‘...’:
  530 |         operator=(_Functor&& __f)
      |                                                                                                                                                  ^
/usr/include/c++/11/bits/std_function.h:530:146: note:         ‘_ArgTypes’
gmake[2]: *** [tmva/tmva/CMakeFiles/TMVA.dir/build.make:2442 : tmva/tmva/CMakeFiles/TMVA.dir/src/DNN/Architectures/Cudnn.cu.o] Erreur 1
gmake[2]: *** [tmva/tmva/CMakeFiles/TMVA.dir/build.make:2372 : tmva/tmva/CMakeFiles/TMVA.dir/src/DNN/Architectures/Cuda.cu.o] Erreur 1
gmake[1]: *** [CMakeFiles/Makefile2:47243 : tmva/tmva/CMakeFiles/TMVA.dir/all] Erreur 2
gmake[1]: *** Attente des tâches non terminées....
[ 96%] Built target G__Eve
gmake: *** [Makefile:156 : all] Erreur 2
root@ganl2427:~/root-cuda/root-6.32.04/build#

Danilo · September 17, 2024, 7:07pm

Dear Dominique,

Thanks for the post and welcome to the ROOT Community!
I am sorry ROOT did not work for you out of the box in this case.

I would like to defer this question to @moneta and @jonas , expert of TMVA and CUDA respectively.

Cheers,
Danilo

touchard · September 23, 2024, 8:14am

Thanks for your answer
Could this configuration work?

I succeeded to compile without tmva_gpu option.

jonas · September 23, 2024, 9:53am

If you’re not intended to use the GPU backend of TMVA, then this configuration perfectly works! You’re intending to use any of Roots CUDA features, namely the TMVA or RooFit CUDA backends?

touchard · September 24, 2024, 10:31am

Hi Jonas
This morning, after discussing with physicists about their needs, CUDA will be used at the edge of ROOT not inside.
I understand they can use a standard instance of ROOT without CUDA to visualize their data
Sorry for the noise
thanks again for your help