Where does ROOT document its LLVM version requirements?

ROOT
1

I am unable to successfully build any of the ROOT versions starting with ROOT 6.18 on the HPC cluster we are using. Generally speaking, all of the failures appear to arise with LLVM incompatibilities.

I have tried turning off the “builtin_llvm” option, but that gives a fairly useless CMake error (this is with ROOT 6.18.02):

CMake Error at interpreter/CMakeLists.txt:282 (message):
  LLVM version different from ROOT supported, please try 5.0.x

I am building against the foss-2018b toolchain, with GCC 10.2.0. The Mesa 20.2.1 distribution that is compatible with that toolchain brings in LLVM 11.0.0. Does ROOT really expect us to go back six major released on LLVM?

Is there any documentation for the individual ROOT versions which actually specifies which LLVM release must be used?

2

Hi @Michael_Kelsey.

You should really build ROOT using the built-in LLVM distribution, i.e. -Dbuiltin_llvm=ON. I think a gcc-10 toolchain should be okay; if you are getting any errors during build, you can paste them here so that we can provide support. :slight_smile:

Cheers,
J.

3

@Michael_Kelsey Note that the build-in LLVM (and Clang) are built in a such a way that they should not interfere with the LLVM libraries used by Mesa.

4

@jalopezg I think that’s the default. Maybe all the apparently LLVM-related errors are a red herring. The only version of ROOT I’ve been able to build successfully so far is 6.18.02 using the foss-2018b toolchain, with GCC 7.3.0.

I have cleaned out my temp area, and am currently building (with a single make thread)

  • ROOT 6.22.08 with foss-2018b (GCC 7.3.0)
  • ROOT 6.18.02 with foss-2020b (GCC 10.2.0)
  • ROOT 6.24.00 with foss-2020b (GCC 10.2.0)

All three of these have failed to build successfully, but this way I’ll have clean log files to post here. All of them are being built with the same combination of CMake flags:

-Dxrootd=OFF -Dmysql=OFF -Dkrb5=OFF -Dodbc=OFF -Doracle=OFF
-Dpgsql=OFF -Dqt=OFF
-Dpython=ON -Dpython3=ON -Dpcre=ON -Dzlib=ON
-Dunuran=ON -Dexplicitlink=ON -Dminuit2=ON -Droofit=ON
-Dfftw3=ON -Dgsl=ON -DOpenGL_GL_PREFERENCE=GLVND

This is basically the same combination that was successful with 6.18.02 under foss-2018b.

We need to be able to move to a more recent toolchain for other reasons, and it would be nice to be able to use more recent ROOT versions. So far, both of those paths are blocked.

Once the three builds finish (much slower with only one thread!), I’ll upload the log files here.

5

@pcanal That’s what I thought. I tried building without the builtin version to try to avoid apparent collisions, but got a bunch of different errors instead.

6

@Michael_Kelsey Can you remind me what was the errors you saw when using the built-in LLVM?

7

I’ve successfully reproduced the errors for all three build combinations above. The log files are several megabytes each. I will post each one individually here, along with a brief description of the first error (each one is different!).

8

easybuild-ROOT-6.18.02-20210913.171019.EurMU.log.gz (193.7 KB)

ROOT-6.18.02 with foss-2020b (GCC 10.2.0)

First error occurs when compiling io/io/src/TGenCollectionStreamer.cxx (line 10855):

/scratch/group/mitchcomp/eb/tmp/build/ROOT/6.18.02/foss-2020b-Python-3.8.6/root-6.18.02/io/io/src/TGenCollectionStreamer.cxx: In function void DispatchConvertArray(int, TGenCollectionProxy::StreamHelper*, TGenCollectionProxy::StreamHelper*, int) [with From = char]:
/scratch/group/mitchcomp/eb/tmp/build/ROOT/6.18.02/foss-2020b-Python-3.8.6/root-6.18.02/io/io/src/TGenCollectionStreamer.cxx:152:12: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=]
  152 |       w[i] = (To)r[i];
      |       ~~~~~^~~~~~~~

This same error repeats multiple times in the log file (why?).

The next error, compiling core/metacling/src/TClingBaseClassInfo.cxx (line 11155) is definitely related to LLVM, and shows that despite the build using the “builtin_llvm” option, it is pulling in code from the LLVM 11.0.0 system installation:

In file included from /sw/eb/sw/LLVM/11.0.0-GCCcore-10.2.0/include/llvm/Support/Casting.h:18,
                 from /scratch/group/mitchcomp/eb/tmp/build/ROOT/6.18.02/foss-2020b-Python-3.8.6/root-6.18.02/interpreter/llvm/src/tools/clang/include/clang/Basic/LLVM.h:22,
                 from /scratch/group/mitchcomp/eb/tmp/build/ROOT/6.18.02/foss-2020b-Python-3.8.6/root-6.18.02/interpreter/llvm/src/tools/clang/include/clang/AST/AttrIterator.h:17,
[...]
9

easybuild-ROOT-6.22.08-20210913.171019.WknEn.log.gz (121.0 KB)

ROOT-6.22.08 with foss-2018b (GCC 7.3.0)

This was a segmentation fault (not a compiler error) when generating G__Core.cxx (line 10677):

[...]
#4 0x000000000053f6fd llvm::cl::opt<VerboseLevel, false, llvm::cl::parser<VerboseLevel> >::getExtraOptionNames(llvm::SmallVectorImpl<llvm::StringRef>&) (/scratch/group/mitchcomp/eb/tmp/build/ROOT/6.22.08/foss-2018b-Python-3.6.6/easybuild_obj/core/rootcling_stage1/src/rootcling_stage1+0x53f6fd)
#5 0x00000000027f46a8 ProvideOption(llvm::cl::Option*, llvm::StringRef, llvm::StringRef, int, char const* const*, int&) (/scratch/group/mitchcomp/eb/tmp/build/ROOT/6.22.08/foss-2018b-Python-3.6.6/easybuild_obj/core/rootcling_stage1/src/rootcling_stage1+0x27f46a8)
#6 0x00000000027fd51f (anonymous namespace)::CommandLineParser::ParseCommandLineOptions(int, char const* const*, llvm::StringRef, llvm::raw_ostream*) (/scratch/group/mitchcomp/eb/tmp/build/ROOT/6.22.08/foss-2018b-Python-3.6.6/easybuild_obj/core/rootcling_stage1/src/rootcling_stage1+0x27fd51f)
#7 0x0000000000531f8c RootClingMain(int, char**, bool) (/scratch/group/mitchcomp/eb/tmp/build/ROOT/6.22.08/foss-2018b-Python-3.6.6/easybuild_obj/core/rootcling_stage1/src/rootcling_stage1+0x531f8c)
[...]

Notice the reference to LLVM and Cling in the traceback.

10

easybuild-ROOT-6.24.00-20210913.171026.GuTvD.log.gz (129.9 KB)

ROOT-6.24.00 with foss-2020b (GCC 10.2.0)

The first error occurs when compiling interpreter/cling/lib/Interpreter/AutoSynthesizer.cpp (line 11127):

/scratch/group/mitchcomp/eb/tmp/build/ROOT/6.24.00/foss-2020b-Python-3.8.6/root-6.24.00/interpreter/llvm/src/tools/clang/include/clang/Basic/FixedPoint.h:168:17: error: could not convert S.llvm::SmallString<40>::str() from llvm::StringRef to std::string {aka std::__cxx11::basic_string<char>}
  168 |     return S.str();
      |            ~~~~~^~
      |                 |
      |                 llvm::StringRef

This is followed by several errors related to being unable to find an operator=() for various LLVM classes.

11

Hi @Michael_Kelsey,

To me, these errors suggest that your toolchain is picking up headers from the system LLVM instead of the LLVM distribution bundled as part of ROOT.

Could you please confirm that the directory /sw/eb/sw/LLVM/11.0.0-GCCcore-10.2.0/include/llvm/ appears in the output of:

$ `/sw/eb/sw/GCCcore/7.3.0/bin/g++ -print-prog-name=cc1plus` -v < /dev/null

Cheers,
J.

12

I can check that by loading the toolchain and ROOT dependencies directly (the HPC I’m on uses EasyBuild for software management).

… Not precisely; the “llvm/” subdirectory is not there. For the ROOT-6.22 attempt (foss-2018b), /scratch/group/mitchcomp/eb/x86_64/sw/LLVM/6.0.1-GCCcore-7.3.0/include is in the path, while for the other two (foss-2020b), it’s /sw/eb/sw/LLVM/11.0.0-GCCcore-10.2.0/include.

Urgh. The module system (Lmod) sets $CPATH with a search list of all of the loaded modules. I’ll have to look up whether that envvar takes precedence over command-line “-I” options. That might explain the conflict.

Do you think the errors in all three builds (including the segfault with ROOT-6.22.08) are due to this? Also, note that I am able to build ROOT-6.18.02 successfully under foss-2018b, with exactly the same setup.

13

Most likely. Since the build is “inconsistent” (with some .o files seeing one set of definition and other another set of definition), the result could be “random/arbitrary” behavior.

14

Ah, that makes sense. Even if the compilation of builtin_llvm was successful, things are still inconsistent.

On a positive note, I was able to modify the EasyBuild config files for the three ROOT versions to unset CPATH. So far, one build completely finished successfully, and the other two finished “building” successfully and are now doing the “installing” step. Thanks for helping me narrow this down, Phillipe!

1 Like
15

easybuild-ROOT-6.22.08-20210914.133354.rsCdt.log.gz (494.8 KB)

The ROOT-6.22.08 build failed at the “installation” stage (which runs make install), even though the package build completed successfully. The errors look like they’re LLVM related again.

Well, that’s unexpected. It seems as though the “install” step is doing a whole bunch of actual build. That’s not how a build-install cycle is supposed to work. I’ll try rerunning adding the “unset CPATH && ” before install, as well as before build.

16

Okay, adding unset CPATH to the install step worked.

I’m surprised that there’s any kind of building going on at the install stage; it has a significant impact on the build process. In particular (with the EasyBuild management system), the installation step is only done with a single thread (make install) rather than many threads. As a result, the build stage takes about 15-20 minutes, while the install stage takes more than half an hour.

Once again, thank you, @pcanal for pointing me in the right direction to work around my problem.

17

Is the “install takes a long time and seems to build stuff” true in only 6.22 or also in the newer version?

18

It was true for both 6.22 and 6.24. For both of them, the “install” stage took around half an hour, and the log files showed stuff actually being compiled. That’s why I ended up adding the “unset CPATH && ” prefix to the installation action as well as the build action.

19

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.