We are atttempting build ROOT 6.22 on an HPC system which uses Lmod and EastyBuild modules. Among our module dependencies, we are pulling in Mesa 20.0.2, which gives us OpenGL. But Mesa also pulls in LLVM 9.0.1.
The ROOT build is failing in the interpreter with a very long stream of complaints all of the form:
/scratch/group/mitchcomp/eb/tmp/build/ROOT/6.22.08/foss-2020a-
Python-3.8.2/root-6.22.08/interpreter/llvm/src/tools/clang/inc
lude/clang/AST/ExprObjC.h:232:58:
error: llvm::isPodLike is not a template
232 | template <> struct isPodLike<clang::ObjCDictionaryElem
ent> : std::true_type {};
| ^
I didn’t run into these problems building ROOT 6.18 with a different toolchain, where the dependence was Mesa 18.1.1 and LLVM 6.0.0.
Has anyone else, particularly building on an HPC cluster, seen these errors? What can we be doing about it?
@Axel We are using GCC 9.3.0 (via the “foss-2020a” toolchain).
I’m doing a single-thread rebuild now; the previous build (which we expected to “just work”) was done with make -j 56, so the output is somewhat jumbled together.
I will extract the specific compilation block once I have it. . . .
The first failure occurs when compiling DictSelectionReader.cxx. I extracted the section of the Make output for just that compilation unit; at 32 kB, I’m attaching it rather than quoting.
Looks like this is the first ever occurrence of error: isPodLike is not a class template in the whole Internet, according to Google. Congratulations for that And this is building llvm, which is build orders of magnitude more commonly than ROOT.
Now - an obvious first cross-check would be to see whether any other compiler can build just this .o: cd /scratch/group/mitchcomp/eb/tmp/build/ROOT/6.22.08/foss-2020a-Python-3.8.2/easybuild_obj/core/dictgen and simple invoke that other compiler with the exact same flags as shown in your invocation.
If that fixes it it’s very likely a compiler issue. (I don’t know “the “foss-2020a” toolchain” so I cannot say anything about its quality and whether I’d expect it to be able to build ROOT.)
Heh. “Toolchains” seem to be a thing on HPC clusters – they define an overall system environment, including compilers, libraries, parallelism support, etc. If you’ve only ever done builds on freestanding machines, or small clusters, they aren’t really relevant.
We’ve built ROOT 6.14 and 6.18 successfully; it’s only the 6.22 distribution which is failing like this. Both of those were built with a dependence on an earlier release of Mesa which pulled in LLVM 8.0.1. This failure is correlated with pulling in LLVM 9.0.1.
It seems like ROOT doesn’t intend to use an external LLVM installation at all; it’s brought in its own partial set of LLVM code for use. Is that correct? Is there any documentation about what environments the ROOT team uses for their reference builds?
I will try doing a 6.22 build against the same foss-2018b toolchain (GCC 7.3.0) which we used for those previous, successful ROOT builds.
My attempt to build ROOT 6.18 against the new compiler failed for an entirely different reason: grid-deployment is now lcgpackages, so all the old ROOT tarballs are broken with DAVIX (see this post). We’re continuing to try to get these things to build successfully.
To make progress please try what I suggested above:
see whether any other compiler can build just this .o: cd /scratch/group/mitchcomp/eb/tmp/build/ROOT/6.22.08/foss-2020a-Python-3.8.2/easybuild_obj/core/dictgen and simple invoke that other compiler with the exact same flags as shown in your invocation.
Building ROOT 6.22.08 fails even earlier in the build with an older compiler (GCC 7.3.0) and LLV 6.0.0 indirect dependence (this is the environment where 6.18.02 built successfully). This time, it segfaulted in rootcling_stage1 at the step of:
Attempting to build ROOT 6.18.02 (patched) with GCC 9.3.0 and LLVM 9.0.1 in the environment failed with the identical “llvm::isPodLike is not a template” error.
OK, what I’m after is the compilation of a source file that triggers llvm::isPodLike is not a template with a different compiler than GCC 9.3.0 that you were using. IIUC your segfault in rootcling_stage1 is just that?
This would mean that the compiler you use is broken. You can try to use it to build llvm9 (for ROOT 6.24 or master), or llvm5 (for ROOT < 6.24) and I’d expect that those fail to build with your GCC9.3.0 the same way.
I’m confused. The “llvm::isPodLike is not a template” error happens at 71% of the build; this new rootcling_stage1 segfault happens at 46% of the build. I guess I was assuming that the ROOT folks did not reorganize the order in which different components got built.
And this is building llvm, which is build orders of magnitude more commonly than ROOT.