ROOT Version (e.g. 6.12/02): 6.10.02
Platform, compiler (e.g. CentOS 7.3, gcc6.2): Cray Programming Environment, with gcc 5.3.0
I am trying to build ROOT 6 on a supercomputer for running on Intel KNL nodes. The environment is a Cray setup, but in principle, the compiler is GCC 5.3
I’m using
$ cmake --version
cmake version 3.5.2
CMake suite maintained and supported by Kitware (kitware.com/cmake).
I point cmake at the Cray compiler wrappers, but if I do the following you see the version:
$ cc --version
gcc (GCC) 5.3.0 20151204 (Cray Inc.)
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
you can see the proper GCC version is there.
I’m running into this error:
– Constructing LLVMBuild project information
– Targeting X86
CMake Error at interpreter/llvm/src/cmake/modules/AddLLVM.cmake:561 (install):
install TARGETS given no ARCHIVE DESTINATION for static library target
“LTO”.
Call Stack (most recent call first):
interpreter/llvm/src/tools/lto/CMakeLists.txt:18 (add_llvm_library)
Any suggestions would be appreciated. It appears that CMake doesn’t know where to install the static libraries it has built.
You won’t be able to ROOT run with static libs, so don’t go down that route.
The problem is with Cray’s wrappers which default to static builds, not with the configuration. Looking into some old build scripts (for flang development, not so much ROOT, but for both the problem is in the LLVM part), adding:
should be the ticket (haven’t tested with ROOT as it was for flang; and this has been a while).
Note that the wrappers exist to provide access to standard HPC tools such as mpi and the Cray networking libraries. The wrappers simply add the appropriate include and link directories based on the loaded PrgEnv and other modules. Since ROOT doesn’t use any of those (nor does LLVM), you may as well invoke gcc directly and be done with it.
Of course, if you do not use icc for KNL-specific optimizations, you can just download and use the canned Linux binaries in the first place.
“Illegal instruction”?! What hardware is the interactive node on which you’re compiling? More specifically, is it that much older that it is missing instructions such as AVX512 that exist on KNL but not on the interactive node’s CPU? (Even so, I’d not expect gcc 5.3 to produce those.)
Thing is, the creation of G__Core.cxx is done by rootcling, which is itself a product of the build process (and it uses the LLVM libraries for parsing). So at this point you do have a configuration problem as rootcling needs to run on the interactive node, not on the big iron. Then again, LLVM needs to run on both …
Any chance you can start over with VERBOSE=1 and show the full CLI arguments of the command that build rootcling (there are a handful, as it entails a couple of intermediate steps)?
No, the verbose output of creating rootcling (stage1), not that of the creation of G__Core.cxx. That will show what the target architecture for rootcling was.
So any platform-specific parameters are still hidden in ‘CC’, as there’s nothing CPU-specific (beyond the innocent -m64) on the CLI.
What is the value of the CRAY_CPU_TARGET envar? If it’s ‘mic-knl’, can you load module ‘craype-haswell’ (or whatever is the cpu of your interactive nodes)?
I can, but then you lose any possible optimization for the worker nodes. I might as well just compile with the g++ installed on the login nodes in that case as you suggested initially.
What optimizations are you expecting that are worthwhile? For that matter, will that ancient gcc5.3 deliver them?
Teasing out rootcling (and rmkdepend) from the build process and building it for the interactive node with the rest for the batch nodes, is going to be a lot of work b/c it needs LLVM, as does the rest of ROOT during run-time.
You could compile the whole of ROOT twice: once on a KNL node, once on an interactive node, no cross-compilation. Then separate out the batch and interactive environments for the end-user.
But a more straightforward option would be to add something like ‘-march=core-avx2’ (assuming Haswell for the interactive nodes) to the build/link flags for rootcling and for LLVM. That way, most of ROOT is still cross-compiled and only dictionaries and wrapper generation will be build compatible for both platforms. Those parts would be potentially slower, but I doubt you’ll notice it for typical HEP workloads, since these are mostly called during initialization (i.e. one-offs) and I/O (i.e. already slowed down).
Hi @wlav,
OK, so after failing to simply use the gcc/g++/gfortran I installed the binaries. The CentOS and redhat versions don’t work. The ubuntu binaries work (Theta uses Suse). However, when I run ‘root’ I see this complaint:
ERROR in cling::CIFactory::createCI(): cannot extract standard library include paths!
Invoking:
LC_ALL=C g+±5 -O2 -DNDEBUG -xc++ -E -v /dev/null 2>&1 >/dev/null | awk ‘/^#include </,/^End of search/{if (!/^#include </ && !/^End of search/){ print }}’ | GREP_OPTIONS= grep -E “(c|g)++”
Results was:
With exit code 256
However my codes still work. Any way to get rid of this error?
