Compiling C++ for Xeon Phi in ROOT

I’m currently working on writing some C++ code for use on Intel’s new coprocessor called the Xeon Phi. The code is supposed to be used for reconstruction of simulation data and we would like to get it running in parallel thus the inclusion of the Xeon Phi. My problem is that for any code to be run on the Phi it needs to be compiled using the Intel compiler with a special flag that will compile it for the Phi’s architecture. Also, the base code I’m working from has been written in such a way that it is supposed to be compiled and run from within ROOT. From the reading I’ve done it would seem that ROOT just uses a standard C++ compiler to compile code, is there any way I can change this and tell ROOT what compiler I want it to use and any flags I want it to use with that compiler?

As long as your ROOT code is a valid C++ program (with a main() function) as opposed to a simple ROOT macro (without a main() function), then you should be able to compile it “outside” of ROOT using whatever other compiler you want.

For example, if I have a ROOT macro called myprogram.C with a function myprogram.C, and it compiles+runs using ACliC (ROOT’s compiler) by doing something like:

, then I can make it compile with g++ by doing the following:

  1. Add a main() function that does nothing but take the command-line arguments, then calls the myprogram() function with the appropriate arguments.

int main(int argc, char * argv[]) { myprogram(argc,argv); return 0; }

  1. compile with g++ with the root libs and includes:

Now you just have to substitute in the second step your intel compiler for g++.

Jean-François

I don’t really share Jean-François’s opinion (in fact I rather expect problems when mixing compilers).

I assume you try to use “icc” / “ifort”.

If this is the case, ROOT “supports” several related “architectures”. Try:
ls -al ./config/Makefile.[iI][cC][cC]
./configure --help

You should find something like this:
linuxicc for i386 Linux with Intel’s icc
linuxia64ecc for Itanium Linux with Intel icc
linuxx8664icc for x86-64 Linux with Intel icc >=10.x
linuxx8664k1omicc for x86-64 Intel Many Integrated Cores Linux with Intel icc >=13.x
macosxicc for MacOS X >= 10.4 with Intel icc (Intel only)

If you need to add some additional “Xeon Phi” related flags, before calling “./configure SomeIccRelatedArchitecture”, edit the corresponding, architecture specific, file called “./config/Makefile.SomeIccRelatedArchitecture” and add these additional flags there (OPTFLAGS, NOOPT, CFLAGS, CXXFLAGS, F77FLAGS, LDFLAGS, SOFLAGS, CINTCFLAGS, CINTCXXFLAGS, CILIBS, SYSLIBS, F77LIBS).
Well, if it is sufficient for you, you could also try to play with simply defining “EXTRA_CFLAGS”, “EXTRA_CXXFLAGS” and “EXTRA_LDFLAGS” before calling “configure”.

BTW. Do share your experience with it here (regardless whether you’ll succeed or not).

Just to give an update, my first time going through the ROOT install for use with the Phi fails after it’s about 30-40 minutes through. The errors I’m getting are like the following:

ipo: warning #11010: file format not recognized for interpreter/cling/lib/Interpreter/ASTDumper.o
ipo: warning #11010: file format not recognized for interpreter/cling/lib/Interpreter/ASTNodeEraser.o
ipo: warning #11010: file format not recognized for interpreter/cling/lib/Interpreter/AutoSynthesizer.o
ipo: warning #11010: file format not recognized for interpreter/cling/lib/Interpreter/CIFactory.o

There are several lines of this each ending with a different .o file. It then turns to this:

x86_64-k1om-linux-ld: skipping incompatible /home/keegan/physics/root/interpreter/llvm/inst/lib/libclangFrontend.a when searching for -lclangFrontend
x86_64-k1om-linux-ld: skipping incompatible /home/keegan/physics/root/interpreter/llvm/inst/lib/libclangFrontend.a when searching for -lclangFrontend
x86_64-k1om-linux-ld: cannot find -lclangFrontend
x86_64-k1om-linux-ld: skipping incompatible /home/keegan/physics/root/interpreter/llvm/inst/lib/libclangSerialization.a when searching for -lclangSerialization
x86_64-k1om-linux-ld: skipping incompatible /home/keegan/physics/root/interpreter/llvm/inst/lib/libclangSerialization.a when searching for -lclangSerialization
x86_64-k1om-linux-ld: cannot find -lclangSerialization
x86_64-k1om-linux-ld: skipping incompatible /home/keegan/physics/root/interpreter/llvm/inst/lib/libclangDriver.a when searching for -lclangDriver
x86_64-k1om-linux-ld: skipping incompatible /home/keegan/physics/root/interpreter/llvm/inst/lib/libclangDriver.a when searching for -lclangDriver
x86_64-k1om-linux-ld: cannot find -lclangDriver

This again continues for a while before finally printing a make error.

make: *** [lib/libCore.so] Error 1

I’m continuing to determine where the error lies and I’ll post my solution when I find it.

Are you using the “head of the v5-34-00-patches branch” or are you trying to use the “head of the master branch”?
I would say you should try the former.

The v5-34-00 branch actually doesn’t have the option to configure for the Intel MIC. I tried using the v5-34-06 branch and got the following error:

#error "No Thread Local Storage (TLS) technology for this platform specified."
Warning: Error occurred during reading source files
Warning: Error occurred during dictionary source generation
!!!Removing core/thread/src/G__Thread.cxx core/thread/src/G__Thread.h !!!
Error: buildtools/core/utils/src/rootcint_tmp: error loading headers…
make: *** [core/thread/src/G__Thread.cxx] Error 1

There have been many changes since v5-34-06 (this version is broken anyhow).

Try the head of the v5-34-00-patches branch:
git clone -b v5-34-00-patches root.cern.ch/git/root.git v5-34-00-patches

If it still fails, attach two files “config.log” and “config.status”.

BTW. Try to configure your ROOT using something like (using “bash”):
unset ROOTSYS
./configure ANY OPTIONS YOU NEED > configure.out.txt 2>&1
and then make it using:
make > make.out.txt 2>&1
and attach also these two files (i.e. “configure.out.txt” and “make.out.txt”).

The install failed again with the same errors as with v5-34-06. I’ve attached the logs, the .status and .log extensions aren’t allowed to be uploaded here so I just renamed them with a .txt.
config.status.txt (18 Bytes)
config.log.txt (194 KB)
make.out.txt (93.7 KB)
configure.out.txt (5.24 KB)

There are two problems that I can see:

  1. configure seems not to search for 64bit libraries (and that’s probably why it missed libpthread)
  2. make seems to mix icc and gcc (actually it begins with a new “configure linuxx8664gcc” call)

Yeah I noticed the mixxing of gcc and icc too which I thought was a little weird. I also realized that it wasn’t seeing the 64 bit libraries so fixed it by running configure as follows:

./configure linuxx8664k1omicc --enable-roofit --enable-minuit2 --with-thread-libdir=/lib64
(This is actually from a conversation I found from Liviu Valsam who was working on getting ROOT running on the Phi as well)

I restarted the make file before I left yesterday, you can find why it failed in the attachment.

It actually sees the problem file (libminicern.so) as a data file instead of a shared object file. I’m not sure how or if I can change that.
make.error.txt (1.79 KB)

So I was able to get ROOT up and running on the Phi. There is a bug in the make file that compiles a few Fortran files without the -mmic flag for the Phi architecture which prevents the build from being successful. recompiling those files with the flag seems to have fixed the problem and I’ve submitted a bug report to hopefully fix this in future releases.

On to the interesting part, running things on the Phi has been much slower than the host. This really is to be expected though as so far I haven’t run anything optimized for the Phi or that has been vectorized and parallized.