ROOT static linkage

Hello,
I am compiling some ROOT C++ programs on my personal laptop and the running time is quite reasonable.

Sent from my local computer:

$> time PrintRoot hist-263324.root
real    0m0.382s
user    0m0.274s
sys    0m0.124s

I am now doing the same on a Lustre FS:

$> time PrintRoot hist-263324.root
real    1m0.655s
user    0m0.864s
sys    0m2.640s

I was told that the problem might comes from the dynamical library linkage. Therefore I was suggested to switch to static linkage. I looked online and I could not find support regarding to this. Basically I would like to replace at the compilation level all those shared library:

meyer1@h2ologin2:/projects/sciteam/balh/production/meyer1/productions> ldd $(which PrintRoot)
    linux-vdso.so.1 =>  (0x00002aaaaaaab000)
    libUIUC.so => /mnt/b/projects/sciteam/balh/production/meyer1/software/escalade/escalade-build/lib/lib/libUIUC.so (0x00002aaaaaaae000)
    libCore.so => /projects/sciteam/balh/opt/x86_64-suse-linux-gcc49/root/6.14.06+pythia8/lib/libCore.so (0x00002aaaaae4c000)
    libCint.so => /projects/sciteam/balh/opt/x86_64-suse-linux-gcc49/root/5.34.36+pythia8/lib/libCint.so (0x00002aaaab515000)
    libRIO.so => /projects/sciteam/balh/opt/x86_64-suse-linux-gcc49/root/6.14.06+pythia8/lib/libRIO.so (0x00002aaaabe89000)
    libNet.so => /projects/sciteam/balh/opt/x86_64-suse-linux-gcc49/root/6.14.06+pythia8/lib/libNet.so (0x00002aaaac473000)
    libHist.so => /projects/sciteam/balh/opt/x86_64-suse-linux-gcc49/root/6.14.06+pythia8/lib/libHist.so (0x00002aaaac74c000)
    libGraf.so => /projects/sciteam/balh/opt/x86_64-suse-linux-gcc49/root/6.14.06+pythia8/lib/libGraf.so (0x00002aaaacd12000)
    libGraf3d.so => /projects/sciteam/balh/opt/x86_64-suse-linux-gcc49/root/6.14.06+pythia8/lib/libGraf3d.so (0x00002aaaad074000)
    libGpad.so => /projects/sciteam/balh/opt/x86_64-suse-linux-gcc49/root/6.14.06+pythia8/lib/libGpad.so (0x00002aaaad320000)
    libTree.so => /projects/sciteam/balh/opt/x86_64-suse-linux-gcc49/root/6.14.06+pythia8/lib/libTree.so (0x00002aaaad5f4000)
    libRint.so => /projects/sciteam/balh/opt/x86_64-suse-linux-gcc49/root/6.14.06+pythia8/lib/libRint.so (0x00002aaaad967000)
    libPostscript.so => /projects/sciteam/balh/opt/x86_64-suse-linux-gcc49/root/6.14.06+pythia8/lib/libPostscript.so (0x00002aaaadb91000)
    libMatrix.so => /projects/sciteam/balh/opt/x86_64-suse-linux-gcc49/root/6.14.06+pythia8/lib/libMatrix.so (0x00002aaaade08000)
    libPhysics.so => /projects/sciteam/balh/opt/x86_64-suse-linux-gcc49/root/6.14.06+pythia8/lib/libPhysics.so (0x00002aaaae1a2000)
    libMathCore.so => /projects/sciteam/balh/opt/x86_64-suse-linux-gcc49/root/6.14.06+pythia8/lib/libMathCore.so (0x00002aaaae3f2000)
    libThread.so => /projects/sciteam/balh/opt/x86_64-suse-linux-gcc49/root/6.14.06+pythia8/lib/libThread.so (0x00002aaaae802000)
    libTreePlayer.so => /projects/sciteam/balh/opt/x86_64-suse-linux-gcc49/root/6.14.06+pythia8/lib/libTreePlayer.so (0x00002aaaaea58000)
    libGeom.so => /projects/sciteam/balh/opt/x86_64-suse-linux-gcc49/root/6.14.06+pythia8/lib/libGeom.so (0x00002aaaaedc7000)
    libstdc++.so.6 => /opt/gcc/4.9.3/snos/lib64/libstdc++.so.6 (0x00002aaaaf1d3000)
    libm.so.6 => /lib64/libm.so.6 (0x00002aaaaf57a000)
    libgcc_s.so.1 => /opt/gcc/4.9.3/snos/lib64/libgcc_s.so.1 (0x00002aaaaf7f3000)
    libc.so.6 => /lib64/libc.so.6 (0x00002aaaafa0a000)
    libz.so.1 => /lib64/libz.so.1 (0x00002aaaafd87000)
    libdl.so.2 => /lib64/libdl.so.2 (0x00002aaaaff9d000)
    libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aaab01a1000)
    /lib64/ld-linux-x86-64.so.2 (0x0000555555554000)
    libssl.so.1.0.0 => /sw/EasyBuild/software/OpenSSL/1.0.2m/lib/libssl.so.1.0.0 (0x00002aaab03bf000)
    libcrypto.so.1.0.0 => /sw/EasyBuild/software/OpenSSL/1.0.2m/lib/libcrypto.so.1.0.0 (0x00002aaab0633000)
    libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00002aaab0a95000)
    libImt.so => /projects/sciteam/balh/opt/x86_64-suse-linux-gcc49/root/6.14.06+pythia8/lib/libImt.so (0x00002aaab0cd0000)
    libfreetype.so.6 => /usr/lib64/libfreetype.so.6 (0x00002aaab0ede000)
    libtbb.so.2 => /projects/sciteam/balh/opt/x86_64-suse-linux-gcc49/root/6.14.06+pythia8/lib/libtbb.so.2 (0x00002aaab1164000)
    libMultiProc.so => /projects/sciteam/balh/opt/x86_64-suse-linux-gcc49/root/6.14.06+pythia8/lib/libMultiProc.so (0x00002aaab13a1000)
    librt.so.1 => /lib64/librt.so.1 (0x00002aaab15b2000)

Perhaps something is implemented at the level of the C++ GCC compilation? or some specific ROOT flag ?

Hi,

before diving into static linkage, a couple of questions.
Is the path /projects/sciteam/balh/opt/x86_64-suse-linux-gcc49/root/6.14.06+pythia8/lib/ on Lustre?
If yes, could you try to do

echo This warms up caches
cat /projects/sciteam/balh/opt/x86_64-suse-linux-gcc49/root/6.14.06+pythia8/lib/* >& /dev/null

and then execute your program?
Naive question, but may be worth asking: are you sure you need all the linked libraries?

The rationale behind my suggestion is that your Lustre instance may be slow at dynamically loading the libs.

Cheers,
D

Hi Danilo,
It is indeed on lustre.


meyer1@h2ologin2:~> echo This warms up caches
meyer1@h2ologin2:~> cat /projects/sciteam/balh/opt/x86_64-suse-linux-gcc49/root/6.14.06+pythia8/lib/* >& /dev/null
meyer1@h2ologin2:~> time PrintRoot hist-263324.root
|real|0m46.585s|
|---|---|
|user|0m0.684s|
|sys|0m1.808s|

This seems to help a bit. Although 45s for that is extremely slow.
And it might just be not the same conditions as yesterday

Lustre instance performances due to dynamically loading libs are indeed also the suggestions from the cluster local team. They suggested me to use static library if I want to take advantage of the performances of my cluster.

EDIT:
Yeah it seems to be performances fluctuations, after the second try I get this :

|real|0m58.812s|
|---|---|
|user|0m0.728s|
|sys|0m2.080s|

Hi,

you could start by reducing the number of linked libraries. In addition, you may consider relocating your ROOT installation. The reply of your cluster team is fine but it is not always easy nor convenient nor even possible to pack code in static executables. For example ROOT needs to load dynamically libraries upon interpreting or reading files :frowning:

Cheers,
D

trying to statically compile C++ is a pain.
even more so for large and complex libraries like ROOT.

especially b/c nobody (or not many people in HEP) is exercizing this modus operandi, so you’re in for a world of pain.

depending on what PrintRoot does, you may want to try out groot.
groot is by default statically compiled.

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.