Distributing ROOT binaries with my program

ROOT
1

Hello!

I developed a C++ program for consumers that internally depends on ROOT. I am now about to produce the first release of my app since ROOT was integrated, and encountered a lot of packaging issues. For instance, using ProcessMonitor I found that my program crashes on startup unless there is a etc directory exists one level above the executable. In this directory, ROOT seems to look for files like allDicts.pch and others.

I can of course try to reverse-engineer this, but that seems like a wrong approach. Instead, I would like to ask if there exist any guidelines for correctly redistributing ROOT binaries in software packages. What is the minimal possible set of files I have to include in addition to libraries like libCore, libThread etc.? Has anybody attempted to do this before?

I would also like to ask about legal considerations. Is it sufficient if I attribute ROOT academically and include ROOT’s license together with all 3rd party software licenses my software uses?

Thanks for any advice you can share!
Best, Petr

ROOT Version: 6.26/10
Platform: Windows SDK 10.0.22621.0 (amd64)
Compiler: MSVC 19.38.33134.0

2

Hello @pmanek,
thanks you for reaching out!

I believe that the minimal possible set of files you should include depends on what your application does. Can you share more details about it?

The ROOT system is being made available under the LGPL 2.1. I read the following:

You may copy, distribute and modify the software provided that you state modifications and license them under LGPL-2.1. Anything statically linked to the library can only be redistributed under LGPL, but applications that use the library don't have to be. You must allow reverse engineering of your application as necessary to debug and relink the library.

So, I guess it is sufficient to distribute your app together with ROOT binaries and their source code.

Best,
Monica

3

I believe that the minimal possible set of files you should include depends on what your application does. Can you share more details about it?

Sure. I have a Qt-based C++ program, which currently links against the following ROOT libraries:

  • libCore
  • libThread
  • libTree
  • libRIO

As for behavior, my current integration with ROOT is as simple as reading/writing TFile objects that contain multiple instances of TTree.

My current packaging script takes the main executable and its dependent DLLs, which are automatically discovered and processed using the windeployqt utility. These files are deposited in the distribution package. As I found, for some reason ROOT libraries are somehow missed by this process.

By trial and error, I found that the following files need to appear in the same directory as my executable:

  • libCore.dll
  • libThread.dll
  • libTree.dll
  • libRIO.dll
  • libNet.dll
  • libImt.dll
  • tbb.dll
  • libCling.dll

Upon startup, it seems that something in libCling.dll looks for a directory named etc in .., then in ../.., then in ../../.. and so on until the path C:\etc is reached. If none of these directories exist, the executable crashes on startup and never opens a GUI window. Otherwise, ROOT loads various file from within this directory.

4

I don’t know what may cause this behavior, I add in the loop our windows expert @bellenot and cling expert @vvassilev as I think they may help…

5

AFAIK, ROOT needs the ${ROOTSYS}/etc directory (and a few other if I’m not mistaking, depending on what you need)

6

AFAIK, ROOT needs the ${ROOTSYS}/etc directory (and a few other if I’m not mistaking, depending on what you need)

Can I resolve this by setting ROOTSYS to a directory within my distributed package, e.g. something like ./root/etc in a directory structure like so:

  • MyExecutable.exe
  • libCore.dll, libTree.dll, …
  • root/
    • etc/

And if so, can this be done programmatically, so that we do not need users to reconfigure environment just because of our program?

7

Well, unless someone already tried this and can reply, I’m afraid you’ll have to experiment this yourself…

8

Thanks for clarifying. I have to admit that I’m a little surprised that this has not come up before.

From your point of view, by including ROOT with my software, am I attempting to do something non-standard or unsupported? If so, is there a better way to deliver my ROOT-dependent software to my users?

9

Yes, ROOTSYS/etc is a folder which host some of the private headers and optimization data structures of ROOT. If you want things to be more distributable you should use -Druntime_cxxmodules=On which will not remove the need of allDicts.pch. However, this approach might not work on Windows.

10

ROOT is at the core of many software stacks at the LHC experiments. It supports such use cases, however that’s still somewhat an expert feature and requires a lot of care.

11

I don’t think so. If you search the forum you will most probably find something… For example Distribution of Root dlls etc

12

Thanks. I will look into that.

Besides ${ROOTSYS}/etc, is there any other directory or set of files that I should not forget about? Something that is perhaps not necessarily needed immediately upon startup, but may cause a runtime crash at a later point?

13

Maybe also fonts and icons

14

Yes, that makes sense, I suppose. Reading through Philippe’s answer in the thread you referenced, the following statement stands out to me:

In most cases, the gain in disk that comes with this pruning is not worth the maintenance effort to properly prune the directories.

I will probably do things the other way around. Instead of distribute a very specific subset of ROOT distribution package, I will ship the entire package and remove parts that my users will not need (manuals, include headers etc.)

Now my only challenge is to figure out how to configure the value of ROOTSYS programmatically before the dynamic linker runs. I can make a wrapper executable, which will just call setenv() and then execve() my current executable, but that seems a little too cumbersome. Would you have any ideas how to accomplish this more efficiently?

15

Unfortunately no, and I’m working on Windows…

16

FYI, I think I just had an idea how this could be accomplished better:

  1. I could move all ROOT-dependent features to an auxiliary library, leaving my main executable completely independent on ROOT.
  2. Then I could simply setenv("ROOTSYS") in my main function before this auxiliary library is loaded.
  3. Once the auxiliary library is loaded, dynamic linker will automatically load ROOT.
  4. ROOT will examine ROOTSYS and find that value of the variable is already configured to the desired location within my distributed package.
17

I have a follow up question. I have implemented the approach I described, but am still wrestling with the behavior of libCling.so. It seems that the library is included as a second-order dependency of some other ROOT libraries I am using. When it is loaded, strace shows it looks for header files in /build/root even though $ROOTSYS has been previously set.

Here is an example:

stat("/build/root/src/build/include/TException.h", 0x7fff0aaf7358) = -1 ENOENT (No such file or directory)
stat("/build/root/src/build/include/TExec.h", 0x7fff0aaf7308) = -1 ENOENT (No such file or directory)
stat("/build/root/src/build/include/TExec.h", 0x7fff0aaf7358) = -1 ENOENT (No such file or directory)
stat("/build/root/src/build/include/TFolder.h", 0x7fff0aaf7308) = -1 ENOENT (No such file or directory)
stat("/build/root/src/build/include/TFolder.h", 0x7fff0aaf7358) = -1 ENOENT (No such file or directory)
stat("/build/root/src/build/include/TMemberInspector.h", 0x7fff0aaf7308) = -1 ENOENT (No such file or directory)
stat("/build/root/src/build/include/TMemberInspector.h", 0x7fff0aaf7358) = -1 ENOENT (No such file or directory)
stat("/build/root/src/build/include/TMessageHandler.h", 0x7fff0aaf7308) = -1 ENOENT (No such file or directory)
stat("/build/root/src/build/include/TMessageHandler.h", 0x7fff0aaf7358) = -1 ENOENT (No such file or directory)
stat("/build/root/src/build/include/TPluginManager.h", 0x7fff0aaf7308) = -1 ENOENT (No such file or directory)
stat("/build/root/src/build/include/TPluginManager.h", 0x7fff0aaf7358) = -1 ENOENT (No such file or directory)
stat("/build/root/src/build/include/TPRegexp.h", 0x7fff0aaf7308) = -1 ENOENT (No such file or directory)
stat("/build/root/src/build/include/TPRegexp.h", 0x7fff0aaf7358) = -1 ENOENT (No such file or directory)
stat("/build/root/src/build/include/TProcessUUID.h", 0x7fff0aaf7308) = -1 ENOENT (No such file or directory)
stat("/build/root/src/build/include/TProcessUUID.h", 0x7fff0aaf7358) = -1 ENOENT (No such file or directory)
stat("/build/root/src/build/include/TQClass.h", 0x7fff0aaf7308) = -1 ENOENT (No such file or directory)
stat("/build/root/src/build/include/TQClass.h", 0x7fff0aaf7358) = -1 ENOENT (No such file or directory)
stat("/build/root/src/build/include/TQCommand.h", 0x7fff0aaf7308) = -1 ENOENT (No such file or directory)
stat("/build/root/src/build/include/TQCommand.h", 0x7fff0aaf7358) = -1 ENOENT (No such file or directory)
stat("/build/root/src/build/include/TQConnection.h", 0x7fff0aaf7308) = -1 ENOENT (No such file or directory)
stat("/build/root/src/build/include/TQConnection.h", 0x7fff0aaf7358) = -1 ENOENT (No such file or directory)
stat("/build/root/src/build/include/TRedirectOutputGuard.h", 0x7fff0aaf7308) = -1 ENOENT (No such file or directory)
stat("/build/root/src/build/include/TRedirectOutputGuard.h", 0x7fff0aaf7358) = -1 ENOENT (No such file or directory)
stat("/build/root/src/build/include/TRemoteObject.h", 0x7fff0aaf7308) = -1 ENOENT (No such file or directory)
stat("/build/root/src/build/include/TRemoteObject.h", 0x7fff0aaf7358) = -1 ENOENT (No such file or directory)
stat("/build/root/src/build/include/TRootIOCtor.h", 0x7fff0aaf7308) = -1 ENOENT (No such file or directory)
stat("/build/root/src/build/include/TRootIOCtor.h", 0x7fff0aaf7358) = -1 ENOENT (No such file or directory)
stat("/build/root/src/build/include/TStringLong.h", 0x7fff0aaf7308) = -1 ENOENT (No such file or directory)
stat("/build/root/src/build/include/TStringLong.h", 0x7fff0aaf7358) = -1 ENOENT (No such file or directory)
stat("/build/root/src/build/include/TSystemDirectory.h", 0x7fff0aaf7308) = -1 ENOENT (No such file or directory)
stat("/build/root/src/build/include/TSystemDirectory.h", 0x7fff0aaf7358) = -1 ENOENT (No such file or directory)
stat("/build/root/src/build/include/TSystemFile.h", 0x7fff0aaf7308) = -1 ENOENT (No such file or directory)
stat("/build/root/src/build/include/TSystemFile.h", 0x7fff0aaf7358) = -1 ENOENT (No such file or directory)
stat("/build/root/src/build/include/TTask.h", 0x7fff0aaf7308) = -1 ENOENT (No such file or directory)
stat("/build/root/src/build/include/TTask.h", 0x7fff0aaf7358) = -1 ENOENT (No such file or directory)
stat("/build/root/src/build/include/TThreadSlots.h", 0x7fff0aaf7308) = -1 ENOENT (No such file or directory)
stat("/build/root/src/build/include/TThreadSlots.h", 0x7fff0aaf7358) = -1 ENOENT (No such file or directory)

My project does not need a live interpreter, is it possible to disable this behavior? And if not, is it possible to at least force libCling.so to honor the value of $ROOTSYS?

Edit: I found this related issue

18

Then maybe @vvassilev or @hahnjo can help…

19

One more observation. I did some digging around using ldd and I am stumped why libCling is loaded in the first place.

As far as I can tell, this is my full chain of dependencies:

/usr/lib/root/libCore.so (interpreter => None)
    libpcre2-8.so.0 => /usr/lib/libpcre2-8.so.0
    libz.so.1 => /usr/lib/libz.so.1
    liblzma.so.5 => /usr/lib/liblzma.so.5
    libxxhash.so.0 => /usr/lib/libxxhash.so.0
    liblz4.so.1 => /usr/lib/liblz4.so.1
    libzstd.so.1 => /usr/lib/libzstd.so.1
    libstdc++.so.6 => /usr/lib/libstdc++.so.6
    libm.so.6 => /usr/lib/libm.so.6
    libgcc_s.so.1 => /usr/lib/libgcc_s.so.1
    libc.so.6 => /usr/lib/libc.so.6
    ld-linux-x86-64.so.2 => /usr/lib/ld-linux-x86-64.so.2
/usr/lib/root/libThread.so (interpreter => None)
    libCore.so => /usr/lib/root/libCore.so
        libpcre2-8.so.0 => /usr/lib/libpcre2-8.so.0
        libz.so.1 => /usr/lib/libz.so.1
        liblzma.so.5 => /usr/lib/liblzma.so.5
        libxxhash.so.0 => /usr/lib/libxxhash.so.0
        liblz4.so.1 => /usr/lib/liblz4.so.1
        libzstd.so.1 => /usr/lib/libzstd.so.1
        libm.so.6 => /usr/lib/libm.so.6
    libtbb.so.12 => /usr/lib/libtbb.so.12
    libstdc++.so.6 => /usr/lib/libstdc++.so.6
    libgcc_s.so.1 => /usr/lib/libgcc_s.so.1
    libc.so.6 => /usr/lib/libc.so.6
    ld-linux-x86-64.so.2 => /usr/lib/ld-linux-x86-64.so.2
/usr/lib/root/libTree.so (interpreter => None)
    libImt.so => /usr/lib/root/libImt.so
        libMultiProc.so => /usr/lib/root/libMultiProc.so
        libtbb.so.12 => /usr/lib/libtbb.so.12
            ld-linux-x86-64.so.2 => /usr/lib/ld-linux-x86-64.so.2
    libNet.so => /usr/lib/root/libNet.so
        libssl.so.3 => /usr/lib/libssl.so.3
        libcrypto.so.3 => /usr/lib/libcrypto.so.3
    libRIO.so => /usr/lib/root/libRIO.so
        liburing.so.2 => /usr/lib/liburing.so.2
    libThread.so => /usr/lib/root/libThread.so
    libCore.so => /usr/lib/root/libCore.so
        libpcre2-8.so.0 => /usr/lib/libpcre2-8.so.0
        libz.so.1 => /usr/lib/libz.so.1
        liblzma.so.5 => /usr/lib/liblzma.so.5
        libxxhash.so.0 => /usr/lib/libxxhash.so.0
        liblz4.so.1 => /usr/lib/liblz4.so.1
        libzstd.so.1 => /usr/lib/libzstd.so.1
    libstdc++.so.6 => /usr/lib/libstdc++.so.6
    libm.so.6 => /usr/lib/libm.so.6
    libgcc_s.so.1 => /usr/lib/libgcc_s.so.1
    libc.so.6 => /usr/lib/libc.so.6
/usr/lib/root/libRIO.so (interpreter => None)
    libThread.so => /usr/lib/root/libThread.so
        libtbb.so.12 => /usr/lib/libtbb.so.12
    liburing.so.2 => /usr/lib/liburing.so.2
    libCore.so => /usr/lib/root/libCore.so
        libpcre2-8.so.0 => /usr/lib/libpcre2-8.so.0
        libz.so.1 => /usr/lib/libz.so.1
        liblzma.so.5 => /usr/lib/liblzma.so.5
        libxxhash.so.0 => /usr/lib/libxxhash.so.0
        liblz4.so.1 => /usr/lib/liblz4.so.1
        libzstd.so.1 => /usr/lib/libzstd.so.1
    libstdc++.so.6 => /usr/lib/libstdc++.so.6
    libm.so.6 => /usr/lib/libm.so.6
    libgcc_s.so.1 => /usr/lib/libgcc_s.so.1
    libc.so.6 => /usr/lib/libc.so.6
    ld-linux-x86-64.so.2 => /usr/lib/ld-linux-x86-64.so.2
/usr/lib/root/libHist.so (interpreter => None)
    libMatrix.so => /usr/lib/root/libMatrix.so
    libMathCore.so => /usr/lib/root/libMathCore.so
        libImt.so => /usr/lib/root/libImt.so
            libMultiProc.so => /usr/lib/root/libMultiProc.so
                libNet.so => /usr/lib/root/libNet.so
                    libssl.so.3 => /usr/lib/libssl.so.3
                    libcrypto.so.3 => /usr/lib/libcrypto.so.3
            libtbb.so.12 => /usr/lib/libtbb.so.12
    libRIO.so => /usr/lib/root/libRIO.so
        libThread.so => /usr/lib/root/libThread.so
        liburing.so.2 => /usr/lib/liburing.so.2
    libCore.so => /usr/lib/root/libCore.so
        libpcre2-8.so.0 => /usr/lib/libpcre2-8.so.0
        libz.so.1 => /usr/lib/libz.so.1
        liblzma.so.5 => /usr/lib/liblzma.so.5
        libxxhash.so.0 => /usr/lib/libxxhash.so.0
        liblz4.so.1 => /usr/lib/liblz4.so.1
        libzstd.so.1 => /usr/lib/libzstd.so.1
    libstdc++.so.6 => /usr/lib/libstdc++.so.6
    libm.so.6 => /usr/lib/libm.so.6
    libgcc_s.so.1 => /usr/lib/libgcc_s.so.1
    libc.so.6 => /usr/lib/libc.so.6
    ld-linux-x86-64.so.2 => /usr/lib/ld-linux-x86-64.so.2

How does libCling.so get loaded, does any of the listed libraries dlopen() it explicitly?

20

See root/core/README at 27def95637ad650c746275c0f5df5ab88f449ef6 · root-project/root · GitHub