Shared libraries in root 6.06.04

ROOT
1

Hi ROOTers.
Apologies to have to bring back up a topic I’ve already seen several instances of in the forums, but despite going through them all I’m still having trouble.
I’m trying to compile a ROOT script that utilizes custom classes. The class definitions are built into a shared library. I load with gSystem->Load(“thelibrary.so”), and in interactive mode, I have no issues. But when trying to compile my ROOT script with .L script.cc+, I get a series of “unknown type name” errors.
I’ve previously had problems and was able to solve them with the magic line R__LOAD_LIBRARY(path/to/library) but this time all this does is print a slew of warnings about attempts to add the native ROOT classes to the TClassTable - but explicitly nothing about the classes I’m actually trying to load.

The dictionary is generated in a makefile via: [code]
ROOTSRC := ./src/WCSimRootEvent.cc ./include/WCSimRootEvent.hh ./src/WCSimRootGeom.cc ./include/WCSimRootGeom.hh ./include/WCSimPmtInfo.hh ./src/WCSimEnumerations.cc ./include/WCSimEnumerations.hh ./include/WCSimRootLinkDef.hh

rootcint: (ROOTSRC) rootcint -f ./src/WCSimRootDict.cc -c -I./include -I(shell root-config --incdir) WCSimRootEvent.hh WCSimRootGeom.hh WCSimPmtInfo.hh WCSimEnumerations.hh WCSimRootLinkDef.hh [/code]
while the library itself is generated via: [code]
ROOTOBJS := (G4WORKDIR)/tmp/(G4SYSTEM)/WCSim/WCSimRootEvent.o (G4WORKDIR)/tmp/(G4SYSTEM)/WCSim/WCSimRootGeom.o (G4WORKDIR)/tmp/(G4SYSTEM)/WCSim/WCSimPmtInfo.o (G4WORKDIR)/tmp/(G4SYSTEM)/WCSim/WCSimEnumerations.o (G4WORKDIR)/tmp/(G4SYSTEM)/WCSim/WCSimRootDict.o

(G4TMPDIR)/%.o: src/%.cc @echo Compiling *.cc …
@if [ ! -d (G4TMPDIR) ] ; then mkdir (G4TMPDIR) ; echo mkdir (G4TMPDIR) ;fi @echo (CXX) (CXXFLAGS) (CPPFLAGS) -c -o (G4TMPDIR)/(*F).o < @(CXX) (CXXFLAGS) (CPPFLAGS) -c -o (G4TMPDIR)/(*F).o $<

libWCSimRoot.so : (ROOTOBJS) @g++ -shared -O ^ -o libWCSimRoot.so $(shell root-config --libs) [/code]

The code I’m using works fine with ROOT 5.34, so presumably the problem lies in the migration to ROOT 6. I think I need a rootmap file to specify the contents of the library? And possibly a pcm file, for I/O of the class? The relevant flags though are a little unclear to me:

In another thread Danilo clarified that [quote]The rml flag specifies the name of the library which should be autoloaded when a class, enum, variable or header file relevant to your selection is encountered[/quote]
That sounds like it should be the file containing my class definitions, libWCSimRoot.so.
Then the -s flag then seems to indicate, if my dictionary file (./src/WCSimRootDict.cc) is compiled into a library, then I should point to that here. Since WCsimRootDict.o is also included in libWCSimRoot.so, then I should also give -s LibWCSimRoot.so …

It seems I can omit the -rmf option, in which case it is “inferred from” the rml file. This could suggest this should be a pre-existing file, and the two must have the appropriate relative naming convention… in which case how do I generate that? But I think it is rather a product to be generated by the rootcint command, and it’s name will be taken from the rml file.

Similarly Danilo clarified that [quote] The pcm file is a special ROOT file and is necessary and should sit next to the library in which the dictionary it refers to has been compiled. It contains metadata to fill the ROOT typesystem at library load time, metadata used to do I/O of selected classes. The -m flag will be necessary in the future when the pcm files will not be ROOT files anymore but rather clang precompiled modules. It allows to specify a hierarchy, a tiered structure, of pre compiled modules.
[/quote]
That somewhat seems to tie up with my interpretation that this is for linking libraries to other libraries, or something similar. Since [quote] The pcm file produced by this invocation will not include any declarations already included in pcm files loaded with -m[/quote] but I don’t have any other .pcm files, and wish this one to contain all my custom classes, I take it I can leave this out.

Ultimately what seems to be missing from my makefile is the options for -s and/or -rml. They both sound like they should be libWCSimRoot.so. In the end I’ve attempted to modify things like this: rootcling -f ./src/WCSimRootDict.cc -s libWCSimRoot.so -rml libWCSimRoot.so -I./include -I$(shell root-config --incdir) WCSimRootEvent.hh WCSimRootGeom.hh WCSimPmtInfo.hh WCSimEnumerations.hh WCSimRootLinkDef.hh

However, upon starting root, with libWCSimRoot.so, libWCSimRootDict.rootmap and WCSimRootDict_rdict.pcm in the current working directory, I get errors error: unknown type name 'WCSimRootEvent'
I have tried with variations of including -s, -rml, -rmf… but have not struck a lucky combination.
Clearly things are a little muddled for me. Can someone please help clarify things?

1 Like
2

Hi,

let me go through the points you mention after a quick introduction.
ROOT can load on demand or automatically shared libraries at runtime. One reason why a particular library is automatically loaded is because cling detects a missing symbol, e.g. a class name which is not known to the interpreter (I leave aside enums, headers and namespaces for the sake of simplicity but the same reasoning applies). We avoid that the compiler error emerges immediately and before it is prompted we try to “autoload” a library which ROOT knows it contains the desired symbol. The mapping between classes’ names and libraries is contained in special files called “rootmap files” which are created automatically by the dictionary generators, rootcling and genreflex. They are ascii, human readable files and ROOT parses them at startup, more precisely when the TROOT singleton is constructed.

The creation of rootmap files is optional but highly recommended.

The rootpcm files are special rootfiles which are generated together with the dictionaries: they can be opened and browsed as any other root file. They contain the information ROOT needs to perform IO operations with the selected classes. They must physically be located next to the libraries which contain the dictionaries associated to them.

As a side note, the generation of rootmaps changed quite a lot from ROOT5 to ROOT6 as far as rootcling/cint is involved. In ROOT5 rootmaps were generated with a separate executable, rlibmap, while in ROOT6 the rootmap files are generated by rootcling/cint. The technical reason behind this is that the entire AST has to be known in order to write down proper rootmap files and that can happen only exposing all the headers to the tool and not only the LinkDef file. As far genreflex is involved, nothing changes.

Now, in some cases, the autoloading mechanism might kick in “too late”. For this reason we introduced the R__LOAD__LIBRARY macro to allow to load unconditionally a library during parsing of the code. Most of the times, this is not needed.

To be now very pragmatic, the command which should make your system work is

rootcling  -f ./src/WCSimRootDict.cc -rml libWCSimRoot.so -I./include -I$(shell root-config --incdir) WCSimRootEvent.hh WCSimRootGeom.hh  WCSimPmtInfo.hh WCSimEnumerations.hh WCSimRootLinkDef.hh

provided that then you build a library called libWCSimRoot.so which includes also the generated WCSimRootDict.cc source file. You will see in the rootmap file the name of the library you required and if the WCSimRootEvent class was selected, the WCSimRootEvent class name as “autoload key”.

If this does not work, check that the library is in the LD_LIBRARY_PATH (or equivalent) and that the WCSimRootEvent name appears in the rootmap file. If this still does not work, do not hesitate to post again.

Cheers,
D

1 Like
3

Hi Danilo,
Thanks for the explanations. I think I understand the gist of most of these commands, although I haven’t actually been using genreflex - is this something I also need to be doing? If not, when, and how does it relate to rootcling’s dictionaries and rootmaps?
I appreciate the remark about R__LOAD__LIBRARY, as I wasn’t clear exactly what was happening with this before.

So, firstly my proposed command looks pretty similar to yours, but is missing the -s flag. As i explained, I had thought this should indicate the location of the library containing the object file built from this dictionary, and this would be libWCSimRoot.so. Could you explain why you omitted it?
Having removed it, things still do not work.
my rootmap file contains the following:

[ libWCSimRoot.so ]
# List of selected classes
class WCSimEnumerations
class WCSimPmtInfo
class WCSimRootCherenkovDigiHit
class WCSimRootCherenkovHit
class WCSimRootCherenkovHitTime
class WCSimRootEvent
class WCSimRootEventHeader
class WCSimRootGeom
class WCSimRootPMT
class WCSimRootPi0
class WCSimRootTrack
class WCSimRootTrigger
header WCSimEnumerations.hh
header WCSimPmtInfo.hh
header WCSimRootEvent.hh
header WCSimRootGeom.hh

I don’t see any explicit mention of “autoload key”.

As something of an aside note, some further info on experiments:
If i open root and try to instantiate a WCSimRootEvent - without calling LoadLibrary - the class name does highlight (it does this if i include the path to the library in LD_LIBRARY_PATH, or if I start root in a directory containing a .rootmap) suggesting the class is known to some degree, but unless I start root in the directory where the dictionary was generated, when I try to instantiate a WCSimRootEvent object, I get the following errors:

Error in cling::AutoloadingVisitor::InsertIntoAutoloadingState:
   Missing FileEntry for WCSimRootEvent.hh
   requested to autoload type WCSimRootCherenkovDigiHit
Error in cling::AutoloadingVisitor::InsertIntoAutoloadingState:
   Missing FileEntry for WCSimRootEvent.hh
   requested to autoload type WCSimRootCherenkovHit
Error in cling::AutoloadingVisitor::InsertIntoAutoloadingState:
   Missing FileEntry for WCSimRootEvent.hh
   requested to autoload type WCSimRootCherenkovHitTime
Error in cling::AutoloadingVisitor::InsertIntoAutoloadingState:
   Missing FileEntry for WCSimRootEvent.hh
   requested to autoload type WCSimRootTrack
Error in cling::AutoloadingVisitor::InsertIntoAutoloadingState:
   Missing FileEntry for WCSimRootEvent.hh
   requested to autoload type WCSimRootEventHeader
Error in cling::AutoloadingVisitor::InsertIntoAutoloadingState:
   Missing FileEntry for WCSimRootEvent.hh
   requested to autoload type WCSimRootTrigger
Error in cling::AutoloadingVisitor::InsertIntoAutoloadingState:
   Missing FileEntry for WCSimRootEvent.hh
   requested to autoload type WCSimRootEvent
Error in cling::AutoloadingVisitor::InsertIntoAutoloadingState:
   Missing FileEntry for WCSimRootEvent.hh
   requested to autoload type WCSimRootPi0
Error in cling::AutoloadingVisitor::InsertIntoAutoloadingState:
   Missing FileEntry for WCSimRootGeom.hh
   requested to autoload type WCSimRootGeom
Error in cling::AutoloadingVisitor::InsertIntoAutoloadingState:
   Missing FileEntry for WCSimRootGeom.hh
   requested to autoload type WCSimRootPMT
Error in cling::AutoloadingVisitor::InsertIntoAutoloadingState:
   Missing FileEntry for WCSimPmtInfo.hh
   requested to autoload type WCSimPmtInfo
Error in cling::AutoloadingVisitor::InsertIntoAutoloadingState:
   Missing FileEntry for WCSimRootEvent.hh
   requested to autoload type WCSimEnumerations
WCSimRootDict dictionary payload:7:10: fatal error: 'WCSimRootEvent.hh' file not found
#include "WCSimRootEvent.hh"
         ^
Error in <TInterpreter::AutoParse>: Error parsing payload code for class WCSimRootEvent with content:

#line 1 "WCSimRootDict dictionary payload"

#ifndef G__VECTOR_HAS_CLASS_ITERATOR
  #define G__VECTOR_HAS_CLASS_ITERATOR 1
#endif

#define _BACKWARD_BACKWARD_WARNING_H
#include "WCSimRootEvent.hh"
#include "WCSimRootGeom.hh"
#include "WCSimPmtInfo.hh"
#include "WCSimEnumerations.hh"

#undef  _BACKWARD_BACKWARD_WARNING_H

Perhaps the rootmap is dependent on the absolute paths to my headers? Do i need to modify the command somehow to provide these?
Incidentally, even if I do start in the directory where i build the dictionary, while I don’t get those errors when instantiating a WCSimRootEvent object, I still can’t compile any analysis scripts.

To throw a final spanner into the works, this is all done by makefiles in the source directory of our application, but i also realized there are CMake commands to make a dictionary in the build directory. I believe these are independent, and for the moment we can leave that aside and just get things working with the Makefiles in the source directory and it’s output. But it would be nice to know how these commands relate to things.
The cmake commands: [code]
ROOT_GENERATE_DICTIONARY(${CMAKE_CURRENT_SOURCE_DIR}/src/WCSimRootDict ${CMAKE_CURRENT_SOURCE_DIR}/include/WCSimRootEvent.hh ${CMAKE_CURRENT_SOURCE_DIR}/include/WCSimRootGeom.hh ${CMAKE_CURRENT_SOURCE_DIR}/include/WCSimPmtInfo.hh ${CMAKE_CURRENT_SOURCE_DIR}/include/WCSimEnumerations.hh LINKDEF ${CMAKE_CURRENT_SOURCE_DIR}/include/WCSimRootLinkDef.hh)

add_library(WCSimRoot SHARED ./src/WCSimRootEvent.cc ./src/WCSimRootGeom.cc ./src/WCSimPmtInfo.cc ./src/WCSimEnumerations.cc ./src/WCSimRootDict.cxx)
target_link_libraries(WCSimRoot ${ROOT_LIBRARIES})
[/code]
This seems to generate an alternate dictionary file (WCSimRootDict.cxx rather than WCSimRootDict.cc) and then link it into the build version of libWCSimRoot.so.

4

Hi,

genreflex and rootcling are the very same executable but with different clis: this is a feature which was implemented to guarantee backward compatibility with the dictionary generators of ROOT5.

I removed everything which is not absolutely vital since there is something not 100% under control here and we need to simplify.

The “autoload keys” are the lines which follow the library name tag. You can see WCSimRootEvent there, so you can see you selected it and it was successfully found and “digested”.

The rootmaps are read only if located in the LD_LIBRARY_PATH on the same footing the library will be found by root only if in the LD_LIRBARY_PATH.
Now, the error you are seeing is there because ROOT is trying to find the header where the interface of your classes are. It needs them for interactivity (to invoke methods, for example). In your case, you need either to run from the directory where these files are or you need to define a variable, ROOT_INCLUDE_PATH, which is analogous to LD_LIBRARY_PATH but refers to the headers needed at runtime and not to the libraries.

I would not drag the ROOT’s CMake into the equation at this point as it is not of core importance for the problem we are solving, despite being very interesting as you note.

Cheers,
Danilo

5

Hi Danilo,
Thanks for the info on genreflex and autoload keys. The terminology isn’t clear to me, so it’s helpful to know things like that.
I’m afraid saying you removed the ‘-s’ flag because it’s not vital doesn’t really clarify things for me. why is it non-vital? when is it vital?
I exported the path to the libWCSimRoot.so in LD_LIBRARY_PATH, and the path to the include folder in ROOT_INCLUDE_PATH. I can now instantiate a WCSimRootEvent in the analysis folder - i.e. not where i compiled the dictionary. However, I can’t compile. As I mentioned previously, even running in the folder where the dictionary was compiled, I can’t compile analysis scripts there either.

6

Hi,

for the -s flag, you can perhaps have a look to the output of “rootcling -h”.

I understand there is progress: you can now interactively use your classes. Now, what do you exactly mean by “cannot compile”?

D

7

Hi Danilo,
I looked at the output of rootcling -h regarding the -s flag and posted my interpretation of it in my opening post. I even quoted it, highlighted the relevant line, and followed up with my interpretation of it. Since the dictionary object is placed in a shared library, i interpret it as needed.

I hate to be a downer, but there’s not really progress. I could already use my classes interactively in root, simply by running gSystem->Load(“libWCSimRoot.so”), even before modifying the makefile.

By cannot compile, I mean, I would like to compile an analysis script analysis.cc using aclic, via

 $root [0] .L /path/to/analysis.cc++g

however when I do so, I receive errors that my classes are not defined. This situation was also described in my opening post. [quote]But when trying to compile my ROOT script with .L script.cc+, I get a series of “unknown type name” errors.[/quote]

8

Hi,

Just wondering, in your project which depends on ROOT, are you building your class dictionary using something like this CMake command?

ROOT_GENERATE_DICTIONARY(EventDict ${Event_INCLUDE_DIR}/Event/Event.h MODULE ${PROJECT_NAME} LINKDEF ${Event_INCLUDE_DIR}/Event/EventLinkDef.h)

Then the pcm file needs to be installed alongside the library so ROOT can find it.

install(FILES ${CMAKE_CURRENT_BINARY_DIR}/lib${PROJECT_NAME}_rdict.pcm DESTINATION ${CMAKE_INSTALL_PREFIX}/lib)

Using this approach, I have not had any problems with the classes being loaded. If you are not currently using CMake in your dependent project, you might consider using it, as the ROOT macros nicely hide all the gory details from you. :slight_smile:

–Jeremy

9

Hi Jeremy,
Thanks for taking the time to give suggestions. The main project does indeed use CMake, but for some reason (perhaps legacy?) the documentation still says the root dictionaries should be built with Makefiles in the source directory. It wasn’t until later that I realized there are also commands in the CMakeLists file to build a root dictionary. Unfortunately neither set of libraries, dictionaries or pcm files appear to work.

I actually posted the CMake command we have. It’s pretty similar, although I notice the absence of a MODULE specification, but I don’t expect that to be the issue.
Unfortunately even with the pcm file next to the library, I still have problems. Perhaps when we get to the bottom of this, migrating to CMake will provide a neater solution.

10

Hi,

I think I lost track of what problem you are facing now.
Let me try to recap:

  • You have the pcm and the rootmap file sitting in the same directory of the library
  • You have correctly set the LD_LIBRARY_PATH
  • You have correctly set the ROOT_INCLUDE_PATH

What is the error you are seeing when doing what?

We’ll get at the bottom of this :slight_smile:

Cheers,
D

11

Hi Danilo,
Thanks for persevering!
Yes, the pcm, rootmap and library are in the same directory (the source directory, in which they are built).
This path (to the source directory) is exported to LD_LIBRARY_PATH
ROOT_INCLUDE_PATH is specified as source_directory/include

When running root with this configuration, when attempting to compile a c++ macro with ACLIC the custom classes are not found. The reported error is: error: unknown type name 'WCSimRootEvent'
and similar for all other custom classes used.

12

For reference, as silly as it may be, a simplified script I’m trying to compile would be:

void simple_testscript(char *filename=NULL)
{
  WCSimRootEvent* wcsimrootsuperevent = new WCSimRootEvent();
}

this is sufficient to generate the complaint regarding the WCSimRootEvent class.

13

Hi,

is this a compilation error? Do you include the right header? What happens if you do not aclic?

D

14

Hi Danilo. Yes, I believe this is a compiler error. As specified, the errors are returned when I execute the command $root[0] .L analysisfilename.cc+, where, as i understand it, the ‘+’ here is to load and compile the file. If I omit the +, the file loads without a problem.

15

Did you add on top of your script?

#include "WCSimRootEvent.hh"
16

No, admittedly i haven’t got to doing that. I do provide ROOT_INCLUDE_PATH, which is the folder containing the header file though. I’d have thought that’d be sufficient…
This does indeed work, (although it requires the path to headers is hard-coded into the analysis script).

17

I see.
The point is that ROOT_INCLUDE_PATH is necessary to the root interpreter while if you use ACLIC (.L xxx.C+) you are transparently invoking the compiler available on your system to create dictionaries and a shared library and load it.

Cheers,
D

18

If you don’t want to hard code the full path to the header, you can do either in your .bashrc

export CPLUS_INCLUDE_PATH=$CPLUS_INCLUDE_PATH:pathtoyourincludefolder

or in your rootlogon.C

gInterpreter->AddIncludePath(pathtoyourincludefolder);
19

Hi,

as ferhue says, these are two ways to make your headers available.
Just remember that CPLUS_INCLUDE_PATH is a very general variable, always picked up by ROOT (cling), clang and gcc.

Cheers,
D