CMake project with custom libraries and python bindings

Dear ROOT experts,

I have a CMake project on my mac-os which was able to be compiled and i was able to create custom object in python via the pyROOT bindings until root 6.18.
I recently upgraded ROOT and compiled it from source using v6-22-00-patches.
When i did this , the code i had successfully compile.
My CMake root-based project “deeply” use the generate_dictionary functionalities to bring my classes into python and to customly load libraries in some MACROS we have.
Then i had a package called kernel for example, which i compile into a library which should be able to be imported and being used inside a python session.

Now, what i see happening is that i can compile the code, the dictionaries are produced properly, however if inside a python session I run

[0] import ROOT as r
[1] r.gSystem.LoadLibraries("libkernel")
[2] e = r.EventType()

where EventType is supposed to be a class i build and for which a dictionary was generated.

the error i get is the following

In [3]: r.gSystem.Load("kernel/libkernel")

RooFit v3.60 -- Developed by Wouter Verkerke and David Kirkby
                Copyright (C) 2000-2013 NIKHEF, University of California & Stanford University
                All rights reserved, please read http://roofit.sourceforge.net/license.txt
Out[3]: 0
In [4]: r.gSystem.Load("kernel/libkernel.dylib")
Out[4]: 1

In [5]: r.EventType()
 *** Break *** segmentation violation
[/Users/lpnhe/root_build/lib/libCore.so] TUnixSystem::DispatchSignals(ESignals) (no debug info)
[/usr/lib/system/libsystem_platform.dylib] _sigtramp (no debug info)
[/Users/lpnhe/root_build/lib/libCling.so] clang::DeclContext::lookup(clang::DeclarationName) const (no debug info)
[/Users/lpnhe/root_build/lib/libCling.so] clang::RecursiveASTVisitor<cling::(anonymous namespace)::StaticVarCollector>::dataTraverseNode(clang::Stmt*, llvm::SmallVectorImpl<llvm::PointerIntPair<clang::Stmt*, 1u, bool, llvm::PointerLikeTypeTraits<clang::Stmt*>, llvm::PointerIntPairInfo<clang::Stmt*, 1u, llvm::PointerLikeTypeTraits<clang::Stmt*> > > >*) (no debug info)
[/Users/lpnhe/root_build/lib/libCling.so] clang::RecursiveASTVisitor<cling::(anonymous namespace)::StaticVarCollector>::TraverseStmt(clang::Stmt*, llvm::SmallVectorImpl<llvm::PointerIntPair<clang::Stmt*, 1u, bool, llvm::PointerLikeTypeTraits<clang::Stmt*>, llvm::PointerIntPairInfo<clang::Stmt*, 1u, llvm::PointerLikeTypeTraits<clang::Stmt*> > > >*) (no debug info)
[/Users/lpnhe/root_build/lib/libCling.so] clang::RecursiveASTVisitor<cling::(anonymous namespace)::StaticVarCollector>::TraverseDecl(clang::Decl*) (no debug info)
[/Users/lpnhe/root_build/lib/libCling.so] clang::RecursiveASTVisitor<cling::(anonymous namespace)::StaticVarCollector>::dataTraverseNode(clang::Stmt*, llvm::SmallVectorImpl<llvm::PointerIntPair<clang::Stmt*, 1u, bool, llvm::PointerLikeTypeTraits<clang::Stmt*>, llvm::PointerIntPairInfo<clang::Stmt*, 1u, llvm::PointerLikeTypeTraits<clang::Stmt*> > > >*) (no debug info)
[/Users/lpnhe/root_build/lib/libCling.so] cling::DeclUnloader::VisitFunctionDecl(clang::FunctionDecl*) (no debug info)
[/Users/lpnhe/root_build/lib/libCling.so] cling::TransactionUnloader::RevertTransaction(cling::Transaction*) (no debug info)
[/Users/lpnhe/root_build/lib/libCling.so] cling::Interpreter::unload(cling::Transaction&) (no debug info)
[/Users/lpnhe/root_build/lib/libCling.so] cling::IncrementalParser::commitTransaction(llvm::PointerIntPair<cling::Transaction*, 2u, cling::IncrementalParser::EParseResult, llvm::PointerLikeTypeTraits<cling::Transaction*>, llvm::PointerIntPairInfo<cling::Transaction*, 2u, llvm::PointerLikeTypeTraits<cling::Transaction*> > >&, bool) (no debug info)
[/Users/lpnhe/root_build/lib/libCling.so] cling::IncrementalParser::Compile(llvm::StringRef, cling::CompilationOptions const&) (no debug inf

I can share the reduced code i am using (CMakes and header/source files ) if needed. It’s part of a private code analysis, thus i can send it only via mail.
Thanks in advance,
Renato

A crash there is a bit unusual. One possible cause is that ROOT and/or the dependent libraries (including the dictionary source files and pcm files) have not been completely rebuild or that older versions are being picked up (because they are on the LD_LIBRARY_PATH or similar).

If you rebuilt everything from scratch already, you can try to reproduce the crash in the debugger and get it to give a list of shared libraries used (for example “info shared” with gdb) and double check only the expected libraries are loaded.

Cheers,
Philippe.

Dear @pcanal,
I managed to figure out one possible reason of the crash.

If i compile my code , i get as output something like :

w-r--r--   1 lpnhe  staff    42K 22 Dic 13:36 CMakeCache.txt
-rw-r--r--   1 lpnhe  staff   2,5K 22 Dic 13:36 cmake_install.cmake
-rwxr-xr-x   1 lpnhe  staff   136K 22 Dic 13:53 toyGenerator.out
-rwxr-xr-x   1 lpnhe  staff   160K 22 Dic 13:53 evaluateIsoBinCuts.out
-rwxr-xr-x   1 lpnhe  staff   127K 22 Dic 13:53 testKernel.out
-rwxr-xr-x   1 lpnhe  staff   140K 22 Dic 13:53 efficiencyCreateScan.out
-rwxr-xr-x   1 lpnhe  staff   175K 22 Dic 13:53 toyStudy.out
-rwxr-xr-x   1 lpnhe  staff   181K 22 Dic 13:53 testFitComponent.out
-rwxr-xr-x   1 lpnhe  staff   163K 22 Dic 13:53 fitGenerator.out
-rwxr-xr-x   1 lpnhe  staff   137K 22 Dic 13:53 tupleCreate.out
-rwxr-xr-x   1 lpnhe  staff   393K 22 Dic 13:53 optimizeSelectionRK.out
-rwxr-xr-x   1 lpnhe  staff   201K 22 Dic 13:53 tupleProcess.out
-rwxr-xr-x   1 lpnhe  staff   133K 22 Dic 13:53 testEventType.out
-rwxr-xr-x   1 lpnhe  staff   179K 22 Dic 13:53 efficiencyCreatePlots.out
-rwxr-xr-x   1 lpnhe  staff   134K 22 Dic 13:53 loadEventType.out
-rwxr-xr-x   1 lpnhe  staff   409K 22 Dic 13:53 optimizeSelectionRKst.out
-rwxr-xr-x   1 lpnhe  staff   262K 22 Dic 13:53 tupleLumi.out
-rwxr-xr-x   1 lpnhe  staff   1,2M 22 Dic 13:53 efficiencyCreateFast.out
-rwxr-xr-x   1 lpnhe  staff   927K 22 Dic 13:53 efficiencyCreate.out
lrwxr-xr-x   1 lpnhe  staff    38B 22 Dic 13:53 libefficiencies_rdict.pcm -> efficiencies/libefficiencies_rdict.pcm
lrwxr-xr-x   1 lpnhe  staff    26B 22 Dic 13:53 libfitter_rdict.pcm -> fitter/libfitter_rdict.pcm
lrwxr-xr-x   1 lpnhe  staff    26B 22 Dic 13:53 libkernel_rdict.pcm -> kernel/libkernel_rdict.pcm
lrwxr-xr-x   1 lpnhe  staff    38B 22 Dic 13:53 liboptimization_rdict.pcm -> optimization/liboptimization_rdict.pcm
lrwxr-xr-x   1 lpnhe  staff    26B 22 Dic 13:53 libroofit_rdict.pcm -> roofit/libroofit_rdict.pcm
lrwxr-xr-x   1 lpnhe  staff    22B 22 Dic 13:53 libtoys_rdict.pcm -> toys/libtoys_rdict.pcm
lrwxr-xr-x   1 lpnhe  staff    26B 22 Dic 13:53 libtuples_rdict.pcm -> tuples/libtuples_rdict.pcm
lrwxr-xr-x   1 lpnhe  staff    22B 22 Dic 13:53 libvelo_rdict.pcm -> velo/libvelo_rdict.pcm
lrwxr-xr-x   1 lpnhe  staff    36B 22 Dic 13:53 libefficiencies.rootmap -> efficiencies/libefficiencies.rootmap
lrwxr-xr-x   1 lpnhe  staff    24B 22 Dic 13:53 libfitter.rootmap -> fitter/libfitter.rootmap
lrwxr-xr-x   1 lpnhe  staff    24B 22 Dic 13:53 libkernel.rootmap -> kernel/libkernel.rootmap
lrwxr-xr-x   1 lpnhe  staff    36B 22 Dic 13:53 liboptimization.rootmap -> optimization/liboptimization.rootmap
lrwxr-xr-x   1 lpnhe  staff    24B 22 Dic 13:53 libroofit.rootmap -> roofit/libroofit.rootmap
lrwxr-xr-x   1 lpnhe  staff    20B 22 Dic 13:53 libtoys.rootmap -> toys/libtoys.rootmap
lrwxr-xr-x   1 lpnhe  staff    24B 22 Dic 13:53 libtuples.rootmap -> tuples/libtuples.rootmap
lrwxr-xr-x   1 lpnhe  staff    20B 22 Dic 13:53 libvelo.rootmap -> velo/libvelo.rootmap
lrwxr-xr-x   1 lpnhe  staff    34B 22 Dic 13:53 libefficiencies.dylib -> efficiencies/libefficiencies.dylib
lrwxr-xr-x   1 lpnhe  staff    22B 22 Dic 13:53 libfitter.dylib -> fitter/libfitter.dylib
lrwxr-xr-x   1 lpnhe  staff    22B 22 Dic 13:53 libkernel.dylib -> kernel/libkernel.dylib
lrwxr-xr-x   1 lpnhe  staff    34B 22 Dic 13:53 liboptimization.dylib -> optimization/liboptimization.dylib
lrwxr-xr-x   1 lpnhe  staff    22B 22 Dic 13:53 libroofit.dylib -> roofit/libroofit.dylib
lrwxr-xr-x   1 lpnhe  staff    18B 22 Dic 13:53 libtoys.dylib -> toys/libtoys.dylib
lrwxr-xr-x   1 lpnhe  staff    22B 22 Dic 13:53 libtuples.dylib -> tuples/libtuples.dylib
lrwxr-xr-x   1 lpnhe  staff    18B 22 Dic 13:53 libvelo.dylib -> velo/libvelo.dylib
drwxr-xr-x   9 lpnhe  staff   288B 22 Dic 13:56 roofit
drwxr-xr-x   9 lpnhe  staff   288B 22 Dic 13:56 velo
drwxr-xr-x   7 lpnhe  staff   224B 22 Dic 13:56 kernel
drwxr-xr-x   9 lpnhe  staff   288B 22 Dic 13:56 tuples
drwxr-xr-x   9 lpnhe  staff   288B 22 Dic 13:56 optimization
drwxr-xr-x   9 lpnhe  staff   288B 22 Dic 13:56 efficiencies
drwxr-xr-x   9 lpnhe  staff   288B 22 Dic 13:56 fitter
drwxr-xr-x   9 lpnhe  staff   288B 22 Dic 13:56 toys
drwxr-xr-x  26 lpnhe  staff   832B 22 Dic 13:56 CMakeFiles
-rw-r--r--   1 lpnhe  staff   393K 22 Dic 13:56 build.ninja

We were creating some soft links in the past for the set of libraries we generate.

Now, if I sit in the build/Darwinf folder and i run

r.>>> import os
>>> r.gSystem.Load("kernel/libkernel")

RooFit v3.60 -- Developed by Wouter Verkerke and David Kirkby
                Copyright (C) 2000-2013 NIKHEF, University of California & Stanford University
                All rights reserved, please read http://roofit.sourceforge.net/license.txt

0
>>> r.ConfigHodler()

It crashes
However, if i cd ./kernel where those files are present :

🍺  lpnhe 🍺  ~/Desktop/Lavoro/2020/ewp-rkstz/analysis/build/Darwin/kernel (master_q2smearing_andOA_investigation)
$|=>ls -lhtr
total 10760
drwxr-xr-x  3 lpnhe  staff    96B 22 Dic 13:55 CMakeFiles
-rw-r--r--  1 lpnhe  staff   1,2K 22 Dic 13:55 cmake_install.cmake
-rw-r--r--  1 lpnhe  staff    14K 22 Dic 13:56 libkernelDict_rdict.pcm
-rw-r--r--  1 lpnhe  staff   2,3K 22 Dic 13:56 libkernelDict.rootmap
-rw-r--r--  1 lpnhe  staff   6,6K 22 Dic 14:01 module.modulemap
-rw-r--r--  1 lpnhe  staff    14K 22 Dic 14:02 libkernel_rdict.pcm
-rw-r--r--  1 lpnhe  staff   2,3K 22 Dic 14:02 libkernel.rootmap
-rw-r--r--  1 lpnhe  staff   261K 22 Dic 14:02 kernelDict.cxx
-rwxr-xr-x  1 lpnhe  staff   4,9M 22 Dic 14:02 libkernel.dylib

The code doesn’t crash.

In practice I think , the issue i have concern this :

Each library i compile produce a set of directories in my build area

🍺  lpnhe 🍺  ~/Desktop/Lavoro/2020/ewp-rkstz/analysis/build/Darwin (master_q2smearing_andOA_investigation)
ls - lhtr kernel/
total 10720
-rw-r--r--  1 lpnhe  staff   6,6K 22 Dic 14:13 module.modulemap
drwxr-xr-x  3 lpnhe  staff    96B 22 Dic 14:13 CMakeFiles
-rw-r--r--  1 lpnhe  staff   1,2K 22 Dic 14:13 cmake_install.cmake
-rw-r--r--  1 lpnhe  staff    14K 22 Dic 14:14 libkernel_rdict.pcm
-rw-r--r--  1 lpnhe  staff   2,3K 22 Dic 14:14 libkernel.rootmap
-rw-r--r--  1 lpnhe  staff   261K 22 Dic 14:14 kernelDict.cxx
-rwxr-xr-x  1 lpnhe  staff   4,9M 22 Dic 14:16 libkernel.dylib
🍺  lpnhe 🍺  ~/Desktop/Lavoro/2020/ewp-rkstz/analysis/build/Darwin (master_q2smearing_andOA_investigation)
$|=>ls -lhtr fitter
total 6968
-rw-r--r--  1 lpnhe  staff   2,4K 22 Dic 14:13 module.modulemap
drwxr-xr-x  3 lpnhe  staff    96B 22 Dic 14:13 CMakeFiles
-rw-r--r--  1 lpnhe  staff   1,2K 22 Dic 14:13 cmake_install.cmake
-rw-r--r--  1 lpnhe  staff    16K 22 Dic 14:13 libfitter_rdict.pcm
-rw-r--r--  1 lpnhe  staff   695B 22 Dic 14:13 libfitter.rootmap
-rw-r--r--  1 lpnhe  staff   205K 22 Dic 14:13 fitterDict.cxx
-rwxr-xr-x  1 lpnhe  staff   3,1M 22 Dic 14:18 libfitter.dylib

for example for the fitter library and the kernel one.
Both folders has this auto-generated module.modulemap file, which has to be exposed next to the libXX. files.
This becomes impossible to do in a clear manner with soft links as I do, thus i wonder how i can eventually bypass this behaviour.

For example if in my build directory i soft link

lrwxr-xr-x   1 lpnhe  staff    21B 22 Dic 14:33 kernelDict.cxx -> kernel/kernelDict.cxx
lrwxr-xr-x   1 lpnhe  staff    23B 22 Dic 14:34 module.modulemap -> kernel/module.modulemap
lrwxr-xr-x   1 lpnhe  staff    24B 22 Dic 14:18 libkernel.rootmap -> kernel/libkernel.rootmap

i can succesffully import stuff in pyROOT, however this can work with 1 library only, given that module.modulemap is generated by all the libraries i compile.

lrwxr-xr-x   1 lpnhe  staff    18B 22 Dic 13:53 libvelo.dylib -> velo/libvelo.dylib

Did you try making those soft link a full path rather than a relative path?

Hi @pcanal, yes. I tried. but it doesn’t work.
In. my reduced example which compiles things in those folders:

🍺  lpnhe 🍺  ~/Desktop/Lavoro/2020/ewp-rkstz/ROOT_TEST/build (master_q2smearing_andOA_investigation)
$|=>ls *
CMakeCache.txt      cmake_install.cmake libkernel.rootmap
Makefile            libkernel.dylib     libkernel_rdict.pcm

CMakeFiles:
3.19.2                          Makefile.cmake
CMakeDirectoryInformation.cmake Makefile2
CMakeError.log                  TargetDirectories.txt
CMakeOutput.log                 cmake.check_cache
CMakeRuleHashes.txt             progress.marks
CMakeTmp

kernel:
CMakeFiles          cmake_install.cmake libkernel.dylib     libkernel_rdict.pcm
Makefile            kernelDict.cxx      libkernel.rootmap   module.modulemap

It seems. like. the only way i have to make

r.gSystem.Load("libkernel")

work is to lunch the pythonn session inside build/kernel folder.
Somehow i have the impression that module.modulemap file has to be exposed in the same directory where the library file is saved.
I wonder therefore is there is still. in ROOT >. 6.20. a. way to bypass the creation of those files.
If I understood correctly from. some. documentation after. 6.20. some updates have been done to use C++ modules (. i am totally. naive. on this ).
NB: before i was using root 6.18 and the soft link of pcm,rootmap and dylib was sufficient. Now it seems like the module.modulemap is crucia as well. However, ROOT_GENERATE_DICTIONARY seems to create this file with the same name everywhere. (for any extra library i create)

I. tried to add in my. kernel dictionary generator

root_generate_dictionary(${PROJECT}Dict ${HEADERS} LINKDEF ${CMAKE_CURRENT_SOURCE_DIR}/LinkDef.h   OPTIONS -cxxmodule MODULE ${PROJECT})
#as before
add_library(${PROJECT} SHARED ${SOURCES} ${PROJECT}Dict.cxx)
#as before
target_link_libraries(${PROJECT} PUBLIC ${PKG_LIBRARIES})

the

OPTIONS -cxxmodule

Now it produce when i compile

-rw-r--r--  1 lpnhe  staff   6,7K 23 Dic 10:46 module.modulemap
-rw-r--r--  1 lpnhe  staff    36K 23 Dic 10:46 Makefile
-rw-r--r--  1 lpnhe  staff   1,2K 23 Dic 10:46 cmake_install.cmake
drwxr-xr-x  6 lpnhe  staff   192B 23 Dic 10:46 CMakeFiles
-rw-r--r--  1 lpnhe  staff   2,3K 23 Dic 10:50 libkernel.rootmap
-rw-r--r--  1 lpnhe  staff    52M 23 Dic 10:50 kernel.pcm
-rw-r--r--  1 lpnhe  staff   165K 23 Dic 10:50 kernelDict.cxx
-rwxr-xr-x  1 lpnhe  staff   4,8M 23 Dic 10:50 libkernel.dylib

so no. kernel_rdict.pcm file
It seems to do something when i try

In [1]: import  ROOT as r
r.
In [2]: r.gSystem.Load("kernel/libkernel")

RooFit v3.60 -- Developed by Wouter Verkerke and David Kirkby
                Copyright (C) 2000-2013 NIKHEF, University of California & Stanford University
                All rights reserved, please read http://roofit.sourceforge.net/license.txt

Error in <TCling::LoadPCM>: ROOT PCM /Users/lpnhe/Desktop/Lavoro/2020/ewp-rkstz/ROOT_TEST/build/kernel/libkernel_rdict.pcm file does not exist
Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libASImageGui_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libASImage_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libEG_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libEve_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libFFTW_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libFitPanel_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libFumili_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libGCocoa_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libGdml_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libGed_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libGenVector_G__GenVector32_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libGenVector_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libGenetic_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libGeomBuilder_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libGeomPainter_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libGeom_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libGuiBld_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libGuiHtml_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libGui_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libGviz3d_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libHistFactory_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libHistPainter_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libHtml_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libMinuit2_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libNetxNG_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libNetx_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libProofBench_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libProofDraw_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libProofPlayer_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libProof_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libPyMVA_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libQuadp_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libRCsg_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libRDAVIX_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libRGL_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libRHTTPSniff_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libRHTTP_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libROOTTPython_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libRecorder_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libRooFitMore_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libRootAuth_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libSPlot_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libSQLIO_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libSessionViewer_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libSmatrix_G__Smatrix32_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libSmatrix_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libSpectrumPainter_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libSpectrum_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libTMVAGui_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libTreeViewer_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libUnfold_rdict.pcm

Info in <TCling::LoadPCM>: In-memory ROOT PCM candidate /Users/lpnhe/root_build/lib/libXMLParser_rdict.pcm

Out[2]: 0

In [3]: r.ConfigHolder()
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyPrj::Prj
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyAnalysis::Analysis
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyQ2Bin::Q2Bin
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyYear::Year
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyPolarity::Polarity
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyTrigger::Trigger
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyTriggerConf::TriggerConf
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyBrem::Brem
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyTrack
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyTrack::Track
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyTrack
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyTrack
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyPrj::Prj
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyAnalysis::Analysis
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyQ2Bin::Q2Bin
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyYear::Year
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyPolarity::Polarity
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyTrigger::Trigger
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyBrem::Brem
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyTrack::Track
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyPrj::Prj
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyAnalysis::Analysis
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyQ2Bin::Q2Bin
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyYear::Year
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyPolarity::Polarity
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyTrigger::Trigger
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyBrem::Brem
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyPrj::Prj
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyAnalysis::Analysis
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyQ2Bin::Q2Bin
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyYear::Year
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyPolarity::Polarity
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyTrigger::Trigger
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyBrem::Brem
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyTrack::Track
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyPrj::Prj
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyAnalysis::Analysis
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyQ2Bin::Q2Bin
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyYear::Year
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyPolarity::Polarity
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyTrigger::Trigger
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyBrem::Brem
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyPrj::Prj
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyAnalysis::Analysis
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyQ2Bin::Q2Bin
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyYear::Year
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyPolarity::Polarity
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyTrigger::Trigger
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyTriggerConf::TriggerConf
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyBrem::Brem
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyTrack::Track
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyPrj::Prj
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyAnalysis::Analysis
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyQ2Bin::Q2Bin
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyYear::Year
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyPolarity::Polarity
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyTrigger::Trigger
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyBrem::Brem
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyTrack::Track
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyPrj::Prj
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyAnalysis::Analysis
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyQ2Bin::Q2Bin
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyYear::Year
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyPolarity::Polarity
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyTrigger::Trigger
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyTriggerConf::TriggerConf
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyBrem::Brem
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyTrack::Track
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyPrj::Prj
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyAnalysis::Analysis
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyQ2Bin::Q2Bin
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyYear::Year
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyPolarity::Polarity
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyTrigger::Trigger
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyBrem::Brem
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyTrack::Track
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyPrj::Prj
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyAnalysis::Analysis
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyQ2Bin::Q2Bin
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyYear::Year
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyPolarity::Polarity
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyTrigger::Trigger
Error in <TInterpreter::TCling::AutoLoad>: failure loading library libkernel.dylib for pyBrem::Brem
Out[3]: <cppyy.gbl.ConfigHolder object at 0x7ffb507d23f0>

and afterwards it can anyway create without. segfault r.ConfigHolder() and use it.

IIUC the underlying issue, you have two options:

a) combine the contents of the produced modulemap files into one and create a link (I am not sure if a modulemap link would be enough for root)
b) Disable the C++ module for this dictionary – by passing NO_CXXMODULE option to the ROOT_GENERATE_DICTIONARY invocation.

Make sure you delete the modulemap and the pcm file from previous trials.

Thanks a lot for the reply @vvassilev, i am not so sure how to realize solution a). I try b) and i report, since it seems to be the easier atm to test for me. Will report asap.

For a) do you mean doing something like
cat build/*/module.modulemap > build/module.modulemap?
I fail to understand the part about creating a link for it.
In practice, i have build/lib1 build/lib2 folders and each one has its own module.modulemap file and some pcm, dylib and.cxx files produced by the root generate dictionary macro

Yes. With the setup you mentioned I am afraid I do not see a way to create a symlink.

Hi @vvassilev, I tried option b) but it was not successfully.
With option a) i get it working, however AFAIK , one must always have module.modulemap located in the same exact directly where the python is executed or at least os.chdir( “$PATHTO/module.modulemap”) before doing r.gSystem.Load( "libkernel") .

I wonder how one can easily handle the module,modulemap files or if there are other solutions.

For reference what i am using now is this :

ROOT_GENERATE_DICTIONARY(G__${PROJECT} ${HEADERS} LINKDEF ${CMAKE_CURRENT_SOURCE_DIR}/LinkDef.h MODULE ${PROJECT} OPTIONS "-cxxmodule-writeEmptyRootPCM")

Thanks a lot for all suggestions and help.

the only thing i can think of is that everytime i try to “load” all my libraries i create a symlink on the fly to the merged module.modulemap, however i would also need at compile time to perform the cat of all of them into a single file

You can use CLING_MODULEMAP_FILES and CLING_MODULES_CACHE_PATH env variables to specify the extra modulemap files (you can name them with custom names) and locations respectively.

What happens if you use:

ROOT_GENERATE_DICTIONARY(G__${PROJECT} ${HEADERS} LINKDEF ${CMAKE_CURRENT_SOURCE_DIR}/LinkDef.h MODULE ${PROJECT} NO_CXXMODULE)

Hi @vvassilev, at the end i did several test, which possibly link to some LinkDef issue and header files i am using. I have not managed to get the NO_CXXMODULE configuration working and also the extra flags -cxxmodule-writeEmptyPCMFiles was not working

I ended up having something working fine proceeding as in the zip file for my reduced CMake project creating libraries and used inside python .

for_root_forum.zip (369.9 KB)

for each library i have i added #include in the LinkDef.h files for all headers
In each CMakeLists.txt i did

ROOT_GENERATE_DICTIONARY(G__${PROJECT}
    ${HEADERS}
    LINKDEF ${CMAKE_CURRENT_SOURCE_DIR}/LinkDef.h
    MODULE ${PROJECT}
add_library(${PROJECT} SHARED ${SOURCES} G__${PROJECT}.cxx)
target_include_directories(${PROJECT} PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/inc)
target_link_libraries(${PROJECT} PUBLIC ${PKG_LIBRARIES})
)

Then i set the PYTHONPATH to python/ folder where analysis folder contains an init.py file which does some trick to get the modulemap and soft links done on the fly.
When analysis is imported i create all soft links on the fly and i os.chdir( BUILDDIRECTORY) where i also cat all the module.modulemap files and i soft link

*/*.pcm, */*.dylib

Everything seems working
in /build folder.
Then i cat */module.modulemap
I noticed that the crash happens if import ROOT as r is not executed in the path where the libraries exists.

I don’t know what would be an alternative to all this the only issue i am left with mograting from ROOT 6.18 to 6.22 is that with all those updates i still get a couple of warning in the python interaction :

from analysis import * 
#this is now prompting  this message 
<<< cling interactive line includer >>>: warning: missing submodule 'roofit' [-Wincomplete-umbrella]
Failed to load module roofit

, however i can create a r.RooIpatia2() function which is expected to be defined inside my roofit extra library.

Ideally i would like to avoid to do the tricks you see in in python/analysis/__init__.py however i guess one has to maybe improve the LinkDef.h files, CMakeLists around and possibly refactor all the code.
Otherwise maybe i am missing something.
If in the script to import the libraries I don’t cd the path where the module map are located before doing import ROOT as r, i get the segfault and many more errors. I don’t know why.

To be clear on my setup :

$|=>root --version
ROOT Version: 6.22/07
Built for macosx64 on Dec 24 2020, 07:38:23
From heads/v6-22-00-patches@v6-22-06-5-g5e69c10422

,

$|=>python
Python 3.9.1 (default, Dec 17 2020, 10:08:12)
[Clang 12.0.0 (clang-1200.0.32.27)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

and

-- CMAKE_CXX_COMPILER_ID = AppleClang
-- Detecting GSL
-- GSL_VERSION   = 2.6
-- GSL_INCLUDES  = /usr/local/Cellar/gsl/2.6/include
-- GSL_LIBRARIES = /usr/local/Cellar/gsl/2.6/lib/libgsl.dylib;/usr/local/Cellar/gsl/2.6/lib/libgslcblas.dylib
-- Detecting ROOT
-- ROOT_VERSION   = 6.22.99
-- ROOT_INCLUDES  = /Users/lpnhe/root_build/include
-- Detecting YAMLCPP
-- YAMLCPP_VERSION   = 0.6.2
-- YAMLCPP_INCLUDES  = /usr/local/include
-- YAMLCPP_LIBRARIES = yaml-cpp
-- YAMLCPP_LIBRARIES = /usr/local/lib/libyaml-cpp.dylib
--
-- Detecting BOOST
-- BOOST_VERSION   = 1.75.0
-- BOOST_INCLUDES  = /usr/local/include
-- BOOST_LIBRARIES = Boost::system;Boost::filesystem

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.