Using boost filesystem in cling

Hi,

I somewhat fail to use boost in cling. I have the below script

#include <boost/filesystem.hpp>
// other includes

void make_some_plots() {
  TString savedir = "some/path/to/figures/";
  boost::filesystem::create_directories(savedir.Data());
  TCanvas* c = new TCanvas();
  // make plots
  c->SaveAs(savedir + "greatLookingPlot.png");
}

which I then try and fail to run as follows:

PROMPT > root -l
root [0] gSystem->Load("/usr/lib/libboost_signals.so.1.49.0");
root [1] gSystem->Load("/usr/lib/libboost_filesystem.so.1.49.0");
root [2] gSystem->Load("/usr/lib/libboost_system.so.1.49.0");
root [3] .L /usr/include/boost/filesystem.hpp
root [4] .L test.C
root [5] test()
IncrementalExecutor::executeFunction: symbol '_ZN5boost10filesystem6detail18create_directoriesERKNS0_4pathEPNS_6system10error_codeE' unresolved while linking [cling interface function]!
You are probably missing the definition of boost::filesystem::detail::create_directories(boost::filesystem::path const&, boost::system::error_code*)
Maybe you need to load the corresponding shared library?
root [6] .q

(also fails though aclic)

PROMPT > root -l
root [0] gSystem->Load("/usr/lib/libboost_signals.so.1.49.0");
root [1] gSystem->Load("/usr/lib/libboost_filesystem.so.1.49.0");
root [2] gSystem->Load("/usr/lib/libboost_system.so.1.49.0");
root [3] .L /usr/include/boost/filesystem.hpp
root [4] .L test.C+
Info in <TUnixSystem::ACLiC>: creating shared library /tmp/tmp.tEFn6LDZ9m/./test_C.so
root [5] test()
/home/pseyfert/coding/recentroot/tmpinstall/bin/root.exe: symbol lookup error: /tmp/tmp.tEFn6LDZ9m/test_C.so: undefined symbol: _ZN5boost10filesystem6detail18create_directoriesERKNS0_4pathEPNS_6system10error_codeE

In which library is the symbol boost::filesystem::detail::create_directories(boost::filesystem::path const&, boost::system::error_code*) located?

You could do this by 'find /usr/lib/ -name *.so -exec nm -AC {} ; | grep ‘_ZN5boost10filesystem6detail18create_directoriesERKNS0_4pathEPNS_6system10error_codeE’

strangely my /usr/lib/libboostso don’t have symbols

PROMPT> nm -C /usr/lib/libboost_filesystem.so.1.49.0 nm: /usr/lib/libboost_filesystem.so.1.49.0: no symbols

I only got that there is only one boost library with filesystem in the name (and i strongly hope that functions defined through boost/filesystem.hpp end up in libboost_filesystem..so.) and that filesystem depends only on system:

PROMPT> ldd /usr/lib/libboost_filesystem.so.1.49.0
	linux-vdso.so.1 (0x00007fff8b775000)
	libboost_system.so.1.49.0 => /usr/lib/libboost_system.so.1.49.0 (0x00007fcfbb6d5000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fcfbb31e000)
	libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fcfbb013000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fcfbad12000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fcfbaafc000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fcfba8df000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fcfba534000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fcfbb526000)

I tried to reproduce the problem with root from cvmfs on lxplus:

source /cvmfs/sft.cern.ch/lcg/views/LCG_87/x86_64-slc6-gcc49-opt/setup.sh

(browsing around the $LD_LIBRARY_PATH this should use boost from /cvmfs/sft.cern.ch/lcg/views/LCG_87/x86_64-slc6-gcc49-opt/lib/libboost*)

Loading the boost headers with .L fails at first due to

root [3] .L /cvmfs/sft.cern.ch/lcg/views/LCG_87/x86_64-slc6-gcc49-opt/include/boost-1_62/boost/filesystem.hpp 
In file included from input_line_17:1:
/cvmfs/sft.cern.ch/lcg/releases/Boost/1.62.0-83c69/x86_64-slc6-gcc49-opt/include/boost-1_62/boost/filesystem.hpp:19:12: fatal error: 
      'boost/filesystem/string_file.hpp' file not found
#  include <boost/filesystem/string_file.hpp>

(trying around I got the suspicion one should restart root/cling here, as updates of the include path don’t repair the loading of filesystem.hpp). so new try:

pseyfert@lxplus091 ~ > root -l
root [0] gInterpreter->AddIncludePath("/cvmfs/sft.cern.ch/lcg/views/LCG_87/x86_64-slc6-gcc49-opt/include/boost-1_62/")
root [1] gSystem->AddIncludePath(Form("-I/cvmfs/sft.cern.ch/lcg/views/LCG_87/x86_64-slc6-gcc49-opt/include/boost-1_62/"))
root [2] gSystem->Load("/cvmfs/sft.cern.ch/lcg/views/LCG_87/x86_64-slc6-gcc49-opt/lib/libboost_signals-gcc49-mt-1_62.so")
(int) 0
// no idea why i have done that … anyway, loading signals on my laptop doesn't help
root [3] gSystem->Load("/cvmfs/sft.cern.ch/lcg/views/LCG_87/x86_64-slc6-gcc49-opt/lib/libboost_filesystem-gcc49-mt-1_62.so")
(int) 0
root [4] gSystem->Load("/cvmfs/sft.cern.ch/lcg/views/LCG_87/x86_64-slc6-gcc49-opt/lib/libboost_system-gcc49-mt-1_62.so")
(int) 0
root [5] .L /cvmfs/sft.cern.ch/lcg/views/LCG_87/x86_64-slc6-gcc49-opt/include/boost-1_62/boost/filesystem.hpp
root [6] .L test.C
root [7] make_some_plots()
Info in <TCanvas::Print>: file /tmp/pseyfert/plotdir/greatLookingPlot.png has been created
root [8] .q

sooooo … might be related to the boost libraries on my PC …

PS: test.C contains

#include <boost/filesystem.hpp>
#include "TString.h"
#include "TCanvas.h"

void make_some_plots() {
  TString savedir = "/tmp/pseyfert/plotdir/";
  boost::filesystem::create_directories(savedir.Data());
  TCanvas* c = new TCanvas();
  // make plots
  c->SaveAs(savedir + "greatLookingPlot.png");
}

Hi Paul,

Very strange - I’m not sure how we can help here :frowning:

If you compile an object file from your code, is this boost symbol undefined?

If you build a binary from these same sources and link against boost, does your binary have this symbol undefined? If not, could you increase the linker verbosity and find out (if all fails using a map file) where your linker, on your system, resolves that symbol from?

Cheers, Axel.

what i have so far:

/// test.C
#include <boost/filesystem.hpp>
#include "TString.h"
#include "TCanvas.h"

void make_some_plots() {
  TString savedir = "/tmp/pseyfert/plotdir/";
  boost::filesystem::create_directories(savedir.Data());
  TCanvas* c = new TCanvas();
  // make plots
  c->SaveAs(savedir + "greatLookingPlot.png");
  delete c;
}

int main() {
  make_some_plots();
  return 0;
}

compiles, links, runs ✓

echo "mostly boring defaults i have in my .bashrc … execessive linking for what we need here"
export CPPFLAGS="-pthread -std=c++11 -Wno-deprecated-declarations -m64 -I/home/pseyfert/coding/recentroot/6-07-04/include -g -Wextra -Wall -Wshadow"
export LDFLAGS="-L/home/pseyfert/coding/recentroot/6-07-04/lib -lCore -lRIO -lNet -lHist -lGraf -lGraf3d -lGpad -lTree -lRint -lPostscript -lMatrix -lPhysics -lMathCore -lThread -lMultiProc -pthread -lm -ldl -rdynamic -lRooStats -lRooFitCore -lRooFit -lMinuit -lFoam -lMathMore -lTMVA -lstdc++ -m64 -g -lboost_system -lboost_filesystem"
/tmp/tmp.6lQa6vXZLf > nm -CA test.o | \grep boost
test.o:0000000000000000 W boost::filesystem::create_directories(boost::filesystem::path const&)
test.o:0000000000000000 W boost::filesystem::path::path(char const*)
test.o:0000000000000000 W boost::filesystem::path::path(char const*)
test.o:0000000000000000 n boost::filesystem::path::path(char const*)
test.o:0000000000000000 W boost::filesystem::path::~path()
test.o:0000000000000000 W boost::filesystem::path::~path()
test.o:0000000000000000 n boost::filesystem::path::~path()
test.o:                 U boost::filesystem::detail::create_directories(boost::filesystem::path const&, boost::system::error_code*)
test.o:0000000000000000 W boost::system::system_error::~system_error()
test.o:0000000000000000 W boost::system::system_error::~system_error()
test.o:0000000000000000 W boost::system::system_error::~system_error()
test.o:0000000000000000 n boost::system::system_error::~system_error()
test.o:                 U boost::system::system_category()
test.o:                 U boost::system::generic_category()
test.o:0000000000000010 b boost::system::errno_ecat
test.o:0000000000000018 b boost::system::native_ecat
test.o:0000000000000008 b boost::system::posix_category
test.o:0000000000000000 W boost::system::error_code::value() const
test.o:0000000000000000 W boost::system::error_code::message() const
test.o:0000000000000000 W boost::system::system_error::what() const
test.o:0000000000000000 V typeinfo for boost::system::system_error
test.o:0000000000000000 V typeinfo name for boost::system::system_error
test.o:0000000000000000 V vtable for boost::system::system_error

so boost::filesystem::detail::create_directories(boost::filesystem::path const&, boost::system::error_code*) is undefined in the .o file

/tmp/tmp.6lQa6vXZLf > nm -CA test | \grep boost
test:00000000004028b1 W boost::filesystem::create_directories(boost::filesystem::path const&)
test:000000000040284e W boost::filesystem::path::path(char const*)
test:000000000040284e W boost::filesystem::path::path(char const*)
test:0000000000402834 W boost::filesystem::path::~path()
test:0000000000402834 W boost::filesystem::path::~path()
test:                 U boost::filesystem::detail::create_directories(boost::filesystem::path const&, boost::system::error_code*)
test:00000000004026fa W boost::system::system_error::~system_error()
test:00000000004026b0 W boost::system::system_error::~system_error()
test:00000000004026b0 W boost::system::system_error::~system_error()
test:                 U boost::system::system_category()
test:                 U boost::system::generic_category()
test:0000000000604630 b boost::system::errno_ecat
test:0000000000604638 b boost::system::native_ecat
test:0000000000604628 b boost::system::posix_category
test:0000000000402650 W boost::system::error_code::value() const
test:0000000000402660 W boost::system::error_code::message() const
test:0000000000402720 W boost::system::system_error::what() const
test:0000000000403030 V typeinfo for boost::system::system_error
test:0000000000403050 V typeinfo name for boost::system::system_error
test:0000000000402fe0 V vtable for boost::system::system_error

i.e. boost::filesystem::detail::create_directories(boost::filesystem::path const&, boost::system::error_code*) is also undefined in the linked executable

/tmp/tmp.6lQa6vXZLf > ldd test
	linux-vdso.so.1 (0x00007fff08fce000)
	libCore.so => /home/pseyfert/coding/recentroot/6-07-04/lib/libCore.so (0x00007fa111485000)
	libRIO.so => /home/pseyfert/coding/recentroot/6-07-04/lib/libRIO.so (0x00007fa110fdf000)
	libNet.so => /home/pseyfert/coding/recentroot/6-07-04/lib/libNet.so (0x00007fa110d28000)
	libHist.so => /home/pseyfert/coding/recentroot/6-07-04/lib/libHist.so (0x00007fa11079a000)
	libGraf.so => /home/pseyfert/coding/recentroot/6-07-04/lib/libGraf.so (0x00007fa11045b000)
	libGraf3d.so => /home/pseyfert/coding/recentroot/6-07-04/lib/libGraf3d.so (0x00007fa1101c3000)
	libGpad.so => /home/pseyfert/coding/recentroot/6-07-04/lib/libGpad.so (0x00007fa10ff1b000)
	libTree.so => /home/pseyfert/coding/recentroot/6-07-04/lib/libTree.so (0x00007fa10fbd3000)
	libRint.so => /home/pseyfert/coding/recentroot/6-07-04/lib/libRint.so (0x00007fa10f9ae000)
	libPostscript.so => /home/pseyfert/coding/recentroot/6-07-04/lib/libPostscript.so (0x00007fa10f74b000)
	libMatrix.so => /home/pseyfert/coding/recentroot/6-07-04/lib/libMatrix.so (0x00007fa10f400000)
	libPhysics.so => /home/pseyfert/coding/recentroot/6-07-04/lib/libPhysics.so (0x00007fa10f1ba000)
	libMathCore.so => /home/pseyfert/coding/recentroot/6-07-04/lib/libMathCore.so (0x00007fa10ee34000)
	libThread.so => /home/pseyfert/coding/recentroot/6-07-04/lib/libThread.so (0x00007fa10ec0a000)
	libMultiProc.so => /home/pseyfert/coding/recentroot/6-07-04/lib/libMultiProc.so (0x00007fa10e9fb000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fa10e7f7000)
	libRooStats.so => /home/pseyfert/coding/recentroot/6-07-04/lib/libRooStats.so (0x00007fa10e46a000)
	libRooFitCore.so => /home/pseyfert/coding/recentroot/6-07-04/lib/libRooFitCore.so (0x00007fa10dcf5000)
	libRooFit.so => /home/pseyfert/coding/recentroot/6-07-04/lib/libRooFit.so (0x00007fa10d8aa000)
	libMinuit.so => /home/pseyfert/coding/recentroot/6-07-04/lib/libMinuit.so (0x00007fa10d640000)
	libFoam.so => /home/pseyfert/coding/recentroot/6-07-04/lib/libFoam.so (0x00007fa10d41d000)
	libMathMore.so => /home/pseyfert/coding/recentroot/6-07-04/lib/libMathMore.so (0x00007fa10d191000)
	libTMVA.so => /home/pseyfert/coding/recentroot/6-07-04/lib/libTMVA.so (0x00007fa10cbe0000)
	libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fa10c8d5000)
	libboost_system.so.1.55.0 => /usr/lib/x86_64-linux-gnu/libboost_system.so.1.55.0 (0x00007fa10c6d1000)
	libboost_filesystem.so.1.55.0 => /usr/lib/x86_64-linux-gnu/libboost_filesystem.so.1.55.0 (0x00007fa10c4b9000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fa10c1b8000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fa10bfa2000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fa10bd85000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa10b9da000)
	libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007fa10b76c000)
	libpcreposix.so.3 => /usr/lib/x86_64-linux-gnu/libpcreposix.so.3 (0x00007fa10b569000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fa10b34e000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fa111a98000)
	libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007fa10b117000)
	libfreetype.so.6 => /usr/lib/x86_64-linux-gnu/libfreetype.so.6 (0x00007fa10ae6d000)
	libTreePlayer.so => /home/pseyfert/coding/recentroot/6-07-04/lib/libTreePlayer.so (0x00007fa10ab3a000)
	libgsl.so.0 => /usr/lib/libgsl.so.0 (0x00007fa10a70d000)
	libgslcblas.so.0 => /usr/lib/libgslcblas.so.0 (0x00007fa10a4cf000)
	libTreeViewer.so => /home/pseyfert/coding/recentroot/6-07-04/lib/libTreeViewer.so (0x00007fa10a239000)
	libMLP.so => /home/pseyfert/coding/recentroot/6-07-04/lib/libMLP.so (0x00007fa109fff000)
	libXMLIO.so => /home/pseyfert/coding/recentroot/6-07-04/lib/libXMLIO.so (0x00007fa109da0000)
	libGed.so => /home/pseyfert/coding/recentroot/6-07-04/lib/libGed.so (0x00007fa109a76000)
	libGui.so => /home/pseyfert/coding/recentroot/6-07-04/lib/libGui.so (0x00007fa10946a000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fa109262000)
	libpng12.so.0 => /lib/x86_64-linux-gnu/libpng12.so.0 (0x00007fa10903b000)

same with root 6.08/02.

In the meanwhile I read the man page of nm to list the symbols in libboost…so files:

/tmp/tmp.6lQa6vXZLf > nm -D /usr/lib/libboost_filesystem.so.1.49.0 | \grep create_directories
0000000000013520 T _ZN5boost11filesystem36detail18create_directoriesERKNS0_4pathEPNS_6system10error_codeE

so … not sure what to think of this, this is almost what cling looks for (copy&paste from initial post)

_ZN5boost10filesystem6detail18create_directoriesERKNS0_4pathEPNS_6system10error_codeE

Hi,

Looking at boost.org/doc/libs/1_48_0/li … /index.htm there seems to be a bit of a mess - i.e. it seems that your boost library (filesystem3) is built with different flags than the headers suggest (looking for “filesystem” without any version number.

Maybe #define BOOST_FILESYSTEM_VERSION to “the right thing” (3?) will do, “in your code, before including any filesystem header” as they say on the boost page?

Axel.

Hi,

thanks for looking that up, I tried a bit around (first setting the filesystem version to 3, w/o success, and then setting it to 2).

// spam bad BOOST_FILESYSTEM_VERSION to see what happens
root [0] #define BOOST_FILESYSTEM_VERSION 2
root [1] TString flags = gSystem->GetFlagsOpt();
root [2] flags+=" -DBOOST_FILESYSTEM_VERSION=2";
root [3] gSystem->SetFlagsOpt(flags);
root [4] gSystem->Load("/usr/lib/libboost_system.so.1.49.0");
root [5] .L /usr/include/boost/filesystem.hpp
In file included from input_line_17:1:
In file included from /usr/include/boost/filesystem.hpp:15:
/usr/include/boost/filesystem/config.hpp:16:5: error: Compiling Filesystem version 3 file with BOOST_FILESYSTEM_VERSION defined != 3
#   error Compiling Filesystem version 3 file with BOOST_FILESYSTEM_VERSION defined != 3
    ^

with this pointer I looked up that boost/filesystem.hpp includes boost/filesystem/config.hpp, which defines BOOST_FILESYSTEM_VERSION to 3 (if it isn’t defined already, and issues this error if it is defined already to a value other than 3).

root [0] gSystem->Load("/usr/lib/libboost_system.so.1.49.0");
root [1] gSystem->Load("/usr/lib/libboost_filesystem.so.1.49.0");
root [2] .L /usr/include/boost/filesystem.hpp
root [3] BOOST_FILESYSTEM_VERSION
(int) 3
root [4] .L test.C
root [5] make_some_plots()
IncrementalExecutor::executeFunction: symbol '_ZN5boost10filesystem6detail18create_directoriesERKNS0_4pathEPNS_6system10error_codeE' unresolved while linking [cling interface function]!
You are probably missing the definition of boost::filesystem::detail::create_directories(boost::filesystem::path const&, boost::system::error_code*)
Maybe you need to load the corresponding shared library?
root [6] 

yet what i came across is that i have a few more libboost_* lying around:

/usr/lib/libboost_filesystem.so.1.46.1
/usr/lib/libboost_filesystem.so.1.48.0
/usr/lib/libboost_filesystem.so.1.49.0
/usr/lib/x86_64-linux-gnu/libboost_filesystem.so
/usr/lib/x86_64-linux-gnu/libboost_filesystem.so.1.55.0
/var/cache/pbuilder/base.cow/usr/lib/x86_64-linux-gnu/libboost_filesystem.so.1.62.0

(was only looking in /usr/lib so far and 1.46.1 or 1.48.0 didn’t work either.)

loading 1.55 instead of 1.49 works.

→ in hindsight: <boost/version.hpp> reveals which boost version the header belongs to (and the shared libraries have it in the filename). what i don’t understand is how/if the symbol filesystem6/filesystem36 relates to the boost filesystem version …

Hi Paul,

That’s the easy part of the mangling :slight_smile: It’s , so you can read 5boost10filesystem6detail18create_directories as “first 5: boost”, “next 10: filesystem”, “next 6: detail”,“next 18: create_directories”. And thus the interesting different was “11filesystem3” versus “10filesystem”.

Axel.