Nonstandard include paths for cling

tobi_s · February 27, 2014, 3:32pm

Hi,

I’m working on porting the belle 2 reconstruction software basf2 to ROOT6. So far, all problems have been due to issues in basf2’s build system, with the exception of the (trivial) rootcint -> rootcling renaming. Congratulations!

Now I’m running into an issue that is probably trivial to solve for someone in the know: during startup of the software I get a message as

$ basf2 In file included from input_line_41:19: ./include/framework/datastore/RelationsObject.h:14:10: fatal error: 'framework/datastore/DataStore.h' file not found #include <framework/datastore/DataStore.h> ^ basf2: /home/schlueter/src/root.git/core/meta/src/TCling.cxx:1148: virtual void TCling::RegisterModule(const char*, const char**, const char**, const char**, const char*, void (*)()): Assertion `cling::Interpreter::kSuccess == compRes && "Payload code of a dictionary could not be parsed correctly."' failed. Aborted (core dumped) $

I can fix this error, by changing the respective include path to

(this is where the file resides relative to my $PWD) but then I get the same error again at the next include statement. I.e. (note that it found the header it complained about before)

$ basf2 In file included from input_line_41:19: In file included from ./include/framework/datastore/RelationsObject.h:14: ./include/framework/datastore/DataStore.h:14:10: fatal error: 'framework/datastore/RelationEntry.h' file not found #include <framework/datastore/RelationEntry.h> ^ basf2: /home/schlueter/src/root.git/core/meta/src/TCling.cxx:1148: virtual void TCling::RegisterModule(const char*, const char**, const char**, const char**, const char*, void (*)()): Assertion `cling::Interpreter::kSuccess == compRes && "Payload code of a dictionary could not be parsed correctly."' failed. Aborted (core dumped) $

Note that the original error message itself is issued inside a file that sits inside “include/”, so there is some kind of inconsistency going on her. I understand that only

is supposed to first look in the directory containing the including file, but FAICT in this case this is some magic inside cling itself that is leading to this place, and the include path should be set correctly because otherwise the initial file should not have been found. The behavior appears to have changed WRT cint.

Hints?

ps backtrace looks like this when the assertion is hit, so it’s trying to initializae an automatically generated dictionary:

#0 TCling::RegisterModule (this=0x6c2190, modulename=0x7ffff74a89d8 "framework_datastore_include_Dict", headers=0x7ffff7780f80 <(anonymous namespace)::TriggerDictionaryInitialization_framework_datastore_include_Dict_Impl()::headers>, allHeaders=<optimized out>, includePaths=<optimized out>, payloadCode=0x7ffff74a8b00 <_ZL12kBitsPerByte> "\b", triggerFunc=0x7ffff746fc98 <(anonymous namespace)::TriggerDictionaryInitialization_framework_datastore_include_Dict_Impl()>) at /home/schlueter/src/root.git/core/meta/src/TCling.cxx:1148 #1 0x00007ffff5f6b46b in TROOT::RegisterModule ( modulename=0x7ffff74a89d8 "framework_datastore_include_Dict", headers=0x7ffff7780f80 <(anonymous namespace)::TriggerDictionaryInitialization_framework_datastore_include_Dict_Impl()::headers>, allHeaders=0x7ffff7780fc0 <(anonymous namespace)::TriggerDictionaryInitialization_framework_datastore_include_Dict_Impl()::allHeaders>, includePaths=0x7ffff77815a0 <(anonymous namespace)::TriggerDictionaryInitialization_framework_datastore_include_Dict_Impl()::includePaths>, payloadCode=0x7ffff74a8b30 "\n#ifndef G__VECTOR_HAS_CLASS_ITERATOR\n #define G__VECTOR_HAS_CLASS_ITERATOR\n#endif\n#ifndef HAS_CALLGRIND\n #define HAS_CALLGRIND\n#endif\n#ifndef HAS_PGSQL\n #define HAS_PGSQL\n#endif\n#ifndef RaveDllExp"..., triggerFunc=0x7ffff746fc98 <(anonymous namespace)::TriggerDictionaryInitialization_framework_datastore_include_Dict_Impl()>) at /home/schlueter/src/root.git/core/base/src/TROOT.cxx:2126 #2 0x00007ffff746fcdc in (anonymous namespace)::TriggerDictionaryInitialization_framework_datastore_include_Dict_Impl () at build/Linux_x86_64/debug/framework_datastore_include_Dict.cc:649 #3 0x00007ffff746fcf7 in (anonymous namespace)::DictInit::DictInit ( this=0x7ffff7786ce8 <_ZN12_GLOBAL__N_1L26__TheDictionaryInitializerE>) at build/Linux_x86_64/debug/framework_datastore_include_Dict.cc:655 #4 0x00007ffff746fd93 in __static_initialization_and_destruction_0 ( __initialize_p=1, __priority=65535) at build/Linux_x86_64/debug/framework_datastore_include_Dict.cc:657 #5 0x00007ffff746fda8 in _GLOBAL__sub_I_framework_datastore_include_Dict.cc(void) () at build/Linux_x86_64/debug/framework_datastore_include_Dict.cc:661 #6 0x00007ffff7de9316 in call_init (l=<optimized out>, argc=1, argv=0x7fffffffd5a8, env=0x7fffffffd5b8) at dl-init.c:85 #7 0x00007ffff7de93ef in call_init (env=<optimized out>, argv=<optimized out>, argc=<optimized out>, l=<optimized out>) at dl-init.c:52 #8 _dl_init (main_map=0x7ffff7ffe2e8, argc=1, argv=0x7fffffffd5a8, env=0x7fffffffd5b8) at dl-init.c:134 #9 0x00007ffff7ddb6fa in _dl_start_user () from /lib64/ld-linux-x86-64.so.2 #10 0x0000000000000001 in ?? () #11 0x00007fffffffd961 in ?? () #12 0x0000000000000000 in ?? ()

Digging a bit further, I find that in the generated dictionary, register_module is called with this as includePaths parameter:

static const char* includePaths[] = { "/home/schlueter/src/belle2/externals/development/include", "/home/schlueter/src/belle2/tools/virtualenv/include/python2.7", "/home/schlueter/src/belle2/externals/development/include/CLHEP", "/home/schlueter/src/belle2/externals/development/include/Geant4", "/home/schlueter/src/belle2/externals/development/include/pgsql", "/home/schlueter/src/root.git/include", "/home/schlueter/src/belle2/externals/development/include/vgm", "/home/schlueter/src/belle2/externals/development/include/HepMC", "/home/schlueter/src/belle2/externals/development/include/pythia", "/home/schlueter/src/belle2/externals/development/include/Photos", "/home/schlueter/src/belle2/externals/development/include/Tauola", "/home/schlueter/src/belle2/externals/development/include/evtgen", "/home/schlueter/src/belle2/externals/development/include/FLC", "/home/schlueter/src/belle2/externals/development/include/Eigen", "/usr/include/libxml2", "/home/schlueter/src/root.git/include", "/home/schlueter/src/basf2_clean/", 0 };

The last entry should probably read “/home/schlueter/src/basf2_clean/include”./ If I make this change manually, the startup sequence completes. I’ll test execution tomorrow

tobi_s · February 28, 2014, 12:55pm

So, I had to battle a little with an external library that was still linked against root 5.34. This resolved, I’m now running into the following:

[code]$ basf2 testbeam/vxd/examples/VXDTFdemoTB.py
[INFO] Steering file: testbeam/vxd/examples/VXDTFdemoTB.py

basf2 Python environment set
Framework object created: fw
In file included from input_line_51:22:
In file included from ./include/mdst/dataobjects/Track.h:11:
/home/schlueter/src/basf2_clean/include/framework/datastore/StoreArray.h:237:44: error: no member named ‘forward’ in namespace ‘std’
return new(nextFreeAddress()) T(std::forward(params)…);

                                  ~~~~~^

/home/schlueter/src/basf2_clean/include/framework/datastore/StoreArray.h:237:52: error: ‘Args’ does not refer to a value
return new(nextFreeAddress()) T(std::forward(params)…);
^
/home/schlueter/src/basf2_clean/include/framework/datastore/StoreArray.h:236:23: note: declared here
template<class …Args> T* appendNew(Args&& … params) {
^
basf2: /home/schlueter/src/root.git/core/meta/src/TCling.cxx:1148: virtual void TCling::RegisterModule(const char*, const char**, const char**, const char**, const char*, void (*)()): Assertion `cling::Interpreter::kSuccess == compRes && “Payload code of a dictionary could not be parsed correctly.”’ failed.
Aborted (core dumped)
$ [/code]
The issue appears to be that cling thinks the source is C++03, where it is C++11. How can I change the language standard used? During compilation the --std=c++11 flag is used, of course.

ps Just to be explicit, I compiled ROOT6 with
cmake -DCMAKE_CXX_FLAGS=‘-std=c++11’
i.e. as C++11.

tobi_s · February 28, 2014, 3:44pm

Ok, I figured out that the proper way of getting C++11 support into ROOT6 is to compile with

cmake -Dcxx11=ON .. (which I recorded in a place where I hope people can actually find the information)

Unfortunately, this doesn’t help me, I still get the same error during startup. The integrated libCling doesn’t digest C++11. I verified that the newly compile libCling is the one being linked (there shouldn’t be another one anywhere, but you never know …)

Edit: hm, the c++11 stuff doesn’t seem to have worked:

ROOT 5.99/05 (heads/master@v5-99-04-821-g1dcbb09, Feb 28 2014, 15:58:00 on linuxx8664gcc) cling C/C++ Interpreter: type .? for help. root [0] #include <unordered_map> In file included from input_line_117:1: In file included from /usr/include/c++/4.7/unordered_map:35: /usr/include/c++/4.7/bits/c++0x_warning.h:32:2: error: This file requires compiler and library support for the ISO C++ 2011 standard. This support is currently experimental, and must be enabled with the -std=c++11 or -std=gnu++11 compiler options. #error This file requires compiler and library support for the \ ^ root [1]

Axel · March 2, 2014, 12:33pm

Hi,
super-weird. This works for configure && make. I’ll ask our CMake guru for help.

Cheers, Axel.

Axel · March 2, 2014, 12:37pm

Regarding

#include <framework/datastore/DataStore.h>

How do you invoke rootcling for this header? It should have a -Iinclude or whatever that allows ROOT at runtime to find the header back (until we have modules).

Cheers, Axel.

tobi_s · March 3, 2014, 10:02am

[quote=“Axel”]Regarding

#include <framework/datastore/DataStore.h>

How do you invoke rootcling for this header? It should have a -Iinclude or whatever that allows ROOT at runtime to find the header back (until we have modules).

Cheers, Axel.[/quote]

There’s an -Iinclude on that line, which should add the respective directory.

For the sake of your and my convenience, I’m repeating the list of include directories contained in the dictionary file. ./include should appear between Eigen and libxml, which it obsviously doesn’t:

        static const char* includePaths[] = {
    "/home/schlueter/src/belle2/externals/development/include",
    "/home/schlueter/src/belle2/tools/virtualenv/include/python2.7",
    "/home/schlueter/src/belle2/externals/development/include/CLHEP",
    "/home/schlueter/src/belle2/externals/development/include/Geant4",
    "/home/schlueter/src/belle2/externals/development/include/pgsql",
    "/home/schlueter/src/root.git/include",
    "/home/schlueter/src/belle2/externals/development/include/vgm",
    "/home/schlueter/src/belle2/externals/development/include/HepMC",
    "/home/schlueter/src/belle2/externals/development/include/pythia",
    "/home/schlueter/src/belle2/externals/development/include/Photos",
    "/home/schlueter/src/belle2/externals/development/include/Tauola",
    "/home/schlueter/src/belle2/externals/development/include/evtgen",
    "/home/schlueter/src/belle2/externals/development/include/FLC",
    "/home/schlueter/src/belle2/externals/development/include/Eigen",
    "/usr/include/libxml2",
    "/home/schlueter/src/root.git/include",
    "/home/schlueter/src/basf2_clean/",
    0
        };

I verified that the same thing happens for our other rootcling directories, which should use the same command line except for the source and output files. I don’t know why they don’t error out like this directory did, maybe i haven’t made it far enough in the execution for them to become relevant.

I didn’t know that the autoconf stuff was still supported, I’ll try a root build with that next. Thanks so far!

tobi_s · March 3, 2014, 12:10pm

Ok, having built with autoconf instead of cmake, I can now make it farther. I still have to correct the dictionary file.

Initially I got lots of these errors, because I used different versions of g++ to compile root and to compile basf2:

schlueter@ucschieck09:~/src/basf2_clean$ basf2 testbeam/vxd/examples/VXDTFdemoTB.py Warning in cling::CIFactory::createCI(): C++ ABI mismatch, compiled with __GLIBCXX__ v20120920 running with v20130411 In file included from input_line_41:19: ./include/framework/datastore/RelationsObject.h:14:10: fatal error: 'framework/datastore/DataStore.h' file not found #include <framework/datastore/DataStore.h> ^ Warning in <TInterpreter::TCling::RegisterModule>: Problems declaring payload for moduleExecution carried on, but then segfaulted further down the line.

I recompiled root with the belle2-g++. So far it works.

Congratulations! This was a fairly easy process. There were only a few hurdles, which I summarize as follows:
One bug in ROOT: rootcling somehow misses ‘-Iinclude’.
Some complexities: making C++11 work as intended, incompatibility between different compiler versions.
Some annoying features of our build system: will address internally.

mato · March 3, 2014, 11:15pm

For the problem of building with the option -Dcxx11=ON I have committed some changes in the master branch. Find attached the changes.
c++11flags.diff (3.35 KB)

tobi_s · March 4, 2014, 10:51am

[quote=“mato”]For the problem of building with the option -Dcxx11=ON I have committed some changes in the master branch. Find attached the changes.[/quote]Thanks for the quick turnaround!

tobi_s · March 4, 2014, 10:54am

Sorry, in all this I forgot to mention one more difference between rootcling and rootcint: we have one class which we read and write even though it contains const members. Somehow ROOT5 manages to make it work. On the other hand, ROOT6 generates a dictionary that the compiler barfs on.

I don’t think it’s possible to rely on the C++ standard and have this working, but it’s definitely a change in behavior compared to ROOT5. An error when building the dictionary / streamer or maybe foregoing the generation of a Streamer would be a more user-friendly way of dealing with this.

Axel · March 5, 2014, 9:22am

Hi,

I have created sft.its.cern.ch/jira/browse/ROOT-6128 for you. Could you please provide a reproducer there and sign up on the watch list?

Cheers, Axel.

tobi_s · March 6, 2014, 10:45am

Well, I don’t have a CERN account, and I don’t want to use my social network accounts outside of the social networks (in particular not from my office desktop), so I will investigate the account options. I was really happy with my Savannah account, BTW. Anyway, I’ll see what I can do.

tobi_s · March 6, 2014, 11:27am

Ok, since I did only mention this in passing before, I will give you some details without reducing the classes first:
We have this in the class (this is the first private member, which is probably why it gives the error):

class TrackFitResult {
  ...
private:
  ...
    //---------------------------------------------------------------------------------------------------------------------------
    /** PDG Code for hypothesis with which the corresponding fit was performed. */
    const unsigned int m_pdg;
   ...
};

Then the disctionary is compiled, and it gives this error (g++ 4.7.3):

g++ -o build/Linux_x86_64/debug/mdst_dataobjects_include_Dict.os -c -Wall -I/home/schlueter/src/belle2/externals/development/include -I/home/schlueter/src/belle2/tools/virtualenv/include/python2.7 -std=c++11 -Wextra -g -I/home/schlueter/src/belle2/externals/development/include/CLHEP -I/home/schlueter/src/belle2/externals/development/include/Geant4 -I/home/schlueter/src/belle2/externals/development/include/pgsql -I/home/schlueter/src/root.autoconf.git/compile/include -I/home/schlueter/src/belle2/externals/development/include/vgm -I/home/schlueter/src/belle2/externals/development/include/HepMC -I/home/schlueter/src/belle2/externals/development/include/pythia -I/home/schlueter/src/belle2/externals/development/include/Photos -I/home/schlueter/src/belle2/externals/development/include/Tauola -I/home/schlueter/src/belle2/externals/development/include/evtgen -I/home/schlueter/src/belle2/externals/development/include/FLC -I/home/schlueter/src/belle2/externals/development/include/Eigen -fPIC -DHAS_CALLGRIND -DHAS_PGSQL -DRaveDllExport= -D_PACKAGE_=\"mdst\" -I. -Iinclude -I/usr/include/libxml2 build/Linux_x86_64/debug/mdst_dataobjects_include_Dict.cc build/Linux_x86_64/debug/mdst_dataobjects_include_Dict.cc: In member function ‘virtual void Belle2::TrackFitResult::Streamer(TBuffer&)’: build/Linux_x86_64/debug/mdst_dataobjects_include_Dict.cc:684:56: error: invalid conversion from ‘unsigned int’ to ‘Char_t* {aka char*}’ [-fpermissive] In file included from /home/schlueter/src/root.autoconf.git/compile/include/TObject.h:232:0, from /home/schlueter/src/root.autoconf.git/compile/include/TNamed.h:26, from /home/schlueter/src/root.autoconf.git/compile/include/TDictionary.h:45, from /home/schlueter/src/root.autoconf.git/compile/include/TClass.h:25, from build/Linux_x86_64/debug/mdst_dataobjects_include_Dict.cc:14: /home/schlueter/src/root.autoconf.git/compile/include/TBuffer.h:341:17: error: initializing argument 2 of ‘TBuffer& operator>>(TBuffer&, Char_t*)’ [-fpermissive]
The code leading up to the erroneous line (last in snippet) looks like this:

[code]void TrackFitResult::Streamer(TBuffer &R__b)
{
// Stream an object of class Belle2::TrackFitResult.

//This works around a msvc bug and should be harmless on other platforms
typedef ::Belle2::TrackFitResult thisClass;
UInt_t R__s, R__c;
if (R__b.IsReading()) {
Version_t R__v = R__b.ReadVersion(&R__s, &R__c); if (R__v) { }
//This works around a msvc bug and should be harmless on other platforms
typedef Belle2::RelationsInterface baseClass0;
baseClass0::Streamer(R__b);
R__b >> const_cast< const unsigned int &>( m_pdg ); // ERROR RAISED HERE
[/code]
We’re going to fix this locally, because writing to a const-member seems like a dangerous proposition anyway, but it’s definitely a change in behavior, and rootlcing should give an error instead of generating bad code.

Axel · March 14, 2014, 11:05am

Hi,

Please use account.cern.ch/account/Externa … count.aspx to get a CERN account. It’s rather trivial.

I’ll upload your reproducer to Jira, thanks!

Cheers, Axel.

Danilo · April 3, 2014, 12:36pm

Hi,

I closed the bug and marked it as non-reproducible.
sft.its.cern.ch/jira/browse/ROOT-6128

Cheers,
Danilo