TMapFile crashes with ROOT Version 6.04

itoh · December 10, 2015, 3:57am

Hi,

I’m now trying to port my Belle2 DAQ software running with ROOT version 5.34 to ROOT version 6.04. The ROOT is running on Scientific Linux 5.7 / x86_64. The Linux kernel version is 2.6.18.

I am using TMapFile for the real-time histogram monitoring and it worked fine with previous version 5, but I found it crashes with version 6. The core dump shows:

Fatal in <operator delete>: storage area overwritten
aborting
#0  0x0000003a85e99dd5 in waitpid () from /lib64/libc.so.6
#1  0x0000003a85e3c4a1 in do_system () from /lib64/libc.so.6
#2  0x00002af3c1315152 in TUnixSystem::Exec (this=0x1fc48ab8, shellcmd=0x1fc8a588 "/home/usr/hltdaq/belle2/externals/v01-01-01/Linux_x86_64/debug/root/etc/gdb-backtrace.sh 13839 1>&2") at /home/usr/hltdaq/belle2/externals/v01-01-01/src/root/core/unix/src/TUnixSystem.cxx:2069
#3  0x00002af3c131599e in TUnixSystem::StackTrace (this=0x1fc48ab8) at /home/usr/hltdaq/belle2/externals/v01-01-01/src/root/core/unix/src/TUnixSystem.cxx:2297
#4  0x00002af3c122377f in DefaultErrorHandler (level=6000, abort_bool=true, location=0x2af3c442de6c "operator delete", msg=0x1fc81a78 "storage area overwritten") at /home/usr/hltdaq/belle2/externals/v01-01-01/src/root/core/base/src/TError.cxx:194
#5  0x00002af3c1223a92 in ErrorHandler(Int_t, const char *, const char *, typedef __va_list_tag __va_list_tag *) (level=6000, location=0x2af3c442de6c "operator delete", fmt=0x2af3c442de30 "%s", ap=0x7fff7dc68538) at /home/usr/hltdaq/belle2/externals/v01-01-01/src/root/core/base/src/TError.cxx:249
#6  0x00002af3c1223f7e in Fatal (location=0x2af3c442de6c "operator delete", fmt=0x2af3c442de30 "%s") at /home/usr/hltdaq/belle2/externals/v01-01-01/src/root/core/base/src/TError.cxx:345
#7  0x00002af3c442d366 in operator delete (ptr=0x1fc89e70) at /home/usr/hltdaq/belle2/externals/v01-01-01/src/root/core/newdelete/src/NewDelete.cxx:270
#8  0x00002af3c5e7600b in clang::driver::toolchains::Linux::Linux(clang::driver::Driver const&, llvm::Triple const&, llvm::opt::ArgList const&) () from /home/usr/hltdaq/belle2/externals/v01-01-01/Linux_x86_64/debug/root/lib/libCling.so
#9  0x00002af3c5e4b9f7 in clang::driver::Driver::getToolChain(llvm::opt::ArgList const&, llvm::StringRef) const () from /home/usr/hltdaq/belle2/externals/v01-01-01/Linux_x86_64/debug/root/lib/libCling.so
#10 0x00002af3c5e541e2 in clang::driver::Driver::BuildCompilation(llvm::ArrayRef<char const*>) () from /home/usr/hltdaq/belle2/externals/v01-01-01/Linux_x86_64/debug/root/lib/libCling.so
#11 0x00002af3c5b9017d in (anonymous namespace)::createCIImpl (buffer=..., argc=8, argv=0x1fc7eca8, llvmdir=0x1fc7d680 "/home/usr/hltdaq/belle2/externals/v01-01-01/Linux_x86_64/debug/root/etc/cling", OnlyLex=false) at /home/usr/hltdaq/belle2/externals/v01-01-01/src/root/interpreter/cling/lib/Interpreter/CIFactory.cpp:635
#12 0x00002af3c5b91131 in cling::CIFactory::createCI (code=..., argc=8, argv=0x1fc7eca8, llvmdir=0x1fc7d680 "/home/usr/hltdaq/belle2/externals/v01-01-01/Linux_x86_64/debug/root/etc/cling") at /home/usr/hltdaq/belle2/externals/v01-01-01/src/root/interpreter/cling/lib/Interpreter/CIFactory.cpp:798
#13 0x00002af3c5c4e2c0 in cling::IncrementalParser::IncrementalParser (this=0x1fc7ed08, interp=0x1fc7d4b8, argc=8, argv=0x1fc7eca8, llvmdir=0x1fc7d680 "/home/usr/hltdaq/belle2/externals/v01-01-01/Linux_x86_64/debug/root/etc/cling") at /home/usr/hltdaq/belle2/externals/v01-01-01/src/root/interpreter/cling/lib/Interpreter/IncrementalParser.cpp:163
#14 0x00002af3c5c5a84b in cling::Interpreter::Interpreter (this=0x1fc7d4b8, argc=8, argv=0x1fc7d458, llvmdir=0x1fc7d680 "/home/usr/hltdaq/belle2/externals/v01-01-01/Linux_x86_64/debug/root/etc/cling", noRuntime=false) at /home/usr/hltdaq/belle2/externals/v01-01-01/src/root/interpreter/cling/lib/Interpreter/Interpreter.cpp:185
#15 0x00002af3c59c5471 in TCling::TCling (this=0x1fc7cc38, name=0x2af3c7186367 "C++", title=0x2af3c7186351 "cling C++ Interpreter") at /home/usr/hltdaq/belle2/externals/v01-01-01/src/root/core/meta/src/TCling.cxx:1088
#16 0x00002af3c59c256c in CreateInterpreter (interpLibHandle=0x1fc6a8c0) at /home/usr/hltdaq/belle2/externals/v01-01-01/src/root/core/meta/src/TCling.cxx:581
#17 0x00002af3c1254cac in TROOT::InitInterpreter (this=0x2af3c16a5480 <ROOT::GetROOT1()::alloc>) at /home/usr/hltdaq/belle2/externals/v01-01-01/src/root/core/base/src/TROOT.cxx:1739
#18 0x00002af3c124e766 in ROOT::GetROOT2 () at /home/usr/hltdaq/belle2/externals/v01-01-01/src/root/core/base/src/TROOT.cxx:353
#19 0x00002af3c124e791 in ROOT::GetROOT () at /home/usr/hltdaq/belle2/externals/v01-01-01/src/root/core/base/src/TROOT.cxx:362
#20 0x00002af3c442da63 in TMapFile::WhichMapFile (addr=0x1fc489e0) at include/TMapFile.h:195
#21 0x00002af3c442d3c7 in operator delete (ptr=0x1fc489e8) at /home/usr/hltdaq/belle2/externals/v01-01-01/src/root/core/newdelete/src/NewDelete.cxx:273
#22 0x00002af3c1346214 in __gnu_cxx::new_allocator<unsigned long>::deallocate (this=0x2af3c16c2630 <TClassEdit::CleanType(char const*, int, char const**)::lengths>, __p=0x1fc489e8) at /home/usr/hltdaq/belle2/externals/v01-01-01/include/c++/ext/new_allocator.h:110
#23 0x00002af3c1345d8a in std::allocator_traits<std::allocator<unsigned long> >::deallocate (__a=..., __p=0x1fc489e8, __n=1) at /home/usr/hltdaq/belle2/externals/v01-01-01/include/c++/bits/alloc_traits.h:386
#24 0x00002af3c13457f0 in std::_Vector_base<unsigned long, std::allocator<unsigned long> >::_M_deallocate (this=0x2af3c16c2630 <TClassEdit::CleanType(char const*, int, char const**)::lengths>, __p=0x1fc489e8, __n=1) at /home/usr/hltdaq/belle2/externals/v01-01-01/include/c++/bits/stl_vector.h:178
#25 0x00002af3c1345974 in std::vector<unsigned long, std::allocator<unsigned long> >::_M_emplace_back_aux<unsigned long>(unsigned long&&) (this=0x2af3c16c2630 <TClassEdit::CleanType(char const*, int, char const**)::lengths>) at /home/usr/hltdaq/belle2/externals/v01-01-01/include/c++/bits/vector.tcc:438
#26 0x00002af3c134507b in std::vector<unsigned long, std::allocator<unsigned long> >::emplace_back<unsigned long>(unsigned long&&) (this=0x2af3c16c2630 <TClassEdit::CleanType(char const*, int, char const**)::lengths>) at /home/usr/hltdaq/belle2/externals/v01-01-01/include/c++/bits/vector.tcc:101
#27 0x00002af3c1344902 in std::vector<unsigned long, std::allocator<unsigned long> >::push_back(unsigned long&&) (this=0x2af3c16c2630 <TClassEdit::CleanType(char const*, int, char const**)::lengths>, __x=<unknown type in /home/usr/hltdaq/belle2/externals/v01-01-01/Linux_x86_64/debug/root/lib/libCore.so, CU 0x4673f9, DIE 0x47674e>) at /home/usr/hltdaq/belle2/externals/v01-01-01/include/c++/bits/stl_vector.h:932
#28 0x00002af3c1340bf1 in TClassEdit::CleanType (typeDesc=0x1fc489d0 "string", mode=0, tail=0x0) at /home/usr/hltdaq/belle2/externals/v01-01-01/src/root/core/metautils/src/TClassEdit.cxx:1110
#29 0x00002af3c133fc17 in TClassEdit::GetSplit (type=0x1fc489d0 "string", output=..., nestedLoc=
0x7fff7dc6abe0: 0, mode=(TClassEdit::kDropStlDefault | TClassEdit::kLong64 | TClassEdit::kDropStd | TClassEdit::kKeepOuterConst)) at /home/usr/hltdaq/belle2/externals/v01-01-01/src/root/core/metautils/src/TClassEdit.cxx:929
#30 0x00002af3c133d142 in TClassEdit::TSplitType::TSplitType (this=0x7fff7dc6abc0, type2split=0x1fc489d0 "string", mode=(TClassEdit::kDropStlDefault | TClassEdit::kLong64 | TClassEdit::kDropStd | TClassEdit::kKeepOuterConst)) at /home/usr/hltdaq/belle2/externals/v01-01-01/src/root/core/metautils/src/TClassEdit.cxx:125
#31 0x00002af3c133f3eb in TClassEdit::GetNormalizedName (norm_name=..., name=...) at /home/usr/hltdaq/belle2/externals/v01-01-01/src/root/core/metautils/src/TClassEdit.cxx:775
#32 0x00002af3c129fec8 in TClassTable::FindElement (cname=0x2af3c137c47c "string", insert=false) at /home/usr/hltdaq/belle2/externals/v01-01-01/src/root/core/cont/src/TClassTable.cxx:503
#33 0x00002af3c12a073f in ROOT::ResetClassVersion (cl=0x0, cname=0x2af3c137c47c "string", newid=2) at /home/usr/hltdaq/belle2/externals/v01-01-01/src/root/core/cont/src/TClassTable.cxx:739
#34 0x00002af3c12e9d5d in ROOT::TGenericClassInfo::SetVersion (this=0x2af3c16a40e0 <ROOT::GenerateInitInstanceLocal(std::string const*)::instance>, version=2) at /home/usr/hltdaq/belle2/externals/v01-01-01/src/root/core/meta/src/TGenericClassInfo.cxx:370
#35 0x00002af3c11f82de in __static_initialization_and_destruction_0 (__initialize_p=1, __priority=65535) at /home/usr/hltdaq/belle2/externals/v01-01-01/src/root/core/base/src/String.cxx:38
#36 0x00002af3c11f82fb in _GLOBAL__sub_I_String.cxx(void) () at /home/usr/hltdaq/belle2/externals/v01-01-01/src/root/core/base/src/String.cxx:38
#37 0x0000003a85a0d4f3 in call_init () from /lib64/ld-linux-x86-64.so.2
#38 0x0000003a85a0d5b5 in _dl_init_internal () from /lib64/ld-linux-x86-64.so.2
#39 0x0000003a85a00aaa in _dl_start_user () from /lib64/ld-linux-x86-64.so.2
#40 0x0000000000000001 in ?? ()
#41 0x00007fff7dc6cb3c in ?? ()
#42 0x0000000000000000 in ?? ()
Abort (core dumped)

I found the same problem occurs even when just invoking rootn.exe.
I also tried to compile and link a simple program “hsimple.cxx” with and without libNew.so,
and found that the same problem occurs when libNew.so is linked, although it never occurs without libNew.so.

Judging from the fact, I suspect the problem occurs when loading “libNew.so”.
Does anyone recognize this problem? If yes, could you tell me how to fix it?

Thanks in advance,
Ryosuke Itoh
Belle II DAQ group

pcanal · December 10, 2015, 4:15am

Hi Itoh,

Do you recall why you need to use libNew and/or rootn? I.e. is no longer using them an option?

Cheers,
Philippe.

itoh · December 10, 2015, 6:21am

Dear Philippe,

thanks for the reply.

The reason why I need libNew is that I get the message

Error in <TMapFile::TMapFile>: no memory mapped file capability available
Use rootn.exe or link application against "-lNew"

when I link my app with TMapFile with ROOT6. Is there any method to use TMapFile without libNew avoiding this message?

Best,

R.Itoh

pcanal · December 10, 2015, 9:18pm

Hi,

If you need the TMapFile functionality … indeed you will need libNew and we would need to get it to work properly in v6.

An alternative to use TMapFile is to use a TMemFile to which you give some shared memory as backing store (in the constructor). The main difference is that you might have to take care of the synchronization (avoiding reader and writer to collide) externally.

Cheers,
Philippe.

itoh · December 11, 2015, 5:24am

Dear Phillipe,

thanks for the info. I will try TMemFile.

As for TMapFile, my colleague tried to use it on MacOSX with ROOT6, and he reported it worked. He could load libNew.so without any trouble. So this problem could be specific to SL5. We are using gcc 5.2.0 to compile ROOT and our apps. Can it be the source of the problem?

Best,

R.Itoh

pcanal · December 11, 2015, 8:25pm

[quote]We are using gcc 5.2.0 to compile ROOT and our apps.[/quote]It could be. gcc v5.2 default to a new ABI that is not yet compatible with clang (and thus cling). Did you revert to use the old ABI?

Cheers,
Philippe.

itoh · December 15, 2015, 4:49am

Dear Philippe,

thank you for the info. I will try again with gcc4. But currently ROOT is compiled together with Belle2 library which requires gcc5 and some effort may be required. Anyway, I will try.

Best,

R.Itoh

pcanal · September 13, 2016, 6:55am

Hi,

Please note that the version in the master (and the upcoming v6.08) now work properly with the new gcc ABI.

Cheers,
Philippe.

jaromrax · November 2, 2016, 9:43pm

Hello, I have the same problem - TMapFile not working

mmap.histo file found Error in <TMapFile::TMapFile>: no memory mapped file capability available Use rootn.exe or link application against "-lNew"
in root.6.06.06. Now I compiled root 6.06.08 with the same result. My system is ubuntu 16.04. Any news/ideas? Thank you.

pcanal · January 18, 2017, 10:09pm

Hi,

The TMapFile problem is finally resolved in the master and the v6.08 patch branch (so in v6.10/00 and v6.08/06.

Our apologies for the long delay and thanks for your patience.

Cheers,
Philippe.