Crash involving TUnixSystem

Viesturs · December 13, 2021, 9:53pm

I see a crash after a successful completion of the binary. The crash involves TUnixSystem.

Is the crash related to ROOT? Can you please suggest what might be wrong and how to check?


#0  0x00007fbb30b05659 in waitpid () from /lib64/libc.so.6
#1  0x00007fbb30a82f62 in do_system () from /lib64/libc.so.6
#2  0x00007fbb30a83311 in system () from /lib64/libc.so.6
#3  0x00007fbb300d2702 in TUnixSystem::StackTrace() () from /cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_4_11/external/slc7_amd64_gcc630/lib/libCore.so
#4  0x00007fbb300d4bac in TUnixSystem::DispatchSignals(ESignals) () from /cvmfs/cms.cern.ch/slc7_amd64_gcc630/cms/cmssw/CMSSW_9_4_11/external/slc7_amd64_gcc630/lib/libCore.so
#5  <signal handler called>
#6  0x00007fbb18ff5f91 in std::_Rb_tree<Definitions::WorkEnum_t, std::pair<Definitions::WorkEnum_t const, char const*>, std::_Select1st<std::pair<Definitions::WorkEnum_t const, char const*> >, std::less<Definitions::WorkEnum_t>, std::allocator<std::pair<Definitions::WorkEnum_t const, char const*> > >::_S_right (__x=0x6574737953657461) at /cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gcc/6.3.0/include/c++/6.3.0/bits/stl_tree.h:701
#7  0x00007fbb18ffddbb in std::_Rb_tree<Definitions::WorkEnum_t, std::pair<Definitions::WorkEnum_t const, char const*>, std::_Select1st<std::pair<Definitions::WorkEnum_t const, char const*> >, std::less<Definitions::WorkEnum_t>, std::allocator<std::pair<Definitions::WorkEnum_t const, char const*> > >::_M_erase (this=0x7fbb23f91040 <Definitions::tag_levels_types_>, __x=0x6574737953657461) at /cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gcc/6.3.0/include/c++/6.3.0/bits/stl_tree.h:1638
#8  0x00007fbb18ffddcd in std::_Rb_tree<Definitions::WorkEnum_t, std::pair<Definitions::WorkEnum_t const, char const*>, std::_Select1st<std::pair<Definitions::WorkEnum_t const, char const*> >, std::less<Definitions::WorkEnum_t>, std::allocator<std::pair<Definitions::WorkEnum_t const, char const*> > >::_M_erase (this=0x7fbb23f91040 <Definitions::tag_levels_types_>, __x=0x5cd9b60) at /cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gcc/6.3.0/include/c++/6.3.0/bits/stl_tree.h:1638
#9  0x00007fbb18ffdc4a in std::_Rb_tree<Definitions::WorkEnum_t, std::pair<Definitions::WorkEnum_t const, char const*>, std::_Select1st<std::pair<Definitions::WorkEnum_t const, char const*> >, std::less<Definitions::WorkEnum_t>, std::allocator<std::pair<Definitions::WorkEnum_t const, char const*> > >::~_Rb_tree (this=0x7fbb23f91040 <Definitions::tag_levels_types_>, __in_chrg=<optimized out>) at /cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gcc/6.3.0/include/c++/6.3.0/bits/stl_tree.h:873
#10 0x00007fbb18ffdaa4 in std::map<Definitions::WorkEnum_t, char const*, std::less<Definitions::WorkEnum_t>, std::allocator<std::pair<Definitions::WorkEnum_t const, char const*> > >::~map (this=0x7fbb23f91040 <Definitions::tag_levels_types_>, __in_chrg=<optimized out>) at /cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gcc/6.3.0/include/c++/6.3.0/bits/stl_map.h:96
#11 0x00007fbb30a79ce9 in __run_exit_handlers () from /lib64/libc.so.6
#12 0x00007fbb30a79d37 in exit () from /lib64/libc.so.6
#13 0x00007fbb318670af in Py_Exit (sts=sts
entry=0) at Python/pythonrun.c:1779
#14 0x00007fbb318671d6 in handle_system_exit () at Python/pythonrun.c:1151
#15 0x00007fbb3186740d in handle_system_exit () at Python/pythonrun.c:1173
#16 PyErr_PrintEx (set_sys_last_vars=set_sys_last_vars
entry=1) at Python/pythonrun.c:1161
#17 0x00007fbb318675ea in PyErr_Print () at Python/pythonrun.c:1064
#18 0x00007fbb31867e14 in PyRun_SimpleFileExFlags (fp=<optimized out>, fp
entry=0x1dc8650, filename=<optimized out>, closeit=closeit
entry=1, flags=flags
entry=0x7ffd67232e80) at Python/pythonrun.c:952
#19 0x00007fbb318683c3 in PyRun_AnyFileExFlags (fp=fp
entry=0x1dc8650, filename=<optimized out>, closeit=closeit
entry=1, flags=flags
entry=0x7ffd67232e80) at Python/pythonrun.c:752
#20 0x00007fbb3187e211 in Py_Main (argc=<optimized out>, argv=<optimized out>) at Modules/main.c:640
#21 0x00007fbb30a62555 in __libc_start_main () from /lib64/libc.so.6
#22 0x0000000000400671 in _start ()

Please read tips for efficient and successful posting and posting code

ROOT Version: Not Provided
Platform: Not Provided
Compiler: Not Provided

pcanal · December 14, 2021, 3:30am

No. The only part involving ROOT is that it provide the stack trace printers. The problem is solely in:

#6  0x00007fbb18ff5f91 in std::_Rb_tree<Definitions::WorkEnum_t, std::pair<Definitions::WorkEnum_t const, char const*>, std::_Select1st<std::pair<Definitions::WorkEnum_t const, char const*> >, std::less<Definitions::WorkEnum_t>, std::allocator<std::pair<Definitions::WorkEnum_t const, char const*> > >::_S_right (__x=0x6574737953657461) at /cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gcc/6.3.0/include/c++/6.3.0/bits/stl_tree.h:701
#7  0x00007fbb18ffddbb in std::_Rb_tree<Definitions::WorkEnum_t, std::pair<Definitions::WorkEnum_t const, char const*>, std::_Select1st<std::pair<Definitions::WorkEnum_t const, char const*> >, std::less<Definitions::WorkEnum_t>, std::allocator<std::pair<Definitions::WorkEnum_t const, char const*> > >::_M_erase (this=0x7fbb23f91040 <Definitions::tag_levels_types_>, __x=0x6574737953657461) at /cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gcc/6.3.0/include/c++/6.3.0/bits/stl_tree.h:1638
#8  0x00007fbb18ffddcd in std::_Rb_tree<Definitions::WorkEnum_t, std::pair<Definitions::WorkEnum_t const, char const*>, std::_Select1st<std::pair<Definitions::WorkEnum_t const, char const*> >, std::less<Definitions::WorkEnum_t>, std::allocator<std::pair<Definitions::WorkEnum_t const, char const*> > >::_M_erase (this=0x7fbb23f91040 <Definitions::tag_levels_types_>, __x=0x5cd9b60) at /cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gcc/6.3.0/include/c++/6.3.0/bits/stl_tree.h:1638
#9  0x00007fbb18ffdc4a in std::_Rb_tree<Definitions::WorkEnum_t, std::pair<Definitions::WorkEnum_t const, char const*>, std::_Select1st<std::pair<Definitions::WorkEnum_t const, char const*> >, std::less<Definitions::WorkEnum_t>, std::allocator<std::pair<Definitions::WorkEnum_t const, char const*> > >::~_Rb_tree (this=0x7fbb23f91040 <Definitions::tag_levels_types_>, __in_chrg=<optimized out>) at /cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gcc/6.3.0/include/c++/6.3.0/bits/stl_tree.h:873
#10 0x00007fbb18ffdaa4 in std::map<Definitions::WorkEnum_t, char const*, std::less<Definitions::WorkEnum_t>, std::allocator<std::pair<Definitions::WorkEnum_t const, char const*> > >::~map (this=0x7fbb23f91040 <Definitions::tag_levels_types_>, __in_chrg=<optimized out>) at /cvmfs/cms.cern.ch/slc7_amd64_gcc630/external/gcc/6.3.0/include/c++/6.3.0/bits/stl_map.h:96
#11 0x00007fbb30a79ce9 in __run_exit_handlers () from /lib64/libc.so.6
#12 0x00007fbb30a79d37 in exit () from /lib64/libc.so.6
#13 0x00007fbb318670af in Py_Exit (sts=sts entry=0) at Python/pythonrun.c:1779
....

It is not clear what the problem is from just the stack trace. Running with valgrind (use `–suppressions=$ROOTSYS/etc/valgrind-root.supp to hide false report related to ROOT). might help pin-point the problem.

One thing of note is that this is happening during the destruction of an std::map during the process tear down. So this might be an order of destruction issue. You might need to change the code patter to avoid using that std::map as a global object.

Viesturs · December 14, 2021, 6:50am

is it safe to use const char * as map values?

Viesturs · December 14, 2021, 7:32am

I identified that there was a shadowing global declaration of map<WorkEnum_t, const char *> which was causing a conflict in the destruction.

But please tell me if it is safe to use const char * as a map value as it is just a pointer to stack and possibly could be deallocated.

Axel · December 16, 2021, 8:34am

In general that should be fine, also because the map destructor will not need to touch these literals.