Segmentation violation - Geant4 MT Build + FairRunSim


Hi, everyone. I have some segmentation violation when i use FairRunSim and Geant4 MT build. I have segmentation violation in both modes - MT and not.
Could anyone prompt place of problem based on stack trace? Thank you.

===========================================================
There was a crash.
This is the entire stack trace of all threads:
===========================================================

Thread 3 (Thread 0x7f6695e09700 (LWP 4162) “root.exe”):
#0 0x00007f66b2c29167 in __GI___wait4 (pid=4166, stat_loc=stat_loc
entry=0x7f6695e01028, options=options
entry=0, usage=usage
entry=0x0) at …/sysdeps/unix/sysv/linux/wait4.c:27
#1 0x00007f66b2c290e7 in __GI___waitpid (pid=, stat_loc=stat_loc
entry=0x7f6695e01028, options=options
entry=0) at waitpid.c:38
#2 0x00007f66b2ba6a07 in do_system (line=) at …/sysdeps/posix/system.c:172
#3 0x00007f66b32584bc in TUnixSystem::StackTrace() () from /opt/fairsoft/install/lib/root/libCore.so.6.16
#4 0x00007f66b325593c in TUnixSystem::DispatchSignals(ESignals) () from /opt/fairsoft/install/lib/root/libCore.so.6.16
#5
#6 0x00007f66a72b535d in TGeoVolumeAssembly::SetNextNodeIndex(int) () from /opt/fairsoft/install/lib/root/libGeom.so.6.16
#7 0x00007f66a728500f in TGeoShapeAssembly::DistFromOutside(double const, double const*, int, double, double*) const [clone .part.0] () from /opt/fairsoft/install/lib/root/libGeom.so.6.16*
#8 0x00007f66a7285173 in TGeoShapeAssembly::DistFromOutside(double const, double const*, int, double, double*) const [clone .part.0] () from /opt/fairsoft/install/lib/root/libGeom.so.6.16*
#9 0x00007f66a72528f7 in TGeoNavigator::FindNextDaughterBoundary(double, double*, int&, bool) () from /opt/fairsoft/install/lib/root/libGeom.so.6.16*
#10 0x00007f66a7256973 in TGeoNavigator::FindNextBoundary(double, char const, bool) () from /opt/fairsoft/install/lib/root/libGeom.so.6.16*
#11 0x00007f669e177b44 in TG4RootNavigator::ComputeStep(CLHEP::Hep3Vector const&, CLHEP::Hep3Vector const&, double, double&) () from /opt/fairsoft/install/lib/libg4root.so
#12 0x00007f669d0e159e in G4PropagatorInField::ComputeStep(G4FieldTrack&, double, double&, G4VPhysicalVolume) () from /opt/fairsoft/install/lib/libG4geometry.so*
#13 0x00007f6697828bd2 in G4Transportation::AlongStepGetPhysicalInteractionLength(G4Track const&, double, double, double&, G4GPILSelection) () from /opt/fairsoft/install/lib/libG4processes.so*
#14 0x00007f669d95c91d in G4SteppingManager::DefinePhysicalStepLength() () from /opt/fairsoft/install/lib/libG4tracking.so
#15 0x00007f669d95acc0 in G4SteppingManager::Stepping() () from /opt/fairsoft/install/lib/libG4tracking.so
#16 0x00007f669d9663ff in G4TrackingManager::ProcessOneTrack(G4Track) () from /opt/fairsoft/install/lib/libG4tracking.so*
#17 0x00007f669d9a3cd2 in G4EventManager::DoProcessing(G4Event) () from /opt/fairsoft/install/lib/libG4event.so*
#18 0x00007f669da4fc14 in G4WorkerRunManager::DoEventLoop(int, char const, int) () from /opt/fairsoft/install/lib/libG4run.so*
#19 0x00007f669da43c5e in G4RunManager::BeamOn(int, char const, int) () from /opt/fairsoft/install/lib/libG4run.so*
#20 0x00007f669da513a4 in G4WorkerRunManager::DoWork() () from /opt/fairsoft/install/lib/libG4run.so
#21 0x00007f669da5aaa1 in G4MTRunManagerKernel::StartThread(G4WorkerThread) () from /opt/fairsoft/install/lib/libG4run.so*
#22 0x00007f66b2e0bed0 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#23 0x00007f66b2a83ea7 in start_thread (arg=) at pthread_create.c:477
#24 0x00007f66b2c5bd4f in clone () at …/sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 2 (Thread 0x7f669660a700 (LWP 4161) “root.exe”):
#0 0x00007f66971a3fc3 in G4PiData::ReactionXSection(double) () from /opt/fairsoft/install/lib/libG4processes.so
#1 0x00007f669716e4d2 in G4ComponentBarNucleonNucleusXsc::ComputeCrossSections(G4ParticleDefinition const, double, int) () from /opt/fairsoft/install/lib/libG4processes.so*
#2 0x00007f669719fb2d in G4NucleonNuclearCrossSection::ComputeCrossSections(G4ParticleDefinition const, double, int) () from /opt/fairsoft/install/lib/libG4processes.so*
#3 0x00007f669719fb72 in G4NucleonNuclearCrossSection::GetElementCrossSection(G4DynamicParticle const, int, G4Material const*) () from /opt/fairsoft/install/lib/libG4processes.so*
#4 0x00007f669712253f in G4BGGNucleonInelasticXS::GetElementCrossSection(G4DynamicParticle const, int, G4Material const*) () from /opt/fairsoft/install/lib/libG4processes.so*
#5 0x00007f669717768b in G4CrossSectionDataStore::GetCrossSection(G4DynamicParticle const, G4Element const*, G4Material const*) () from /opt/fairsoft/install/lib/libG4processes.so*
#6 0x00007f6697178869 in G4CrossSectionDataStore::ComputeCrossSection(G4DynamicParticle const, G4Material const*) () from /opt/fairsoft/install/lib/libG4processes.so*
#7 0x00007f66971b8545 in G4HadronicProcess::GetMeanFreePath(G4Track const&, double, G4ForceCondition) () from /opt/fairsoft/install/lib/libG4processes.so*
#8 0x00007f66977ef74c in G4VDiscreteProcess::PostStepGetPhysicalInteractionLength(G4Track const&, double, G4ForceCondition) () from /opt/fairsoft/install/lib/libG4processes.so*
#9 0x00007f669d95c7d0 in G4SteppingManager::DefinePhysicalStepLength() () from /opt/fairsoft/install/lib/libG4tracking.so
#10 0x00007f669d95acc0 in G4SteppingManager::Stepping() () from /opt/fairsoft/install/lib/libG4tracking.so
#11 0x00007f669d9663ff in G4TrackingManager::ProcessOneTrack(G4Track) () from /opt/fairsoft/install/lib/libG4tracking.so*
#12 0x00007f669d9a3cd2 in G4EventManager::DoProcessing(G4Event) () from /opt/fairsoft/install/lib/libG4event.so*
#13 0x00007f669da4fc14 in G4WorkerRunManager::DoEventLoop(int, char const, int) () from /opt/fairsoft/install/lib/libG4run.so*
#14 0x00007f669da43c5e in G4RunManager::BeamOn(int, char const, int) () from /opt/fairsoft/install/lib/libG4run.so*
#15 0x00007f669da513a4 in G4WorkerRunManager::DoWork() () from /opt/fairsoft/install/lib/libG4run.so
#16 0x00007f669da5aaa1 in G4MTRunManagerKernel::StartThread(G4WorkerThread) () from /opt/fairsoft/install/lib/libG4run.so*
#17 0x00007f66b2e0bed0 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#18 0x00007f66b2a83ea7 in start_thread (arg=) at pthread_create.c:477
#19 0x00007f66b2c5bd4f in clone () at …/sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 1 (Thread 0x7f66b2932980 (LWP 4132) “root.exe”):
#0 futex_wait_cancelable (private=0, expected=0, futex_word=0x560d5b05a5ec) at …/sysdeps/nptl/futex-internal.h:183
#1 __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x560d5b05a598, cond=0x560d5b05a5c0) at pthread_cond_wait.c:508
#2 __pthread_cond_wait (cond=0x560d5b05a5c0, mutex=0x560d5b05a598) at pthread_cond_wait.c:638
#3 0x00007f66b2e0690c in std::condition_variable::wait(std::unique_lockstd::mutex&) () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#4 0x00007f669cddbdaa in G4MTBarrier::Wait() () from /opt/fairsoft/install/lib/libG4global.so
#5 0x00007f669da4e93c in G4MTRunManager::RunTermination() () from /opt/fairsoft/install/lib/libG4run.so
#6 0x00007f669da43c67 in G4RunManager::BeamOn(int, char const, int) () from /opt/fairsoft/install/lib/libG4run.so*
#7 0x00007f669e276d75 in TG4RunManager::ProcessRun(int) () from /opt/fairsoft/install/lib/libgeant4vmc.so
#8 0x00007f66a76e3c3c in FairMCApplication::RunMC (this=0x560d581bff90, nofEvents=100) at /opt/fairroot/base/sim/FairMCApplication.cxx:465
#9 0x00007f66a7adbc95 in ?? ()
#10 0x0000560d5a0241b0 in ?? ()
#11 0x00007f66aff4b20e in std::_Hashtable<unsigned int, std::pair<unsigned int const, llvm::SmallVector<llvm::RelocationEntry, 64u> >, std::allocator<std::pair<unsigned int const, llvm::SmallVector<llvm::RelocationEntry, 64u> > >, std::__detail::_Select1st, std::equal_to, std::hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::_M_insert_unique_node(unsigned int const&, unsigned long, unsigned long, std::__detail::_Hash_node<std::pair<unsigned int const, llvm::SmallVector<llvm::RelocationEntry, 64u> >, false>, unsigned long) [clone .isra.0] () from /opt/fairsoft/install/lib/root/libCling.so*
#12 0x00007f66b2c95ad4 in do_sym (handle=, name=0x560d5960bcf0 “Object array iterator”, who=0x7f66b1a67afc <llvm::sys::DynamicLibrary::SearchForAddressOfSymbol(char const)+892>, vers=0x0, flags=) at dl-sym.c:165*
#13 0x0000560d5d2eaf30 in ?? ()
#14 0x0000560d5d2eaf30 in ?? ()
#15 0x0000560d5d2eaf30 in ?? ()
#16 0x0000560d5d2eaf30 in ?? ()
#17 0x0000560d5d2eaf30 in ?? ()
#18 0x0000560d54ba37d0 in ?? ()
#19 0x0000560d54ba37d0 in ?? ()
#20 0x0000560d59d02e20 in ?? ()
#21 0x0000560d5a11eb60 in ?? ()
#22 0x0000560d5a11eb60 in ?? ()
#23 0x0000560d5a11eb60 in ?? ()
#24 0x0000000000000000 in ?? ()
===========================================================

The lines below might hint at the cause of the crash.
You may get help by asking at the ROOT forum http://root.cern.ch/forum
Only if you are really convinced it is a bug in ROOT then please submit a
report at http://root.cern.ch/bugs Please post the ENTIRE stack trace
from above as an attachment in addition to anything else
that might help us fixing this issue.
===========================================================
#6 0x00007f66a72b535d in TGeoVolumeAssembly::SetNextNodeIndex(int) () from /opt/fairsoft/install/lib/root/libGeom.so.6.16
#7 0x00007f66a728500f in TGeoShapeAssembly::DistFromOutside(double const, double const*, int, double, double*) const [clone .part.0] () from /opt/fairsoft/install/lib/root/libGeom.so.6.16*
#8 0x00007f66a7285173 in TGeoShapeAssembly::DistFromOutside(double const, double const*, int, double, double*) const [clone .part.0] () from /opt/fairsoft/install/lib/root/libGeom.so.6.16*
#9 0x00007f66a72528f7 in TGeoNavigator::FindNextDaughterBoundary(double, double*, int&, bool) () from /opt/fairsoft/install/lib/root/libGeom.so.6.16*
#10 0x00007f66a7256973 in TGeoNavigator::FindNextBoundary(double, char const, bool) () from /opt/fairsoft/install/lib/root/libGeom.so.6.16*
#11 0x00007f669e177b44 in TG4RootNavigator::ComputeStep(CLHEP::Hep3Vector const&, CLHEP::Hep3Vector const&, double, double&) () from /opt/fairsoft/install/lib/libg4root.so
#12 0x00007f669d0e159e in G4PropagatorInField::ComputeStep(G4FieldTrack&, double, double&, G4VPhysicalVolume) () from /opt/fairsoft/install/lib/libG4geometry.so*
#13 0x00007f6697828bd2 in G4Transportation::AlongStepGetPhysicalInteractionLength(G4Track const&, double, double, double&, G4GPILSelection) () from /opt/fairsoft/install/lib/libG4processes.so*
#14 0x00007f669d95c91d in G4SteppingManager::DefinePhysicalStepLength() () from /opt/fairsoft/install/lib/libG4tracking.so
#15 0x00007f669d95acc0 in G4SteppingManager::Stepping() () from /opt/fairsoft/install/lib/libG4tracking.so
#16 0x00007f669d9663ff in G4TrackingManager::ProcessOneTrack(G4Track) () from /opt/fairsoft/install/lib/libG4tracking.so*
#17 0x00007f669d9a3cd2 in G4EventManager::DoProcessing(G4Event) () from /opt/fairsoft/install/lib/libG4event.so*
#18 0x00007f669da4fc14 in G4WorkerRunManager::DoEventLoop(int, char const, int) () from /opt/fairsoft/install/lib/libG4run.so*
#19 0x00007f669da43c5e in G4RunManager::BeamOn(int, char const, int) () from /opt/fairsoft/install/lib/libG4run.so*
#20 0x00007f669da513a4 in G4WorkerRunManager::DoWork() () from /opt/fairsoft/install/lib/libG4run.so
#21 0x00007f669da5aaa1 in G4MTRunManagerKernel::StartThread(G4WorkerThread) () from /opt/fairsoft/install/lib/libG4run.so*
#22 0x00007f66b2e0bed0 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#23 0x00007f66b2a83ea7 in start_thread (arg=) at pthread_create.c:477
#24 0x00007f66b2c5bd4f in clone () at …/sysdeps/unix/sysv/linux/x86_64/clone.S:95

===========================================================

Root > [INFO] Filling MCTrack array…
[INFO] Number of primaries = 10
[INFO] Total number of particles = 208
[INFO] Number of tracks in output = 10
[INFO] Updating track indizes…
[INFO] …stack and 7 collections updated.

  • *** Break *** segmentation violation*

ROOT Version: 6 // FairRoot
Platform: Debian 11
Compiler: Gcc 10.0.2


Hi, very hard to say just from the trace which shows a standard Geant4_VMC calling sequence dying in geometry in a setter, which signals a corrupted pointer. You may need to run again with a Debug build and print the state (data members and local variables) when the crash happens.

@agheata

You should check stack trace - segfault comes from debug part of the code, which disabled by default - I guess. Not sure if this code regularly tested and all threads data (where it fails) initialized correctly.