Segmentation violation only with some ntuples

Hi everyone,
I have a big problem that I can’t find a solution to.

I have a macro, Run.py, where I run my code on several ntuples.
For most of them I don’t have any type of problem, the code works good and it’s ok.
But for two nutples the code crash and make Segmentation fault (ntuples has only different data but the structure is the same for all of them).
I’ll report the crash:

 > Loading and compiling C++ codes...

reading file/eos/user/e/epompapa/public/datasets/ntuples/signal/isoWP/applied/passed/user.epompapa.signal.passedIso.16Jul2023.511308.e8427_s3126_r10201_p5085_ANALYSIS.root/user.epompapa.34091913._000001.ANALYSIS.root
reading entry
0
--------------isMC is---------------
1
 > Creating new TChain with tree T
 > Adding ntuples:
   /eos/user/e/epompapa/public/datasets/ntuples/signal/isoWP/applied/passed/user.epompapa.signal.passedIso.16Jul2023.511308.e8427_s3126_r10201_p5085_ANALYSIS.root/user.epompapa.34091913._000001.ANALYSIS.root
   /eos/user/e/epompapa/public/datasets/ntuples/signal/isoWP/applied/passed/user.epompapa.signal.passedIso.16Jul2023.511308.e8427_s3126_r10724_p5085_ANALYSIS.root/user.epompapa.34091914._000001.ANALYSIS.root
   /eos/user/e/epompapa/public/datasets/ntuples/signal/isoWP/applied/passed/user.epompapa.signal.passedIso.16Jul2023.511308.e8427_s3126_r9364_p5085_ANALYSIS.root/user.epompapa.34091915._000001.ANALYSIS.root
/eos/user/e/epompapa/public/datasets/ntuples/signal/isoWP/applied/passed/user.epompapa.signal.passedIso.16Jul2023.511308.e8427_s3126_r10201_p5085_ANALYSIS.root/user.epompapa.34091913._000001.ANALYSIS.root
is campaign d
0.0
734007.625
0.0
/eos/user/e/epompapa/public/datasets/ntuples/signal/isoWP/applied/passed/user.epompapa.signal.passedIso.16Jul2023.511308.e8427_s3126_r10724_p5085_ANALYSIS.root/user.epompapa.34091914._000001.ANALYSIS.root
is campaign e
0.0
734007.625
954245.625
/eos/user/e/epompapa/public/datasets/ntuples/signal/isoWP/applied/passed/user.epompapa.signal.passedIso.16Jul2023.511308.e8427_s3126_r9364_p5085_ANALYSIS.root/user.epompapa.34091915._000001.ANALYSIS.root
is campaign a
587222.9375
734007.625
954245.625
 > Output histograms will be saved in 6000MeV.root
 > Total entries:  309999

 > OK, Setup done, running C++ code now
 > Called DPAnalyzer::Begin 
 *** Break *** segmentation violation



===========================================================
There was a crash.
This is the entire stack trace of all threads:
===========================================================

Thread 2 (Thread 0x7fad55ae5700 (LWP 23125)):
#0  0x00007fad93d86de2 in pthread_cond_timedwait

GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007fad9431b465 in PyCOND_TIMEDWAIT (cond=0x7fad94712c88 <_PyRuntime+1184>, mut=0x7fad94712cb8 <_PyRuntime+1232>, us=<optimized out>) at /workspace/build/externals/Python-3.8.6/src/Python/3.8.6/Python/condvar.h:73
#2  take_gil (ceval=0x7fad94712a30 <_PyRuntime+584>, tstate=0x4ea3320) at /workspace/build/externals/Python-3.8.6/src/Python/3.8.6/Python/ceval_gil.h:206
#3  0x00007fad9431ba10 in PyEval_RestoreThread (tstate=0x4ea3320) at /workspace/build/externals/Python-3.8.6/src/Python/3.8.6/Python/ceval.c:399
#4  0x00007fad943c1b76 in pysleep (secs=<optimized out>) at /workspace/build/externals/Python-3.8.6/src/Python/3.8.6/Modules/timemodule.c:1850
#5  time_sleep (self=<optimized out>, obj=<optimized out>) at /workspace/build/externals/Python-3.8.6/src/Python/3.8.6/Modules/timemodule.c:338
#6  0x00007fad942957c4 in cfunction_vectorcall_O (func=0x7fad9487b270, args=0x7fad2570a1c0, nargsf=<optimized out>, kwnames=<optimized out>) at /workspace/build/externals/Python-3.8.6/src/Python/3.8.6/Objects/methodobject.c:482
#7  0x00007fad943257b1 in _PyObject_Vectorcall (callable=0x7fad9487b270, args=0x7fad2570a1c0, nargsf=9223372036854775809, kwnames=0x0) at /workspace/build/externals/Python-3.8.6/src/Python/3.8.6/Include/cpython/abstract.h:127
#8  call_function (tstate=<optimized out>, pp_stack=0x7fad55ae4668, oparg=<optimized out>, kwnames=0x0) at /workspace/build/externals/Python-3.8.6/src/Python/3.8.6/Python/ceval.c:4963
#9  0x00007fad94322669 in _PyEval_EvalFrameDefault (f=0x7fad2570a040, throwflag=<optimized out>) at /workspace/build/externals/Python-3.8.6/src/Python/3.8.6/Python/ceval.c:3469
#10 0x00007fad94326863 in PyEval_EvalFrameEx (f=0x7fad2570a040, throwflag=0) at /workspace/build/externals/Python-3.8.6/src/Python/3.8.6/Python/ceval.c:741
#11 _PyEval_EvalCodeWithName (_co=<optimized out>, globals=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, kwnames=0x7fad2570a1a8, kwargs=0x7fad8c7dc4e0, kwcount=0, kwstep=1, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0, name=0x7fad256ffe90, qualname=0x7fad256f5880) at /workspace/build/externals/Python-3.8.6/src/Python/3.8.6/Python/ceval.c:4298
#12 0x00007fad942581c9 in _PyFunction_Vectorcall (func=<optimized out>, stack=0xffffffffffffffff, nargsf=<optimized out>, kwnames=<optimized out>) at /workspace/build/externals/Python-3.8.6/src/Python/3.8.6/Objects/call.c:435
#13 0x00007fad94257a39 in PyVectorcall_Call (callable=0x7fad25707820, tuple=<optimized out>, kwargs=<optimized out>) at /workspace/build/externals/Python-3.8.6/src/Python/3.8.6/Objects/call.c:199
#14 0x00007fad94257b18 in PyObject_Call (callable=<optimized out>, args=<optimized out>, kwargs=<optimized out>) at /workspace/build/externals/Python-3.8.6/src/Python/3.8.6/Objects/call.c:227
#15 0x00007fad9432296d in do_call_core (tstate=0x4ea3320, func=0x7fad25707820, callargs=0x7fad8c7dc4c0, kwdict=0x7fad947ee4c0) at /workspace/build/externals/Python-3.8.6/src/Python/3.8.6/Python/ceval.c:5010
#16 _PyEval_EvalFrameDefault (f=0x7fad25b30e40, throwflag=<optimized out>) at /workspace/build/externals/Python-3.8.6/src/Python/3.8.6/Python/ceval.c:3559
#17 0x00007fad9425804f in function_code_fastcall (co=<optimized out>, args=0x7fad255dfbb8, nargs=<optimized out>, globals=<optimized out>) at /workspace/build/externals/Python-3.8.6/src/Python/3.8.6/Objects/call.c:283
#18 0x00007fad943257b1 in _PyObject_Vectorcall (callable=0x7fad8c8375e0, args=0x7fad255dfbb8, nargsf=9223372036854775809, kwnames=0x0) at /workspace/build/externals/Python-3.8.6/src/Python/3.8.6/Include/cpython/abstract.h:127
#19 call_function (tstate=<optimized out>, pp_stack=0x7fad55ae4b18, oparg=<optimized out>, kwnames=0x0) at /workspace/build/externals/Python-3.8.6/src/Python/3.8.6/Python/ceval.c:4963
#20 0x00007fad9432264b in _PyEval_EvalFrameDefault (f=0x7fad255dfa40, throwflag=<optimized out>) at /workspace/build/externals/Python-3.8.6/src/Python/3.8.6/Python/ceval.c:3486
#21 0x00007fad9425804f in function_code_fastcall (co=<optimized out>, args=0x7fad25b30538, nargs=<optimized out>, globals=<optimized out>) at /workspace/build/externals/Python-3.8.6/src/Python/3.8.6/Objects/call.c:283
#22 0x00007fad943257b1 in _PyObject_Vectorcall (callable=0x7fad8c8378b0, args=0x7fad25b30538, nargsf=9223372036854775809, kwnames=0x0) at /workspace/build/externals/Python-3.8.6/src/Python/3.8.6/Include/cpython/abstract.h:127
#23 call_function (tstate=<optimized out>, pp_stack=0x7fad55ae4cf8, oparg=<optimized out>, kwnames=0x0) at /workspace/build/externals/Python-3.8.6/src/Python/3.8.6/Python/ceval.c:4963
#24 0x00007fad9432264b in _PyEval_EvalFrameDefault (f=0x7fad25b303c0, throwflag=<optimized out>) at /workspace/build/externals/Python-3.8.6/src/Python/3.8.6/Python/ceval.c:3486
#25 0x00007fad9425804f in function_code_fastcall (co=<optimized out>, args=0x7fad55ae4e28, nargs=<optimized out>, globals=<optimized out>) at /workspace/build/externals/Python-3.8.6/src/Python/3.8.6/Objects/call.c:283
#26 0x00007fad9425a74c in _PyObject_Vectorcall (callable=0x7fad8c837670, args=0x7fad55ae4e28, nargsf=1, kwnames=0x0) at /workspace/build/externals/Python-3.8.6/src/Python/3.8.6/Include/cpython/abstract.h:127
#27 method_vectorcall (method=<optimized out>, args=0x7fad948be058, nargsf=<optimized out>, kwnames=<optimized out>) at /workspace/build/externals/Python-3.8.6/src/Python/3.8.6/Objects/classobject.c:67
#28 0x00007fad94257a39 in PyVectorcall_Call (callable=0x7fad9487a340, tuple=<optimized out>, kwargs=<optimized out>) at /workspace/build/externals/Python-3.8.6/src/Python/3.8.6/Objects/call.c:199
#29 0x00007fad94257b18 in PyObject_Call (callable=<optimized out>, args=<optimized out>, kwargs=<optimized out>) at /workspace/build/externals/Python-3.8.6/src/Python/3.8.6/Objects/call.c:227
#30 0x00007fad943c41d8 in t_bootstrap (boot_raw=0x7fad257090c0) at /workspace/build/externals/Python-3.8.6/src/Python/3.8.6/Modules/_threadmodule.c:1002
#31 0x00007fad94376496 in pythread_wrapper (arg=<optimized out>) at /workspace/build/externals/Python-3.8.6/src/Python/3.8.6/Python/thread_pthread.h:232
#32 0x00007fad93d82ea5 in start_thread () from /lib64/libpthread.so.0
#33 0x00007fad933a2b0d in clone () from /lib64/libc.so.6

Thread 1 (Thread 0x7fad948fe740 (LWP 23052)):
#0  0x00007fad93369659 in waitpid () from /lib64/libc.so.6
#1  0x00007fad932e6f62 in do_system () from /lib64/libc.so.6
#2  0x00007fad932e7311 in system () from /lib64/libc.so.6
#3  0x00007fad3708a092 in TUnixSystem::StackTrace() () from /cvmfs/sft.cern.ch/lcg/views/LCG_99/x86_64-centos7-clang10-opt/lib/libCore.so
#4  0x00007fad3750d321 in (anonymous namespace)::TExceptionHandlerImp::HandleException(int) () from /cvmfs/sft.cern.ch/lcg/views/LCG_99/x86_64-centos7-clang10-opt/lib/libcppyy_backend3_8.so
#5  0x00007fad3708d972 in TUnixSystem::DispatchSignals(ESignals) () from /cvmfs/sft.cern.ch/lcg/views/LCG_99/x86_64-centos7-clang10-opt/lib/libCore.so
#6  <signal handler called>
#7  0x00007fad25566d8a in ?? ()
#8  0x000000002774bc60 in ?? ()
#9  0x000000002774bcc0 in ?? ()
#10 0x00007ffe4f0e55b0 in ?? ()
#11 0x00007fad255c946e in ?? ()
#12 0x00007ffe4f0e5580 in ?? ()
#13 0x00007fad255c8d20 in ?? ()
#14 0x00000000276a55a0 in ?? ()
#15 0x00000000276a5540 in ?? ()
#16 0x000000002774bc60 in ?? ()
#17 0x00000000276a55a0 in ?? ()
#18 0x00000000276a5540 in ?? ()
#19 0x000000002774bc60 in ?? ()
#20 0x000000002774bc60 in ?? ()
#21 0x000000002774bcc0 in ?? ()
#22 0x000000002774bcc0 in ?? ()
#23 0x00007fad255cdbb4 in ?? ()
#24 0x00007ffe4f0e5980 in ?? ()
#25 0x00007fad2558a3c7 in ?? ()
#26 0x000000002774bcc0 in ?? ()
#27 0x0000000023a056d0 in ?? ()
#28 0x00007ffe4f0e5980 in ?? ()
#29 0x00007ffe4f0e5970 in ?? ()
#30 0x00007ffe4f0e5640 in ?? ()
#31 0x00007fad255c373c in ?? ()
#32 0x000000002774bcc0 in ?? ()
#33 0x00007ffe4f0e5958 in ?? ()
#34 0x000000002774bcc0 in ?? ()
#35 0x000000002774bcc0 in ?? ()
#36 0x00007ffe4f0e5710 in ?? ()
#37 0x00007ffe4f0e5958 in ?? ()
#38 0x0000000026155840 in ?? ()
#39 0x0000000000000001 in ?? ()
#40 0x01007fad255982c0 in ?? ()
#41 0x00007ffe4f0e5958 in ?? ()
#42 0x00007ffe4f0e5680 in ?? ()
#43 0x00000000237493a0 in ?? ()
#44 0x00007ffe4f0e56a0 in ?? ()
#45 0x0000000000000000 in ?? ()
===========================================================


The lines below might hint at the cause of the crash.
You may get help by asking at the ROOT forum https://root.cern.ch/forum
Only if you are really convinced it is a bug in ROOT then please submit a
report at https://root.cern.ch/bugs Please post the ENTIRE stack trace
from above as an attachment in addition to anything else
that might help us fixing this issue.
===========================================================
#7  0x00007fad25566d8a in ?? ()
#8  0x000000002774bc60 in ?? ()
#9  0x000000002774bcc0 in ?? ()
#10 0x00007ffe4f0e55b0 in ?? ()
#11 0x00007fad255c946e in ?? ()
#12 0x00007ffe4f0e5580 in ?? ()
#13 0x00007fad255c8d20 in ?? ()
#14 0x00000000276a55a0 in ?? ()
#15 0x00000000276a5540 in ?? ()
#16 0x000000002774bc60 in ?? ()
#17 0x00000000276a55a0 in ?? ()
#18 0x00000000276a5540 in ?? ()
#19 0x000000002774bc60 in ?? ()
#20 0x000000002774bc60 in ?? ()
#21 0x000000002774bcc0 in ?? ()
#22 0x000000002774bcc0 in ?? ()
#23 0x00007fad255cdbb4 in ?? ()
#24 0x00007ffe4f0e5980 in ?? ()
#25 0x00007fad2558a3c7 in ?? ()
#26 0x000000002774bcc0 in ?? ()
#27 0x0000000023a056d0 in ?? ()
#28 0x00007ffe4f0e5980 in ?? ()
#29 0x00007ffe4f0e5970 in ?? ()
#30 0x00007ffe4f0e5640 in ?? ()
#31 0x00007fad255c373c in ?? ()
#32 0x000000002774bcc0 in ?? ()
#33 0x00007ffe4f0e5958 in ?? ()
#34 0x000000002774bcc0 in ?? ()
#35 0x000000002774bcc0 in ?? ()
#36 0x00007ffe4f0e5710 in ?? ()
#37 0x00007ffe4f0e5958 in ?? ()
#38 0x0000000026155840 in ?? ()
#39 0x0000000000000001 in ?? ()
#40 0x01007fad255982c0 in ?? ()
#41 0x00007ffe4f0e5958 in ?? ()
#42 0x00007ffe4f0e5680 in ?? ()
#43 0x00000000237493a0 in ?? ()
#44 0x00007ffe4f0e56a0 in ?? ()
#45 0x0000000000000000 in ?? ()
===========================================================


Traceback (most recent call last):
  File "Run.py", line 177, in <module>
    SC = ch.Process(NtupleReader,argumentsStr)
TypeError: none of the 2 overloaded methods succeeded. Full details:
  Long64_t TChain::Process(TSelector* selector, const char* option = "", Long64_t nentries = kMaxEntries, Long64_t firstentry = 0) =>
    SegmentationViolation: segfault in C++; program state was reset
  Long64_t TChain::Process(const char* filename, const char* option = "", Long64_t nentries = kMaxEntries, Long64_t firstentry = 0) =>
    TypeError: could not convert argument 1 (bad argument type for built-in operation)
 *** Break *** segmentation violation
Segmentation fault (core dumped)

The code pretty big, so for the moment I avoid to post it. The strange thing is that works for almost every ntuples and not in few ones.
Someone has ideas?

Platform: lxplus

HI,
Sorry for the late reply. It is difficult to say what the problem is, we would need your full code and run it. However if you have some big code running, it is possible you have some issues in your code that can cause this crash. II would suggest you to debug it and run some tools like valgrind to understand what the problem is. Othwwise try to reduce the code to a smaller reproduceble that cab be better investigated

Lorenzo

Thanks, by reducing the code I found some bugs. The thing that surprises me (it’s never happened to me, I’ve been working with ROOT for a few months) is that with certain ntuples made with the same architecture I have no problem, with others I have a crash. In general, what could this be due to? Maybe too much computational memory that lxplus doesn’t support?

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.