cause of crash?

our program is crashing and we have the core file (no debug version, -g is
not used). How can I determine the cause of the crash? Is working with a
core file documented somewhere? Is there a way to determine if something
killed it, got a memory fault, etc?

Joseph Witherspoon <jspoon@us.ibm.com> wrote:
JW > our program is crashing and we have the core file (no debug version, -g is
JW > not used). How can I determine the cause of the crash? Is working with a
JW > core file documented somewhere? Is there a way to determine if something
JW > killed it, got a memory fault, etc?

Try running coreinfo on the core file. It will tell you much including
the IP when the process died.

If the program generated a link map when it was linked you can look up
that address in the link map. Otherwise, if the source code hasn’t
changed just re-compile and re-link it.

Bear in mind that in the link map, the entry points with each modual are
sorted alphabeticly by symbol name, NOT by address.

“That will teach you to put so many things in a single source modual!”

Bill Caroselli wrote:

Try running coreinfo on the core file. It will tell you much including
the IP when the process died.

If the program generated a link map when it was linked you can look up
that address in the link map. Otherwise, if the source code hasn’t
changed just re-compile and re-link it.

Bear in mind that in the link map, the entry points with each modual are
sorted alphabeticly by symbol name, NOT by address.

“That will teach you to put so many things in a single source modual!”

I added the -M to the LD options and got a .map file for each object (each
o, each .so, each executable). When I do the coreinfo, it shows our 8
threads and thread three had a SIGSEGV:-) How do I tie that back to the
map files to determine where the crash occured… There are 12 shared
libs and an executable. Any chance of discussing this off-line?

Joseph Witherspoon <jspoon@us.ibm.com> wrote:
JW > Bill Caroselli wrote:

Try running coreinfo on the core file. It will tell you much including
the IP when the process died.

If the program generated a link map when it was linked you can look up
that address in the link map. Otherwise, if the source code hasn’t
changed just re-compile and re-link it.

Bear in mind that in the link map, the entry points with each modual are
sorted alphabeticly by symbol name, NOT by address.

“That will teach you to put so many things in a single source modual!”

JW > I added the -M to the LD options and got a .map file for each object (each
JW > o, each .so, each executable). When I do the coreinfo, it shows our 8
JW > threads and thread three had a SIGSEGV:-) How do I tie that back to the
JW > map files to determine where the crash occured… There are 12 shared
JW > libs and an executable. Any chance of discussing this off-line?

Here for example is a core info from a program that I am working on.

/home/bill>coreinfo /var/dumps/HSC.core
/var/dumps/HSC.core:
processor=X86 num_cpus=1
cpu 1 cpu=1586 name=Intel ?86 F15M2S7 speed=2389
flags=0xc0003fff FPU MMU CPUID RDTSC INVLPG WP BSWAP MMX CMOV PSE PGE MTRR SEP SIMD FXSR
cyc/sec=2386720000 tod_adj=1094153271000000000 nsec=157112317137073 inc=999847
boot=1094153271 epoch=1970 intr=0
rate=838095345 scale=-15 load=1193
HOSTNAME=“dell” MACHINE=“x86pc”
pid=31670309 parent=675881 child=0 pgrp=31670309 sid=675881
flags=0x002000 umask=02 base_addr=0x8048000 init_stack=0x80477fc
ruid=100 euid=100 suid=100 rgid=100 egid=100 sgid=100
ign=0000000000000000 queue=ff00000000000000 pending=0000000000000000
fds=15 threads=2 timers=0 chans=4
thread 1 SIGNALLED-SIGSEGV code=1 MAPERR refaddr=32202c48 fltno=11
ip=0x805bf98 sp=0x8046ed0 stkbase=0x7fc7000 stksize=528384
state=STOPPED flags=0 last_cpu=1 timeout=00000000
pri=10 realpri=10 policy=RR
thread 2
ip=0xb032d681 sp=0x7fc6d20 stkbase=0x7fc2000 stksize=20480
state=SIGWAITINFO flags=4000000 last_cpu=1 timeout=0x1000800
pri=20 realpri=20 policy=FIFO

It shows that thread #1 had SIGSEGV’d. Under that it shows that the IP
for that thread was 0x805bf98.

Here is a short piece of my linkmap:
.text 0x0805bef4 0x112c /tmp/obj/nto-x86/Q-lib.a(tracer.o)
0x0805cb96 Tracer::SetShowCalls()
0x0805ceb4 Tracer::Spaces()
0x0805cca6 Tracer::ShowCalls()
0x0805bef4 tracer_thread_attr::tracer_thread_attr(int)
0x0805c6de Tracer::~Tracer()
0x0805cb0e Tracer::ClearLogSwitch()
0x0805bf40 tracer_thread_attr::tracer_thread_attr(int)
0x0805c334 Tracer::~Tracer()
0x0805cc1e Tracer::ClearShowCalls()
0x0805bf8c tracer_thread_attr::FindTracer()
0x0805c044 Tracer::Init(char const*, bool, bool)
0x0805ca88 Tracer::SetLogSwitch(void*)

This shows that my htread died somewhere in a library routine I have
called /tmp/obj/nto-x86/Q-lib.a(tracer.o). These are the entry points in
that modual. They are unfortunately sorted alphabetically for that modual.
However a little manual sorting shows that my program died 12 bytes
into tracer_thread_attr::FindTracer() (805bf98 - 805bf8c).

I could have disassenbled that source modual but looking at the source I
immediately saw what my bug was.

Good luck

Load it into the debugger.

ntosh-gdb <executable.core>

You can also do it with the IDE

Then select thread 3 and you will see the point it crashed at.

Joseph Witherspoon wrote:

Bill Caroselli wrote:


Try running coreinfo on the core file. It will tell you much including
the IP when the process died.


If the program generated a link map when it was linked you can look up
that address in the link map. Otherwise, if the source code hasn’t
changed just re-compile and re-link it.


Bear in mind that in the link map, the entry points with each modual are
sorted alphabeticly by symbol name, NOT by address.


“That will teach you to put so many things in a single source modual!”


I added the -M to the LD options and got a .map file for each object (each
o, each .so, each executable). When I do the coreinfo, it shows our 8
threads and thread three had a SIGSEGV:-) How do I tie that back to the
map files to determine where the crash occured… There are 12 shared
libs and an executable. Any chance of discussing this off-line?
\


cburgess@qnx.com