Heap usage stats in QNX6

Hi all,

I have a program which looks just great and seems to pass all
functionality tests. Among the things I need to do to qualify it for
shipping is to run a long term heap analysis on it.

That is, I want to start the program running with my automated testing
harness and watch the heap usage over the next few
hours/days/weeks/months to make sure there are no subtle memory leaks in
the program.

I thought that pidin might report heap usage somewhere, but I can’t find
the right information anywhere. I know pidin will give me code and data
usage for each library. I know that pidin will give me stack usage for
each thread. I do not see where it will give me heap usage for a given
process.

Can someone suggest a way to get heap usage stats for a process?

Looking ahead, let’s say I determine that there is a memory leak in the
program. What tools/APIs would I use to instrument my code to look for
those leaks?

Thanks in advance,
Eric

malloc() use mmap(MAP_ANON) which is mmap(fd=open("/dev/zero")).
The /dev/zero mappings can be read directly from procfs, but pidin includes
them into data size.

I’m working on a tool to show aggregate statistics in a readable manner.
Something like extended ‘top’ utility. Almost all pieces are relatively easy
to get, but they are organized as a damed puzzle, it’s real pain in the neck
to see the big picture…

  • igor

“Eric Berdahl” <berdahl@intelligentparadigm.com> wrote in message
news:berdahl-32FAA6.16453305022001@nntp.qnx.com

Hi all,

I have a program which looks just great and seems to pass all
functionality tests. Among the things I need to do to qualify it for
shipping is to run a long term heap analysis on it.

That is, I want to start the program running with my automated testing
harness and watch the heap usage over the next few
hours/days/weeks/months to make sure there are no subtle memory leaks in
the program.

I thought that pidin might report heap usage somewhere, but I can’t find
the right information anywhere. I know pidin will give me code and data
usage for each library. I know that pidin will give me stack usage for
each thread. I do not see where it will give me heap usage for a given
process.

Can someone suggest a way to get heap usage stats for a process?

Looking ahead, let’s say I determine that there is a memory leak in the
program. What tools/APIs would I use to instrument my code to look for
those leaks?

Thanks in advance,
Eric

In article <berdahl-32FAA6.16453305022001@nntp.qnx.com>,
Eric Berdahl <berdahl@intelligentparadigm.com> wrote:

Hi all,

I have a program which looks just great and seems to pass all
functionality tests. Among the things I need to do to qualify it for
shipping is to run a long term heap analysis on it.

That is, I want to start the program running with my automated testing
harness and watch the heap usage over the next few
hours/days/weeks/months to make sure there are no subtle memory leaks in
the program.

What you describe is one one of looking. We’ll deal with that one first
– there is another.

I thought that pidin might report heap usage somewhere, but I can’t find
the right information anywhere. I know pidin will give me code and data
usage for each library. I know that pidin will give me stack usage for
each thread. I do not see where it will give me heap usage for a given
process.

As Igor says in his reply the mapped chunks are reported by pidin in the
data size for the program. Some of it is the static initialized (.data) and
static uninitialized data (.bss) for the program, so you would have to
factor out that, the corresponding data for all dynamic libraries and any
explicit mmap() requests.

There are also circumstances (reserved mmap() virtual address mappings
with no physical pages committed) that can cause this to be unreliable.
Okay, so it’s rare, but some programs do it.

Can someone suggest a way to get heap usage stats for a process?

You could factor out the data from pidin, or you could construct /proc
requests to find out the page mappings directly and analyze them.

However, if you are willing to modify the program somewhat, the malloc
library has some undocumented profiling features.
The _malloc_stats variable has low-level information about the
amount of data in use (blocks and bands), and the total heap as well
as user heap (the amount actually requested by the program). It is
of type “struct malloc_stats”, defined in malloc.h.

Basically, m_freemem tells the amount of available space in the free list,
while m_allocmem tells the amount of space taken up for big blocks in use.
Likewise, m_small_freemem tells the amount of space available in bands
(buckets for small allocations, ranging in size from 8-64 bytes), while
m_small_allocmem tells the amount of space in use by the program for
small blocks. In contrast m_heapsize gives the full size of the heap,
including internal fragmentation.

The other members give profiling information about the number of times
various operations have been performed.

Looking ahead, let’s say I determine that there is a memory leak in the
program. What tools/APIs would I use to instrument my code to look for
those leaks?

Read the tech note on heap analysis. It describes how the malloc_g
library can be used to detect heap data that is no longer referred
to by the program. To use it on a running program, you would
need some way to trigger a call to malloc_dump_unreferenced.

Thanks in advance,
Eric

Steve Furr email: furr@qnx.com
QNX Software Systems, Ltd.

In article <95ofrg$mj1$1@inn.qnx.com>, “Igor Kovalenko”
<kovalenko@home.com> wrote:

malloc() use mmap(MAP_ANON) which is mmap(fd=open("/dev/zero")).
The /dev/zero mappings can be read directly from procfs, but pidin
includes
them into data size.

pidin includes the /dev/zero mappings into the data size? Interesting.

Ok, so if I do a pidin mem, I get something like this:

pid tid name prio STATE code data stack
1155099 1 ./Foo 10o READY 16K 24K 8192(516K)*
1155099 2 ./Foo 10o RECEIVE 16K 24K 4096(132K)
ldqnx.so.1 @b0300000 300K 12K
libstdc++.so.2.10 @b034e000 248K 60K
libm.so.1 @b039b000 88K 4096

…and if I do a pidin pmem, I get something like this:

pid tid name prio STATE code data stack
1155099 1 ./Foo 10o READY 652K 100K 8192(516K)*
1155099 2 ./Foo 10o NANOSLEEP 652K 100K 4096(132K)

From the mem data set, I know which shared libraries my app uses and the
sizes of the code and data segments for those libraries. Yes?

From the pmem data set, do I then know the aggregate code size and the
aggregate data size?

From this data, how do I determine how big the malloc heap is? If I can
figure out how big the malloc heap is, and if I observe that number over
a period of time, and if that number does not increase, I can determine
that a leak is not likely.

To more properly demonstrate a lack of leaks requires something more
like Steve Furr’s suggestion.

I’m working on a tool to show aggregate statistics in a readable manner.
Something like extended ‘top’ utility. Almost all pieces are relatively
easy
to get, but they are organized as a damed puzzle, it’s real pain in the
neck
to see the big picture…

Sounds like a very interesting tool. I’d love to see it.

Thanks,
Eric

Eric Berdahl wrote:

In article <95ofrg$mj1$> 1@inn.qnx.com> >, “Igor Kovalenko”
kovalenko@home.com> > wrote:

malloc() use mmap(MAP_ANON) which is mmap(fd=open("/dev/zero")).
The /dev/zero mappings can be read directly from procfs, but pidin
includes
them into data size.

pidin includes the /dev/zero mappings into the data size? Interesting.

Ok, so if I do a pidin mem, I get something like this:

pid tid name prio STATE code data stack
1155099 1 ./Foo 10o READY 16K 24K 8192(516K)*
1155099 2 ./Foo 10o RECEIVE 16K 24K 4096(132K)
ldqnx.so.1 @b0300000 300K 12K
libstdc++.so.2.10 @b034e000 248K 60K
libm.so.1 @b039b000 88K 4096

…and if I do a pidin pmem, I get something like this:

pid tid name prio STATE code data stack
1155099 1 ./Foo 10o READY 652K 100K 8192(516K)*
1155099 2 ./Foo 10o NANOSLEEP 652K 100K 4096(132K)

From the mem data set, I know which shared libraries my app uses and the
sizes of the code and data segments for those libraries. Yes?

From the pmem data set, do I then know the aggregate code size and the
aggregate data size?

Yes, but it is somewhat misleading because shared libraries may be
indeed shared. In which case every process will report the same memory
as belonging to it. In fact shared libs should be counted separately.

From this data, how do I determine how big the malloc heap is? If I can
figure out how big the malloc heap is, and if I observe that number over
a period of time, and if that number does not increase, I can determine
that a leak is not likely.

Static data segments are usually rather small. Most of the memory
reported in ‘data’ is heap.

To more properly demonstrate a lack of leaks requires something more
like Steve Furr’s suggestion.

I’m working on a tool to show aggregate statistics in a readable manner.
Something like extended ‘top’ utility. Almost all pieces are relatively
easy
to get, but they are organized as a damed puzzle, it’s real pain in the
neck
to see the big picture…

Sounds like a very interesting tool. I’d love to see it.

Hehe, me too, but I figured it’s not gonna appear by itself :wink:
Stay tuned, it’s about 80% complete.

  • igor

In article <3A804AD1.8C07740E@motorola.com>, Igor Kovalenko
<Igor.Kovalenko@motorola.com> wrote:

Eric Berdahl wrote:
From the mem data set, I know which shared libraries my app uses and
the
sizes of the code and data segments for those libraries. Yes?

From the pmem data set, do I then know the aggregate code size and the
aggregate data size?


Yes, but it is somewhat misleading because shared libraries may be
indeed shared. In which case every process will report the same memory
as belonging to it. In fact shared libs should be counted separately.

I’m hep.

For the experiment I’m running, that’s not as big a concern. My
experiment is only meant to allow me to watch the process running
continuously over a bazillion hours and see that the heap space isn’t
slowly growing, suggesting a memory leak.

If the experiment were, instead, to calculate how my RAM was being used
by the various processes (perhaps to estimate RAM requirements for the
shipping device), then I’d have to be more clever to deal with the
shared situations.

From this data, how do I determine how big the malloc heap is? If I can
figure out how big the malloc heap is, and if I observe that number
over
a period of time, and if that number does not increase, I can determine
that a leak is not likely.


Static data segments are usually rather small. Most of the memory
reported in ‘data’ is heap.

Boffo. The fact that heap space is recorded in “data” was the piece I
needed to confirm. Thanks.

Eric