memory fragmentation?

QNX Newsgroups (Archives) qdn.public.qnxrtp.os
#1

I’m looking for some information about memory fragmentation in RTP.
lets say we have a 1000 yyte heap
and I am doing three allocs on 250 bytes, => b1, b2, b3
so the heap will look something like this
|–b1–|–b2–|–b3–|------|
if I’m now freeing the block b2 the heap will look like this
|–b1–|------|–b3–|------|
now I want to alloc a block of 500 bytes, what will the system do?
will I get an error?
or is the system doing a reordering of the blocks, like
|–b1–|–b3–|-------------|

Any hints?
TIA
hp

#2

“HP Reichert” <hp.reichert@idea.REMOVE.soft.de> wrote in message
news:a4vd8k$sra$1@inn.qnx.com…

I’m looking for some information about memory fragmentation in RTP.
lets say we have a 1000 yyte heap
and I am doing three allocs on 250 bytes, => b1, b2, b3
so the heap will look something like this
|–b1–|–b2–|–b3–|------|
if I’m now freeing the block b2 the heap will look like this
|–b1–|------|–b3–|------|
now I want to alloc a block of 500 bytes, what will the system do?
will I get an error?
or is the system doing a reordering of the blocks, like
|–b1–|–b3–|-------------|

From the user point of view things will look like:
|–b1–|------|–b3–|–b4—|

If the heap isn’t big enough, it will be grown. Hence you will
have “lost” 250 bytes.

Now how the heap is allocated in physical memory is a different story,
because of virtual memory, the heap could be spread all over.

My understanding is that asking the OS to allocate memory is expensive,
that’s why operation are mostly handle within the program environment (heap)
and only when the heap needs to be grow are the OS services involved.

I haven’t really look at this up close, but that’s what my observations tell
me
is happening.


Hope this helps.


Any hints?



TIA
hp

#3

I remember that there was a fairly big performance issue with python when
you create large numbers of list entries because of the way python’s
allocator works. On Linux, when a program asks for more memory, the OS
keeps giving it blocks that are double the size of the previous so there
isn’t a big performance benefit to managing your own heap. QNX is much more
conservative and only gives what is asked for. This causes a certain amount
of thrashing in the allocator if you’re asking for lots of small blocks. I
believe that the network guys here have their own allocator that grabs big
chunks at a time and then splits it off internally.

Cheers,

Kris

“Mario Charest” <goto@nothingness.com> wrote in message
news:a505lg$gue$1@inn.qnx.com…

“HP Reichert” <> hp.reichert@idea.REMOVE.soft.de> > wrote in message
news:a4vd8k$sra$> 1@inn.qnx.com> …
I’m looking for some information about memory fragmentation in RTP.
lets say we have a 1000 yyte heap
and I am doing three allocs on 250 bytes, => b1, b2, b3
so the heap will look something like this
|–b1–|–b2–|–b3–|------|
if I’m now freeing the block b2 the heap will look like this
|–b1–|------|–b3–|------|
now I want to alloc a block of 500 bytes, what will the system do?
will I get an error?
or is the system doing a reordering of the blocks, like
|–b1–|–b3–|-------------|

From the user point of view things will look like:
|–b1–|------|–b3–|–b4—|

If the heap isn’t big enough, it will be grown. Hence you will
have “lost” 250 bytes.

Now how the heap is allocated in physical memory is a different story,
because of virtual memory, the heap could be spread all over.

My understanding is that asking the OS to allocate memory is expensive,
that’s why operation are mostly handle within the program environment
(heap)
and only when the heap needs to be grow are the OS services involved.

I haven’t really look at this up close, but that’s what my observations
tell
me
is happening.


Hope this helps.



Any hints?


TIA
hp
\

#4

Note that the source code is available from cvs.qnx.com! :v)

Kris Warkentin <kewarken@qnx.com> wrote:

I remember that there was a fairly big performance issue with python when
you create large numbers of list entries because of the way python’s
allocator works. On Linux, when a program asks for more memory, the OS
keeps giving it blocks that are double the size of the previous so there
isn’t a big performance benefit to managing your own heap. QNX is much more
conservative and only gives what is asked for. This causes a certain amount
of thrashing in the allocator if you’re asking for lots of small blocks. I
believe that the network guys here have their own allocator that grabs big
chunks at a time and then splits it off internally.

Cheers,

Kris

“Mario Charest” <> goto@nothingness.com> > wrote in message
news:a505lg$gue$> 1@inn.qnx.com> …

“HP Reichert” <> hp.reichert@idea.REMOVE.soft.de> > wrote in message
news:a4vd8k$sra$> 1@inn.qnx.com> …
I’m looking for some information about memory fragmentation in RTP.
lets say we have a 1000 yyte heap
and I am doing three allocs on 250 bytes, => b1, b2, b3
so the heap will look something like this
|–b1–|–b2–|–b3–|------|
if I’m now freeing the block b2 the heap will look like this
|–b1–|------|–b3–|------|
now I want to alloc a block of 500 bytes, what will the system do?
will I get an error?
or is the system doing a reordering of the blocks, like
|–b1–|–b3–|-------------|

From the user point of view things will look like:
|–b1–|------|–b3–|–b4—|

If the heap isn’t big enough, it will be grown. Hence you will
have “lost” 250 bytes.

Now how the heap is allocated in physical memory is a different story,
because of virtual memory, the heap could be spread all over.

My understanding is that asking the OS to allocate memory is expensive,
that’s why operation are mostly handle within the program environment
(heap)
and only when the heap needs to be grow are the OS services involved.

I haven’t really look at this up close, but that’s what my observations
tell
me
is happening.


Hope this helps.



Any hints?


TIA
hp

\

–

cburgess@qnx.com

#5

“Colin Burgess” <cburgess@qnx.com> wrote in message
news:a5141d$db2$2@nntp.qnx.com…

Note that the source code is available from cvs.qnx.com! :v)

You’ll find another terrific resource in the programmers guide, chapter:
Heap Analysis - Making Memory Errors a Thing of the Past
Here’s a snip from the top of the online doc…

Introduction
Dynamic Memory Management
Heap Corruption
Detecting and Reporting Errors
Manual Checking (Bounds Checking)
Memory Leaks
Compiler Support
Summary
Introduction

If you develop a program that dynamically allocates memory, you’re also
responsible for tracking any memory that you allocate whenever a task is
performed, and for releasing that memory when it’s no longer required. If
you fail to track the memory correctly you may introduce “memory leaks,” or
unintentionally write to an area outside of the memory space.

Conventional debugging techniques usually prove to be ineffective for
locating the source of corruption or leak because memory-related errors
typically manifest themselves in an unrelated part of the program. Tracking
down an error in a multithreaded environment becomes even more complicated
because the threads all share the same memory address space.

In this chapter, we’ll introduce you to a special version of our memory
management functions that’ll help you to diagnose your memory management
problems.