1 - Can a process free memory back to the system?
2 - If so, does the allocation order matter?
3 - If I new a 1M block and after that a 100K block, if I free the 1M block
I do not seem to get it back. When I free the 100K block, I get all of it
back (i.e. 1M + 100K). If I only free the 1M block, can my process reuse it,
even though it is not freed to the system?
Is there a document/site where I can get a clear explanation/tips of how
memory allocation works under 6.0/6.1
Thanks.
Jamal Benbrahim <jamal.benbrahim@igt.com> wrote:
1 - Can a process free memory back to the system?
2 - If so, does the allocation order matter?
3 - If I new a 1M block and after that a 100K block, if I free the 1M block
I do not seem to get it back. When I free the 100K block, I get all of it
back (i.e. 1M + 100K). If I only free the 1M block, can my process reuse it,
even though it is not freed to the system?
This is really a question that you should post to comp.lang.c++ . That said,
I’ll try to answer you question(s).
)-Adam
Jamal Benbrahim <jamal.benbrahim@igt.com> wrote:
1 - Can a process free memory back to the system?
Yes, but it depends on the encoding for the allocator.
2 - If so, does the allocation order matter?
It depends on how the allocator is written, with the current QNX allocator
order does matter.
3 - If I new a 1M block and after that a 100K block, if I free the 1M block
I do not seem to get it back. When I free the 100K block, I get all of it
back (i.e. 1M + 100K). If I only free the 1M block, can my process reuse it,
even though it is not freed to the system?
The current QNX allocator will only return to the OS free list blocks at
the end of the processes free list – but not internal blocks in the list.
Internal blocks in the list that are marked as free will be re-used within
the process itself.
QNX Training Services
dagibbs@qnx.com
For what I know, the malloc() is essentially a cover for mmap(MAP_ANON).
When you request some amount of memory malloc() checks if there’s a
sufficient block in the process pool. If there is, it just puts that
block into used list and returns its address. If not, the mmap() will
ask system for a number of bytes not less than _amblksiz and rounded to
pagesize boundary. The newly mmap()-ed memory goes into process pool and
then used for allocations. When you free() a block it is moved into free
list and allocator tries to coalesce blocks on the free list into larger
blocks. I don’t know exactly under what circumstances allocator returns
memory back to system but apparently it does sometimes, probably when a
whole mmap()-ed chunk is not used anymore.
So, to answer your question yes your process can reuse free()-d memory
even if it is not returned to the system. If you want more precise
control over memory usage you need to allocate memory in large chunks,
using mmap() directly. Then you manage your own pools inside those
chunks, mmap()-ing new chunks when you need to and munmap()-ing them
when they are not needed anymore. The munmap() always returns memory
back to system. It might be that standard allocator does the same thing,
but you can only guess when and what happens.
Jamal Benbrahim wrote:
1 - Can a process free memory back to the system?
2 - If so, does the allocation order matter?
3 - If I new a 1M block and after that a 100K block, if I free the 1M block
I do not seem to get it back. When I free the 100K block, I get all of it
back (i.e. 1M + 100K). If I only free the 1M block, can my process reuse it,
even though it is not freed to the system?Is there a document/site where I can get a clear explanation/tips of how
memory allocation works under 6.0/6.1Thanks.
Igor Kovalenko <Igor.Kovalenko@motorola.com> wrote:
: For what I know, the malloc() is essentially a cover for mmap(MAP_ANON).
I’ll say a few words here.
Most malloc() routines will use mmap() or sbrk()/brk() to get memory from
the system, from then different algorithms will be use to manage
the memory. General purpose allocator are usually tradeoffs
between space-efficiency and space-inefficiency. It will vary
from OS to OS.
: When you request some amount of memory malloc() checks if there’s a
: sufficient block in the process pool. If there is, it just puts that
: block into used list and returns its address. If not, the mmap() will
: ask system for a number of bytes not less than _amblksiz and rounded to
: pagesize boundary. The newly mmap()-ed memory goes into process pool and
: then used for allocations. When you free() a block it is moved into free
: list and allocator tries to coalesce blocks on the free list into larger
: blocks. I don’t know exactly under what circumstances allocator returns
: memory back to system but apparently it does sometimes, probably when a
: whole mmap()-ed chunk is not used anymore.
Yes.
But trying to second guest an allocator is non-portable and simply
wrong. Some allocator will try to unmmap() back pagesize to the
OS on certain circumstances, in many case it is not worth the effort.
QNX uses two different algorithms depending on the size of the allocation.
If it is small, it will fall into the “band allocator”, the bands are
power of two free lists i.e. a request of 10 will be honoured by the
band/arena 16 etc …
This provide some overhead but reduce fragmentation for small buffers.
: So, to answer your question yes your process can reuse free()-d memory
: even if it is not returned to the system. If you want more precise
: control over memory usage you need to allocate memory in large chunks,
: using mmap() directly. Then you manage your own pools inside those
: chunks, mmap()-ing new chunks when you need to and munmap()-ing them
: when they are not needed anymore. The munmap() always returns memory
: back to system. It might be that standard allocator does the same thing,
: but you can only guess when and what happens.
Agreed 100 %. If one need this level of granularity i.e. to know when
memory is return back to the system, rolling your own cover up to mmap()
is the only way to go. I do not know of any user-level C library malloc
that allows such flexiblity, unless one have intimate knowledge of the
allocator.
p.s.: This may look obvious, but sometime people ask me this.
Yes memory(and all resources) is return back to the system when the
process exit.
–
alain
Alain Magloire <alain@qnx.com> wrote:
p.s.: This may look obvious, but sometime people ask me this.
Yes memory(and all resources) is return back to the system when the
process exit.
Actually, that isn’t quite true. Resources that are asssociated with
a name may (will) endure past process death. For example, if you do
a shm_open(…, O_CREAT) then ftruncate() to allocate a shared memory area,
that memory will not be released until the name that is passed to shm_open()
has been unlinked. Similarly for open(), sem_open(), mqueue_open(), etc.
The rules for when something are released are, generally:
If associated with a name, it will be released after the name has been
unlinked and all references to the object have been closed/released.
(e.g. for shared memory when all fds to the object are closed and all
mappings have been unmapped.)
An object that is associated with memory, essentially most if not all of
the synchronisation objects such as unnammed semaphores, mutexes, condition
variables, etc will be released when the memory they are in is released.
Other resources, including normal process memory, fds, mappings, irq handlers,
pids, chids, coids, etc (generally anything that has a numeric identifier)
will be released/freed when a process exits. Since memory is released,
this also means that in most cases semaphores, mutexes, etc get automatically
released when a process exits, since they are usually in process local
memory.
QNX Training Services
dagibbs@qnx.com