Maximum Memory allocation in a QNX 6.5 process

[maschoen]

I’m working with a client who has QNX 6.6 boxes with 16Gig (same as 6.5 for these purposes) and he wants to be able to allocate the maximum memory in a process. We know how to get QNX to see the memory above 4Gig (-x parameter on start-*) and we know how to start a process that is allowed to use that memory (# on -P program). We know that malloc() has a maximum allocation of 2Gig per call. We seem to be limited to about 2Gig, or maybe a bit more. We’ve found this limit occurs with both malloc() allocations and getrlimit() never shows it being updated. We can’t figure out what’s wrong. Some documenation alludes to 3.5Gig being a limit. It would be great if we could allocate that much.

The shell command ulimit -a shows memory unlimited.

Any ideas out there? This is really troubling.

[maschoen]

This is very strange. I saw a reply on my cell phone to this question but I don’t see the reply here?
Now I can’t find it on my cell. Maybe better to email me at maschoen@gmail.com.

In answer, I don’t have a 6.6 machine so I’m testing on 6.5. We are seeing the same results on both OS’s. There have been two kinds of tests. Using malloc() in a loop a large amount until it fails. Using mmap() in a loop in which I allocate progressively large amounts until it fails. After each allocation I unmap() it.

[maschoen]

Sorry for responding here but a reply to the email I received did not work.

Thanks for getting back to me. This is confusing because I saw the first response briefly on my phone but then it disappeared and it wasn’t in my email. I’m crossing my fingers that just by replying here you will get it.

On Tue, Mar 12, 2024 at 1:14 AM John Kearney <community-noreply@qnx.com> wrote:

The available user address space on a 32bit system on qnx depends on the architecture.
see QNX® Software Development Platform 6.6 <br/> QNX® SDK for Apps and Media 1.1 <br/> QNX® Wireless Framework 1.0

In one case it is a very vanilla x86 16Gig box. That’s at the customer site running 6.6. I’ve been using an 8Gig VM I created using VMware with QNX 6.5. The results we’ve seen have been identical.

so having 2/3.5g available user address space doesn’t mean you are able to do a contiguous virtual address space allocation of that size.

My understanding is that using page tables you can have a continuous virtual address space even if the real memory is not continuous. Even if that is not so, when we run a program, there is nothing above 4Gig so it should be possible to have close to 4Gig allocated.

There is space also used for the stack, heap, executable, libs, etc…

Max stack space (I saw this today) is 1/2 a meg. The executables are relatively tiny. I don’t know about the libs, but It’s hard to imagine that there is close to 1gig being used. There are also read only system areas that are virtualized into each address space, but again I don’t think that is all that much. When using malloc() I think that adds to the heap, but these programs are using almost no data space.

There’s one more obvious issue which may be the whole problem. Why can’t setrlimit() set the max size above 2Gig? It’s not clear to me that this limit is the real limit but I’d like to adjust it to see if it is. One option in setrlimit() is RLIMIT_INFINITY which means unlimited. I’ve tried to set this and it doesn’t work. setrlimit() doesn’t return an error either.

BTW, thank you so much for responding. I feel like I’m out in the cold here. There’s no going through the QNX front door. They seem quite uninterested these days in anything but the current version which now seems to be 8.0.

Mitchell

[Tim]

Mitchell,

I wonder if you are being constrained by the arena caching features introduced in later versions of QNX. Maybe it might be best to turn them off since presumably your customer never plans to release the RAM they allocate.

MALLOC_ARENA_SIZE
The size of the malloc() arena. It also controls the amount by which the arena grows.

By default, this is set to 32 KB, which results in a minimum allocation of 64 KB by the allocator. You can lower this as long as it's a multiple of the page size (4096). The allocator will still attempt to get memory from the system in multiples of this value, and since it creates buckets of varying sizes, it may choose to allocate core memory for a bucket of blocks of a new size, even though an earlier allocation of core memory for a block of a different size hasn't been fully exhausted.

Memory in the allocator is allocated with different policies for blocks smaller than 128 bytes and blocks larger than 128 bytes:

    Smaller blocks are allocated using a fixed chunk allocator, which gets system memory in bunches of pieces of the size specified by MALLOC_ARENA_SIZE and creates smaller buckets from them.
    Larger memory blocks are created by also getting core memory from the system in multiples of MALLOC_ARENA_SIZE and carving out the appropriate user pieces from this, putting the remaining space onto a free list.

So, for example, if you make a simple pair of allocations such as:

malloc(100);
malloc(200);
  

the system creates a 32 KB arena allocation for the bucket size of 128 bytes to service the 100-byte allocation, and a 32 KB arena allocation to service the 200-byte allocation.

Lowering MALLOC_ARENA_SIZE can reduce the amount of memory required. 

https://www.qnx.com/developers/docs/6.4.1/neutrino/lib_ref/m/malloc.html
https://www.qnx.com/developers/docs/7.1/#com.qnx.doc.neutrino.prog/topic/hat_DynamicMemoryManagement.html

Tim