I have a nice new Server board with 8 cores and 4GB of memory installed and I can not for the life of me work out how to get QNX to recognize the whole 4Gb.
I’ve spent the best part of a week trawling documentation, but I can not find any further reference to this extended memory support. I presume there is some magic (obscure setting) that needs to be enabled to allow QNX to access the entire 4G.
Does any one know how to enable the extended memory support for x86 or where I can find some more detailed information on extended memory in QNX?
How much is being recognized? How do you know this? I have a machine with 4G but I’ve never checked. Let me know and I will check mine and report back.
Does Windows recognize it? This seems to be a very common issue and there are dozens of google hits on it. Here is one off the Asus website:
Question
I have install total 4GB memory on my motherboard. However, it can only recognize around 3.1GB or less. My friend’s SLI board even less than 3GB. What’s up? How should I do to recognize 4GB totally?
Answer
If you installed total 4GB memory, the system will detect less than 4GB of total memory because of address space allocation for other critical functions, such as:
System BIOS (including motherboard, add-on cards, etc…)
Motherboards resources
Memory mapped I/O
configuration for AGP/PCI-Ex/PCI
Other memory allocations for PCI devices
Different onboard devices and different add-on cards (devices) will result of different total memory size.
e.g. more PCI cards installed will require more memory resources, resulting of less memory free for other uses.
On a SLI system, since PCI-Ex graphic cards will occupy around 256MB, another 256MB will be occupied after you install a 2nd PCI-Ex graphic card. Hence, 2.75GB memory left only if two SLI cards installed on A8N-SLI Premium while 3.0GB memory left with one graphic card without other add-on devices.
This limitation applies to most chipsets & Windows XP 32-bit version operating system.
If you install Windows XP 32-bit version operating system, we recommend that you install less than 3GB of total memory. If more than 3GB memory is required for your system, then below two conditions must be met:
The memory controller which supports memory swap functionality is used. The latest chipsets like Intel 975X, 955X, Nvidia NF4 SLI Intel Edition, Nvidia NF4 SLI X16, and AMD K8 CPU architecture can support the memory swap function.
Windows XP Pro X64 Ed. (64-bit) or other OS which can address more than 4GB memory.
This is really enlightening. We’re up to a 4GB address space, and we’re back to bank switching. I saw this twice before. With 8080’s running MPM swaping out 48K of user space. 16K had to remain for the OS. You might want to read that last sentence again if you eye’s didn’t pop out. Later, 8088/8086 machines running DOS (and of course QNX) ran into the 16Meg barrier, and 64K pages were paged in and out.
Yes, it’s frustrating. I’ve got a Dell D820 notebook with 4GB and can only use 3GB of it due to hardware limitations. I can’t say that has been a bottleneck - yet. But it would be nice to use all the resources that you pay for. From a manufacturers standpoint it’s a little disingenuous to sell 4GB systems knowing full well it can’t all be used.
Only 3.5GB is being recognized, this amount comes about from the ability to shift the PCI & IO addresses to the high address range above the 3.5GB mark in the bios. If this bios setting is lower so is the available memory.
As for determining how much memory is available. I have been using the system information perspective from momentous to determine how much memory QNX thinks it has.
Since my original post I have discovered the “-xâ€
QNX does support up to 64GB of physical memory on x86 (and PPC and MIPS) processor versions that implement it.
It seems that your particular processor does not implement extended addressing. The issue you are seeing is a known bug, but the bug is not that QNX should see the memory above 4GB (it can’t) only that the PCI server should not crash when you specify “-x” on a machine that doesn’t actually support extended addressing.
To summarize:
your hardware does not support extended addressing (not all x86 do)
we have a bug where if you force extended addressing on, the memory configuration for PCI is not setup properly and the server crashes.
Obviously, while it is not pleasant that QNX crashes in case 2, it is also not that serious, because if this bug didn’t exist, you still wouldn’t get your extended memory anyway (due to hardware limitations).
If you want to confirm this, then I believe if you supply enough "-v"s to startup-bios, then it should print out the processor flags which would show whether the CPU supports extended addressing or not…
Note that for me it’s not working. Although the 4Gig of ram is fully detected the machine is unstable and will crash with an hour or so. I’m waiting for the next Trinity build to give it another go.
That is a possibility. But I think one of the role of the “patched” pci-bios is to make sure the hardware all lives below 4G. I also think the OS will not allocated DMA’able memory beyond the 4G barrier.
Mario the OS can allocate contiguous memory above 4GB (I think you’d be very upset if it couldn’t .
The pci server may insure that the PCI space is below 4GB, but that is unrelated to where a DMA gets sent. If you mmap(ANON|CONTIG) and you happen to get memory above 4GB, then you program it into a 32bit DMA register, bad things are going to happen… Of course, things will work great until you get the first buffer above 4GB (and possibly for a while after that).
I can check if the drivers I use have special option to prevent usage of memory above 4G, but I think not.
I think it’s up to the OS/drivers to deal with that not to the user. If the driver sees that the hardware doesn’t have 32bit DMA register it should make sure the memory is allocated below 4G.
That being said I beleive all DMA’able memory should be allocated below 4G unless asked, for otherwise it could break all existing driver that are out there.
The driver should be able to tell automatically by IDing the chip and knowing its capabilities.
I disagree that the OS should have anything to do with this. The drivers however, should know what the hardware it controls is capable of and select from a pool of compatible memory.
How does the OS know what is DMA’able and what isn’t?
I think memory partitioning will allow you to associate a partition with a posix typed memory name and then you can run legacy drivers in that partition. This will work, but this is sub-optimal since the .text, stack and plain heap will also be located in that partition (thereby consuming valuable sub 32bit address space unnecessarily).
All new drivers should use Posix typed memory to insure that they allocate from a pool that is guaranteed to have physical address requirements that matches the hardware they are driving.
If a driver is talking to hardware with a 32bit DMA register, then it allocates from a special pool, otherwise it can alloc from “system”.
There should probably be a typed pool that can be configured from the startup command line called something like “phys32”. The default should allocate something like 256MB to this, and a legacy memory partition should be created by diskboot and all drivers placed in it (for desktop systems).
I agree but since I’m only using QNX drivers then that does appear to be the case.
For backward compatibility I don’t see how the OS cannot get involved.
Well prior to the yet unavailable 6.4 it is not possible as memory partitionning isn’t there. But QNX has been claiming support for more then 4Gig of ram since 6.3 I believe… Basically what you are telling me is that 4G capability is broken ?
It’s not 100% but usage of MAP_PHYS and MAP_ANON is a pretty good guess.
If memory partitionning, when available, is like Adaptive Partioning, it’s going to be unrealistically expensive, money wise, to be use to solve the 4G problem.
Ok what about all the existing drivers, you surely cannot expect to break compatibility with all the drivers that are using DMA.
No. The kernel supports 4GB just fine. You are talking about driver support. I don’t believe that we have made any claims about >4GB physical memory and legacy drivers.
Many applications (particularly applications that allocate a lot of memory - i.e. exactly the type that need to be allocated above 4GB ) need to allocate physically contiguous memory. Automatically stuffing these below 4GB would be completely useless (you’d run out of memory with plenty of physical RAM available).
I am not sure I understand your point here…
Thats what memory partitioning handles… Not sure what you don’t like about that.
Now if none of the QNX drivers that use DMA (IDE, SCSI, Network) can be use it rather pointless don’t you think and would surely deserve a big WARNING that is no where to be found.
Application cannot map more then 3.5 Gig of ram anyway so there is still a limit on how much ram you can access per process. You probably know more about what QNX customers need then I do but from my experience applications that needs lots of ram don’t care if it’s contiguous or not. I can see that it may be needed for vision application where lots of data is grabbed but these applications are a pretty rare occurrence and i don’t believe it justify breaking existing driver.
My comment about using Memory Partitioning (with the upcoming 6.4) versus price is that Adaptive Partition when I last check was an option that cost over 50K. If it’s sold in the same fashion as AP, having to shell out 50K just to be able to use 4G of ram doesn’t make sense.
It certainly isn’t none of the drivers, just not all of the drivers. I don’t think the correct approach is to hamstring the applications just so the drivers don’t need to be fixed (it is, after all, a simple fix).
Exactly my point. This is a kludge, and asking for it (as a customer) makes it happen. I would prefer if customers didn’t ask for problems…
Yeah, but if (in a large system) you ask for 2GB contiguous RAM, and it has to come from below 4GB do you think you’ll actually get it?
Think of a large networking application with a huge routing table. The routing table is read/updated by an ASIC (that knows nothing of page mappings). Since this table is likely to be variable in size (based on configuration), the kludge you mention implies that the posix typed memory region needs to be dynamically resized by that application (or worse that you have to waste memory to allow for the biggest configuration). The app has to do this simply to work around the fact the drivers ( which don’t require dynamic memory sizing) don’t use the Posix calls.
This is the tail (drivers) wagging the dog (application code) IMO…
The memory partitioning option gives you the ability to throw legacy drivers in there while they are fixed…
Seeing that you can download the source for AP from Foundry27, I am not quite sure how we can charge for it. It is only payable in runtime royalties AFAIK…
Can you tell me which driver are and which aren’t 64bit aware?
If it’s so simple why hasn’t it been done?
I didn’t ask for 4G not working with 6.3.2
Well there is no other way around it because Memory Partitioning isn’t available and QNX own driver are broken and I do not have control over 3rd party. I also still think that Memory Partitioning is overkill for such a problem. What you are basically saying is that in 1 year from now everybody will need Memory Partitionning. (Does using Memory Partitioning adds overhead?). Note that I’ve never used MP so I don’t really know what I’m talking about
I don’t know how QNX allocate memory but if it does it from TOP to BOTTOM most probably yes it will get it but I see what you mean.
Note that in my case I’m not even trying to go over 4G i’m trying to go from 3.5 to 4G which requires enabling >4G support.
I don’t see how the “kludge” creates a problem if mmap has a flag that allows memory to be allocated from above 4G. I’m not sure about this but I believe you would want the DMA’able memory to live below 4G for performance reason anyway.
That being said it will look weird in 5 years from now when everything ( even the OS) is 64 bits and 4G won’t mean a thing.
No it doesn’t cause it’s not available, and I can just imagine the number of tech support call this will generated…
Yes I’m talking about royalties.
By the way I just checked mmap and posix_mem* stuff and I couldn’t find a way to specify I want memory below 4G. That means even though I would like to “fix” my own existing driver, I cannot (something similar to MAP_BELOW16M).
.