32K

Hi,

Is there a memory manipulation limitation ? (32K) …

Thanks for your time in advance !

Arinc

ARINC <arincscutr@hotmail.com> wrote:

Hi,

Is there a memory manipulation limitation ? (32K) …

Yes. 4G.

-David

Hey David, last I heard the hooks were in for >4G memory stuff but it hadn’t
materialized totally yet. I’d love to be able to play with one of those 12G
8-CPU monsters for a couple of projects on my table right now, and I’d rather
not go the Linux route. Any sort of WAG or gut feeling about the priority of
such a feature? Any possibility I can whip up some sort of a resource manager
that can page in stuff without giving the O.S. hemmerhoids? Any idea on where
I can find data on what sort of standard has been adopted by everyone (I mean,
there are still only 32-bits aren’t there?) so even if Pete tells me it’s a
bad idea I can still go beat myself senseless trying to do it anyway?

-Warren “I want it all!” Peece


“David Gibbs” <dagibbs@qnx.com> wrote in message
news:8tq59s$hbq$2@nntp.qnx.com
| ARINC <arincscutr@hotmail.com> wrote:
| > Hi,
|
| > Is there a memory manipulation limitation ? (32K) …
|
| Yes. 4G.
|
| -David

“Warren Peece” <warren@nospam.com> wrote in message
news:8tqjbq$n0q$1@inn.qnx.com

Hey David, last I heard the hooks were in for >4G memory stuff but it
hadn’t
materialized totally yet. I’d love to be able to play with one of those
12G
8-CPU monsters for a couple of projects on my table right now, and I’d
rather
not go the Linux route. Any sort of WAG or gut feeling about the priority
of
such a feature? Any possibility I can whip up some sort of a resource
manager
that can page in stuff without giving the O.S. hemmerhoids? Any idea on
where
I can find data on what sort of standard has been adopted by everyone (I
mean,
there are still only 32-bits aren’t there?) so even if Pete tells me
it’s a
bad idea I can still go beat myself senseless trying to do it anyway?

I vaguely remember the watcom compiler having a far 32 bit model that
would give 48 bits pointers

-Warren “I want it all!” Peece


“David Gibbs” <> dagibbs@qnx.com> > wrote in message
news:8tq59s$hbq$> 2@nntp.qnx.com> …
| ARINC <> arincscutr@hotmail.com> > wrote:
| > Hi,
|
| > Is there a memory manipulation limitation ? (32K) …
|
| Yes. 4G.
|
| -David

Outstanding! Now all you have to do is port the WATCOM compiler and we’re
golden :wink:


-Warren



“Mario Charest” <mcharest@zinformatic.com> wrote in message
news:8tqlup$pjv$1@inn.qnx.com

I vaguely remember the watcom compiler having a far 32 bit model that
would give 48 bits pointers

Warren Peece <warren@nospam.com> wrote:

Hey David, last I heard the hooks were in for >4G memory stuff but it hadn’t
materialized totally yet. I’d love to be able to play with one of those 12G
8-CPU monsters for a couple of projects on my table right now, and I’d rather
not go the Linux route. Any sort of WAG or gut feeling about the priority of
such a feature? Any possibility I can whip up some sort of a resource manager
that can page in stuff without giving the O.S. hemmerhoids? Any idea on where
I can find data on what sort of standard has been adopted by everyone (I mean,
there are still only 32-bits aren’t there?) so even if Pete tells me it’s a
bad idea I can still go beat myself senseless trying to do it anyway?

Sorry, no idea on schedule or priority for this.

-David

David Gibbs <dagibbs@qnx.com> wrote:

Warren Peece <> warren@nospam.com> > wrote:
Hey David, last I heard the hooks were in for >4G memory stuff but it hadn’t
materialized totally yet. I’d love to be able to play with one of those 12G
8-CPU monsters for a couple of projects on my table right now, and I’d rather
not go the Linux route. Any sort of WAG or gut feeling about the priority of
such a feature? Any possibility I can whip up some sort of a resource manager
that can page in stuff without giving the O.S. hemmerhoids? Any idea on where
I can find data on what sort of standard has been adopted by everyone (I mean,
there are still only 32-bits aren’t there?) so even if Pete tells me it’s a
bad idea I can still go beat myself senseless trying to do it anyway?

Sorry, no idea on schedule or priority for this.

Actually, the PCI 2.2 spec specifies a 64 bit address bus, so there really
will be 64 bits going onto it.

If you’re really as masochistic as you make out, then it ought to be
possible to grab some unused address space via mmap, and then have some
assembler stubs do the magic to make RAM that is physically not in the low 4
GB appear in the window you mmaped.

pete@qnx.com wrote:

David Gibbs <> dagibbs@qnx.com> > wrote:
Warren Peece <> warren@nospam.com> > wrote:
Hey David, last I heard the hooks were in for >4G memory stuff but it hadn’t
materialized totally yet. I’d love to be able to play with one of those 12G
8-CPU monsters for a couple of projects on my table right now, and I’d rather
not go the Linux route. Any sort of WAG or gut feeling about the priority of
such a feature? Any possibility I can whip up some sort of a resource manager
that can page in stuff without giving the O.S. hemmerhoids? Any idea on where
I can find data on what sort of standard has been adopted by everyone (I mean,
there are still only 32-bits aren’t there?) so even if Pete tells me it’s a
bad idea I can still go beat myself senseless trying to do it anyway?

Sorry, no idea on schedule or priority for this.

Actually, the PCI 2.2 spec specifies a 64 bit address bus, so there really
will be 64 bits going onto it.

If you’re really as masochistic as you make out, then it ought to be
possible to grab some unused address space via mmap, and then have some
assembler stubs do the magic to make RAM that is physically not in the low 4
GB appear in the window you mmaped.

I don’t think this is masochistic at all; I think it’s pretty cool :slight_smile:
Then you can use this mmaped space as a swap file system, virtually
increasing your core space without the penalty of disk i/o.

Yeah Pete, that’s what I had in mind. Take over a bunch of pages from the
O.S., then twiddle whatever hardware needs twiddling to make >4G memory appear
and disappear in those pages as needed. A couple of issues immediately present
themselves:

  1. If I have a box that supports 12G, and I stick 12G in it, isn’t the PCI BIOS
    going to assign memory addresses above the 12G range for my add-in cards? How
    is the O.S. & device drivers & whatnot going to feel about this?

  2. Where is the documentation on the hardware that I need to twiddle? I assume
    there has to be some sort of standard for manipulating 64-bit addresses with a
    32-bit CPU. I’ve got no idea how they went about implementing this, so any
    suggestions for research material are welcomed.

  3. Will I indeed be able to twiddle this hardware (once I find out what and
    where it is) without causing any grief for the O.S.? I can trap into the
    kernel and raise all kinds of hell as far as mapping and whatnot if need be (I
    hope I don’t have to), but all’s for naught if the system becomes even slightly
    unstable.

This whole scheme smells of extended memory or expanded memory or whatever it
used to be called in good ol’ DOS, doesn’t it? Real huge memory support would
be welcomed, but for the short term my choices are Linux or the magic window
approach with QNX6.

-Warren



<pete@qnx.com> wrote in message news:8tsevj$rui$2@nntp.qnx.com
| David Gibbs <dagibbs@qnx.com> wrote:
| > Warren Peece <warren@nospam.com> wrote:
| >> Hey David, last I heard the hooks were in for >4G memory stuff but it
hadn’t
| >> materialized totally yet. I’d love to be able to play with one of those
12G
| >> 8-CPU monsters for a couple of projects on my table right now, and I’d
rather
| >> not go the Linux route. Any sort of WAG or gut feeling about the priority
of
| >> such a feature? Any possibility I can whip up some sort of a resource
manager
| >> that can page in stuff without giving the O.S. hemmerhoids? Any idea on
where
| >> I can find data on what sort of standard has been adopted by everyone (I
mean,
| >> there are still only 32-bits aren’t there?) so even if Pete tells me
it’s a
| >> bad idea I can still go beat myself senseless trying to do it anyway?
|
| > Sorry, no idea on schedule or priority for this.
|
| Actually, the PCI 2.2 spec specifies a 64 bit address bus, so there really
| will be 64 bits going onto it.
|
| If you’re really as masochistic as you make out, then it ought to be
| possible to grab some unused address space via mmap, and then have some
| assembler stubs do the magic to make RAM that is physically not in the low 4
| GB appear in the window you mmaped.
|

Warren Peece <warren@nospam.com> wrote:

Yeah Pete, that’s what I had in mind. Take over a bunch of pages from the
O.S., then twiddle whatever hardware needs twiddling to make >4G memory appear
and disappear in those pages as needed. A couple of issues immediately present
themselves:

  1. If I have a box that supports 12G, and I stick 12G in it, isn’t the PCI BIOS
    going to assign memory addresses above the 12G range for my add-in cards? How
    is the O.S. & device drivers & whatnot going to feel about this?

It better not put any 32-bit devices above 4-gig, or they won’t work.

  1. Where is the documentation on the hardware that I need to twiddle? I assume
    there has to be some sort of standard for manipulating 64-bit addresses with a
    32-bit CPU. I’ve got no idea how they went about implementing this, so any
    suggestions for research material are welcomed.

The Intel Architecture Software Developers Manual will give you an
insight into why you would you won’t be able to do this.

  1. Will I indeed be able to twiddle this hardware (once I find out what and
    where it is) without causing any grief for the O.S.? I can trap into the
    kernel and raise all kinds of hell as far as mapping and whatnot if need be (I
    hope I don’t have to), but all’s for naught if the system becomes even slightly
    unstable.

No. Only the kernel can execute the priviliged instructions you would
need to execute.

This whole scheme smells of extended memory or expanded memory or whatever it
used to be called in good ol’ DOS, doesn’t it? Real huge memory support would
be welcomed, but for the short term my choices are Linux or the magic window
approach with QNX6.

-Warren



pete@qnx.com> > wrote in message news:8tsevj$rui$> 2@nntp.qnx.com> …
| David Gibbs <> dagibbs@qnx.com> > wrote:
| > Warren Peece <> warren@nospam.com> > wrote:
| >> Hey David, last I heard the hooks were in for >4G memory stuff but it
hadn’t
| >> materialized totally yet. I’d love to be able to play with one of those
12G
| >> 8-CPU monsters for a couple of projects on my table right now, and I’d
rather
| >> not go the Linux route. Any sort of WAG or gut feeling about the priority
of
| >> such a feature? Any possibility I can whip up some sort of a resource
manager
| >> that can page in stuff without giving the O.S. hemmerhoids? Any idea on
where
| >> I can find data on what sort of standard has been adopted by everyone (I
mean,
| >> there are still only 32-bits aren’t there?) so even if Pete tells me
it’s a
| >> bad idea I can still go beat myself senseless trying to do it anyway?
|
| > Sorry, no idea on schedule or priority for this.
|
| Actually, the PCI 2.2 spec specifies a 64 bit address bus, so there really
| will be 64 bits going onto it.
|
| If you’re really as masochistic as you make out, then it ought to be
| possible to grab some unused address space via mmap, and then have some
| assembler stubs do the magic to make RAM that is physically not in the low 4
| GB appear in the window you mmaped.
|

David Donohoe <ddonohoe@qnx.com> wrote:

Warren Peece <> warren@nospam.com> > wrote:

  1. Will I indeed be able to twiddle this hardware (once I find out what and
    where it is) without causing any grief for the O.S.? I can trap into the
    kernel and raise all kinds of hell as far as mapping and whatnot if need be (I
    hope I don’t have to), but all’s for naught if the system becomes even slightly
    unstable.

No. Only the kernel can execute the priviliged instructions you would
need to execute.

Under QNX4 you could put such a privileged instruction (ring0) into an
interrupt handler and it would work, as irq handlers are called by the
kernel…

-David

“David Gibbs” <dagibbs@qnx.com> wrote in message
news:8tt98p$cbg$1@nntp.qnx.com

David Donohoe <> ddonohoe@qnx.com> > wrote:
Warren Peece <> warren@nospam.com> > wrote:

  1. Will I indeed be able to twiddle this hardware (once I find out what
    and
    where it is) without causing any grief for the O.S.? I can trap into
    the
    kernel and raise all kinds of hell as far as mapping and whatnot if
    need be (I
    hope I don’t have to), but all’s for naught if the system becomes even
    slightly
    unstable.

No. Only the kernel can execute the priviliged instructions you would
need to execute.

Under QNX4 you could put such a privileged instruction (ring0) into an
interrupt handler and it would work, as irq handlers are called by the
kernel…

-David

Exactamundo. Hence the trap thingie, unless I’m unduly thwarted by some
sublime architecture change that prevents such behavior. (I’d blame Martin
for that, of course :slight_smile:

I suppose if a 32-bit device is assigned an address >4G it’s going to be a
problem. Perhaps my answer lies in the documentation of the differences
between the 32- and 64-bit PCI busses. Is the >4G memory maybe an add-in
card on the 64-bit bus? Is there a “memory hole” around the 4G mark
(similar to the 1M memory hole on a standard PC) to stuff the 32-bit pCI
addresses below the 4G mark? Time to dig around on the Intel web site (as
suggested), I guess. Perhaps some of the Linux code might lend some hints
in this area as well.

-Warren “Kernel !>Klink<!” Peece

Warren Peece <warren@nospam.com> wrote:

Yeah Pete, that’s what I had in mind. Take over a bunch of pages from the
O.S., then twiddle whatever hardware needs twiddling to make >4G memory appear
and disappear in those pages as needed. A couple of issues immediately present
themselves:

  1. If I have a box that supports 12G, and I stick 12G in it, isn’t the PCI BIOS
    going to assign memory addresses above the 12G range for my add-in cards? How
    is the O.S. & device drivers & whatnot going to feel about this?

I’m not sure it would. I suspect (as a gut feeling) that they will try to
keep all your hardware down in `book’ 0 just to be sure that hardware that
doesn’t support 64 bit addresses will still work.

When I say `book’ here, I’m referring to one of the 4 billion chunks of 4GB
address space.

  1. Where is the documentation on the hardware that I need to twiddle? I assume
    there has to be some sort of standard for manipulating 64-bit addresses with a
    32-bit CPU. I’ve got no idea how they went about implementing this, so any
    suggestions for research material are welcomed.

Hmmm… the PCI 2.2 spec would be the place to start I would think.

  1. Will I indeed be able to twiddle this hardware (once I find out what and
    where it is) without causing any grief for the O.S.? I can trap into the
    kernel and raise all kinds of hell as far as mapping and whatnot if need be (I
    hope I don’t have to), but all’s for naught if the system becomes even slightly
    unstable.

I think you would only be able to read/write the memory using assembler code
you would have to write yourself. I don’t think you could do this in a way
that happilly allows any process to access the other books… only processes
that had your special stuff linked in could use it, and then only via
special routines that know how to do the paging.

This whole scheme smells of extended memory or expanded memory or whatever it
used to be called in good ol’ DOS, doesn’t it? Real huge memory support would
be welcomed, but for the short term my choices are Linux or the magic window
approach with QNX6.

I think the `proper’ way to do this is to use some form of extended TLB’s.

Each process has a page table that tells it how to translate virtual
addresses to physical addresses. It is now possible for the page table to
specify pages that come from other `books’, but this would definitely require
kernel support.

Warren Peece <Warren@nospam.com> wrote:

“David Gibbs” <> dagibbs@qnx.com> > wrote in message
news:8tt98p$cbg$> 1@nntp.qnx.com> …
David Donohoe <> ddonohoe@qnx.com> > wrote:
Warren Peece <> warren@nospam.com> > wrote:

  1. Will I indeed be able to twiddle this hardware (once I find out what
    and
    where it is) without causing any grief for the O.S.? I can trap into
    the
    kernel and raise all kinds of hell as far as mapping and whatnot if
    need be (I
    hope I don’t have to), but all’s for naught if the system becomes even
    slightly
    unstable.

No. Only the kernel can execute the priviliged instructions you would
need to execute.

Under QNX4 you could put such a privileged instruction (ring0) into an
interrupt handler and it would work, as irq handlers are called by the
kernel…

-David

Exactamundo. Hence the trap thingie, unless I’m unduly thwarted by some
sublime architecture change that prevents such behavior. (I’d blame Martin
for that, of course > :slight_smile:

The interrupt handler trick still happens to work. But there aren’t
many Ring 0 instructions that you will be able to execute without
screwing up the kernel. For example, to access physical memory
above 4 gig, you would need to turn on PAE (Physical address
extensions). This would fundmentally alter the meaning of the
page tables, in a way that the kernel would certainly not be
expecting… BOOM! :wink:

“David Donohoe” <ddonohoe@qnx.com> wrote in message
news:8tuhoe$5gn$1@nntp.qnx.com
|
| The interrupt handler trick still happens to work. But there aren’t
| many Ring 0 instructions that you will be able to execute without
| screwing up the kernel. For example, to access physical memory
| above 4 gig, you would need to turn on PAE (Physical address
| extensions). This would fundmentally alter the meaning of the
| page tables, in a way that the kernel would certainly not be
| expecting… BOOM! :wink:

BOOM is undesireable. DOOM, good. BOOM, bad. Not gonna do it. Wouldn’t be
prudent.

-Warren

Gah. So whom do I have to bribe to get this bumped up on the QNX priority
list? I used to be able to lean on Steve thanks to a few photographs I’d
managed to acquire, but since he’s bailed I don’t know what the proper
procedure is these days… :wink:

I was thinking along the lines of a server process that would receive
read/write requests and handle all of the funky memory stuff on behalf of the
client processes. Rather like a block special device, one could say. So will
setting PAE really blow up the colonel?

-Warren


<pete@qnx.com> wrote in message news:8tufqa$496$2@nntp.qnx.com
| Warren Peece <warren@nospam.com> wrote:
| > Yeah Pete, that’s what I had in mind. Take over a bunch of pages from the
| > O.S., then twiddle whatever hardware needs twiddling to make >4G memory
appear
| > and disappear in those pages as needed. A couple of issues immediately
present
| > themselves:
|
| > 1) If I have a box that supports 12G, and I stick 12G in it, isn’t the PCI
BIOS
| > going to assign memory addresses above the 12G range for my add-in cards?
How
| > is the O.S. & device drivers & whatnot going to feel about this?
|
| I’m not sure it would. I suspect (as a gut feeling) that they will try to
| keep all your hardware down in book' 0 just to be sure that hardware that | doesn't support 64 bit addresses will still work. | | When I say book’ here, I’m referring to one of the 4 billion chunks of 4GB
| address space.
|
| > 2) Where is the documentation on the hardware that I need to twiddle? I
assume
| > there has to be some sort of standard for manipulating 64-bit addresses
with a
| > 32-bit CPU. I’ve got no idea how they went about implementing this, so any
| > suggestions for research material are welcomed.
|
| Hmmm… the PCI 2.2 spec would be the place to start I would think.
|
| > 3) Will I indeed be able to twiddle this hardware (once I find out what and
| > where it is) without causing any grief for the O.S.? I can trap into the
| > kernel and raise all kinds of hell as far as mapping and whatnot if need be
(I
| > hope I don’t have to), but all’s for naught if the system becomes even
slightly
| > unstable.
|
| I think you would only be able to read/write the memory using assembler code
| you would have to write yourself. I don’t think you could do this in a way
| that happilly allows any process to access the other books… only processes
| that had your special stuff linked in could use it, and then only via
| special routines that know how to do the paging.
|
| > This whole scheme smells of extended memory or expanded memory or whatever
it
| > used to be called in good ol’ DOS, doesn’t it? Real huge memory support
would
| > be welcomed, but for the short term my choices are Linux or the magic
window
| > approach with QNX6.
|
| I think the proper' way to do this is to use some form of extended TLB's. | | Each process has a page table that tells it how to translate virtual | addresses to physical addresses. It is now possible for the page table to | specify pages that come from other books’, but this would definitely require
| kernel support.
|

PAE is a Pentium XENON feature, wasn’t it on Intel’s web pages?

Along with a sample RAM disk driver. (for OEM’s to test / develop code?)

<pete@qnx.com> wrote in message news:8tufqa$496$2@nntp.qnx.com

Warren Peece <> warren@nospam.com> > wrote:
Yeah Pete, that’s what I had in mind. Take over a bunch of pages from
the
O.S., then twiddle whatever hardware needs twiddling to make >4G memory
appear
and disappear in those pages as needed. A couple of issues immediately
present
themselves:

  1. If I have a box that supports 12G, and I stick 12G in it, isn’t the
    PCI BIOS
    going to assign memory addresses above the 12G range for my add-in
    cards? How
    is the O.S. & device drivers & whatnot going to feel about this?

I’m not sure it would. I suspect (as a gut feeling) that they will try to
keep all your hardware down in `book’ 0 just to be sure that hardware that
doesn’t support 64 bit addresses will still work.

When I say `book’ here, I’m referring to one of the 4 billion chunks of
4GB
address space.

  1. Where is the documentation on the hardware that I need to twiddle? I
    assume
    there has to be some sort of standard for manipulating 64-bit addresses
    with a
    32-bit CPU. I’ve got no idea how they went about implementing this, so
    any
    suggestions for research material are welcomed.

Hmmm… the PCI 2.2 spec would be the place to start I would think.

  1. Will I indeed be able to twiddle this hardware (once I find out what
    and
    where it is) without causing any grief for the O.S.? I can trap into
    the
    kernel and raise all kinds of hell as far as mapping and whatnot if need
    be (I
    hope I don’t have to), but all’s for naught if the system becomes even
    slightly
    unstable.

I think you would only be able to read/write the memory using assembler
code
you would have to write yourself. I don’t think you could do this in a
way
that happilly allows any process to access the other books… only
processes
that had your special stuff linked in could use it, and then only via
special routines that know how to do the paging.

This whole scheme smells of extended memory or expanded memory or
whatever it
used to be called in good ol’ DOS, doesn’t it? Real huge memory support
would
be welcomed, but for the short term my choices are Linux or the magic
window
approach with QNX6.

I think the `proper’ way to do this is to use some form of extended TLB’s.

Each process has a page table that tells it how to translate virtual
addresses to physical addresses. It is now possible for the page table to
specify pages that come from other `books’, but this would definitely
require
kernel support.

PAE is a Pentium XENON feature, wasn’t it on Intel’s web pages?

Along with a sample RAM disk driver. (for OEM’s to test / develop code?)

http://support.intel.com/support/performancetools/pse36/sup4gmem.htm
http://support.intel.com/support/performancetools/pse36/index.htm

<pete@qnx.com> wrote in message news:8tufqa$496$2@nntp.qnx.com

Warren Peece <> warren@nospam.com> > wrote:
Yeah Pete, that’s what I had in mind. Take over a bunch of pages from
the
O.S., then twiddle whatever hardware needs twiddling to make >4G memory
appear
and disappear in those pages as needed. A couple of issues immediately
present
themselves:

  1. If I have a box that supports 12G, and I stick 12G in it, isn’t the
    PCI BIOS
    going to assign memory addresses above the 12G range for my add-in
    cards? How
    is the O.S. & device drivers & whatnot going to feel about this?

I’m not sure it would. I suspect (as a gut feeling) that they will try to
keep all your hardware down in `book’ 0 just to be sure that hardware that
doesn’t support 64 bit addresses will still work.

When I say `book’ here, I’m referring to one of the 4 billion chunks of
4GB
address space.

  1. Where is the documentation on the hardware that I need to twiddle? I
    assume
    there has to be some sort of standard for manipulating 64-bit addresses
    with a
    32-bit CPU. I’ve got no idea how they went about implementing this, so
    any
    suggestions for research material are welcomed.

I think the `proper’ way to do this is to use some form of extended TLB’s.

Each process has a page table that tells it how to translate virtual
addresses to physical addresses. It is now possible for the page table to
specify pages that come from other `books’, but this would definitely
require
kernel support.

Ahh, apparently there are two separate mechanisms for getting at >4G memory.
PAE, which creates 8-byte PTE’s and requires extensive O.S. support, and PSE
which supposedly requires less O.S. support since the page tables remain the
same size. With PSE, page size becomes 4MB though, so I don’t think it’s
something I can just turn on and not affect anything else in the system…


“Michael J. Ferrador” <n2kra@orn.com> wrote in message
news:8tv3tg$5gg$1@inn.qnx.com
| PAE is a Pentium XENON feature, wasn’t it on Intel’s web pages?
|
| Along with a sample RAM disk driver. (for OEM’s to test / develop code?)
|
| http://support.intel.com/support/performancetools/pse36/sup4gmem.htm
| http://support.intel.com/support/performancetools/pse36/index.htm
|