Hi 'yall,
I am under the impression that QNET isn’t currently able to speak to QNX
nodes that are not of the same endianness. If this is true, is this going
to change? Seems a flag could be part of a QNET header in some way
indicating what order the data is in and do conversions only if necessary
(unlike TCPIP where all traffic is put in netowrk byte order).
Thanks,
Kevin
Xiaodan Tang wrote:
Kevin Stallard <> kevin@ffflyingrobots.com> > wrote:
The problem to really support cross-endian IPC is to turn ALL the QNX
resource manager to be endian aware, and being able to “swap” a data
structure. We are currently design something more generic to do the
“swaping”, and hope there will be some API that user application could
use…
Also, can’t detection, and handling of this flag be built into default
io handlers in the resmgr lib ? At least that way, one would get the
stat information handled for free with simple IO managers (which is a
fairly large class of iomanagers). Hmmm, there should also be some way
for a client to query a resmgr to find out if it handles x-endianess…
Rennie
Kevin Stallard wrote:
Hi 'yall,
I am under the impression that QNET isn’t currently able to speak to QNX
nodes that are not of the same endianness.
PVM supports communication between nodes of different architectures
(endianness …)
If this is true, is this going
to change? Seems a flag could be part of a QNET header in some way
indicating what order the data is in and do conversions only if necessary
(unlike TCPIP where all traffic is put in netowrk byte order).
PVM is available at http://www.sf.net/projects/openqnx
Cheers
Armin
Thanks,
Kevin
What would I do with PVM? It adds overhead to what QNX already does. I
want lean and mean. PVM looks like an after-thought for the poor blokes
that have to deal with MS and Linux 
. QSSL didn’t make it an after thought
and so it is faster. And I am not linking different OS’s… I’m a QNX kind
of guy and all my systems (if I have anything to say about it) will and will
always be QNX based…I just wanted to see if I could use a hybrid of
processors and link them via QNET.
Ok, so there are more issues to the virtual machine concept than is covered
by transparent IPC, but I would bet that making all CPU’s resources
transparent across the net is is 70% to 80% of that. I do a pretty good
job of managing parallel resources on my own, thank you very much!
Further more, if I can avoid TCP/IP, I will. I have what could be
classified as an unfounded dislike of TCP/IP, but I dislike it
none-the-less. I’m glad I have it so I can write this note, and post it so
everyone can read. But I’m not going to use it as the main protocol for any
multi CPU embedded system I design/write, unless that embedded system needs
to talk across networks.
So PVM isn’t the answer…and thus my question remains.
Kevin
“Armin Steinhoff” <a-steinhoff@web_.de> wrote in message
news:3CDFADF5.492B2669@web_.de…
Kevin Stallard wrote:
Hi 'yall,
I am under the impression that QNET isn’t currently able to speak to QNX
nodes that are not of the same endianness.PVM supports communication between nodes of different architectures
(endianness …)If this is true, is this going
to change? Seems a flag could be part of a QNET header in some way
indicating what order the data is in and do conversions only if
necessary
(unlike TCPIP where all traffic is put in netowrk byte order).PVM is available at > http://www.sf.net/projects/openqnx
Cheers
Armin
Thanks,Kevin
Kevin Stallard <kevin@ffflyingrobots.com> wrote:
Hi 'yall,
I am under the impression that QNET isn’t currently able to speak to QNX
nodes that are not of the same endianness. If this is true, is this going
to change? Seems a flag could be part of a QNET header in some way
indicating what order the data is in and do conversions only if necessary
(unlike TCPIP where all traffic is put in netowrk byte order).
The “plan” to support cross-endian is still there, but unfortunatly, not
too much push on it, it’s in a low priority.
However, the latest release Kernel and QNET does support cross-endian IPC,
just not our resource managers.
The idea is this, the server (who did MsgReceive()), will have a flag in
its info->flags to tell that the client (sender) is from a different endian.
It’s upto the server to “swap” the message it received, and, after process
it, reply a “swaped” message back. This “flag”, is already implemented
(_NTO_MI_ENDIAN_DIFF).
So you should be able to send/receive “raw” message cross endian over QNET.
(Hm, I will have to confirm that cause it’s a long time since last time
I checked
The problem to really support cross-endian IPC is to turn ALL the QNX
resource manager to be endian aware, and being able to “swap” a data
structure. We are currently design something more generic to do the
“swaping”, and hope there will be some API that user application could
use…
-xtang
Armin Steinhoff <a-steinhoff@web_.de> wrote:
Kevin Stallard wrote:
Hi 'yall,
I am under the impression that QNET isn’t currently able to speak to QNX
nodes that are not of the same endianness.PVM supports communication between nodes of different architectures
(endianness …)
Armin, since you mentioned PVM again
Don’t you want to make PVM aware of the existing of QNET (between
QNX architectures), and try to use that if possiable ? I mean if
the peer is also a QNX machine, use QNET to directly pass data to
remote process, instead of go through all the daemons and tcpip layer ?
I did that on QNX4 long time ago…
-xtang
Cool!
So basically the messages will get through just fine, I just need to perform
the usual ntohs/l on the received data…thats ok by me!
Kevin
“Xiaodan Tang” <xtang@qnx.com> wrote in message
news:aboj45$drd$1@nntp.qnx.com…
Kevin Stallard <> kevin@ffflyingrobots.com> > wrote:
Hi 'yall,I am under the impression that QNET isn’t currently able to speak to QNX
nodes that are not of the same endianness. If this is true, is this
going
to change? Seems a flag could be part of a QNET header in some way
indicating what order the data is in and do conversions only if
necessary
(unlike TCPIP where all traffic is put in netowrk byte order).The “plan” to support cross-endian is still there, but unfortunatly, not
too much push on it, it’s in a low priority.However, the latest release Kernel and QNET does support cross-endian IPC,
just not our resource managers.The idea is this, the server (who did MsgReceive()), will have a flag in
its info->flags to tell that the client (sender) is from a different
endian.
It’s upto the server to “swap” the message it received, and, after process
it, reply a “swaped” message back. This “flag”, is already implemented
(_NTO_MI_ENDIAN_DIFF).So you should be able to send/receive “raw” message cross endian over
QNET.
(Hm, I will have to confirm that cause it’s a long time since last time
I checked >The problem to really support cross-endian IPC is to turn ALL the QNX
resource manager to be endian aware, and being able to “swap” a data
structure. We are currently design something more generic to do the
“swaping”, and hope there will be some API that user application could
use…-xtang
“Xiaodan Tang” <xtang@qnx.com> wrote in message
news:aboj45$drd$1@nntp.qnx.com…
The problem to really support cross-endian IPC is to turn ALL the QNX
resource manager to be endian aware, and being able to “swap” a data
structure. We are currently design something more generic to do the
“swaping”, and hope there will be some API that user application could
use…
Holly Shrimp Batman!
You’re going to try to have net managers recognize the data structures from
EVERY other resource manager and know the difference between a ‘long int’
(which needs to be swapped) and a ‘char[4]’ (which should not be swapped)?
I don’t envy the guy who has to keep that up to date?
I never though it through before, but I like the TCP/IP method of putting
everything on the net in a predefined order much better than I did
yesterday.
“Bill Caroselli (Q-TPS)” <QTPS@earthlink.net> wrote:
“Xiaodan Tang” <> xtang@qnx.com> > wrote in message
news:aboj45$drd$> 1@nntp.qnx.com> …The problem to really support cross-endian IPC is to turn ALL the QNX
resource manager to be endian aware, and being able to “swap” a data
structure. We are currently design something more generic to do the
“swaping”, and hope there will be some API that user application could
use…
Holly Shrimp Batman!You’re going to try to have net managers recognize the data structures from
EVERY other resource manager and know the difference between a ‘long int’
(which needs to be swapped) and a ‘char[4]’ (which should not be swapped)?
Not really. The net manager will pass the data “as is” to resource
manager. Cause only the “resource manager” knowes what structure
they expect. To us, the resource manager library will have code to
do those “swap” on all QNX message. (Note the user application still
have to do their own “swap” if they have private structure).
The API I am talking about is the “endian_swap()” in /usr/include/gulliver.h
It pass in a pointer, a “structure instruction string”, maybe a length,
and return you a swapped structure…
I don’t envy the guy who has to keep that up to date?
I never though it through before, but I like the TCP/IP method of putting
everything on the net in a predefined order much better than I did
yesterday.
Remember application suppose to not aware of local/remote satuation,
you almost suggest EVERY message send/receive/reply have to do that.
Even if we only do it on cross network case, for “same endian” nodes,
it is still a waste. Especially consider Little Endian talk to LE
node.
-xtang
How is it more work than it is for TCP/IP? You’d have to do the same for it
as well.
There aren’t many resource managers that talk to other resource managers
anyway. However you look at it its clients talking to servers. The servers
can look at the flag described by Xtang and make modifications to the data
structres passed. Some resource managers won’t care (writing to disk for
example) they just deal with raw data anyway (unless its a some seeking
going on, but then a predefined struct is used and the resource manager will
know which need to be swapped).
I don’t see this as being much of a maintenance issue at all. Clients can
be written to not care about endianess.
Kevin
“Bill Caroselli (Q-TPS)” <QTPS@EarthLink.net> wrote in message
news:abovk3$h19$1@inn.qnx.com…
“Xiaodan Tang” <> xtang@qnx.com> > wrote in message
news:aboj45$drd$> 1@nntp.qnx.com> …The problem to really support cross-endian IPC is to turn ALL the QNX
resource manager to be endian aware, and being able to “swap” a data
structure. We are currently design something more generic to do the
“swaping”, and hope there will be some API that user application could
use…
Holly Shrimp Batman!You’re going to try to have net managers recognize the data structures
from
EVERY other resource manager and know the difference between a ‘long int’
(which needs to be swapped) and a ‘char[4]’ (which should not be swapped)?I don’t envy the guy who has to keep that up to date?
I never though it through before, but I like the TCP/IP method of putting
everything on the net in a predefined order much better than I did
yesterday.
Yes, I see now. Sorry. I mis-understood what/where the work was being
done.
“Xiaodan Tang” <xtang@qnx.com> wrote in message
news:abp1ei$q3m$1@nntp.qnx.com…
“Bill Caroselli (Q-TPS)” <> QTPS@earthlink.net> > wrote:
“Xiaodan Tang” <> xtang@qnx.com> > wrote in message
news:aboj45$drd$> 1@nntp.qnx.com> …The problem to really support cross-endian IPC is to turn ALL the QNX
resource manager to be endian aware, and being able to “swap” a data
structure. We are currently design something more generic to do the
“swaping”, and hope there will be some API that user application could
use…
Holly Shrimp Batman!You’re going to try to have net managers recognize the data structures
from
EVERY other resource manager and know the difference between a ‘long
int’
(which needs to be swapped) and a ‘char[4]’ (which should not be
swapped)?Not really. The net manager will pass the data “as is” to resource
manager. Cause only the “resource manager” knowes what structure
they expect. To us, the resource manager library will have code to
do those “swap” on all QNX message. (Note the user application still
have to do their own “swap” if they have private structure).The API I am talking about is the “endian_swap()” in
/usr/include/gulliver.h >
It pass in a pointer, a “structure instruction string”, maybe a length,
and return you a swapped structure…I don’t envy the guy who has to keep that up to date?
I never though it through before, but I like the TCP/IP method of
putting
everything on the net in a predefined order much better than I did
yesterday.Remember application suppose to not aware of local/remote satuation,
you almost suggest EVERY message send/receive/reply have to do that.Even if we only do it on cross network case, for “same endian” nodes,
it is still a waste. Especially consider Little Endian talk to LE
node.-xtang
Some resource managers won’t care (writing to disk for example)
they just deal with raw data anyway (unless its a some seeking
going on, but then a predefined struct is used and the resource manager
will
know which need to be swapped).
Nah bad example since read/write have size data attached to them
that need to be dealth with, not to mention the actual size of the operation
and status returned to the client.
I don’t see this as being much of a maintenance issue at all. Clients can
be written to not care about endianess.Kevin
“Bill Caroselli (Q-TPS)” <> QTPS@EarthLink.net> > wrote in message
news:abovk3$h19$> 1@inn.qnx.com> …
“Xiaodan Tang” <> xtang@qnx.com> > wrote in message
news:aboj45$drd$> 1@nntp.qnx.com> …The problem to really support cross-endian IPC is to turn ALL the QNX
resource manager to be endian aware, and being able to “swap” a data
structure. We are currently design something more generic to do the
“swaping”, and hope there will be some API that user application could
use…
Holly Shrimp Batman!You’re going to try to have net managers recognize the data structures
from
EVERY other resource manager and know the difference between a ‘long
int’
(which needs to be swapped) and a ‘char[4]’ (which should not be
swapped)?I don’t envy the guy who has to keep that up to date?
I never though it through before, but I like the TCP/IP method of
putting
everything on the net in a predefined order much better than I did
yesterday.
\
Not at all…read write take what ever size the client gives for raw data
(RM doesn’t need to interpret raw data)
As for structs…good 'ole sizeof() takes care of that.
If the RM see’s a different endian based client, it swaps return values
before it puts 'em on the wire.
Kevin
“Mario Charest” <goto@nothingness.com> wrote in message
news:abp20k$ieu$1@inn.qnx.com…
Some resource managers won’t care (writing to disk for example)
they just deal with raw data anyway (unless its a some seeking
going on, but then a predefined struct is used and the resource manager
will
know which need to be swapped).Nah bad example since read/write have size data attached to them
that need to be dealth with, not to mention the actual size of the
operation
and status returned to the client.
I don’t see this as being much of a maintenance issue at all. Clients
can
be written to not care about endianess.Kevin
“Bill Caroselli (Q-TPS)” <> QTPS@EarthLink.net> > wrote in message
news:abovk3$h19$> 1@inn.qnx.com> …
“Xiaodan Tang” <> xtang@qnx.com> > wrote in message
news:aboj45$drd$> 1@nntp.qnx.com> …The problem to really support cross-endian IPC is to turn ALL the
QNX
resource manager to be endian aware, and being able to “swap” a data
structure. We are currently design something more generic to do the
“swaping”, and hope there will be some API that user application
could
use…
Holly Shrimp Batman!You’re going to try to have net managers recognize the data structures
from
EVERY other resource manager and know the difference between a ‘long
int’
(which needs to be swapped) and a ‘char[4]’ (which should not be
swapped)?I don’t envy the guy who has to keep that up to date?
I never though it through before, but I like the TCP/IP method of
putting
everything on the net in a predefined order much better than I did
yesterday.
\
“Kevin Stallard” <kevin@ffflyingrobots.com> wrote in message
news:abp31v$jes$1@inn.qnx.com…
Not at all…read write take what ever size the client gives for raw data
(RM doesn’t need to interpret raw data)
As for structs…good 'ole sizeof() takes care of that.
I misread your post, sorry.
If the RM see’s a different endian based client, it swaps return values
before it puts 'em on the wire.Kevin
“Mario Charest” <> goto@nothingness.com> > wrote in message
news:abp20k$ieu$> 1@inn.qnx.com> …
Some resource managers won’t care (writing to disk for example)
they just deal with raw data anyway (unless its a some seeking
going on, but then a predefined struct is used and the resource
manager
will
know which need to be swapped).Nah bad example since read/write have size data attached to them
that need to be dealth with, not to mention the actual size of the
operation
and status returned to the client.
I don’t see this as being much of a maintenance issue at all. Clients
can
be written to not care about endianess.Kevin
“Bill Caroselli (Q-TPS)” <> QTPS@EarthLink.net> > wrote in message
news:abovk3$h19$> 1@inn.qnx.com> …
“Xiaodan Tang” <> xtang@qnx.com> > wrote in message
news:aboj45$drd$> 1@nntp.qnx.com> …The problem to really support cross-endian IPC is to turn ALL the
QNX
resource manager to be endian aware, and being able to “swap” a
data
structure. We are currently design something more generic to do
the
“swaping”, and hope there will be some API that user application
could
use…
Holly Shrimp Batman!You’re going to try to have net managers recognize the data
structures
from
EVERY other resource manager and know the difference between a ‘long
int’
(which needs to be swapped) and a ‘char[4]’ (which should not be
swapped)?I don’t envy the guy who has to keep that up to date?
I never though it through before, but I like the TCP/IP method of
putting
everything on the net in a predefined order much better than I did
yesterday.
\
Kevin Stallard wrote:
What would I do with PVM?
With PVM you would simply solve your problem in a convenient way
It adds overhead to what QNX already does.
What does QNX/QNET already? Does it allow communication between QNX6
systems with different CPU architectures? Does it allow communication
between QNX4 and QNX6 ??
Nope!
I want lean and mean. PVM looks like an after-thought for the poor blokes
that have to deal with MS and Linux >
No … I see here only a poor ‘bloke’ (?) which have to realize
communication between two QNX6 machines with different architectures )
QSSL didn’t make it an after thought and so it is faster.
PVM communication (after setup of a connection) is based on a lean and
secure UDP protocol. What is the base of QNET?? How could QNET be
faster??
BTW … are there implementation of QNET for shared memory systems,
Myrinet, Gigabit Ethernet or other communication media??
And I am not linking different OS’s… I’m a QNX kind
of guy and all my systems (if I have anything to say about it) will and will
always be QNX based…I just wanted to see if I could use a hybrid of
processors and link them via QNET.
Message passing by PVM allows to communicate between different operating
systems (QNX4-QNX6) and between system with different CPU architectures.
Both is until now NOT possible with QNET.
Ok, so there are more issues to the virtual machine concept than is covered
by transparent IPC,
Wrong … there are at first library calls for transparent message
passing and based on it service calls for running the virtual machine.
but I would bet that making all CPU’s resources
transparent across the net is is 70% to 80% of that.
PVM is simply a system independent message passing library … plus
management function for the VM.
I do a pretty good job of managing parallel resources on my own, thank you very much!
Oh … thank you for your reply
Further more, if I can avoid TCP/IP, I will.
So you should not use QNET )
I have what could be classified as an unfounded dislike of TCP/IP, but I dislike it
none-the-less. I’m glad I have it so I can write this note, and post it so
everyone can read. But I’m not going to use it as the main protocol for any
multi CPU embedded system I design/write, unless that embedded system needs
to talk across networks.So PVM isn’t the answer…and thus my question remains.
)
Cheers
Armin
Kevin
“Armin Steinhoff” <a-steinhoff@web_.de> wrote in message
news:3CDFADF5.492B2669@web_.de…
Kevin Stallard wrote:Hi 'yall,
I am under the impression that QNET isn’t currently able to speak to QNX
nodes that are not of the same endianness.PVM supports communication between nodes of different architectures
(endianness …)If this is true, is this going
to change? Seems a flag could be part of a QNET header in some way
indicating what order the data is in and do conversions only if
necessary
(unlike TCPIP where all traffic is put in netowrk byte order).PVM is available at > http://www.sf.net/projects/openqnx
Cheers
Armin
Thanks,Kevin
Xiaodan Tang wrote:
Armin Steinhoff <a-steinhoff@web_.de> wrote:
Kevin Stallard wrote:
Hi 'yall,
I am under the impression that QNET isn’t currently able to speak to QNX
nodes that are not of the same endianness.PVM supports communication between nodes of different architectures
(endianness …)Armin, since you mentioned PVM again >
Yes … it is simply the answer - or better - the solution for Kevins
problem
Don’t you want to make PVM aware of the existing of QNET (between
QNX architectures), and try to use that if possiable ? I mean if
the peer is also a QNX machine, use QNET to directly pass data to
remote process, instead of go through all the daemons
The demons are mainly used for connection setup … the rest of the
peer-to-peer communication is UDP based. So there is no overhead added!
and tcpip layer ?
PVM goes only ‘through’ the UPD ‘layer’
It would make sense to make PVM aware of native message passing of QNX6
for locale IPC … that’s on my to-do list .
Cheers
Armin
I did that on QNX4 long time ago…
-xtang
Xiaodan Tang wrote:
[ clip …]
You’re going to try to have net managers recognize the data structures from
EVERY other resource manager and know the difference between a ‘long int’
(which needs to be swapped) and a ‘char[4]’ (which should not be swapped)?Not really. The net manager will pass the data “as is” to resource
manager. Cause only the “resource manager” knowes what structure
they expect. To us, the resource manager library will have code to
do those “swap” on all QNX message. (Note the user application still
have to do their own “swap” if they have private structure).
Just to make it clear: QNET doesn’t support transparent message passing
between different CPU architectures!!
A clean implementation of transparent message passing between different
CPU architecture would be to convert the contens of the send buffers to
the ‘external data representation’ (RPC) if the architecture of the
receiver
is unknown.
At the receiving site must be signaled that the received buffer comes
from a side with a different architecture, which means the buffer
contens should be in an ‘external data representation’.
Armin
The API I am talking about is the “endian_swap()” in /usr/include/gulliver.h >
It pass in a pointer, a “structure instruction string”, maybe a length,
and return you a swapped structure…I don’t envy the guy who has to keep that up to date?
I never though it through before, but I like the TCP/IP method of putting
everything on the net in a predefined order much better than I did
yesterday.Remember application suppose to not aware of local/remote satuation,
you almost suggest EVERY message send/receive/reply have to do that.Even if we only do it on cross network case, for “same endian” nodes,
it is still a waste. Especially consider Little Endian talk to LE
node.-xtang
What does QNX/QNET already? Does it allow communication between QNX6
systems with different CPU architectures? Does it allow communication
between QNX4 and QNX6 ??
It only doesn’t support communication between CPUs of different endians.
I use qnet to stream audio from realplayer on my x86 PC to my iPaq (ARM)
over 802.11 all the time at home. Nice, portable, wireless internet radio. 
And once the resmgr layer is endian “aware” this won’t even be an issue.
Chris McKillop <cdm@qnx.com> “The faster I go, the behinder I get.”
Software Engineer, QSSL – Lewis Carroll –
http://qnx.wox.org/
Kevin Stallard wrote:
Uhhmm…seems that TCP have these ntohs, ntohl, htons and htonl functions
that do this. If I use TCP/IP to talk to differnet CPU architectures, I
have to use these functions before I place binary data on the wire or take
it off the wire…how is that transparent?
Firstly, I agree that QNET is light-years ahead of PVM; however, Armin
was refering to XDR which is a more complete solution to endianess
problems than anything I have heard planned for QNET at this point (it
has a standardized meta language that describes the format of the data -
but does assume little endian within an octet - which is XDR’s
concession to efficiency). That said, I’m with you and I think the
overhead of XDR’s more generalized solution would be inappropriate for
IPC in an operating system like QNX where lean and mean is the order of
the day. One can always use XDR to communicate outside of the QNX
“computer” (where the QNX “computer” can be more than 1 node of
differing processor types).
Rennie
“Armin Steinhoff” <a-steinhoff@web_.de> wrote in message
news:3CE0B643.7F2E45E2@web_.de…
Kevin Stallard wrote:
What would I do with PVM?
With PVM you would simply solve your problem in a convenient way >
huh? Can I do open("/net/somewhere/dev/ser1") with PVM?
Can I write a file on a different machine with open/read/write/close?
I mean it does sound extremly usefull at user application level, but
it doesn’t solve the problem at the OS level.