I am looking for advice on developing a simple message passing strategy that
needs to be cross platform. The operating systems it will need to work on
include QNX 4, Neutrino, Linux, FreeBSD and Open BSD. The requirements are
pretty simple. I need to have clients do atomic writes to the queue with
variable length messages. The clients always just write (and shouldn’t
block) and a server will always just read from the queue and process the
messages based on message type. Initially I thought about using fifos but
limits.h (on QNX 4) showed an atomic write size of only 512 bytes. Doing a
pathconf() on an existing QNX 4 fifo residing on a ramdisk showed 5120. The
same call on my Open BSD box showed 512 which unfortunately isn’t large
enough for all possible message types.
. So, I started to investigate
message queues. All operating systems that I need to support except Open
BSD support Posix Message Queues. Open BSD appears only to support Sys V
message queues. It appears that the symantics for these two messaging
strategies are fundamentally different and thus it might be difficult to
create a wrapper class that would hide the differences. Has any one had to
solve a similar problem in the past and if so what was your solution? I am
looking for some advice. Thanks.
Lawrence R. Sweet <lsweet@fct.ca> wrote:
I am looking for advice on developing a simple message passing strategy that
needs to be cross platform. The operating systems it will need to work on
include QNX 4, Neutrino, Linux, FreeBSD and Open BSD.
By “cross-platform” do you mean communicating between different OSes,
or do you mean portable between different OSes?
If you need to communicate cross-platform, generally TCP/IP is your
best choice.
The requirements are
pretty simple. I need to have clients do atomic writes to the queue with
variable length messages. The clients always just write (and shouldn’t
block) and a server will always just read from the queue and process the
messages based on message type. Initially I thought about using fifos but
limits.h (on QNX 4) showed an atomic write size of only 512 bytes. Doing a
pathconf() on an existing QNX 4 fifo residing on a ramdisk showed 5120. The
same call on my Open BSD box showed 512 which unfortunately isn’t large
enough for all possible message types.>
message queues. All operating systems that I need to support except Open
BSD support Posix Message Queues. Open BSD appears only to support Sys V
message queues. It appears that the symantics for these two messaging
strategies are fundamentally different and thus it might be difficult to
create a wrapper class that would hide the differences. Has any one had to
solve a similar problem in the past and if so what was your solution? I am
looking for some advice. Thanks.
The problem with FIFOs (and by extension TCP sockets), is that while you
might get an atomic write size of 512 (or whatever), this doesn’t promise
you read seperation, because it is a stream. So, if two clients did
write()s of 250 and 400 bytes, and you did a read() of 512 bytes, you could
get all of the first write and 262 bytes of the second one in that read().
I wish I knew enough about Open BSD to comment further on portability
choices, but I don’t really.
-David
David Gibbs
QNX Training Services
dagibbs@qnx.com
“David Gibbs” <dagibbs@qnx.com> wrote in message
news:d7ppfp$c1c$1@inn.qnx.com…
Lawrence R. Sweet <> lsweet@fct.ca> > wrote:
I am looking for advice on developing a simple message passing strategy
that
needs to be cross platform. The operating systems it will need to work
on
include QNX 4, Neutrino, Linux, FreeBSD and Open BSD.By “cross-platform” do you mean communicating between different OSes,
or do you mean portable between different OSes?
I mean portable between different OSes.
If you need to communicate cross-platform, generally TCP/IP is your
best choice.The requirements are
pretty simple. I need to have clients do atomic writes to the queue with
variable length messages. The clients always just write (and shouldn’t
block) and a server will always just read from the queue and process the
messages based on message type. Initially I thought about using fifos but
limits.h (on QNX 4) showed an atomic write size of only 512 bytes. Doing
a
pathconf() on an existing QNX 4 fifo residing on a ramdisk showed 5120.
The
same call on my Open BSD box showed 512 which unfortunately isn’t large
enough for all possible message types.>
message queues. All operating systems that I need to support except Open
BSD support Posix Message Queues. Open BSD appears only to support Sys V
message queues. It appears that the symantics for these two messaging
strategies are fundamentally different and thus it might be difficult to
create a wrapper class that would hide the differences. Has any one had
to
solve a similar problem in the past and if so what was your solution? I
am
looking for some advice. Thanks.The problem with FIFOs (and by extension TCP sockets), is that while you
might get an atomic write size of 512 (or whatever), this doesn’t promise
you read seperation, because it is a stream. So, if two clients did
write()s of 250 and 400 bytes, and you did a read() of 512 bytes, you
could
get all of the first write and 262 bytes of the second one in that read().
If I can guarantee atomic writes this wouldn’t be a problem b/c the protocol
has us adding a message header that tells us the length of the data that
follows.
I wish I knew enough about Open BSD to comment further on portability
choices, but I don’t really.
Me too
-David
David Gibbs
QNX Training Services
dagibbs@qnx.com
Lawrence R. Sweet <lsweet@fct.ca> wrote:
“David Gibbs” <> dagibbs@qnx.com> > wrote in message
news:d7ppfp$c1c$> 1@inn.qnx.com> …If I can guarantee atomic writes this wouldn’t be a problem b/c the protocol
has us adding a message header that tells us the length of the data that
follows.
Ah, so with a stream, you could read( sizeof(hdr) );
read( hdr->bytes_to_follow) or something similar.
Hm… maybe Unix domain TCP sockets. Problem is, they are point-to-point,
so the server would, really, have to have one socket per client, and
select() on the list of fds to wait on all of them. With seperate
streams, you wouldn’t have to worry about intermixture of messages.
I’m not sure what atomicity you get, or why you need atomic operations – but
if you need the atomicity to prevent intermixture of messages, the seperate
streams might give you what you need.
Of course, there may be certain scalability issues – if you have a LOT of
clients, you do get into the issue of a bunch of sockets/streams all in
existence at once. This would, also, require io-net and npm-tcpip.so for
QNX, whereas a mqueue solution would be lighter – if you don’t already have
requirements for networking, that is.
(Just brainstorming, dunno if it will help or not.)
-David
David Gibbs
QNX Training Services
dagibbs@qnx.com
Lawrence R. Sweet wrote:
The problem with FIFOs (and by extension TCP sockets), is that while you
might get an atomic write size of 512 (or whatever), this doesn’t promise
you read seperation, because it is a stream. So, if two clients did
write()s of 250 and 400 bytes, and you did a read() of 512 bytes, you
could get all of the first write and 262 bytes of the second one in that read().
If I can guarantee atomic writes this wouldn’t be a problem b/c the protocol
has us adding a message header that tells us the length of the data that
follows.
This only works if you have a single reader, so the two read() necessary
to implement this will both be done by the same process.
I think for QNX4 the atomic write size is the same as the buffer size,
so believe the 5120 value rather than 512. But you are correct that for
some OS their atomic write is smaller: I’ve tested NetBSD 1.6.2 = 512,
SUNOS 5.8 = 5120, Linux RH9 = 4096, and QNX6 = 5120).