Need clarification about mutexes and semaphores

I need some clarification regarding the following paragraphs from the
QNX6 System Architecture book:

"Another useful property of semaphores is that they were defined to
operate between processes. Although our mutexes work between processes,
the POSIX thread standard considers this an optional capability and as
such may not be portable across systems. For synchronization between
threads in a single process, mutexes will be more efficient than semaphores.

As a useful variation, a named semaphore service is also available. It
uses a resource manager and as such allows semaphores to be used between
processes on different machines connected by a network. "


Q1. How can I make mutexes work between processes?

Q2. How can I make un-named semaphores work between processes? (If
process A calls sem_init(&sem, 1, 3) where the second argument (1)
indicates shared memory, what does process B do to get to the shared
semaphore?

Q3. What is the difference between recursive mutexes and un-named
semaphores (other than efficiency)?

Hi Alain,

Alain Achkar wrote:

I need some clarification regarding the following paragraphs from the
QNX6 System Architecture book:

"Another useful property of semaphores is that they were defined to
operate between processes. Although our mutexes work between processes,
the POSIX thread standard considers this an optional capability and as
such may not be portable across systems. For synchronization between
threads in a single process, mutexes will be more efficient than
semaphores.

As a useful variation, a named semaphore service is also available. It
uses a resource manager and as such allows semaphores to be used between
processes on different machines connected by a network. "


Q1. How can I make mutexes work between processes?

Put the mutex in shared memory. Map the shared memory into any process
here the mutex should be used.

Attached is a rough example of how to do this.

Q2. How can I make un-named semaphores work between processes? (If
process A calls sem_init(&sem, 1, 3) where the second argument (1)
indicates shared memory, what does process B do to get to the shared
semaphore?

Like mutexes for threads in separate processes, un-named semaphores have
to go in shared memory. Both process A and process B would have to map
in the shared memory, thus each getting a pointer, which they can pass
into the 1st arg of the sem_init() call.

Q3. What is the difference between recursive mutexes and un-named
semaphores (other than efficiency)?

One important difference is that mutexes by default priority inherit,
and semaphores (both named and unnamed) do not. This feature prevents a
priority inversion situation where a low prio thread could lock a mutex
that a high prio thread will want. If there is an intermediate prio
thread that starves the low prio thread, the low prio thread is
prevented from making progress towards unlocking the mutex so that the
high prio thread can get it. The inheritance feature causes the prio of
the low prio thread to be bumped up to equal that of the high prio
thread, thus preempting the intermediate prio thread, and preventing the
problem.

For this reason, mutexes, even across processes are more desirable that
semaphores.

–Chris Foran

Beware that portability of POSIX semaphores in real life is not much better
than mutexes. Most Unix systems stick with System V IPC due to efficiency
(they are direct kernel calls, as opposed to fugly user level libs that
implement POSIX IPC on many systems). On Linux you can’t share POSIX
semaphores between processes either (last time I checked anyway).

Next caveat is the sharing mechanism - that in itself is not very portable.
While mmap() is available almost everywhere its semantics differ. On Linux
you do not allocate memory with mmap(), on QNX or BSD you can with MAP_ANON,
on Solaris you can too but older versions require you to use fd of /dev/zero
while newer allow MAP_ANON. You can use a named filesystem object, but some
systems will want ltrunc() while others will want ftruncate() and you get
into other kinds of trouble too (bugs, especially on Solaris and neccessity
to deal with locking and cleanup of the files - those are PITA in
themselves). Due to this and other reasons most Unixes stick with System V
API (which does not smell like roses either).

Once you decide how you want to share, choice between semaphore and mutex is
fairly simple. Semaphore has two fundamental differences - one that it has a
counter and the other is that it has no ‘owner’ (i.e., it can be ‘posted’ by
anyone, whereas mutex can only be unlocked by the owner). So semaphores are
naturally suited for flow-control type of operation (producer-consumer
scenario) and mutex as its aptly chosen name suggests is best for mutual
exclusion. A semaphore can be emulated with combination of mutex and
condvar.

– igor

“Chris Foran” <cforan@qnx.com> wrote in message
news:db6evf$h73$1@inn.qnx.com

Hi Alain,

Alain Achkar wrote:
I need some clarification regarding the following paragraphs from the
QNX6 System Architecture book:

"Another useful property of semaphores is that they were defined to
operate between processes. Although our mutexes work between processes,
the POSIX thread standard considers this an optional capability and as
such may not be portable across systems. For synchronization between
threads in a single process, mutexes will be more efficient than
semaphores.

As a useful variation, a named semaphore service is also available. It
uses a resource manager and as such allows semaphores to be used between
processes on different machines connected by a network. "


Q1. How can I make mutexes work between processes?

Put the mutex in shared memory. Map the shared memory into any process
here the mutex should be used.

Attached is a rough example of how to do this.


Q2. How can I make un-named semaphores work between processes? (If
process A calls sem_init(&sem, 1, 3) where the second argument (1)
indicates shared memory, what does process B do to get to the shared
semaphore?

Like mutexes for threads in separate processes, un-named semaphores have
to go in shared memory. Both process A and process B would have to map
in the shared memory, thus each getting a pointer, which they can pass
into the 1st arg of the sem_init() call.

Q3. What is the difference between recursive mutexes and un-named
semaphores (other than efficiency)?

One important difference is that mutexes by default priority inherit,
and semaphores (both named and unnamed) do not. This feature prevents a
priority inversion situation where a low prio thread could lock a mutex
that a high prio thread will want. If there is an intermediate prio
thread that starves the low prio thread, the low prio thread is
prevented from making progress towards unlocking the mutex so that the
high prio thread can get it. The inheritance feature causes the prio of
the low prio thread to be bumped up to equal that of the high prio
thread, thus preempting the intermediate prio thread, and preventing the
problem.

For this reason, mutexes, even across processes are more desirable that
semaphores.

–Chris Foran




/*

  • shmemuser.c
  • This one is meant to be run in tandem with shmemcreator.c.
  • Run it as: shmemuser shared_memory_object_name
  • Example: shmemuser /wally

*/

#include <errno.h
#include <fcntl.h
#include <stdio.h
#include <stdlib.h
#include <string.h
#include <unistd.h
#include <sys/mman.h
#include <sys/stat.h
#include <pthread.h

/* shmem.h contains the structure that is overlayed on the shared memory
*/
#include “shmem.h”

/*

  • our global variables.
    */

char *progname = “shmemuser”;

int main( int argc, char *argv[] )
{
int fd;
shmem_t *ptr;

setvbuf (stdout, NULL, _IOLBF, 0);

if ( argc != 2 ) {
printf( “use: shmemuser shared_memory_object_name\n” );
printf( “Example: shmemuser /wally\n” );
exit( EXIT_FAILURE );
}

/* open the shared memory object */

fd = shm_open( argv[1], O_RDWR, S_IRWXU );
if ( fd == -1 ) {
printf( “%s: error opening the shared memory object ‘%s’: %s\n”,
progname, argv[1], strerror(errno) );
exit( EXIT_FAILURE );
}

/* get a pointer to a piece of the shared memory */

ptr = mmap( 0, sizeof(shmem_t), PROT_READ|PROT_WRITE, MAP_SHARED, fd,
0 );

/* don’t need fd anymore */
close( fd );

/* wait on the semaphore */

printf( “%s: Waiting for the mutex to be unlocked. Run ‘pidin’. I
should be mutex blocked.\n”, progname );

pthread_mutex_lock(&ptr->mutex);
printf("%s: mutex was unlocked\n", progname);
printf( “%s: The shared memory contains ‘%s’\n”, progname, ptr->text );
pthread_mutex_unlock(&ptr->mutex);

munmap( ptr, sizeof(shmem_t) );
return 0;
}




#include <pthread.h

#define MAX_TEXT_LEN 100

typedef struct {
pthread_mutex_t mutex;
char text[MAX_TEXT_LEN+1];
} shmem_t;




/*

  • shmemcreator.c
  • This module demonstrates shared memory and using a mutex to sync. the
    access
  • of multiple processes. It creates some shared memory and puts the data
    and
  • mutex in it.
  • This one is meant to be run in tandem with shmemuser.c.
  • Run it as: shmemcreator shared_memory_object_name
  • Example: shmemcreator /wally

*/

#include <errno.h
#include <fcntl.h
#include <stdio.h
#include <stdlib.h
#include <string.h
#include <unistd.h
#include <sys/mman.h
#include <sys/stat.h
#include <pthread.h

/* shmem.h contains the structure that is overlayed on the shared memory
*/
#include “shmem.h”

/*

  • our global variables.
    */

char *progname = “shmemcreator”;

int main( int argc, char *argv[] )
{
char *text = “Text by shmemcreator.c”;
int fd;
shmem_t ptr;
pthread_mutexattr_t mutex_attr; //mutex attributes
pthread_mutex_t
mutex; //will point to the mutex in shmem

setvbuf (stdout, NULL, _IOLBF, 0);

if ( argc != 2 ) {
printf( “use: shmemcreator shared_memory_object_name\n” );
printf( “Example: shmemcreator /wally\n” );
exit( EXIT_FAILURE );
}

/* create the shared memory object */

fd = shm_open( argv[1], O_RDWR | O_CREAT, S_IRWXU );
if ( fd == -1 ) {
printf( “%s: error creating the shared memory object ‘%s’: %s\n”,
progname, argv[1], strerror(errno) );
exit( EXIT_FAILURE );
}

/* set the size of the shared memory object */

ftruncate( fd, sizeof(shmem_t) );

/* get a pointer to a piece of the shared memory */

ptr = mmap( 0, sizeof(shmem_t), PROT_READ|PROT_WRITE, MAP_SHARED, fd,
0 );

/* don’t need fd anymore, so close it */
close( fd );

pthread_mutexattr_init(&mutex_attr);
pthread_mutexattr_setpshared(&mutex_attr, PTHREAD_PROCESS_SHARED);
mutex = (pthread_mutex_t*)ptr;
pthread_mutex_init(mutex, &mutex_attr);

pthread_mutex_lock(&ptr->mutex);
strcpy( ptr->text, text ); /* write to the shared memory */

printf( “%s: Shared memory created and mutex initialized\n”
“%s: Wrote text ‘%s’ to shared memory.\n”
“%s: Sleeping for 20 seconds. While this program is
sleeping\n”
“%s: run ‘shmemuser %s’.\n”,
progname, progname, ptr->text, progname, progname, argv[1] );
sleep( 20 );

printf( “%s: Woke up. Now I’ll unlock the mutex\n”, progname );
pthread_mutex_unlock(&ptr->mutex);

/* another delay for people to look at things */
sleep( 10 );

munmap( ptr, sizeof(shmem_t) );
shm_unlink( argv[1] );

return 0;
}

First, you may not need shared memory at all. It’s rare
that you actually need shared memory under QNX, because QNX
has good interprocess communication mechanisms. Under Linux
and UNIX, interprocess communication is so slow that shared
memory is often needed for performance reasons. This is less
often the case under QNX. If you want to communicate
between processes, use MsgSend/MsgReceive, which is hundreds
of times faster than socket-based communication under Linux
or UNIX.

A multithreaded program is much easier to organize than
shared memory. The compiler and loader handle memory
allocation for you. With shared memory, you have to carefully
lay out your shared area and coordinate its use.

Worth noting is that mutexes under QNX do not make a system
call when they don’t block. They’re no more expensive than
an if statement. Semaphores are slower.

John Nagle
Team Overbot

Alain Achkar wrote:

I need some clarification regarding the following paragraphs from the
QNX6 System Architecture book:

"Another useful property of semaphores is that they were defined to
operate between processes. Although our mutexes work between processes,
the POSIX thread standard considers this an optional capability and as
such may not be portable across systems. For synchronization between
threads in a single process, mutexes will be more efficient than
semaphores.

As a useful variation, a named semaphore service is also available. It
uses a resource manager and as such allows semaphores to be used between
processes on different machines connected by a network. "


Q1. How can I make mutexes work between processes?

Q2. How can I make un-named semaphores work between processes? (If
process A calls sem_init(&sem, 1, 3) where the second argument (1)
indicates shared memory, what does process B do to get to the shared
semaphore?

Q3. What is the difference between recursive mutexes and un-named
semaphores (other than efficiency)?

“John Nagle” <nagle@downside.com> wrote in message
news:dba5bt$9qp$1@inn.qnx.com

First, you may not need shared memory at all. It’s rare
that you actually need shared memory under QNX, because QNX
has good interprocess communication mechanisms. Under Linux
and UNIX, interprocess communication is so slow that shared
memory is often needed for performance reasons. This is less
often the case under QNX. If you want to communicate
between processes, use MsgSend/MsgReceive, which is hundreds
of times faster than socket-based communication under Linux
or UNIX.

That is a stretch. It is faster, but not hundreds of times. But to offset
that, sockets are faster under Linux (ok, they were when I measured 2 years
ago and may or may not hold now). The reasons to use shared memory are many
and they may have nothing to do with the speed. Sometimes it is just
convinient.

As of speed, the message passing taken as universal approach to do
everything is something like a credit card. You go around spending little
innocent amounts like $20 or so. And then you get a bill for $5000. At first
you could not believe it actually added up to that much and reach for
calculator… The problem lies in the fact that complex systems tend to be
designed in layered fashion (and for good reasons). Unless the benefits and
costs of message passing are carefully considered, you end up with 20
context switches to do something seemingly trivial. For that reason QNX has
never used message passing between their own drivers and respective I/O
managers.

A multithreaded program is much easier to organize than
shared memory. The compiler and loader handle memory
allocation for you. With shared memory, you have to carefully
lay out your shared area and coordinate its use.

Threads may seem deceptively easy at first, but as your design evolves they
become increasingly hard to keep tidy. The careful coordination that you put
into shared memory design might as well pay off in terms of reliability and
resilience to errors/crashes/etc. It is possible to design for resilience
with threads, but it actually takes more effort/skills than with processes.

As far as I have seen, beginners almost inevitably abuse threads by failing
to organize them into any kind of formal construction with well-thought
policies and then the mess ensues.

Worth noting is that mutexes under QNX do not make a system
call when they don’t block. They’re no more expensive than
an if statement. Semaphores are slower.

Indeed, but they do more. No free lunch.

John Nagle
Team Overbot

Alain Achkar wrote:
I need some clarification regarding the following paragraphs from the
QNX6 System Architecture book:

"Another useful property of semaphores is that they were defined to
operate between processes. Although our mutexes work between processes,
the POSIX thread standard considers this an optional capability and as
such may not be portable across systems. For synchronization between
threads in a single process, mutexes will be more efficient than
semaphores.

As a useful variation, a named semaphore service is also available. It
uses a resource manager and as such allows semaphores to be used between
processes on different machines connected by a network. "


Q1. How can I make mutexes work between processes?

Q2. How can I make un-named semaphores work between processes? (If
process A calls sem_init(&sem, 1, 3) where the second argument (1)
indicates shared memory, what does process B do to get to the shared
semaphore?

Q3. What is the difference between recursive mutexes and un-named
semaphores (other than efficiency)?

Alain Achkar <js@simplytech.com> wrote:

I need some clarification regarding the following paragraphs from the
QNX6 System Architecture book:

"Another useful property of semaphores is that they were defined to
operate between processes. Although our mutexes work between processes,
the POSIX thread standard considers this an optional capability and as
such may not be portable across systems. For synchronization between
threads in a single process, mutexes will be more efficient than semaphores.

As a useful variation, a named semaphore service is also available. It
uses a resource manager and as such allows semaphores to be used between
processes on different machines connected by a network. "



Q1. How can I make mutexes work between processes?

Put them in a shared memory area (See shm_open() & mmap() for that setup.)
and use the pthread_mutexattr_setpshared() routine to set the
PTHREAD_PROCESS_SHARED flag in the attribute structure you use to
initialize the mutex.

Q2. How can I make un-named semaphores work between processes? (If
process A calls sem_init(&sem, 1, 3) where the second argument (1)
indicates shared memory, what does process B do to get to the shared
semaphore?

Put the semaphore in shared memory, at a known offset from the start
of the shared memory, and have process B map in the shared memory.
(Basically the same as for a mutex.)

Q3. What is the difference between recursive mutexes and un-named
semaphores (other than efficiency)?

Mutexes have ownership, semaphores don’t. What that means is two
big things – 1) for a mutex, only the thread that locked the mutex
can unlock it – whereas any thread can post a semaphore. And 2) because
of (1), the OS can (and does) boost the priority of a thread that owns
a mutex if a higher priority thread tries to lock the mutex, preventing
possible priority inversion problems.

The above is true for both normal, and recursive mutexes.

Recursive mutexes differ from ordinary ones in that they allow
“relocking” by the same thread that already has them locked (rather
than failing that with EDEADLCK).

(Also, mutexes can be used with condvars, semaphores can’t. Recursive
mutexes can not safely be used with condvars.)

(And, in particular, a recursive mutex allows any number of relocks
of the mutex – a counting semaphore will block when the count gets
to zero – they are almost the inverse of each other. By that, I
mean that a mutex is kind of like a semaphore that is only used for
values of 1 or 0; with a recursive mutex, it allows a successful
lock (sem_wait) when the value is 0 (or negative), if it is the same
thread; whereas with a semaphore that will never happen – but with
the semaphore, the value can be taken positive – pre-allowing several
sem_wait() calls to not block, but only a fixed pre-determined number
of those.)

Hopefully this clarifies things a bit.

-David

David Gibbs
QNX Training Services
dagibbs@qnx.com

“David Gibbs” <dagibbs@qnx.com> wrote in message
news:dbgscq$65r$1@inn.qnx.com

Alain Achkar <> js@simplytech.com> > wrote:
I need some clarification regarding the following paragraphs from the
QNX6 System Architecture book:

Q3. What is the difference between recursive mutexes and un-named
semaphores (other than efficiency)?

Mutexes have ownership, semaphores don’t. What that means is two
big things – 1) for a mutex, only the thread that locked the mutex
can unlock it – whereas any thread can post a semaphore. And 2) because
of (1), the OS can (and does) boost the priority of a thread that owns
a mutex if a higher priority thread tries to lock the mutex, preventing
possible priority inversion problems.

It is logically misleading to think about semaphores as ‘another type of
mutex’ with no priority inheritance.

In the most abstract sense:

You use mutexes to prevent two entities from accessing something (a
logically singular resource) simultaneously. There the priority inversion is
a valid problem since higher priority entity may be indeed held by lower
priority one. An important characteristic of this case is that your entities
do not care at all about their relative pace of work. As far as higher
priority one is concerned, the lower priority one may go to hell altogether.

But you use semaphores in a fundamentally different case - to regulate the
relative pace of work of two (or more) entities accessing some vector-type
resource. One entity producing something for the other one to consume
through a queue would be the most classic example. There the priority
inversion is a void problem, because one of the entities can not usefully
work if the other one is starved of the CPU. So it does not make a lot of
sense to boost the priority (in all likelyhood your entities would have
equal priority, unless you want to slightly favor one). In this scenario
semaphore would actually provide better performance since you have fewer
potential kernel calls (in the worst case).

– igor

Igor Kovalenko <kovalenko@comcast.net> wrote:

“David Gibbs” <> dagibbs@qnx.com> > wrote in message

It is logically misleading to think about semaphores as ‘another type of
mutex’ with no priority inheritance.

I agree.

But, I’ve seen a lot of code that does (and taught developpers who
think that)

sem_init( count = 1 )
sem_wait()
access_critical_data
sem_post()

Is the equivalent of:
pthread_mutex_init()
pthread_mutex_lock()
access_critical_data
pthread_mutex_unlock()

If semaphores are used in this way – and it DOES happen – they are
like a mutex, but without priority inheritance.

The fact that the original poster was asking:
“what is the difference between a recursive mutex and a semaphore”
strongly suggests the original posted had this confusion.

In the most abstract sense:

Yup. I’m quite well aware of this.

semaphores can be (well) used for a form of dispatch. In this use,
they actually compare more to a condition variable than a mutex, as
both are forms of dispatch.

semaphores can be (badly) used for a form of locking. They should
not be used this way if mutexes are available.

-David

David Gibbs
QNX Training Services
dagibbs@qnx.com

“David Gibbs” <dagibbs@qnx.com> wrote in message
news:dbjo4p$aap$1@inn.qnx.com

Igor Kovalenko <> kovalenko@comcast.net> > wrote:
“David Gibbs” <> dagibbs@qnx.com> > wrote in message

It is logically misleading to think about semaphores as ‘another type of
mutex’ with no priority inheritance.

I agree.

But, I’ve seen a lot of code that does (and taught developpers who
think that)

sem_init( count = 1 )
sem_wait()
access_critical_data
sem_post()

Is the equivalent of:
pthread_mutex_init()
pthread_mutex_lock()
access_critical_data
pthread_mutex_unlock()

If semaphores are used in this way – and it DOES happen – they are
like a mutex, but without priority inheritance.

The fact that the original poster was asking:
“what is the difference between a recursive mutex and a semaphore”
strongly suggests the original posted had this confusion.

I know, for most part this is just an old Unix-ism. People simply chose the
most efficient mechanism and SysV semaphores were about the fastest
synchronization primitive available there. Those folk usually don’t care
about priority inversion either, given a typical Unix scheduler. It was very
much the same story on Linux, until lately. And even now, lot of people are
just not used to PTHREADS APIs and stick to something they know. Old dogs,
new tricks… Of course it does not help that PTHREADS on Linux still suck,
although they kinda suck less now.

I’d be surprized though to see a lot of this done with POSIX semaphores -
they are just about as new on most *nix as PTHREADS are. Then it is mostly
just confusion about things - lot of books go to great lengths to explain
minute details about semaphores and mutexes and give lot of code examples,
but fail to explain when and why those two particular formalisms should be
used in a generic sense.

In fact I have observed that when I say ‘semaphore’ a lot of people don’t
necessarily think about the specific concept defined by Dijkstra and
implemented by POSIX. They often think that semaphore is a catch-all word
refering to any sort of syncrhonization mechanism. The fact that SysV
semaphores behave quite differently keeps things confused enough.

– igor

Igor Kovalenkowrote:


[…]

As of speed, the message passing taken as universal approach to do
everything is something like a credit card. You go around spending
little
innocent amounts like $20 or so. And then you get a bill for $5000.
At first
you could not believe it actually added up to that much and reach
for
calculator… The problem lies in the fact that complex systems tend
to be
designed in layered fashion (and for good reasons). Unless the
benefits and

Yup; message passing can be abused (if you ever come across a
mechanism/concept/abstraction/entity/religion/ideology that can’t be
abused I would sure like to know about it :wink:.

Igor Kovalenkowrote:
costs of message passing are carefully considered, you end up with 20



context switches to do something seemingly trivial. For that reason
QNX has
never used message passing between their own drivers and respective
I/O
managers.

Is that why ? Perhaps it’s simply because message passing between the
iomanager and the driver would be an inappropriate use of message
passing ? (i.e. message passing certainly would’nt be significant
performance hit for say, serial ports - but it would still be a dumb
design).

Sure shared memory has a place, but it is way overused by those who
are too lazy to learn about message passing, and in this respect I
think that Johns comments were an appropriate response to the
question.

Rennie

“rgallen” <rallen@csical-dot-com.no-spam.invalid> wrote in message
news:dbou34$45d$1@inn.qnx.com

Igor Kovalenkowrote:
costs of message passing are carefully considered, you end up with 20

context switches to do something seemingly trivial. For that reason
QNX has
never used message passing between their own drivers and respective
I/O
managers.


Is that why ? Perhaps it’s simply because message passing between the
iomanager and the driver would be an inappropriate use of message
passing ? (i.e. message passing certainly would’nt be significant
performance hit for say, serial ports - but it would still be a dumb
design).

Bad example to make your point Rennie. Serial drivers in QNX never had a
‘respective iomanager’ in the first place. Most other I/O subsystems would
incur a performance hit indeed. Also I would not advice to use an
inefficient method just because in a specific case the penalty is
irrelevant. That way you end up with a system that has lot of small pieces
each of them designed under assumption that it would not care about
inefficiency. Windows anyone?

Sure shared memory has a place, but it is way overused by those who
are too lazy to learn about message passing, and in this respect I
think that Johns comments were an appropriate response to the
question.

After using QNX for 11 years and really learning how to use message
passing, I’d say that if I was designing something I’d avoid using raw
message passing almost at all costs. I’d use it [implicitly] for a resource
manager of course (and I have written my fair share of those). But direct
raw message passing between a client (user app) and some ‘server’ - it’d be
cold day in hell before I did it again. It is just Bad Idea ™.

Nevertheless, I did not say John’s response was inappropriate. Just wanted
to show the other side of the coin. You know I like playing a little devil’s
advocate here and there.

– igor

“Igor Kovalenko” <kovalenko@comcast.net> wrote in message
news:dbt2pu$41n$1@inn.qnx.com

Bad example to make your point Rennie. Serial drivers in QNX never had a
‘respective iomanager’ in the first place. Most other I/O subsystems would

I meant QNX6.

– igor

I’d like to take exception to your opinion that raw message passing is a bad
idea. I use the resource manager method to establist the initial
connection. After that I use raw message passing. This allows me to pass a
complicted structure with a command field and other parameters. The
‘resource manager’ can then reply with a similar complex structure as and
when appropriate. I also have a whole library of code to convert a function
call to a raw send/receive. It is very efficient.

What is wrong with that?


“Igor Kovalenko” <kovalenko@comcast.net> wrote in message
news:dbt2pu$41n$1@inn.qnx.com

After using QNX for 11 years and really learning how to use message
passing, I’d say that if I was designing something I’d avoid using raw
message passing almost at all costs. I’d use it [implicitly] for a
resource
manager of course (and I have written my fair share of those). But direct
raw message passing between a client (user app) and some ‘server’ - it’d
be
cold day in hell before I did it again. It is just Bad Idea ™.

What is wrong with SyncMutexLock()? Or TimerCreate()? The same as
MsgSend()… they are QNX-specific kernel calls. Lowest level possible. If
you don’t use the former two directly (I presume you use their POSIX covers)
then why do you feel that the latter must be an exception?

If as you said you already have resmgr, then why not use ioctl() to pass
your complicated structure? It enforces a lot more discipline and leads to
much more robust and portable code. And yes I know you don’t care about
portability for religious reasons. It is still a Bad Idea to mix up calls
from different abstraction layers in the user-level code (at least when you
can help it).

“Bill Caroselli” <qtps@earthlink.net> wrote in message
news:dbu77f$r4f$1@inn.qnx.com

I’d like to take exception to your opinion that raw message passing is a
bad
idea. I use the resource manager method to establist the initial
connection. After that I use raw message passing. This allows me to pass
a
complicted structure with a command field and other parameters. The
‘resource manager’ can then reply with a similar complex structure as and
when appropriate. I also have a whole library of code to convert a
function
call to a raw send/receive. It is very efficient.

What is wrong with that?


“Igor Kovalenko” <> kovalenko@comcast.net> > wrote in message
news:dbt2pu$41n$> 1@inn.qnx.com> …

After using QNX for 11 years and really learning how to use message
passing, I’d say that if I was designing something I’d avoid using raw
message passing almost at all costs. I’d use it [implicitly] for a
resource
manager of course (and I have written my fair share of those). But direct
raw message passing between a client (user app) and some ‘server’ - it’d
be
cold day in hell before I did it again. It is just Bad Idea ™.

“Igor Kovalenko” <kovalenko@comcast.net> wrote in message
news:dbudhp$24t$1@inn.qnx.com

What is wrong with SyncMutexLock()? Or TimerCreate()? The same as
MsgSend()… they are QNX-specific kernel calls. Lowest level possible. If
you don’t use the former two directly (I presume you use their POSIX
covers) then why do you feel that the latter must be an exception?

If as you said you already have resmgr, then why not use ioctl() to pass
your complicated structure?

I usually even go to the lenght of not using ioctl and sticking with
read/write and cleaver use of path space. I find ioctl to be a bitch :wink:

much more robust and portable code. And yes I know you don’t care about
portability for religious reasons. It is still a Bad Idea to mix up calls
from different abstraction layers in the user-level code (at least when
you can help it).


“Bill Caroselli” <> qtps@earthlink.net> > wrote in message
news:dbu77f$r4f$> 1@inn.qnx.com> …
I’d like to take exception to your opinion that raw message passing is a
bad
idea. I use the resource manager method to establist the initial
connection. After that I use raw message passing. This allows me to
pass a
complicted structure with a command field and other parameters. The
‘resource manager’ can then reply with a similar complex structure as and
when appropriate. I also have a whole library of code to convert a
function
call to a raw send/receive. It is very efficient.

What is wrong with that?


“Igor Kovalenko” <> kovalenko@comcast.net> > wrote in message
news:dbt2pu$41n$> 1@inn.qnx.com> …

After using QNX for 11 years and really learning how to use message
passing, I’d say that if I was designing something I’d avoid using raw
message passing almost at all costs. I’d use it [implicitly] for a
resource
manager of course (and I have written my fair share of those). But
direct
raw message passing between a client (user app) and some ‘server’ - it’d
be
cold day in hell before I did it again. It is just Bad Idea ™.

\

Igor Kovalenko wrote:

Bad example to make your point Rennie. Serial drivers in QNX never had a
‘respective iomanager’ in the first place. Most other I/O subsystems would
incur a performance hit indeed. Also I would not advice to use an
inefficient method just because in a specific case the penalty is
irrelevant.

You (should) know that this is not the point I was making. I was making
exactly the point that there are many reasons why a design decision is
made. I have (many times) ruled out the performance penalty of message
passing as a concern early in the process, and still ended up without
message passing for reasons other than performance; in fact, there are
only a handful of cases I can recall where I chose not to use message
passing with the primary factor being performance concerns.

After using QNX for 11 years and really learning how to use message
passing, I’d say that if I was designing something I’d avoid using raw
message passing almost at all costs. I’d use it [implicitly] for a resource
manager of course (and I have written my fair share of those). But direct
raw message passing between a client (user app) and some ‘server’ - it’d be
cold day in hell before I did it again. It is just Bad Idea ™.

Then you are ruling out a viable option for no apparently good reason
other than that you believe it to be a “Bad Idea”.

The point is that on systems without message passing, it isn’t an
option; on QNX it is, and to not consider it as a design option because
of unfamiliarity is doing a disservice to the application.

Nevertheless, I did not say John’s response was inappropriate. Just wanted
to show the other side of the coin. You know I like playing a little devil’s
advocate here and there.

You, play the devils advocate ??? :wink:

Rennie

“Rennie Allen” <rnogspamallen@comcast.net> wrote in message
news:dc08lt$c43$1@inn.qnx.com

Igor Kovalenko wrote:

Bad example to make your point Rennie. Serial drivers in QNX never had a
‘respective iomanager’ in the first place. Most other I/O subsystems
would incur a performance hit indeed. Also I would not advice to use an
inefficient method just because in a specific case the penalty is
irrelevant.

You (should) know that this is not the point I was making. I was making

I do. But everyone else does not necessarily know and your post could be
interpreted in more than one way.

After using QNX for 11 years and really learning how to use message
passing, I’d say that if I was designing something I’d avoid using raw
message passing almost at all costs. I’d use it [implicitly] for a
resource manager of course (and I have written my fair share of those).
But direct
raw message passing between a client (user app) and some ‘server’ - it’d
be cold day in hell before I did it again. It is just Bad Idea ™.

Then you are ruling out a viable option for no apparently good reason
other than that you believe it to be a “Bad Idea”.

The point is that on systems without message passing, it isn’t an option;
on QNX it is, and to not consider it as a design option because of
unfamiliarity is doing a disservice to the application.

Again with ‘unfamiliarity’. Did not I say that I am pretty damn familiar
with it? Or do I need to prove it to you?
See my reply to Bill for one of my reasons. There are others too, but I am
getting tired of this discussion.

It is clearly off topic by now.

– igor

Well, thanks for everyone for their input!

I surely got a lot of explanations to help me with my design decision,
which is a great thing on this newsgroup!

Bill Caroselli wrote:

I’d like to take exception to your opinion that raw message passing is a bad
idea. I use the resource manager method to establist the initial
connection. After that I use raw message passing. This allows me to pass a
complicted structure with a command field and other parameters. The
‘resource manager’ can then reply with a similar complex structure as and
when appropriate. I also have a whole library of code to convert a function
call to a raw send/receive. It is very efficient.

What is wrong with that?

I don’t see the enthusiasm for resource managers. I’ve written one, for
FireWire cameras, but that was because it really was something for which
stream-based read and write made sense.

If you want an interprocess subroutine call, MsgSend is the right
mechanism. Putting stream buffering on top of messaging, and then
having to delimit the streams in some way, is complicated and
inefficient. Yes, the sockets-based world does it that way,
and pays a heavy price for it. See any CORBA implementation.

MsgSend is not that slow. See the benchmarks. I run uncompressed video
through MsgSend, and it’s using a few percent of a Pentium III class machine.
If you’re worried about overhead for bulk transfers, use big messages.
I use one message per frame.

The main problem with the MsgSend world is that the directory services
needed to set up connections are weak. Trying to get a process family to
intercommunicate is a mess. There’s name_attach, but that’s global,
and didn’t work right until QNX 6.3. Inter-machine spawn didn’t
work right in 6.21, either. (Is it fixed in 6.3)?

John Nagle

“John Nagle” <nagle@overbot.com> wrote in message
news:dc6cdf$qke$1@inn.qnx.com

Bill Caroselli wrote:
I’d like to take exception to your opinion that raw message passing is a
bad
idea. I use the resource manager method to establist the initial
connection. After that I use raw message passing. This allows me to
pass a
complicted structure with a command field and other parameters. The
‘resource manager’ can then reply with a similar complex structure as and
when appropriate. I also have a whole library of code to convert a
function
call to a raw send/receive. It is very efficient.

What is wrong with that?

I don’t see the enthusiasm for resource managers. I’ve written one, for
FireWire cameras, but that was because it really was something for which
stream-based read and write made sense.

The enthusiasm here is better organisation of the code, portability of
client side and number of conviniences in terms of infrastructure of the
entire system.

I’ve written quite enough of that stuff to know that making your server a
formal resmgr will force you to do things [more or less] right, whereas
doing it raw will allow to cut too many corners and end up with sloppy code.
I don’t doubt that you could write a good code with raw messages, but
working in a large project you have to think in statistical terms and use
methods that give statistically better results.

In fact I consider even the resmgr-level API of QNX too ‘raw’ for casual
use. There’s way to many chances to miss some angles and then a long painful
process of debugging and fixing ensues. When you’re doing it alone, you
might be fine. When it is done in parallel by half a dozen or more people
(doing different components) you’re much better off with a more high-level
[and yes, less flexible] abstraction to hide some complexities and eliminate
most common logic mistakes. Once you design and debug that layer, you get
much better quality of code from its users.

And given how you have to contemplate moving to another platform now,
portability of client side is not a minor advantage either.

If you want an interprocess subroutine call, MsgSend is the right
mechanism. Putting stream buffering on top of messaging, and then
having to delimit the streams in some way, is complicated and
inefficient. Yes, the sockets-based world does it that way,
and pays a heavy price for it. See any CORBA implementation.

Indeed. But it is up to you whether or not your resmgr will do stream
buffering at all. You can say that your read() or write() will only handle
this much of data at a time and avoid streaming altogether. And ioctl()
normally does not stream at all, althoug AFAIR it can only pass up to 64k of
data at a time.

In my experience we found that even very specialized resmgrs that have no
corresponding hardware and operate exclusively through ioctl() with
predefined structures can benefit by using resmgr approach. First, you get
the benefit of consistent and convinient open(). It can be used by shell
scripts that start your system to check for readiness of the subsystems
(instead of stuffing dumb sleeps after myserver &). Also you can use
read/write as debug/test interfaces (like you do cat /proc/ipstats to see
status of tiny TCP stack or you can echo"xxx" > /dev/myserver to automate
testing, etc). After I have written a high level library to make creation of
that sort of resmgrs real easy, it was used throughout the project very
successfully by number of people and they found the whole scheme very nice.
In fact they loved it - both the developers and users. Elegant and
consistent debug interface into your IPC chains allows better testing and
provides built-in ‘instrumentation’ for gathering stats.

MsgSend is not that slow. See the benchmarks. I run uncompressed video
through MsgSend, and it’s using a few percent of a Pentium III class
machine.
If you’re worried about overhead for bulk transfers, use big messages.
I use one message per frame.
The main problem with the MsgSend world is that the directory services
needed to set up connections are weak. Trying to get a process family to
intercommunicate is a mess. There’s name_attach, but that’s global,
and didn’t work right until QNX 6.3. Inter-machine spawn didn’t
work right in 6.21, either. (Is it fixed in 6.3)?

If you did your resmgr right, you don’t have to pay for the added luxury
either.

– igor