John A. Murphy <murf@perftech.com> wrote:
We’ve tried to build a server using a thread_pool. We open a socket, do
a select_attach() and a listen(), and start a thread_pool to handle
connections, expecting each incoming connection to be handed off to a
separate thread. But we have observed that the connections get handled
strictly one at a time, as if the select_attach() gets disabled when a
connection comes in and is given to a thread, and doesn’t get enabled
again until that thread finishes its work. Anybody got any idea where
we went wrong?
I know the resmgr library does locking at the attribute level for
devices that have been attached.
I speculate that it is also doing locking at the (internal attribute) to
represent (fd selected on), and that it locks before calling out into
your code, so that it doesn’t end up dispatching multiple threads to
handle the same fd. In fact, I suspect it doesn’t “re-select” on it
until after your handler function returns (because it can’t know when
you’ve actually processed your event – so if it re-selects before
you’ve read/accepted/whatever, it may just keep dispatching threads
for the still active select() condition). I also speculate that you
are accept()ing the connection, and then continuing to handle that
connection in that same callout. So, of course, the library is not
re-selecting on it.
If it doesn’t do this, there are (as I kind of noted) some real
synchronisation problems. If you check the docs for select() you
will see that it is not safe for use in a multi-threaded process,
at least partially due to these dispatch issues. (But, also due
to the way the notification is delivered for select, select_attach()
works around this.)
So, do you need to be a resmgr for other reasons? Or were you doing
so just to get the thread_pool()? Can you work with your own pool
of threads, and replace your use of select_attach() (or select())
with ionotify()?
Or, within your accept() code – can you instead of expecting the
accept()ing thread to completely handle that fd/connection, can
you select_attach() the new fd?
-David
QNX Training Services
http://www.qnx.com/support/training/
Please followup in this newsgroup if you have further questions.