Correct way to exit/cleanup in multi-threaded resource manag

Upon consideration of a thread in qdn.public.qnxrtp.devtools in which John
Garvey makes the following “throw away” comment:

“I have successfully profiled multi-threaded resource managers (but with the
initial thread waiting in SignalWaitinfo() and then explicitly calling
exit() if that is indeed a factor) …”

I have a question about the correct way to do clean-up processing in a
multi-threaded resource manager.

The function thread_pool_start() normally either never returns, or returns
immediately (depending on flags). So based on John’s comment, is the
following the correct psuedo-code for clean-up processing?

tpp = thread_pool_create( &attr, 0 ); // 0 means return after pool created

thread_pool_start( tpp );
SignalWaitinfo( SIGQUIT etc );
my_cleanup_code();
exit();

An example would be very helpful! Neither the online docs nor Rob Krten’s
book seem to cover this in any detail.

Rob Rutherford
Ruzz Technology

John Garvey sent this to me privately as (for some mysterious reason) he was
unable to post it to the NG. Since it is so relevant and useful I thought I
should share it…

Robert Rutherford <ruzz@nospamplease.ruzz.com> wrote:

Upon consideration of a thread in qdn.public.qnxrtp.devtools in which

John Garvey makes the following “throw away” comment:

I obviously didn’t throw it away far enough :slight_smile: Comments inline …

tpp = thread_pool_create( &attr, 0 );

Here you’d want to block all signals in the pool threads so they don’t
inadvertently get targetted; for example (since you haven’t set
POOL_FLAG_USE_SELF the first pool thread will inherit this mask which will
then also pass it on to all other pool threads):

sigfillset(&signals), pthread_sigmask(SIG_BLOCK, &signals, &old);

thread_pool_start(pool);

pthread_sigmask(SIG_SETMASK, &old, NULL);

thread_pool_start( tpp );

SignalWaitinfo( SIGQUIT etc );

I presume you really meant SIGTERM here, a more appropriate signal? After
this point you may want to exercise a little care, as your pool threads may
be actively working on something. I don’t use thread cancellations (messy
with pushing cleanup handlers and keeping track of what are cancellation
points and so on) so let the threads finish their current work and have the
pool de-populate itself with:

thread_pool_limits(pool, 0, 0, 0, 1, 0);

Note that you’ll have to set up the pool ‘unblock_func’ to rip out
RECV-blocked threads (there is either the libc _message_unblock() or I
always just supply my own that sends an invalid pulse code (-1) to my known
resmgr coid).

Now you want to take care of any open client connections, so you can clean
up any resources associated with those, so for each attached pathname you do
this (which will result in the close_ocb() callout being made to you as if
every client had close()d):

resmgr_detach(dispatch, pathid, _RESMGR_DETACH_CLOSE);

my_cleanup_code();

Your cleanup code now nows there are no other pool threads running around,
there are no incomplete/partial/active operations so you are safe with any
global/shared data structures, and all clients have been cleaned-up after as
if they nicely close()d themselves. So there’s not too much for this code to
do (release any one-off global resource allocations your server made).

So, there you go, that’s pretty much the thread pool cleanup handling of
QNX6 io-blk.so …



“Robert Rutherford” <ruzz@NoSpamPlease.ruzz.com> wrote in message
news:b2h9kg$dj7$1@inn.qnx.com

Upon consideration of a thread in qdn.public.qnxrtp.devtools in which John
Garvey makes the following “throw away” comment:

“I have successfully profiled multi-threaded resource managers (but with
the
initial thread waiting in SignalWaitinfo() and then explicitly calling
exit() if that is indeed a factor) …”

I have a question about the correct way to do clean-up processing in a
multi-threaded resource manager.

The function thread_pool_start() normally either never returns, or returns
immediately (depending on flags). So based on John’s comment, is the
following the correct psuedo-code for clean-up processing?

tpp = thread_pool_create( &attr, 0 ); // 0 means return after pool created

thread_pool_start( tpp );
SignalWaitinfo( SIGQUIT etc );
my_cleanup_code();
exit();

An example would be very helpful! Neither the online docs nor Rob Krten’s
book seem to cover this in any detail.

Rob Rutherford
Ruzz Technology

Interesting,

-cut-

I presume you really meant SIGTERM here, a more appropriate signal? After
this point you may want to exercise a little care, as your pool threads may
be actively working on something. I don’t use thread cancellations (messy
with pushing cleanup handlers and keeping track of what are cancellation
points and so on) so let the threads finish their current work and have the
pool de-populate itself with:

thread_pool_limits(pool, 0, 0, 0, 1, 0);


Do we have explicitly to wait for the threads having finished their work

BEFORE calling this function, or pthread_pool_limits() will just prevent
to create new threads, waiting actual running threads to stop before
killing them?

-cut-

Alain.

Alain Bonnefoy <alain.bonnefoy@icbt.com> wrote:

this point you may want to exercise a little care, as your pool threads may
be actively working on something. I don’t use thread cancellations (messy
with pushing cleanup handlers and keeping track of what are cancellation
points and so on) so let the threads finish their current work and have the
pool de-populate itself with:
thread_pool_limits(pool, 0, 0, 0, 1, 0);
Do we have explicitly to wait for the threads having finished their work
BEFORE calling this function, or pthread_pool_limits() will just prevent
to create new threads, waiting actual running threads to stop before
killing them?

It will all be done by this function, which you can call at any time; it
will adjust the thread pool to the limits specified (in my example
lo == hi == max == 0, which prevents any new threads being created); it
will also terminate any superfluous threads as and when they return from
the dispatch/resmgr callout they are currently handling. When the pool
lies within the limits specified, then (and only then, because I didn’t
specify the THREAD_POOL_CONTROL_NONBLOCK flag) this function returns.