multiple threads versus select()

I have a application that will be reading numerous file handles.
The files can be written to at any time. My first solution to this
problem was to have one thread for every file and have the read
calls blocking so that when data is available the thread would empty
process the file until empty. I am wondering if having 100 threads
is less efficient than using the select() call? The problem with the
select() call
is that I would have to implement some type of time sharing between the
file handles. Has any one done an analysis between multiple threads versus
select for reading file or sockets?

Thanks

mik <mkonstantinides@connecttech.com> wrote:

I have a application that will be reading numerous file handles.
The files can be written to at any time. My first solution to this
problem was to have one thread for every file and have the read
calls blocking so that when data is available the thread would empty
process the file until empty. I am wondering if having 100 threads
is less efficient than using the select() call? The problem with the
select() call
is that I would have to implement some type of time sharing between the
file handles. Has any one done an analysis between multiple threads versus
select for reading file or sockets?

Using multiple threads, you pay your overhead in memory, the stack for
each thread, the data structures to describe the thread, etc. You also
have the issue of being multi-threaded, so have the cost/concern of proper
protection/synchronisation of access to any data structures that more than
one thread will want to modify.

For select(), the cost of a select() on 100 fds is going to be moderately
high in message pass exchanges and resmgr work to do this. So, it will
save on memory at the cost of CPU – but you have the simplicity of a
single-threaded design as well.

Neither answer is, neccessarily, correct.

-David

QNX Training Services
http://www.qnx.com/services/training/
Please followup in this newsgroup if you have further questions.

Very challenging design decision. The critical point is that with a
select() I must implement
some type of scheduling. I would rather not implement such an algorithm for
generic input streams.
It would seem it would be best to implement threads. I will just have to
minimize the stack size.
Just as a side note, this process with multiple read threads will also
contain a dispatch thread from the io-char library,
and yes I plan to call tti() everytime one of these threads wakes up … I
don’t really know if this is the most efficent
way to do it but it is the most obvious way to piece everything together.

Your adivce probably saved me a couple days of experimentation.

mk

“David Gibbs” <dagibbs@qnx.com> wrote in message
news:c5h20k$ng7$1@nntp.qnx.com

mik <> mkonstantinides@connecttech.com> > wrote:
I have a application that will be reading numerous file handles.
The files can be written to at any time. My first solution to this
problem was to have one thread for every file and have the read
calls blocking so that when data is available the thread would empty
process the file until empty. I am wondering if having 100 threads
is less efficient than using the select() call? The problem with the
select() call
is that I would have to implement some type of time sharing between the
file handles. Has any one done an analysis between multiple threads
versus
select for reading file or sockets?

Using multiple threads, you pay your overhead in memory, the stack for
each thread, the data structures to describe the thread, etc. You also
have the issue of being multi-threaded, so have the cost/concern of proper
protection/synchronisation of access to any data structures that more than
one thread will want to modify.

For select(), the cost of a select() on 100 fds is going to be moderately
high in message pass exchanges and resmgr work to do this. So, it will
save on memory at the cost of CPU – but you have the simplicity of a
single-threaded design as well.

Neither answer is, neccessarily, correct.

-David

QNX Training Services
http://www.qnx.com/services/training/
Please followup in this newsgroup if you have further questions.

mik <mkonstantinides@connecttech.com> wrote:

Very challenging design decision. The critical point is that with a
select() I must implement
some type of scheduling. I would rather not implement such an algorithm for
generic input streams.
It would seem it would be best to implement threads. I will just have to
minimize the stack size.

Stack is demand-paged, in 4K increments as needed.


Your adivce probably saved me a couple days of experimentation.

You’re quite welcome.

-David

QNX Training Services
http://www.qnx.com/services/training/
Please followup in this newsgroup if you have further questions.

I totally agree with Dave …
Just one small addiditon when you use select() it’s much easier to debug …

BTW, i wonder if asynchonous io has been implemented in QNX 6.3

cheers,
Igor

“mik” <mkonstantinides@connecttech.com> wrote in message
news:c5hhbb$9j7$1@inn.qnx.com

Very challenging design decision. The critical point is that with a
select() I must implement
some type of scheduling. I would rather not implement such an algorithm
for
generic input streams.
It would seem it would be best to implement threads. I will just have to
minimize the stack size.
Just as a side note, this process with multiple read threads will also
contain a dispatch thread from the io-char library,
and yes I plan to call tti() everytime one of these threads wakes up … I
don’t really know if this is the most efficent
way to do it but it is the most obvious way to piece everything together.

Your adivce probably saved me a couple days of experimentation.

mk

“David Gibbs” <> dagibbs@qnx.com> > wrote in message
news:c5h20k$ng7$> 1@nntp.qnx.com> …
mik <> mkonstantinides@connecttech.com> > wrote:
I have a application that will be reading numerous file handles.
The files can be written to at any time. My first solution to this
problem was to have one thread for every file and have the read
calls blocking so that when data is available the thread would empty
process the file until empty. I am wondering if having 100 threads
is less efficient than using the select() call? The problem with the
select() call
is that I would have to implement some type of time sharing between
the
file handles. Has any one done an analysis between multiple threads
versus
select for reading file or sockets?

Using multiple threads, you pay your overhead in memory, the stack for
each thread, the data structures to describe the thread, etc. You also
have the issue of being multi-threaded, so have the cost/concern of
proper
protection/synchronisation of access to any data structures that more
than
one thread will want to modify.

For select(), the cost of a select() on 100 fds is going to be
moderately
high in message pass exchanges and resmgr work to do this. So, it will
save on memory at the cost of CPU – but you have the simplicity of a
single-threaded design as well.

Neither answer is, neccessarily, correct.

-David

QNX Training Services
http://www.qnx.com/services/training/
Please followup in this newsgroup if you have further questions.

I have yet to compile anything under QNX 6.3. That will happen sometime
later.

“Igor Levko” <spama@HuxpeHa.HeT> wrote in message
news:c5jl8c$5qa$1@inn.qnx.com

I totally agree with Dave …
Just one small addiditon when you use select() it’s much easier to debug

BTW, i wonder if asynchonous io has been implemented in QNX 6.3

cheers,
Igor

“mik” <> mkonstantinides@connecttech.com> > wrote in message
news:c5hhbb$9j7$> 1@inn.qnx.com> …
Very challenging design decision. The critical point is that with a
select() I must implement
some type of scheduling. I would rather not implement such an algorithm
for
generic input streams.
It would seem it would be best to implement threads. I will just have
to
minimize the stack size.
Just as a side note, this process with multiple read threads will also
contain a dispatch thread from the io-char library,
and yes I plan to call tti() everytime one of these threads wakes up …
I
don’t really know if this is the most efficent
way to do it but it is the most obvious way to piece everything
together.

Your adivce probably saved me a couple days of experimentation.

mk

“David Gibbs” <> dagibbs@qnx.com> > wrote in message
news:c5h20k$ng7$> 1@nntp.qnx.com> …
mik <> mkonstantinides@connecttech.com> > wrote:
I have a application that will be reading numerous file handles.
The files can be written to at any time. My first solution to this
problem was to have one thread for every file and have the read
calls blocking so that when data is available the thread would empty
process the file until empty. I am wondering if having 100 threads
is less efficient than using the select() call? The problem with
the
select() call
is that I would have to implement some type of time sharing between
the
file handles. Has any one done an analysis between multiple threads
versus
select for reading file or sockets?

Using multiple threads, you pay your overhead in memory, the stack for
each thread, the data structures to describe the thread, etc. You
also
have the issue of being multi-threaded, so have the cost/concern of
proper
protection/synchronisation of access to any data structures that more
than
one thread will want to modify.

For select(), the cost of a select() on 100 fds is going to be
moderately
high in message pass exchanges and resmgr work to do this. So, it
will
save on memory at the cost of CPU – but you have the simplicity of a
single-threaded design as well.

Neither answer is, neccessarily, correct.

-David

QNX Training Services
http://www.qnx.com/services/training/
Please followup in this newsgroup if you have further questions.
\

I have just realized what you mean by asynchonous io in QNX.
Do you mean that I would be able to recieve pulses from the process
responsible for network socket io indicating that data is available for a
particular socket?
This would be great and would remove the need to have my code create 2N
threads for reading data from
numerous sockets.

I would have assume the select() statment would do this, but if it did I
guess I would have expected
to register sockets and then recieve a pulse in my main message loop.
It might be worth while to check and see if this feature is available in QNX
6.3 beta.

mk

“Igor Levko” <spama@HuxpeHa.HeT> wrote in message
news:c5jl8c$5qa$1@inn.qnx.com

I totally agree with Dave …
Just one small addiditon when you use select() it’s much easier to debug

BTW, i wonder if asynchonous io has been implemented in QNX 6.3

cheers,
Igor

“mik” <> mkonstantinides@connecttech.com> > wrote in message
news:c5hhbb$9j7$> 1@inn.qnx.com> …
Very challenging design decision. The critical point is that with a
select() I must implement
some type of scheduling. I would rather not implement such an algorithm
for
generic input streams.
It would seem it would be best to implement threads. I will just have
to
minimize the stack size.
Just as a side note, this process with multiple read threads will also
contain a dispatch thread from the io-char library,
and yes I plan to call tti() everytime one of these threads wakes up …
I
don’t really know if this is the most efficent
way to do it but it is the most obvious way to piece everything
together.

Your adivce probably saved me a couple days of experimentation.

mk

“David Gibbs” <> dagibbs@qnx.com> > wrote in message
news:c5h20k$ng7$> 1@nntp.qnx.com> …
mik <> mkonstantinides@connecttech.com> > wrote:
I have a application that will be reading numerous file handles.
The files can be written to at any time. My first solution to this
problem was to have one thread for every file and have the read
calls blocking so that when data is available the thread would empty
process the file until empty. I am wondering if having 100 threads
is less efficient than using the select() call? The problem with
the
select() call
is that I would have to implement some type of time sharing between
the
file handles. Has any one done an analysis between multiple threads
versus
select for reading file or sockets?

Using multiple threads, you pay your overhead in memory, the stack for
each thread, the data structures to describe the thread, etc. You
also
have the issue of being multi-threaded, so have the cost/concern of
proper
protection/synchronisation of access to any data structures that more
than
one thread will want to modify.

For select(), the cost of a select() on 100 fds is going to be
moderately
high in message pass exchanges and resmgr work to do this. So, it
will
save on memory at the cost of CPU – but you have the simplicity of a
single-threaded design as well.

Neither answer is, neccessarily, correct.

-David

QNX Training Services
http://www.qnx.com/services/training/
Please followup in this newsgroup if you have further questions.
\

There is no async I/O in 6.3. POSIX definition of async read implies that a
caller passes an address that is later on used to write data into… This is
very messy when I/O is outside of the kernel… not saying that it can’t be
done, but I have doubts they will bother.

OTOH, 6.3 has some half-cooked implementation of poll() which is in some
ways nicer to use than select().
Still, using either select() or poll() does not replace async I/O. You can’t
wait for all common event types in one blocking entry point.

– igor

“mik” <mkonstantinides@connecttech.com> wrote in message
news:c5muma$1g7$1@inn.qnx.com

I have just realized what you mean by asynchonous io in QNX.
Do you mean that I would be able to recieve pulses from the process
responsible for network socket io indicating that data is available for a
particular socket?
This would be great and would remove the need to have my code create 2N
threads for reading data from
numerous sockets.

I would have assume the select() statment would do this, but if it did I
guess I would have expected
to register sockets and then recieve a pulse in my main message loop.
It might be worth while to check and see if this feature is available in
QNX
6.3 beta.

mk

“Igor Levko” <> spama@HuxpeHa.HeT> > wrote in message
news:c5jl8c$5qa$> 1@inn.qnx.com> …
I totally agree with Dave …
Just one small addiditon when you use select() it’s much easier to debug

BTW, i wonder if asynchonous io has been implemented in QNX 6.3

cheers,
Igor

“mik” <> mkonstantinides@connecttech.com> > wrote in message
news:c5hhbb$9j7$> 1@inn.qnx.com> …
Very challenging design decision. The critical point is that with a
select() I must implement
some type of scheduling. I would rather not implement such an
algorithm
for
generic input streams.
It would seem it would be best to implement threads. I will just have
to
minimize the stack size.
Just as a side note, this process with multiple read threads will also
contain a dispatch thread from the io-char library,
and yes I plan to call tti() everytime one of these threads wakes up

I
don’t really know if this is the most efficent
way to do it but it is the most obvious way to piece everything
together.

Your adivce probably saved me a couple days of experimentation.

mk

“David Gibbs” <> dagibbs@qnx.com> > wrote in message
news:c5h20k$ng7$> 1@nntp.qnx.com> …
mik <> mkonstantinides@connecttech.com> > wrote:
I have a application that will be reading numerous file handles.
The files can be written to at any time. My first solution to
this
problem was to have one thread for every file and have the read
calls blocking so that when data is available the thread would
empty
process the file until empty. I am wondering if having 100
threads
is less efficient than using the select() call? The problem with
the
select() call
is that I would have to implement some type of time sharing
between
the
file handles. Has any one done an analysis between multiple
threads
versus
select for reading file or sockets?

Using multiple threads, you pay your overhead in memory, the stack
for
each thread, the data structures to describe the thread, etc. You
also
have the issue of being multi-threaded, so have the cost/concern of
proper
protection/synchronisation of access to any data structures that
more
than
one thread will want to modify.

For select(), the cost of a select() on 100 fds is going to be
moderately
high in message pass exchanges and resmgr work to do this. So, it
will
save on memory at the cost of CPU – but you have the simplicity of
a
single-threaded design as well.

Neither answer is, neccessarily, correct.

-David

QNX Training Services
http://www.qnx.com/services/training/
Please followup in this newsgroup if you have further questions.


\

Igor Kovalenko <kovalenko@attbi.com> wrote:

OTOH, 6.3 has some half-cooked implementation of poll() which is in some
ways nicer to use than select().

If you’re finding issues with poll(), please post them
to the beta conference.

Thanks,

-seanb

Have you looked at the ionotify() function? It does what you describe below
(io-net can send you a sigevent struct when data is available to read (and
much more for that matter)).

“mik” <mkonstantinides@connecttech.com> wrote in message
news:c5muma$1g7$1@inn.qnx.com

I have just realized what you mean by asynchonous io in QNX.
Do you mean that I would be able to recieve pulses from the process
responsible for network socket io indicating that data is available for a
particular socket?
This would be great and would remove the need to have my code create 2N
threads for reading data from
numerous sockets.

I would have assume the select() statment would do this, but if it did I
guess I would have expected
to register sockets and then recieve a pulse in my main message loop.
It might be worth while to check and see if this feature is available in
QNX
6.3 beta.

mk

“Igor Levko” <> spama@HuxpeHa.HeT> > wrote in message
news:c5jl8c$5qa$> 1@inn.qnx.com> …
I totally agree with Dave …
Just one small addiditon when you use select() it’s much easier to debug

BTW, i wonder if asynchonous io has been implemented in QNX 6.3

cheers,
Igor

“mik” <> mkonstantinides@connecttech.com> > wrote in message
news:c5hhbb$9j7$> 1@inn.qnx.com> …
Very challenging design decision. The critical point is that with a
select() I must implement
some type of scheduling. I would rather not implement such an
algorithm
for
generic input streams.
It would seem it would be best to implement threads. I will just have
to
minimize the stack size.
Just as a side note, this process with multiple read threads will also
contain a dispatch thread from the io-char library,
and yes I plan to call tti() everytime one of these threads wakes up

I
don’t really know if this is the most efficent
way to do it but it is the most obvious way to piece everything
together.

Your adivce probably saved me a couple days of experimentation.

mk

“David Gibbs” <> dagibbs@qnx.com> > wrote in message
news:c5h20k$ng7$> 1@nntp.qnx.com> …
mik <> mkonstantinides@connecttech.com> > wrote:
I have a application that will be reading numerous file handles.
The files can be written to at any time. My first solution to
this
problem was to have one thread for every file and have the read
calls blocking so that when data is available the thread would
empty
process the file until empty. I am wondering if having 100
threads
is less efficient than using the select() call? The problem with
the
select() call
is that I would have to implement some type of time sharing
between
the
file handles. Has any one done an analysis between multiple
threads
versus
select for reading file or sockets?

Using multiple threads, you pay your overhead in memory, the stack
for
each thread, the data structures to describe the thread, etc. You
also
have the issue of being multi-threaded, so have the cost/concern of
proper
protection/synchronisation of access to any data structures that
more
than
one thread will want to modify.

For select(), the cost of a select() on 100 fds is going to be
moderately
high in message pass exchanges and resmgr work to do this. So, it
will
save on memory at the cost of CPU – but you have the simplicity of
a
single-threaded design as well.

Neither answer is, neccessarily, correct.

-David

QNX Training Services
http://www.qnx.com/services/training/
Please followup in this newsgroup if you have further questions.


\

I only meant to say it is incomplete.

“Sean Boudreau” <seanb@qnx.com> wrote in message
news:c5p3db$qnt$1@inn.qnx.com

Igor Kovalenko <> kovalenko@attbi.com> > wrote:

OTOH, 6.3 has some half-cooked implementation of poll() which is in some
ways nicer to use than select().

If you’re finding issues with poll(), please post them
to the beta conference.

Thanks,

-seanb

poll() is complete. The docs haven’t been updated for beta
two but the content has been generated. Various managers
may have to start triggering the extended events. Please
post the ones you’re aware of to the beta conference.

-seanb


Igor Kovalenko <kovalenko@attbi.com> wrote:

I only meant to say it is incomplete.

“Sean Boudreau” <> seanb@qnx.com> > wrote in message
news:c5p3db$qnt$> 1@inn.qnx.com> …
Igor Kovalenko <> kovalenko@attbi.com> > wrote:

OTOH, 6.3 has some half-cooked implementation of poll() which is in some
ways nicer to use than select().

If you’re finding issues with poll(), please post them
to the beta conference.

Thanks,

-seanb

Ok Sean. Sorry I said that.

“Sean Boudreau” <seanb@qnx.com> wrote in message
news:c5tpnb$meg$1@inn.qnx.com

poll() is complete. The docs haven’t been updated for beta
two but the content has been generated. Various managers
may have to start triggering the extended events. Please
post the ones you’re aware of to the beta conference.

-seanb


Igor Kovalenko <> kovalenko@attbi.com> > wrote:
I only meant to say it is incomplete.

“Sean Boudreau” <> seanb@qnx.com> > wrote in message
news:c5p3db$qnt$> 1@inn.qnx.com> …
Igor Kovalenko <> kovalenko@attbi.com> > wrote:

OTOH, 6.3 has some half-cooked implementation of poll() which is in
some
ways nicer to use than select().

If you’re finding issues with poll(), please post them
to the beta conference.

Thanks,

-seanb

I have not looked at ionotify() until you mentioned it. According to the
resource manager documenation is similar to the select() …
so I assume that the cost will be the same “moderately high in message pass
exchanges and resmgr work to do this”?
If this is the case, I would rather consume more memory than CPU time.

mk

“Brian Meinke” <RoverFan_SE7@Hotmail.com> wrote in message
news:c5pvbe$iaq$1@inn.qnx.com

Have you looked at the ionotify() function? It does what you describe
below
(io-net can send you a sigevent struct when data is available to read (and
much more for that matter)).

“mik” <> mkonstantinides@connecttech.com> > wrote in message
news:c5muma$1g7$> 1@inn.qnx.com> …
I have just realized what you mean by asynchonous io in QNX.
Do you mean that I would be able to recieve pulses from the process
responsible for network socket io indicating that data is available for
a
particular socket?
This would be great and would remove the need to have my code create 2N
threads for reading data from
numerous sockets.

I would have assume the select() statment would do this, but if it did I
guess I would have expected
to register sockets and then recieve a pulse in my main message loop.
It might be worth while to check and see if this feature is available in
QNX
6.3 beta.

mk

“Igor Levko” <> spama@HuxpeHa.HeT> > wrote in message
news:c5jl8c$5qa$> 1@inn.qnx.com> …
I totally agree with Dave …
Just one small addiditon when you use select() it’s much easier to
debug

BTW, i wonder if asynchonous io has been implemented in QNX 6.3

cheers,
Igor

“mik” <> mkonstantinides@connecttech.com> > wrote in message
news:c5hhbb$9j7$> 1@inn.qnx.com> …
Very challenging design decision. The critical point is that with a
select() I must implement
some type of scheduling. I would rather not implement such an
algorithm
for
generic input streams.
It would seem it would be best to implement threads. I will just
have
to
minimize the stack size.
Just as a side note, this process with multiple read threads will
also
contain a dispatch thread from the io-char library,
and yes I plan to call tti() everytime one of these threads wakes up

I
don’t really know if this is the most efficent
way to do it but it is the most obvious way to piece everything
together.

Your adivce probably saved me a couple days of experimentation.

mk

“David Gibbs” <> dagibbs@qnx.com> > wrote in message
news:c5h20k$ng7$> 1@nntp.qnx.com> …
mik <> mkonstantinides@connecttech.com> > wrote:
I have a application that will be reading numerous file handles.
The files can be written to at any time. My first solution to
this
problem was to have one thread for every file and have the read
calls blocking so that when data is available the thread would
empty
process the file until empty. I am wondering if having 100
threads
is less efficient than using the select() call? The problem
with
the
select() call
is that I would have to implement some type of time sharing
between
the
file handles. Has any one done an analysis between multiple
threads
versus
select for reading file or sockets?

Using multiple threads, you pay your overhead in memory, the stack
for
each thread, the data structures to describe the thread, etc. You
also
have the issue of being multi-threaded, so have the cost/concern
of
proper
protection/synchronisation of access to any data structures that
more
than
one thread will want to modify.

For select(), the cost of a select() on 100 fds is going to be
moderately
high in message pass exchanges and resmgr work to do this. So, it
will
save on memory at the cost of CPU – but you have the simplicity
of
a
single-threaded design as well.

Neither answer is, neccessarily, correct.

-David

QNX Training Services
http://www.qnx.com/services/training/
Please followup in this newsgroup if you have further questions.




\

The cost is not necessarily the same, when considered in the large.

Since select() is multi-plexing, the round of message exchanges may
have to be duplicated multiple time (to resource managers that are
irrelevant to the I/O condition).

In many instances, it can be more efficient to have specific threads
to handle I/O from particular file descriptors, including for example
a single thread on each accept() call, or a thread waiting for data
to become available. In such cases, there is no querying resource
managers for all the extraneous file descriptors.

With many file descriptors, you can have better scalability at the expense
of
some memory and thread slots.

The real issue is that select(), poll(), etc. pre-date the common
availability of threads, so they provide a mechanism that is not
strictly necessary in a threaded environment. Now you have more
choices, with more trade-offs.

“mik” <mkonstantinides@connecttech.com> wrote in message
news:c60pni$9jt$1@inn.qnx.com

I have not looked at ionotify() until you mentioned it. According to the
resource manager documenation is similar to the select() …
so I assume that the cost will be the same “moderately high in message
pass
exchanges and resmgr work to do this”?
If this is the case, I would rather consume more memory than CPU time.

mk

“Brian Meinke” <> RoverFan_SE7@Hotmail.com> > wrote in message
news:c5pvbe$iaq$> 1@inn.qnx.com> …

Have you looked at the ionotify() function? It does what you describe
below
(io-net can send you a sigevent struct when data is available to read
(and
much more for that matter)).

“mik” <> mkonstantinides@connecttech.com> > wrote in message
news:c5muma$1g7$> 1@inn.qnx.com> …
I have just realized what you mean by asynchonous io in QNX.
Do you mean that I would be able to recieve pulses from the process
responsible for network socket io indicating that data is available
for
a
particular socket?
This would be great and would remove the need to have my code create
2N
threads for reading data from
numerous sockets.

I would have assume the select() statment would do this, but if it did
I
guess I would have expected
to register sockets and then recieve a pulse in my main message loop.
It might be worth while to check and see if this feature is available
in
QNX
6.3 beta.

mk

“Igor Levko” <> spama@HuxpeHa.HeT> > wrote in message
news:c5jl8c$5qa$> 1@inn.qnx.com> …
I totally agree with Dave …
Just one small addiditon when you use select() it’s much easier to
debug

BTW, i wonder if asynchonous io has been implemented in QNX 6.3

cheers,
Igor

“mik” <> mkonstantinides@connecttech.com> > wrote in message
news:c5hhbb$9j7$> 1@inn.qnx.com> …
Very challenging design decision. The critical point is that with
a
select() I must implement
some type of scheduling. I would rather not implement such an
algorithm
for
generic input streams.
It would seem it would be best to implement threads. I will just
have
to
minimize the stack size.
Just as a side note, this process with multiple read threads will
also
contain a dispatch thread from the io-char library,
and yes I plan to call tti() everytime one of these threads wakes
up

I
don’t really know if this is the most efficent
way to do it but it is the most obvious way to piece everything
together.

Your adivce probably saved me a couple days of experimentation.

mk

“David Gibbs” <> dagibbs@qnx.com> > wrote in message
news:c5h20k$ng7$> 1@nntp.qnx.com> …
mik <> mkonstantinides@connecttech.com> > wrote:
I have a application that will be reading numerous file
handles.
The files can be written to at any time. My first solution to
this
problem was to have one thread for every file and have the
read
calls blocking so that when data is available the thread would
empty
process the file until empty. I am wondering if having 100
threads
is less efficient than using the select() call? The problem
with
the
select() call
is that I would have to implement some type of time sharing
between
the
file handles. Has any one done an analysis between multiple
threads
versus
select for reading file or sockets?

Using multiple threads, you pay your overhead in memory, the
stack
for
each thread, the data structures to describe the thread, etc.
You
also
have the issue of being multi-threaded, so have the cost/concern
of
proper
protection/synchronisation of access to any data structures that
more
than
one thread will want to modify.

For select(), the cost of a select() on 100 fds is going to be
moderately
high in message pass exchanges and resmgr work to do this. So,
it
will
save on memory at the cost of CPU – but you have the simplicity
of
a
single-threaded design as well.

Neither answer is, neccessarily, correct.

-David

QNX Training Services
http://www.qnx.com/services/training/
Please followup in this newsgroup if you have further questions.






\

mik <mkonstantinides@connecttech.com> wrote:

I have not looked at ionotify() until you mentioned it. According to the
resource manager documenation is similar to the select() …
so I assume that the cost will be the same “moderately high in message pass
exchanges and resmgr work to do this”?

Not nearly as bad as select. select() is implemented on top of ionotify().
It ends up doing lots of ionotify() calls for every select() call. If
you are just using ionotify() with pulses to a MsgReceive() you are going
to get far less messaging overhead.

select() basically calls ionotify() twice for every fd every time you call
select(). If you are using ionotify() directly, you can call it once
for each fd, then usually need another call for each fd as that fd
activates – but not 2xfds calls for every time any fd goes active. So,
using ionotify() directly is far less costly than using select(),
especially for largish numbers of fds.

-David

QNX Training Services
http://www.qnx.com/services/training/
Please followup in this newsgroup if you have further questions.

David Gibbs <dagibbs@qnx.com> wrote:

mik <> mkonstantinides@connecttech.com> > wrote:

select() basically calls ionotify() twice for every fd every time you call
select().

And, this behaviour has been improved for 6.3.0 – select() now tries to
send one message per manager, rather than per fd, to a server to make things
more efficient.

-David

QNX Training Services
http://www.qnx.com/services/training/
Please followup in this newsgroup if you have further questions.