Notify trigger all devices of a resource manager

QNX Newsgroups (Archives) qdn.public.qnxrtp.os
1

I wrote a resource manager to control a particular communication device. It
is capable of one write per millisecond. When a write occurs, all further
writes are blocked for one millisecond. After that millisecond, writes are
possible again.

There are multiple nodes in the file hierarchy for the various endpoints of
this device, although they all control the same actual device. Each file has
its own iofunc_notify_t set.

When the millisecond completes and writes are possible again, is there a
quick way to iofunc_notify_trigger() every endpoint quickly? Perhaps they
could somehow share the same write iofunc_notify_t. They still need separate
notify structs for read though. The only alternative I can come up with is
to for(i=0; i<N; i++) iofunc_notify_trigger( …); which doesn’t appeal to
me, especially since N can be large, and likely at most one file might have
a notify waiting on it (although this isn’t a limit).

Thanks,
Shaun

2

Shaun Jackman <sjackman@nospam.vortek.com> wrote:

I wrote a resource manager to control a particular communication device. It
is capable of one write per millisecond. When a write occurs, all further
writes are blocked for one millisecond. After that millisecond, writes are
possible again.

There are multiple nodes in the file hierarchy for the various endpoints of
this device, although they all control the same actual device. Each file has
its own iofunc_notify_t set.

When the millisecond completes and writes are possible again, is there a
quick way to iofunc_notify_trigger() every endpoint quickly? Perhaps they
could somehow share the same write iofunc_notify_t. They still need separate
notify structs for read though. The only alternative I can come up with is
to for(i=0; i<N; i++) iofunc_notify_trigger( …); which doesn’t appeal to
me, especially since N can be large, and likely at most one file might have
a notify waiting on it (although this isn’t a limit).

Some thoughts.

I’m assuming all these end-points are handled by the same resource
manager.

Is your resource manager multi-threaded?

Do you really need to notify your clients that the 1ms is up?

That is, what should happen if:

client1 writes to device A
less than 1ms passes
client 2 writes to device B (or could be A, doesn’t matter, right?)

What does your resource manager do to client 2’s request?
– do you even see it? If you are single-threaded, do you spend the
full 1msec handling the write in the io_write() handler? If not,
you’ll never see it until you’re done client 1’s request, and then
you’ll be safe to handle client 2’s request
– do you block client 2 and make it wait until client 1’s request is
done? (has client 2 open O_NONBLOCK ? Is that even meaningful?)
– this one is pretty easy to handle
– condvars might be useful for saying you can process the next client
internally
– do you fail the client? What errno? EAGAIN?
– can the client KNOW that it is a 1ms delay, and just try again itself
without needing to be signaled from you?
– could it be a variable delay?

If you have multiple clients to signal, each is going to get it nearly
simultaneously, most will fail…do they have to re-activate their
notification each time? Is this more overhead than you want every 1ms?


The iofunc_notify_* helper functions are available in source from
cvs.qnx.com – they require an array of iofunc_notify_t structures,
so you can’t share – but you may find that you can make your own
special case set based on that code that does what you want.

-David

QNX Training Services
http://www.qnx.com/support/training/
Please followup in this newsgroup if you have further questions.

3

Some thoughts.

I’m assuming all these end-points are handled by the same resource
manager.

The one resource manager handles all the end-points.

Is your resource manager multi-threaded?

The resource manager is multithreaded.

Do you really need to notify your clients that the 1ms is up?

I need to notify the client app when it can send again (after the 1ms is
up). At this point it needs to assemble the data and write() it. I can’t
call the write() before and have it block, because by the time the write
executes, the data may be stale (real-time app).

That is, what should happen if:

client1 writes to device A
less than 1ms passes
client 2 writes to device B (or could be A, doesn’t matter, right?)

What does your resource manager do to client 2’s request?
– do you even see it? If you are single-threaded, do you spend the
full 1msec handling the write in the io_write() handler? If not,
you’ll never see it until you’re done client 1’s request, and then
you’ll be safe to handle client 2’s request

It is multithreaded, so I will see both requests.

– do you block client 2 and make it wait until client 1’s request is
done? (has client 2 open O_NONBLOCK ? Is that even meaningful?)
– this one is pretty easy to handle
– condvars might be useful for saying you can process the next
client
internally

Currently blocked writes are not implemented as there’s no use for a blocked
write message. By the time the blocked write could send, the data that was
blocked is probably stale.

– do you fail the client? What errno? EAGAIN?

I fail the client with EWOULDBLOCK (EAGAIN). My resource manager assumes
O_NONBLOCK was specified for all writable files.

– can the client KNOW that it is a 1ms delay, and just try again
itself
without needing to be signaled from you?

That’s sort of the system I’m using right now. I’m using an ad-hoc way of
waiting for the next system clock tick, at which point I know I can send
again. (the transmit is synced to the system clock)

– could it be a variable delay?

The delay is always 1ms.

If you have multiple clients to signal, each is going to get it nearly
simultaneously, most will fail…do they have to re-activate their
notification each time? Is this more overhead than you want every 1ms?

There’s likely only one client waiting, and if there is two only one will
get to write this time (the other will get it next). This case won’t occur
often enough that I’m worried about the overhead involved in re-activating
the notification. There is an iofunc_notify_t structure associated with
every file (managed by this one resource manager) though, and I want to
avoid calling iofunc_notify_trigger() on each one. I might work around this
by maintaining a list of each iofunc_notify_t structures that has a client
waiting on it, and iterating through that list when the timer is up.

The iofunc_notify_* helper functions are available in source from
cvs.qnx.com – they require an array of iofunc_notify_t structures,
so you can’t share – but you may find that you can make your own
special case set based on that code that does what you want.

I’ll look into this option too.

Thanks for your help,
Shaun

4

Shaun Jackman <sjackman@nospam.vortek.com> wrote:

There are multiple nodes in the file hierarchy for the various endpoints of
this device, although they all control the same actual device. Each file has
its own iofunc_notify_t set.
When the millisecond completes and writes are possible again, is there a
quick way to iofunc_notify_trigger() every endpoint quickly? Perhaps they
could somehow share the same write iofunc_notify_t. They still need separate
notify structs for read though. The only alternative I can come up with is
to for(i=0; i<N; i++) iofunc_notify_trigger( …); which doesn’t appeal to
me, especially since N can be large, and likely at most one file might have
a notify waiting on it (although this isn’t a limit).

Since each client has its own (potentially different mode of) notification
event, which must be delivered on the unique rcvid to that client, there
is no shortcut to triggering each write notification individually. And
since the standard iofunc_notify*() routines all expect an [3], you can’t
share part of them. So I’m assuming you worried about looping through
many empty list heads? But since an iofunc_attr_t itself does not contain
any iofunc_notify_t structures, you’ve had to extend it yourself to include
one, so you could also add a linked list of active/waiting writers or
files which are being waited on or so on, which you could manipulate
yourself in parallel to the standard array, and then just traverse this
shorter list on your 1ms ticks … ?

5

Shaun Jackman <sjackman@nospam.vortek.com> wrote:

I’m assuming all these end-points are handled by the same resource
manager.

The one resource manager handles all the end-points.

Is your resource manager multi-threaded?

The resource manager is multithreaded.

Do you really need to notify your clients that the 1ms is up?

I need to notify the client app when it can send again (after the 1ms is
up). At this point it needs to assemble the data and write() it. I can’t
call the write() before and have it block, because by the time the write
executes, the data may be stale (real-time app).

Right, makes sense.



– do you fail the client? What errno? EAGAIN?

I fail the client with EWOULDBLOCK (EAGAIN). My resource manager assumes
O_NONBLOCK was specified for all writable files.

Makes sense.

– can the client KNOW that it is a 1ms delay, and just try again
itself
without needing to be signaled from you?

That’s sort of the system I’m using right now. I’m using an ad-hoc way of
waiting for the next system clock tick, at which point I know I can send
again. (the transmit is synced to the system clock)

Right, but there’s no easy way to synch to the system clock tick… were
you the person who asked for the “wait for tick” function somewhere?

If you have multiple clients to signal, each is going to get it nearly
simultaneously, most will fail…do they have to re-activate their
notification each time? Is this more overhead than you want every 1ms?

There’s likely only one client waiting, and if there is two only one will
get to write this time (the other will get it next). This case won’t occur
often enough that I’m worried about the overhead involved in re-activating
the notification. There is an iofunc_notify_t structure associated with
every file (managed by this one resource manager) though, and I want to
avoid calling iofunc_notify_trigger() on each one. I might work around this
by maintaining a list of each iofunc_notify_t structures that has a client
waiting on it, and iterating through that list when the timer is up.

Do you need the other iofunc_notify_* stuff – that is notification for
read() or exceptional conditions?

It is beginning to sound like you really only need the notification that
it is ok to write.

If you only need the write, I can see two solutions:
– don’t bother with the iofunc_notify_* stuff, just do the handling
yourself, possibly with a register message when a client connects,
and automatically activating the notification any time you fail a
write()
– this might/could be more efficient than the more general purpose
ionotify_* helper routines (custom solutions often are)
– have all the attribute structures point to a shared ionotify_t[3]
array. You’ll only be setting/checking the write condition, but
the library helper functions should work ok.

If you need both write and read/except, then it gets a bit messier
– again, you might go with an array per attribute & a pointer to
a shared array
– in this case, in the iofunc_notify() message, you’d have to look
at the request… and determine whether it was a read/except or
write request & pass it off to the correct array
– if it was both, you’d have to split it – set/unset the correct
bits for each call so that you don’t get read/except notifications
appended to you write list and vice versa

-David

QNX Training Services
http://www.qnx.com/support/training/
Please followup in this newsgroup if you have further questions.