Mark Welo <mwelo@logisync.com> wrote:
I have a pretty simple question… I have created a resource manager to
handle some hardware and I want to be sure that access to the hardware is
“controlled”. If I have 2 threads, and one opens the RM(resource manager)
for read access and the other opens it for write access, can the threads
‘get in each others way’? Or does the RM read() code finish executing
before the RM write() code? Does having 2 separate file handles help? Am I
right in thinking that the following could happen?
Thread1 fd1 = open(/dev/rmgr);
Thread2 fd2 = open(/dev/rmgr);
// …
Thread2 write(fd2, “hello there”, 11);
Thread1 read(fd1, buf, sizeof(buf));
Thread2 write(fd2, “goodbye”, 7);
Thread1 read(fd1, buf, sizeof(buf));
Can Thread1 read back ‘goodbye’ instead of ‘hello there’? (or something
else?) Can the RM read() code get interrupted by Thread2 before it finishes
the read()? Is it guaranteed that I will get back both ‘hello there’ and
‘goodbye’?
Lots of good questions (and good answers as well). I’ll just
add a little bit.
Each open() of the same name “should” (ie unless you
know why you don’t want to) map to the same attribute
structure in your resource manager. So for the above
situation you should end up with:
Client | Server
fd1 ----|—> ocb1 —> attr1 (for /dev/rmgr)
fd2 ----|—> ocb2 —> also to attr1
Now if your resource manager is single threaded then
you are guaranteed that the operations will be performed
serially (assuming that your threads do in fact run
as you have ordered them the client has to
guarantee that). No surprise the server only have one
thread.
Once you start using the thread pool functions then
there is a different chain of events.
Each read/write operation within the resource manager
is “atomic” if the opens were bound to the same attribute
structure. The attribute structures are locked (which
is why seanb mentioned iofunc_attr_lock()) before heading
into any io operation. This means that unless you
explicitely unlock the attribute in your read/write
handler (which means that read/write operations would
not be atomic) you will be guaranteed that only one
of the operations will take place. This means that in the
above scenario of the first read/write one will get
in before the other, then the other will block waiting.
So it could be possible that the read() reads from an
empty buffer.
So now knowing this we can look at your questions assuming
that things work the way we would like them to:
Can Thread1 read back ‘goodbye’ instead of ‘hello there’?
(or something else?).
I would say an unqualified “maybe”. The behaviour entirely
depends on the behaviour of your resource manager. If your
resource manager was a “normal” device then the data would
be written to a buffer that would look like:
Initially:
device buffer: []
ocb1 offset = 0, ocb2 offset = 0
(ocbs are equivalent to the client fds but in the server)
Thread2 write(fd2, “hello there”, 11);
device buffer: [hello there]
ocb1 offset = 0, ocb2 offset = 11
Thread1 read(fd1, buf, sizeof(buf));
device buffer: [hello there]
ocb1 offset = 11, ocb2 offset = 11
Thread2 write(fd2, “goodbye”, 7);
device buffer: [hello theregoodbye]
ocb1 offset = 11, ocb2 offset = 18
Now I’ve acted as if these requests take place serially
when we know that the ordering could be any of the
following (only looking at the three calls):
write(), read(), read()
write(), write(), read()
read(), write(), write()
Now since the server is the one who controls the advancement
of the offset pointer in the ocb, it is up to the resource
manager writer to decide what behaviour they want. Perhaps
this server was meant to be a logging server which only
records the last event. In this case rather than
[hello theregoodby] we would just have [goodbye] on the
last call. In this case readers might never have their
offset advanced, but would always read from the start of
the buffer.
Can the RM read() code get interrupted by Thread2 before
it finishes the read()?
Again an big “maybe”. This can happen under two conditions:
- The server doesn’t use the same attribute structure for
both opens. Then the read() and the write() will be locking
different structures so there could be contention (ie if you
were reading from global resources which weren’t attached to
the attribute structure)
- You do bind to the same attribute, but in your read/write
handler you unlock the attribute structure by calling
iofunc_attr_unlock(). You can’t use the attribute again
until you lock the structure with iofunc_attr_lock().
You have to go out of your way to do these things, but you
should be aware of the behaviour of your server. So the
answer would be “no” in the default case.
Is it guaranteed that I will get back both ‘hello there’
and ‘goodbye’?
There are no guarantees =;-) It is up to you what you do
in your resource manager. Presumably you are storing the
writes somewhere … I think we covered the possible cases
above enough for you to be able to take a look and determine
for yourself. If you aren’t sure then ask again or drop
me a message.
How can I be guaranteed that I don’t drop data?
You should be able to guarantee that you don’t loose data
if you follow the guidelines above in terms of locking.
Now in terms of guaranteeing that you don’t run into a
situations where your client reads and empty buffer before
you have put anything there. That is a synchronization
and design issue you need to look at from both the
client and server point of view. Not knowing what you
are trying to design some suggestions might be:
- Have the thread continuously re-try (not so good)
- Use the iofunc_notify/select functions (requires work
in the server).
- Have the server block any reads when there is no
data available (ala pipe … again requires work in
the server).
Hope this helps you somewhat.
Thomas
