MsgDeliverEvent and QNET

Quote from the mannual under the MsgDeliverEvent entry:

Native networking When you use MsgDeliverEvent() to communicate across a network, the return code isn't "reliable". In the local case, MsgDeliverEvent() always returns a correct success or failure value. But since MsgDeliverEvent() must be nonblocking, in the networked case, the return value isn't guaranteed to reflect the actual result on the client's node. This is because MsgDeliverEvent() would have to block waiting for the communications between the two npm-qnets.

Generally, this isn’t a problem, because MsgDeliverEvent() is for the
benefit of the client anyway – if the client no longer exists, then the
client obviously doesn’t care that it didn’t get the event. The server
usually delivers the event and then goes about its business, regardless of
the success or failure of the event delivery.

Question:

Will the error code indicate a communications failure? In other words,
will MsgDeliverEvent() fail if the link to the destination node is down or
some other error prevented the remote npm-qnet from receiving the request?

I don’t care about notification that the client on the remote node actually
received it, but I do think it is important that the event at least reach
the remote node (qnet on the local node indicates a successful
transmission).

Kevin

One other item. What does it mean (reading from the docs again) under the
sub-heading “Blocking” when it says:

In the network case, lower priority threads may run. ?

Is this reffering to the lower priority threads on the remote node, lower
priority threads associated with io-net, or lower priority threads on the
local node?
Then…lower than what? The caller of MsgDeliverEvent, or the client that
is supposed to receive it?

BTW: I’ll try to keep myself from replying to myself from this point onward.
:slight_smile:

Thanks
Kevin

“Kevin Stallard” <kevin@ffflyingrobots.com> wrote in message
news:apa662$nec$1@inn.qnx.com

Quote from the mannual under the MsgDeliverEvent entry:

Quote
Native networking
When you use MsgDeliverEvent() to communicate across a network, the return
code isn’t “reliable”. In the local case, MsgDeliverEvent() always returns
a
correct success or failure value. But since MsgDeliverEvent() must be
nonblocking, in the networked case, the return value isn’t guaranteed to
reflect the actual result on the client’s node. This is because
MsgDeliverEvent() would have to block waiting for the communications
between
the two npm-qnets.

Generally, this isn’t a problem, because MsgDeliverEvent() is for the
benefit of the client anyway – if the client no longer exists, then the
client obviously doesn’t care that it didn’t get the event. The server
usually delivers the event and then goes about its business, regardless of
the success or failure of the event delivery.

/Quote

Question:

Will the error code indicate a communications failure? In other words,
will MsgDeliverEvent() fail if the link to the destination node is down or
some other error prevented the remote npm-qnet from receiving the request?

I don’t care about notification that the client on the remote node
actually
received it, but I do think it is important that the event at least reach
the remote node (qnet on the local node indicates a successful
transmission).

Kevin

\

Kevin Stallard <kevin@ffflyingrobots.com> wrote in message
news:apa662$nec$1@inn.qnx.com

Quote from the mannual under the MsgDeliverEvent entry:

Quote
Native networking
When you use MsgDeliverEvent() to communicate across a network, the return
code isn’t “reliable”. In the local case, MsgDeliverEvent() always returns
a
correct success or failure value. But since MsgDeliverEvent() must be
nonblocking, in the networked case, the return value isn’t guaranteed to
reflect the actual result on the client’s node. This is because
MsgDeliverEvent() would have to block waiting for the communications
between
the two npm-qnets.

Generally, this isn’t a problem, because MsgDeliverEvent() is for the
benefit of the client anyway – if the client no longer exists, then the
client obviously doesn’t care that it didn’t get the event. The server
usually delivers the event and then goes about its business, regardless of
the success or failure of the event delivery.

/Quote

Question:

Will the error code indicate a communications failure? In other words,
will MsgDeliverEvent() fail if the link to the destination node is down or
some other error prevented the remote npm-qnet from receiving the request?

No. In fact it seems it will never fail cross QNET. (Not if something like
the
delievered event pointer is point to invalid memory space)

I don’t care about notification that the client on the remote node
actually
received it, but I do think it is important that the event at least reach
the remote node (qnet on the local node indicates a successful
transmission).

Why? What the difference of the event is reached the other node or not ?
The idea of QNET is your application should not aware it is talking cross
network or not…

-xtang

Kevin Stallard <kevin@ffflyingrobots.com> wrote in message
news:apa6im$nns$1@inn.qnx.com

One other item. What does it mean (reading from the docs again) under the
sub-heading “Blocking” when it says:

In the network case, lower priority threads may run. ?

Is this reffering to the lower priority threads on the remote node, lower
priority threads associated with io-net, or lower priority threads on the
local node?
Then…lower than what? The caller of MsgDeliverEvent, or the client that
is supposed to receive it?

I believe this is refer to the calling process thread (the server).

In local case, MsgDeliverEvent goes into kernel and come back from kernel
(non blocking). If this is going to cross network, the kernel will actually
put
the calling thread into a special status (waiting for QNET to response), and
the kernel will then do a re-schedule, so low priority thread could be run.
Effectively, MsgDeliverEvent() cross network will be “block” for a very
short period.

Same happens to MsgWrite() and MsgWrite() kercall.

-xtang

Interesting…this makes sense…it’s too bad QNET’s success in deliver that
event to the remote node isn’t considered by the kernel.

One thing to consider about the non-blocking aspect of this. I consder it
to be syncronous if the call has to wait for a client ack (i.e. the client
is actually run and responds as a result of receiving the event before I
unblock). But I think one could argue that it is still asyncronous even if
we block waiting for the remote qnet to acknowledge receipt of the event as
the client destined to receive it hasn’t been made ‘READY’ and run as a
result of the event.

If a server could absolutly not block because of a call to MsgDeliverEvent,
how about using a flag as one of MsgDeliverEvents parameters? Then we could
choose what we wanted…

Kevin

“Xiaodan Tang” <xtang@qnx.com> wrote in message
news:apaah9$lr1$1@nntp.qnx.com

Kevin Stallard <> kevin@ffflyingrobots.com> > wrote in message
news:apa6im$nns$> 1@inn.qnx.com> …
One other item. What does it mean (reading from the docs again) under
the
sub-heading “Blocking” when it says:

In the network case, lower priority threads may run. ?

Is this reffering to the lower priority threads on the remote node,
lower
priority threads associated with io-net, or lower priority threads on
the
local node?
Then…lower than what? The caller of MsgDeliverEvent, or the client
that
is supposed to receive it?

I believe this is refer to the calling process thread (the server).

In local case, MsgDeliverEvent goes into kernel and come back from kernel
(non blocking). If this is going to cross network, the kernel will
actually
put
the calling thread into a special status (waiting for QNET to response),
and
the kernel will then do a re-schedule, so low priority thread could be
run.
Effectively, MsgDeliverEvent() cross network will be “block” for a very
short period.

Same happens to MsgWrite() and MsgWrite() kercall.

-xtang

Why? What the difference of the event is reached the other node or not ?
The idea of QNET is your application should not aware it is talking cross
network or not…

I guess this is my point. Because our current application relies heavly on
the reliability of asyncronous event notification (asyncronous in the sense
that I do not need the client to run before it returns, just know that it is
at least queued up on the client), we can’t use them because we now have to
know if we are using the network or not.

If we aren’t using the network, then MsgDeliverEvent() appears to give the
feed back we need to guarantee delivery.

If we are using the network, then we get no such feedback.

I would really like my application to not have to be aware of the network…
:slight_smile: Just aware if the event is going to be delivered or not.

Kevin

-xtang

See one of the aspects of QNX that I argue a lot is the fact that QNX can be
very resource aware. If I’m having to clog the network because I constantly
have to query for state information, then I can’t use that anymore as a
reason and they might as well use streaming UDP (ick!).

Additionally…when it comes to state behavior…if a remote process is
relying on accurate state change information and has to query for it, it is
possible that states could change w/o the client knowing (aliasing
problems…not sampling fast enough). This is why there has to be a way
to know if an event at least made it across the wire. If it makes it across
the wire I think we can safely say it will make it to the client if it is
still running…if it isn’t running…then, yes, there is no need for
concern.

Kevin

“Xiaodan Tang” <xtang@qnx.com> wrote in message
news:apaa3j$lh9$1@nntp.qnx.com

Kevin Stallard <> kevin@ffflyingrobots.com> > wrote in message
news:apa662$nec$> 1@inn.qnx.com> …
Quote from the mannual under the MsgDeliverEvent entry:

Quote
Native networking
When you use MsgDeliverEvent() to communicate across a network, the
return
code isn’t “reliable”. In the local case, MsgDeliverEvent() always
returns
a
correct success or failure value. But since MsgDeliverEvent() must be
nonblocking, in the networked case, the return value isn’t guaranteed to
reflect the actual result on the client’s node. This is because
MsgDeliverEvent() would have to block waiting for the communications
between
the two npm-qnets.

Generally, this isn’t a problem, because MsgDeliverEvent() is for the
benefit of the client anyway – if the client no longer exists, then the
client obviously doesn’t care that it didn’t get the event. The server
usually delivers the event and then goes about its business, regardless
of
the success or failure of the event delivery.

/Quote

Question:

Will the error code indicate a communications failure? In other words,
will MsgDeliverEvent() fail if the link to the destination node is down
or
some other error prevented the remote npm-qnet from receiving the
request?

No. In fact it seems it will never fail cross QNET. (Not if something like
the
delievered event pointer is point to invalid memory space)

I don’t care about notification that the client on the remote node
actually
received it, but I do think it is important that the event at least
reach
the remote node (qnet on the local node indicates a successful
transmission).

Why? What the difference of the event is reached the other node or not ?
The idea of QNET is your application should not aware it is talking cross
network or not…

-xtang

OK. Kevin actually send me an email explained their system and what they are
tring to do. After some mindstorm with our CAOs and old timer FLEET
expert, let me put all the thoughts together :slight_smile:

  1. MsgDelieverEvent() is not “reliable” in the manner of “delievering
    event”, local or remote.

In local, MsgDelieverEvent() look at the client (the destination of
deliverring), and queue
the event, return success. The client could then gone, and event
delievere will fail late.

In remote, MsgDelieverEvent() won’t be able to look at the client (at
remote site), it just
successed, handing data to QNET. QNET then move the event cross network,
and
deliever it to client. Either step could fail, and server won’t know.

So the question goes to, is it important to KNOW that a client is
existing at the moment
of MsgDelieverEvent() ?

  1. Kevin mentioned FLEET and remote proxy trigger. It turns out FLEET did
    the same thing.
    After Trigger() a remote proxy, and return EOK, you can only sure the
    VC is existing.
    Data is not moving accross network at that time, and could still fail
    later.

  2. In QNX6, if a clint (who have an open connection with a server) goes
    away, the server
    will get told (_NTO_CHF_DISCONNECT). This means the server will
    “reliable” know
    whenever a client (local or remote) goes away. If you using resource
    manager framework,
    this flag is automaticly set on channel for you.

  3. You seems more concern of QNET can’t move data cross network. In general,
    QNET
    provide reliable data transfer. if data can’t be reliable transfered
    (network down, remote
    node shutdown, nic error), QNET will decided the other node is no longer
    exist, and
    wipe out ALL the clients from that node, which in turn, the server will
    get it’s DISCONNECT
    pulse.
    (The are more complicate things happened in multiple interface, and QoS,
    but …)

-xtang

Kevin Stallard <kevin@ffflyingrobots.com> wrote in message
news:apae5a$25s$1@inn.qnx.com

See one of the aspects of QNX that I argue a lot is the fact that QNX can
be
very resource aware. If I’m having to clog the network because I
constantly
have to query for state information, then I can’t use that anymore as a
reason and they might as well use streaming UDP (ick!).

Additionally…when it comes to state behavior…if a remote process is
relying on accurate state change information and has to query for it, it
is
possible that states could change w/o the client knowing (aliasing
problems…not sampling fast enough). This is why there has to be a
way
to know if an event at least made it across the wire. If it makes it
across
the wire I think we can safely say it will make it to the client if it is
still running…if it isn’t running…then, yes, there is no need for
concern.

Kevin

“Xiaodan Tang” <> xtang@qnx.com> > wrote in message
news:apaa3j$lh9$> 1@nntp.qnx.com> …

Kevin Stallard <> kevin@ffflyingrobots.com> > wrote in message
news:apa662$nec$> 1@inn.qnx.com> …
Quote from the mannual under the MsgDeliverEvent entry:

Quote
Native networking
When you use MsgDeliverEvent() to communicate across a network, the
return
code isn’t “reliable”. In the local case, MsgDeliverEvent() always
returns
a
correct success or failure value. But since MsgDeliverEvent() must be
nonblocking, in the networked case, the return value isn’t guaranteed
to
reflect the actual result on the client’s node. This is because
MsgDeliverEvent() would have to block waiting for the communications
between
the two npm-qnets.

Generally, this isn’t a problem, because MsgDeliverEvent() is for the
benefit of the client anyway – if the client no longer exists, then
the
client obviously doesn’t care that it didn’t get the event. The server
usually delivers the event and then goes about its business,
regardless
of
the success or failure of the event delivery.

/Quote

Question:

Will the error code indicate a communications failure? In other
words,
will MsgDeliverEvent() fail if the link to the destination node is
down
or
some other error prevented the remote npm-qnet from receiving the
request?

No. In fact it seems it will never fail cross QNET. (Not if something
like
the
delievered event pointer is point to invalid memory space)

I don’t care about notification that the client on the remote node
actually
received it, but I do think it is important that the event at least
reach
the remote node (qnet on the local node indicates a successful
transmission).

Why? What the difference of the event is reached the other node or not ?
The idea of QNET is your application should not aware it is talking
cross
network or not…

-xtang
\

“Xiaodan Tang” <xtang@qnx.com> wrote in message
news:apbnuv$kds$1@nntp.qnx.com

OK. Kevin actually send me an email explained their system and what they
are
tring to do. After some mindstorm with our CAOs and old timer FLEET
expert, let me put all the thoughts together > :slight_smile:
Heh…I pressed the wrong button when I went to reply to you…I was meaning

to post it to the ng, but alas, it filled up your mail box…sorry about
that :slight_smile: It was a bit too wordy anyway…that’s why I didn’t post it here.

  1. MsgDelieverEvent() is not “reliable” in the manner of “delievering
    event”, local or remote.

In local, MsgDelieverEvent() look at the client (the destination of
deliverring), and queue
the event, return success. The client could then gone, and event
delievere will fail late.

That’s actualy just fine. We don’t need to know that the client actually
received it (i.e. acknowledged it and acted upon it) just that it got queued
to the client, or that it reached the remote node where we can be fairly
sure it would be queued.

In remote, MsgDelieverEvent() won’t be able to look at the client (at
remote site), it just
successed, handing data to QNET. QNET then move the event cross
network,
and
deliever it to client. Either step could fail, and server won’t know.

So the question goes to, is it important to KNOW that a client is
existing at the moment
of MsgDelieverEvent() ?

No it is not important. Obviously MsgDeliverEvent returns in error in the
local sense if the client doesn’t exist, and the traffic that would have to
occur to insure a client existed over network is an unreasonable demand.
However, one can reasonably expect that if the event makes it across the
wire, that it won’t get lost because of ambiant noise or a link failure, and
thus the remote qnet will attempt to find the client and deliver it to that
client. If the client has died, then there is no reason to care. It’s just
that I think there needs to be some indication that the event has reached
the remote node.


  1. Kevin mentioned FLEET and remote proxy trigger. It turns out FLEET did
    the same thing.
    After Trigger() a remote proxy, and return EOK, you can only sure the
    VC is existing.
    Data is not moving accross network at that time, and could still fail
    later.

Yes…and if a client has died, the VC is cleaned up. I believe that the vc
becomes unavailable because of a broken link, and the proxy can’t make it
across the network, Trigger() will indicates this. It is this type of
functionality upon which we rely…

  1. In QNX6, if a clint (who have an open connection with a server) goes
    away, the server
    will get told (_NTO_CHF_DISCONNECT). This means the server will
    “reliable” know
    whenever a client (local or remote) goes away. If you using resource
    manager framework,
    this flag is automaticly set on channel for you.

I remember a conversation I had with someone who indicated that the
_NTO_CHF_DISCONNECT was not a reliable way of determining if a client gets
disconnected. I don’t remember the details of the conversation, but I
believe I was having problems with clients that were reply blocked on a
server and was concerned because I could not force the clients to die as the
server would not let them go (I can’t remember the flag that is set that
allows a server to not let go of a client). If I remember correctly, I was
told that _NTO_CHF_DISCONNECT was not 100% reliable, and so it was necessary
that servers hold on to clients until they wish to reply to them so as to
prevent duplicate rcvid from messing things up.

However, it’s not that I care about a client going away, I care about the
ability to get the event from the server’s node to the remote node with
verification that it at least arrive at the remote node. We can assume that
if the event arrives at the remote node and the client is runnig, that it
will then get queued up at the client. The problem occurs if the client is
running and the event gets lost because of network interruptions, I want to
be able to recover gracefully from it.

  1. You seems more concern of QNET can’t move data cross network. In
    general,
    QNET
    provide reliable data transfer. if data can’t be reliable transfered
    (network down, remote
    node shutdown, nic error), QNET will decided the other node is no
    longer
    exist, and
    wipe out ALL the clients from that node, which in turn, the server
    will
    get it’s DISCONNECT
    pulse.
    (The are more complicate things happened in multiple interface, and
    QoS,
    but …)

Is the DISCONNECT pulse guaranteed to be delivered to the server in this
situation? If not, under what circumstances might a server miss a
DISCONNECT pulse?

I know you guys are very busy over there, so I really appreciate you time
xtang…take care,

Kevin

-xtang

Kevin Stallard <> kevin@ffflyingrobots.com> > wrote in message
news:apae5a$25s$> 1@inn.qnx.com> …
See one of the aspects of QNX that I argue a lot is the fact that QNX
can
be
very resource aware. If I’m having to clog the network because I
constantly
have to query for state information, then I can’t use that anymore as a
reason and they might as well use streaming UDP (ick!).

Additionally…when it comes to state behavior…if a remote process is
relying on accurate state change information and has to query for it, it
is
possible that states could change w/o the client knowing (aliasing
problems…not sampling fast enough). This is why there has to be a
way
to know if an event at least made it across the wire. If it makes it
across
the wire I think we can safely say it will make it to the client if it
is
still running…if it isn’t running…then, yes, there is no need for
concern.

Kevin

“Xiaodan Tang” <> xtang@qnx.com> > wrote in message
news:apaa3j$lh9$> 1@nntp.qnx.com> …

Kevin Stallard <> kevin@ffflyingrobots.com> > wrote in message
news:apa662$nec$> 1@inn.qnx.com> …
Quote from the mannual under the MsgDeliverEvent entry:

Quote
Native networking
When you use MsgDeliverEvent() to communicate across a network, the
return
code isn’t “reliable”. In the local case, MsgDeliverEvent() always
returns
a
correct success or failure value. But since MsgDeliverEvent() must
be
nonblocking, in the networked case, the return value isn’t
guaranteed
to
reflect the actual result on the client’s node. This is because
MsgDeliverEvent() would have to block waiting for the communications
between
the two npm-qnets.

Generally, this isn’t a problem, because MsgDeliverEvent() is for
the
benefit of the client anyway – if the client no longer exists, then
the
client obviously doesn’t care that it didn’t get the event. The
server
usually delivers the event and then goes about its business,
regardless
of
the success or failure of the event delivery.

/Quote

Question:

Will the error code indicate a communications failure? In other
words,
will MsgDeliverEvent() fail if the link to the destination node is
down
or
some other error prevented the remote npm-qnet from receiving the
request?

No. In fact it seems it will never fail cross QNET. (Not if something
like
the
delievered event pointer is point to invalid memory space)

I don’t care about notification that the client on the remote node
actually
received it, but I do think it is important that the event at least
reach
the remote node (qnet on the local node indicates a successful
transmission).

Why? What the difference of the event is reached the other node or not
?
The idea of QNET is your application should not aware it is talking
cross
network or not…

-xtang


\

Kevin Stallard wrote:

One thing to consider about the non-blocking aspect of this. I consder it
to be syncronous if the call has to wait for a client ack (i.e. the client
is actually run and responds as a result of receiving the event before I
unblock). But I think one could argue that it is still asyncronous even if
we block waiting for the remote qnet to acknowledge receipt of the event as
the client destined to receive it hasn’t been made ‘READY’ and run as a
result of the event.

What if the remote kernel, was on the other side of an RFC 1149 link ?
Your “non-blocking” MsgDeliverEvent could suddenly take weeks to return
(even if the link is operating within design parameters). Not a good
situation for call that should be non-blocking…

Obviously, this is an extreme case, but non-blocking should mean
non-blocking (i.e. that the call will return in a relatively short
and deterministic period).

Rennie

Miguel Simon wrote:

Also, if you need reliable knowledge that something happened, then you
can not use asynchronous architecture for this; you must use some sort
of synchronous notification.

Yup… and a deterministic communication system, so that you know
when to register a failure.

Hi Kevin…

Xiaodan Tang wrote:
<…>

  1. In QNX6, if a clint (who have an open connection with a server) goes
    away, the server
    will get told (_NTO_CHF_DISCONNECT). This means the server will
    “reliable” know
    whenever a client (local or remote) goes away. If you using resource
    manager framework,
    this flag is automaticly set on channel for you.

We use UDP + this feature above very reliably. The resource manager
model is very reliable indeed.

Also, if you need reliable knowledge that something happened, then you
can not use asynchronous architecture for this; you must use some sort
of synchronous notification. Again, this is our experience, and it works
well for us.

regards…

Miguel.

Also, if you need reliable knowledge that something happened, then you
can not use asynchronous architecture for this; you must use some sort
of synchronous notification.

Yup… and a deterministic communication system, so that you know
when to register a failure.

That depends though.

There are a couple of ways to look at something being asynchronous.

Asynchronous communication with a client to me means that the
MsgDeliverEvent() can return before the client can even deal with the
message sent to it. You can still participate in asyncronous communication
with a client but have the communication meduim itself be syncronous.

I’m not synchronizing the client process with the server process. I am
synchronizing the local npm-qnet with the remote npm-qnet. But that is all,
I’m not waiting for the client to respond. QNX already pretty much
guarantees that events make it to where they are supposed to once they
arrive to their destination node.

I realize that the communication medium has to be taken into account. I
doubt you would pick a medium that would affect real-time performance anyway
if you were going to use QNET across it (i.e. a RFC 1149 link).

I think that if QNET is really supposed to extend the OS over the network,
then the communication aspect of it should by definition be syncronous.
MsgDeliverEvent() is there to deliver events, not stream information
packets, if you want to stream packets, UDP is the best choice.
MsgDeliverEvent() is not for streaming it should be for the reliable
asyncronous delivery of events Again asyncronous from the perspective of
the server and client only.

Kevin

Is there an “after the fact” method to query the success of the last
attempted transmission?

“Kevin Stallard” <kevin@ffflyingrobots.com> wrote in message
news:apadcq$1k7$1@inn.qnx.com

Interesting…this makes sense…it’s too bad QNET’s success in deliver
that
event to the remote node isn’t considered by the kernel.

One thing to consider about the non-blocking aspect of this. I consder it
to be syncronous if the call has to wait for a client ack (i.e. the client
is actually run and responds as a result of receiving the event before I
unblock). But I think one could argue that it is still asyncronous even
if
we block waiting for the remote qnet to acknowledge receipt of the event
as
the client destined to receive it hasn’t been made ‘READY’ and run as a
result of the event.

If a server could absolutly not block because of a call to
MsgDeliverEvent,
how about using a flag as one of MsgDeliverEvents parameters? Then we
could
choose what we wanted…

Kevin

“Xiaodan Tang” <> xtang@qnx.com> > wrote in message
news:apaah9$lr1$> 1@nntp.qnx.com> …

Kevin Stallard <> kevin@ffflyingrobots.com> > wrote in message
news:apa6im$nns$> 1@inn.qnx.com> …
One other item. What does it mean (reading from the docs again) under
the
sub-heading “Blocking” when it says:

In the network case, lower priority threads may run. ?

Is this reffering to the lower priority threads on the remote node,
lower
priority threads associated with io-net, or lower priority threads on
the
local node?
Then…lower than what? The caller of MsgDeliverEvent, or the client
that
is supposed to receive it?

I believe this is refer to the calling process thread (the server).

In local case, MsgDeliverEvent goes into kernel and come back from
kernel
(non blocking). If this is going to cross network, the kernel will
actually
put
the calling thread into a special status (waiting for QNET to response),
and
the kernel will then do a re-schedule, so low priority thread could be
run.
Effectively, MsgDeliverEvent() cross network will be “block” for a very
short period.

Same happens to MsgWrite() and MsgWrite() kercall.

-xtang
\