Forcing QNET to query for existing nodes.

Hi Kevin…

Kevin Stallard wrote:

Hey Miguel,

Nice to hear from you, how are you doing? Are you going to be in SJC
anytime soon? We still need to do lunch > :slight_smile:

I’ll be there next week, as a matter of fact. Hope to see you!

I wonder if it would be possible to create another npm-name.so. A simple
network protocol that would be loaded along with npm-qnet.so that could be
asked to broadcast a request of names and the record them and make them
available to the rest of the apps locally. When ever it received this
broadcast message it would send back it’s own name. That might suffice, I
wonder how hard it would be to implement.

I suppose that you could implement a resource manager that could do
something similar with the current tools, but something like a native
implementation would be nice indeed.

Regards…

Miguel.

This would essentially do what you are suggesting w/o using tcpip stack and
I wouldn’t have to muck around with IP addresses…

Best Regards,
Kevin

Well, this is a long argument ever since QNX4 FLEET.

“Broadcast” is not “reliable” (that along is argumable, but), so it could
take
time (un-determinable) when a service be propergated on network.
That’s why nameloc on FLEET do not based on broadcasting.

-xtang

<kabe@sra-tohoku.co.jp> wrote in message news:ape8kq$c11$1@inn.qnx.com

[full snip]
The more I read this thread, the thing you need is
already provided in NetBIOS name registering scheme…

(NetBIOS uses broadcast to announce itself, each node listens to them
and remembers it locally, use timeout to expire stale entry,
could establish “master browser” without any administration…)

The trouble is QNET doesn’t use that
(but you can run Samba nmbd on each node and glean browser data from
/var/samba)

kabe

“Xiaodan Tang” <xtang@qnx.com> wrote in message
news:api297$f3n$1@nntp.qnx.com

Well, this is a long argument ever since QNX4 FLEET.

“Broadcast” is not “reliable” (that along is argumable, but), so it could
take
time (un-determinable) when a service be propergated on network.

That’s why nameloc on FLEET do not based on broadcasting.

Well ARP has been pretty reliable enought for me resolving IP versus MAC
addresses? You will have to be more convincing xtang :wink:

-xtang

kabe@sra-tohoku.co.jp> > wrote in message news:ape8kq$c11$> 1@inn.qnx.com> …
[full snip]
The more I read this thread, the thing you need is
already provided in NetBIOS name registering scheme…

(NetBIOS uses broadcast to announce itself, each node listens to them
and remembers it locally, use timeout to expire stale entry,
could establish “master browser” without any administration…)

The trouble is QNET doesn’t use that
(but you can run Samba nmbd on each node and glean browser data from
/var/samba)

kabe

Mario Charest postmaster@127.0.0.1 wrote in message
news:apjfis$71c$1@inn.qnx.com

“Xiaodan Tang” <> xtang@qnx.com> > wrote in message
news:api297$f3n$> 1@nntp.qnx.com> …
Well, this is a long argument ever since QNX4 FLEET.

“Broadcast” is not “reliable” (that along is argumable, but), so it
could
take
time (un-determinable) when a service be propergated on network.

That’s why nameloc on FLEET do not based on broadcasting.

Well ARP has been pretty reliable enought for me resolving IP versus MAC
addresses? You will have to be more convincing xtang > :wink:

This is something people keeps on arguing :slight_smile:

On LAN, “Broadcast Once” is not reliable, but “Broadcast Enough Times”
could considered “reliable” (there are protocols out there based on this).

However, using broadcast increase the “time of undeterminable” (which
against
the “real time”).You understand ARP could lost and then your first packet
time
“seconds” to find it’s way. Or, some shared folder takes more time to show
up.
Now, does people use global name services could accept that ?

And of cause, if QNET is riding on IP and on something like 2 different IP
subnet,
you won’t be able to announce/lookup global services cross subnet.

-xtang

-xtang

kabe@sra-tohoku.co.jp> > wrote in message
news:ape8kq$c11$> 1@inn.qnx.com> …
[full snip]
The more I read this thread, the thing you need is
already provided in NetBIOS name registering scheme…

(NetBIOS uses broadcast to announce itself, each node listens to them
and remembers it locally, use timeout to expire stale entry,
could establish “master browser” without any administration…)

The trouble is QNET doesn’t use that
(but you can run Samba nmbd on each node and glean browser data from
/var/samba)

kabe
\

“Xiaodan Tang” <xtang@qnx.com> wrote in message
news:apjl31$es1$1@nntp.qnx.com

Mario Charest postmaster@127.0.0.1 wrote in message
news:apjfis$71c$> 1@inn.qnx.com> …

“Xiaodan Tang” <> xtang@qnx.com> > wrote in message
news:api297$f3n$> 1@nntp.qnx.com> …
Well, this is a long argument ever since QNX4 FLEET.

“Broadcast” is not “reliable” (that along is argumable, but), so it
could
take
time (un-determinable) when a service be propergated on network.

That’s why nameloc on FLEET do not based on broadcasting.

Well ARP has been pretty reliable enought for me resolving IP versus MAC
addresses? You will have to be more convincing xtang > :wink:

This is something people keeps on arguing > :slight_smile:

On LAN, “Broadcast Once” is not reliable, but “Broadcast Enough Times”
could considered “reliable” (there are protocols out there based on this).

However, using broadcast increase the “time of undeterminable” (which
against
the “real time”).You understand ARP could lost and then your first packet
time
“seconds” to find it’s way. Or, some shared folder takes more time to show
up.
Now, does people use global name services could accept that ?

And of cause, if QNET is riding on IP and on something like 2 different IP
subnet,
you won’t be able to announce/lookup global services cross subnet.

I see, in the context of QNET it seems to make sense, however I don’t think
that was the reason QNX4 didn’t use broadcast for global name and the
deterministic argument doesn’t fly. nameloc is definitely not
deterministic, unless you want to take deterministic beyond the capability
of a normal human brain :wink:

-xtang



-xtang

kabe@sra-tohoku.co.jp> > wrote in message
news:ape8kq$c11$> 1@inn.qnx.com> …
[full snip]
The more I read this thread, the thing you need is
already provided in NetBIOS name registering scheme…

(NetBIOS uses broadcast to announce itself, each node listens to
them
and remembers it locally, use timeout to expire stale entry,
could establish “master browser” without any administration…)

The trouble is QNET doesn’t use that
(but you can run Samba nmbd on each node and glean browser data from
/var/samba)

kabe


\

xtang@qnx.com sed in <apjl31$es1$1@nntp.qnx.com>

However, using broadcast increase the “time of undeterminable” (which
against
the “real time”).You understand ARP could lost and then your first packet
time

Using CDMA (i.e Ethernet) already breaks realtime requirement IMHO.
Having reliable global name service doesn’t help if it’s over Ethernet.

(So if you want real realtime, all connections should be Point-to-Point…)

kabe

<kabe@sra-tohoku.co.jp> wrote in message news:apkhlk$dmk$2@inn.qnx.com

xtang@qnx.com > sed in <apjl31$es1$> 1@nntp.qnx.com

However, using broadcast increase the “time of undeterminable” (which
against
the “real time”).You understand ARP could lost and then your first
packet
time

Using CDMA (i.e Ethernet) already breaks realtime requirement IMHO.
Having reliable global name service doesn’t help if it’s over Ethernet.

(So if you want real realtime, all connections should be
Point-to-Point…)

Well, “global name service” is about “regist and find”, once the service is
found.
A point to point connection well be made…

By “reliable global name service”, we want to if a name is registed
globally,
then it must be able to be found (lookup should not fail). Or is this no
longer
important anymore?

-xtang

My advice here is to:

  1. write a name server application
  2. put it on a reliable and well know host
  3. allow other processes to register with it
  4. require them to handshake with it periodically - either side may initiate
  5. allow other processes to become a backup of a registered server if when
    the main registered server goes down.
  6. allow clients to query the name server for the location of the registered
    servers.
  7. design mirroring into the name server so that several name servers could
    all keep each other informed
  8. if a client process ever fails to reach a registered server it should
    automatically try to find a new registered server by the same name.
  9. the clients should have a list of well know name servers - extension of
    number 2.
  10. allow for secondary name servers that can run on local machines to speed
    up access to the name data base.

This should take, oh what, about 45 minutes? OK. Maybe a little longer.

I’ve written several master servers like this in QNX4. It works quite well.
And what I like is that once you write it, you know how it works. You can
privateize the protocol or open it up (or both). You can build in your own
levels of security, if needed.

The issue that scares me is “Who do you trust?”

If a new host comes along and says “Hi, My name is Bill. What’s yours? Do
you want to go out some time?” The other host has no way of knowing if
“Bill” is a trusted new member of the community, or some predator that snuck
in. At least that’s what all the girls tell me when I try internet dating.

Somewhere there NEEDS to be a trusted, authoritative list of who’s who.

I guess that a host could contact all of their already trusted friends and
ask if they know “Bill”. If they do and Bill is trusted this host could
just add him to it’s list of trusted friends. But as the trusted network
gets bigger and bigger, this becomes unreasonable pretty fast.

“Kevin Stallard” <kevin@ffflyingrobots.com> wrote in message
news:apcnk2$lng$1@inn.qnx.com

“Kevin Stallard” <> kevin@ffflyingrobots.com> > wrote in message
news:apbri7$lla$> 1@inn.qnx.com> …
Well, for now, yes…but this is something we are working on moving away
from. This would remove an additional dependancy from the application
if
we
didn’t have to care about node names.


Let me clarify this a little bit.

Node names are still extremely useful. It helps you identify which
physical
piece of hardware to which you are talking. Very important when it comes
to
diagnostics. Robotics it is especially important. In the robotics case,
you may want to introduce a new machine w/o having to tell everyone else
its name.

I don’t think a global name locator service is the answer, but the ability
to discover who is there and what resources they have is. I really think
you guys hit a home run when you decided to have servers show up in the
pathname space. Yeah…when you query for this info there are a lot of
stat()'s going on, but this should only occur during initial discovery.
I’m
not sure its that big a deal as when you know what services are there, you
connect to them usuall for the lifetime of what ever operation you are
performing.

Anyone knows that if you are going to access a file, you don’t open and
close, then re-open it for every record.

We just need to be able to broadcast a “tell me who you are” message to
currently running nodes and get responses on demand. The broadcast idea
will work, but like you said xtang, it is going to result in a lot of
chatter. It would be nice if remote qnets could respond to a “tell my who
you are” broadcast.

Kevin



Kevin

“Xiaodan Tang” <> xtang@qnx.com> > wrote in message
news:apbrf9$n7s$> 1@nntp.qnx.com> …
Kevin,

On a perticular system, don’t you always KNOW all your node names? You
know
you can
talk to a node even if it is not show up under /net, do you?

-xtang

Kevin Stallard <> kevin@ffflyingrobots.com> > wrote in message
news:apbp18$ipj$> 1@inn.qnx.com> …

On the other hand, the undocumented “broadcast=” option, would
make
it
chatty, but
somewhat reliable (on node being show up under /net).

Let me point this out, there is NO reliable way to make “all node
show
up”
from the design
view of QNET. What if QNET is based on tcpip, do you want to know
ALL
the
nodes
on the otherside of earth is come and go? What if some node is on
a
flaky
link, and not
broadcast reachable?

The TCPIP part of this is understandable. That would be too hard to
do,
but
it is unlikley that we would do what we want using TCPIP as it would
be
local LAN only (TCPIP has too much overhead, and too much has to be
done
to
make it dynamic (have to use DHCP, etc). This is how we intend to
use
it
(local LAN only)…

Thanks for your feedback xtang…it’s appreciated.

Kevin

This will only work on some special condition (a LAN, reasonable
reliable
media).

-xtang


Kevin

“Xiaodan Tang” <> xtang@qnx.com> > wrote in message
news:apaapi$lvb$> 1@nntp.qnx.com> …

Kevin Stallard <> kevin@ffflyingrobots.com> > wrote in message
news:apa9o0$r0s$> 1@inn.qnx.com> …
Hey xtang…how’s things goin?

I want to be able to start a server on any node and be able
to
find
it
w/o
knowning the name of the node on which it is residing. This
would
allow

This is what name_attach(name, NAME_ATTACH_FLAG_GLOBAL)
and name_open(name, NAME_ATTACH_FLAG_GLOBAL) suppose
to do.

Unfortunatly this is not true right now. We will have a
“Global
Name
Services”
manager to address this.

-xtang

dynamic shifting of processes and the dynamic discovery of
them.

I need a way to find out the names of the nodes on the
network
so
I
can
query each one…

Thanks,
Kevin
“Xiaodan Tang” <> xtang@qnx.com> > wrote in message
news:apa9ik$l7b$> 1@nntp.qnx.com> …
No, netmgr_ctl() won’t help. In fact, there is no way in
QNET
to
re-learn
the existing node.
Node names under /net is not that “reliable” anyway.

It might be interesting to know why you want to do this?
and
maybe
there
are
otherways
to do it.

-xtang

Kevin Stallard <> kevin@ffflyingrobots.com> > wrote in message
news:ap9ieq$337$> 1@inn.qnx.com> …
I found this function mentioned in the QNET Networking
docs,
netmgr_ctl(),
but it doesn’t seem to exist in the library reference.
Would
netmgr_ctl()
help me out with this?

Thanks
Kevin

“Kevin Stallard” <> kevin@ffflyingrobots.com> > wrote in
message
news:ap9gpk$1e2$> 1@inn.qnx.com> …
Hi,

I’d like to be able to send a message to io-net/qnet
to
force
it
to
query
for existing nodes running qnet. I am going to be
looking
for
server
names
by scanning the /net directory, and have noticed that
unless
I
know
a
name
of a node, I can’t get qnet to display until it
decides
to
take
a
look
and
see who is there.

Is it possible?

Thanks
Kevin


















\

See my comment on trusted friends.

<kabe@sra-tohoku.co.jp> wrote in message news:ape8kq$c11$1@inn.qnx.com

[full snip]
The more I read this thread, the thing you need is
already provided in NetBIOS name registering scheme…

(NetBIOS uses broadcast to announce itself, each node listens to them
and remembers it locally, use timeout to expire stale entry,
could establish “master browser” without any administration…)

The trouble is QNET doesn’t use that
(but you can run Samba nmbd on each node and glean browser data from
/var/samba)

kabe

Or a server on that new node that provides some authentication that is
recognized and hard to duplicate (public/private key combination sort of
thing). Interesting point.

Kevin

“Bill Caroselli (Q-TPS)” <QTPS@EarthLink.net> wrote in message
news:appc89$bm$1@inn.qnx.com

The issue that scares me is “Who do you trust?”

If a new host comes along and says “Hi, My name is Bill. What’s yours?
Do
you want to go out some time?” The other host has no way of knowing if
“Bill” is a trusted new member of the community, or some predator that
snuck
in. At least that’s what all the girls tell me when I try internet
dating.

Somewhere there NEEDS to be a trusted, authoritative list of who’s who.

I guess that a host could contact all of their already trusted friends and
ask if they know “Bill”. If they do and Bill is trusted this host could
just add him to it’s list of trusted friends. But as the trusted network
gets bigger and bigger, this becomes unreasonable pretty fast.

“Kevin Stallard” <> kevin@ffflyingrobots.com> > wrote in message
news:apcnk2$lng$> 1@inn.qnx.com> …

“Kevin Stallard” <> kevin@ffflyingrobots.com> > wrote in message
news:apbri7$lla$> 1@inn.qnx.com> …
Well, for now, yes…but this is something we are working on moving
away
from. This would remove an additional dependancy from the application
if
we
didn’t have to care about node names.


Let me clarify this a little bit.

Node names are still extremely useful. It helps you identify which
physical
piece of hardware to which you are talking. Very important when it
comes
to
diagnostics. Robotics it is especially important. In the robotics
case,
you may want to introduce a new machine w/o having to tell everyone else
its name.

I don’t think a global name locator service is the answer, but the
ability
to discover who is there and what resources they have is. I really
think
you guys hit a home run when you decided to have servers show up in the
pathname space. Yeah…when you query for this info there are a lot of
stat()'s going on, but this should only occur during initial discovery.
I’m
not sure its that big a deal as when you know what services are there,
you
connect to them usuall for the lifetime of what ever operation you are
performing.

Anyone knows that if you are going to access a file, you don’t open and
close, then re-open it for every record.

We just need to be able to broadcast a “tell me who you are” message to
currently running nodes and get responses on demand. The broadcast
idea
will work, but like you said xtang, it is going to result in a lot of
chatter. It would be nice if remote qnets could respond to a “tell my
who
you are” broadcast.

Kevin



Kevin

“Xiaodan Tang” <> xtang@qnx.com> > wrote in message
news:apbrf9$n7s$> 1@nntp.qnx.com> …
Kevin,

On a perticular system, don’t you always KNOW all your node names?
You
know
you can
talk to a node even if it is not show up under /net, do you?

-xtang

Kevin Stallard <> kevin@ffflyingrobots.com> > wrote in message
news:apbp18$ipj$> 1@inn.qnx.com> …

On the other hand, the undocumented “broadcast=” option, would
make
it
chatty, but
somewhat reliable (on node being show up under /net).

Let me point this out, there is NO reliable way to make “all
node
show
up”
from the design
view of QNET. What if QNET is based on tcpip, do you want to
know
ALL
the
nodes
on the otherside of earth is come and go? What if some node is
on
a
flaky
link, and not
broadcast reachable?

The TCPIP part of this is understandable. That would be too hard
to
do,
but
it is unlikley that we would do what we want using TCPIP as it
would
be
local LAN only (TCPIP has too much overhead, and too much has to
be
done
to
make it dynamic (have to use DHCP, etc). This is how we intend to
use
it
(local LAN only)…

Thanks for your feedback xtang…it’s appreciated.

Kevin

This will only work on some special condition (a LAN, reasonable
reliable
media).

-xtang


Kevin

“Xiaodan Tang” <> xtang@qnx.com> > wrote in message
news:apaapi$lvb$> 1@nntp.qnx.com> …

Kevin Stallard <> kevin@ffflyingrobots.com> > wrote in message
news:apa9o0$r0s$> 1@inn.qnx.com> …
Hey xtang…how’s things goin?

I want to be able to start a server on any node and be
able
to
find
it
w/o
knowning the name of the node on which it is residing.
This
would
allow

This is what name_attach(name, NAME_ATTACH_FLAG_GLOBAL)
and name_open(name, NAME_ATTACH_FLAG_GLOBAL) suppose
to do.

Unfortunatly this is not true right now. We will have a
“Global
Name
Services”
manager to address this.

-xtang

dynamic shifting of processes and the dynamic discovery of
them.

I need a way to find out the names of the nodes on the
network
so
I
can
query each one…

Thanks,
Kevin
“Xiaodan Tang” <> xtang@qnx.com> > wrote in message
news:apa9ik$l7b$> 1@nntp.qnx.com> …
No, netmgr_ctl() won’t help. In fact, there is no way in
QNET
to
re-learn
the existing node.
Node names under /net is not that “reliable” anyway.

It might be interesting to know why you want to do this?
and
maybe
there
are
otherways
to do it.

-xtang

Kevin Stallard <> kevin@ffflyingrobots.com> > wrote in
message
news:ap9ieq$337$> 1@inn.qnx.com> …
I found this function mentioned in the QNET Networking
docs,
netmgr_ctl(),
but it doesn’t seem to exist in the library reference.
Would
netmgr_ctl()
help me out with this?

Thanks
Kevin

“Kevin Stallard” <> kevin@ffflyingrobots.com> > wrote in
message
news:ap9gpk$1e2$> 1@inn.qnx.com> …
Hi,

I’d like to be able to send a message to io-net/qnet
to
force
it
to
query
for existing nodes running qnet. I am going to be
looking
for
server
names
by scanning the /net directory, and have noticed
that
unless
I
know
a
name
of a node, I can’t get qnet to display until it
decides
to
take
a
look
and
see who is there.

Is it possible?

Thanks
Kevin




















\

Xiaodan Tang <xtang@qnx.com> wrote:

However, using broadcast increase the “time of undeterminable” (which
against the “real time”).

In all honesty, you can’t even contemplate reatime/determinism until
AFTER you have established communications. So, with that in mind the
ability to identify a node that is present shouldn’t be considered a
realtime operation, for all the reasons you mention. That does not
mean that this isn’t useful, I just think you are perhaps a bit too
concerned about being able to get a response from all nodes within a
reasonable time frame. What is important is that in most cases you
will get a response in a reasonable time, but the application will
be the best judge of what is reasonable and what is not.

I think the best solution is a combination of what some others have
suggested.

1a) periodic “I’m here!” broadcasts, time period configurable on the
command line, with a default of disabled
1b) devctl() capability to change this at runtime
(this would allow for a chatty node to be told to shut up)
2) devctl() capability to send an “I’m here!” broadcast on demand
3) devctl() capability to request a “new host discovered” pulse

You could implement the on demand broadcast as a one-off, or the
send multiple thingie that you mentioned.

With these you now enable a developer to make the decisions that are
most important to thier application. Is the trade-off of a chatty
network worth having new nodes appear in a more timely manner worth
it? Depends on the application. If the application does not want
to rely on that, then it could force a broadcast.

QSS isn’t required to solve ALL our realtime issues, just to give us
the tools required to do so ourselves. You need to remember that
when you see too many sceanrios where things might not be reliable.


And of cause, if QNET is riding on IP and on something like 2 different IP
subnet, you won’t be able to announce/lookup global services cross subnet.

I’ve already mentioned that without placing the entire network behind
a firewall (which pretty much forces it to be a LAN) makes the use of
QNET over IP pretty much useless.

The advantage we had with FLEET was that we didn’t have node names,
and it was always possible to query the OS for the max number of
licensed nodes and then essentially poll them. This was possible
because we could pre-determine the node numbers that could exist in
the network, and that allowed us to generate messages for all nodes
whether they were present or not. Was it deterministic? No way.
Was it reliable? Nope. It did give the information we wanted most
of the time… and if a node wasn’t there, then there isn’t any
“realtime” aspect of communicating with it, is there?
QNX4 was at a disadvantage since it didn’t have threads, which could
make the timeout’s pretty brutal when an attempt to talk to an
offline node was made. With threading in QNX6 we can more easily
work around this to prevent the “poll” program from hanging.

Cheers,
Camz.