Armin Steinhoff wrote:
Robert Craig wrote:
Yes… “Transparent to software” implies a lot of things, most of them
untrue. With RapidIO, once you’ve configured things (which, of
course, ISN’T transparent), then you can do most things without the
rest of the software having to be aware of the fact that it’s
communicating through RapidIO.
IMHO … initialisation for the chip-to-chip communication is the task
of the firmware, this is not a task for the OS.
I think that you may be missing out on the whole philosophy behind the
RapidIO technology. Is PCI enumeration, configuration and operation a
job for firmware? RapidIO is designed to handle everything from
chip-to-chip up to board to board in the backplane environment. It
makes more sense to think of it as a combination of networking and
device bus (e.g. processor-to-processor using messaging, or device in
the sense that you can have, for example, a “dumb” Ethernet MAC that
acts as a RapidIO endpoint that maps it’s buffers and configuration
space into a processors address space). Given the dynamic nature of
RapidIO, enumeration and configuration are usually done by the OS to
determine what’s in the system and how to use it. (As an aside, even
for processor-to-processor you still need some sort of efficient
protocol to handle your communications.)
Essentially, RapidIO as a spec offers a number of mechanisms for hosts
to communicate with each other (a lot of this can be read from the
specs at > www.rapidio.org> ). The key features are large bandwidths with
low CPU usages and guaranteed (in hardware) message delivery. It’s
very different from PCI / PCI-Express in that it’s a peer-to-peer,
message based transport layer, so you get a lot of flexibility in
terms of topology. It also has a number of different PHYs (serial and
Parallel. Serial RapidIO is being made available on PQ-III chips from
Freescale in the very near term).
These include (almong others):
Maintenance transactions: low level register access transactions for
configuration of such elements as switches and other endpoints.
Doorbells: Small (16 bit) messages (Think Neutrino “pulse”)
RapidIO messages: Discrete message based transport allowing a 4K
message size. Gives you a full network communication capability
between end points. (Kind of likeEthernet with guaranteed delivery
and lower CPU usage).
Non-coherent I/O: Memory mapped interfaces providing a couple of
different transport mechanisms in which reading / writing to a block
of memory results in reading / writing a remote hosts memory (and
hence the “transparent” to software after it’s been set up.)
Globally Shared memory: (Not yet implemented in current hardware, to
my knowledge)… Cache coherent ops for memory / OS. Think SMP over
RapidIO…
QNX will be providing a library giving access to all of these
transport mechanisms including QNET over RapdidIO messages or
non-coherent I/O.
That means QNET on RapidIO makes just sense for a board-to-board
communication if RapidIO is used as a serial backplane?
Not really. It’s just as useful if you want to have a consistent
mechanism for communications amongst a cluster of processors on a single
board. You can then, of course, interconnect multiple cards over a
RapidIO backplane and have a cluster of clusters transparently
interconnected using exactly the same communication infrastructure with
no specific knowledge of this required at the application level. The
system will just see “X” processors that an application can be
seamlessly distributed over (where X can increase as new boards are added).
Of course, having transparent communication capability is pretty much
old-hat for anyone using QNET anyway, but it’s nice to know that the
concept fits so well with some of the more leading edge technologies
being introduced today.
And why is RapidIO so exiting for QSSL ?
Backplane communication between processor boards isn’t new …
Backplane communication isn’t new, but using a technology that does it
with more versatility, higher speeds, lower CPU utilization, lower
software complexity and lower cost than an existing technology does have
advantages… Intel also feels that there’s a compelling technology
area to be addressed here if you look at their PCI-Advanced Switching
initiative.
Processor communication is just one of the features of RapidIO and the
new breed of interconnects coming out. Best way to think of this is to
look at some of RapidIO’s competitors. For example: Infiniband,
PCI-Express, HyperTransport, Gigabit Ethernet, PCI-Advanced Switching.
Some of the benchmarks that we’ve done in comparison with Gigabit
Ethernet on the same chip show pretty impressive results (a factor of
2.5 times the thoughput using QNET (a la TDP) for the same CPU usage)
But Gigabit Ethernet is compared to RapidIO a WAN > 
The minimum packetsize of RapidIO is 256 bytes … Gigabit Ethernet is
using 4k … that comparison make no sense for me.
Absolutely, GigE ca be used as a WAN technology (although I still think
of it as more of a LAN technology ). So why do so many people want
to put it into a backplane? Interconnect technologies like RapidIO and
PCI-AdvancedSwitching are emerging to help address issues with these
areas as distributed processing becomes more and more important and
things like versatility, cost, software complexity, speed, CPU
utilization etc. etc. etc. become higher forerunners. Ethernet is a
great LAN technology, but let’s face it… Is that level of complexity
really needed for a backplane? That’s the comparison area that we’re
addressing.
The messaging capability built into QNET (TDP) melds very cleanly with
these new interconnect technologies and allows applications to take
advantage of them very easily and very cleanly. And again, there’s much
more to RapidIO than just communications.
The newer interconnects being developed are seen as being enablers to
denser and more interesting distributed computing systems (among other
things) and, with distributed computing being such a key element of
Neutrino, we are, of course, excited about being able to support them so
cleanly.
(OK… That was my marketing line for the week… Can’t do that too
often or I’ll get taken away from writing code (:-o)).
Robert.
Robert Craig
P.S. Suprisingly enough, we do actually have something very similar
to QNET over PCI. It’s used by a number of customers.
Backplane communication between CPUs … not new.
Regards
Armin
Armin Steinhoff wrote:
John Nagle wrote:
Armin Steinhoff wrote:
Is someone able to explain what is special with the QNX support of
RapidIO?
The text below comes from a FAQ of the RapidIO org:
Q: How is software development impacted by the RapidIO technology?
A: Software development is independent of the RapidIO interconnect
architecture. RapidIO technology is transparent to the existing
software base. To software, the RapidIO interconnect can look just
like a traditional microprocessor and peripheral bus. RapidIO
technology also bridges easily to PCI and PCI-X. These features
allow users to mix legacy software and PCI chips with RapidIO
chips, and without requiring special device drivers.
So what is QSSL doing with that ‘transparent’ RapidIO stuff??
Any ideas?
QNET over RapidIO, apparently:
RapidIO is a replacement for parallel system buses like PCI a.s.o.
It is designed for chip-to-chip communication and can be used as a
serial backplane bus. And … it ‘is transparent to the existing
software base’.
So … makes QNET over PCI any sense for you ?
Regards
Armin
http://www.qnxzone.com/?q=node/view/335
This makes sense for CPU to CPU communication in big
routers. RapidIO as a peripheral interconnect seems
to have been eclipsed by PCI Express, FireWire, etc.
John Nagle