[x-posted from apps]: Re: fs-qnx filesystem questions

“Operating System Tech Support” <os@qnx.com> wrote in message
news:9r1kou$kcd$1@nntp.qnx.com

  1. How does it perform in terms of recovering from the system crash?
    How
    does it compare to UNIX filesystem? How about journalled filesystem?

It depends by what you mean as “recovering from a system crash”. Many of
the journalling filesystems, use journaling to restore/fix their
filesystem
Meta data and FS integrity but it does not do data recovery.

I don’t think that’s quite true. Unixes will lose files being written at the
time of crash, just like QNX will.

The problem is that many of the *NIX like FS’s is that they have very
agressive cache mechanisms to avoid disk access. This is great for
general
use, but if the system goes down, before those caches are flushed and
commited, you data is gone.

Hmm. That’s not really as much ‘aggressive caching’ as well designed buffer
cache system. In most modern Unixes buffer cache and VM are unified and OS
can automatically adjust buffer cache as usage pattern suggests. Granted,
that (unification) may have adverse effect on realtime, but in QNX they are
not only non-unified, but every devb-xxx has its own cache. That does not
help performance at all and does not help reliability as far as I can see.

In short, the QNX FS might not be as fast as some of the journaled FS/*NIX
but it’s very reliable.

It is reliable, but not really so much more reliable than modern UFS
variants to justify such poor piss performance anymore. I spent lot of
efforts trying to get maximum performance of it and results were at best
modest.

Truth is, QNX4 FS was good enough in its days, but by now is simply too damn
old. It does not reflect any of developments in this area of past decade
(which in computer industry means it is relict). Write performance is
especially bad and it also seems to suffer severely from cache saturation
effects. Under some conditions throughput can drop as low as 200kb/sec when
copying a large file within same drive, and that is using DMA mode of EIDE
(!)

It is really sad part of QNX (along with VM) and I suggest you guys stop
using excuses already and do something about it. Claims of ‘leading realtime
OS’ and ‘most advanced architecture’ are starting to look somewhat
ridiculous.

  • igor

P.S. cross-posted to OS group where it belongs

Hello Igor

I agree that several aspects of V6 still require some improvements.
However, why do you think that it is the file system?

I’m thinking that the drivers are where most of the improvements need to go.

QSSL, these points should not be taken lightly. I’m finding that almost
across the board QNX V4 is noticably faster that the corrisponding V6
equivlent. V6 compile times are 4 to 6 times slower. Graphics are slower.
Disk I/O is slower. The packager is an amuseing idea but should never be
used in an actual running system.

I am curious, what are QSSL’s own benchmarks showing in these areas?


Bill Caroselli – 1(530) 510-7292
Q-TPS Consulting
QTPS@EarthLink.net


“Igor Kovalenko” <kovalenko@home.com> wrote

Truth is, QNX4 FS was good enough in its days, but by now is simply too
damn
old. It does not reflect any of developments in this area of past decade
(which in computer industry means it is relict). Write performance is
especially bad and it also seems to suffer severely from cache saturation
effects. Under some conditions throughput can drop as low as 200kb/sec
when
copying a large file within same drive, and that is using DMA mode of EIDE
(!)

It is really sad part of QNX (along with VM) and I suggest you guys stop
using excuses already and do something about it. Claims of ‘leading
realtime
OS’ and ‘most advanced architecture’ are starting to look somewhat
ridiculous.

  • igor

Bill Caroselli (Q-TPS) wrote:

Hello Igor

I agree that several aspects of V6 still require some improvements.
However, why do you think that it is the file system?

I’m thinking that the drivers are where most of the improvements need > to go.

QSSL, these points should not be taken lightly. I’m finding that
almost across the board QNX V4 is noticably faster that the
corrisponding V6 equivlent. V6 compile times are 4 to 6 times slower. > Graphics are slower.
Disk I/O is slower. The packager is an amuseing idea but should never > be used in an actual running system.

Don’t use the packer installer in a productive system … it shuts down
the packet files system and restarts it. Well … and this happens in a
multi user system.

Is there a way not to use the package file system in an embedded
configuration ??

Armin



I am curious, what are QSSL’s own benchmarks showing in these areas?


Bill Caroselli – 1(530) 510-7292
Q-TPS Consulting
QTPS@EarthLink.net


“Igor Kovalenko” <> kovalenko@home.com> > wrote

Truth is, QNX4 FS was good enough in its days, but by now is simply too

damn

old. It does not reflect any of developments in this area of past decade
(which in computer industry means it is relict). Write performance is
especially bad and it also seems to suffer severely from cache saturation
effects. Under some conditions throughput can drop as low as 200kb/sec

when

copying a large file within same drive, and that is using DMA mode of EIDE
(!)

It is really sad part of QNX (along with VM) and I suggest you guys stop using excuses already and do something about it. Claims of 'leading
realtime

OS’ and ‘most advanced architecture’ are starting to look somewhat
ridiculous.

  • igor
    \

Hello all,

“Igor Kovalenko” <kovalenko@home.com> wrote in message
news:9r1psq$sdk$1@inn.qnx.com

“Operating System Tech Support” <> os@qnx.com> > wrote in message
news:9r1kou$kcd$> 1@nntp.qnx.com> …

It is really sad part of QNX (along with VM) and I suggest you guys stop
using excuses already and do something about it. Claims of ‘leading
realtime
OS’ and ‘most advanced architecture’ are starting to look somewhat
ridiculous.

I have to agree with Igor about the above.

QNX could use a journalling filesystem. Advanced architecture and features
in a FS design would be:

  • journalling
  • meta data support
  • live snapshot (freeze a view of the filesystem, for backup, while other
    applications
    can continue modifying/writing).
  • for a whole-system advanced design, having a journalling cache in flash
    memory
    would bring substantional performance (think of write behind merging of
    commits)
    and robustness (prevent data loss in case of unexpected system shutdown).

That being said, the current QNX filesystem is very robust. It’s just a
previous
generation filesystem (which is seldomly a system design constraint).

Regards, Leon Woestenberg.

“Operating System Tech Support” <os@qnx.com> wrote in message
news:9r3ugc$4qf$1@nntp.qnx.com

“Igor Kovalenko” <> kovalenko@home.com> > wrote in message
news:9r1psq$sdk$> 1@inn.qnx.com> …

I don’t think that’s quite true. Unixes will lose files being written at
the
time of crash, just like QNX will.

I wasn’t suggesting that the QNXFS was immune to data lost, just the
window
of opportunity is smaller (due to the way writes are done).

The way writes are done is optimized in most (if not all) other systems as
well.

The Linux Filesystem is a good example of “agressive caching”, in fact
it’s
much “faster” than many of the UFS implementations.

If you mean ‘classic UFS’ by ‘many of the UFS’, then yes. If you consider
more modern variants, like Solaris UFS or FreeBSD UFS+softupdates, I doubt
that statement would hold.

You’ll notice that the
disk isn’t even touched many times, which IMHO is “agressive caching”.

Modern computer design is all about ‘aggressive caching’. CPU has 2 or 3
levels of caching, speculative branch perdiction unit, etc. Disk subsystems
follow the same path (read-ahead/write-behind) because it is the only
realistic way to get the best of hardware performance. Now whether your FS
can afford ‘aggressive caching’ or not, is big question. If the design
allows fast and reliable recovery, may be yes.

Also, there are hardware designs which make aggressive caching more safe.
For example, some harddrives (expensive SCSI ones) have feature which allows
them to use rotational energy of drive to finish writing in case of sudden
power loss. Not to mention UPS devices (isn’t it shame QNX has no drivers
for any of them?)

To summarize my point, there is nothing wrong with aggressive caching if
used properly. Good OS design would allow users to have aggressive caching
when it is welcomed and don’t use it when it is not. Problem here is, QNX
does not provide much capability to use it even when we want it.

The
fact that each devb-xxx has it’s own cache is a side effect of having
modularity and a micro-kernel.

You’re not saying it is good side effect, are you? Micro-kernel and
modularity by themselves do not mandate separate cache for each driver. Mach
OS is microkernel, and it does not have this problem. I would say it is side
effect of evident lack of respect/attention to this part of OS functionality
on QSSL side.

I don’t think a VFS in the kernel is very
good design, and having a VFS layer outside is simply going to add more
layers of communication (ie. slower).

We figured that much.

I think you’re comparing apples and oranges, you can’t use UFS completely
in
a vacum - the OS and it’s kernel (monolithic) are going to have major
effects on speed.

Did not I said that myself in another posting?

I don’t think “age” is a very good argument in this domain. Many process
controllers and other systems are still build around i386, i486 and very
low
speed processors. I would not consider them to be "relict"s.

I would not consider them ‘most advanced architecture’ either, would you?

To assume that we’re not doing “something about it” is a bad assumption.

Good to hear. Actually I know you do, but adding some gasoline into flame
helps it burn faster =8-}

The problem is QNX4FS is a proven FS, and can’t just be surplanted with
the
next “best thing” or next industry buzzword compliant term (ie. Journaling
FS’s are not that great for recovery). One last point, speed is not what
a
RTOS is about - it is about determinism.

You aren’t saying QNX4FS is ‘deterministic’, are you? There are some
experimental works in that direction (guaranteed time-bounds on I/O
requests), but AFAIK none of them are happening in QSSL. Which is a shame.

  • igor

To summarize my point, there is nothing wrong with aggressive caching
if
used properly. Good OS design would allow users to have aggressive
caching
when it is welcomed and don’t use it when it is not. Problem here is,
QNX
does not provide much capability to use it even when we want it.

It is certainly true that there is nothing wrong with aggressive caching
for general purpose computing, in fact, for general purpose computing,
everything is “right” with it; however, QNX is not targeted at general
purpose computing, it is targeted at systems that must demonstrate high
availability, and high predictability, and typically do little writing
to disk (many systems never write to disk at all). What is driving QSSL
is customer demand (paying customer demand that is), if the guys who are
paying for QNX wanted fast writes, my guess is it would be a priority.

You’re not saying it is good side effect, are you? Micro-kernel and
modularity by themselves do not mandate separate cache for each
driver. Mach
OS is microkernel, and it does not have this problem. I would say it
is side

Mach OS is not a microkernel. Simply naming a kernel a microkernel
doesn’t make it one.

I don’t think “age” is a very good argument in this domain. Many
process
controllers and other systems are still build around i386, i486 and
very

low

speed processors. I would not consider them to be "relict"s.


I would not consider them ‘most advanced architecture’ either, would
you?

What is the “most advanced” in the domain of process control, is seldom
equal to what is the most advanced in other domains. We manufacture a
QNX based process controller. It is quite advanced compared to the
current state of the art from our competitors, and it uses QNX4. If we
were using QNX6 (with the filesystem to which you direct the term
“relic”) we would undoubtedly have the most advanced process controller
on the market. It will take years before we have a QNX6 version
available. In process control, reliability is the ultimate technology
(not gee-whiz features).

Rennie Allen wrote:

Mach OS is not a microkernel. Simply naming a kernel a microkernel
doesn’t make it one.

Ohh… That’s quite a statement. What makes ‘microkernel’ is modular
structure with modules running outside of kernel memory space. Mach3 is
exactly that (it even allows user-land pager daemon). It is also based
on message passing, only more sophisticated than QNX. Mach
message-passing, unlike QNX is secure (has permissions attached to
‘ports’) and works in 0-copy mode when possible, using copy-on-write
mechanism. And on top of that it is binary-compatible with Unix. Which
is what killed it, ironically since it was competing against monolithic
kernels and they were faster anyway, especially on the processors of
those days.

Nevertheless, most modern Unix variants were influenced heavily by ideas
introduced in Mach. Too bad QNX was not (yet).

What is the “most advanced” in the domain of process control, is seldom
equal to what is the most advanced in other domains. We manufacture a
QNX based process controller. It is quite advanced compared to the
current state of the art from our competitors, and it uses QNX4. If we
were using QNX6 (with the filesystem to which you direct the term
“relic”) we would undoubtedly have the most advanced process controller
on the market. It will take years before we have a QNX6 version
available. In process control, reliability is the ultimate technology
(not gee-whiz features).

Well, if you watch QNX corporate video, they don’t really say ‘most
advanced OS for process control’. They say something more in line with
‘the most advanced architecture in the world’.

Also i would challenge calling good filesystem performance a ‘gee-whiz’
feature. May be for you it is, but not for most of us. We (Motorola) use
QNX6 on ‘high-end’ server-type system and it needs to read and write
data efficiently. We have bypassed writing problems by using complicated
queueing, but there’s nothing we can do with reading. It gives us
3mb/sec on UltraWide SCSI and that translates directly into slow
startup, because we need to read large database at startup. Slow startup
translates into longer downtime which is hardly tolerable in telecom
industry. And you bet we did tell that to our sales rep (couple years
back and kept telling ever since). See how things look differently from
different perspective?

  • igor

“Igor Kovalenko” <kovalenko@home.com> wrote in message
news:9r1psq$sdk$1@inn.qnx.com

I don’t think that’s quite true. Unixes will lose files being written at
the
time of crash, just like QNX will.

I wasn’t suggesting that the QNXFS was immune to data lost, just the window
of opportunity is smaller (due to the way writes are done).

Hmm. That’s not really as much ‘aggressive caching’ as well designed
buffer
cache system. In most modern Unixes buffer cache and VM are unified and OS
can automatically adjust buffer cache as usage pattern suggests. Granted,
that (unification) may have adverse effect on realtime, but in QNX they
are
not only non-unified, but every devb-xxx has its own cache. That does not
help performance at all and does not help reliability as far as I can see.

The Linux Filesystem is a good example of “agressive caching”, in fact it’s
much “faster” than many of the UFS implementations. You’ll notice that the
disk isn’t even touched many times, which IMHO is “agressive caching”. The
fact that each devb-xxx has it’s own cache is a side effect of having
modularity and a micro-kernel. I don’t think a VFS in the kernel is very
good design, and having a VFS layer outside is simply going to add more
layers of communication (ie. slower).

It is reliable, but not really so much more reliable than modern UFS
variants to justify such poor piss performance anymore. I spent lot of
efforts trying to get maximum performance of it and results were at best
modest.

I think you’re comparing apples and oranges, you can’t use UFS completely in
a vacum - the OS and it’s kernel (monolithic) are going to have major
effects on speed.

Truth is, QNX4 FS was good enough in its days, but by now is simply too
damn
old. It does not reflect any of developments in this area of past decade
(which in computer industry means it is relict). Write performance is
especially bad and it also seems to suffer severely from cache saturation
effects. Under some conditions throughput can drop as low as 200kb/sec
when
copying a large file within same drive, and that is using DMA mode of EIDE
(!)

I don’t think “age” is a very good argument in this domain. Many process
controllers and other systems are still build around i386, i486 and very low
speed processors. I would not consider them to be "relict"s.

It is really sad part of QNX (along with VM) and I suggest you guys stop
using excuses already and do something about it. Claims of ‘leading
realtime
OS’ and ‘most advanced architecture’ are starting to look somewhat
ridiculous.

To assume that we’re not doing “something about it” is a bad assumption.
The problem is QNX4FS is a proven FS, and can’t just be surplanted with the
next “best thing” or next industry buzzword compliant term (ie. Journaling
FS’s are not that great for recovery). One last point, speed is not what a
RTOS is about - it is about determinism.

-Adam

Igor Kovalenko wrote:

Also i would challenge calling good filesystem performance a ‘gee-whiz’
feature. May be for you it is, but not for most of us. We (Motorola) use
QNX6 on ‘high-end’ server-type system and it needs to read and write
data efficiently. We have bypassed writing problems by using complicated
queueing, but there’s nothing we can do with reading. It gives us
3mb/sec on UltraWide SCSI and that translates directly into slow
startup, because we need to read large database at startup. Slow startup
translates into longer downtime which is hardly tolerable in telecom
industry. And you bet we did tell that to our sales rep (couple years
back and kept telling ever since). See how things look differently from
different perspective?

Just out of interest, what made you choose QNX6 for a high-end
server-type system?

Tom

Operating System Tech Support wrote:

Ohh… That’s quite a statement. What makes ‘microkernel’ is modular
structure with modules running outside of kernel memory space. Mach3 is
exactly that (it even allows user-land pager daemon). It is also based
on message passing, only more sophisticated than QNX. Mach
message-passing, unlike QNX is secure (has permissions attached to
‘ports’) and works in 0-copy mode when possible, using copy-on-write
mechanism. And on top of that it is binary-compatible with Unix. Which
is what killed it, ironically since it was competing against monolithic
kernels and they were faster anyway, especially on the processors of
those days.

Having modules running outside of kernel space does not make an OS a
micro-kernel. That is simply a side-effect.

Will you educate us please? What does make a microkernel? Size? To me
size is not really serious argument. And before someone pops up to say
‘QNX microkernel is only 40k’ I will remind that you can’t get those 40k
on their own. They come ‘bundled’ with process manager and share address
space with it. So it is about 300K. And that’s with very rudimentary VM
subsystem.

Also i would challenge calling good filesystem performance a ‘gee-whiz’
feature. May be for you it is, but not for most of us. We (Motorola) use
QNX6 on ‘high-end’ server-type system and it needs to read and write
data efficiently. We have bypassed writing problems by using complicated
queueing, but there’s nothing we can do with reading. It gives us
3mb/sec on UltraWide SCSI and that translates directly into slow
startup, because we need to read large database at startup. Slow startup
translates into longer downtime which is hardly tolerable in telecom
industry. And you bet we did tell that to our sales rep (couple years
back and kept telling ever since). See how things look differently from
different perspective?

Why not write a driver to support you “high-end server-type system”, if the
generic ones don’t perform the way you’d like. The resources you spent on
developing “complicated queueing” for only 1/2 of a solution seems poorly
spent.

I just love to hear this. How did not we think about that on our own, I
wonder…
Where’s disk driver DDK Adam? Last time we asked, there was no DDK and
no documented interfaces and sample source was not available either.

As for poorly spent resources, we needed queueing for multiple reasons,
some of them outside of ‘disk performance’ problem. So we felt our
resources would be better spent that way than on developing SCSI driver
for an already ‘supported’ chipset. The problem with performance is due
to QNX driver not supporting sync/wide operation of SCSI (it works in
ancient async mode). We hoped/assumed that it would be eventually fixed
(and at some point we were told it would be), but 2 years passed and it
is still not there.

Chipset in question is LSI/Symbios 53C8XX and it is not something
esoteric. It is second most wiedly used SCSI chipset after Adaptec and
it is practically the only one available in CompactPCI ‘mezzanine’
form-factor. I think it is really short-sighted for QNX to not support
it properly, given apparent importance of CompactPCI platform for their
markets.

  • igor

Just out of interest, what made you choose QNX6 for a high-end
server-type system?

Just what I was thinking :wink:

Hi Armin…

Armin Steinhoff wrote:

Don’t use the packer installer in a productive system … it shuts down
the packet files system and restarts it. Well … and this happens in a
multi user system.

Is there a way not to use the package file system in an embedded
configuration ??

What do you mean by this question? What do you call an ‘embedded
system’?

In an stand alone, autonomous? embedded system, you write your *.ifs
which becomes your .boot. No package file system required. Either this,
or I completely did not get your point above!

Regards…

Miguel.

Armin

my opinions are mine, only mine, solely mine, and they are not related
in any possible way to the institution(s) in which I study and work.

Miguel Simon
Research Engineer
School of Aerospace and Mechanical Engineering
University of Oklahoma
http://www.amerobotics.ou.edu/
http://www.saic.com

Hi…

Rennie Allen wrote:


everything is “right” with it; however, QNX is not targeted at general
purpose computing, it is targeted at systems that must demonstrate high
availability, and high predictability, and typically do little writing
to disk (many systems never write to disk at all). What is driving QSSL

Agree. Our embedded control applications (running in both QNX4 and QNX6)
do not even have a disk to write to, and they are very robust because in
every sense, QNX is a robust OS. In many ways it is good that QNX is not
targeted at general purpose computing…

<…>

on the market. It will take years before we have a QNX6 version
available. In process control, reliability is the ultimate technology
(not gee-whiz features).

Once again, agree. We have some robotic systems that have been
continuously running for months on end. It is a good thing that the OS
behaves in such a predictable and reliable way.

Regards…

Miguel.

my opinions are mine, only mine, solely mine, and they are not related
in any possible way to the institution(s) in which I study and work.

Miguel Simon
Research Engineer
School of Aerospace and Mechanical Engineering
University of Oklahoma
http://www.amerobotics.ou.edu/
http://www.saic.com

Ohh… That’s quite a statement. What makes ‘microkernel’ is modular
structure with modules running outside of kernel memory space. Mach3
is
exactly that (it even allows user-land pager daemon).

What makes a microkernel is a philosphy of simplicity of design and
minimalism. One could easily construct a massively over-engineered
kernel that demonstrated high modularity and ran many services in user
space, and this would not be a microkernel.

“user land pager daemon” != “user land cache management”.

It is also based
on message passing, only more sophisticated than QNX.

Message passing is (by definition of a microkernel message passing
architecture) implemented in the kernel.

“more sophisticated” == “more code” == “more complex”

If you don’t subscribe to the QNX philosophy of simplicity, then perhaps
Mach is the O/S you should be using ?

Mach
message-passing, unlike QNX is secure (has permissions attached to
‘ports’)

Oh yeah, just what I need, security data for IPC on my iPaq, or my
process controller :wink:

and works in 0-copy mode when possible, using copy-on-write
mechanism.

and what, exactly is the impact on scheduling latency (not to mention
footprint) for all this “cool” stuff ?

And on top of that it is binary-compatible with Unix. Which
is what killed it, ironically since it was competing against
monolithic
kernels and they were faster anyway, especially on the processors of
those days.

Hmmm, QNX didn’t die, I wonder why “superior” technology that was “open
source” “died”, when QNX didn’t ? Could it be that QNX actually met a
market need better than Mach ?

Nevertheless, most modern Unix variants were influenced heavily by
ideas
introduced in Mach.

Except for anything remotely related to microkernel architecture…

Too bad QNX was not (yet).

No, your right, QNX is heavily influenced by simplicity, and microkernel
architecture, not featuritis and Rube Goldbergism … Personally, I hope
it remains that way.

Well, if you watch QNX corporate video, they don’t really say ‘most
advanced OS for process control’. They say something more in line with
‘the most advanced architecture in the world’.

Often, marketing material isn’t designed to convey detailed information
about the target applications that a product is designed to address
(typically because marketing campaigns can encompass many revisions of a
product). I think that QNX6 is a very advanced architecture for the
market it is designed to address; I concede that the marketing material
does not clearly convey what the target market is; but I am also willing
to bet that QSSL sales/marketing rarely show up at customers who aren’t
in that target market. Those who read any marketing material in a
vacuum deserve what they get.

Also i would challenge calling good filesystem performance a
‘gee-whiz’
feature. May be for you it is, but not for most of us.

No. In my experience (15 years of QNX at 5 different companies in 3
different industries), a high performance filesystem, is a ghee-wiz
feature for most of the customers in the QNX target market (it always
has been). Many of the customers don’t even need a filesystem, those
that do, typically need a compact and robust filesystem, above a high
performance filesystem. As proof, I submit that QNX’s largest
competitor (and dominant player in the market that QSSL currently
operates in) is VxWorks which (for all intents and purposes) has no
filesystem; and certainly not a high performance filesystem by anyones
definition of the term…

IMO there are a minority of vocal advocates of a HPFS and IBC that make
a lot of noise, while the majority of users are FDAH without these, and
see no reason to get involved in these discussions. I just want to make
it clear that there are those of us out here for whom a HPFS is very low
on the priority list.

We (Motorola) use
QNX6 on ‘high-end’ server-type system and it needs to read and write
data efficiently. We have bypassed writing problems by using
complicated
queueing,

QNX is a real-time operating system. Even the most novice real-time
system designer knows the rule #2 of real-time system design: "no queues

  • certainly not complicated queues". Therefore, if you solved your
    applications “problems” by using queues, then I submit that your
    application is not hard real-time. Perhaps QNX is not the best choice
    for your application.

but there’s nothing we can do with reading. It gives us
3mb/sec on UltraWide SCSI and that translates directly into slow
startup, because we need to read large database at startup. Slow
startup
translates into longer downtime which is hardly tolerable in telecom
industry. And you bet we did tell that to our sales rep (couple years
back and kept telling ever since).

I’m not going to get into what QSSL sales told you or didn’t tell you
(since I haven’t a clue about that subject), I am simply voicing my
opinion, that I don’t believe a high performance filesystem, or an
integrated buffer cache is a high priority for those in our industry
(process control), or for most current QNX customers (including telecom

  • haven’t seen any filesystems on routers lately…).

Would I like it if QNX had these features (at no cost in scheduling
latency or base footprint), certainly; but I wouldn’t be willing to
sacrifice very much to get them. QNX6 is starting to look very good in
the areas that I care about, but it isn’t quite there yet, and I
represent the traditional market; IMO they need to serve the traditional
market first, and then they can look at new markets (as long as it
doesn’t negatively impact the existing markets).

See how things look differently from
different perspective?

I am just trying to give you my perspective which is different to yours,
and (I believe) common to the majority of those who currently have
products based on QNX (and the QNX philosophy).

Hi,

Agree. Our embedded control applications (running in both QNX4 and QNX6)
do not even have a disk to write to, and they are very robust because in
every sense, QNX is a robust OS. In many ways it is good that QNX is not
targeted at general purpose computing…

But QSSL claims QNX 6 to be “extremely scalable”. I think that term covers a

whole lot of different systems in term of size/performance.

I think general purpose computing fits in somewhere in the “scalable” range.

Leon Woestenberg.

Rennie Allen wrote:

Just out of interest, what made you choose QNX6 for a high-end
server-type system?

Just what I was thinking > :wink:

At first i thought to keep you dying in curiosity, but then my mood
improved :slight_smile:
The short answer is, nothing made me choose it. Decision was made before
I came. As for reasons why that decision was made, it is long story. I
can tell you, there was very good salesperson @ QNX who ‘sold’ it to our
group. Too bad he’s not there anymore :wink:

Technically wise, we needed both realtime capabilities and
high-performance I/O (ethernet and disk). The project was rather
high-risk/low-budget, so we were given relative flexibility in choice,
otherwise we would have been forced to use something from ‘approved’
list. Also at the time the project was conceived, hardware platform was
still in development, peripherial boards did not exist (some were in
development, some were designed specifically for us). So we did not mind
to use ‘bleeding edge’ OS as well. We were using Neutrino 2 since its
first beta in late 1998 (which did not boot at all due to bug in that
same SCSI driver) and over those years many of bugs were found because
we used the system in rather ‘unusual’ (high load) way.

  • igor

Igor Kovalenko <Igor.Kovalenko@motorola.com> wrote:
[snip]

Well, if you watch QNX corporate video, they don’t really say ‘most
advanced OS for process control’. They say something more in line with
‘the most advanced architecture in the world’.

“Architecture, architecture, architecture,” that’s what I say :slight_smile:

Sorry, couldn’t resist!

Robert Krten, PARSE Software Devices +1 613 599 8316.
Realtime Systems Architecture, Consulting and Training at www.parse.com

“Igor Kovalenko” <Igor.Kovalenko@motorola.com> wrote in message
news:3BD61861.E2A9B1D9@motorola.com

Rennie Allen wrote:

Mach OS is not a microkernel. Simply naming a kernel a microkernel
doesn’t make it one.


Ohh… That’s quite a statement. What makes ‘microkernel’ is modular
structure with modules running outside of kernel memory space. Mach3 is
exactly that (it even allows user-land pager daemon). It is also based
on message passing, only more sophisticated than QNX. Mach
message-passing, unlike QNX is secure (has permissions attached to
‘ports’) and works in 0-copy mode when possible, using copy-on-write
mechanism. And on top of that it is binary-compatible with Unix. Which
is what killed it, ironically since it was competing against monolithic
kernels and they were faster anyway, especially on the processors of
those days.

Having modules running outside of kernel space does not make an OS a
micro-kernel. That is simply a side-effect.

Also i would challenge calling good filesystem performance a ‘gee-whiz’
feature. May be for you it is, but not for most of us. We (Motorola) use
QNX6 on ‘high-end’ server-type system and it needs to read and write
data efficiently. We have bypassed writing problems by using complicated
queueing, but there’s nothing we can do with reading. It gives us
3mb/sec on UltraWide SCSI and that translates directly into slow
startup, because we need to read large database at startup. Slow startup
translates into longer downtime which is hardly tolerable in telecom
industry. And you bet we did tell that to our sales rep (couple years
back and kept telling ever since). See how things look differently from
different perspective?

Why not write a driver to support you “high-end server-type system”, if the
generic ones don’t perform the way you’d like. The resources you spent on
developing “complicated queueing” for only 1/2 of a solution seems poorly
spent.

-Adam

“Rennie Allen” <RAllen@csical.com> wrote in message
news:64F00D816A85D51198390050046F80C9012A59@exchangecal.hq.csical.com

Ohh… That’s quite a statement. What makes ‘microkernel’ is modular
structure with modules running outside of kernel memory space. Mach3
is
exactly that (it even allows user-land pager daemon).

What makes a microkernel is a philosphy of simplicity of design and
minimalism.

That is very subjective definition since there is no formal way to define
‘simple’ or ‘minimalistic’. It is just what you like to think and don’t try
to sell it for anything more than that.

One could easily construct a massively over-engineered
kernel that demonstrated high modularity and ran many services in user
space, and this would not be a microkernel.

“user land pager daemon” != “user land cache management”.

Really?
“Pager daemon” = “VM subsystem”, which (when unified with buffer cache) =
“cache management”.

It is also based
on message passing, only more sophisticated than QNX.

Message passing is (by definition of a microkernel message passing
architecture) implemented in the kernel.

“more sophisticated” == “more code” == “more complex”

If you don’t subscribe to the QNX philosophy of simplicity, then perhaps
Mach is the O/S you should be using ?

I don’t subscribe to philosophy of blindly dismissing anything what lies
beyond something you’re used to. Mach was research project, exploring new
concepts of OS design. There would be nothing wrong with QNX using some
ideas from it. There are other research projects QNX would learn from, like
L4 (which is realtime u-kernel).

and works in 0-copy mode when possible, using copy-on-write
mechanism.

and what, exactly is the impact on scheduling latency (not to mention
footprint) for all this “cool” stuff ?

I don’t see how copy-on-write applied to message-passing relates to
scheduling latency.

And on top of that it is binary-compatible with Unix. Which
is what killed it, ironically since it was competing against
monolithic
kernels and they were faster anyway, especially on the processors of
those days.

Hmmm, QNX didn’t die, I wonder why “superior” technology that was “open
source” “died”, when QNX didn’t ? Could it be that QNX actually met a
market need better than Mach ?

QNX4 was operating in rather small ‘niche market’, for which it was
adequate. Mach was a research project on general OS design principles and to
say ‘QNX met market needs better’ is just nonsense.

Still, new Apple’s MacOS X is largely based on BSD and Mach. And they seem
to be doing fine.

Also i would challenge calling good filesystem performance a
‘gee-whiz’
feature. May be for you it is, but not for most of us.

No. In my experience (15 years of QNX at 5 different companies in 3
different industries), a high performance filesystem, is a ghee-wiz
feature for most of the customers in the QNX target market (it always
has been). Many of the customers don’t even need a filesystem, those
that do, typically need a compact and robust filesystem, above a high
performance filesystem. As proof, I submit that QNX’s largest
competitor (and dominant player in the market that QSSL currently
operates in) is VxWorks which (for all intents and purposes) has no
filesystem; and certainly not a high performance filesystem by anyones
definition of the term…

IMO there are a minority of vocal advocates of a HPFS and IBC that make
a lot of noise, while the majority of users are FDAH without these, and
see no reason to get involved in these discussions. I just want to make
it clear that there are those of us out here for whom a HPFS is very low
on the priority list.

I am getting sick of this kind of arguments. Reminds me very much the old
days when large chorus of people in comp.os.qnx (except Armin) argued to me
how unnecessary and unwanted would it be to have SMP and threads in QNX.
Until Dan Dodge came up and told them otherwise, lol.

‘It has always has been’ is very silly thing to hang on to. Industry changes
and players in the industry do so as well. If in the past none of major
players bothered to offer any embedded/realtime solution, then now we have
NT/Embedded and RTLinux pushing into traditional QNX market. If QNX does not
make any counter-moves, the balance may change just enough to kill them.
Yes, QNX has better realtime technology, but that’s not the only issue
considered when decisions are made. If general-purpose OSes are getting more
realtime capabilities, QNX needs to counter that with more general-purpose
features, especially as technology evolves and new opportunities arise.

QNX is a real-time operating system. Even the most novice real-time
system designer knows the rule #2 of real-time system design: "no queues

  • certainly not complicated queues". Therefore, if you solved your
    applications “problems” by using queues, then I submit that your
    application is not hard real-time. Perhaps QNX is not the best choice
    for your application.

We use queueing into circular buffers to write logs and billing information
(potentially coming from thousands of calls). Since those data are produced
by realtime components, we did not want those components to be held by
undeterministic I/O operations. Writing into queue is deterministic and then
there are multiple low-priority ‘dequeuers’, such as remote log readers (who
potentionally have different connection speed and) and flush daemon which
saves all that stuff to disk as CPU cycles permit.

If you’re so advanced realtime designer as to tell us that queues are bad
even without knowing our requirements, please educate me how would you
implement that thing better. Notice, the new ‘slogger’ manager in QNX6 is
basically similar thing. Our solution was just more flexible and scalable
(multiple queues and multithreaded manager).

I’m not going to get into what QSSL sales told you or didn’t tell you
(since I haven’t a clue about that subject), I am simply voicing my
opinion, that I don’t believe a high performance filesystem, or an
integrated buffer cache is a high priority for those in our industry
(process control), or for most current QNX customers (including telecom

  • haven’t seen any filesystems on routers lately…).

‘Telecom’ is not limited to routers. Our system is mobile phone switching
center.

Would I like it if QNX had these features (at no cost in scheduling
latency or base footprint), certainly; but I wouldn’t be willing to
sacrifice very much to get them. QNX6 is starting to look very good in
the areas that I care about, but it isn’t quite there yet, and I
represent the traditional market; IMO they need to serve the traditional
market first, and then they can look at new markets (as long as it
doesn’t negatively impact the existing markets).

See how things look differently from
different perspective?

I am just trying to give you my perspective which is different to yours,
and (I believe) common to the majority of those who currently have
products based on QNX (and the QNX philosophy).

I will just quote one QNX employee (who I will not name): ‘Anyone who’s
defending our disk I/O system is lying to themselves’.

  • igor

Rennie Allen wrote:

[ clip …]

What makes a microkernel is a philosphy of simplicity of design

I hope the ‘simplicity’ is not in the design of the QNX kernel :slight_smile:

and minimalism.
One could easily construct a massively over-engineered
kernel that demonstrated high modularity and ran many services in user
space, and this would not be a microkernel.

Is there a clear separation of the process manager and the QNX 6 kernel??

If not … where are then the differences compared to a ‘massively
over-engineered’ kernel ?? Are there any performance reasons ??

“user land pager daemon” != “user land cache management”.

[ clip …]
Message passing is (by definition of a microkernel message passing
architecture) implemented in the kernel.

“more sophisticated” == “more code” == “more complex”

Wrong … the ‘more sophisticated code’ should be handled by a partner
process (running in its own protected address space).

Such a configuration is compliant to the micro kernel design … so
there are no excuse for not providing more sophisticated services.

If you don’t subscribe to the QNX philosophy of simplicity, then
perhaps Mach is the O/S you should be using ?

Wrong conclusion … se above.

Armin Steinhoff


Mach
message-passing, unlike QNX is secure (has permissions attached to
‘ports’)


Oh yeah, just what I need, security data for IPC on my iPaq, or my
process controller > :wink:


and works in 0-copy mode when possible, using copy-on-write
mechanism.


and what, exactly is the impact on scheduling latency (not to mention
footprint) for all this “cool” stuff ?


And on top of that it is binary-compatible with Unix. Which
is what killed it, ironically since it was competing against

monolithic

kernels and they were faster anyway, especially on the processors of
those days.


Hmmm, QNX didn’t die, I wonder why “superior” technology that was “open
source” “died”, when QNX didn’t ? Could it be that QNX actually met a
market need better than Mach ?


Nevertheless, most modern Unix variants were influenced heavily by

ideas

introduced in Mach.


Except for anything remotely related to microkernel architecture…


Too bad QNX was not (yet).


No, your right, QNX is heavily influenced by simplicity, and microkernel
architecture, not featuritis and Rube Goldbergism … Personally, I hope
it remains that way.


Well, if you watch QNX corporate video, they don’t really say ‘most
advanced OS for process control’. They say something more in line with
‘the most advanced architecture in the world’.


Often, marketing material isn’t designed to convey detailed information
about the target applications that a product is designed to address
(typically because marketing campaigns can encompass many revisions of a
product). I think that QNX6 is a very advanced architecture for the
market it is designed to address; I concede that the marketing material
does not clearly convey what the target market is; but I am also willing
to bet that QSSL sales/marketing rarely show up at customers who aren’t
in that target market. Those who read any marketing material in a
vacuum deserve what they get.


Also i would challenge calling good filesystem performance a

‘gee-whiz’

feature. May be for you it is, but not for most of us.


No. In my experience (15 years of QNX at 5 different companies in 3
different industries), a high performance filesystem, is a ghee-wiz
feature for most of the customers in the QNX target market (it always
has been). Many of the customers don’t even need a filesystem, those
that do, typically need a compact and robust filesystem, above a high
performance filesystem. As proof, I submit that QNX’s largest
competitor (and dominant player in the market that QSSL currently
operates in) is VxWorks which (for all intents and purposes) has no
filesystem; and certainly not a high performance filesystem by anyones
definition of the term…

IMO there are a minority of vocal advocates of a HPFS and IBC that make
a lot of noise, while the majority of users are FDAH without these, and
see no reason to get involved in these discussions. I just want to make
it clear that there are those of us out here for whom a HPFS is very low
on the priority list.


We (Motorola) use
QNX6 on ‘high-end’ server-type system and it needs to read and write
data efficiently. We have bypassed writing problems by using

complicated

queueing,


QNX is a real-time operating system. Even the most novice real-time
system designer knows the rule #2 of real-time system design: "no queues

  • certainly not complicated queues". Therefore, if you solved your
    applications “problems” by using queues, then I submit that your
    application is not hard real-time. Perhaps QNX is not the best choice
    for your application.


    but there’s nothing we can do with reading. It gives us
    3mb/sec on UltraWide SCSI and that translates directly into slow
    startup, because we need to read large database at startup. Slow

startup

translates into longer downtime which is hardly tolerable in telecom
industry. And you bet we did tell that to our sales rep (couple years
back and kept telling ever since).


I’m not going to get into what QSSL sales told you or didn’t tell you
(since I haven’t a clue about that subject), I am simply voicing my
opinion, that I don’t believe a high performance filesystem, or an
integrated buffer cache is a high priority for those in our industry
(process control), or for most current QNX customers (including telecom

  • haven’t seen any filesystems on routers lately…).

Would I like it if QNX had these features (at no cost in scheduling
latency or base footprint), certainly; but I wouldn’t be willing to
sacrifice very much to get them. QNX6 is starting to look very good in
the areas that I care about, but it isn’t quite there yet, and I
represent the traditional market; IMO they need to serve the traditional
market first, and then they can look at new markets (as long as it
doesn’t negatively impact the existing markets).


See how things look differently from
different perspective?


I am just trying to give you my perspective which is different to yours,
and (I believe) common to the majority of those who currently have
products based on QNX (and the QNX philosophy).