[x-posted from apps]: Re: fs-qnx filesystem questions

Miguel Simon wrote:

Hi Armin…

Armin Steinhoff wrote:


Don’t use the packer installer in a productive system … it shuts down
the packet files system and restarts it. Well … and this happens in a
multi user system.

Is there a way not to use the package file system in an embedded
configuration ??


What do you mean by this question? What do you call an ‘embedded
system’?

‘Embedded systems’ are the systems targeted by QNX :slight_smile:


In an stand alone, autonomous? embedded system, you write your *.ifs
which becomes your .boot.

Yes … that’s one of the posibillities. But there are also embedded
systems with real mass storage … e.g. for data logging.

So the question is: how can I install the OS into the plain files system
without Photon and the package installer (which presumes an active
package fs)??

Armin


No package file system required. Either this,
or I completely did not get your point above!

Regards…

Miguel.


Armin

Igor Kovalenko <kovalenko@home.com> wrote:
: I will just quote one QNX employee (who I will not name): ‘Anyone who’s
: defending our disk I/O system is lying to themselves’.

They would only be lying about what bit actually needs defending …

So the question is: how can I install the OS into the plain files system
without Photon and the package installer (which presumes an active
package fs)??

There is no way to install a system without the package installer.
Just like you would with QNX4 when created a embedded target,
you copy the files “manualy” from a fully install setup.
There is no equivalent to QNX4 /etc/install.

  • Mario

“Miguel Simon” <simon@ou.edu> wrote in message
news:3BD70ED7.23656E27@ou.edu

Hi…

Rennie Allen wrote:


everything is “right” with it; however, QNX is not targeted at general
purpose computing, it is targeted at systems that must demonstrate high
availability, and high predictability, and typically do little writing
to disk (many systems never write to disk at all). What is driving QSSL

Agree. Our embedded control applications (running in both QNX4 and QNX6)
do not even have a disk to write to, and they are very robust because in
every sense, QNX is a robust OS. In many ways it is good that QNX is not
targeted at general purpose computing…

What does a decent performance file system has to do with “general purpose
computing”?

IMO there are a minority of vocal advocates of a HPFS and IBC that make
a lot of noise, while the majority of users are FDAH without these, and
see no reason to get involved in these discussions. I just want to make
it clear that there are those of us out here for whom a HPFS is very low
on the priority list.

We don’t hear about the people with requirement for HPFS that checks QNX
out and then turn away.

I am getting sick of this kind of arguments. Reminds me very much the old
days when large chorus of people in comp.os.qnx (except Armin)

and me, and me :wink:

argued to me how unnecessary and unwanted would it be to have SMP
and threads

and for non x86 CPU

in QNX. Until Dan Dodge came up and told them otherwise, lol.



I am just trying to give you my perspective which is different to yours,
and (I believe) common to the majority of those who currently have
products based on QNX (and the QNX philosophy).

I though one of the QNX philosophy was to get the most out of the hardware.

“Armin Steinhoff” <a-steinhoff@web_.de> wrote in message
news:3BD82B52.90406@web_.de…

Mario Charest wrote:
So the question is: how can I install the OS into the plain files system
without Photon and the package installer (which presumes an active
package fs)??



There is no way to install a system without the package installer.
Just like you would with QNX4 when created a embedded target,
you copy the files “manualy” from a fully install setup.
There is no equivalent to QNX4 /etc/install.

Thanks for your reply.

I found out in the meantime a way to install *.qpk archives:

  • rename the *.qpk archive to *.tar.gz and install it with gunzip/pax
    (QNX4 oldtimers can also use /etc/install > :slight_smile: > )

Got idea, it never occur to me to copy /etc/install over to QNX6.

qpk are packaged in a way that when gunzip/pax, they are installed
under /pkgs/repository and not under /. They also often have extra
directory level present to keep the /pkgs/repository/… organised.

  • don’t forget to excecute the ‘post_install’ script included in the
    *.qpk archive

Do I miss something ?

Armin







\

  • Mario


    \

Armin Steinhoff <a-steinhoff@web_.de> wrote:

I found out in the meantime a way to install *.qpk archives:

Here’s a little ditty that I cooked up to pull the contents of a package
into a directory tree. It’s not perfect - it doesn’t preserve user/group
id’s nor setuid bits. If someone wants to go ahead and enhance it, be
my guest.

=======================================================================
#! /bin/sh
#ifdef __USAGE
#%C - pull packages into a directory structure

#%C <target_dir> …

#Where:

<target_dir> root of installed tree

name of *.qpk file to be installed.

#endif

function process_manifest
{
dir=${1%/}
fgrep -e QPM:Dir -e QPM:Link $1 | (
while read -r type remainder; do
case ${type} in
“<QPM:Dir”)
path=$(echo ${remainder} | sed -e ‘s/name="//’ -e 's/".
//’)
if [ “${path}” ]; then
dir=${dir}/${path}
fi
;;
“</QPM:Dir>”)
dir=${dir%/}
;;
“<QPM:Link”)
src=$(echo ${remainder} | sed -e ‘s/link="//’ -e 's/".
//’)
dst=$(echo ${remainder} | sed -e ‘s/^.">//’ -e 's/<.//’)
ln -sf ${src} ${dir}/${dst}
;;
esac
done
)
}

base=$1
shift
list=$*
for pkg in ${list}; do
echo Working on ${pkg}…
SIFS=${IFS}
IFS=-
set – ${pkg%.qpk}
IFS=${SIFS}

Find version number in package name (first thing starting with a number)

while [ “${1##[0-9]*}” != “” ]; do
shift
done
version=$1

Get build number, if present

if [ “${2#bld}” != “$2” ]; then
build=-$2
else
build=
fi
zcat -d ${pkg} | pax -rov -s~^.*-${version}${build}/~${base}/~ 2>/tmp/pp.$$
man_list=$(fgrep MANIFEST /tmp/pp.$$)
for manifest in ${man_list}; do
process_manifest ${manifest}
done
rm -f /tmp/pp.$$ ${man_list}
done

Mario Charest wrote:

So the question is: how can I install the OS into the plain files system
without Photon and the package installer (which presumes an active
package fs)??



There is no way to install a system without the package installer.
Just like you would with QNX4 when created a embedded target,
you copy the files “manualy” from a fully install setup.
There is no equivalent to QNX4 /etc/install.

Thanks for your reply.

I found out in the meantime a way to install *.qpk archives:

  • rename the *.qpk archive to *.tar.gz and install it with gunzip/pax
    (QNX4 oldtimers can also use /etc/install :slight_smile: )

  • don’t forget to excecute the ‘post_install’ script included in the
    *.qpk archive

Do I miss something ?

Armin







  • Mario
    \

“Operating System Tech Support” <os@qnx.com> wrote in message
news:9r97g0$meg$1@nntp.qnx.com

I never said size was what “made” a microkernel, please don’t guess what I
think, then counter it - it’s not a very good debating strategy. Size is
also not what makes a microkernel (and I’m shocked you would assume I
would
think that),

I did not really think you would think that. I said ‘someone’ as I simply
wanted to avoid that kind of argument before it even pops up.

it’s about the minimialist view of what services the kernel is
to provide (ie. as few as possible).

So you share Rennie’s view. I already said that such definition is rather
useless because it does not provide for a formal way to tell what is
microkernel and what is not.

I just love to hear this. How did not we think about that on our own, I
wonder…

Ouch, sarcasm was hardly called for.

Perhaps. But then you should not have [sarcastically] asked why did not we
write a proper driver ourselves, especially when you know better that you do
not provide a way to do it.

  • igor

“Mario Charest” <mcharest@clipzinformatic.com> wrote in message
news:9r91a5$jn6$1@inn.qnx.com

I am getting sick of this kind of arguments. Reminds me very much the
old
days when large chorus of people in comp.os.qnx (except Armin)

and me, and me > :wink:

argued to me how unnecessary and unwanted would it be to have SMP
and threads

and for non x86 CPU

Sorry Mario, silly me forgot :wink:

  • igor

Hi…

Igor Kovalenko wrote:

“Operating System Tech Support” <> os@qnx.com> > wrote in message
news:9r97g0$meg$> 1@nntp.qnx.com> …

it’s about the minimialist view of what services the kernel is
to provide (ie. as few as possible).

So you share Rennie’s view. I already said that such definition is rather
useless because it does not provide for a formal way to tell what is
microkernel and what is not.

…reminds me of the argument of “…what is a planet, and what is not a
planet…, humm, Is Pluto a planet or is Pluto not a planet?”

An answer with regard to the planet would be… “who cares, as long as
it gets the job done!” :slight_smile:)

Now, for those folks who do embedded controls, a minimalist,
deterministic, robust and reliable kernel whose performance and behavior
does not change every four months or every other developer, well…,
that type of kernel is a rather welcomed development in the world of
applied computing.

Thanks QSSL. I for one do appreciate your product. :slight_smile:


Regards… :slight_smile:

Miguel.


  • igor

my opinions are mine, only mine, solely mine, and they are not related
in any possible way to the institution(s) in which I study and work.

Miguel Simon
Research Engineer
School of Aerospace and Mechanical Engineering
University of Oklahoma
http://www.amerobotics.ou.edu/
http://www.saic.com

I am getting sick of this kind of arguments. Reminds me very much the
old
days when large chorus of people in comp.os.qnx (except Armin) argued
to me
how unnecessary and unwanted would it be to have SMP and threads in
QNX.
Until Dan Dodge came up and told them otherwise, lol.

I recall this argument. IIRC you were arguing that real-time systems
need SMP, and that QNX4 could, and should support SMP, and that ASMP
would be totally useless for realtime. You don’t honestly believe that
those working at QSSL didn’t know that Dan was working on SMP do you ?
It appears that you confused the owners perogative to make product
announcements with some sort of retribution of employees.

Now it is 5 years later, and how many SMP QNX6 systems are there as a
percentage of total QNX6 systems ? If SMP were so vitally important (as
you made it out to be at the time) don’t you think that more than (some
small number here)% of the (paying customer) QNX6 systems would be
shipping SMP systems ? QNX6 was designed to be SMP from the very
beginning (which btw was a long time before your SMP tirade), so
actually implementing it was relatively low cost.

Our industry has no requirement for SMP today. You continuously
complain that QNX is “not advanced” yet it has many features that the
vast majority of customers still don’t use, and probably won’t use for
years. I don’t disagree with you about the technical coolness of SMP or
a HPFS, but where is the customer base ?

Yes, QNX has better realtime technology, but that’s not the only issue
considered when decisions are made. If general-purpose OSes are
getting more
realtime capabilities, QNX needs to counter that with more
general-purpose
features, especially as technology evolves and new opportunities
arise.

Or perhaps the M$ strategy might be to get QSSL to spend inordinate
amounts of it’s R&D effort trying to provide something that is
essentially useless to their market, and to bleed off their customer
base ?

We use queueing into circular buffers to write logs and billing
information
(potentially coming from thousands of calls). Since those data are
produced
by realtime components, we did not want those components to be held by
undeterministic I/O operations.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

OK. Now let’s get back to the root of the discussion here. You want
QSSL to invest a lot of time and effort in to producing a HPFS, not a
deterministic file system. If you were calling for a deterministic
filesystem I would still say it isn’t that high a priority, but I would
concede that it is a higher priority than a HPFS. Please note that a
deterministic filesystem would be much slower than the current QNX
filesystem (since determinism would require an acknowledgement that the
data is on the media before proceeding with any addition output
operations), which would (most likely) mean that you would need to
provide queuing, which is what your complaining about having to have
done.

Writing into queue is deterministic and then
there are multiple low-priority ‘dequeuers’, such as remote log
readers (who
potentionally have different connection speed and) and flush daemon
which
saves all that stuff to disk as CPU cycles permit.

If your system is a hard real-time system then you must know exactly how
many CPU cycles it takes to flush “all that stuff” to disk. If you do,
then there is absolutely no point in having queues, simply allocate a
single buffer big enough to hold “all the stuff” that could be produced
in the number of CPU cycles it takes to write “that stuff” to disk, and
your done (if you don’t have enough memory in your system to accomodate
that single buffer, then your H/W is inadequate to support the
application, since the queues would ultimately allocate that much memory
at some point anyway - all you are doing by providing queues along with
inadequate memory is designing failure into your system).

The entire concept of dynamically allocated queues flies in the face of
predictability (which is a synonym for determinism), they are, in fact,
an admission by the programmer, that he can’t predict how much memory is
required.

People who use queues in HRT systems remind me of people who buy into
pyramid schemes, they think that as long as they can keep recruiting
more memory they’ll be OK (both approaches ultimately lead to failure).

If you’re so advanced realtime designer as to tell us that queues are
bad
even without knowing our requirements, please educate me how would you
implement that thing better.

There are always exceptions to rules, but I don’t think what you’ve
described counts as one. My guess is that the system to which you refer
above is soft real-time.

Notice, the new ‘slogger’ manager in QNX6 is
basically similar thing. Our solution was just more flexible and
scalable
(multiple queues and multithreaded manager).

slogger is a standard unix thing, it has nothing to do with QNX’s
real-time features.

‘Telecom’ is not limited to routers. Our system is mobile phone
switching
center.

Certainly telecom is not limited to routers, neither is it limited to
mobile phone switch centers.

We do process control. We need deterministic behavior from our
controllers, but our engineering and operator workstations all run on
NT. Just because QNX is aimed at the process control industry doesn’t
mean they perfectly address all of the applications within that domain
(IMO QNX addresses its part of our system better than NT addresses its
part of our system).

I will just quote one QNX employee (who I will not name): ‘Anyone
who’s
defending our disk I/O system is lying to themselves’.

I am not defending any part of the disk I/O system. All I am saying is
that it’s good enough for my requirements, and those of most current QNX
customers; and before investing a lot of effort in making it better, I’d
like to see all the real-time features of QNX working flawlessly first.

“John Garvey” <jgarvey@qnx.com> wrote in message
news:9r8tg6$fur$1@nntp.qnx.com

Igor Kovalenko <> kovalenko@home.com> > wrote:
: I will just quote one QNX employee (who I will not name): ‘Anyone who’s
: defending our disk I/O system is lying to themselves’.

They would only be lying about what bit actually needs defending …

Well John, you should know better. Let me think of the ‘bits’ we have here
(speaking about all aspects of block I/O, not just QNX4 FS per say):

  • very poor writing performance
  • very modest reading performance
  • very high memory consumption (eide driver alone can consume 73Mb on my
    system, and if we have SCSI too, then there’s another buffer cache to double
    it)
  • inability to auto-tune cache (due to lack of unification between drivers
    and with VM)
  • apparently dumb directory lookup algorithms (sucks beyond belief on large
    directories)
  • very poor media error handling (writing to a bad DVD-RAM media will lock
    the drive in busy state until power cycle, Solaris reports SCSI errors and
    drive remains ok in such case)
  • no useful abstract device interface, like CAM or ASPI (impossible to add
    drivers for new types of SCSI devices without resorting to pass-thru, which
    is undocumented anyway)
  • very poor device enumeration/identification scheme (can you tell what the
    heck is dev/hd0? Is that your SCSI or your EIDE drive? And what is your
    removable drive?)
  • no support for tapes at all
  • no support for jukeboxes
  • no support for CD-R/RW
  • no published interfaces for either device drivers or filesystem modules
  • only 48-byte filenames (did you try to use napster with qnx4fs?)
  • only 16 bits for UID/GID (hard to use in Unix NFS networks)
  • only 2Gb for max. file size

Hmm, that’s 15 bits man. There must be some 16th bit which you think is
worth defending. What’s that? Tell us any single feature of this QNX
subsystem which sets it apart from other OSes and offsets all or even any of
the above mentioned ‘bits’. Just please don’t tell ‘robustness’ unless
you’re prepared to elaborate how it is more ‘rubust’ than other systems.

  • igor

Hi…

Mario Charest wrote:


What does a decent performance file system has to do with “general purpose
computing”?

…educate me if you will, please… What is so bad about the QNX6
file system?? I suppose that I have not pushed the system to its
limits, have I? In any case, if you enlighten me, then I would know
what is the complain all about.

I get the feeling that some people use QNX because it seems to be the
least upper bound of all the bad RTOS’s. Which is or what constitutes a
good RTOS, and why would QNX6 not be one of them? Because of the file
system performance? How relevant is a ‘decent performance file system’
to the real-time aspect of the os? Many have said -and is indeed my own
experience- that the file system is irrelevant to some type of hard real
time systems and/or applications. Then, if you have an RTOS, would you
have to have a ‘decent performance file system’ for the RTOS to be
considered good?? I would suspect that this is a matter of opinion as
opposed to a matter of fact, would you not?

As an example: what file system did the Apollo program folks utilized in
their embedded real-time computers?

Notice, I just want to know, and as far as I am concerned, freedom from
my own ignorance is what I look for. I gladly be ignorant now rather
than when I am giving a presentation. Thanks.

Regards…

Miguel.

\


my opinions are mine, only mine, solely mine, and they are not related
in any possible way to the institution(s) in which I study and work.

Miguel Simon
Research Engineer
School of Aerospace and Mechanical Engineering
University of Oklahoma
http://www.amerobotics.ou.edu/
http://www.saic.com

“Miguel Simon” <simon@ou.edu> wrote in message
news:3BD8468C.44EB6109@ou.edu

Hi…

Now, for those folks who do embedded controls, a minimalist,
deterministic, robust and reliable kernel whose performance and behavior
does not change every four months or every other developer, well…,
that type of kernel is a rather welcomed development in the world of
applied computing.

LOL, you must mean ‘when performance of all other systems goes up and
performance of yours is stuck on 5 years old level’ :wink:

Thanks QSSL. I for one do appreciate your product. > :slight_smile:

I never said I don’t appreciat etheir product. I would not be wasting my
time posting here if I did not. But product is quite a broad term and some
parts of it really do need revamping. Just because you don’t care about that
part, does not make it less important.

  • igor

Hi…

Igor Kovalenko wrote:

LOL, you must mean ‘when performance of all other systems goes up and
performance of yours is stuck on 5 years old level’ > :wink:

…then enlighten me if you will, so that I can LOL too!. :slight_smile: Why is
rtp6 file system performance 5 years behind when compared to those other
OS’s??

Regards…

Miguel

  • igor

my opinions are mine, only mine, solely mine, and they are not related
in any possible way to the institution(s) in which I study and work.

Miguel Simon
Research Engineer
School of Aerospace and Mechanical Engineering
University of Oklahoma
http://www.amerobotics.ou.edu/
http://www.saic.com

Miguel Simon wrote:

Hi…

Igor Kovalenko wrote:

LOL, you must mean ‘when performance of all other systems goes up and
performance of yours is stuck on 5 years old level’ > :wink:


…then enlighten me if you will, so that I can LOL too!. > :slight_smile: > Why is
rtp6 file system performance 5 years behind when compared to those other
OS’s??

Well, I said some thing already in reply to John. To summarize, there
are problems with FS design per say which are mostly related to lack of
writing optimisations and too tight limits everywhere.

There are [worse] problems with the infrastructure around FS, which
contributes a lot into poor performance, as well as into general
usability and extensibility of system. Again, see my reply to John (who
afaik is in charge of those affairs).

  • igor

I did not really think you would think that. I said ‘someone’ as I
simply
wanted to avoid that kind of argument before it even pops up.

Ahhh, yes, contrary opinion preemption. This is intended to prevent all
that messy exchange of ideas that debates normally consist of :wink:

So you share Rennie’s view. I already said that such definition is
rather
useless because it does not provide for a formal way to tell what is
microkernel and what is not.

Ohhh, you have already unilaterally dismissed this view as useless have
you ? How dare anyone continue to hold a view that has previously been
determined as useless by you.

FYI there are many things in the field of architecture for which there
is no formalism suitable for differentiating styles. This does not make
these styles unidentifiable, although certainly there are gray areas.
If you put the Mach kernel next to the QNX kernel few people will
disagree that the QNX kernel is more minimalist than the Mach kernel, in
fact your entire position when comparing the kernels was to cite all of
the cool things that the Mach kernel has that the QNX kernel doesn’t,
which seems to indicate that you recognize this difference also.

“Rennie Allen” <RAllen@csical.com> wrote in message
news:64F00D816A85D51198390050046F80C9012CEC@exchangecal.hq.csical.com

So you share Rennie’s view. I already said that such definition is
rather
useless because it does not provide for a formal way to tell what is
microkernel and what is not.

Ohhh, you have already unilaterally dismissed this view as useless have
you ? How dare anyone continue to hold a view that has previously been
determined as useless by you.

I don’t dismiss any views Rennie. Views by definition are subjective, so it
does not make sense to dismiss them. But I do dismiss subjective
definitions because they are indeed practically useless. Definitions exist
to provide a way to identify something. If you can’t reliably use a
definition, then the definition is vague and it will only lead to confusion.
That is my view.

FYI there are many things in the field of architecture for which there
is no formalism suitable for differentiating styles. This does not make
these styles unidentifiable, although certainly there are gray areas.
If you put the Mach kernel next to the QNX kernel few people will
disagree that the QNX kernel is more minimalist than the Mach kernel, in
fact your entire position when comparing the kernels was to cite all of
the cool things that the Mach kernel has that the QNX kernel doesn’t,
which seems to indicate that you recognize this difference also.

Yes I do. However, majority of people in computer science do classify Mach 3
as microkernel. The reason behind that, they follow more formal definition
of microkernel that you seem to like.

  • igor

“Igor Kovalenko” <Igor.Kovalenko@motorola.com> wrote in message
news:3BD6EA0D.9473B713@motorola.com

Operating System Tech Support wrote:
Having modules running outside of kernel space does not make an OS a
micro-kernel. That is simply a side-effect.

Will you educate us please? What does make a microkernel? Size? To me
size is not really serious argument. And before someone pops up to say
‘QNX microkernel is only 40k’ I will remind that you can’t get those 40k
on their own. They come ‘bundled’ with process manager and share address
space with it. So it is about 300K. And that’s with very rudimentary VM
subsystem.

I never said size was what “made” a microkernel, please don’t guess what I
think, then counter it - it’s not a very good debating strategy. Size is
also not what makes a microkernel (and I’m shocked you would assume I would
think that), it’s about the minimialist view of what services the kernel is
to provide (ie. as few as possible).

I just love to hear this. How did not we think about that on our own, I
wonder…

Ouch, sarcasm was hardly called for.

-Adam