My guess is that it has something to do with the filesystem stuff needing to
be written and flushed to the storage media in a synchronous fashion. I
think they do careful updating of all of the directory and bitmap data which
may involve a couple of writes followed by the writing of the actual data.
None of that directory/bitmap stuff goes on when you hit the raw device
directly. Given that the writes are the slow part, if there were two more
writes involved in the filesystem overhead that would give you your 3X
performance hit. Without actually getting in there and mucking around with
it myself, I’d say your best bets are to fiddle with the synchronous and/or
caching options for the filesystem (perhaps not IDE specific ya know?), or
go with the readblock()/writeblock() functions. Personally I’d go with the
block stuff as it doesn’t sound like you need anything more than a
sequentially growing quantity of log data, and that will be not only your
fastest option but probably your most recoverable in the event of a
disaster.
Sounds like a cool application.
-Warren “Can I get a ride?” Peece
“Jason Farque” <jasonf@pigging.com> wrote in message
news:a4jpdt$j6i$1@inn.qnx.com…
Warren, thanks for the response.
My data application is for a battery powered device that travels along
inside of oil and gas pipelines at speeds up to 13 feet per second.
Hitting
a 90’ bend inside of a pipeline at 9 miles per hour isn’t good for
rotating
media. Heck it’s not even good for solid state media, but it’s the
environment I’m in. I’m aware of the performance issues of Flash itself,
but what I can’t understand is why writing data through the devc-eide
driver
is ~3 times slower than writing raw blocks to /dev/hd1. Is the devc-eide
driver that slow?
I have tuned the driver to use dma, and have tried other various
configurations of the driver but none of them seem to have a noticible
effect on performance. I thought this was purely a Flash write-time issue
until I experimented with writing to the raw /dev/hd1. Much to my
surprise
I can sustain ~350k-400/sec to the raw Flash device, but only ~140k
through
the devc-eide device driver in RTP 6.1.
Jason
“Warren Peece” <> Warren@NoSpam.com> > wrote in message
news:a4jgds$d5d$> 1@inn.qnx.com> …
Hiya Jason,
First off FLASH speeds are FLASH speeds, it takes (comparatively) a long
time to write anything to a FLASH device regardless of filesystem
overhead.
It’s the internal write algorithm that takes all the time. Also FLASH
devices have a limited life cycle (number of erases & writes) before
they
start developing bad spots, so FLASH is just a bad idea in general for
logging any substantial amount of data. As for IDE performance, IDE is
inherrantly more CPU intensive than say bus mastering SCSI so CPU speed
is
going to play a fairly large factor in overall performance. Knowing
that,
make sure your IDE controller chipset as well as your IDE drive of
choice
both have DMA capability and that you have that option enabled within
the
driver. Once you get that sorted out, you need to address how you write
your data to disk. I’m making a few assumptions here, namely that
you’re
writing your own application. Writing non-block sized data to any file
is
going to be slower than writing block sized data. As an example say
you’re
writing data to a file and the end of the data occupies the first 256
bytes
of a 512 byte disk block. If that disk block isn’t in the filesystem
cache,
which it probably won’t be in a logging application, then it will need
to
be
read from the drive, modified, then written back out. You can see where
this would be slower than if you always wrote 512 byte blocks. So,
buffer
the data up to say 32K at a time, then slap it to disk in one write.
This
has the added benefit of reducing the overhead of the system calls. You
may
want to look at readblock() and writeblock() as well, as they are
totally
block oriented and may even be slightly less overhead than the standard
read()/write() calls (?). There are of course other factors (disk cache
size, synchronous writes and so on), but hopefully you’ll find some of
the
above suggestions helpful.
-Warren “Log THIS!” Peece
“Jason Farque” <> jasonf@pigging.com> > wrote in message
news:a4h5fl$l40$> 1@inn.qnx.com> …
Hello all,
I’ve been playing around with the RTP’s QNX file system on an embedded
system with a PCMCIA IDE interface solid state hard disk (capacity
1GB).
The computer is a 133MHz AMD 520 x86-compatible machine (DIMM-PC from
JumpTec) with 32MB of RAM. I’ve found that I can only achieve about
140k
a
second writing to the flash disk drive using devb-eide, but that if I
write
to the raw disk in /dev/hd1 (for instance with “time dd if=/dev/hd0
of=/dev/hd1 count=100000”) I get about 350k a second.
Yuck.
My system is going into a data logging application where I will be
accumulating data at rates as high as 200kb/sec. Is there any way to
squeeze better performace out of the QNX/RTP devb-eide file system or
must
I
write to the raw disk to get the transfer rate that I want? Is there
a
different/better way?
Thanks,
Jason Farque
\