high performance disk solution

We need to record video to disk, some 20Mbytes/second sustained.
It needs to be a compact rackmount PC solution, preferrebly
some PCI adapter to put in a 1U (or maybe 2U) computer.
Anything on the market with QNX6.1 support? A 1U external disk
box with suitable interface connected to a 1U computer could be an
alternative.; perhaps a nice alternative since a standard (SCSI) controller can be used.
The framegrabber will sit in the same PC. Will the PCI bus choke?
Could it be possible to make PCI DMA directly from he framegrabber to
the disk controller to remove the unnecessary data copying via the RAM?

Cheers / Tom

“Tomas Högström” <tomas@scandicraft.se> wrote in message
news:3BD206A0.7C390B07@scandicraft.se

We need to record video to disk, some 20Mbytes/second sustained.
It needs to be a compact rackmount PC solution, preferrebly
some PCI adapter to put in a 1U (or maybe 2U) computer.
Anything on the market with QNX6.1 support? A 1U external disk
box with suitable interface connected to a 1U computer could be an
alternative.; perhaps a nice alternative since a standard (SCSI)
controller can be used.

I think that would be the only solution, if there’s any at all. I was unable
to get more than 10mb/sec sustained write under QNX. I did get something
like 20Mb/sec (on EIDE) sustained under Win2k, but it could be due to
auto-adjusting buffer cache and anyway it would not be safe to run ‘on
edge’. UltraSCSI did not do any better, i did not try Ultra160.

The framegrabber will sit in the same PC. Will the PCI bus choke?

PCI should not choke, it is designed to handle much more than that.

Could it be possible to make PCI DMA directly from he framegrabber to
the disk controller to remove the unnecessary data copying via the RAM?

Not without custom driver and then you’d have to treat the drive as ‘raw
block device’ (no filesystem).

You might need a custom driver in any case, i am afraid. You probably want
stripping RAID external drive unit to handle 20mb/sec reliably. I don’t
think there are any drivers available. About the only type of SCSI
controller you can find for CompactPCI is LSI Logic (Symbios) and QNX driver
for that is too damned bad (ASYNC mode only, never mind RAID). If your
system is not CompactPCI, you could try Adaptec (better driver under QNX).
Perhaps with write-behind cache enabled (in SCSI BIOS) and one of new 15000
RPM drives you can do without RAID.

  • igor

How about the boards with EIDE RAID (MSI has several), does QNX support those
out of the box? Has anyone tested what performance it is possible to get from
this solution?

Another question (have lots to day :slight_smile: ):
If one would use a dedicated disk and use it as a raw block device with the
drivers from QNX (i.e. writing directly to /dev/hd… bypassing the file
system), would this give a significant performance boost?

/Mikael K.

Igor Kovalenko wrote:

“Tomas Högström” <> tomas@scandicraft.se> > wrote in message
news:> 3BD206A0.7C390B07@scandicraft.se> …
We need to record video to disk, some 20Mbytes/second sustained.
It needs to be a compact rackmount PC solution, preferrebly
some PCI adapter to put in a 1U (or maybe 2U) computer.
Anything on the market with QNX6.1 support? A 1U external disk
box with suitable interface connected to a 1U computer could be an
alternative.; perhaps a nice alternative since a standard (SCSI)
controller can be used.

I think that would be the only solution, if there’s any at all. I was unable
to get more than 10mb/sec sustained write under QNX. I did get something
like 20Mb/sec (on EIDE) sustained under Win2k, but it could be due to
auto-adjusting buffer cache and anyway it would not be safe to run ‘on
edge’. UltraSCSI did not do any better, i did not try Ultra160.

The framegrabber will sit in the same PC. Will the PCI bus choke?

PCI should not choke, it is designed to handle much more than that.

Could it be possible to make PCI DMA directly from he framegrabber to
the disk controller to remove the unnecessary data copying via the RAM?


Not without custom driver and then you’d have to treat the drive as ‘raw
block device’ (no filesystem).

You might need a custom driver in any case, i am afraid. You probably want
stripping RAID external drive unit to handle 20mb/sec reliably. I don’t
think there are any drivers available. About the only type of SCSI
controller you can find for CompactPCI is LSI Logic (Symbios) and QNX driver
for that is too damned bad (ASYNC mode only, never mind RAID). If your
system is not CompactPCI, you could try Adaptec (better driver under QNX).
Perhaps with write-behind cache enabled (in SCSI BIOS) and one of new 15000
RPM drives you can do without RAID.

  • igor

Thanks for the advice. Yes, some kind of RAID is perhaps the
way to go. There are RAID solutions which looks like a
standard disk (SCSI 160MB/s) to the controller which perhaps is
an interesting alternative since we can use standard controllers.
There is a solutions from transtech which contains 8 EIDE disks behind a
SCSI 160 interface which gives rather good price / performance due to the
lower EIDE disk price. Somewhat too large for our needs, and we could
probably do with only four such disks.

What about maximum file size? 1hr uncompressed video would give a 72GB file.
But if bypassing the filesystem that problem is of course also bypassed.

Thanks / Tom

Igor Kovalenko wrote:

“Tomas Högström” <> tomas@scandicraft.se> > wrote in message
news:> 3BD206A0.7C390B07@scandicraft.se> …
We need to record video to disk, some 20Mbytes/second sustained.
It needs to be a compact rackmount PC solution, preferrebly
some PCI adapter to put in a 1U (or maybe 2U) computer.
Anything on the market with QNX6.1 support? A 1U external disk
box with suitable interface connected to a 1U computer could be an
alternative.; perhaps a nice alternative since a standard (SCSI)
controller can be used.

I think that would be the only solution, if there’s any at all. I was unable
to get more than 10mb/sec sustained write under QNX. I did get something
like 20Mb/sec (on EIDE) sustained under Win2k, but it could be due to
auto-adjusting buffer cache and anyway it would not be safe to run ‘on
edge’. UltraSCSI did not do any better, i did not try Ultra160.

The framegrabber will sit in the same PC. Will the PCI bus choke?

PCI should not choke, it is designed to handle much more than that.

Could it be possible to make PCI DMA directly from he framegrabber to
the disk controller to remove the unnecessary data copying via the RAM?


Not without custom driver and then you’d have to treat the drive as ‘raw
block device’ (no filesystem).

You might need a custom driver in any case, i am afraid. You probably want
stripping RAID external drive unit to handle 20mb/sec reliably. I don’t
think there are any drivers available. About the only type of SCSI
controller you can find for CompactPCI is LSI Logic (Symbios) and QNX driver
for that is too damned bad (ASYNC mode only, never mind RAID). If your
system is not CompactPCI, you could try Adaptec (better driver under QNX).
Perhaps with write-behind cache enabled (in SCSI BIOS) and one of new 15000
RPM drives you can do without RAID.

  • igor

“Tomas Högström” <tomas@scandicraft.se> wrote in message
news:3BD31416.CF4380B1@scandicraft.se

Thanks for the advice. Yes, some kind of RAID is perhaps the
way to go. There are RAID solutions which looks like a
standard disk (SCSI 160MB/s) to the controller which perhaps is
an interesting alternative since we can use standard controllers.
There is a solutions from transtech which contains 8 EIDE disks behind a
SCSI 160 interface which gives rather good price / performance due to the
lower EIDE disk price. Somewhat too large for our needs, and we could
probably do with only four such disks.

As long as you got a driver for that :wink:

What about maximum file size? 1hr uncompressed video would give a 72GB
file.
But if bypassing the filesystem that problem is of course also bypassed.

QNX filesystem would not support such big files. There are functions in libc
for ‘large file support’ but I don’t have experience with them and not sure
how they can be used with QNX filesystem.

If you have reasonable business case, give a call to their sales. If enough
people will do that, they will come up with solution. So far standard excuse
was something like ‘we do not have much market for high-end storage
systems’.

  • igor

Writing to /dev/hd0 is not much more efficient. The extra overhead of the
file system is born only during the open() and when the file must be grown
beyond it’s current extent. I.E. there is a very nominal performance
improvement when you pregrow a file.


Bill Caroselli – 1(530) 510-7292
Q-TPS Consulting
QTPS@EarthLink.net


“Mikael Karlsson” <mikkar@bredband.net> wrote in message
news:3BD2AA19.510036AA@bredband.net

If one would use a dedicated disk and use it as a raw block device with
the
drivers from QNX (i.e. writing directly to /dev/hd… bypassing the file
system), would this give a significant performance boost?

/Mikael K.

“Armin Steinhoff” <a-steinhoff@web_.de> wrote in message
news:3BD45A61.200@web_.de…

wxGTK has a stupid bmp file loader. It takes 5 sec to load a 900K bmp
file with SUSE 7.2 … but it takes 40 sec with QNX 6.1 and the
dev-eide eats up 80% of the CPU power of a 700MHz machine =:-/

Isn’t is a great multi media system ?

It sounds like your EIDE is not working in DMA mode. Note, even if you
passed dma option to driver, that does not automatically mean it is used :wink:

  • igor

Igor Kovalenko wrote:

“Tomas Högström” <> tomas@scandicraft.se> > wrote in message
news:> 3BD31416.CF4380B1@scandicraft.se> …

Thanks for the advice. Yes, some kind of RAID is perhaps the
way to go. There are RAID solutions which looks like a
standard disk (SCSI 160MB/s) to the controller which perhaps is
an interesting alternative since we can use standard controllers.
There is a solutions from transtech which contains 8 EIDE disks behind a
SCSI 160 interface which gives rather good price / performance due to the
lower EIDE disk price. Somewhat too large for our needs, and we could
probably do with only four such disks.



As long as you got a driver for that > :wink:


What about maximum file size? 1hr uncompressed video would give a 72GB

file.

But if bypassing the filesystem that problem is of course also bypassed.


QNX filesystem would not support such big files. There are functions in libc
for ‘large file support’ but I don’t have experience with them and not sure
how they can be used with QNX filesystem.

If you have reasonable business case, give a call to their sales. If enough
people will do that, they will come up with solution. So far standard excuse
was something like ‘we do not have much market for high-end storage
systems’.

Hmm … could it be that QSSL is overlooking something?? How is it
possible to support (high performance) multi media with out a performant
file system ??

wxGTK has a stupid bmp file loader. It takes 5 sec to load a 900K bmp
file with SUSE 7.2 … but it takes 40 sec with QNX 6.1 and the
dev-eide eats up 80% of the CPU power of a 700MHz machine =:-/

Isn’t is a great multi media system ?

Armin Steinhoff

We have solution for video recording for security purposes under QNX6.
The device consist of:

  • PCI video compression board with wavelet compression engine and
    multiplexer
    ( Compact PCI board is being prepared for next year )
  • RAID 1 EIDE drivers with hotswap and autorebuid functions
    We use raw disk access.


    Tomas Högström <tomas@scandicraft.se> píse v diskusním
    pøíspìvku:3BD206A0.7C390B07@scandicraft.se

We need to record video to disk, some 20Mbytes/second sustained.
It needs to be a compact rackmount PC solution, preferrebly
some PCI adapter to put in a 1U (or maybe 2U) computer.
Anything on the market with QNX6.1 support? A 1U external disk
box with suitable interface connected to a 1U computer could be an
alternative.; perhaps a nice alternative since a standard (SCSI)
controller can be used.
The framegrabber will sit in the same PC. Will the PCI bus choke?
Could it be possible to make PCI DMA directly from he framegrabber to
the disk controller to remove the unnecessary data copying via the RAM?

Cheers / Tom

“Armin Steinhoff” <a-steinhoff@web_.de> wrote in message
news:3BD53F18.6090906@web_.de…

Igor Kovalenko wrote:
“Armin Steinhoff” <a-steinhoff@web_.de> wrote in message
news:3BD45A61.200@web_.de…

wxGTK has a stupid bmp file loader. It takes 5 sec to load a 900K bmp
file with SUSE 7.2 … but it takes 40 sec with QNX 6.1 and the
dev-eide eats up 80% of the CPU power of a 700MHz machine =:-/

Isn’t is a great multi media system ?



It sounds like your EIDE is not working in DMA mode. Note, even if you
passed dma option to driver, that does not automatically mean it is used
:wink:

I’m sure that dev-eide is using DMA …

How you are sure about that? There’e whole bunch of reasons why it could
not use DMA even when you told it to do so…

  • igor

Igor Kovalenko wrote:

“Armin Steinhoff” <a-steinhoff@web_.de> wrote in message
news:3BD45A61.200@web_.de…

wxGTK has a stupid bmp file loader. It takes 5 sec to load a 900K bmp
file with SUSE 7.2 … but it takes 40 sec with QNX 6.1 and the
dev-eide eats up 80% of the CPU power of a 700MHz machine =:-/

Isn’t is a great multi media system ?



It sounds like your EIDE is not working in DMA mode. Note, even if you
passed dma option to driver, that does not automatically mean it is used > :wink:

I’m sure that dev-eide is using DMA …

Armin

  • igor

Jiri Kristek wrote:

We have solution for video recording for security purposes under QNX6.
The device consist of:

  • PCI video compression board with wavelet compression engine and
    multiplexer
    ( Compact PCI board is being prepared for next year )
  • RAID 1 EIDE drivers with hotswap and autorebuid functions
    We use raw disk access.

… and what about your disk performance??

Armin

Tomas Högström <> tomas@scandicraft.se> > píse v diskusním
pøíspìvku:> 3BD206A0.7C390B07@scandicraft.se> …

We need to record video to disk, some 20Mbytes/second sustained.
It needs to be a compact rackmount PC solution, preferrebly
some PCI adapter to put in a 1U (or maybe 2U) computer.
Anything on the market with QNX6.1 support? A 1U external disk
box with suitable interface connected to a 1U computer could be an
alternative.; perhaps a nice alternative since a standard (SCSI)

controller can be used.

The framegrabber will sit in the same PC. Will the PCI bus choke?
Could it be possible to make PCI DMA directly from he framegrabber to
the disk controller to remove the unnecessary data copying via the RAM?

Cheers / Tom
\

“Armin Steinhoff” <a-steinhoff@web_.de> wrote in message
news:3BD53FF1.6070804@web_.de…

Jiri Kristek wrote:
We have solution for video recording for security purposes under QNX6.
The device consist of:

  • PCI video compression board with wavelet compression engine and
    multiplexer
    ( Compact PCI board is being prepared for next year )
  • RAID 1 EIDE drivers with hotswap and autorebuid functions
    We use raw disk access.

… and what about your disk performance?

I guess not very important in this case since it appears the PCI does
video compression. Looks like they circle around the problem :wink:

Armin



Tomas Högström <> tomas@scandicraft.se> > píse v diskusním
pøíspìvku:> 3BD206A0.7C390B07@scandicraft.se> …

We need to record video to disk, some 20Mbytes/second sustained.
It needs to be a compact rackmount PC solution, preferrebly
some PCI adapter to put in a 1U (or maybe 2U) computer.
Anything on the market with QNX6.1 support? A 1U external disk
box with suitable interface connected to a 1U computer could be an
alternative.; perhaps a nice alternative since a standard (SCSI)

controller can be used.

The framegrabber will sit in the same PC. Will the PCI bus choke?
Could it be possible to make PCI DMA directly from he framegrabber to
the disk controller to remove the unnecessary data copying via the RAM?

Cheers / Tom


\

I don’t know if that is “circling around the problem”. Seems like a
common sense thing to do anyway, after all how many minutes of
uncompressed video can you store on a 80GB drive at 20MB/sec ?

-----Original Message-----
From: Mario Charest [mailto:mcharest@clipzinformatic.com]
Posted At: Tuesday, October 23, 2001 5:14 AM
Posted To: os
Conversation: high performance disk solution
Subject: Re: high performance disk solution



“Armin Steinhoff” <a-steinhoff@web_.de> wrote in message
news:3BD53FF1.6070804@web_.de…

Jiri Kristek wrote:
We have solution for video recording for security purposes under
QNX6.
The device consist of:

  • PCI video compression board with wavelet compression engine and
    multiplexer
    ( Compact PCI board is being prepared for next year )
  • RAID 1 EIDE drivers with hotswap and autorebuid functions
    We use raw disk access.

… and what about your disk performance?

I guess not very important in this case since it appears the PCI does
video compression. Looks like they circle around the problem :wink:

Armin



Tomas Högström <> tomas@scandicraft.se> > píse v diskusním
pøíspìvku:> 3BD206A0.7C390B07@scandicraft.se> …

We need to record video to disk, some 20Mbytes/second sustained.
It needs to be a compact rackmount PC solution, preferrebly
some PCI adapter to put in a 1U (or maybe 2U) computer.
Anything on the market with QNX6.1 support? A 1U external disk
box with suitable interface connected to a 1U computer could be an
alternative.; perhaps a nice alternative since a standard (SCSI)

controller can be used.

The framegrabber will sit in the same PC. Will the PCI bus choke?
Could it be possible to make PCI DMA directly from he framegrabber
to
the disk controller to remove the unnecessary data copying via the
RAM?

Cheers / Tom


\

Thank you for your reply Bill!

“Bill Caroselli (Q-TPS)” wrote:

Writing to /dev/hd0 is not much more efficient. The extra overhead of the
file system is born only during the open() and when the file must be grown
beyond it’s current extent. I.E. there is a very nominal performance
improvement when you pregrow a file.

I see. In the application we are developing, the file will probably grow constantly since we are
doing data logging with large amount of data and we don’t know how much data it will be.

In this case, will the overhead of the file system be significant?
Perhaps one could do the growing in larger steps, but will this be costly? We are probably already
on the edge of performance here, so we don’t want the get au uneven data flow to the disk.

/Mikael


Mikael Karlsson (MSc)
Department of Radar Sensors
Swedish Defence Research Agency E-MAIL: mikkar@foi.se
PO. Box 1165 PHONE: Int. +46 13 37 84 75
SE-581 11 Linkoping, SWEDEN FAX: Int. +46 13 37 84 88

“Rennie Allen” <RAllen@csical.com> wrote in message
news:64F00D816A85D51198390050046F80C9012872@exchangecal.hq.csical.com

I don’t know if that is “circling around the problem”. Seems like a
common sense thing to do anyway,

Not if you don’t want to loose information. Most common video compression
generate significant lost of data. I spoke to someone doing advance
object recognition in 3D with multiple cameras (14), they had the same
type of required (20Mb/Sec) per camera… I suggested they compress the
video,
and he looked close to being offended :wink: The noise and artifact
generated by video compression render their software almost
inoperable.

A Stradivarius doesn’t sound like a Stardivarius if listen through a mp3
files.
That’s important to some people :wink:

uncompressed video can you store on a 80GB drive at 20MB/sec ?

-----Original Message-----
From: Mario Charest [mailto:> mcharest@clipzinformatic.com> ]
Posted At: Tuesday, October 23, 2001 5:14 AM
Posted To: os
Conversation: high performance disk solution
Subject: Re: high performance disk solution



“Armin Steinhoff” <a-steinhoff@web_.de> wrote in message
news:3BD53FF1.6070804@web_.de…
Jiri Kristek wrote:
We have solution for video recording for security purposes under
QNX6.
The device consist of:

  • PCI video compression board with wavelet compression engine and
    multiplexer
    ( Compact PCI board is being prepared for next year )
  • RAID 1 EIDE drivers with hotswap and autorebuid functions
    We use raw disk access.

… and what about your disk performance?

I guess not very important in this case since it appears the PCI does
video compression. Looks like they circle around the problem > :wink:


Armin



Tomas Högström <> tomas@scandicraft.se> > píse v diskusním
pøíspìvku:> 3BD206A0.7C390B07@scandicraft.se> …

We need to record video to disk, some 20Mbytes/second sustained.
It needs to be a compact rackmount PC solution, preferrebly
some PCI adapter to put in a 1U (or maybe 2U) computer.
Anything on the market with QNX6.1 support? A 1U external disk
box with suitable interface connected to a 1U computer could be an
alternative.; perhaps a nice alternative since a standard (SCSI)

controller can be used.

The framegrabber will sit in the same PC. Will the PCI bus choke?
Could it be possible to make PCI DMA directly from he framegrabber
to
the disk controller to remove the unnecessary data copying via the
RAM?

Cheers / Tom




\

It looks like we perhaps needs to consider using another OS after all.
Background: we are developing a stabilized camera system which shall
use QNX for data processing. We also need to be able to store uncompressed
video together with accurate time stamping for each frame. The video storage can

really be done in any OS, but we chose QNX since:

  1. We need to come up with something fast.
  2. We need to customize the system in order to be able to store time stamp with
    each frame
  3. We are more experienced in QNX than other OS:es (i.e. it will be faster to
    achieve 1+2 )

But if we cannot get some 20MBytes/s throughput despite the right hardware,
we have to rethink. Can we?

It’s a research project, so I guess the business case towards QSSL isn’t very
good…

“Mikael Karlsson” <mikkar@foi.se> wrote in message
news:3BD67F4A.4A474A55@foi.se

I see. In the application we are developing, the file will probably grow
constantly since we are
doing data logging with large amount of data and we don’t know how much
data it will be.

In this case, will the overhead of the file system be significant?
Perhaps one could do the growing in larger steps, but will this be costly?
We are probably already
on the edge of performance here, so we don’t want the get au uneven data
flow to the disk.

I had to deal with an application like this once. It consumed 150-250 MB a

day. What we did was to allocate 300 MB each day. I.E.
open() for create
lseek() to 300000000
write() 1 bytes
close()
Then during the day we could write sequentially to the file without growing
it. At the end of the 24 hour day we truncated the file at whatever the
size needed to be. This way any unused disk space would be freed back to
the file system.

Note: This was on QNX4 but I think the file system works pretty much the
same.


Bill Caroselli – 1(530) 510-7292
Q-TPS Consulting
QTPS@EarthLink.net

“Bill Caroselli (Q-TPS)” wrote:

“Mikael Karlsson” <> mikkar@foi.se> > wrote in message
news:> 3BD67F4A.4A474A55@foi.se> …
I see. In the application we are developing, the file will probably grow
constantly since we are
doing data logging with large amount of data and we don’t know how much
data it will be.

In this case, will the overhead of the file system be significant?
Perhaps one could do the growing in larger steps, but will this be costly?
We are probably already
on the edge of performance here, so we don’t want the get au uneven data
flow to the disk.

I had to deal with an application like this once. It consumed 150-250 MB a
day. What we did was to allocate 300 MB each day. I.E.
open() for create
lseek() to 300000000
write() 1 bytes
close()
Then during the day we could write sequentially to the file without growing
it. At the end of the 24 hour day we truncated the file at whatever the
size needed to be. This way any unused disk space would be freed back to
the file system.

Note: This was on QNX4 but I think the file system works pretty much the
same.

That may be a solution, but the data flow we are talking about is at least 20 MB/s sustained for
20-30 minutes.

As Tomas and Igor has discussed, the maximum file size is an issue here as well. But it might be
possible to allocate several files with the maximum file size allowed, thus getting a file pool that
can be filled with data as needed. The question will then be if this will give us the performance we
need? As other answers in this thread suggests, this may not be the case.

/Mikael


Mikael Karlsson (MSc)
Department of Radar Sensors
Swedish Defence Research Agency E-MAIL: mikkar@foi.se
PO. Box 1165 PHONE: Int. +46 13 37 84 75
SE-581 11 Linkoping, SWEDEN FAX: Int. +46 13 37 84 88