QNX2 osconfig, cache, misc disk related questions

QNX USERS QNX2 and QNX4
1

I’ve got a handful of questions about just some assorted stuff…

  • osconfig - does this really offer that much in the way of “configuration?” The QNX manual doesn’t really say anything useful about it… I’d almost say the screen seems self-explanatory, but there’s no context about the “number of open files” or “number of extra segments”
    – what is too many open files? Our software sets its default value to 128, but would it be of any benefit for me to increase that value?
    – what are “extra segments?” our systems are set at 800… again, is higher better? Is lower better?

These systems use 2.21atpb, and as I understand it, the “p” is protected and “b” is for “big files…”

Some of our logged data can reach +50mb in size occasionally… Those files are not specifically opened or manipulated on these systems, they typically get transferred to a server via FTP, but is their size any concern as the software writes them to the disk?

  • mount cache/xcache/bmcache - by default, our software configures the sys.init file as such:
mount cache d=3 s=16k
mount xcache s=10k
mount bmcache d=3
  • the manual says 48k is “a good size for a cache,” so I assume I could bump that number up a bit…I was thinking at least 32k. Is there any greater benefit of setting this even higher than 32? What if I did 64k?
  • If I mount multiple disks (say two or three extra partitions), is there any harm in enabling caches for all disks? Should I increase xcache size?

I am assuming the system only sees like 16mb of the 1gb installed memory; when I run tsk mem, it does show a chunk of 13.8mb available at address 201800…

  • does the system know well enough to utilize that extra memory, or are programs/commands/background routines beholden to only a narrow range of working memory potentially specified by how they were written?
  • is bumping up disk cache cannibalizing integral ram for other functions?

I appreciate the help thus far, I feel like I’ve gotten to a place where I can at least navigate these systems and I don’t feel too intimidated about trying to make them do what I need as far as file/system management. There’s still a lot of undocumented parts of the actual software we use, but this QNX nut definitely feels like it’s nearly cracked for most of what I need at the moment.

2

I am not sure that the cache/xcache/bmcache options matter any more.

When QNX2 was released, physical drives were very slow and did not have any local caching so any disk operation (even doing directories) meant spinning the disk and waiting for results. These days they are orders of magnitude faster and most have megabytes worth of caching. I can’t recall the last time I actually adjusted disk caching of any kind (probably close to 20 years ago) or seeing O/S software caching matter in terms of performance.

Also, the larger the cache, the more data that gets into the cache before it gets written to disk. Which means possible data loss if you have unexpected power off (I assume you have UPS so that’s not an issue bug figured I’d mention it).

Tim

3

osconfig allows a number of uses. For example if you have an 8 port serial card you can plug in the io-port values. The number of open files is just that, the maximum number of files that can be open at one time. I never found a need to increase the default. It only matters if your application needs to open more. I should note that in each task there is memory overhead for opening a file, so one task may not be able to exceed the system maximum on its own.

Extra-segments: QNX 2 runs in a 16 bit mode. Programs can have 64K of memory as well as 64K of code. There are ways around both of these. In the case of code, the linker supports the use of overlays. This is a little like virtual memory swapped to disk, but not as flexible. If you need more data space you can allocate an extra segement, up to 64k and access it by setting the ES register. I think the command is set_es_reg(unsigned seg) or something similar. Once set you can access the extra segment using the non-standard ‘@’ symbol instead of ‘*’. For example:

#define SEG_SIZE 0x8000
unsigned seg;
unsigned char *p;
int i;

seg = get_extra_seg(SEG_SIZE); ← This is fake, I don’t remember the call, but it exists.
set_extra_seg(seg); <— again I dont remember the call name

/* init segment *.
p = 0;
for(i=0;i<SEG_SIZE;i++)
{
@p++ = 0;
}

"These systems use 2.21atpb, and as I understand it, the “p” is protected and “b” is for “big files…”

If you have an 8088 or 8086 machine it cannot use the ‘p’ version.
I’m not sure about the ‘b’. I thought it was a version that supported more tasks.

We’ve discussed file size before and I’m not sure of the QNX 2 limit.

  • is bumping up disk cache cannibalizing integral ram for other functions?"

Yes!

The only issue I see with cache is that you are using up RAM. If you can afford to, you should use it. It’s at most 64K on a protected mode system which can have 16Meg.

I do not think what Tim suggests, that cached data has a delayed write. This was true in QNX 4 and QNX 6. I believe that the cache is read only. When you write, the driver is called immediately to send data to the disk. When write() returns, the data is on disk. I’m not sure what the maximum per disk is, however mount will fail if it is too large so start at 64K and work your way down.

There is one issue with pulling the plug, even with the non-caching disk.at driver. When you create a file or increase its size, the extent information is not written to disk immediately. One way to mitigate this is whenever you create a file, fill it out to its maximum size and close it. Then re-opening it in “rw” mode. There is also a programmatic way to force the extent information to disk, but any specific program might not handle this.

I have software that does this in case you need it, but with a UPS probably not.

Well, what I said about write() is mostly true. I believe that all the QNX written drivers push write data directly to disk. There is a disk.atc driver that does some read caching in the driver. If you read a sector, more than one continguous sector in an extent can be read all at once. The QNX drivers do not do write caching.

However, I wrote a driver for the Iomega Bernoulli Box. It’s rotational speed was very low. To get good performance I would read an entire track on any read. I also would delay writing in case continuous writes could be used to speed things up. This created the possibility of corruption. To mitigate this I locked the drives and required the user to run an unlock/spin-down command. This did not protect against someone pulling the plug, but neither does the disk.atc driver. It just has less damage that you can do.

xcashe, cashes extent information. Each file is made up of groups of continuous sectors called extents. There is a 16 byte extent header on each extent. This means that the first sector of a file always has at most 512-16 bytes. Other sectors that are part of the extent have 512 bytes. A small xcache is worth while.

bmcache caches the bitmap of a disk. The bitmap is at the begining of a disk after the first extent of the root directory. Each bit in the bitmap file represents the allocation state of one sector. Using the bmcache is worthwhile.