QNX2 omit directories when using "backup" command?

QNX USERS QNX2 and QNX4
1

Not sure how strict the etiquette is on the forums but I feel like my progress has evolved enough from my last post to warrant a new question… I am now trying to figure out how to omit a directory (or directories) from the backup command.

I have the manual in front of me and it lists the option pd=[^]pattern - “Indicates that only directories with names that match (or don’t match) this pattern are to be copied.”

It doesn’t have a usage example but it seems self-explanatory… except I can’t seem to get it to work. I have a directory 3:/log/data/[other directories] that houses gigabytes of data which are not a priority when trying to make an otherwise full drive backup, and no matter how I structure the option, it still tries to copy the /data/ directory.
Do I need an asterisk somewhere at the end as a wild card?
Do I need a slash at the end or no?
Do I need quotes anywhere?
Is the position of the variable in the command important?

This is the sequence I go through when performing this process, basically an established running machine and I come in with a fresh blank hard drive.

install backup drive as slave
boot from floppy

mount disk 3 1:/drivers/disk.ata pa=7
mount disk 4 d=3 p=2 pa=7
fdisk 4

os type 7, start cyl 0, end cyl 999, save quit

mount disk 4 d=3 p=2 pa=7
dinit 4 +h
backup 3:/ 4:/ +a s=c p=^bitmap +q
logoff

power machine off
connect second drive as single drive
reboot from floppy

mount disk 3 1:/drivers/disk.ata pa=7
boot 3:/netboot/os.2.21atpb c=3:/netboot/config_file  d=3:/drivers/disk.ata +H +P +q
fdisk 3

make bootable, save quit
restart and confirm boot/function

For whatever reason, I have not needed to provide t/h/n info when specifying the drive, not sure if that’s the bios or just the drive controllers themselves but it’s a small luxury.

I was previously specifying 1023 as the “end cylinder” when formatting, but got repeated failure to copy errors so I dialed it back to 999 (still about 8 gigs at least).

What might I be overlooking when trying to omit these directories?

2

A few things to say here.

I’ve never tried to use the pattern parameter of backup so I don’t know h ow to advise you on it. If I were in your shoes, I would just get a listing of directories below the root and use backup on each one of them except data.

The driver you mentioned, disk.ata. I think I wrote it. QNX did not have a driver for IDE that used LBA (logical block addressing). The drive is addressed by sector number. 0, 1 … last. It can show a t.h/n configuration but it is fake. You need this to boot using older BIOS’s that don’t support LBA booting.

8 Gig you say. I’m amazed. I thought the files system limit was .5 Gig, but it’s been quite while.

3

I was looking at a copy of the docs here:

If you post the exact command you are trying to use it might help us figure out why it isn’t matching because there are a lot of pattern matching rules for files vs directories etc.

Tim

4

I’m also willing to play around with it on one of my Q2 machines.

5

I could potentially do directory by directory, but in time I’d like to make this process as streamlined as possible because I still have other fires within our software or this OS that I need to discover and put out.

The driver is just one included with the software package, has a file date of 10-Sep-96. Don’t see any specific remarks about it, but are there any limitations you might recall? Should I configure the bios for CHS instead of LBA or the “auto” mode that I think it is currently on?

A small part of me has been wondering if we’ve been too trusting of the OS to blindly utilize disks so large (for ~8gb partition, does a bitmap file of apx 2mb or ~4000 blocks sound about right?)… but some of our tests really do log like dozens or hundreds of files between 10 - 50mb a piece, so the storage is necessary.

6

Oof, the OCR scan of that manual butchered a lot. I’ve got copies printed from either Tenox’s files or the wayback machine, thankfully :o

I tried the string two or three ways before giving up and just trying to storm ahead (the drive I was trying to backup was on its way out and i couldn’t keep screwing around… it finally fully died in the process though, so hopefully enough of our test configurations were copied.)

backup 3:/ 4:/ +a s=c p=^bitmap pd=^/log/data +q
backup 3:/ 4:/ +a s=c p=^bitmap pd=^3:/log/data/* +q

I forget if I just tried pd=^log but now it irks me if that was the trick, because there is only one directory named “log” in these systems so I wouldn’t have had to worry about unknowingly omitting anything elsewhere on the drive.

I just tested a few quick things myself here, here are some captures;

3:/$ backup /testing /testing2 pd=^*3                     Sep 20  8:49:44 am  ý
Load disks in drives and hit <CR>
Making directory: /testing2/dir1
Making directory: /testing2/dir2
Making directory: /testing2/dir4
3:/$ cd testing2
3:/testing2$ rmdir dir1 dir2 dir4
3:/testing2$ ls
3:/testing2$ cd ..
3:/$ backup /testing /testing2 pd=^3
Load disks in drives and hit <CR>
Making directory: /testing2/dir1
Making directory: /testing2/dir2
Making directory: /testing2/dir3
Making directory: /testing2/dir4
3:/$ cd testing2
3:/testing2$ rmdir dir1 dir2 dir3 dir4
3:/testing2$ ls
3:/testing2$ cd ..
3:/$ backup /testing /testing2 pd=^dir3
Load disks in drives and hit <CR>
Making directory: /testing2/dir1
Making directory: /testing2/dir2
Making directory: /testing2/dir4
3:/$

3:/testing$ cd /testing2                                  Sep 20  8:55:01 am  ý
3:/testing2$ ls
3:/testing2$ cd ..
3:/$ backup /testing /testing 2 pd=^*a*
BACKUP: invalid option   2
Type carriage return to continue.
3:/$ backup /testing /testing2 pd=^*a*
Load disks in drives and hit <CR>
Making directory: /testing2/dir1
Making directory: /testing2/dir1/dir1b
Making directory: /testing2/dir1/dir1c
Making directory: /testing2/dir1/dir1d
Making directory: /testing2/dir2
Making directory: /testing2/dir2/dir2b
Making directory: /testing2/dir2/dir2c
Making directory: /testing2/dir2/dir2d
Making directory: /testing2/dir3
Making directory: /testing2/dir3/dir3b
Making directory: /testing2/dir3/dir3c
Making directory: /testing2/dir3/dir3d
Making directory: /testing2/dir4
Making directory: /testing2/dir4/dir4b
Making directory: /testing2/dir4/dir4c
Making directory: /testing2/dir4/dir4d
3:/$

            ³                                             Sep 20  9:00:29 am  ý
3:/testing$ cd ..
3:/$ backup /testing /testing2 pd=^/dir3/*b
Load disks in drives and hit <CR>
Making directory: /testing2/dir1
Making directory: /testing2/dir1/dir1a
Making directory: /testing2/dir1/dir1b
Making directory: /testing2/dir1/dir1c
Making directory: /testing2/dir1/dir1d
Making directory: /testing2/dir2
Making directory: /testing2/dir2/dir2a
Making directory: /testing2/dir2/dir2b
Making directory: /testing2/dir2/dir2c
Making directory: /testing2/dir2/dir2d
Making directory: /testing2/dir3
Making directory: /testing2/dir3/dir3a
Making directory: /testing2/dir3/dir3b
Making directory: /testing2/dir3/dir3c
Making directory: /testing2/dir3/dir3d
Making directory: /testing2/dir4
Making directory: /testing2/dir4/dir4a
Making directory: /testing2/dir4/dir4b
Making directory: /testing2/dir4/dir4c
Making directory: /testing2/dir4/dir4d
3:/$

it looks like you can’t specify any kind of directory structure within the pattern of the argument… and I guess that means I could have just added pd=^log and been done with it.

I stuck the failing drive in a freezer overnight in hopes that i can squeeze a little more life out of it, as soon as I am able to get over there to grab it, this is the next thing I’ll try… if it’s even readable.

7

Yeah, from your tests I think it’s clear that it only pattern matches alphanumeric data.

That makes a lot of sense because the pattern match algorithm is probably the same for file names and directories and file names of course must be alphanumeric and can’t contain a /.

Tim

8

I could potentially do directory by directory, but in time I’d like to make this process as streamlined as possible because I still have other fires within our software or this OS that I need to discover and put out.

I wasn’t suggesting you do this manually. You could create a script to do this. If the script will change often, you could write a program to generate the script dynamically.

The driver is just one included with the software package, has a file date of 10-Sep-96. Don’t see any specific remarks about it, but are there any limitations you might recall? Should I configure the bios for CHS instead of LBA or the “auto” mode that I think it is currently on?

The agreement I had with QNX at the time was, they gave me the disk.at or disk.atc driver source and I would give back the result. The software package people might have gotten the driver from QNX. I have sold drivers in the past, but the ata driver was a freebee.

My vague recollection is that there were problems getting a BIOS that understood LBA to boot with that option. There are ways around this. A drive that is LBA will also have on startup a head/track/sector configuration. The trick is to figure out what that is and plug it into the BIOS. This allows the BIOS to properly find the OS file and load it into memory along with an image of the driver that was created with the boot command. I remember that getting this to work was not always easy. You might have to give disk.ata those parameters when you initially mount it before running the boot command.

4000 blocks is an 8Gig disk. 4000 x 512(bytes per sector) * 8 bits per byte = 16M sectors or 8 gig.

9

oh for sure, I would ideally make a script or program, but coding and scripting are not in my repertoire; I can probably figure it out in time but sadly I’m not afforded that time right now.

Wait, what’s that about the boot command? There’s a way to specify h/t/n information?

If I roll with how the utilities manual outlines the boot command, it suggests that you have to point it directly to a driver file… the d= argument…

If I have prepared a disk that I am going to make into a boot disk, and have mounted it with specific h/t/n information, would I just type d=3 instead of d=#:/location/of/driver.xyz ?
.

when I go into the fdisk command, it reports values for the heads/cylinders/etc for the drive…

But I know this is just whatever the BIOS is reporting, since the bios acts as a middleman to the disk. The disks are 40gb ultra-ata barracudas.

When I’m in the bios, it does show the actual physical cylinder/track/head/etc information from the disk, though.

If I switch the bios to CHS mode, fdisk then reports the exact same values shown in the bios. the bios also has a “large” disk mode, which translates the physical disk different than LBA mode, and fdisk also accordingly reports those different values.

The QNX manual actually makes a claim that it is possible to access a disk near to one terabyte in storage, but in my experience with re-re-re-reformatting one of these disks, I get errors whenever I try to write to a disk that has been formatted larger than 8Gb (like literally 8000-8010mb is the threshold, not 8192, not 8100, not 8032.5)

10

I admit that it’s been a while since I had to make this work, but I’m pretty sure I’m correct on this.

Wait, what’s that about the boot command? There’s a way to specify h/t/n information?

I don’t recall there being a way to specify h/t/n info with the boot command. When you mount the driver you can specify these.

If I roll with how the utilities manual outlines the boot command, it suggests that you have to point it directly to a driver file… the d= argument…

And there’s some trickiness here. It has to be the driver file that is currently mounted. If you were to instead mount the driver from a floppy, this would not work even though the two driver files might be identical.

If I have prepared a disk that I am going to make into a boot disk, and have mounted it with specific h/t/n information, would I just type d=3 instead of d=#:/location/of/driver.xyz ?
.

You are thinking of the mount command where d=3 can be used to mount another disk using the already mounted driver. The boot command want you to point at the driver file.

when I go into the fdisk command, it reports values for the heads/cylinders/etc for the drive…
But I know this is just whatever the BIOS is reporting, since the bios acts as a middleman to the disk. The disks are 40gb ultra-ata barracudas.

You are half right. These are the values that the driver grabs from the bios. These values can be different if you give “mount” h/t/n info.

When I’m in the bios, it does show the actual physical cylinder/track/head/etc information from the disk, though.

Hmmmm. How to explain this? You can think of those values as actual, but they are not. The real h/t/n information is hidden in the driver. WIth EIDE you are theoretically able to tell the drive which geometry configuration you prefer, but I don’t know a way to do this with QNX 2. If you choose LBA mode, the BIOS picks some values which as I recall include n=63.

If I switch the bios to CHS mode, fdisk then reports the exact same values shown in the bios. the bios also has a “large” disk mode, which translates the physical disk different than LBA mode, and fdisk also accordingly reports those different values.

Fdisk is asking the driver for this information and without modifying the info when you use the mount command, it reflects what the driver found in the BIOS.

The QNX manual actually makes a claim that it is possible to access a disk near to one terabyte in storage, but in my experience with re-re-re-reformatting one of these disks, I get errors whenever I try to write to a disk that has been formatted larger than 8Gb (like literally 8000-8010mb is the threshold, not 8192, not 8100, not 8032.5)

Can you describe where that claim is made? There are limitations that I know of. If you are in h/t/n mode there are limitations on the size of those parameters. I think that one of the objectives in writing the LBA driver was to overcome this limit. I read that MS-DOS ended up with a limitation of a 2G partition, which I suspect was also true of QNX2. With 4 partitions you could bump this to 8G. 2^31 bytes.

11

This is correct. It was a limitation of the 16 bit O/S. So the partition table was limited to 65535 entries (16 bit number) and each cluster was 32K in size so the max disk space was 65535*32K = 2 Gig. It also mean that each file written no matter how small used at least 32K in disk space which lead to a lot of wasted space on a drive that had lots of small files.

Tim

12

So over the past few weeks, I have been buried in QNX in an effort to rebuild two different machines which suffered two different problems… my head has been spinning with all of this because getting them back into operation is a big priority and all this stuff effectively fell in my lap over the course of this summer.

I think I misinterpreted what you said about getting head/track/sector information into the driver configuration at boot; I realize now that you meant to simply configure the BIOS with the appropriate profile for the drive, and then when the driver gets loaded, it will automatically have the correct drive size.

The main reason I have been going through all this format and configuration stuff has been an attempt to solve a strange periodic stuttering problem in our software that was occurring every time a background disk monitoring program ran its check routine; this was after I had already reinstalled everything from the ground up and used completely new hardware.
I figured it HAD to be related to some part of the disk configuration through bios settings or formatting and driver settings.

After exhausting just about all my efforts and options yesterday, I tried re-enabling the L1 and L2 cache, which I had disabled so the floppy drive would work. After turning the CPU cache back on, the stuttering went away…
I had actually typed up a long post, outlining the entire scenario and everything I had tried, that I was minutes away from submitting before I discovered that the CPU cache solved the issue.

The takeaway from all of this ^^^ is that I am now much more familiar with all the song and dance needed to format a disk for whatever our purposes may be lol

And the reason I was trying to omit directories when running a backup command was because the hard drive for a different machine, which had some ctree database error, fell off my desk and I knew it was going to fail after such a hard fall; so I was trying to copy as much off of it as I could before that happened, but the logged data files were so big that I think the drive died before it finished copying all the actual program and configuration files (which were Much More Important than the log files).

I managed to get a bulk of stuff from the dead drive but I won’t know if it’s everything until I start with a new system and try to rebuild our software configuration.

This was also good experience for the backup routine, as this will enable me to hopefully streamline my process of going around and backing things up from all of these machines over time.

Page 13 in the 2.2x manual under “Step 1b - Partitioning Your Hard Disk”
There are 4 partition slots. Use the up/down cursor keys to select one that is not in
use. Type the letter c to select the change option. You should now enter the OS
type (7 for QNX) and the start and end, cylinder to be used for the QNX partition.
Enter a carriage return after entering each number. The QNX partition may be any
size up to 1 terabyte which easily covers all disks on the market today.

A small part of me was hoping I could potentially access the entirety of these 40gb drives, it would make it so I could back up multiple machines’ drives onto just a couple drives, but like my previous coworker and I discovered separately, the backup command spits out a bunch of “unable to create file” and etc errors when a drive is just a hair larger than 8 gigs.

13

Well this sure got me curious. I could not find any documentation stating the maximum disk size supported, so I did some investigating.

To begin with, the EIDE interface that the AT driver worked with has a 137Gig limit, so definitly not close to 1T. A SCSI drive might have a higher limit. There is an enhanced interface that would allow more sectors, but it didn’t exist at the time of QNX2.

Then I looked at fdisk. Somehow I remember that max heads is 16 and max sectors is 63. The QNX2 fdisk will only allow up to 9999 cylinders. Adding this up you get about a 5Gig limit.

I still think that there is some other lower limit, but that’s all I can find.

I want to revisit the LBA booting issue. Consider what happens at boot time. The BIOS when not configured in LBA mode looks at the BIOS CMOS info and reads the h/t/n. It then must load the OS file using these parameters. When initializing the drive with the LBA driver to boot off the hard drive, the LBA offsets must coincide with the h/t/n offset or booting will fail. This is the jist of the problem.

What you need to know is what the default power on values of h/t/n are on the drive, and that is not easy to determine. The numbers listed on the outside of the drive may or may not be valid. That’s what makes this process so tricky.

Update, I found the limits in the back of the manual. It’s limits based on the variable sizes in the file system and it suggests 1T, but as I pointed out, there are other limits.

14

Oh I know there are other practical limits, but we could certainly take advantage of larger disk sizes in some of our setups. Right now at least the 8gb suits most of the machines that are in operation.

as far as the current disk, it’s a seagate model ST340014A.
According to the spec-sheet its “guaranteed sectors” total 78,165,360…
But the sectors/cylinders/heads don’t exactly add up… their sheet claims the same numbers across the 5 different capacities; 63 sectors per track, 16 heads, 16,383 cylinders.

If you multiply those, it’s nowhere near the 78m sectors for the 40gb model, much less near the total sectors listed for the larger drives

In the bios of this Pentium 4 machine, it lists different values at different settings…

Mode:  |   CHS   |   Large   |    LBA   |
Cap    | 40022MB |  40016 MB | 40018 MB |
Cyl    |  19158  |   1277    |   4865   |
Head   |    16   |    240    |    255   |
Sector |   255   |    255    |     63   |

(not sure why the monospaced font changes color as it pleases)

19,158 cylinders, 16 heads, and 255 sectors per track… multiply those and you get 78,164,640, so I am inclined to think the bios’ numbers are accurate. When I leave the bios in “auto” mode and run fdisk in qnx, it reports the same values as what the bios says in LBA mode.

It was here that I specified starting block 0 and ending block 999; qnx, or mount, or fdisk, or whichever component is responsible, doesn’t balk at the “255” heads, and the drive formats just fine at apx 8000mb. I had briefly tried formatting with 10000 cylinders in CHS mode (16 H, 255 N) which resulted in approximately a 20 gig volume, but i’m onto the new recovery project and I don’t recall whether that resulted in an actually functional disk or not at this point.

I’m definitely gonna circle back to this if I eventually get some spare time to dabble.

15

If you have figured out the magic needed to QNX 2 to boot off of the HD, you are ready to leave grasshopper.