How do you detect renames?

[OS: QNX4.25]

Hi all,

I’m looking for a way to reliably detect file/directory renames or
moves. My first thought was to keep track of the serial number (the
st_ino field in the stat struct): If the serial number is the same, but
the name has changed, then the file was renamed/moved.

Wrong! It turns out that serial numbers sometimes get re-used :frowning:. Now I
also keep track of the file modification time (st_mtime), and add the
condition that the st_mtime should be unchanged. Are these conditions
enough to uniquely detect renames?

[I’ve also considered resource managers and might go that way in some future point in time. But that’s a rather big step right now…]

Also, can anyone shed some light on the allocation policy of file serial
numbers, or give some pointers to reading material?


Thanks a million,

rick

In article <39BEA5CC.CBB3DA55@SPAM.REDIRECTED.TO.DEV.NULL>,
Rick Lake <rwlake@SPAM.REDIRECTED.TO.DEV.NULL> wrote:

[OS: QNX4.25]

Hi all,

I’m looking for a way to reliably detect file/directory renames or
moves. My first thought was to keep track of the serial number (the
st_ino field in the stat struct): If the serial number is the same, but
the name has changed, then the file was renamed/moved.

Wrong! It turns out that serial numbers sometimes get re-used > :frowning:> . Now I
also keep track of the file modification time (st_mtime), and add the
condition that the st_mtime should be unchanged. Are these conditions
enough to uniquely detect renames?

[I’ve also considered resource managers and might go that way in some future point in time. But that’s a rather big step right now…]

Also, can anyone shed some light on the allocation policy of file serial
numbers, or give some pointers to reading material?

So you want to detect file/directory renames or moves…hmm. Its the
‘or moves’ listed as being distinct from renames which makes me wonder
what you really want to do. Within a filesystem, rename() is the function
that accomplishes a move of a file to any location within the filesystem.
However, if you also count as a move an operation that spans filesytems
(such as the mv utility would do in this case, doing a cp and an rm) then
you will have to resort to other techniques such as looking at checksums
etc. There is also the issue of ‘what don’t you want to see’ since your
basic technique will also identify new hard links. That said…

I think you’re on the right track. You might try the QNX-specific st_ftime
(file creation time) in combination with st-ino/st-dev. It still isn’t
bulletproof, but as long as your system time doesn’t get set back
substantially the risk should be very minimal. Your use of st_mtime seems
odd - you wouldn’t see a file that had been appended to as being the
same file, even if it was. But, perhaps that is your intent in which
case you could look at both st_ftime and st_mtime rather than rely on
just st_ftime. If this doesn’t work for some reason remember this advice
is probably worth exactly what you paid for it. :slight_smile:

Please also note that st_ino isn’t unique except in combination with st_dev!
QNX’s prefix mappings allow a new st_dev to pop up pretty well anywhere as
you traverse a pathname space, so beware. Another caveat - I’d be extra
cautious about relying on this for anything but a QNX filesystem. With DOS
filesystems, CIFS, NFS,… all bets are off. In these cases st_ino and/or
st_ftime are fictional characters.


Eric Johnson
QA Mgr, QNX Software Systems Ltd.

Eric Johnson wrote:

In article <> 39BEA5CC.CBB3DA55@SPAM.REDIRECTED.TO.DEV.NULL> >,
Rick Lake <> rwlake@SPAM.REDIRECTED.TO.DEV.NULL> > wrote:
[OS: QNX4.25]

Hi all,

I’m looking for a way to reliably detect file/directory renames or
moves. My first thought was to keep track of the serial number (the
st_ino field in the stat struct): If the serial number is the same, but
the name has changed, then the file was renamed/moved.

Wrong! It turns out that serial numbers sometimes get re-used > :frowning:> . Now I
also keep track of the file modification time (st_mtime), and add the
condition that the st_mtime should be unchanged. Are these conditions
enough to uniquely detect renames?

[I’ve also considered resource managers and might go that way in some future point in time. But that’s a rather big step right now…]

Also, can anyone shed some light on the allocation policy of file serial
numbers, or give some pointers to reading material?

So you want to detect file/directory renames or moves…hmm. Its the
‘or moves’ listed as being distinct from renames which makes me wonder
what you really want to do. Within a filesystem, rename() is the function
that accomplishes a move of a file to any location within the filesystem.
However, if you also count as a move an operation that spans filesytems
(such as the mv utility would do in this case, doing a cp and an rm) then
you will have to resort to other techniques such as looking at checksums

Ah yes. I should have narrowed down the conditions a bit more. Indeed
all files and directories are on the same device, same partition, same
filesystem. I said “moves” to imply the case where a file or directory
gets “renamed” to a different directory, but keeping it’s original name
and contents.

What I’m actually building is a special FTP mirror daemon which sends a
dir tree to a remote host, keeping track of what has already been sent.
If the local file gets modified, it gets re-sent. If the local file gets
renamed, the remote counterpart is also renamed using the RNFR/RNTO FTP
command pair. This is more efficient than DELEting the old file and
re-sending it. (Especially when renaming directories)

etc. There is also the issue of ‘what don’t you want to see’ since your
basic technique will also identify new hard links. That said…

Hard links would normally not be present on the local tree. However if
there are, I speculate the daemon would find the first link in its
readdir(), send it and then rename it remotely when it finds the second
link. Then in the next loop, it would find the first link and then
rename it again, and so on. I would have to test that sometime to be
sure…

I think you’re on the right track. You might try the QNX-specific st_ftime
(file creation time) in combination with st-ino/st-dev. It still isn’t
bulletproof, but as long as your system time doesn’t get set back
substantially the risk should be very minimal. Your use of st_mtime seems
odd - you wouldn’t see a file that had been appended to as being the
same file, even if it was. But, perhaps that is your intent in which

Are you saying that appending to a file can sometimes lead to it getting
a different serial number?

Just to be clear, this is how I test now:

if (serialnum_unchanged && fullpath_changed && modtime_unchanged)
it_was_a_rename_indeed();

case you could look at both st_ftime and st_mtime rather than rely on
just st_ftime. If this doesn’t work for some reason remember this advice
is probably worth exactly what you paid for it. > :slight_smile:

Then I got more value than what I paid for :wink:

Please also note that st_ino isn’t unique except in combination with st_dev!

So in other words: within a given device, the st_ino is unique, right?

QNX’s prefix mappings allow a new st_dev to pop up pretty well anywhere as
you traverse a pathname space, so beware. Another caveat - I’d be extra
cautious about relying on this for anything but a QNX filesystem. With DOS
filesystems, CIFS, NFS,… all bets are off. In these cases st_ino and/or
st_ftime are fictional characters.

Agreed. But I specifically chose to do this in QNX.


Eric Johnson
QA Mgr, QNX Software Systems Ltd.

Thanks very much for the detailed outlining.

rick

In article <39BFC4F4.6B295D49@SPAM.REDIRECTED.TO.DEV.NULL>,
Rick Lake <rwlake@SPAM.REDIRECTED.TO.DEV.NULL> wrote:

What I’m actually building is a special FTP mirror daemon which sends a
dir tree to a remote host, keeping track of what has already been sent.
If the local file gets modified, it gets re-sent. If the local file gets
renamed, the remote counterpart is also renamed using the RNFR/RNTO FTP
command pair. This is more efficient than DELEting the old file and
re-sending it. (Especially when renaming directories)

cool


I think you’re on the right track. You might try the QNX-specific st_ftime
(file creation time) in combination with st-ino/st-dev. It still isn’t
bulletproof, but as long as your system time doesn’t get set back
substantially the risk should be very minimal. Your use of st_mtime seems
odd - you wouldn’t see a file that had been appended to as being the
same file, even if it was. But, perhaps that is your intent in which

Are you saying that appending to a file can sometimes lead to it getting
a different serial number?

No, the serial number will remain the same. I was more getting at what
your criteria of ‘sameness’ was. i.e. if a file has the same name and
same st_dev, st_ino, and st_ftime, is that sufficient, or are you interested
in sameness of contents, which could easily be different for a file that
had been appended or written to. If you want to add sameness of contents
to the sameness criteria, you would add the requirement that st_mtime
also match before you assumed it was the same.


\

Eric Johnson
QA Mgr, QNX Software Systems Ltd.

Rick Lake wrote:

I’m looking for a way to reliably detect file/directory renames or
moves. My first thought was to keep track of the serial number (the
st_ino field in the stat struct): If the serial number is the same, but
the name has changed, then the file was renamed/moved.

Wrong! It turns out that serial numbers sometimes get re-used > :frowning:> . Now I
also keep track of the file modification time (st_mtime), and add the
condition that the st_mtime should be unchanged. Are these conditions
enough to uniquely detect renames?

Also, can anyone shed some light on the allocation policy of file serial
numbers, or give some pointers to reading material?

The QNX4 filesystem does not use ino in the traditional UNIX sense
(ie an index in a fixed inodes file; an entry is only placed in “.inodes”
if the file has >14 char filename length or has multiple hard links).
Otherwise the ino is fabricated from the physical location of the files
directory entry (also a unique number). So, a rename (within the same
filesystem) will cause the ino (position-generated) of a file to change
UNLESS it was already in an invariant position in “.inodes” because of
the name length or a “ln” operation OR if there is no need to change it,
because it is renamed within the same directory.

So, I’m not sure it can be done, reliably (causes problems for NFS too :frowning:.