Support for excluding files from analysis

[jvassev]

Hi,
I am using rdfind to find duplicate files in single folder which is used by a downloader service. Occasionally, the downloader fetches the same file under a different name and rdfind is able to successfully dedup.

I noticed that rdfind (which I run in a loop every 30s) does a lot redundant work while the file is being downloaded.

Is it possible to say "ignore file that were modified in the last X seconds from analysis"? Or maybe use a globbing pattern to exclude *.part files?

[bes-internal]

You can use system find as in an example in man rfdind:

 Search for duplicate files in directories called foo:
              find . -type d -name foo -print0 |xargs -0 rdfind

In your case is something like this:

find pathtodir -type f  ! -name '*.part' -print0 |xargs -0 rdfind
find pathtodir -type f  -mmin 1 -print0 |xargs -0 rdfind
                          ^-- File's data was last modified n minutes ago.

[pauldreik]

Sorry for the late answer. The suggestion by @bes-internal is excellent!

[entrity]

I think the xargs answer isn't sufficient in large collections. I want to do this on a large tree, and I get xargs: argument line too long.