aws s3 sync --exclude does not properly skip files/directories that are not readable

[ianslai]

I have a script that automates aws s3 sync of the contents of a disk, and I want to exclude the lost+found directory. However, the user the script runs at cannot read lost+found, which means when everything else is transferred properly, aws s3 sync ... --exclude 'lost+found' always fails with status 2. Ideally aws s3 sync should not even try to examine the directory and simply obey the pattern passed to --exclude.

Steps to reproduce:

mkdir /tmp/backup
touch /tmp/backup/foo
mkdir /tmp/backup/lost+found
chmod go-rx /tmp/backup/lost+found
sudo chown root:root /tmp/backup/lost+found
# Replace s3://mybucket/foo with a URL the current user has access to
aws s3 sync --dryrun /tmp/backup s3://mybucket/foo --exclude 'lost+found'

This generates the following output:

warning: Skipping file /tmp/backup/lost+found. File/Directory is not readable.
(dryrun) upload: ../backup/foo to s3://kyk-bi-backups/foo/foo 
$ echo $?
2

If you skip the sudo chown line, then this works properly and exits with status 0.

[joguSD]

Very similar to #2473
Fix would be the same (apply excludes at the traversal level). I'm gonna close this issue in favor of the existing one.

[ianslai]

Thanks @joguSD! I didn't see the other issue when I did my search earlier.