aws s3 sync warning: Skipping file "." File does not exist.

[Fantus]

aws --version
aws-cli/1.6.10 Python/2.6.6 Linux/2.6.32-504.3.3.el6.x86_64

Hi,
iam experiencing a strange be behavior when running aws s3 sync

if i run

aws s3 sync s3://testing /home/vagrant/test-folder/

i get the following error in the very first line, the rest of the bucket is synced as expected

warning: Skipping file /home/vagrant/test-folder/. File does not exist.

however if i go into the folder and run

aws s3 sync s3://testing .

it runs successfully.

Best
Justus

[jamesls]

Confirmed. Looks to be a bug on our end, the warning is not necessary. It happens whenever the destination directory does not exist.

Repro Steps

1. Create a file in s3: $ echo "hello world" | aws s3 cp - s3://bucket/foo.txt
2. Sync down from S3 to a local directory that does not exist:

$ aws s3 sync s3://bucket/ dir-does-not-exist/
warning: Skipping file /private/tmp/scratch/dir-does-not-exist/. File does not exist.
download: s3://bucket/foo.txt to dir-does-not-exist/foo.txt

1. Note you get the warning message above but we actually do create the directory and we do copy the file down:

$ tree
.
└── dir-does-not-exist
    └── foo.txt

1 directory, 1 file
$ cat dir-does-not-exist/foo.txt
hello world

[ChrisSLT]

I see this as well when syncing from my home directory on my mac with --exclude "Library/" - it still gives me pages and pages and pages of "warning: skipping file" for every single file in my Library folder.

Is this part of the bug?

[ChrisSLT]

I had a thought - since this happens with the Library directory in my home directory, and Library is hidden, is that confusing sync?

[jamesls]

@ChrisSLT I believe you need to use a wildcard Library/* to exclude that directory. Here are some more examples: http://docs.aws.amazon.com/cli/latest/reference/s3/index.html#use-of-exclude-and-include-filters

[iMoses]

I get the same error message when using the sync command along with the --delete flag. Warnings aren't my biggest problem, the real issue is that the script exit code is 2 instead of 0.

warning: Skipping file /my/path/file.json. File does not exist.
[Errno 2] No such file or directory: '/my_path/another_file.json'

All of these files, which are causing the warnings and errors, are located on my local machine and not on s3, so I assume the right behavior should have been simply deleting the files, as the --delete flag should do.

[chiefy]

Adding onto this, not sure if it should be a separate bug:

In the following example my keys are set in my ENV vars, and there is no such location as app-configs/hello-world/dev

$ aws s3 sync s3://app-configs/hello-world/dev .

$ echo $?
0

In a fresh shell w/o AWS keys set:

$ aws s3 sync s3://app-configs/hello-world/dev .
Unable to locate credentials
Completed 1 part(s) with ... file(s) remaining

$ echo $?
1

[felipedemacedo]

I just found the same situation (lots of warnings while using aws s3 sync). Turns out the solution was simply about permissions.

Instead of
"aws s3 sync ..."
use
"sudo aws s3 sync"

[jinet]

Thank you Felipe. With sudo, it works for me too :)

…

On 11 August 2017 at 18:27, Felipe de Macêdo ***@***.***> wrote: I just found the same situation (lots of warnings while using aws s3 sync). Turns out the solution was simply about permissions. Instead of "aws s3 sync ..." use "sudo aws s3 sync" — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1082 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABYc6YqKMtDUVbkzDI8uhDot90_P1ziPks5sXP-YgaJpZM4DO35U> .

-- /* Regards, Jinet Alphonsa Jose */

[gus0074]

Didn't work for me... Windows 10 :(

[Ayase2e]

Didn't work for me... Windows 10 :(

yes same for me on windows 2012 R2.

[zhaolewen]

I'm on Windows and for me it is because either my file path is too long (thus it's cut and the file of the cut path does not exist), either it's because there are spaces in my folder path.

[3ggaurav]

Solution(Answer) -- Yes, Same problem was occurring for me also i.e. 'File does not exist' warning.
So aws s3 sync is used to synchronize folders only not for a particular file/files. It's expecting both source and target to be folders (local folders or S3 URIs). If you put your file into a folder and then sync your folder to s3 folder. It work fine. Thank you.

[dallasclouddev]

I have the same problem on Windows server. The issue is that file paths are limited to 255 characters on Windows. The "aws s3 sync" command produces the following:
16:58:42 warning: Skipping file C:\Program Files (x86)\Jenkins\workspace\Athena_Sonar_S3_upload\platform-services\Services\ResourceAccess\retail-da\sip-retail-persistence-configuration-impl\src\main\java\com\sabre\synxis\sip\retail\persistence\configuration\impl\RetailPersistenceConfiguration.java. File does not exist.

The workaround:
As JDK 11 is installed on the server, I used the jar utility from the SDK to zip up the files.
Then the files can be successfully uploaded to S3 with the aws s3 cp commnad, for example"aws cp myzip.zip s3://bucket/myzip.zip".

[pratapkk63]

I just found the same situation (lots of warnings while using aws s3 sync). Turns out the solution was simply about permissions.

Instead of
"aws s3 sync ..."
use
"sudo aws s3 sync"

Thanks man :)

[jasel-lewis]

I have the same problem on Windows server. The issue is that file paths are limited to 255 characters on Windows. The "aws s3 sync" command produces the following: 16:58:42 warning: Skipping file C:\Program Files (x86)\Jenkins\workspace\Athena_Sonar_S3_upload\platform-services\Services\ResourceAccess\retail-da\sip-retail-persistence-configuration-impl\src\main\java\com\sabre\synxis\sip\retail\persistence\configuration\impl\RetailPersistenceConfiguration.java. File does not exist.

The workaround: As JDK 11 is installed on the server, I used the jar utility from the SDK to zip up the files. Then the files can be successfully uploaded to S3 with the aws s3 cp commnad, for example"aws cp myzip.zip s3://bucket/myzip.zip".

This was my issue... the 255 character limit (running Win10). Thank you for bringing this to light, @dallasclouddev!

My resolution was a little more trivial, I just moved my root folder (the one to copy) up to C:\temp which shaved enough characters from the full path that I could use the aws s3 cp command. I realize this is only a situational fix - the real fix would come from >255 character support from the Windows install of the AWS CLI. :/

[z0mg]

I had the same problem on Windows while in C:\temp\folder1.

When using aws s3 sync s3://bucket/prefix --exclude * --include "2024-07*", aws would say warning: Skipping file C:\folder2\..., which was in no way involved. It would also take forever to start the transfer.

I updated and the issue went away, the transfer started immediately.