Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cp command downloads file from S3 even though the file does not exist on the S3 bucket #348

Closed
harisbr opened this issue Sep 1, 2021 · 3 comments · Fixed by #477
Closed

Comments

@harisbr
Copy link

harisbr commented Sep 1, 2021

When cp command is executed to download a file from the S3 bucket I get the AWS error in console 'NoSuchKey: The specified key does not exist.' but the file with the same name is created on my local storage.
Is there any way to prevent this behaviour?

@harisbr harisbr changed the title cp command downloads file from S3 event it does not exist cp command downloads file from S3 even though the file does not exist on the S3 bucket Sep 1, 2021
@ocakhasan
Copy link
Contributor

Hi @harisbr, this should not happen. Can you please provide the s5cmd version and steps to reproduce the problem. We need a bit more detail to handle the issue.

@harisbr
Copy link
Author

harisbr commented Sep 2, 2021

Hi @ocakhasan, thank you for the response.
Here are all details:

  • version is 0.0.0-dev, I've installed it using Go, not sure if I did something wrong, I suppose it installed the latest version from the master branch
  • executed command:
    s5cmd cp s3://harisbr-test/filename.txt .downloads/

This S3 bucket is empty, no files there.
I am sending you the screenshot of the error as well as the created file in the downloads folder.

error
file

@ocakhasan
Copy link
Contributor

ocakhasan commented Sep 3, 2021

Hi again @harisbr,
I tried to debug the code and the steps below happens in order.

  1. Since this is a download task, doDownload is called.
  2. Then it tries to download the file, but it gives an error. You can check it from here.
  3. Since there is an error, we need to delete the generated file (filename.txt in your case), but when I tried to delete the file, code part, windows give me an error saying
remove deneme/filename.txt The process cannot access the file because it is being used by another process.

When I tried the same command in ubuntu, it works just fine, currently this seems to be a windows problem and there should be a fix in the new release, so stay tuned.

igungor pushed a commit that referenced this issue Aug 22, 2022
Before downloading a file from s3 a local target file is created. If the download fails the created file should've been deleted. But since the file was not closed, the delete operation fails in Windows.

Fixes #348
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants