Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sftp: fails to sync with “Failed to copy: file does not exist” #5077

Closed
ncw opened this issue Feb 28, 2021 · 2 comments
Closed

sftp: fails to sync with “Failed to copy: file does not exist” #5077

ncw opened this issue Feb 28, 2021 · 2 comments
Labels
bug Remote: SFTP waiting for upstream...

Comments

@ncw
Copy link
Member

ncw commented Feb 28, 2021

The symptom of this problem is when trying to do a copy from certain SFTP servers, rclone errors with

ERROR : file.txt: Failed to copy: file does not exist

This is because the server can only read each file once - when the file is read it gets deleted.

There are some upstream issues about this

This issue was originally reported in the forum for more details see there: https://forum.rclone.org/t/sftp-fails-to-sync-to-local-failed-to-copy-file-does-not-exist/21759
@ncw
Copy link
Member Author

ncw commented Feb 28, 2021 •
edited
Loading

I have made a fix with a test version of the sftp library from: pkg/sftp#408

v1.55.0-beta.5240.9b01663c0.fix-5077-sftp-read-once on branch fix-5077-sftp-read-once (uploaded in 15-30 mins)

Please can you test and report back. You'll need to use the --sftp-disable-concurrent-reads flag or set disable_concurrent_reads = true in the config file.

@ncw ncw mentioned this issue Feb 28, 2021
Closed
ncw added a commit that referenced this issue Mar 13, 2021
@ncw
sftp: fix "file not found" errors for read once servers - fixes #5077
45b2b05 
It introduces a new flag --sftp-disable-concurrent-reads to stop the
problematic behaviour in the SFTP library for read-once servers.

This upgrades the sftp library to v1.13.0 which has the fix.
@ncw ncw closed this as completed in 713f8f3 Mar 13, 2021
@ncw
Copy link
Member Author

ncw commented Mar 13, 2021

The upstream version of this has now been merged so I'm going to pull it in to rclone and merge it here

I've merged this to master now which means it will be in the latest beta in 15-30 mins and released in v1.55

negative0 pushed a commit to negative0/rclone that referenced this issue Aug 13, 2021
@ncw @negative0
sftp: fix "file not found" errors for read once servers - fixes rclon…
c580119 
…e#5077

It introduces a new flag --sftp-disable-concurrent-reads to stop the
problematic behaviour in the SFTP library for read-once servers.

This upgrades the sftp library to v1.13.0 which has the fix.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Remote: SFTP waiting for upstream...
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@ncw