-
Notifications
You must be signed in to change notification settings - Fork 423
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Copy tasks between some different file-systems (remote) hang with (os error 95)
#1775
Comments
Seems a dup of #1703 Could you try this comment #1703 (comment)? |
Thanks a lot for the quick reply.
|
You can try Joshuto which it's also a file manager built with Rust |
I've run into (what I think is) the same issue. Interestingly, running a shell command within yazi using the ":" key binding to create a file (e.g. It doesn't look like you have to be operating between different file systems either, as I was able to reproduce this yank-pasting a file from a samba share into the same directory. Also, it looks like the operation succeeds in creating the new file, but hangs at some point after that. I've seen other discussions elsewhere with people having similar problems with the std::fs::copy function, and the hypothesis that "NTFS does not have chmod permissions, therefore attempting to set chmod permissions on a samba share will fail." If I find the time, I'll try to play around with different versions of copy functions in Rust using my mounted file system, and see if I can isolate the issue and possibly suggest a fix. |
Thanks @AidanJHMurphy, this info is really helpful! It seems like Rust's I tried // fs::copy()
statx(3, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_ALL, {stx_mask=STATX_ALL|STATX_MNT_ID, stx_attributes=0, stx_mode=S_IFREG|0644, stx_size=13, ...}) = 0
openat(AT_FDCWD, "b", O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0100644) = 4
statx(4, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_ALL, {stx_mask=STATX_ALL|STATX_MNT_ID, stx_attributes=0, stx_mode=S_IFREG|0644, stx_size=0, ...}) = 0
fchmod(4, 0100644) = 0
copy_file_range(3, NULL, 4, NULL, 1073741824, 0) = 13
copy_file_range(3, NULL, 4, NULL, 1073741824, 0) = 0
close(4) = 0
close(3) = 0 // io::copy()
statx(3, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_ALL, {stx_mask=STATX_ALL|STATX_MNT_ID, stx_attributes=0, stx_mode=S_IFREG|0644, stx_size=13, ...}) = 0
openat(AT_FDCWD, "b", O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0100644) = 4
fchmod(4, 0100644) = 0
statx(3, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_ALL, {stx_mask=STATX_ALL|STATX_MNT_ID, stx_attributes=0, stx_mode=S_IFREG|0644, stx_size=13, ...}) = 0
statx(4, "", AT_STATX_SYNC_AS_STAT|AT_EMPTY_PATH, STATX_ALL, {stx_mask=STATX_ALL|STATX_MNT_ID, stx_attributes=0, stx_mode=S_IFREG|0644, stx_size=0, ...}) = 0
copy_file_range(3, NULL, 4, NULL, 1073741824, 0) = 13
copy_file_range(3, NULL, 4, NULL, 1073741824, 0) = 0
close(4) = 0
close(3) = 0 But there are also some issues with this. It's currently unclear how macOS should implement it, since macOS's That said, I'm not even sure if this problem exists on macOS, as no one has reported it yet. Although Rust's Maybe we can try fixing it for Linux first. I'll create a PR for it later. |
Please try #1805 |
Thanks for looking into this so quickly! BUT: in the progressbar an error is still reported (red) and the task never finishes (even after the files have actually been copied). The errors are still reported (per file?), which is no problem by itself, but needs an error/warning distinction (see below). So the error handling is not yet fixed. Also, it would probably be helpful in general if in the detailed report of a selected task (i.e. It is probably necessary to distinguish between "errors" and "warnings", so that we could arrive at a task state where a task could be "finished with errors" or "finished with warnings". /reopen |
Thanks for the test @ppenguin!
Can you elaborate a bit more? What error is it? Is it still the same Are you using the latest commit df78827 (what does your |
Yes, it's the same list of same errors
(last commit on master as of yesterday, two newer than the fix). A side by side test "before/after" the fix: Before: Now it gets interesting, with the same (limited) file set, everything is copied, even without errors, but some of the copied files are shown as With another file set (nested dirs and more/larger files), I get this: So in other words: the number of files for this case that is successfully copied went from "zero" to "more", but doesn't reliably work. You might be able to reproduce this behaviour yourself by mounting an |
This is really strange, I've checked the code several times and can't find any errors, basically, Yazi is just using yazi/yazi-shared/src/fs/fns.rs Lines 255 to 276 in 24a77d8
My best guess is that It could also be related to your environment. If possible, I suggest trying to call I'll mark this issue as "help wanted" and reopen it for a while to see if anyone can reproduce it on their system and debug it at the code level. Any help would be greatly appreciated! 🙏🏻 |
Just as some extra info: if I just do
But all files are copied completely. This probably means that in the above code (I don't know any Indeed, since both operations are async (copying content is decoupled from metadata?), this may also explain how we can end up with incomplete files? To make sure that the contents (everything else than the metadata) has been indeed copied without difference, I'm not sure whether the part on Further, inspired by this thread, one might consider to make the metadata part a bit more elaborate, basically trying to copy different attributes from the metadata separately and allow failure for them (handle as warning). You could maybe even handle this in "fall-through" fashion, first try everything and then less, based on what errors come back based on the target file system. |
Yes, the code above runs exactly as you described, with only the The subsequent So, I really can’t imagine what could be causing the issue besides a problem with |
I just took a quick look for fun and education at the source of the "original" (I'd say this is pretty much mandatory, a copy program that can't handle the same as the |
Yeah, I would love to switch to such an API if it exists and would accept a PR as long as it's reliable and implemented in Rust — I don't want to introduce a C library because that would complicate the build. I noticed that another issue you raised with In this case, I'm going to close this issue since it's beyond what Yazi can handle and also not just specific to Yazi. I really suggest testing |
I actually raised that to make the point that probably many/most rust based programs suffer from this
I understand, but this is a huge thing IMO, which would warrant a big red sticker on the product saying it's severly limited and only works on a limited set of file systems.
Basically I think we have already proven that both |
I just read the tokio source and noticed that its I'm not sure if this is causing the issue, but I replaced it with |
Yes, I can confirm that #1817 works for the above test case! |
I'm going to lock this issue because it has been closed for 30 days. ⏳ |
What system are you running Yazi on?
Linux Wayland
What terminal are you running Yazi in?
kitty
yazi --debug
outputDid you try the latest nightly build to see if the problem got fixed?
No, and I'll explain why below
Describe the bug
I'm copying files from a mounted
ntfs3
volume (external USB HD) to agvfs
mounted samba share (i.e. accessed through/run/user/1000/gvfs/...
).I get many
Failed to work on this task: Operation not supported (os error 95)
errors, and all tasks with the same source/destination hang. No further information is given.
It is not unusual to get some errors messages when copying between file-systems, these almost always refer to the inability to copy some kind of extended attribute, however, this type of error is/should be always treated non-fatally (more like a warning), this is the behaviour of all other known software (at least on linux).
Therefore, "not trying harder" to just copy the files and hanging instead should be considered a bug.
Expectation: issues regarding copying of extended attributes are reported as warnings, but the copying tasks are completed if no "real" errors (i.e. no write premission, disk full, ...) occur.
Minimal reproducer
Not sure, probably copying files with some kind of extended attributes to a
gvfs
mounted remote samba share?Anything else?
No response
The text was updated successfully, but these errors were encountered: