Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FAT32 USB flash, can't copy/paste anything #2036

Closed
3 tasks done
skafiend opened this issue Dec 12, 2024 · 6 comments · Fixed by #2040
Closed
3 tasks done

FAT32 USB flash, can't copy/paste anything #2036

skafiend opened this issue Dec 12, 2024 · 6 comments · Fixed by #2040
Labels
bug Something isn't working

Comments

@skafiend
Copy link

skafiend commented Dec 12, 2024

What system are you running Yazi on?

Linux X11

What terminal are you running Yazi in?

kitty 0.37.0

yazi --debug output

Yazi
    Version: 0.4.2 (c78f39b5 2024-12-12)
    Debug  : false
    Triple : x86_64-unknown-linux-gnu (linux-x86_64)
    Rustc  : 1.82.0 (f6e511ee 2024-10-15)

Ya
    Version: 0.4.2 (c78f39b5 2024-12-12)

Emulator
    Brand.from_env      : Some(Kitty)
    Emulator.detect     : Emulator { kind: Left(Kitty), light: false, cell_size: Some((10, 23)) }
    Emulator.detect_full: Ok(Emulator { kind: Left(Kitty), light: false, cell_size: Some((10, 23)) })

Adapter
    Adapter.matches: Kgp

Desktop
    XDG_SESSION_TYPE           : Some("tty")
    WAYLAND_DISPLAY            : None
    DISPLAY                    : Some(":0")
    SWAYSOCK                   : None
    HYPRLAND_INSTANCE_SIGNATURE: None
    WAYFIRE_SOCKET             : None

SSH
    shared.in_ssh_connection: false

WSL
    WSL: false

NVIM
    NVIM          : false
    Neovim version: 0.10.2

Variables
    SHELL           : Some("/bin/bash")
    EDITOR          : Some("nvim")
    VISUAL          : Some("nvim")
    YAZI_FILE_ONE   : None
    YAZI_CONFIG_HOME: None
    YAZI_ZOXIDE_OPTS: None
    FZF_DEFAULT_OPTS: Some("--color=fg:#af00ff,bg:#011627,hl:#ffffff --color=fg+:#011627,bg+:#af00ff,hl+:#ffffff --color=info:#af00ff,prompt:#ffffff,pointer:#011627 --color=marker:#FFFFFF,spinner:#af00ff,header:#0000ff,gutter:#011627,border:#011627")

Text Opener
    default     : Some(Opener { run: "${EDITOR:-vi} \"$@\"", block: true, orphan: false, desc: "$EDITOR", for_: None, spread: true })
    block-create: Some(Opener { run: "${EDITOR:-vi} \"$@\"", block: true, orphan: false, desc: "$EDITOR", for_: None, spread: true })
    block-rename: Some(Opener { run: "${EDITOR:-vi} \"$@\"", block: true, orphan: false, desc: "$EDITOR", for_: None, spread: true })

Multiplexers
    TMUX               : 0
    tmux version       : tmux 3.5a
    tmux build flags   : enable-sixel=Unknown
    ZELLIJ_SESSION_NAME: None
    Zellij version     : No such file or directory (os error 2)

Dependencies
    file          : 5.45
    ueberzugpp    : No such file or directory (os error 2)
    ffmpeg/ffprobe: 6.1.2 / 6.1.2
    pdftoppm      : No such file or directory (os error 2)
    magick        : 7.1.1-41
    fzf           : 0.56.3
    fd/fdfind     : 10.1.0 / No such file or directory (os error 2)
    rg            : 14.1.1
    chafa         : 1.10.3
    zoxide        : 0.9.4
    7z/7zz        : 16.02 / No such file or directory (os error 2)
    jq            : 1.7.1

Clipboard
    wl-copy/paste: No such file or directory (os error 2) / No such file or directory (os error 2)
    xclip        : 0.13
    xsel         : 1.2.1

Describe the bug

yazi seems to be extremely sluggish when I copy files on an external fat 32 drive (I've tried two different ones and had the same issue). it takes about an hour maybe even more to transfer less than 1000 MB. if I do the same using cp or ranger it takes just several minutes

Minimal reproducer

  1. format a usb stick to fat 32
  2. copy a bunch of files (300-500 mb total)
  3. progress goes from 0 to 93% pretty fast
  4. ...
  5. you are stuck and nothing happens

Anything else?

2024-12-12_18-58
2024-12-12_19-10
2024-12-12_19-17
2024-12-12_19-22

Checklist

  • I tried the latest nightly build, and the issue is still reproducible
  • I updated the debug information (yazi --debug) input box to the nightly that I tried
  • I can reproduce it after disabling all custom configs/plugins (mv ~/.config/yazi ~/.config/yazi-backup)
@skafiend skafiend added the bug Something isn't working label Dec 12, 2024
@skafiend skafiend changed the title Performance Issues: Slow File Transfers on FAT32 Drives Slow File Transfers on FAT32 Drives Dec 12, 2024
@skafiend skafiend changed the title Slow File Transfers on FAT32 Drives FAT32 USB flash, can't copy/paste anything Dec 12, 2024
@sxyazi
Copy link
Owner

sxyazi commented Dec 12, 2024

It looks like a duplicate of #1858. Yazi simply calls Rust's std::io::copy() when copying files, which could be a bug in Rust itself. I have a few things to confirm:

  • Does this issue exist in the previous version v0.3.3? Yazi 0.4 changed the file copy API from std::fs::copy() to std::io::copy() because fs::copy() had a bug causing network file permission errors.
  • Is the file you're copying never completing, or does it eventually complete but take a long time? Are there any error logs—can you inspect the task in Task Manager by pressing <enter>?
  • Does this happen occasionally, or is it 100% of the time? Does it happen with a single file or only when copying multiple files?
  • What happens if you manually call std::io::copy() on these files in a Rust app?

@sxyazi sxyazi added the waiting on op Waiting for more information from the original poster label Dec 12, 2024
@skafiend
Copy link
Author

skafiend commented Dec 13, 2024

  • yazi 0.3.3
    start
    middle
    end
  • yazi 0.4.2
    start
    middle
    end

I've done a couple more tests, and the yazi version doesn't affect the behavior, what usb stick I use and how many files are being copied at a time does.

  • Actions Semiconductor Co., Ltd D-Wave 2GB MP4 Player / AK1025 MP3/MP4 Player - I can reproduce the issue every time as long as I transfer more than 20-25 files (each from 5 - 10mb), but when the files are compressed into a zip archive, everything works flawlessly.
  • SanDisk Corp. Cruzer Blade - works fine either way, even tho it has the same FAT32 fs

cp gives consistent results for both drives:
cp_test

  • What happens if you manually call std::io::copy() on these files in a Rust app?

I have no idea. I've never messed with rust before. Is there any way to launch yazi in debug mode, so it would generate a log file with the last file operations?

@github-actions github-actions bot removed the waiting on op Waiting for more information from the original poster label Dec 13, 2024
@skafiend
Copy link
Author

as a temporary solution, I guess I can just hardcode cp command for this particular drive and call it a day.

@sxyazi
Copy link
Owner

sxyazi commented Dec 13, 2024

In this case, if it only happens on certain USB drives, I guess it might be a hardware issue — some hardware might have issues with concurrent read/write operations, although I can't reproduce it on my FAT32 USB.

You could try setting both micro_workers and macro_workers to 1 to revert the cp behavior, meaning force Yazi to only use a single worker for file copying, and see if it works:

[tasks]
micro_workers = 10
macro_workers = 25

@sxyazi sxyazi added the waiting on op Waiting for more information from the original poster label Dec 13, 2024
@skafiend
Copy link
Author

2024-12-14_00-23
I think it did the trick. Thank you

@github-actions github-actions bot removed the waiting on op Waiting for more information from the original poster label Dec 13, 2024
@sxyazi
Copy link
Owner

sxyazi commented Dec 13, 2024

Nice to see it's working!

I'll create a PR later to choose more conservative values for the preset micro_workers and macro_workers, even though it might be a bit slower, but the goal is to ensure it's safe for all users as much as possible.

But of course, users can always adjust them in their own config as needed, I think that's enough.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
2 participants