Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't run compsize on / #14

Open
ghost opened this issue Mar 4, 2018 · 6 comments
Open

Can't run compsize on / #14

ghost opened this issue Mar 4, 2018 · 6 comments

Comments

@ghost
Copy link

ghost commented Mar 4, 2018

Hello, thanks for your great program!
I have a little issue - I can't run compsize on / directory:

 yard@arch  ~  sudo compsize /
Not btrfs (or SEARCH_V2 unsupported).

But btrfs command can detect that it's btrfs:

 yard@arch  ~  sudo btrfs fi us /
Overall:
    Device size:                 340.00GiB
    Device allocated:             28.06GiB
    Device unallocated:          311.94GiB
    Device missing:                  0.00B
    Used:                         18.18GiB
    Free (estimated):            320.55GiB      (min: 164.58GiB)
    Data ratio:                       1.00
    Metadata ratio:                   2.00
    Global reserve:               43.91MiB      (used: 0.00B)

Data,single: Size:26.00GiB, Used:17.38GiB
   /dev/sda2      26.00GiB

Metadata,DUP: Size:1.00GiB, Used:406.47MiB
   /dev/sda2       2.00GiB

System,DUP: Size:32.00MiB, Used:16.00KiB
   /dev/sda2      64.00MiB

Unallocated:
   /dev/sda2     311.94GiB

Is it a limitation?

@kilobyte
Copy link
Owner

kilobyte commented Mar 4, 2018

I can't reproduce: compsize / starts fine for me; only some time later failing once recursing to a filesystem like /proc or /dev (unless given -x).

I've messed recently with this code, but changed only the error message.

@kilobyte
Copy link
Owner

kilobyte commented Mar 4, 2018

... I'm an idiot. Obviously, I do get the error message you say, it just takes a while — unless the cache is hot or one of special filesystems gets enumerated by readdir before something that takes a substantial time to process; in both of these cases the error is returned quickly.

And, the error message doesn't say what file the ioctl failed on, making it hard for the user to see what's wrong.

In the short term, you can run compsize -x; I wonder what would be a proper fix. Defaulting to a sort of -x whenever recursing to non-btrfs? Making the error fatal only on the first level but recursing silently into non-btrfs so any child filesystems (usually subvolumes) get handled?

@ghost
Copy link
Author

ghost commented Mar 4, 2018

@kilobyte yeah, compsize -x works:

 ✘ yard@arch  ~  sudo compsize -x /
Processed 308087 files, 228092 regular extents (234987 refs), 171354 inline.
Type       Perc     Disk Usage   Uncompressed Referenced  
TOTAL       63%       17G          27G          27G       
none       100%       12G          12G          12G       
zstd        32%      4.7G          14G          15G

I don't really know which method is better, but we can add blacklist for folders like /proc and /dev/ and use one of the methods you've described

@ckujau
Copy link

ckujau commented Aug 9, 2019

I get a slightly different message with the latest checkout:

opensuse0# ./compsize /
/dev/.blkid.tab: Not btrfs (or SEARCH_V2 unsupported).

opensuse0# cat /dev/.blkid.tab
<device DEVNO="0xfe02" TIME="1565340143.900144" UUID="e85f7bc4-460a-4341-bbe2-4f5721a6c7ae" UUID_SUB="3ee258c4-05cb-4d5c-9617-fe2226d07823" TYPE="btrfs" PTTYPE="dos" PARTUUID="000dbe6c-02">/dev/vda2</device>
<device DEVNO="0xfe01" TIME="1565340143.897162" UUID="11d15fa0-0321-4343-b30f-52712bd5d571" TYPE="swap" PARTUUID="000dbe6c-01">/dev/vda1</device>

But -x works here too - thanks for the quick fix!

@cmurf
Copy link

cmurf commented Mar 25, 2021

I'm running into this problem as well, and -x doesn't really help because it ends up skipping nested btrfs subvolumes on the same file system. So what's really needed is a way to skip non-btrfs file systems.

Example:

$ sudo compsize /var/
/var/lib/nfs/rpc_pipefs/gssd/clntXX/info: Not btrfs (or SEARCH_V2 unsupported).
$ grep /var/lib/nfs/ /proc/self/mounts
sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0
$ sudo compsize -x /var/
Processed 12358 files, 57051 regular extents (69255 refs), 7707 inline.
Type       Perc     Disk Usage   Uncompressed Referenced  
TOTAL       62%      4.0G         6.4G         7.4G       
none       100%      2.3G         2.3G         2.4G       
zstd        40%      1.6G         4.0G         5.0G       
prealloc   100%       12K          12K         8.0K       
$ sudo du -sh /var
16G	/var
$ 

In the case of compsize / it needs to skip /boot/efi which is FAT, but include /boot/ /home/ /var/log/ all of which are subvolumes on the same Btrfs. I think there's a bit of a challenge distinguishing the case where a mountpoint in the compsize search path is actually a different btrfs file system, whether to include that or not.

@kilobyte
Copy link
Owner

Actually, there are multiple schemes people can legitimately want:

  • recursing all in-depth, counting only btrfs
  • recursing but pruning immediately
    And:
  • same sub-volume (ie, same "filesystem", ie st.st_dev)
  • same real filesystem
  • any btrfs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants