[Feature Request] Remove ZFS ARC from Total Memory Usage

[jerias]

Great project! I was looking for a lightweight replacement for Zabbix for home. I thought I'd settled on Netdata, but I just found this and it may perfect for my needs.

On a system with ZFS, the total memory usage does not accurately reflect actual usage as it includes the ZFS ARC. Unfortuanly, Linux doesn't include this in the cache size. It would be great if ARC could be removed from the Total Memory usage and included in the cache (or as a separate ARC stat).

Thanks!

[henrygd]

What does it report currently and what would you expect it to report?

Can you please tell me what free -m reports? And htop / btop if you have those.

Also please paste the output of cat /proc/meminfo if possible, thanks.

[jerias]

Standard Linux cache doesn't include ARC:

$ free -h
               total        used        free      shared  buff/cache   available
Mem:            31Gi        22Gi       4.7Gi        35Mi       4.3Gi       8.6Gi
Swap:           15Gi          0B        15Gi

~15G of the "used" is actually ARC (This can also be obtained from "/proc/spl/kstat/zfs/arcstats"):

$ arcstat
    time  read  miss  miss%  dmis  dm%  pmis  pm%  mmis  mm%  size     c  avail
15:22:01     0     0      0     0    0     0    0     0    0   14G   15G   6.7G

Htop actually handles ARC as expected (7G used ~ 22G-14.8G)

Mem[|||||||||||||||||||||||||||||||||7.06G/31.3G]

/proc/meminfo doesn't account for ARC either (I assume this is where "free" gets it's stats from).

$ cat /proc/meminfo
MemTotal:       32785244 kB
MemFree:         5399700 kB
MemAvailable:    9430060 kB
Buffers:          284584 kB
Cached:          3714452 kB
SwapCached:            0 kB
Active:          1126512 kB
Inactive:        6627064 kB
Active(anon):      18528 kB
Inactive(anon):  3771600 kB
Active(file):    1107984 kB
Inactive(file):  2855464 kB
Unevictable:       97716 kB
Mlocked:           94644 kB
SwapTotal:      16777212 kB
SwapFree:       16777212 kB
Dirty:               236 kB
Writeback:             0 kB
AnonPages:       3852404 kB
Mapped:          1602996 kB
Shmem:             36940 kB
KReclaimable:     530836 kB
Slab:            5436008 kB
SReclaimable:     530836 kB
SUnreclaim:      4905172 kB
KernelStack:       39904 kB
PageTables:        56440 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    33169832 kB
Committed_AS:   25456008 kB
VmallocTotal:   34359738367 kB
VmallocUsed:     1237280 kB
VmallocChunk:          0 kB
Percpu:            62592 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
FileHugePages:         0 kB
FilePmdMapped:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:               0 kB
DirectMap4k:     6972652 kB
DirectMap2M:    24420352 kB
DirectMap1G:     2097152 kB

Here is the current stats in Beszel showing ~22G used, expected should be ~7G used taking ARC into account:

[henrygd]

Thanks, I'll look further into it but I have some higher priority items to address first.

I also don't have a machine using ZFS, so I may need someone to help with testing at some point.

Probably adding a separate category for it in the chart would be better than moving it into cache / buff.

[jerias]

I would be happy to help test this when available.

[henrygd]

Just added this in 0.5.2. No rush, but please update when you can and let me know if it's coming in properly.

[jerias]

I've checked this with the latest release. For the summary page and 1 hour stats it looks great:

Unfortunately, all the other time ranges seem to be missing the ZFS ARC (12 hour for example):

[henrygd]

Gotcha thanks, must have overlooked that. I'll try to have a patch release out Mon or Tue that fixes it.

[henrygd]

This should be fixed now in 0.5.3. Let me know if that's the case.

[jerias]

Verified on the 0.5.3. I now see the full memory stats being populated on graphs for all time ranges.

Thanks for adding this!

[henrygd]

No worries 👍