arcstat l2_misses can be very incorrect

[adamdmoss]

System information

Type	Version/Name
Distribution Name	Ubuntu
Distribution Version	18.04.05
Linux Kernel	Linux version 5.4.0-42-generic (buildd@lgw01-amd64-023) (gcc version 7.5.0 (Ubuntu 7.5.0-3ubuntu118.04)) #4618.04.1-Ubuntu SMP Fri Jul 10 07:21:24 UTC 2020
Architecture	x86_64
ZFS Version	git zfs-2.0-release @ 29bc31f
SPL Version	git zfs-2.0-release @ 29bc31f

Describe the problem you're observing

I was wondering why my L2 hit ratio always seemed so low, so finally decided to look into it...

The way that arc_read() currently accounts L2 hits/misses, if ANY pool has an l2arc then ALL reads that don't hit an l2arc are considered to be a miss, even if the read is from a pool that doesn't have an l2arc configured.

edit: The same over-eager miss accounting appears to also happen for reads from datasets which have secondarycache=none but whose pools have l2arc; this shouldn't count as a miss.

Describe how to reproduce the problem

Set up two pools on the same system, one backed by an l2arc and one not. Read data from the pool which doesn't have an l2arc; these reads cause l2_misses to rise (i.e. in /proc/spl/kstat/zfs/arcstats)

Include any warning/errors/backtraces from the system logs

n/a

[adamdmoss]

Pinging @gamanakis just 'cos they're an L2ARC expert at this point. :)

[adamdmoss]

ZFS version should be:
2.0.0 RC1 (just to be clear this isn't release, in the future)

It's the name of the upstream branch I was using - I'm not making any claims about its naming accuracy. :)

[adamdmoss]

I gave the commit id - 29bc31f