Disable Linux readahead on zvols

[ryao]

Linux has a readahead logic designed to accelerate sequential workloads.
ZFS has its own readahead logic called zprefetch that operates on both
zvols and datasets. Having two prefetchers active at the same time
causes overprefetching, which unnecessarily reduces IOPS performance on
CoW filesystems like ZFS.

We currently disable the Linux readahead logic on datasets in the belief
that zprefetch is superior because it is aware of the internal geometry
used in ZFS while the Linux readahead logic is not. In particular,
zprefetch aligns readahead to ZFS' internal geometry and is integrated
with ARC to ensure that prefetches are not considered hits. This makes
zprefetch more desireable than the Linux readahead logic on both
datasets and zvols.

This patch disables Linux's readahead logic on zvols to be consistent
with datasets. This has the side benefit of making the behavior of ZFS
more consistent across platforms.

Signed-off-by: Richard Yao ryao@gentoo.org

[ryao]

This seems to hurt sequential reads on zvols. I had previously been seeing some abaurd numbers with Linux's readahead enabled on zvols that had deduplication enabled and now the numbers are a fraction of that of a regular file. In specific, quick informal benchmarks produced figures of 1.1GBps vs. 140MBps on an Intel 730 SSD.

This should benefit random IO, but it can signficantly hurt sequential IO. There are three possible explanations that come to mind:

1. Linux's readahead implements pipelining via asynchronous mechanisms while zfetch us fully synchronous.
2. My zvols use volblocksize=4KB such that Linux's readahead has no alignment/geometry handicap.
3. The page cache is far closer to userland than ARC because of the bio_overhead and we need to implement file and inode operations structs to alleviate that.

This started as a curiosity when I noticed Linux's read ahead mechanisms in use when I was debugging a regression that prevented me from publishing #2484 in usable condition, but the potential for improved random IO and merits more investigation. I will leave this open until it has been more thoroughly examined.

The following paper describes the Linux readahead code in depth:

http://www.ece.eng.wayne.edu/~sjiang/Tsinghua-2010/linux-readahead.pdf

[behlendorf]

@ryao Thanks for following up on this. We'll of course need to get some performance testing done for these kind of proposed changes.

[behlendorf]

Closing until we have some actual data details if this is a good idea or not.