Add Linux kmemleak support to ZTS

[szubersk]

Motivation and Context

Add code quality checks.

Description

• Kmemleak clear is invoked right before every test case run.
• Kmemleak scan is requested right after each test case is finished.
• Kmemleak instrumentation is not used for
  setup/cleanup/pretest/posttest/failsafe stages to shorten the test
  case execution time.
• Kmemleak periodic scan is disabled (scan=0) before the test suite
  run to avoid interfering with the on-demand scan results.
• There are unavoidable potential false positives coming from kernel
  areas other than OpenZFS module.
• The ZTS with kmemleak enabled duration is increased by ~50%.

Example run

Running Time:   07:12:13
Percent passed: 98.3%

unreferenced object 0xffff9da82aea5410 (size 80):
  comm "kworker/u32:10", pid 942206, jiffies 4296749716 (age 2615.516s)
  hex dump (first 32 bytes):
    00 30 30 00 00 00 00 00 ff 8f 30 00 00 00 00 00  .00.......0.....
    51 e6 77 05 a8 9d ff ff 00 00 00 00 00 00 00 00  Q.w.............
  backtrace:
    [<000000005cf1fea2>] alloc_extent_state+0x1d/0xb0 [btrfs]
    [<0000000083f78ae5>] set_extent_bit+0x2ff/0x670 [btrfs]
    [<00000000de29249e>] lock_extent_bits+0x6b/0xa0 [btrfs]
    [<00000000b241f424>] lock_and_cleanup_extent_if_need+0xaf/0x1c0 [btrfs]
    [<0000000093ca72b5>] btrfs_buffered_write+0x297/0x7d0 [btrfs]
    [<000000002c2938c8>] btrfs_file_write_iter+0x127/0x390 [btrfs]
    [<00000000b888f720>] do_iter_readv_writev+0x152/0x1b0
    [<00000000320f0bcc>] do_iter_write+0x7c/0x1c0
    [<000000000b5a8fe0>] lo_write_bvec+0x62/0x150 [loop]
    [<000000009aa03c73>] loop_process_work+0x250/0xbd0 [loop]
    [<00000000c7487d8a>] process_one_work+0x1f1/0x390
    [<000000000b236831>] worker_thread+0x53/0x3e0
    [<0000000023cb3e57>] kthread+0x127/0x150
    [<000000002d48676a>] ret_from_fork+0x22/0x30

How Has This Been Tested?

ZTS passes

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Performance enhancement (non-breaking change which improves efficiency)
• Code cleanup (non-breaking change which makes code smaller or more readable)
• Breaking change (fix or feature that would cause existing functionality to change)
• Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
• Documentation (a change to man pages or other documentation)

Checklist:

• My code follows the OpenZFS code style requirements.
• I have updated the documentation accordingly.
• I have read the contributing document.
• I have added tests to cover my changes.
• I have run the ZFS Test Suite with this change applied.
• All commit messages are properly formatted and contain Signed-off-by.

[gmelikov]

Maybe enable it in zfs-tests-sanity worker? it's ~20 minutes per tests themselves now, looks ok for me if it will be 30 minutes but with kmemleak enabled (if it passes).

[szubersk]

Maybe enable it in zfs-tests-sanity worker?

I'd be all for it if not

/boot/config-5.11.0-1028-azure:CONFIG_HAVE_DEBUG_KMEMLEAK=y
/boot/config-5.11.0-1028-azure:# CONFIG_DEBUG_KMEMLEAK is not set

Looks like a dedicated buildbot slave would be needed with custom Linux kernel configuration. @behlendorf had one, if I recall correctly.

[behlendorf]

In fact we do. The coverage builder uses an older kmemleak and gcov enabled kernel. You can still request it from the CI by adding Requires-builders: coverage to your top commit comment. We could enable it again by default, but I think we'd need to better document or automate the process for building that image so it could be kept more current and properly maintained.

[szubersk]

Rebased.

[szubersk]

@behlendorf, thanks for spotting documentation and usability deficiencies. On a side note: man page != test-runner.py -h output.

[samwyc]

This check fails on fedora-based systems for non-root users, as permission to /sys/kernel/debug/kmemleak is denied.
Hence we're not able to enable the feature, even if CONFIG_DEBUG_KMEMLEAK is set.

[rocky@rocky-kernel ~]$ cat /sys/kernel/debug/kmemleak
cat: /sys/kernel/debug/kmemleak: Permission denied
[rocky@rocky-kernel ~]$ grep "CONFIG_DEBUG_KMEMLEAK" /boot/config-4.18.0+
CONFIG_DEBUG_KMEMLEAK=y
[rocky@rocky-kernel ~]$ python3 -c "import os;print(os.path.exists('/sys/kernel/debug/kmemleak'))"
False