Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix some FreeBSD VOPs to synchronize properly with teardown #12704

Closed
wants to merge 2 commits into from

Conversation

markjdb
Copy link
Contributor

@markjdb markjdb commented Oct 28, 2021

Motivation and Context

This change fixes a kernel use-after-free encountered while testing a fix for a deadlock in zfs_getpages(). After fixing the problem in zfs_getpages(), I noticed the same bug in a couple of other VOPs, specifically setattr and rename.

Description

The pointer to the objset for a dataset must be loaded while in a ZFS_ENTER/EXIT block. Otherwise a concurrent rollback or recv can invalidate the pointer, resulting in a use after free when VOPs resume (assuming that they don't fail immediately).

How Has This Been Tested?

I have a stress test for the aforementioned deadlock. It simply consists of running kernel builds in a snapshotted dataset, and rolling back the dataset periodically. It exposed the use-after-free in zfs_getpages() very quickly, but now runs without issues. I've also run the change on a desktop system which uses ZFS-on-root, and on a host running syzkaller (which on FreeBSD uses ZFS to create VM image snapshots and clones, so is a decent regression test).

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Performance enhancement (non-breaking change which improves efficiency)
  • Code cleanup (non-breaking change which makes code smaller or more readable)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
  • Documentation (a change to man pages or other documentation)

Checklist:

@behlendorf behlendorf requested review from amotin and a user October 28, 2021 19:59
Copy link

@ghost ghost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this makes sense.

module/os/freebsd/zfs/zfs_vnops_os.c Outdated Show resolved Hide resolved
module/os/freebsd/zfs/zfs_vnops_os.c Outdated Show resolved Hide resolved
The objset object is reallocated during certain dataset operations, such
as rollbacks, so the objset pointer must be loaded after acquiring the
teardown lock.

Signed-off-by: Mark Johnston <markj@FreeBSD.org>
We have to hold the teardown lock while dereferencing zfsvfs->z_os and,
I believe, when committing to the ZIL.

Note that jumping to the "out" label, "error" is always non-zero.

Signed-off-by: Mark Johnston <markj@FreeBSD.org>
@ghost
Copy link

ghost commented Oct 28, 2021

Thanks. The stable/12 build failure is unrelated and should be fixed by a buildbot config update I have opened a PR for, so don't worry about that :)

@behlendorf behlendorf added the Status: Accepted Ready to integrate (reviewed, tested) label Oct 29, 2021
behlendorf pushed a commit that referenced this pull request Oct 29, 2021
We have to hold the teardown lock while dereferencing zfsvfs->z_os and,
I believe, when committing to the ZIL.

Note that jumping to the "out" label, "error" is always non-zero.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Ryan Moeller <ryan@ixsystems.com>
Signed-off-by: Mark Johnston <markj@FreeBSD.org>
Closes #12704
tonyhutter pushed a commit to tonyhutter/zfs that referenced this pull request Dec 13, 2021
The objset object is reallocated during certain dataset operations, such
as rollbacks, so the objset pointer must be loaded after acquiring the
teardown lock.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Ryan Moeller <ryan@ixsystems.com>
Signed-off-by: Mark Johnston <markj@FreeBSD.org>
Closes openzfs#12704
tonyhutter pushed a commit to tonyhutter/zfs that referenced this pull request Dec 13, 2021
We have to hold the teardown lock while dereferencing zfsvfs->z_os and,
I believe, when committing to the ZIL.

Note that jumping to the "out" label, "error" is always non-zero.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Ryan Moeller <ryan@ixsystems.com>
Signed-off-by: Mark Johnston <markj@FreeBSD.org>
Closes openzfs#12704
tonyhutter pushed a commit to tonyhutter/zfs that referenced this pull request Dec 13, 2021
The objset object is reallocated during certain dataset operations, such
as rollbacks, so the objset pointer must be loaded after acquiring the
teardown lock.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Ryan Moeller <ryan@ixsystems.com>
Signed-off-by: Mark Johnston <markj@FreeBSD.org>
Closes openzfs#12704
tonyhutter pushed a commit to tonyhutter/zfs that referenced this pull request Dec 13, 2021
We have to hold the teardown lock while dereferencing zfsvfs->z_os and,
I believe, when committing to the ZIL.

Note that jumping to the "out" label, "error" is always non-zero.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Ryan Moeller <ryan@ixsystems.com>
Signed-off-by: Mark Johnston <markj@FreeBSD.org>
Closes openzfs#12704
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Status: Accepted Ready to integrate (reviewed, tested)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants