Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Newer Illumos code #1775

Closed
wants to merge 50 commits into from
Closed

Newer Illumos code #1775

wants to merge 50 commits into from

Conversation

ryao
Copy link
Contributor

@ryao ryao commented Oct 9, 2013

This depends on openzfs/spl#300. It supersedes #1696, #1760 and #1772.

It includes some fixes that I wrote when porting the changes. I split them out and placed them before the Illumos commits to make backporting them easier. This is not quite ready to merge yet, but it is ready for comments.

I ran git log usr/src/uts/common/fs/zfs and reviewed all commits made since the start of 2012 to obtain a list of commits that we were missing. In addition, I used git blame to identify the causes of merge conflicts, which identified some even older commits that we were missing. The following commits have been omitted for any of the following reasons:

  • They add new feature flags and I would rather not include them in this pull request.
  • They don't make sense on Linux.
  • They require additional improvements to the Illumos code before they can be merged (e.g. nop write)
  • They have already been fixed in ZFSOnLinux (e.g. LZ4 stack overflow, txg commit callbacks don't work)
commit 0713e232b7712cd27d99e1e935ebb8d5de61c57d
Author: George Wilson <george.wilson@delphix.com>
Date:   Tue Oct 1 13:25:53 2013 -0800

    4101 metaslab_debug should allow for fine-grained control
    4102 space_maps should store more information about themselves
    4103 space map object blocksize should be increased
    4104 ::spa_space no longer works
    4105 removing a mirrored log device results in a leaked object
    4106 asynchronously load metaslab
    Reviewed by: Matthew Ahrens <mahrens@delphix.com>
    Reviewed by: Adam Leventhal <ahl@delphix.com>
    Reviewed by: Sebastien Roy <seb@delphix.com>
    Approved by: Garrett D'Amore <garrett@damore.org>

commit cf746768a898264fa491791a8252865e5338dc67
Author: Bryan Cantrill <bryan@joyent.com>
Date:   Sat Aug 3 16:27:57 2013 -0700

    1974 pages_pp_maximum/lotsfree discrepancy leads to mlock() failure
    Reviewed by: Keith Wesolowski <keith.wesolowski@joyent.com>
    Reviewed by: Brendan Gregg <brendan.gregg@joyent.com>
    Approved by: Garrett D'Amore <garrett@damore.org>

commit 810e43b2eb0e320833671a403fdda51917e8b036
Author: Bill Pijewski <wdp@joyent.com>
Date:   Thu Aug 22 19:03:54 2013 +0000

    2932 support crash dumps to raidz, etc. pools
    Reviewed by: Jerry Jelinek <jerry.jelinek@joyent.com>
    Reviewed by: Matthew Ahrens <mahrens@delphix.com>
    Approved by: Dan McDonald <danmcd@nexenta.com>

commit 18e649782fcf9da52d755b2074b4b9382426c818
Author: Marcel Telka <marcel.telka@nexenta.com>
Date:   Tue Aug 13 07:32:14 2013 +0200

    4039 zfs_rename()/zfs_link() needs stronger test for XDEV
    Reviewed by: Gordon Ross <gordon.ross@nexenta.com>
    Reviewed by: Kevin Crowe <kevin.crowe@nexenta.com>
    Reviewed by: Saso Kiselkov <skiselkov.ml@gmail.com>
    Reviewed by: Matthew Ahrens <mahrens@delphix.com>
    Approved by: Dan McDonald <danmcd@nexenta.com>

commit a5b577712a34346841d970e0827b4920ace408af
Author: Dan McDonald <danmcd@nexenta.com>
Date:   Wed Jul 10 10:58:11 2013 -0400

    348 ZFS should handle DKIOCGMEDIAINFOEXT failure
    Reviewed by: Garrett D'Amore <garrett@damore.org>
    Reviewed by: George Wilson <george.wilson@delphix.com>
    Approved by: Garrett D'Amore <garrett@damore.org>

commit 3cb69f734bc60bbb4d56a28c83706db862bec082
Author: Will Andrews <will@firepipe.net>
Date:   Tue Jun 11 09:13:55 2013 -0800

    3749 zfs event processing should work on R/O root filesystems
    Reviewed by: Matthew Ahrens <mahrens@delphix.com>
    Reviewed by: Eric Schrock <eric.schrock@delphix.com>
    Approved by: Christopher Siden <christopher.siden@delphix.com>

commit b3d9f2e26021d3f55a281af30720589d303b9806
Author: Will Andrews <willa@spectralogic.com>
Date:   Tue Jun 11 09:13:51 2013 -0800

    3747 txg commit callbacks don't work
    Reviewed by: Matthew Ahrens <mahrens@delphix.com>
    Reviewed by: Eric Schrock <eric.schrock@delphix.com>
    Approved by: Christopher Siden <christopher.siden@delphix.com>

commit d8fa96c3021236b21f47b84036cbe4f5d4854a06
Author: Sa?o Kiselkov <skiselkov.ml@gmail.com>
Date:   Wed Apr 24 08:45:43 2013 -0800

    3705 stack overflow due to zfs lz4 compression
    Reviewed by: Matthew Ahrens <mahrens@delphix.com>
    Approved by: Christopher Siden <christopher.siden@delphix.com>

commit d5ee8a1311accef11ec2057f70da38d1dd687088
Author: Matthew Ahrens <mahrens@delphix.com>
Date:   Sun Apr 7 18:40:39 2013 -0800

    3654 zdb should print number of ganged blocks
    3655 ::rrwlock and ::refcount print errors on non-debug kernels
    3656 remove unused function zap_cursor_move_to_key()
    3657 mdb_ctf_vread() should allow conversion of pointer to uintptr_t
    Reviewed by: George Wilson <george.wilson@delphix.com>
    Reviewed by: Christopher Siden <christopher.siden@delphix.com>
    Reviewed by: Dan McDonald <danmcd@nexenta.com>
    Approved by: Garrett D'Amore <garrett@damore.org>

commit c61ea5668634a215fd1c9000beb7f6d2853b3468
Author: George Wilson <george.wilson@delphix.com>
Date:   Wed Feb 20 22:50:16 2013 -0800

    3557 dumpvp_size is not updated correctly when a dump zvol's size is changed
    3558 setting the volsize on a dump device does not return back ENOSPC
    3559 setting a volsize larger than the space available sometimes succeeds
    3560 dumpadm should be able to remove a dump device
    Reviewed by: Adam Leventhal <ahl@delphix.com>
    Reviewed by: Matthew Ahrens <mahrens@delphix.com>
    Reviewed by: Christopher Siden <christopher.siden@delphix.com>
    Approved by: Albert Lee <trisk@nexenta.com>

commit 80901aea8e78a2c20751f61f01bebd1d5b5c2ba5
Author: George Wilson <george.wilson@delphix.com>
Date:   Tue Nov 13 14:55:48 2012 -0800

    3236 zio nop-write
    Reviewed by: Matt Ahrens <matthew.ahrens@delphix.com>
    Reviewed by: Adam Leventhal <ahl@delphix.com>
    Reviewed by: Christopher Siden <chris.siden@delphix.com>
    Approved by: Garrett D'Amore <garrett@damore.org>

commit cd1c8b85eb30b568e9816221430c479ace7a559d
Author: Matthew Ahrens <mahrens@delphix.com>
Date:   Thu Aug 30 05:13:49 2012 -0700

    3112 ztest does not honor ZFS_DEBUG
    3113 ztest should use watchpoints to protect frozen arc bufs
    3114 some leaked nvlists in zfsdev_ioctl
    3115 poll(2) returns prematurely in presence of spurious wakeups
    Reviewed by: Adam Leventhal <ahl@delphix.com>
    Reviewed by: Matt Amdur <Matt.Amdur@delphix.com>
    Reviewed by: George Wilson <george.wilson@delphix.com>
    Reviewed by: Christopher Siden <chris.siden@delphix.com>
    Approved by: Eric Schrock <eric.schrock@delphix.com>

commit b420f3adeb349714478d1a7813d2c0e069d41555
Author: Richard Lowe <richlowe@richlowe.net>
Date:   Wed Aug 1 16:40:39 2012 -0400

    backout 3006: causes 3046 (panics after mounting root)

commit 94dd93aee32d1616436eb51fb7b58094b9a8d3e8
Author: George Wilson <gwilson@delphix.com>
Date:   Sun May 6 18:49:02 2012 -0700

    1618 zfs causing system to hang in vmem_xalloc()
    Reviewed by: Eric Schrock <eric.schrock@delphix.com>
    Reviewed by: Adam Leventhal <ahl@delphix.com>
    Reviewed by: Matt Ahrens <matt@delphix.com>
    Reviewed by: Richard Elling <richard.elling@richardelling.com>
    Reviewed by: Garrett D'Amore <garrett@damore.org>
    Reviewed by: Steve Gonczi <gonczi@comcast.net>
    Approved by: Gordon Ross <gwr@nexenta.com>

commit c08b16378bf1d808956fc23b8c8ac30f3172a034
Author: Josef 'Jeff' Sipek <jeffpc@josefsipek.net>
Date:   Sat Feb 18 18:33:56 2012 -0500

    2130 zvol DKIOCFREE uses nested DMU transactions
    Reviewed by: Matthew Ahrens <mahrens@delphix.com>
    Reviewed by: Dan McDonald <danmcd@nexenta.com>
    Reviewed by: Garrett D'Amore <garrett@damore.org>
    Reviewed by: Richard Elling <richard.elling@richardelling.com>
    Reviewed by: George Wilson <george.wilson@delphix.com>
    Approved by: Richard Lowe <richlowe@richlowe.net>    

commit 654b400c387942fc00d4f6869539adbd7b25fbce
Author: Joshua M. Clulow <josh@sysmgr.org>
Date:   Sat Feb 4 16:38:08 2012 +1100

    763 FMD msg URLs should refer to something visible
    Reviewed by: Richard Lowe <richlowe@richlowe.net>
    Reviewed by: Yuri Pankov <yuri.pankov@gmail.com>
    Reviewed by: Garrett D'Amore <garrett@damore.org>
    Approved by: Richard Lowe <richlowe@richlowe.net>

@ryao
Copy link
Contributor Author

ryao commented Oct 9, 2013

As an additional note, I seem to have gotten the userland tools building with C99, but as illumos 4047 demonstrated, the kernel modules are still being built as C89.

@ryao
Copy link
Contributor Author

ryao commented Oct 9, 2013

I did a rebase to fix a mistake where I gave myself authorship on one of the illumos commits... now the history being shown by github is incorrect (although my own local history seems right). All of my commits are supposed to precede the Illumos commits.

@behlendorf
Copy link
Contributor

@ryao This is great! I'll make the time to get you feedback on this, I'd love it if the 0.6.3 caught us largely up with Illumos.

@behlendorf
Copy link
Contributor

@dweeezil Since you're familiar with this issues involved in bring in changes from Illumos it would be great if you could also review these patches.

@ryao
Copy link
Contributor Author

ryao commented Oct 12, 2013

I have refreshed the pull request with the following changes:

  1. It has been rebased on today's ZFSOnLinux HEAD. No significant changes were needed to do this, but people who pull this branch for local testing will not be missing recent commits.
  2. The commit messages have been reformatted to be consistent with existing commits taken from Illumos. Specifically, a reference section has been added and a short subject line has been prepended.
  3. Illumos 4061 has been added to the list of commits. I had missed this because it was a change specific to userland.
  4. An off-by-one error in my commit to change zfs_unmount_snap() that @prakashsurya caught has been fixed.
  5. A commit to define MUTEX_SPIN in libzpool has been dropped. It crept into this from something else that I have in development.
  6. The two commits to fix missing copyright notices have been merged into one with additional details added.
  7. I fixed some typos in the commit messages that I introduced by mistake.

With that said, someone still needs to go through and revert modifications that occurred because ZFSOnLinux only called the constructors/destructors once per SLAB. @nedbass' work to port 4045 does this and a few other places seem to do it too. I have not touched this yet, but the corresponding SPL pull request includes the modifications required to support this.

@ryao
Copy link
Contributor Author

ryao commented Oct 12, 2013

A minor refresh. I just did a review of the output of git log usr/src/lib/libzfs, git log usr/src/common/zfs and git log $(ls ./usr/src/man/man*/z*) on the illumos-gate repository. I found some commits in userland that we were also missing. Specifically, I have added:

  • Illumos 3745, 3811
  • Illumos 3818
  • Illumos 3909

Also, I found that my above list is missing the following two commits:

commit 71dbfc287ce623fa58f67b024d14ef6924fa1b2a
Author: Paul B. Henson <henson@acm.org>
Date:   Fri Oct 26 13:33:52 2012 -0700

    3254 add support in zfs for aclmode=restricted
    Reviewed by: Albert Lee <trisk@nexenta.com>
    Reviewed by: Gordon Ross <gwr@nexenta.com>
    Approved by: Richard Lowe <richlowe@richlowe.net>

commit a3c49ce110f325a563c245bedc4d533adddb7211
Author: Albert Lee <trisk@nexenta.com>
Date:   Sat May 14 00:29:13 2011 -0400

    742 Resurrect the ZFS "aclmode" property
    664 Umask masking "deny" ACL entries.
    279 Bug in the new ACL (post-PSARC/2010/029) semantics
    Reviewed by: Aram Hăvărneanu <aram@nexenta.com>
    Reviewed by: Gordon Ross <gwr@nexenta.com>
    Reviewed by: Robert Gordon <rbg@openrbg.com>
    Reviewed by: Mark.Maybee@oracle.com
    Approved by: Garrett D'Amore <garrett@nexenta.com>

These might fall under the "do not make sense on Linux" category given that we don't have ACL support working yet.

ryao referenced this pull request in ryao/zfs Oct 12, 2013
This lock unnecessarily serializes an important path. The fast write
variables are updated using atomic operations, which need no protection.
The code snippet being protected is already protected by the spa_lock.
We remove mc_fastwrite_lock and add an assertion to verify that we have
a spa_lock. This permits multiple threads to run through the
metaslab_alloc_dva() function in parallel as they do on other
implementations.

Signed-off-by: Richard Yao <ryao@gentoo.org>
@dweeezil
Copy link
Contributor

@behlendorf I took a cursory glance at this wad as soon as it was posted and was planning on giving it a more thorough look this weekend.

@dweeezil
Copy link
Contributor

I've still not had much time to look at this yet (it's huge!) but after fiddling with it a bit, I came up with a couple of patches in dweeezil/zfs@1a91683; one to deal with a nvlist allocation and the other to deal with a duplicate preprocessor definition. I've been simultaneously looking the code and trying to run it. Another observation is that I can't unload the zfs.ko module (if anything nontrivial has been done subsequent to its loading) without reverting the recent constructor/destructor patch to spl. @ryao, is this to be expected at this point? The rmmod process blocks in D state but I've not had a chance to figure out why or where it's blocking.

FInally, I'll note that trying to do a zfs destroy <whatever>@snapshot always fails saying that the snapshot (or filesystem, I forgot which) can't be found.

Hopefully I'll have more time to look at this throughout the coming week.

@ryao
Copy link
Contributor Author

ryao commented Oct 14, 2013

@dweeezil You seem to have found a couple of regressions. I am not sure what caused the module unloading issue, but the zfs destroy dataset@snapshot issue is caused by a slight error in "Fix incorrect usage of strdup() in zfs_unmount_snap()" where I used strncpy instead of strlcpy. I will push a revised version soon.

@dweeezil
Copy link
Contributor

@ryao I suspect you'll want to remove the extern declaration of zfs_txg_synctime_ms in dsl_pool.h as part of the 4045 patch.

@prakashsurya
Copy link
Member

@dweeezil @nedbass Has been working on porting and testing the 4045 changes over. I'd have to ask him to be sure, but my hunch is that parameter will disappear when that patch lands. I don't think it makes sense anymore with 4045.

@dweeezil
Copy link
Contributor

@kpande Did you also adjust the length parameter? See dweeezil/zfs@29b2094.

@dweeezil
Copy link
Contributor

@kpande Try dweeezil/zfs@fdffb9e. The failure condition you discovered was a result of the illumos 3744 patch.

@ryao
Copy link
Contributor Author

ryao commented Oct 16, 2013

@kpande that would be a different bug. Thanks for letting us know about it.

@ryao
Copy link
Contributor Author

ryao commented Oct 24, 2013

I have pushed a refresh. Here are the changes.

  1. The snapshot destruction regression has been squashed. It was caused by changes to zfs_unmount_snap() when porting 3744 zfs shouldn't ignore errors unmounting snapshots.
  2. zfs_unmount_snap() now uses strlcpy() instead of strncpy(), which guarentees proper termination.
  3. I expanded on the commit message for 'Add missing code to zfs_debug.{c,h}'.

I spent the better part of the past week trying to get a new workstation working (sadly, the hardware is not cooperating), so there is not as much here as I would have liked to have. Here is what is still left to do:

  1. If possible, 'Add missing code to zfs_debug.{c,h} ' should be broken into the original Illumos commits for consistency.
  2. The module unloading regression needs to be tackled. From what I am told, it is probably caused by the SPL changes.
  3. Look into @kpande's issue.
  4. Resolve the issue building the kernel modules as C99. Determine if we will be making a switch to C99 with this pull request.

ryao added 6 commits October 25, 2013 09:14
This lock unnecessarily serializes an important path. The fast write
variables are updated using atomic operations, which need no protection.
The code snippet being protected is already protected by the spa_lock.
We remove mc_fastwrite_lock and add an assertion to verify that we have
a spa_lock. This permits multiple threads to run through the
metaslab_alloc_dva() function in parallel as they do on other
implementations.

Signed-off-by: Richard Yao <ryao@gentoo.org>
This was accidentally removed by overzealous commenting.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Modifying the length of a string returned by strdup() is incorrect
because strfree() is allowed to use strlen() to determine which slab
cache was used to do the allocation.

Signed-off-by: Richard Yao <ryao@gentoo.org>
The resolution of a merge conflict when merging Illumos openzfs#3464 caused us
to invert the order couple of function calls in zio_free_sync() versus
what they are in Illumos.

Signed-off-by: Richard Yao <ryao@gentoo.org>
This resolves merge conflicts when merging Illumos openzfs#3588 and Illumos openzfs#4047.

Signed-off-by: Richard Yao <ryao@gentoo.org>
Upstream Illumos uses C99, but we have been building as C89. This
required backporting patches from Illumos from C99 to C89. There are no
known ABI compatibility issues when mixing C89 and C99 code. so lets
just build as C99 and save ourselves some trouble. This patch modifies
the autotools checks, userland code and kernel code to use C89.

Note that the Linux kernel passes -Wdeclaration-after-statement, so we
must pass -Wno-declaration-after-statement to counteract that. We also
do a little cleanup to simplify the makefiles.

Signed-off-by: Richard Yao <ryao@gentoo.org>
unya pushed a commit to unya/zfs that referenced this pull request Dec 13, 2013
References:
  illumos/illumos-gate@d39ee14

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue openzfs#1775

Porting notes:

1. This commit was so old that only two lines applied to the modern
   code base.
unya pushed a commit to unya/zfs that referenced this pull request Dec 13, 2013
References:
  illumos/illumos-gate@44bffe0

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue openzfs#1775
unya pushed a commit to unya/zfs that referenced this pull request Dec 13, 2013
3582 zfs_delay() should support a variable resolution
3584 DTrace sdt probes for ZFS txg states

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Reviewed by: Dan McDonald <danmcd@nexenta.com>
Reviewed by: Richard Elling <richard.elling@dey-sys.com>
Approved by: Garrett D'Amore <garrett@damore.org>

References:
    https://www.illumos.org/issues/3582
    illumos/illumos-gate@0689f76

Ported by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue openzfs#1775
unya pushed a commit to unya/zfs that referenced this pull request Dec 13, 2013
3699 zfs hold or release of a non-existent snapshot does not output error
3739 cannot set zfs quota or reservation on pool version < 22
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Eric Shrock <eric.schrock@delphix.com>
Approved by: Dan McDonald <danmcd@nexenta.com>

References:
  https://www.illumos.org/issues/3699
  https://www.illumos.org/issues/3739
  illumos/illumos-gate@013023d

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue openzfs#1775
unya pushed a commit to unya/zfs that referenced this pull request Dec 13, 2013
3741 zfs needs better comments
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Eric Schrock <eric.schrock@delphix.com>
Approved by: Christopher Siden <christopher.siden@delphix.com>

References:
  https://www.illumos.org/issues/3741
  illumos/illumos-gate@3e30c24

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue openzfs#1775
unya pushed a commit to unya/zfs that referenced this pull request Dec 13, 2013
3742 zfs comments need cleaner, more consistent style
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Eric Schrock <eric.schrock@delphix.com>
Approved by: Christopher Siden <christopher.siden@delphix.com>

References:
  https://www.illumos.org/issues/3742
  illumos/illumos-gate@f717074

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue openzfs#1775

Porting notes:

1. The change to zfs_vfsops.c was dropped because it involves
   zfs_mount_label_policy, which does not exist in the Linux port.
unya pushed a commit to unya/zfs that referenced this pull request Dec 13, 2013
3743 zfs needs a refcount audit
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Eric Schrock <eric.schrock@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Christopher Siden <christopher.siden@delphix.com>

References:
  https://www.illumos.org/issues/3743
  illumos/illumos-gate@b287be1

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue openzfs#1775
unya pushed a commit to unya/zfs that referenced this pull request Dec 13, 2013
3744 zfs shouldn't ignore errors unmounting snapshots
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Christopher Siden <christopher.siden@delphix.com>

References:
  https://www.illumos.org/issues/3744
  illumos/illumos-gate@fc7a6e3

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue openzfs#1775

Porting notes:

1. There is no clear way to distinguish between a failure when we
   tried to unmount the snapdir of a zvol (which does not exist)
   and the failure when we try to unmount a snapdir of a dataset,
   so the changes to zfs_unmount_snap() were dropped in favor of
   an altered Linux function that unconditionally returns 0.
unya pushed a commit to unya/zfs that referenced this pull request Dec 13, 2013
3745 zpool create should treat -O mountpoint and -m the same
3811 zpool create -o altroot=/xyz -O mountpoint=/mnt ignores
     the mountpoint option
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Christopher Siden <christopher.siden@delphix.com>

References:
  https://www.illumos.org/issues/3745
  https://www.illumos.org/issues/3811
  illumos/illumos-gate@8b71377

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue openzfs#1775
unya pushed a commit to unya/zfs that referenced this pull request Dec 13, 2013
3740 Poor ZFS send / receive performance due to snapshot
     hold / release processing
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Christopher Siden <christopher.siden@delphix.com>

References:
  https://www.illumos.org/issues/3740
  illumos/illumos-gate@a7a845e

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue openzfs#1775

Porting notes:

1. 13fe019 introduced a merge conflict
   in dsl_dataset_user_release_tmp where some variables were moved
   outside of the preprocessor directive.

2. dea9dfe made the previous merge
   conflict worse by switching KM_SLEEP to KM_PUSHPAGE. This is notable
   because this commit refactors the code, adding a new KM_SLEEP
   allocation. It is not clear to me whether this should be converted
   to KM_PUSHPAGE.

3. We had a merge conflict in libzfs_sendrecv.c because of copyright
   notices.

4. Several small C99 compatibility fixed were made.
unya pushed a commit to unya/zfs that referenced this pull request Dec 13, 2013
3818 zpool status -x should report pools with removed l2arc devices
Reviewed by: Saso Kiselkov <skiselkov.ml@gmail.com>
Reviewed by: George Wilson <gwilson@zfsmail.com>
Approved by: Christopher Siden <christopher.siden@delphix.com>

References:
  https://www.illumos.org/issues/3818
  illumos/illumos-gate@7f2416e

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue openzfs#1775
unya pushed a commit to unya/zfs that referenced this pull request Dec 13, 2013
3829 fix for 3740 changed behavior of zfs destroy/hold/release ioctl
Reviewed by: Matt Amdur <matt.amdur@delphix.com>
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>

References:
  https://www.illumos.org/issues/3829
  illumos/illumos-gate@bb6e707

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue openzfs#1775
unya pushed a commit to unya/zfs that referenced this pull request Dec 13, 2013
3894 zfs should not allow snapshot of inconsistent dataset
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Gordon Ross <gwr@nexenta.com>

References:
  https://www.illumos.org/issues/3894
  illumos/illumos-gate@ca48f36

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue openzfs#1775
unya pushed a commit to unya/zfs that referenced this pull request Dec 13, 2013
3888 zfs recv -F should destroy any snapshots created since
     the incremental source
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: Peng Dai <peng.dai@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>

References:
  https://www.illumos.org/issues/3888
  illumos/illumos-gate@34f2f8c

Porting notes:

1. Commit 1fde1e3 wrapped a
   declaration in dsl_dataset_modified_since_lastsnap in ASSERTV().
   The ASSERTV() and local variable have been removed to avoid an
   unused variable warning.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Ported-by: Richard Yao <ryao@gentoo.org>
Issue openzfs#1775
unya pushed a commit to unya/zfs that referenced this pull request Dec 13, 2013
3875 panic in zfs_root() after failed rollback
Reviewed by: Jerry Jelinek <jerry.jelinek@joyent.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Gordon Ross <gwr@nexenta.com>

References:
  https://www.illumos.org/issues/3875
  illumos/illumos-gate@91948b5

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue openzfs#1775
unya pushed a commit to unya/zfs that referenced this pull request Dec 13, 2013
3836 zio_free() can be processed immediately in the common case
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Adam Leventhal <ahl@delphix.com>
Approved by: Dan McDonald <danmcd@nexenta.com>

References:
  https://www.illumos.org/issues/3836
  illumos/illumos-gate@9cb154a

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue openzfs#1775
unya pushed a commit to unya/zfs that referenced this pull request Dec 13, 2013
3834 incremental replication of 'holey' file systems is slow
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>

References:
  https://www.illumos.org/issues/3834
  illumos/illumos-gate@ca48f36

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue openzfs#1775
unya pushed a commit to unya/zfs that referenced this pull request Dec 13, 2013
3909 "zfs send -D" does not work
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Christopher Siden <christopher.siden@delphix.com>

References:
  https://www.illumos.org/issues/3909
  illumos/illumos-gate@36f7455

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue openzfs#1775
unya pushed a commit to unya/zfs that referenced this pull request Dec 13, 2013
3973 zfs_ioc_rename alters passed in zc->zc_name
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Christopher Siden <christopher.siden@delphix.com>

References:
  https://www.illumos.org/issues/3973
  illumos/illumos-gate@a0c1127

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue openzfs#1775
unya pushed a commit to unya/zfs that referenced this pull request Dec 13, 2013
3955 ztest failure: assertion refcount_count(&tx->tx_space_written) +
     delta <= tx->tx_space_towrite
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>

References:
  https://www.illumos.org/issues/3955
  illumos/illumos-gate@be9000c

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue openzfs#1775
unya pushed a commit to unya/zfs that referenced this pull request Dec 13, 2013
3949 ztest fault injection should avoid resilvering devices
3950 ztest: deadman fires when we're doing a scan
3951 ztest hang when running dedup test
3952 ztest: ztest_reguid test and ztest_fault_inject don't place nice together
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Adam Leventhal <ahl@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>

References:
  https://www.illumos.org/issues/3949
  https://www.illumos.org/issues/3950
  https://www.illumos.org/issues/3951
  https://www.illumos.org/issues/3952
  illumos/illumos-gate@2c1e2b4

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue openzfs#1775

Porting notes:

1. The deadman thread was removed from ztest during the original
   port because it depended on Solaris thr_create() interface.
   This functionality should be reintroduced using the more
   portable pthreads.
unya pushed a commit to unya/zfs that referenced this pull request Dec 13, 2013
3996 want a libzfs_core API to rollback to latest snapshot
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Andy Stormont <andyjstormont@gmail.com>
Approved by: Richard Lowe <richlowe@richlowe.net>

References:
  https://www.illumos.org/issues/3996
  illumos/illumos-gate@a7027df

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue openzfs#1775
unya pushed a commit to unya/zfs that referenced this pull request Dec 13, 2013
4047 panic from dbuf_free_range() from dmu_free_object() while
     doing zfs receive
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Dan McDonald <danmcd@nexenta.com>

References:
  https://www.illumos.org/issues/4047
  illumos/illumos-gate@713d6c2

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue openzfs#1775

Porting notes:

1. The exported symbol dmu_free_object() was renamed to
   dmu_free_long_object() in Illumos.
unya pushed a commit to unya/zfs that referenced this pull request Dec 13, 2013
4061 libzfs: memory leak in iter_dependents_cb()
Reviewed by: Jeffry Molanus <jeffry.molanus@nexenta.com>
Reviewed by: Boris Protopopov <boris.protopopov@nexenta.com>
Reviewed by: Andy Stormont <andyjstormont@gmail.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Dan McDonald <danmcd@nexenta.com>

References:
  https://www.illumos.org/issues/4061
  illumos/illumos-gate@2fbdf8d

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue openzfs#1775
unya pushed a commit to unya/zfs that referenced this pull request Dec 13, 2013
4046 dsl_dataset_t ds_dir->dd_lock is highly contended
Reviewed by: Eric Schrock <eric.schrock@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Garrett D'Amore <garrett@damore.org>

References:
  https://www.illumos.org/issues/4046
  illumos/illumos-gate@b62969f

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue openzfs#1775

Porting notes:

1. This commit removed dsl_dataset_namelen in Illumos, but that
   appears to have been removed from ZFSOnLinux in an earlier commit.
unya pushed a commit to unya/zfs that referenced this pull request Dec 13, 2013
3954 metaslabs continue to load even after hitting zfs_mg_alloc_failure limit
4080 zpool clear fails to clear pool
4081 need zfs_mg_noalloc_threshold
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>

References:
  https://www.illumos.org/issues/3954
  https://www.illumos.org/issues/4080
  https://www.illumos.org/issues/4081
  illumos/illumos-gate@22e3098

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue openzfs#1775
unya pushed a commit to unya/zfs that referenced this pull request Dec 13, 2013
4082 zfs receive gets EFBIG from dmu_tx_hold_free()
Reviewed by: Eric Schrock <eric.schrock@delphix.com>
Reviewed by: Christopher Siden <christopher.siden@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>

References:
  https://www.illumos.org/issues/4082
  illumos/illumos-gate@5253393

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue openzfs#1775
unya pushed a commit to unya/zfs that referenced this pull request Dec 13, 2013
4168 ztest assertion failure in dbuf_undirty
4169 verbatim import causes zdb to segfault
4170 zhack leaves pool in ACTIVE state
Reviewed by: Adam Leventhal <ahl@delphix.com>
Reviewed by: Eric Schrock <eric.schrock@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Dan McDonald <danmcd@nexenta.com>

References:
  https://www.illumos.org/issues/4168
  https://www.illumos.org/issues/4169
  https://www.illumos.org/issues/4170
  illumos/illumos-gate@7fdd916

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue openzfs#1775
unya pushed a commit to unya/zfs that referenced this pull request Dec 13, 2013
3995 Memory leak of compressed buffers in l2arc_write_done

References:
  https://illumos.org/issues/3995

Ported-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes openzfs#1688
Issue openzfs#1775
unya pushed a commit to unya/zfs that referenced this pull request Dec 13, 2013
A couple of kmem_alloc() allocations were using KM_SLEEP in
the sync thread context.  These were accidentally introduced
by the recent set of Illumos patches.  The solution is to
switch to KM_PUSHPAGE.

dsl_dataset_promote_sync() -> promote_hold() -> snaplist_make() ->
kmem_alloc(sizeof (*snap), KM_SLEEP);

dsl_dataset_user_hold_sync() -> dsl_onexit_hold_cleanup() ->
kmem_alloc(sizeof (*ca), KM_SLEEP)

Signed-off-by: Richard Yao <ryao@gentoo.org>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue openzfs#1775
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants