Proposed zfs-0.8.4 patchset #10209
Merged
Proposed zfs-0.8.4 patchset #10209
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
I think we would like to have the send-dedup deprecation and "zstream dump" command in this release as well. |
|
I propose to backport #10148 too. |
Codecov Report
@@ Coverage Diff @@
## zfs-0.8-release #10209 +/- ##
====================================================
- Coverage 79.02% 67.37% -11.66%
====================================================
Files 381 319 -62
Lines 117796 101220 -16576
====================================================
- Hits 93093 68194 -24899
- Misses 24703 33026 +8323
Continue to review full report at Codecov.
|
Codecov Report
@@ Coverage Diff @@
## zfs-0.8-release #10209 +/- ##
===================================================
- Coverage 78.82% 78.82% -0.01%
===================================================
Files 381 381
Lines 118574 118574
===================================================
- Hits 93468 93465 -3
- Misses 25106 25109 +3
Continue to review full report at Codecov.
|
|
Very nice - good selection of cherrypicks. Only one I'm a little doubtful of is f1646c7 as I'm unsure the risk/reward is worth it for 0.8. |
|
Please no. Can we just have a stable no new features patch. |
|
Could I suggest bd0d24e: Linux 5.5 compat: blkg_tryget() for support of 5.5+ PREEMPT_RCU kernels |
|
May I request #10216 please. |
|
@tonyhutter: our patch stack mimics master a bit closer than what's proposed here. I cherry picked the commits in this PR atop our stack and added the recent ZVOL changes (cant cherry pick that cleanly since the whole os-specific thing moves where the diffs apply). This is what we've been building up since 0.8.1. Since we use grsecurity, we reset the stack atop whatever spender's latest patch is, but this setup has worked for us for quite some time (the particular stack listed there just finished 12h of zloop with no issue against 5.4.33-grsec+). |
|
All - thanks for the patch suggestions. I'll try to get in as many as I can, provided they are stable and merge relatively cleanly. @behlendorf has also sent me an updated patchlist. Note that progress has been somewhat slow, as our build server got deprecated and it's taking me some extra time to get our new build server up and running. |
|
Any chance of getting a version of #10187 into 0.8.4 so that 0.8.4 works with kernel 5.7 (assuming no further breaking changes before the final release)? I am using a version of that patch (here: https://launchpadlibrarian.net/474321719/k5.7-rc1-zfs.patch) applied to Ubuntu's zfs-dkms "successfully" with Linux kernel 5.7-rc2 . I posted that to https://bugs.launchpad.net/ubuntu/+source/zfs-linux/+bug/1872373 but someone with actual expertise should probably figure out if the zvol.c changes are appropriate and not just a working hack. Also I agree with #10187 (comment) that |
|
@satmandu thanks for that patch! I'll include it in the patchset. |
On some systems - openSUSE, for example - there is not yet a writeable temporary file system available, so bash bails out with an error, 'cannot create temp file for here-document: Read-only file system', on the here documents in zfs-mount-generator. The simple fix is to change these into a multi-line echo statement. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-By: Richard Laager <rlaager@wiktel.com> Reviewed-by: George Melikov <mail@gmelikov.ru> Signed-off-by: Lorenz Hüdepohl <dev@stellardeath.org> Closes openzfs#9802
If someone is using both multipathd and ZFS, they are probably using them together. Ordering the zpool imports after multipathd is ready fixes import issues for multipath configurations. Tested-by: Mike Pastore <mike@oobak.org> Reviewed-by: George Melikov <mail@gmelikov.ru> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Richard Laager <rlaager@wiktel.com> Closes openzfs#9863
When growing the size of a (VMEM or KVMEM) kmem cache, spl_cache_grow() always does taskq_dispatch(spl_cache_grow_work), and then waits for the KMC_BIT_GROWING to be cleared by the taskq thread. The taskq thread (spl_cache_grow_work()) does: 1. allocate new slab and add to list 2. wake_up_all(skc_waitq) 3. clear_bit(KMC_BIT_GROWING) Therefore, the waiting thread can wake up before GROWING has been cleared. It will see that the growing has not yet completed, and go back to sleep until it hits the 100ms timeout. This can have an extreme performance impact on workloads that alloc/free more than fits in the (statically-sized) magazines. These workloads allocate and free slabs with high frequency. The problem can be observed with `funclatency spl_cache_grow`, which on some workloads shows that 99.5% of the time it takes <64us to allocate slabs, but we spend ~70% of our time in outliers, waiting for the 100ms timeout. The fix is to do `clear_bit(KMC_BIT_GROWING)` before `wake_up_all(skc_waitq)`. A future investigation should evaluate if we still actually need to taskq_dispatch() at all, and if so on which kernel versions. Reviewed-by: Paul Dagnelie <pcd@delphix.com> Reviewed-by: Pavel Zakharov <pavel.zakharov@delphix.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-by: George Wilson <gwilson@delphix.com> Signed-off-by: Matthew Ahrens <mahrens@delphix.com> Closes openzfs#9989
The correct name for the mount unit for / is "-.mount", not ".mount". Reviewed-by: InsanePrawn <insane.prawny@gmail.com> Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Co-authored-by: Antonio Russo <antonio.e.russo@gmail.com> Signed-off-by: Richard Laager <rlaager@wiktel.com> Closes openzfs#9970
Commit torvalds/linux@9e8d42a0f accidentally converted the static inline function blkg_tryget() to GPL-only for kernels built with CONFIG_PREEMPT_RCU=y and CONFIG_BLK_CGROUP=y. Resolve the build issue by providing our own equivalent functionality when needed which uses rcu_read_lock_sched() internally as before. Reviewed-by: Tony Hutter <hutter2@llnl.gov> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes openzfs#9745 Closes openzfs#10072
When configuring as builtin (--enable-linux-builtin) for kernels without loadable module support (CONFIG_MODULES=n) only the object file is created. Never a loadable kmod. Update ZFS_LINUX_TRY_COMPILE to handle this in a manor similar to the ZFS_LINUX_TEST_COMPILE_ALL macro. Reviewed-by: George Melikov <mail@gmelikov.ru> Reviewed-by: Tony Hutter <hutter2@llnl.gov> Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov> Closes openzfs#9887 Closes openzfs#10063
tonyhutter
force-pushed
the
zfs-0.8.4-hutter
branch
2 times, most recently
from
ddf0d44 to
2c18481
Compare
|
Looks like I need to update |
tonyhutter
force-pushed
the
zfs-0.8.4-hutter
branch
2 times, most recently
from
aac5a98 to
39ef9cc
Compare
|
Looks like Centos 7 and possibly Centos 6 may be panicking early in the test suite. I'm looking into it. |
|
Recently fixed #10239 Would be extremely useful |
tonyhutter
force-pushed
the
zfs-0.8.4-hutter
branch
from
39ef9cc to
e210e31
Compare
|
May I suggest #10246 ? It is an one line fix. |
|
@gamanakis thanks, I'll pull that one in. |
|
I feel obligated to point out that the included systemd mount generator commits are a little more than 'bugfixes' by most standards, but
Overall I think those 'feature' changes are very benign and I think it's much better to include them as a whole than not at all, just wanted to give everyone a heads up. |
|
My last push fixed the crashes I was seeing on Centos 6-7 (thanks @behlendorf). It looks like a lot of builders are now failing |
tonyhutter
force-pushed
the
zfs-0.8.4-hutter
branch
3 times, most recently
from
7658c24 to
7b908e5
Compare
Otherwise when running with reference_tracking_enable=TRUE mounting and unmounting an encrypted dataset panics with: Call Trace: dump_stack+0x66/0x90 slab_err+0xcd/0xf2 ? __kmalloc+0x174/0x260 ? __kmem_cache_shutdown+0x158/0x240 __kmem_cache_shutdown.cold+0x1d/0x115 shutdown_cache+0x11/0x140 kmem_cache_destroy+0x210/0x230 spl_kmem_cache_destroy+0x122/0x3e0 [spl] zfs_refcount_fini+0x11/0x20 [zfs] spa_fini+0x4b/0x120 [zfs] zfs_kmod_fini+0x6b/0xa0 [zfs] _fini+0xa/0x68c [zfs] __x64_sys_delete_module+0x19c/0x2b0 do_syscall_64+0x5b/0x1a0 entry_SYSCALL_64_after_hwframe+0x44/0xa9 Reviewed-By: Brian Behlendorf <behlendorf1@llnl.gov> Reviewed-By: Tom Caputi <tcaputi@datto.com> Signed-off-by: George Amanakis <gamanakis@gmail.com> Closes openzfs#10246
META file and changelog updated. Signed-off-by: Tony Hutter <hutter2@llnl.gov> TEST_ZIMPORT_SKIP="yes"
tonyhutter
force-pushed
the
zfs-0.8.4-hutter
branch
from
0106cad to
f5fd31f
Compare
tonyhutter
changed the title
|
I do not see #10163 and #10184 in this patchset, any chance they could be still included? I recon ZVOL performance is actually an important aspect of basic ZFS functionality, at least for some users (like me). They've been first mentioned in this thread over two weeks ago, and merged in master at least 3 weeks ago. |
I think a feature deprecation would too big of a change for a minor version bump. OpenZFS 2.0 would be a much better target IMO. Would it block/complicate any of the proposed 0.8.4 changes if it wasn't deprecated? |
|
I'm working on updating the official Debian packaging, and I'm testing this patch set on Debian unstable. I'm consistently getting a test failure for alloc_class_013_pos
Tailing last 200 lines of zfs_dbgmsg log
Tailing last 200 lines of dmesg logThis is on a (hosted offsite) shared VM. The EDIT: This test case passes on Debian's Linux kernel Outdated: I was able to get the full test suite to pass (after significant adjustments) on kernel 5.4 with ZFS 0.8.3, so I suspect this represents something actually failing. There are other failures that I can reproduce, but this is the first one that occurs, and I've learned not to trust subsequent errors. If this is helpful, let me know and I can feed more test failures. |
|
@Atemu We want to add the deprecation notice (which does NOT remove any functionality) so that folks can be aware of this in advance of the next major release (2.0), which will have the functionality removed. |
|
Quick update - I've been testing the prospective 0.8.4 packages on Centos 6-8 and Fedora 30-32. They seem to be passing Centos, but are failing on the Fedoras with mostly the same tests: @aerusso that's the same @Bronek yea the zvol changes are really great. I'd like to include them in the next release though, just to reduce risk in this release. |
|
@tonyhutter The problem doesn't appear for me in Linux 5.5, but does in 5.6. (See my edit) Also, I have seen some of those failures in my testing, but I haven't tried to understand any ones after the first failing test because the VM I'm running in is flaky and earlier failing results seem to trigger cascades of failures (i.e., "unable to create testpool" because it already exists). |
|
@ahrens thanks for clarifying, that makes a lot of sense actually! |
|
@tonyhutter @aerusso we're also seeing these test failures on the master branch with the 5.6 kernel. |
|
May I ask for #10283 to be considered? Now this is technically a new feature enabling mirrored pools with indirect vdevs to be split, and was just merged, but the added code is minimal. |
|
@gamanakis thanks. We just put out the 0.8.4 release, but we'll take a look at that patch for 0.8.5. |
12 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation and Context
WIP proposed 0.8.4 patchset. This is not a complete list yet, more patches to follow. The target here is to get ZFS building under Fedora 32 w/5.6 kernel and aspirationally the 5.7 kernel.
Description
Mostly build fixes, and a few low-risk fixes/optimizations.
How Has This Been Tested?
Only build tested on F32
Types of changes
Checklist:
Signed-off-by.