Linux 5.8 __vmalloc compat #10422

c0d3z3r0 · 2020-06-08T08:57:38Z

The pgprot argument has been removed from __vmalloc in Linux 5.8 [1],
being PAGE_KERNEL always now.

Detect this during configure to use the right function call in spl.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/mm/vmalloc.c?h=next-20200605&id=88dca4ca5a93d2c09e5bbc6a62fbfc3af83c4fca

Co-authored-by: Sebastian Gottschall s.gottschall@dd-wrt.com
Co-authored-by: Michael Niewöhner foss@mniewoehner.de
Signed-off-by: Sebastian Gottschall s.gottschall@dd-wrt.com
Signed-off-by: Michael Niewöhner foss@mniewoehner.de

Motivation and Context

Description

How Has This Been Tested?

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Performance enhancement (non-breaking change which improves efficiency)
Code cleanup (non-breaking change which makes code smaller or more readable)
Breaking change (fix or feature that would cause existing functionality to change)
Documentation (a change to man pages or other documentation)

Checklist:

My code follows the ZFS on Linux code style requirements.
I have updated the documentation accordingly.
I have read the contributing document.
I have added tests to cover my changes.
I have run the ZFS Test Suite with this change applied.
All commit messages are properly formatted and contain Signed-off-by.

c0d3z3r0 · 2020-06-08T09:00:56Z

An alternative would be redefining __vmalloc in a new include/os/linux/kernel/linux/vmalloc.h

BrainSlayer · 2020-06-08T10:31:46Z

An alternative would be redefining __vmalloc in a new include/os/linux/kernel/linux/vmalloc.h

good point.. i think i will change this in this way

BrainSlayer · 2020-06-08T11:13:36Z

@c0d3z3r0 made a new commit which follows your suggestion. please review

codecov · 2020-06-08T15:49:34Z

Codecov Report

Merging #10422 into master will decrease coverage by 13.36%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           master   #10422       +/-   ##
===========================================
- Coverage   79.39%   66.03%   -13.37%     
===========================================
  Files         391      308       -83     
  Lines      123633   106348    -17285     
===========================================
- Hits        98162    70225    -27937     
- Misses      25471    36123    +10652

Flag	Coverage Δ
#kernel	`?`
#user	`66.03% <ø> (+0.62%)`	⬆️

Impacted Files	Coverage Δ
module/zfs/objlist.c	`0.00% <0.00%> (-100.00%)`	⬇️
module/zfs/pathname.c	`0.00% <0.00%> (-100.00%)`	⬇️
include/sys/dmu_redact.h	`0.00% <0.00%> (-100.00%)`	⬇️
include/sys/dmu_traverse.h	`0.00% <0.00%> (-100.00%)`	⬇️
module/zfs/zfs_rlock.c	`0.00% <0.00%> (-96.36%)`	⬇️
module/lua/ltablib.c	`2.34% <0.00%> (-95.32%)`	⬇️
module/zfs/bqueue.c	`0.00% <0.00%> (-94.45%)`	⬇️
module/zcommon/zfs_deleg.c	`0.00% <0.00%> (-92.46%)`	⬇️
module/zfs/dmu_diff.c	`0.00% <0.00%> (-87.88%)`	⬇️
module/zfs/zcp_set.c	`0.00% <0.00%> (-87.10%)`	⬇️
... and 233 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c9e319f...cbc8971. Read the comment docs.

c0d3z3r0 · 2020-06-08T16:02:11Z

@BrainSlayer yup that's what I meant, thx! I slightly simplified it (tbh I was surprised that it works this way \o/)

codecov · 2020-06-08T16:02:35Z

Codecov Report

Merging #10422 into master will increase coverage by 0.06%.
The diff coverage is 66.66%.

@@            Coverage Diff             @@
##           master   #10422      +/-   ##
==========================================
+ Coverage   79.39%   79.45%   +0.06%     
==========================================
  Files         391      391              
  Lines      123633   123632       -1     
==========================================
+ Hits        98162    98237      +75     
+ Misses      25471    25395      -76

Flag	Coverage Δ
#kernel	`79.94% <66.66%> (+0.10%)`	⬆️
#user	`65.54% <ø> (+0.14%)`	⬆️

Impacted Files	Coverage Δ
module/os/linux/spl/spl-kmem.c	`89.23% <50.00%> (-0.17%)`	⬇️
module/os/linux/spl/spl-kmem-cache.c	`84.81% <100.00%> (+9.22%)`	⬆️
module/zfs/vdev_indirect.c	`73.33% <0.00%> (-11.67%)`	⬇️
cmd/zvol_id/zvol_id_main.c	`76.31% <0.00%> (-5.27%)`	⬇️
module/zfs/vdev_removal.c	`93.90% <0.00%> (-2.76%)`	⬇️
module/zcommon/zfs_fletcher.c	`75.65% <0.00%> (-2.64%)`	⬇️
module/os/linux/zfs/vdev_file.c	`80.37% <0.00%> (-1.87%)`	⬇️
cmd/zed/agents/fmd_api.c	`88.61% <0.00%> (-1.78%)`	⬇️
module/zfs/vdev_indirect_mapping.c	`97.10% <0.00%> (-1.45%)`	⬇️
cmd/zed/agents/zfs_diagnosis.c	`77.55% <0.00%> (-1.17%)`	⬇️
... and 52 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c9e319f...507245a. Read the comment docs.

c0d3z3r0 · 2020-06-08T16:03:23Z

yay, this even made codecov green :D

behlendorf

Thanks for sorting this out!

behlendorf · 2020-06-08T16:30:03Z

include/os/linux/kernel/linux/vmalloc_compat.h

+#ifdef HAVE_VMALLOC_PAGE_KERNEL
+#define	__vmalloc(size, gfp_flags) \
+    __vmalloc(size, gfp_flags, PAGE_KERNEL)
+#endif /* HAVE_VMALLOC_PAGE_KERNEL */


Since we only intend to use this wrapper from the SPL, let's move it to include/os/linux/spl/sys/kmem.h with the other Linux kmem compatibility code and add the spl_ prefix. I was thinking something like this after the void *spl_kvmalloc(size_t size, gfp_t flags); prototype.

/* * 5.8 API change, pgprot_t argument removed. */ #ifdef HAVE_VMALLOC_PAGE_KERNEL #define spl_vmalloc(size, flags) __vmalloc(size, flags, PAGE_KERNEL) #else #define spl_vmalloc(size, flags) __vmalloc(size, flags) #endif

ack, we can even move the __GFP_RETRY_MAYFAIL compat code there

@c0d3z3r0 yep. makes sense

@behlendorf include/os/linux/spl/sys is for userspace and kernel space. but include/os/linux/kernel/linux is for kernel only. so kernel wrappers should be placed in this folder as i initially did. spl_vmalloc should never be used from userspace and should be out of reach by its declaration

c0d3z3r0 · 2020-06-08T17:07:29Z

@BrainSlayer by moving to spl code, this get's relicensed from CCDL to GPL. do you agree with that?

BrainSlayer · 2020-06-08T17:56:57Z

@c0d3z3r0 i dont mind its nothing i see as intellectual property. do as you wish. but take care about the cstyle error i see

c0d3z3r0 · 2020-06-08T18:08:34Z

@c0d3z3r0 i dont mind its nothing i see as intellectual property. do as you wish. but take care about the cstyle error i see

have the same opinion on that, but in that area better be safe than sorry

BrainSlayer · 2020-06-08T18:12:31Z

all okay. i'm currently preparing all changes in my tree. just need to finish all compiling tests locally. gimme 5 minutes. then check my PR

The `pgprot` argument has been removed from `__vmalloc` in Linux 5.8, being `PAGE_KERNEL` always now [1]. Detect this during configure and redefine `__vmalloc` older kernels. [1] https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/mm/vmalloc.c?h=next-20200605&id=88dca4ca5a93d2c09e5bbc6a62fbfc3af83c4fca Co-authored-by: Brian Behlendorf <behlendorf1@llnl.gov> Co-authored-by: Sebastian Gottschall <s.gottschall@dd-wrt.com> Co-authored-by: Michael Niewöhner <foss@mniewoehner.de> Signed-off-by: Sebastian Gottschall <s.gottschall@dd-wrt.com> Signed-off-by: Michael Niewöhner <foss@mniewoehner.de>

The `pgprot` argument has been removed from `__vmalloc` in Linux 5.8, being `PAGE_KERNEL` always now [1]. Detect this during configure and define a wrapper for older kernels. [1] https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/mm/vmalloc.c?h=next-20200605&id=88dca4ca5a93d2c09e5bbc6a62fbfc3af83c4fca Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Co-authored-by: Sebastian Gottschall <s.gottschall@dd-wrt.com> Co-authored-by: Michael Niewöhner <foss@mniewoehner.de> Signed-off-by: Sebastian Gottschall <s.gottschall@dd-wrt.com> Signed-off-by: Michael Niewöhner <foss@mniewoehner.de> Closes openzfs#10422

The `pgprot` argument has been removed from `__vmalloc` in Linux 5.8, being `PAGE_KERNEL` always now [1]. Detect this during configure and define a wrapper for older kernels. [1] https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/mm/vmalloc.c?h=next-20200605&id=88dca4ca5a93d2c09e5bbc6a62fbfc3af83c4fca Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Co-authored-by: Sebastian Gottschall <s.gottschall@dd-wrt.com> Co-authored-by: Michael Niewöhner <foss@mniewoehner.de> Signed-off-by: Sebastian Gottschall <s.gottschall@dd-wrt.com> Signed-off-by: Michael Niewöhner <foss@mniewoehner.de> Closes #10422

The `pgprot` argument has been removed from `__vmalloc` in Linux 5.8, being `PAGE_KERNEL` always now [1]. Detect this during configure and define a wrapper for older kernels. [1] https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/mm/vmalloc.c?h=next-20200605&id=88dca4ca5a93d2c09e5bbc6a62fbfc3af83c4fca Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Co-authored-by: Sebastian Gottschall <s.gottschall@dd-wrt.com> Co-authored-by: Michael Niewöhner <foss@mniewoehner.de> Signed-off-by: Sebastian Gottschall <s.gottschall@dd-wrt.com> Signed-off-by: Michael Niewöhner <foss@mniewoehner.de> Closes openzfs#10422 (cherry picked from commit 080102a)

Adding this patch makes zfs compile against 5.8-rc7 kernels. Upstream PR: openzfs/zfs#10422 Modifications: - spl_kvmalloc does not exists in zfs 0.8.4 - related code was dropped.

The `pgprot` argument has been removed from `__vmalloc` in Linux 5.8, being `PAGE_KERNEL` always now [1]. Detect this during configure and define a wrapper for older kernels. [1] https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/mm/vmalloc.c?h=next-20200605&id=88dca4ca5a93d2c09e5bbc6a62fbfc3af83c4fca Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Co-authored-by: Sebastian Gottschall <s.gottschall@dd-wrt.com> Co-authored-by: Michael Niewöhner <foss@mniewoehner.de> Signed-off-by: Sebastian Gottschall <s.gottschall@dd-wrt.com> Signed-off-by: Michael Niewöhner <foss@mniewoehner.de> Closes openzfs#10422

The `pgprot` argument has been removed from `__vmalloc` in Linux 5.8, being `PAGE_KERNEL` always now [1]. Detect this during configure and define a wrapper for older kernels. [1] https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/mm/vmalloc.c?h=next-20200605&id=88dca4ca5a93d2c09e5bbc6a62fbfc3af83c4fca Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Co-authored-by: Sebastian Gottschall <s.gottschall@dd-wrt.com> Co-authored-by: Michael Niewöhner <foss@mniewoehner.de> Signed-off-by: Sebastian Gottschall <s.gottschall@dd-wrt.com> Signed-off-by: Michael Niewöhner <foss@mniewoehner.de> Closes openzfs#10422 (cherry picked from commit 080102a) - apply to 0.8.4 before certain files were moved around - config/kernel-kmem.m4 exists in git but not release tarballs because it is unused; introduce it in a new file to prevent conflicts - linux/mm.h is included in git master via sys/kmem.h; do not remove it here or the build will error due to undefined is_vmalloc_addr() Original-patch-by: Michael Niewöhner <c0d3z3r0@users.noreply.github.com> Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>

eli-schwartz · 2020-08-17T22:49:50Z

module/os/linux/spl/spl-kmem.c

@@ -26,7 +26,6 @@
 #include <sys/sysmacros.h>
 #include <sys/kmem.h>
 #include <sys/vmem.h>
-#include <linux/mm.h>


The file still uses is_vmalloc_addr() and trying to backport the patch to 0.8.x results in compile errors. It turns out sys/kmem.h includes linux/mm.h but only in git master.

Is this indirection intentional?

The `pgprot` argument has been removed from `__vmalloc` in Linux 5.8, being `PAGE_KERNEL` always now [1]. Detect this during configure and define a wrapper for older kernels. [1] https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/mm/vmalloc.c?h=next-20200605&id=88dca4ca5a93d2c09e5bbc6a62fbfc3af83c4fca Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Co-authored-by: Sebastian Gottschall <s.gottschall@dd-wrt.com> Co-authored-by: Michael Niewöhner <foss@mniewoehner.de> Signed-off-by: Sebastian Gottschall <s.gottschall@dd-wrt.com> Signed-off-by: Michael Niewöhner <foss@mniewoehner.de> Closes #10422 (cherry picked from commit 080102a) - apply to 0.8.4 before certain files were moved around - config/kernel-kmem.m4 exists in git but not release tarballs because it is unused; introduce it in a new file to prevent conflicts - linux/mm.h is included in git master via sys/kmem.h; do not remove it here or the build will error due to undefined is_vmalloc_addr() Original-patch-by: Michael Niewöhner <c0d3z3r0@users.noreply.github.com> Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>

backport kernel 5.8 support from openzfs/zfs#10728 (see openzfs/zfs#10422)

The `pgprot` argument has been removed from `__vmalloc` in Linux 5.8, being `PAGE_KERNEL` always now [1]. Detect this during configure and define a wrapper for older kernels. [1] https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/mm/vmalloc.c?h=next-20200605&id=88dca4ca5a93d2c09e5bbc6a62fbfc3af83c4fca Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Co-authored-by: Sebastian Gottschall <s.gottschall@dd-wrt.com> Co-authored-by: Michael Niewöhner <foss@mniewoehner.de> Signed-off-by: Sebastian Gottschall <s.gottschall@dd-wrt.com> Signed-off-by: Michael Niewöhner <foss@mniewoehner.de> Closes openzfs#10422 (cherry picked from commit 080102a) - apply to 0.8.4 before certain files were moved around - config/kernel-kmem.m4 exists in git but not release tarballs because it is unused; introduce it in a new file to prevent conflicts - linux/mm.h is included in git master via sys/kmem.h; do not remove it here or the build will error due to undefined is_vmalloc_addr() Original-patch-by: Michael Niewöhner <c0d3z3r0@users.noreply.github.com> Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>

Adding this patch makes zfs compile against 5.8-rc7 kernels. Upstream PR: openzfs/zfs#10422 Modifications: - spl_kvmalloc does not exists in zfs 0.8.4 - related code was dropped.

The `pgprot` argument has been removed from `__vmalloc` in Linux 5.8, being `PAGE_KERNEL` always now [1]. Detect this during configure and define a wrapper for older kernels. [1] https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/mm/vmalloc.c?h=next-20200605&id=88dca4ca5a93d2c09e5bbc6a62fbfc3af83c4fca Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Co-authored-by: Sebastian Gottschall <s.gottschall@dd-wrt.com> Co-authored-by: Michael Niewöhner <foss@mniewoehner.de> Signed-off-by: Sebastian Gottschall <s.gottschall@dd-wrt.com> Signed-off-by: Michael Niewöhner <foss@mniewoehner.de> Closes openzfs#10422

backport kernel 5.8 support from openzfs/zfs#10728 (see openzfs/zfs#10422)

c0d3z3r0 mentioned this pull request Jun 8, 2020

Introduce ZSTD compression to ZFS #10278

Closed

17 tasks

c0d3z3r0 force-pushed the for-upstream/kernel-next branch from cbc8971 to 9fb9eaf Compare June 8, 2020 16:02

c0d3z3r0 force-pushed the for-upstream/kernel-next branch from 9fb9eaf to 52a54a9 Compare June 8, 2020 16:05

behlendorf reviewed Jun 8, 2020

View reviewed changes

behlendorf added the Status: Code Review Needed Ready for review and testing label Jun 8, 2020

c0d3z3r0 force-pushed the for-upstream/kernel-next branch 4 times, most recently from f764564 to dbb9d4a Compare June 8, 2020 18:37

c0d3z3r0 force-pushed the for-upstream/kernel-next branch from dbb9d4a to 507245a Compare June 8, 2020 18:43

c0d3z3r0 mentioned this pull request Jun 8, 2020

Move GFP flags kernel compatibility code #10424

Merged

12 tasks

behlendorf approved these changes Jun 8, 2020

View reviewed changes

behlendorf added Status: Accepted Ready to integrate (reviewed, tested) and removed Status: Code Review Needed Ready for review and testing labels Jun 8, 2020

behlendorf merged commit 080102a into openzfs:master Jun 8, 2020

nivedita76 mentioned this pull request Jun 9, 2020

Build fixes #10421

Closed

12 tasks

rtreffer mentioned this pull request Jul 31, 2020

zfs: backport linux 5.8 compatibility patch to 0.8.4 release NixOS/nixpkgs#94377

Merged

10 tasks

Atemu mentioned this pull request Aug 17, 2020

Linux 5.8 build error #10533

Closed

eli-schwartz reviewed Aug 17, 2020

View reviewed changes

eli-schwartz mentioned this pull request Aug 18, 2020

[backport] Linux 5.8+: various compat fixes (__vmalloc, make_request_fn, NR_SLAB_RECLAIMABLE etc.) #10728

Merged

12 tasks

eli-schwartz added a commit to eli-schwartz/pkgbuilds that referenced this pull request Aug 21, 2020

upgpkg: zfs-dkms 0.8.4-2

2e13664

backport kernel 5.8 support from openzfs/zfs#10728 (see openzfs/zfs#10422)

plevart mentioned this pull request Sep 1, 2020

Fedora 32 updated to kernel 5.8.4-200.fc32.x86_64 dkms build errors: too many arguments to function ‘__vmalloc’ #10862

Closed

zer0def pushed a commit to zer0def/PKGBUILDs that referenced this pull request Sep 10, 2021

upgpkg: zfs-dkms 0.8.4-2

737fea4

backport kernel 5.8 support from openzfs/zfs#10728 (see openzfs/zfs#10422)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Linux 5.8 __vmalloc compat #10422

Linux 5.8 __vmalloc compat #10422

c0d3z3r0 commented Jun 8, 2020

c0d3z3r0 commented Jun 8, 2020

BrainSlayer commented Jun 8, 2020

BrainSlayer commented Jun 8, 2020

codecov bot commented Jun 8, 2020

c0d3z3r0 commented Jun 8, 2020

codecov bot commented Jun 8, 2020 •

edited

Loading

c0d3z3r0 commented Jun 8, 2020

behlendorf left a comment

behlendorf Jun 8, 2020

c0d3z3r0 Jun 8, 2020

BrainSlayer Jun 8, 2020

BrainSlayer Jun 8, 2020

c0d3z3r0 commented Jun 8, 2020

BrainSlayer commented Jun 8, 2020

c0d3z3r0 commented Jun 8, 2020

BrainSlayer commented Jun 8, 2020

eli-schwartz Aug 17, 2020

Linux 5.8 __vmalloc compat #10422

Linux 5.8 __vmalloc compat #10422

Conversation

c0d3z3r0 commented Jun 8, 2020

Motivation and Context

Description

How Has This Been Tested?

Types of changes

Checklist:

c0d3z3r0 commented Jun 8, 2020

BrainSlayer commented Jun 8, 2020

BrainSlayer commented Jun 8, 2020

codecov bot commented Jun 8, 2020

Codecov Report

c0d3z3r0 commented Jun 8, 2020

codecov bot commented Jun 8, 2020 • edited Loading

Codecov Report

c0d3z3r0 commented Jun 8, 2020

behlendorf left a comment

Choose a reason for hiding this comment

behlendorf Jun 8, 2020

Choose a reason for hiding this comment

c0d3z3r0 Jun 8, 2020

Choose a reason for hiding this comment

BrainSlayer Jun 8, 2020

Choose a reason for hiding this comment

BrainSlayer Jun 8, 2020

Choose a reason for hiding this comment

c0d3z3r0 commented Jun 8, 2020

BrainSlayer commented Jun 8, 2020

c0d3z3r0 commented Jun 8, 2020

BrainSlayer commented Jun 8, 2020

eli-schwartz Aug 17, 2020

Choose a reason for hiding this comment

codecov bot commented Jun 8, 2020 •

edited

Loading