Fail to build on ppc64be linux 6.1 due to using GPL-only symbol

[jiaolovekt]

System information

Type	Version/Name
Distribution Name	Debian
Distribution Version	sid
Kernel Version	Linux 6.1.0-5-powerpc64
Architecture	powerpc64
OpenZFS Version	zfs-2.1.9 and zfs-2.1.99-1746-gbf1bec394

Describe the problem you're observing

Cannot build zfs.ko

Describe how to reproduce the problem

Build zfs

Include any warning/errors/backtraces from the system logs

ERROR: modpost: GPL-incompatible module zfs.ko uses GPL-only symbol 'cpu_feature_keys'
make[2]: *** [/usr/src/linux-headers-6.1.0-5-common/scripts/Makefile.modpost:126: /root/src/zfs/build.6.1.0-5-powerpc64/module/Module.symvers] Error 1
make[1]: *** [/usr/src/linux-headers-6.1.0-5-common/Makefile:1982: modpost] Error 2
make[1]: Leaving directory '/usr/src/linux-headers-6.1.0-5-powerpc64'
make: *** [Makefile:56: modules-Linux] Error 2

[Low-power]

It seems a left over of #12590 (comment), where Linux powerpc64le somehow didn't get affected by this bug. And of course the issue is in Linux, as it should export cpu_feature_keys to all modules instead of GPL-only.

[Low-power]

I can reproduce the issue with Linux 6.2.1 once I turned on CONFIG_JUMP_LABEL and CONFIG_JUMP_LABEL_FEATURE_CHECKS, which are enabled in most distributions. OpenZFS version in this test is commit 620a977.

build log:

...
make -f ../scripts/Makefile.modpost
   scripts/mod/modpost -m      -o /home/whr/src/zfs/build.6.2.1-rivoreo-powerpc64-largepage/module/Module.symvers -T /home/whr/src/zfs/build.6.2.1-rivoreo-powerpc64-largepage/module/modules.order -i Module.symvers -e 
ERROR: modpost: GPL-incompatible module zfs.ko uses GPL-only symbol 'cpu_feature_keys'
make[2]: *** [../scripts/Makefile.modpost:138: /home/whr/src/zfs/build.6.2.1-rivoreo-powerpc64-largepage/module/Module.symvers] Error 1
make[1]: *** [../Makefile:1973: modpost] Error 2
make[1]: Leaving directory '/home/whr/src/linux-6.2.1/build.rivoreo-powerpc64-largepage'
make: *** [Makefile:56: modules-Linux] Error 2

gcc version:

Using built-in specs.
COLLECT_GCC=gcc-13-20230226
COLLECT_LTO_WRAPPER=/opt/gcc-13-20230226/bin/../lib/gcc/powerpc-unknown-linux-gnu/13.0.1/lto-wrapper
Target: powerpc-unknown-linux-gnu
Configured with: ../gcc-13-20230226/configure --build=powerpc-unknown-linux-gnu --prefix=/usr/local --sysconfdir=/etc --localstatedir=/var --libexecdir='/usr/local/lib' --enable-version-specific-runtime-libs --disable-rpath --with-system-zlib --enable-languages=c,c++,objc,obj-c++,fortran,lto --enable-initfini-array --enable-gnu-indirect-function --program-suffix=-13-20230226 --enable-gnu-unique-object --enable-plugin --disable-softfloat --enable-targets=powerpc-linux,powerpc64-linux --enable-multiarch --with-long-double-128 --disable-multilib
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 13.0.1 20230226 (experimental) (GCC) 

lscpu:

Architecture:            ppc64
  CPU op-mode(s):        32-bit, 64-bit
  Byte Order:            Big Endian
CPU(s):                  4
  On-line CPU(s) list:   0-3
Model name:              POWER7 (architected), altivec supported
  Model:                 2.3 (pvr 003f 0203)
  Thread(s) per core:    4
  Core(s) per socket:    1
  Socket(s):             1
Virtualization features: 
  Hypervisor vendor:     pHyp
  Virtualization type:   para
Caches (sum of all):     
  L1d:                   32 KiB (1 instance)
  L1i:                   32 KiB (1 instance)
  L2:                    256 KiB (1 instance)
  L3:                    4 MiB (1 instance)
NUMA:                    
  NUMA node(s):          1
  NUMA node0 CPU(s):     0-3

[Low-power]

Symbol references analysis in linux-6.2.1 and zfs-2.1.99-1780-g620a977

The individual translation units that references cpu_feature_keys are (excluding zfs.o of course):

whr@debian:~/src/zfs/build.6.2.1-rivoreo-powerpc64-largepage/module$ grep -F cpu_feature_keys -r *
Binary file icp/algs/sha2/sha256_impl.o matches
Binary file icp/algs/sha2/sha512_impl.o matches
Binary file os/linux/zfs/zfs_znode.o matches
Binary file os/linux/zfs/zfs_vnops_os.o matches
Binary file zfs/vdev_raidz_math_powerpc_altivec.o matches
Binary file zfs.o matches

By digging around the assembly code in above files, I found the dependency chains are as follows:

• cpu_feature_keys<-cpu_has_feature<-zfs_vsx_available<-sha256_have_vsx<-icp/algs/sha2/sha256_impl.o
• cpu_feature_keys<-cpu_has_feature<-zfs_vsx_available<-sha512_have_vsx<-icp/algs/sha2/sha512_impl.o
• cpu_feature_keys<-cpu_has_feature<-flush_dcache_page<-zfs_zero_partial_page<-os/linux/zfs/zfs_znode.o
• cpu_feature_keys<-cpu_has_feature<-flush_dcache_page<-update_pages<-os/linux/zfs/zfs_vnops_os.o
• cpu_feature_keys<-cpu_has_feature<-flush_dcache_page<-mappedread<-os/linux/zfs/zfs_vnops_os.o
• cpu_feature_keys<-cpu_has_feature<-zfs_altivec_available<-raidz_will_powerpc_altivec_work<-zfs/vdev_raidz_math_powerpc_altivec.o

I think OpenZFS can workaround this bug by reimplementing cpu_has_feature without using cpu_feature_keys on affected Linux versions, which basically just disables the CONFIG_JUMP_LABEL_FEATURE_CHECKS optimization for cpu_has_feature from zfs codes.

[behlendorf]

Closed by #14590.