Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OpenZFS 2.2.4 build fail on Debian 12 #16155

Open
dguskovv opened this issue May 3, 2024 · 11 comments
Open

OpenZFS 2.2.4 build fail on Debian 12 #16155

dguskovv opened this issue May 3, 2024 · 11 comments
Labels
Component: Packaging custom packages Type: Defect Incorrect behavior (e.g. crash, hang)

Comments

@dguskovv
Copy link

dguskovv commented May 3, 2024

System information

Type Version/Name
Distribution Name Debian
Distribution Version 12
Kernel Version 6.1.0-20-amd64
Architecture x86_64
OpenZFS Version 2.2.4

Describe the problem you're observing

I tried to compile version 2.2.4 exactly the same as versions 2.2.x before it

sh autogen.sh
./configure
make -j$(nproc) deb-utils

In the end I get the error

RPM build errors:
    File not found: /tmp/zfs-build-root-A9VcYovo/BUILDROOT/zfs-2.2.4-1.x86_64/usr/lib/python3/dist-packages/libzfs_core/*
    File not found: /tmp/zfs-build-root-A9VcYovo/BUILDROOT/zfs-2.2.4-1.x86_64/usr/lib/python3/dist-packages/pyzfs*
make[1]: *** [Makefile:14373: rpm-common] Error 1
make[1]: Leaving directory '/usr/src/zfs/2.2.4/6.1.0-20/zfs-2.2.4'
make: *** [Makefile:14327: rpm-utils-initramfs] Error 2

At the same time, in the directory /tmp/zfs-build-root-A9VcYovo/BUILDROOT/zfs-2.2.4-1.x86_64/usr/lib/ there is python3.11, but not python3 as indicated in the error

# ls -la /tmp/zfs-build-root-A9VcYovo/BUILDROOT/zfs-2.2.4-1.x86_64/usr/lib/python3.11/site-packages/
total 0
drwxr-xr-x 4 root root  80 May  3 06:39 .
drwxr-xr-x 3 root root  60 May  3 06:39 ..
drwxr-xr-x 5 root root 240 May  3 06:39 libzfs_core
drwxr-xr-x 2 root root 160 May  3 06:39 pyzfs-2.2.4-py3.11.egg-info

Versions 2.2.1, 2.2.2, 2.2.3 - compiled previously without this problem.

All these packages are installed:
build-essential autoconf automake libtool gawk alien fakeroot dkms libblkid-dev uuid-dev libudev-dev libssl-dev zlib1g-dev libaio-dev libattr1-dev libelf-dev linux-headers-generic python3 python3-dev python3-setuptools python3-cffi libffi-dev python3-packaging git libcurl4-openssl-dev debhelper-compat dh-python po-debconf python3-all-dev python3-sphinx parallel

Describe how to reproduce the problem

Build OpenZFS 2.2.4 on Debian 12 from scratch

Include any warning/errors/backtraces from the system logs

@dguskovv dguskovv added the Type: Defect Incorrect behavior (e.g. crash, hang) label May 3, 2024
@robn
Copy link
Member

robn commented May 3, 2024

Confirmed. Quick fix: revert 71216b9 (#16126 #16129).

(My Python skills are nonexistent, so I'm not much use on a fix).

I have the same output as above. Note also dist-packages vs site-packages.

Here the Python bits of config.log, if they're useful:

configure:18082: checking for python version
configure:18090: result: 3.11
configure:18095: checking for python platform
configure:18103: result: linux
configure:18164: checking for GNU default python prefix
configure:18166: result: ${prefix}
configure:18232: checking for GNU default python exec_prefix
configure:18234: result: ${exec_prefix}
configure:18265: checking for python script directory (pythondir)
configure:18307: result: ${PYTHON_PREFIX}/lib/python3.11/site-packages
configure:18315: checking for python extension module directory (pyexecdir)
configure:18357: result: ${PYTHON_EXEC_PREFIX}/lib/python3.11/site-packages
configure:18479: checking for python3 module: packaging
configure:18484: result: yes
configure:18563: checking for python3.11
configure:18598: result: /usr/bin/python3
configure:18620: checking for a version of Python >= '2.1.0'
configure:18660: result: yes
configure:18673: checking for a version of Python >= '3.6.0'
configure:18700: result: yes
configure:18728: checking for the sysconfig Python package
configure:18732: result: yes
configure:18766: checking for Python include path
configure:18791: result: -I/usr/include/python3.11
configure:18798: checking for Python library path
configure:18885: result: -L/usr/lib/x86_64-linux-gnu -lpython3.11
configure:18892: checking for Python site-packages path
configure:18916: result: /usr/lib/python3.11/site-packages
configure:18923: checking for Python platform specific site-packages path
configure:18947: result: /usr/lib/python3.11/site-packages
configure:18954: checking python extra libraries
configure:18961: result: -ldl -lm
configure:18968: checking python extra linking flags
configure:18977: result: -Xlinker -export-dynamic -Wl,-O1 -Wl,-Bsymbolic-functions
configure:18984: checking consistency of all components of python development environment
configure:19012: gcc -o conftest -O2 -g  -I/usr/include/python3.11  -Xlinker -export-dynamic -Wl,-O1 -Wl,-Bsymbolic-functions conftest.c  -L/usr/lib/x86_64-linux-gnu -lpython3.11 -ldl -lm >&5
configure:19012: $? = 0
configure:19031: result: yes
configure:19610: checking for python3 module: setuptools
configure:19615: result: yes
configure:19648: checking for python3 module: cffi
configure:19653: result: yes

@dguskovv
Copy link
Author

dguskovv commented May 3, 2024

Even Debian sid (unstable) doesn't use python 3.12. Only Debian experimental contains python 3.12.

At the moment I'm just using

./configure --disable-pyzfs

@thisisnotmyrealname
Copy link

Same issue for me on Ubuntu 22.04.04 which has python 3.10.12

@rincebrain
Copy link
Contributor

I bet I even know why.

IIRC Debian does evil nonsense to some but not all of the override mechanisms for where packages go, so after 71216b9, we probably hit one of the ones where it doesn't override it, and things end up in site-packages not dist-packages. (See also).

This explains the insanity. So basically, Debian monkeypatches only distutils for this, and not relying on distutils thereby produces unexpected results.

@pcd1193182
Copy link
Contributor

pcd1193182 commented May 3, 2024

We just ran into this internally, and @rincebrain's analysis is correct. The proper fix here probably involves tweaking ax_python_devel.m4 to use a scheme that puts the file in the right place on debian (the 'posix_local' or maybe 'deb_system' scheme appears to work), and then to have zfs.spec.in use the $PYTHON_SITE_PKG variable to define where it expects the file to end up. That may require some more tweaking, though, because the installation process uses the --prefix flag to setup.py to tweak the install location further.

@pcd1193182
Copy link
Contributor

https://gist.github.com/pcd1193182/f177c2899219ad874fa45e61a6cf3d3b Might be a starting point for a fix?

@m-ueberall
Copy link

https://gist.github.com/pcd1193182/f177c2899219ad874fa45e61a6cf3d3b Might be a starting point for a fix?

@pcd1193182: Applying the above patch as-is inside an Ubuntu Focal/20.04 build container (followed by ./autogen.sh; ./configure; make deb) changes the resulting errors from

RPM build errors:
    File not found: /tmp/zfs-build-root-q0ionx4M/BUILDROOT/zfs-2.2.4-1.x86_64/usr/lib/python3/dist-packages/libzfs_core/*
    File not found: /tmp/zfs-build-root-q0ionx4M/BUILDROOT/zfs-2.2.4-1.x86_64/usr/lib/python3/dist-packages/pyzfs*
make[1]: *** [Makefile:14374: rpm-common] Error 1
make[1]: Leaving directory '/tmp/zfs_focal'
make: *** [Makefile:14328: rpm-utils-initramfs] Error 2

to

RPM build errors:
    File not found: /tmp/zfs-build-root-d6NFk0pc/BUILDROOT/zfs-2.2.4-1.x86_64/usr/local/lib/python3.8/site-packages/libzfs_core/*
    File not found: /tmp/zfs-build-root-d6NFk0pc/BUILDROOT/zfs-2.2.4-1.x86_64/usr/local/lib/python3.8/site-packages/pyzfs*
make[1]: *** [Makefile:14374: rpm-common] Error 1
make[1]: Leaving directory '/tmp/zfs_focal'
make: *** [Makefile:14328: rpm-utils-initramfs] Error 2

Are you sure this is supposed to read /usr/local/? python3-pyzfs v2.2.3-1 previously used /usr/ (which is what I'd expect).

@rincebrain
Copy link
Contributor

rincebrain commented May 6, 2024 via email

@wangmice
Copy link

the same error on ubuntu 24.04 ,python 3.12

@aubreybailey
Copy link

aubreybailey commented Jun 8, 2024

Edit: looks like it's fixed in mainline 24.01 now

@comrade-meowski comrade-meowski mentioned this issue Aug 6, 2024
13 tasks
@PeterFalken
Copy link

PeterFalken commented Aug 8, 2024

Just tried to build 2.2.5 on debian bookworm and still get this error.
Had to use the workaround:
./configure --disable-pyzfs --enable-systemd

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Component: Packaging custom packages Type: Defect Incorrect behavior (e.g. crash, hang)
Projects
None yet
Development

No branches or pull requests

10 participants