Manylinux 2022

manylinux/README.md

@@ -0,0 +1,44 @@
+Manylinux build scripts for wxPython
+====================================
+
+The code in this folder can be used to build a manylinux version of wxPython.
+This means that a single wxPython wheel can be used on more than a single Linux
+distro like what we've been limited to so far. This is accomplished by including
+in the wheel file the shared libraries for all of the dependencies of wxWidgets.
+This is primarily the GTK3 libs and all of its dependencies, although there are
+some others too.
+
+In general this works, but it's not a perfect solution. Here are some potential
+issues:
+
+* Since the wheel has its own instance of GTK and other dependent libraries, the
+  wheel file is quite large compared to one created specifically for the distro
+  where it will be deployed. (Around 2+ times larger.)
+
+* The private copy of the GTK libs will not play well with GTK themes and other
+  resources that are installed on the system. In fact, there may be warnings
+  printed indicating that "your installation may be broken."
+
+* In order to get new enough versions of wxWidgets' dependencies, we need to use
+  the newest (at the time of this writing) version of the manylinux images,
+  resulting in the platform tag being set to "manylinux_2_28". In other words,
+  the version of glibc on the target system needs to be >= 2.28, and the wheels
+  will not work on older systems. A newish version of Pip is also required, one
+  that recognizes this platform tag.
+
+How to build
+------------
+
+Building is simple. You need a system with docker available, and you need a
+wxPython sdist to be present in the ../dist folder. This makes it easy to build
+using the exact same source code that was used to build already released
+versions of wxPython, and also saves the time needed for the code generation
+steps. The Python invoke package is also needed.
+
+To start a build run a command like the following:
+
+```
+$ invoke build --pythons "3.8, 3.9, 3.10"
+```
+
+The resulting manylinux wheels will be moved to ../dist when they are done.

manylinux/do-build.sh

@@ -0,0 +1,104 @@
+#!/bin/bash
+#--------------------------------------------------------------------------
+# This script is run inside the manylinux Docker container to perform the
+# actual build of the Phoenix wheels. See tasks.py and README.md for details.
+#--------------------------------------------------------------------------
+set -o errexit
+#set -o xtrace
+
+export PYTHONUNBUFFERED=1
+
+VERSION=$1
+shift
+PYTHONS="$@"
+if [ "" == "$PYTHONS" ]; then
+    PYTHONS=3.8
+fi
+
+echo VERSION: $VERSION
+echo PYTHONS: $PYTHONS
+
+
+function do_setup() {
+    # Install extra pacakges needed for the build
+    echo Installing Linux packages...
+    yum -y -q install \
+        freeglut-devel \
+        mesa-libGL-devel \
+        mesa-libGLU-devel \
+        gstreamer1-devel \
+        gstreamer1-plugins-base-devel \
+        gtk3-devel \
+        libjpeg-devel \
+        libnotify \
+        libnotify-devel \
+        libpng-devel \
+        libSM-devel \
+        libtiff-devel \
+        libXtst-devel \
+        SDL2-devel \
+        webkit2gtk3-devel
+
+    # Unpack the source archive
+    mkdir /build
+    cd /build
+    tar xzf /dist/wxPython-$VERSION.tar.gz
+}
+
+
+function do_build() {
+    # Remove the '.' from the version. IOW, 3.9 --> 39
+    py=$(echo $1 | sed 's/\.//')
+    cd /build/wxPython-$VERSION
+
+    # There's something odd about how the Pythons are set up on this image,
+    # and/or how virtual envs are created with these Pythons. When using a
+    # virtual env the python-config tools give the wrong paths for includes,
+    # libs, etc. Maybe it's too many layers of symlinks? Anyway, we need to use
+    # the real python binary instead of a virual env for our build because waf
+    # needs a working python-config.
+    PYTHON=/opt/python/cp$py-*/bin/python
+    OLD_PATH=$PATH
+    export PATH=$(dirname $PYTHON):$PATH
+
+    # build
+    $PYTHON -m pip install -U -r requirements.txt
+    $PYTHON -m pip install -U pip setuptools wheel
+    $PYTHON build.py build_wx
+    $PYTHON build.py build_py
+    $PYTHON build.py bdist_wheel
+
+    # do the manylinux magic
+    auditwheel show dist/wxPython-$VERSION-*linux*.whl
+    auditwheel repair -w dist  dist/wxPython-$VERSION-*linux*.whl
+
+    # do a quickie smoke-test with a virtual env
+    mkdir tmp
+    $PYTHON -m venv tmp/test
+    tmp/test/bin/pip install -U pip
+    tmp/test/bin/pip install dist/wxPython-*manylinux*.whl
+    tmp/test/bin/python -c "import wx; print(wx.version())"
+    rm -rf tmp
+
+    # Save the manylinux wheel to the host folder
+    mv dist/wxPython-*-manylinux*.whl /dist
+    rm dist/wxPython-*.whl
+    export PATH=$OLD_PATH
+
+    # Clean up the Python parts of this build, since we can do more than one
+    # build per invocation of the docker image.
+    rm -rf build/waf wx/__pycache__
+    rm -f wx/*.so || true
+}
+
+
+
+do_setup
+for pyver in $PYTHONS; do
+    do_build $pyver
+done
+
+if [ "$INTERACTIVE" == "yes" ]; then
+    exec /bin/bash -i
+fi
+

manylinux/tasks.py

@@ -0,0 +1,65 @@
+import os
+import glob
+import sys
+from invoke import task, run
+
+HERE = os.path.abspath(os.path.dirname(__file__))
+IMAGE = 'quay.io/pypa/manylinux_2_28_{}'
+
+
+@task(
+    help={
+        'cmd': "If given will run this command instead of the image's default command.",
+        'keep': "Keep the container when it exits. Otherwise it will be auto-removed.",
+        'extra': "Extra flags to pass to the docker command line.",
+        'arch': 'The architecture to build for. "x86_64" (default) or "aarch64".'
+    },
+)
+def run(ctx, cmd=None, keep=False, extra='', arch='x86_64'):
+    """
+    Run the manylinux docker image.
+    """
+    os.chdir(HERE)
+    dist = os.path.abspath('../dist')
+    cwd = os.getcwd()
+    cmd = '' if cmd is None else cmd
+    rm = '' if keep else '--rm'
+    image = IMAGE.format(arch)
+    ctx.run(
+        f'docker run -it {rm} -v {dist}:/dist -v {cwd}:/scripts {extra} {image} {cmd}',
+        pty=True, echo=True)
+
+
+@task(
+    help={
+        'pythons': "Comma separated list of Python verions to build for.",
+        'keep':    "Keep the container when it exits. Otherwise it will be auto-removed.",
+        'interactive': "Run a shell when the build script is done so the container can be examined.",
+        'arch': 'The architecture to build for. "x86_64" (default) or "aarch64".'
+    }
+)
+def build(ctx, pythons='', keep=False, interactive=False, arch='x86_64'):
+    """
+    Run the build(s)
+    """
+    # Ensure we've got a source archive available
+    source = glob.glob('../dist/wxPython-*.tar.gz')
+    if not source:
+        print('ERROR: no source archive found in ../dist')
+        sys.exit(1)
+    if len(source) > 1:
+        print('ERROR: Too many source archives found in ../dist')
+        sys.exit(1)
+    source = source[0]
+    version = source[17:-7]
+    if pythons == 'all':
+        pythons = '3.8, 3.9, 3.10'
+    if pythons == '':
+        pythons = '3.8'
+
+    pythons = pythons.split(',')
+    pythons = ' '.join(pythons)
+    cmd = f'/scripts/do-build.sh {version} {pythons}'
+    extra = '-e INTERACTIVE=yes' if interactive else ''
+    run(ctx, cmd, keep, extra, arch)
+

packaging/HOWTO-Release.rst

@@ -80,15 +80,18 @@ HOWTO Release wxPython Phoenix
 17. Tag the released revision in git, using a name like wxPython-4.0.0 (using
     the actual version number of course.) Push the tag to all remotes.
 
-18. Bump the version numbers in buildtools/version.py appropriately for the
+18. Create a release at GitHub. Copy the text about the "Source Code" downloads
+    from the previous release, and upload the official source tarball.
+
+19. Bump the version numbers in buildtools/version.py appropriately for the
     next anticipated release, so future snapshot builds will be recognized as
     pre-release development versions for the next official release, not the
     one just completed.
 
-19. If making an announcement about this release, (I think it's okay not to
+20. If making an announcement about this release, (I think it's okay not to
     for minor releases or smallish bug fixes,) send the text in
     packaging/ANNOUNCE.txt to the email addresses listed at the top of the
     file.
 
-20. Add a news post to the wxPython site about the release.
+21. Add a news post to the wxPython site about the release.