|
| 1 | +==================== |
| 2 | +Python on iOS README |
| 3 | +==================== |
| 4 | + |
| 5 | +:Authors: |
| 6 | + Russell Keith-Magee (2023-11) |
| 7 | + |
| 8 | +This document provides a quick overview of some iOS specific features in the |
| 9 | +Python distribution. |
| 10 | + |
| 11 | +These instructions are only needed if you're planning to compile Python for iOS |
| 12 | +yourself. Most users should *not* need to do this. If you're looking to |
| 13 | +experiment with writing an iOS app in Python, tools such as `BeeWare's Briefcase |
| 14 | +<https://briefcase.readthedocs.io>`__ and `Kivy's Buildozer |
| 15 | +<https://buildozer.readthedocs.io>`__ will provide a much more approachable |
| 16 | +user experience. |
| 17 | + |
| 18 | +Compilers for building on iOS |
| 19 | +============================= |
| 20 | + |
| 21 | +Building for iOS requires the use of Apple's Xcode tooling. It is strongly |
| 22 | +recommended that you use the most recent stable release of Xcode. This will |
| 23 | +require the use of the most (or second-most) recently released macOS version, |
| 24 | +as Apple does not maintain Xcode for older macOS versions. The Xcode Command |
| 25 | +Line Tools are not sufficient for iOS development; you need a *full* Xcode |
| 26 | +install. |
| 27 | + |
| 28 | +If you want to run your code on the iOS simulator, you'll also need to install |
| 29 | +an iOS Simulator Platform. You should be prompted to select an iOS Simulator |
| 30 | +Platform when you first run Xcode. Alternatively, you can add an iOS Simulator |
| 31 | +Platform by selecting an open the Platforms tab of the Xcode Settings panel. |
| 32 | + |
| 33 | +iOS specific arguments to configure |
| 34 | +=================================== |
| 35 | + |
| 36 | +* ``--enable-framework=DIR`` |
| 37 | + |
| 38 | + This argument specifies the location where the Python.framework will be |
| 39 | + installed. This argument is required for all iOS builds; a directory *must* |
| 40 | + be specified. |
| 41 | + |
| 42 | +* ``--with-framework-name=NAME`` |
| 43 | + |
| 44 | + Specify the name for the Python framework; defaults to ``Python``. |
| 45 | + |
| 46 | +Building Python on iOS |
| 47 | +====================== |
| 48 | + |
| 49 | +ABIs and Architectures |
| 50 | +---------------------- |
| 51 | + |
| 52 | +iOS apps can be deployed on physical devices, and on the iOS simulator. Although |
| 53 | +the API used on these devices is identical, the ABI is different - you need to |
| 54 | +link against different libraries for an iOS device build (``iphoneos``) or an |
| 55 | +iOS simulator build (``iphonesimulator``). |
| 56 | + |
| 57 | +Apple uses the ``XCframework`` format to allow specifying a single dependency |
| 58 | +that supports multiple ABIs. An ``XCframework`` is a wrapper around multiple |
| 59 | +ABI-specific frameworks that share a common API. |
| 60 | + |
| 61 | +iOS can also support different CPU architectures within each ABI. At present, |
| 62 | +there is only a single supported architecture on physical devices - ARM64. |
| 63 | +However, the *simulator* supports 2 architectures - ARM64 (for running on Apple |
| 64 | +Silicon machines), and x86_64 (for running on older Intel-based machines). |
| 65 | + |
| 66 | +To support multiple CPU architectures on a single platform, Apple uses a "fat |
| 67 | +binary" format - a single physical file that contains support for multiple |
| 68 | +architectures. It is possible to compile and use a "thin" single architecture |
| 69 | +version of a binary for testing purposes; however, the "thin" binary will not be |
| 70 | +portable to machines using other architectures. |
| 71 | + |
| 72 | +Building a single-architecture framework |
| 73 | +---------------------------------------- |
| 74 | + |
| 75 | +The Python build system will create a ``Python.framework`` that supports a |
| 76 | +*single* ABI with a *single* architecture. Unlike macOS, iOS does not allow a |
| 77 | +framework to contain non-library content, so the iOS build will produce a |
| 78 | +``bin`` and ``lib`` folder in the same output folder as ``Python.framework``. |
| 79 | +The ``lib`` folder will be needed at runtime to support the Python library. |
| 80 | + |
| 81 | +If you want to use Python in a real iOS project, you need to produce multiple |
| 82 | +``Python.framework`` builds, one for each ABI and architecture. iOS builds of |
| 83 | +Python *must* be constructed as framework builds. To support this, you must |
| 84 | +provide the ``--enable-framework`` flag when configuring the build. The build |
| 85 | +also requires the use of cross-compilation. The minimal commands for building |
| 86 | +Python for the ARM64 iOS simulator will look something like:: |
| 87 | + |
| 88 | + $ export PATH="`pwd`/iOS/Resources/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin" |
| 89 | + $ ./configure \ |
| 90 | + AR=arm64-apple-ios-simulator-ar \ |
| 91 | + CC=arm64-apple-ios-simulator-clang \ |
| 92 | + CPP=arm64-apple-ios-simulator-cpp \ |
| 93 | + CXX=arm64-apple-ios-simulator-clang \ |
| 94 | + --enable-framework=/path/to/install \ |
| 95 | + --host=arm64-apple-ios-simulator \ |
| 96 | + --build=arm64-apple-darwin \ |
| 97 | + --with-build-python=/path/to/python.exe |
| 98 | + $ make |
| 99 | + $ make install |
| 100 | + |
| 101 | +In this invocation: |
| 102 | + |
| 103 | +* ``iOS/Resources/bin`` has been added to the path, providing some shims for the |
| 104 | + compilers and linkers needed by the build. Xcode requires the use of ``xcrun`` |
| 105 | + to invoke compiler tooling. However, if ``xcrun`` is pre-evaluated and the |
| 106 | + result passed to ``configure``, these results can embed user- and |
| 107 | + version-specific paths into the sysconfig data, which limits the portability |
| 108 | + of the compiled Python. Alternatively, if ``xcrun`` is used *as* the compiler, |
| 109 | + it requires that compiler variables like ``CC`` include spaces, which can |
| 110 | + cause significant problems with many C configuration systems which assume that |
| 111 | + ``CC`` will be a single executable. |
| 112 | + |
| 113 | + To work around this problem, the ``iOS/Resources/bin`` folder contains some |
| 114 | + wrapper scripts that present as simple compilers and linkers, but wrap |
| 115 | + underlying calls to ``xcrun``. This allows configure to use a ``CC`` |
| 116 | + definition without spaces, and without user- or version-specific paths, while |
| 117 | + retaining the ability to adapt to the local Xcode install. These scripts are |
| 118 | + included in the ``bin`` directory of an iOS install. |
| 119 | + |
| 120 | + These scripts will, by default, use the currently active Xcode installation. |
| 121 | + If you want to use a different Xcode installation, you can use |
| 122 | + ``xcode-select`` to set a new default Xcode globally, or you can use the |
| 123 | + ``DEVELOPER_DIR`` environment variable to specify an Xcode install. The |
| 124 | + scripts will use the default ``iphoneos``/``iphonesimulator`` SDK version for |
| 125 | + the select Xcode install; if you want to use a different SDK, you can set the |
| 126 | + ``IOS_SDK_VERSION`` environment variable. (e.g, setting |
| 127 | + ``IOS_SDK_VERSION=17.1`` would cause the scripts to use the ``iphoneos17.1`` |
| 128 | + and ``iphonesimulator17.1`` SDKs, regardless of the Xcode default.) |
| 129 | + |
| 130 | + The path has also been cleared of any user customizations. A common source of |
| 131 | + bugs is for tools like Homebrew to accidentally leak macOS binaries into an iOS |
| 132 | + build. Resetting the path to a known "bare bones" value is the easiest way to |
| 133 | + avoid these problems. |
| 134 | + |
| 135 | +* ``/path/to/install`` is the location where the final ``Python.framework`` will |
| 136 | + be output. |
| 137 | + |
| 138 | +* ``--host`` is the architecture and ABI that you want to build, in GNU compiler |
| 139 | + triple format. This will be one of: |
| 140 | + |
| 141 | + - ``arm64-apple-ios`` for ARM64 iOS devices. |
| 142 | + - ``arm64-apple-ios-simulator`` for the iOS simulator running on Apple |
| 143 | + Silicon devices. |
| 144 | + - ``x86_64-apple-ios-simulator`` for the iOS simulator running on Intel |
| 145 | + devices. |
| 146 | + |
| 147 | +* ``--build`` is the GNU compiler triple for the machine that will be running |
| 148 | + the compiler. This is one of: |
| 149 | + |
| 150 | + - ``arm64-apple-darwin`` for Apple Silicon devices. |
| 151 | + - ``x86_64-apple-darwin`` for Intel devices. |
| 152 | + |
| 153 | +* ``/path/to/python.exe`` is the path to a Python binary on the machine that |
| 154 | + will be running the compiler. This is needed because the Python compilation |
| 155 | + process involves running some Python code. On a normal desktop build of |
| 156 | + Python, you can compile a python interpreter and then use that interpreter to |
| 157 | + run Python code. However, the binaries produced for iOS won't run on macOS, so |
| 158 | + you need to provide an external Python interpreter. This interpreter must be |
| 159 | + the same version as the Python that is being compiled. To be completely safe, |
| 160 | + this should be the *exact* same commit hash. However, the longer a Python |
| 161 | + release has been stable, the more likely it is that this constraint can be |
| 162 | + relaxed - the same micro version will often be sufficient. |
| 163 | + |
| 164 | +For a full CPython build, you also need to specify the paths to iOS builds of |
| 165 | +the binary libraries that CPython depends on (XZ, BZip2, LibFFI and OpenSSL). |
| 166 | +This can be done by defining the ``LIBLZMA_CFLAGS``, ``LIBLZMA_LIBS``, |
| 167 | +``BZIP2_CFLAGS``, ``BZIP2_LIBS``, ``LIBFFI_CFLAGS``, and ``LIBFFI_LIBS`` |
| 168 | +environment variables, and the ``--with-openssl`` configure option. Versions of |
| 169 | +these libraries pre-compiled for iOS can be found in `this repository |
| 170 | +<https://github.com/beeware/cpython-apple-source-deps/releases>`__. |
| 171 | + |
| 172 | +By default, Python will be compiled with an iOS deployment target (i.e., the |
| 173 | +minimum supported iOS version) of 12.0. To specify a different deployment |
| 174 | +target, provide the version number as part of the ``--host`` argument - for |
| 175 | +example, ``--host=arm64-apple-ios15.4-simulator`` would compile an ARM64 |
| 176 | +simulator build with a deployment target of 15.4. |
| 177 | + |
| 178 | +Merge thin frameworks into fat frameworks |
| 179 | +----------------------------------------- |
| 180 | + |
| 181 | +Once you've built a ``Python.framework`` for each ABI and and architecture, you |
| 182 | +must produce a "fat" framework for each ABI that contains all the architectures |
| 183 | +for that ABI. |
| 184 | + |
| 185 | +The ``iphoneos`` build only needs to support a single architecture, so it can be |
| 186 | +used without modification. |
| 187 | + |
| 188 | +If you only want to support a single simulator architecture, (e.g., only support |
| 189 | +ARM64 simulators), you can use a single architecture ``Python.framework`` build. |
| 190 | +However, if you want to create ``Python.xcframework`` that supports *all* |
| 191 | +architectures, you'll need to merge the ``iphonesimulator`` builds for ARM64 and |
| 192 | +x86_64 into a single "fat" framework. |
| 193 | + |
| 194 | +The "fat" framework can be constructed by performing a directory merge of the |
| 195 | +content of the two "thin" ``Python.framework`` directories, plus the ``bin`` and |
| 196 | +``lib`` folders for each thin framework. When performing this merge: |
| 197 | + |
| 198 | +* The pure Python standard library content is identical for each architecture, |
| 199 | + except for a handful of platform-specific files (such as the ``sysconfig`` |
| 200 | + module). Ensure that the "fat" framework has the union of all standard library |
| 201 | + files. |
| 202 | + |
| 203 | +* Any binary files in the standard library, plus the main |
| 204 | + ``libPython3.X.dylib``, can be merged using the ``lipo`` tool, provide by |
| 205 | + Xcode:: |
| 206 | + |
| 207 | + $ lipo -create -output module.dylib path/to/x86_64/module.dylib path/to/arm64/module.dylib |
| 208 | + |
| 209 | +* The header files will be indentical on both architectures, except for |
| 210 | + ``pyconfig.h``. Copy all the headers from one platform (say, arm64), rename |
| 211 | + ``pyconfig.h`` to ``pyconfig-arm64.h``, and copy the ``pyconfig.h`` for the |
| 212 | + other architecture into the merged header folder as ``pyconfig-x86_64.h``. |
| 213 | + Then copy the ``iOS/Resources/pyconfig.h`` file from the CPython sources into |
| 214 | + the merged headers folder. This will allow the two Python architectures to |
| 215 | + share a common ``pyconfig.h`` header file. |
| 216 | + |
| 217 | +At this point, you should have 2 Python.framework folders - one for ``iphoneos``, |
| 218 | +and one for ``iphonesimulator`` that is a merge of x86+64 and ARM64 content. |
| 219 | + |
| 220 | +Merge frameworks into an XCframework |
| 221 | +------------------------------------ |
| 222 | + |
| 223 | +Now that we have 2 (potentially fat) ABI-specific frameworks, we can merge those |
| 224 | +frameworks into a single ``XCframework``. |
| 225 | + |
| 226 | +The initial skeleton of an ``XCframework`` is built using:: |
| 227 | + |
| 228 | + xcodebuild -create-xcframework -output Python.xcframework -framework path/to/iphoneos/Python.framework -framework path/to/iphonesimulator/Python.framework |
| 229 | + |
| 230 | +Then, copy the ``bin`` and ``lib`` folders into the architecture-specific slices of |
| 231 | +the XCframework:: |
| 232 | + |
| 233 | + cp path/to/iphoneos/bin Python.xcframework/ios-arm64 |
| 234 | + cp path/to/iphoneos/lib Python.xcframework/ios-arm64 |
| 235 | + |
| 236 | + cp path/to/iphonesimulator/bin Python.xcframework/ios-arm64_x86-64-simulator |
| 237 | + cp path/to/iphonesimulator/lib Python.xcframework/ios-arm64_x86-64-simulator |
| 238 | + |
| 239 | +Note that the name of the architecture-specific slice for the simulator will |
| 240 | +depend on the CPU architecture that you build. |
| 241 | + |
| 242 | +Then, add symbolic links to "common" platform names for each slice:: |
| 243 | + |
| 244 | + ln -si ios-arm64 Python.xcframework/iphoneos |
| 245 | + ln -si ios-arm64_x86-64-simulator Python.xcframework/iphonesimulator |
| 246 | + |
| 247 | +You now have a Python.xcframework that can be used in a project. |
| 248 | + |
| 249 | +Testing Python on iOS |
| 250 | +===================== |
| 251 | + |
| 252 | +The ``iOS/testbed`` folder that contains an Xcode project that is able to run |
| 253 | +the iOS test suite. This project converts the Python test suite into a single |
| 254 | +test case in Xcode's XCTest framework. The single XCTest passes if the test |
| 255 | +suite passes. |
| 256 | + |
| 257 | +To run the test suite, configure a Python build for an iOS simulator (i.e., |
| 258 | +``--host=arm64-apple-ios-simulator`` or ``--host=x86_64-apple-ios-simulator`` |
| 259 | +), setting the framework location to the testbed project:: |
| 260 | + |
| 261 | + --enable-framework="./iOS/testbed/Python.xcframework/ios-arm64_x86_64-simulator" |
| 262 | + |
| 263 | +Then run ``make all install testiOS``. This will build an iOS framework for your |
| 264 | +chosen architecture, install the Python iOS framework into the testbed project, |
| 265 | +and run the test suite on an "iPhone SE (3rd generation)" simulator. |
| 266 | + |
| 267 | +While the test suite is running, Xcode does not display any console output. |
| 268 | +After showing some Xcode build commands, the console output will print ``Testing |
| 269 | +started``, and then appear to stop. It will remain in this state until the test |
| 270 | +suite completes. On a 2022 M1 MacBook Pro, the test suite takes approximately 12 |
| 271 | +minutes to run; a couple of extra minutes is required to boot and prepare the |
| 272 | +iOS simulator. |
| 273 | + |
| 274 | +On success, the test suite will exit and report successful completion of the |
| 275 | +test suite. No output of the Python test suite will be displayed. |
| 276 | + |
| 277 | +On failure, the output of the Python test suite *will* be displayed. This will |
| 278 | +show the details of the tests that failed. |
| 279 | + |
| 280 | +Debugging test failures |
| 281 | +----------------------- |
| 282 | + |
| 283 | +The easiest way to diagnose a single test failure is to open the testbed project |
| 284 | +in Xcode and run the tests from there using the "Product > Test" menu item. |
| 285 | + |
| 286 | +Running specific tests |
| 287 | +^^^^^^^^^^^^^^^^^^^^^^ |
| 288 | + |
| 289 | +As the test suite is being executed on an iOS simulator, it is not possible to |
| 290 | +pass in command line arguments to configure test suite operation. To work around |
| 291 | +this limitation, the arguments that would normally be passed as command line |
| 292 | +arguments are configured as a static string at the start of the XCTest method |
| 293 | +``- (void)testPython`` in ``iOSTestbedTests.m``. To pass an argument to the test |
| 294 | +suite, add a a string to the ``argv`` defintion. These arguments will be passed |
| 295 | +to the test suite as if they had been passed to ``python -m test`` at the |
| 296 | +command line. |
| 297 | + |
| 298 | +Disabling automated breakpoints |
| 299 | +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 300 | + |
| 301 | +By default, Xcode will inserts an automatic breakpoint whenever a signal is |
| 302 | +raised. The Python test suite raises many of these signals as part of normal |
| 303 | +operation; unless you are trying to diagnose an issue with signals, the |
| 304 | +automatic breakpoints can be inconvenient. However, they can be disabled by |
| 305 | +creating a symbolic breakpoint that is triggered at the start of the test run. |
| 306 | + |
| 307 | +Select "Debug > Breakpoints > Create Symbolic Breakpoint" from the Xcode menu, and |
| 308 | +populate the new brewpoint with the following details: |
| 309 | + |
| 310 | +* **Name**: IgnoreSignals |
| 311 | +* **Symbol**: UIApplicationMain |
| 312 | +* **Action**: Add debugger commands for: |
| 313 | + - ``process handle SIGINT -n true -p true -s false`` |
| 314 | + - ``process handle SIGUSR1 -n true -p true -s false`` |
| 315 | + - ``process handle SIGUSR2 -n true -p true -s false`` |
| 316 | + - ``process handle SIGXFSZ -n true -p true -s false`` |
| 317 | +* Check the "Automatically continue after evaluating" box. |
| 318 | + |
| 319 | +All other details can be left blank. When the process executes the |
| 320 | +``UIApplicationMain`` entry point, the breakpoint will trigger, run the debugger |
| 321 | +commands to disable the automatic breakpoints, and automatically resume. |
0 commit comments