Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python 3.13 support #442

Merged
merged 47 commits into from
Oct 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
a9144ea
chore(deps): upgrade numpy to support Python 3.13
Xmader Aug 21, 2024
d80c162
fix: `_PyDictView` removed from Python's public API
Xmader Aug 21, 2024
dac79df
fix: Python 3.13 removed `_PyThreadState_GetDict(PyThreadState *)` fr…
Xmader Aug 21, 2024
9b7c598
chore(CI): enable Python 3.13 build in the CI
Xmader Aug 21, 2024
fc734e2
Merge branch 'main' into Xmader/feat/python-3.13-support
Xmader Aug 21, 2024
dc4442d
WIP: remove the use of the `_PyArg_CheckPositional` private API
Xmader Aug 21, 2024
3feb5b7
fix: Python 3.13 moved the undocumented `_PyArg_UnpackKeywords` funct…
Xmader Aug 22, 2024
e4a4fda
fix: Python 3.13 moved the undocumented `_PyErr_SetKeyError` function…
Xmader Aug 22, 2024
428353c
fix: Python 3.13 added the `PyLong_AsNativeBytes` API, but changed th…
Xmader Aug 22, 2024
a96d52b
Merge branch 'main' into Xmader/feat/python-3.13-support
Xmader Sep 12, 2024
2f30f99
cleanup unused code
philippedistributive Sep 17, 2024
db5ba71
fix: `_Py_IsFinalizing` is renamed to `Py_IsFinalizing` in Python 3.13
Xmader Sep 20, 2024
07da1db
refactor: move the `_PyDictViewObject` shim to the `pyshim.hh` header…
Xmader Sep 20, 2024
a8ac1f6
feat: add shim for `_PyDictView_New`
Xmader Sep 20, 2024
9fd5412
fix: `_PyErr_SetKeyError` is removed in Python 3.13
Xmader Sep 20, 2024
8d2f1a1
WIP: `_PyThreadState_GetDict(tstate)` API gets removed in Python 3.13
Xmader Sep 20, 2024
9826939
WIP: `_PyArg_UnpackKeywords` is removed
Xmader Sep 20, 2024
f6de9c3
fix: `_PyDictViewObject` typedef redefinition with different types on…
Xmader Sep 20, 2024
7d07ee1
fix: `Py_NewRef` API is not available in Python < 3.10
Xmader Sep 20, 2024
3368d3b
Revert "WIP: remove the use of the `_PyArg_CheckPositional` private API"
Xmader Sep 20, 2024
943a431
fix: `_PyArg_CheckPositional` function became an internal API since P…
Xmader Sep 20, 2024
7c451e5
refactor: move the `_PyDictView_New` shim to pyshim.hh
Xmader Sep 20, 2024
d6e4cff
refactor: shim for `_PyArg_CheckPositional`
Xmader Sep 20, 2024
7c03098
Revert "refactor: shim for `_PyArg_CheckPositional`"
Xmader Sep 20, 2024
d3c0f85
fix: `_PyArg_UnpackKeywords` API is removed, use `PyArg_ParseTupleAnd…
Xmader Sep 20, 2024
3706a47
fix: `_PyArg_UnpackKeywords` API is removed, use `PyArg_ParseTupleAnd…
Xmader Sep 20, 2024
92798a8
fix: `_PyThreadState_GetDict(tstate)` API gets removed in Python 3.13
Xmader Sep 20, 2024
ec20bdf
fix: replacement for the `_PyThreadState_GetDict(tstate)` API in Pyth…
Xmader Sep 20, 2024
aea53bd
get rid of the build warnings
Xmader Sep 20, 2024
b2fb169
refactor: `_getLoopOnThread` getting thread dict
Xmader Sep 20, 2024
a5fbef8
feat: to get the running event-loop in Python 3.13 is as simple as `t…
Xmader Sep 20, 2024
29b6bae
feat: keep track of the local variables inside Python `eval`/`exec` t…
Xmader Sep 20, 2024
7f02431
fix: `_PyArg_CheckPositional` may also be when unpacking a tuple
Xmader Sep 23, 2024
a2d9ce8
refactor: use the `_PyDictView_New` function in Python < 3.13
Xmader Sep 23, 2024
623d83e
refactor: keep using `_PyErr_SetKeyError` in Python < 3.13
Xmader Sep 23, 2024
a1cdfb3
refactor: move `_PyErr_SetKeyError` shim to pyshim.hh
Xmader Sep 23, 2024
56b6409
refactor: rename the `PyDictViewObject_new` shim to `PyDictView_New`
Xmader Sep 25, 2024
3f7f722
refactor: add shim for `Py_SET_SIZE`
Xmader Sep 25, 2024
bcfcbf3
refactor: add shim for `PyObject_CallOneArg`
Xmader Oct 1, 2024
68c288e
refactor: `_PyObject_CallOneArg` is already an alias for `PyObject_Ca…
Xmader Oct 1, 2024
23d0576
revert commit 68c288eadaf6fe37f62b09b2107e2363eedc46b1.
Xmader Oct 1, 2024
6bf3d70
refactor: replace `PyObject_CallNoArgs(func)` calls with `PyObject_Ca…
Xmader Oct 1, 2024
c4d9405
refactor: add shim for `PyLong_AsByteArray` to work with different fu…
Xmader Oct 1, 2024
3e2aad2
feat: add back the ability to mutate parent scope variables with `pm.…
Xmader Oct 1, 2024
7ebe149
fix: add back `python.getenv` that was accidentally deleted in `3e2aad2`
Xmader Oct 1, 2024
bb78bbd
CI: use the release version of Python 3.13
Xmader Oct 8, 2024
1a98043
fix the reference count for dicts `test_get_default_not_found`
Xmader Oct 1, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .github/workflows/test-and-publish.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ on:
- '3.10'
- '3.11'
- '3.12'
- '3.13'
build_type:
type: choice
description: 'Choose the build type to use'
Expand Down Expand Up @@ -132,7 +133,7 @@ jobs:
fail-fast: false
matrix:
os: [ 'ubuntu-20.04', 'macos-12', 'macos-14', 'windows-2022', 'pi' ]
python_version: [ '3.8', '3.9', '3.10', '3.11', '3.12' ]
python_version: [ '3.8', '3.9', '3.10', '3.11', '3.12', '3.13' ]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
Expand Down
7 changes: 3 additions & 4 deletions include/JSArrayProxy.hh
Original file line number Diff line number Diff line change
Expand Up @@ -251,11 +251,10 @@ public:
*
* @param self - The JSArrayProxy
* @param args - arguments to the sort method (not used)
* @param nargs - number of arguments to the sort method
* @param kwnames - keyword arguments to the sort method (reverse=True|False, key=keyfunction)
* @param kwargs - keyword arguments to the sort method (reverse=True|False, key=keyfunction)
* @return PyObject* NULL on exception, None otherwise
*/
static PyObject *JSArrayProxy_sort(JSArrayProxy *self, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames);
static PyObject *JSArrayProxy_sort(JSArrayProxy *self, PyObject *args, PyObject *kwargs);

/**
* @brief tp_traverse
Expand Down Expand Up @@ -404,7 +403,7 @@ static PyMethodDef JSArrayProxy_methods[] = {
{"index", (PyCFunction)JSArrayProxyMethodDefinitions::JSArrayProxy_index, METH_FASTCALL, list_index__doc__},
{"count", (PyCFunction)JSArrayProxyMethodDefinitions::JSArrayProxy_count, METH_O, list_count__doc__},
{"reverse", (PyCFunction)JSArrayProxyMethodDefinitions::JSArrayProxy_reverse, METH_NOARGS, list_reverse__doc__},
{"sort", (PyCFunction)JSArrayProxyMethodDefinitions::JSArrayProxy_sort, METH_FASTCALL|METH_KEYWORDS, list_sort__doc__},
{"sort", (PyCFunction)JSArrayProxyMethodDefinitions::JSArrayProxy_sort, METH_VARARGS|METH_KEYWORDS, list_sort__doc__},
{NULL, NULL} /* sentinel */
};

Expand Down
1 change: 1 addition & 0 deletions include/JSObjectItemsProxy.hh
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
#include <jsapi.h>

#include <Python.h>
#include "include/pyshim.hh"


/**
Expand Down
1 change: 1 addition & 0 deletions include/JSObjectKeysProxy.hh
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
#include <jsapi.h>

#include <Python.h>
#include "include/pyshim.hh"


/**
Expand Down
1 change: 1 addition & 0 deletions include/JSObjectValuesProxy.hh
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
#include <jsapi.h>

#include <Python.h>
#include "include/pyshim.hh"


/**
Expand Down
145 changes: 145 additions & 0 deletions include/pyshim.hh
Original file line number Diff line number Diff line change
@@ -0,0 +1,145 @@
/**
* @file pyshim.hh
* @author Tom Tang (xmader@distributive.network)
* @brief Python's C APIs are constantly changing in different versions of CPython.
* PythonMonkey has a wide variety of CPython versions' support. (Currently Python 3.8-3.13)
* This file helps our Python API calls work with different Python versions in the same code base.
* @date 2024-09-20
*
* @copyright Copyright (c) 2024 Distributive Corp.
*
*/

#ifndef PythonMonkey_py_version_shim_
#define PythonMonkey_py_version_shim_

#include <Python.h>

/**
* @brief `_Py_IsFinalizing` becomes a stable API in Python 3.13,
* and renames to `Py_IsFinalizing`
*/
#if PY_VERSION_HEX >= 0x030d0000 // Python version is greater than 3.13
#define Py_IsFinalizing Py_IsFinalizing
#else
#define Py_IsFinalizing _Py_IsFinalizing
#endif

/**
* @brief `_PyDictViewObject` type definition moved from Python's public API
* to the **internal** header file `internal/pycore_dict.h` in Python 3.13.
*
* @see https://github.com/python/cpython/blob/v3.13.0rc1/Include/internal/pycore_dict.h#L64-L72
*/
#if PY_VERSION_HEX >= 0x030d0000 // Python version is greater than 3.13
typedef struct {
PyObject_HEAD
PyDictObject *dv_dict;
} _PyDictViewObject;
#endif

/**
* @brief Shim for `_PyArg_CheckPositional`.
* Since Python 3.13, `_PyArg_CheckPositional` function became an internal API.
* @see Modified from https://github.com/python/cpython/blob/v3.13.0rc1/Python/getargs.c#L2738-L2780
*/
#if PY_VERSION_HEX >= 0x030d0000 // Python version is greater than 3.13
inline int _PyArg_CheckPositional(const char *name, Py_ssize_t nargs, Py_ssize_t min, Py_ssize_t max) {
if (!name) { // _PyArg_CheckPositional may also be when unpacking a tuple
name = "unpacked tuple"; // https://github.com/python/cpython/blob/v3.13.0rc1/Python/getargs.c#L2746
}

if (nargs < min) {
PyErr_Format(
PyExc_TypeError,
"%.200s expected %s%zd argument%s, got %zd",
name, (min == max ? "" : "at least "), min, min == 1 ? "" : "s", nargs);
return 0;
}

if (nargs == 0) {
return 1;
}

if (nargs > max) {
PyErr_Format(
PyExc_TypeError,
"%.200s expected %s%zd argument%s, got %zd",
name, (min == max ? "" : "at most "), max, max == 1 ? "" : "s", nargs);
return 0;
}

return 1;
}
#endif

/**
* @brief Shim for `_PyDictView_New`.
* Since Python 3.13, `_PyDictView_New` function became an internal API.
* @see Modified from https://github.com/python/cpython/blob/v3.13.0rc1/Objects/dictobject.c#L5806-L5827
*/
inline PyObject *PyDictView_New(PyObject *dict, PyTypeObject *type) {
#if PY_VERSION_HEX < 0x030d0000 // Python version is lower than 3.13
return _PyDictView_New(dict, type);
#else
_PyDictViewObject *dv;
dv = PyObject_GC_New(_PyDictViewObject, type);
if (dv == NULL)
return NULL;
Py_INCREF(dict);
dv->dv_dict = (PyDictObject *)dict;
PyObject_GC_Track(dv);
return (PyObject *)dv;
#endif
}

/**
* @brief Shim for `_PyErr_SetKeyError`.
* Since Python 3.13, `_PyErr_SetKeyError` function became an internal API.
*/
inline void PyErr_SetKeyError(PyObject *key) {
// Use the provided API when possible, as `PyErr_SetObject`'s behaviour is more complex than originally thought
// see also: https://github.com/python/cpython/issues/101578
#if PY_VERSION_HEX < 0x030d0000 // Python version is lower than 3.13
return _PyErr_SetKeyError(key);
#else
return PyErr_SetObject(PyExc_KeyError, key);
#endif
}

/**
* @brief Shim for `Py_SET_SIZE`.
* `Py_SET_SIZE` is not available in Python < 3.9
*/
#ifndef Py_SET_SIZE
static inline void _Py_SET_SIZE(PyVarObject *ob, Py_ssize_t size) {
ob->ob_size = size;
}
#define Py_SET_SIZE(ob, size) _Py_SET_SIZE((PyVarObject *)(ob), size)
#endif

/**
* @brief Shim for `PyObject_CallOneArg`.
* `PyObject_CallOneArg` is not available in Python < 3.9
*/
#if PY_VERSION_HEX < 0x03090000 // Python version is less than 3.9
inline PyObject *PyObject_CallOneArg(PyObject *func, PyObject *arg) {
return PyObject_CallFunction(func, "O", arg);
}
#endif

/**
* @brief Shim for `_PyLong_AsByteArray`.
* Python 3.13.0a4 added a new public API `PyLong_AsNativeBytes()` to replace the private `_PyLong_AsByteArray()`.
* But this change also modified the function signature of `_PyLong_AsByteArray()`.
* @see https://github.com/python/cpython/issues/111140
*/
inline int PyLong_AsByteArray(PyLongObject *v, unsigned char *bytes, size_t n, bool little_endian, bool is_signed) {
#if PY_VERSION_HEX >= 0x030d0000 // Python version is 3.13 or higher
return _PyLong_AsByteArray(v, bytes, n, little_endian, is_signed, /*with_exceptions*/ false);
#else
return _PyLong_AsByteArray(v, bytes, n, little_endian, is_signed);
#endif
}

#endif // #ifndef PythonMonkey_py_version_shim_
Xmader marked this conversation as resolved.
Show resolved Hide resolved
Xmader marked this conversation as resolved.
Show resolved Hide resolved
Xmader marked this conversation as resolved.
Show resolved Hide resolved
Loading
Loading