Skip to content

Commit

Permalink
Merge remote-tracking branch 'upstream/main' into tailcall_for_windows
Browse files Browse the repository at this point in the history
  • Loading branch information
Fidget-Spinner committed Feb 13, 2025
2 parents c811b8a + c357d69 commit 904b7ee
Show file tree
Hide file tree
Showing 45 changed files with 1,600 additions and 941 deletions.
1 change: 1 addition & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -621,6 +621,7 @@ jobs:
- build_wasi
- build_windows
- build_windows_msi
- cross-build-linux
- test_hypothesis
- build_asan
- build_tsan
Expand Down
15 changes: 14 additions & 1 deletion .github/workflows/tail-call.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ jobs:
- aarch64-apple-darwin/clang
- x86_64-unknown-linux-gnu/gcc
- aarch64-unknown-linux-gnu/gcc
- free-threading
llvm:
- 19
include:
Expand All @@ -67,6 +68,9 @@ jobs:
- target: aarch64-unknown-linux-gnu/gcc
architecture: aarch64
runner: ubuntu-22.04-arm
- target: free-threading
architecture: x86_64
runner: ubuntu-24.04
steps:
- uses: actions/checkout@v4
with:
Expand Down Expand Up @@ -115,11 +119,20 @@ jobs:
./python.exe -m test --multiprocess 0 --timeout 4500 --verbose2 --verbose3
- name: Native Linux (debug)
if: runner.os == 'Linux'
if: runner.os == 'Linux' && matrix.target != 'free-threading'
run: |
sudo bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)" ./llvm.sh ${{ matrix.llvm }}
export PATH="$(llvm-config-${{ matrix.llvm }} --bindir):$PATH"
CC=clang-19 ./configure --with-tail-call-interp --with-pydebug
make all --jobs 4
./python -m test --multiprocess 0 --timeout 4500 --verbose2 --verbose3
- name: Native Linux with free-threading (release)
if: matrix.target == 'free-threading'
run: |
sudo bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)" ./llvm.sh ${{ matrix.llvm }}
export PATH="$(llvm-config-${{ matrix.llvm }} --bindir):$PATH"
CC=clang-19 ./configure --with-tail-call-interp --disable-gil
make all --jobs 4
./python -m test --multiprocess 0 --timeout 4500 --verbose2 --verbose3
2 changes: 1 addition & 1 deletion Doc/c-api/typeobj.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1830,7 +1830,7 @@ and :c:data:`PyType_Type` effectively act as defaults.)
dictionary, so it is may be more efficient to call :c:func:`PyObject_GetAttr`
when accessing an attribute on the object.

It is an error to set both the :c:macro:`Py_TPFLAGS_MANAGED_WEAKREF` bit and
It is an error to set both the :c:macro:`Py_TPFLAGS_MANAGED_DICT` bit and
:c:member:`~PyTypeObject.tp_dictoffset`.

**Inheritance:**
Expand Down
5 changes: 5 additions & 0 deletions Doc/library/traceback.rst
Original file line number Diff line number Diff line change
Expand Up @@ -257,6 +257,11 @@ Module-Level Functions

.. versionadded:: 3.5

.. versionchanged:: 3.14
This function previously returned a generator that would walk the stack
when first iterated over. The generator returned now is the state of the
stack when ``walk_stack`` is called.

.. function:: walk_tb(tb)

Walk a traceback following :attr:`~traceback.tb_next` yielding the frame and
Expand Down
3 changes: 3 additions & 0 deletions Doc/using/configure.rst
Original file line number Diff line number Diff line change
Expand Up @@ -575,6 +575,9 @@ also be used to improve performance.
.. versionchanged:: 3.12
Use ThinLTO as the default optimization policy on Clang if the compiler accepts the flag.

.. versionchanged:: next
Revert to using full LTO as the default optimization policy on Clang.

.. option:: --enable-bolt

Enable usage of the `BOLT post-link binary optimizer
Expand Down
17 changes: 16 additions & 1 deletion Doc/whatsnew/3.14.rst
Original file line number Diff line number Diff line change
Expand Up @@ -165,7 +165,7 @@ Improved error messages
error message prints the received number of values in more cases than before.
(Contributed by Tushar Sadhwani in :gh:`122239`.)

.. code-block:: pycon
.. code-block:: python
>>> x, y, z = 1, 2, 3, 4
Traceback (most recent call last):
Expand All @@ -175,6 +175,17 @@ Improved error messages
ValueError: too many values to unpack (expected 3, got 4)
* When incorrectly closed strings are detected, the error message suggests
that the string may be intended to be part of the string. (Contributed by
Pablo Galindo in :gh:`88535`.)

.. code-block:: python
>>> "The interesting object "The important object" is very important"
Traceback (most recent call last):
SyntaxError: invalid syntax. Is this intended to be part of the string?
.. _whatsnew314-pep741:

PEP 741: Python Configuration C API
Expand Down Expand Up @@ -1268,6 +1279,10 @@ Build changes
* GNU Autoconf 2.72 is now required to generate :file:`configure`.
(Contributed by Erlend Aasland in :gh:`115765`.)

* CPython now uses Full LTO as the default link time optimization policy
on Clang. This reverts an earlier change in CPython 3.12.
(Contributed by Ken Jin in :gh:`130049`.)

.. _whatsnew314-pep761:

PEP 761: Discontinuation of PGP signatures
Expand Down
3 changes: 3 additions & 0 deletions Grammar/python.gram
Original file line number Diff line number Diff line change
Expand Up @@ -1178,6 +1178,9 @@ invalid_type_param:
}

invalid_expression:
| STRING a=(!STRING expression_without_invalid)+ STRING {
RAISE_SYNTAX_ERROR_KNOWN_RANGE( PyPegen_first_item(a, expr_ty), PyPegen_last_item(a, expr_ty),
"invalid syntax. Is this intended to be part of the string?") }
# !(NAME STRING) is not matched so we don't show this error with some invalid string prefixes like: kf"dsfsdf"
# Soft keywords need to also be ignored because they can be parsed as NAME NAME
| !(NAME STRING | SOFT_KEYWORD) a=disjunction b=expression_without_invalid {
Expand Down
58 changes: 29 additions & 29 deletions Lib/sysconfig/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -666,34 +666,34 @@ def get_platform():

# Set for cross builds explicitly
if "_PYTHON_HOST_PLATFORM" in os.environ:
return os.environ["_PYTHON_HOST_PLATFORM"]

# Try to distinguish various flavours of Unix
osname, host, release, version, machine = os.uname()

# Convert the OS name to lowercase, remove '/' characters, and translate
# spaces (for "Power Macintosh")
osname = osname.lower().replace('/', '')
machine = machine.replace(' ', '_')
machine = machine.replace('/', '-')

if osname[:5] == "linux":
if sys.platform == "android":
osname = "android"
release = get_config_var("ANDROID_API_LEVEL")

# Wheel tags use the ABI names from Android's own tools.
machine = {
"x86_64": "x86_64",
"i686": "x86",
"aarch64": "arm64_v8a",
"armv7l": "armeabi_v7a",
}[machine]
else:
# At least on Linux/Intel, 'machine' is the processor --
# i386, etc.
# XXX what about Alpha, SPARC, etc?
return f"{osname}-{machine}"
osname, _, machine = os.environ["_PYTHON_HOST_PLATFORM"].partition('-')
release = None
else:
# Try to distinguish various flavours of Unix
osname, host, release, version, machine = os.uname()

# Convert the OS name to lowercase, remove '/' characters, and translate
# spaces (for "Power Macintosh")
osname = osname.lower().replace('/', '')
machine = machine.replace(' ', '_')
machine = machine.replace('/', '-')

if osname == "android" or sys.platform == "android":
osname = "android"
release = get_config_var("ANDROID_API_LEVEL")

# Wheel tags use the ABI names from Android's own tools.
machine = {
"x86_64": "x86_64",
"i686": "x86",
"aarch64": "arm64_v8a",
"armv7l": "armeabi_v7a",
}[machine]
elif osname == "linux":
# At least on Linux/Intel, 'machine' is the processor --
# i386, etc.
# XXX what about Alpha, SPARC, etc?
return f"{osname}-{machine}"
elif osname[:5] == "sunos":
if release[0] >= "5": # SunOS 5 == Solaris 2
osname = "solaris"
Expand Down Expand Up @@ -725,7 +725,7 @@ def get_platform():
get_config_vars(),
osname, release, machine)

return f"{osname}-{release}-{machine}"
return '-'.join(map(str, filter(None, (osname, release, machine))))


def get_python_version():
Expand Down
40 changes: 0 additions & 40 deletions Lib/test/test_ast/test_ast.py
Original file line number Diff line number Diff line change
Expand Up @@ -3239,46 +3239,6 @@ def test_folding_tuple(self):

self.assert_ast(code, non_optimized_target, optimized_target)

def test_folding_comparator(self):
code = "1 %s %s1%s"
operators = [("in", ast.In()), ("not in", ast.NotIn())]
braces = [
("[", "]", ast.List, (1,)),
("{", "}", ast.Set, frozenset({1})),
]
for left, right, non_optimized_comparator, optimized_comparator in braces:
for op, node in operators:
non_optimized_target = self.wrap_expr(ast.Compare(
left=ast.Constant(1), ops=[node],
comparators=[non_optimized_comparator(elts=[ast.Constant(1)])]
))
optimized_target = self.wrap_expr(ast.Compare(
left=ast.Constant(1), ops=[node],
comparators=[ast.Constant(value=optimized_comparator)]
))
self.assert_ast(code % (op, left, right), non_optimized_target, optimized_target)

def test_folding_iter(self):
code = "for _ in %s1%s: pass"
braces = [
("[", "]", ast.List, (1,)),
("{", "}", ast.Set, frozenset({1})),
]

for left, right, ast_cls, optimized_iter in braces:
non_optimized_target = self.wrap_statement(ast.For(
target=ast.Name(id="_", ctx=ast.Store()),
iter=ast_cls(elts=[ast.Constant(1)]),
body=[ast.Pass()]
))
optimized_target = self.wrap_statement(ast.For(
target=ast.Name(id="_", ctx=ast.Store()),
iter=ast.Constant(value=optimized_iter),
body=[ast.Pass()]
))

self.assert_ast(code % (left, right), non_optimized_target, optimized_target)

def test_folding_type_param_in_function_def(self):
code = "def foo[%s = 1 + 1](): pass"

Expand Down
128 changes: 128 additions & 0 deletions Lib/test/test_build_details.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
import json
import os
import sys
import sysconfig
import string
import unittest

from test.support import is_android, is_apple_mobile, is_emscripten, is_wasi


class FormatTestsBase:
@property
def contents(self):
"""Install details file contents. Should be overriden by subclasses."""
raise NotImplementedError

@property
def data(self):
"""Parsed install details file data, as a Python object."""
return json.loads(self.contents)

def key(self, name):
"""Helper to fetch subsection entries.
It takes the entry name, allowing the usage of a dot to separate the
different subsection names (eg. specifying 'a.b.c' as the key will
return the value of self.data['a']['b']['c']).
"""
value = self.data
for part in name.split('.'):
value = value[part]
return value

def test_parse(self):
self.data

def test_top_level_container(self):
self.assertIsInstance(self.data, dict)
for key, value in self.data.items():
with self.subTest(key=key):
if key in ('schema_version', 'base_prefix', 'base_interpreter', 'platform'):
self.assertIsInstance(value, str)
elif key in ('language', 'implementation', 'abi', 'suffixes', 'libpython', 'c_api', 'arbitrary_data'):
self.assertIsInstance(value, dict)

def test_base_prefix(self):
self.assertIsInstance(self.key('base_prefix'), str)

def test_base_interpreter(self):
"""Test the base_interpreter entry.
The generic test wants the key to be missing. If your implementation
provides a value for it, you should override this test.
"""
with self.assertRaises(KeyError):
self.key('base_interpreter')

def test_platform(self):
self.assertEqual(self.key('platform'), sysconfig.get_platform())

def test_language_version(self):
allowed_characters = string.digits + string.ascii_letters + '.'
value = self.key('language.version')

self.assertLessEqual(set(value), set(allowed_characters))
self.assertTrue(sys.version.startswith(value))

def test_language_version_info(self):
value = self.key('language.version_info')

self.assertEqual(len(value), sys.version_info.n_fields)
for part_name, part_value in value.items():
with self.subTest(part=part_name):
self.assertEqual(part_value, getattr(sys.version_info, part_name))

def test_implementation(self):
for key, value in self.key('implementation').items():
with self.subTest(part=key):
if key == 'version':
self.assertEqual(len(value), len(sys.implementation.version))
for part_name, part_value in value.items():
self.assertEqual(getattr(sys.implementation.version, part_name), part_value)
else:
self.assertEqual(getattr(sys.implementation, key), value)


needs_installed_python = unittest.skipIf(
sysconfig.is_python_build(),
'This test can only run in an installed Python',
)


@unittest.skipIf(os.name != 'posix', 'Feature only implemented on POSIX right now')
@unittest.skipIf(is_wasi or is_emscripten, 'Feature not available on WebAssembly builds')
class CPythonBuildDetailsTests(unittest.TestCase, FormatTestsBase):
"""Test CPython's install details file implementation."""

@property
def location(self):
if sysconfig.is_python_build():
projectdir = sysconfig.get_config_var('projectbase')
with open(os.path.join(projectdir, 'pybuilddir.txt')) as f:
dirname = os.path.join(projectdir, f.read())
else:
dirname = sysconfig.get_path('stdlib')
return os.path.join(dirname, 'build-details.json')

@property
def contents(self):
with open(self.location, 'r') as f:
return f.read()

@needs_installed_python
def test_location(self):
self.assertTrue(os.path.isfile(self.location))

# Override generic format tests with tests for our specific implemenation.

@needs_installed_python
@unittest.skipIf(is_android or is_apple_mobile, 'Android and iOS run tests via a custom testbed method that changes sys.executable')
def test_base_interpreter(self):
value = self.key('base_interpreter')

self.assertEqual(os.path.realpath(value), os.path.realpath(sys.executable))


if __name__ == '__main__':
unittest.main()
2 changes: 1 addition & 1 deletion Lib/test/test_cmd_line_script.py
Original file line number Diff line number Diff line change
Expand Up @@ -660,7 +660,7 @@ def test_syntaxerror_invalid_escape_sequence_multi_line(self):
self.assertEqual(
stderr.splitlines()[-3:],
[ b' foo = """\\q"""',
b' ^^^^^^^^',
b' ^^',
b'SyntaxError: "\\q" is an invalid escape sequence. '
b'Did you mean "\\\\q"? A raw string is also an option.'
],
Expand Down
2 changes: 1 addition & 1 deletion Lib/test/test_compile.py
Original file line number Diff line number Diff line change
Expand Up @@ -798,7 +798,7 @@ def check_same_constant(const):
f3 = lambda x: x in {("not a name",)}
self.assertIs(f1.__code__.co_consts[0],
f2.__code__.co_consts[0][0])
self.assertIs(next(iter(f3.__code__.co_consts[0])),
self.assertIs(next(iter(f3.__code__.co_consts[1])),
f2.__code__.co_consts[0])

# {0} is converted to a constant frozenset({0}) by the peephole
Expand Down
Loading

0 comments on commit 904b7ee

Please sign in to comment.