posix.sched_{get|set}affinity(-1) no longer returns ProcessLookup causing test failures on FreeBSD

[tiran]

BPO	47205
Nosy	@tiran, @koobs, @miss-islington, @Fidget-Spinner
PRs	bpo-47205: Skip error check of sched_get/setaffinity on FreeBSD (GH-32285) #32285 [3.10] bpo-47205: Skip error check of sched_get/setaffinity on FreeBSD (GH-32285) #32286 [3.9] bpo-47205: Skip error check of sched_get/setaffinity on FreeBSD (GH-32285) #32287

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2022-04-03.12:36:46.409>
labels = ['type-bug', 'tests', '3.9', '3.10', '3.11']
title = 'posix.sched_{get|set}affinity(-1) no longer returns ProcessLookup causing test failures on FreeBSD'
updated_at = <Date 2022-04-05.23:07:44.290>
user = 'https://github.com/tiran'

bugs.python.org fields:

activity = <Date 2022-04-05.23:07:44.290>
actor = 'koobs'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Tests', 'FreeBSD']
creation = <Date 2022-04-03.12:36:46.409>
creator = 'christian.heimes'
dependencies = []
files = []
hgrepos = []
issue_num = 47205
keywords = ['patch']
message_count = 9.0
messages = ['416616', '416620', '416621', '416623', '416624', '416627', '416640', '416647', '416830']
nosy_count = 4.0
nosy_names = ['christian.heimes', 'koobs', 'miss-islington', 'kj']
pr_nums = ['32285', '32286', '32287']
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue47205'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

[tiran]

Two test cases have been failing on FreeBSD buildbot for a while. Let's skip the error check on FreeBSD for now.

0:27:24 load avg: 1.97 Re-running test_posix in verbose mode (matching: test_sched_getaffinity, test_sched_setaffinity)
test_sched_getaffinity (test.test_posix.PosixTester.test_sched_getaffinity) ... FAIL
test_sched_setaffinity (test.test_posix.PosixTester.test_sched_setaffinity) ... FAIL
======================================================================
FAIL: test_sched_getaffinity (test.test_posix.PosixTester.test_sched_getaffinity)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/usr/home/buildbot/python/pull_request.koobs-freebsd-564d/build/Lib/test/test_posix.py", line 1197, in test_sched_getaffinity
    self.assertRaises(OSError, posix.sched_getaffinity, -1)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: OSError not raised by sched_getaffinity

======================================================================
FAIL: test_sched_setaffinity (test.test_posix.PosixTester.test_sched_setaffinity)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/usr/home/buildbot/python/pull_request.koobs-freebsd-564d/build/Lib/test/test_posix.py", line 1215, in test_sched_setaffinity
    self.assertRaises(OSError, posix.sched_setaffinity, -1, mask)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: OSError not raised by sched_setaffinity

---

Ran 2 tests in 0.015s

[tiran]

Koobs, please take a look.

[Fidget-Spinner]

FWIW, seems like the buildbot started failing at either commit cb495a1 or 7088120. But I have no clue how commits changing the Windows build config can lead to test_posix failing.

[tiran]

Koob's build bot is a FreeBSD 14 host. FreeBSD 14 is under development and has not reached stable yet. I bet that the issue was caused by a recent system and Kernel upgrade.

[tiran]

New changeset b82cdd1 by Christian Heimes in branch 'main':
bpo-47205: Skip error check of sched_get/setaffinity on FreeBSD (GH-32285)
b82cdd1

[miss-islington]

New changeset 490ccbd by Miss Islington (bot) in branch '3.9':
bpo-47205: Skip error check of sched_get/setaffinity on FreeBSD (GH-32285)
490ccbd

[miss-islington]

New changeset 470dfe2 by Miss Islington (bot) in branch '3.10':
bpo-47205: Skip error check of sched_get/setaffinity on FreeBSD (GH-32285)
470dfe2

[koobs]

Investigating

[koobs]

From one of our base/kernel developers:

--------------------------------------------------------------

koobs wrote:

I don't grok the system call semantics, but it appears the issue is calls
with the '-1' argument:

mask = posix.sched_getaffinity(0)
...
self.assertRaises(OSError, posix.sched_getaffinity, -1)
...
self.assertRaises(OSError, posix.sched_setaffinity, -1, mask)

This line was added 10 years ago in:

8486987

It wants an OSError [1] and presumably was getting it before recent
failures, but isn't anymore:

[1] https://docs.python.org/3/library/exceptions.html#OSError

So this is the difference:

Python 3.11.0a6+ (heads/main:38ae5b8c0c, Apr  5 2022, 03:27:23) [GCC 11.2.0] on freebsd13
Type "help", "copyright", "credits" or "license" for more information.
>>> import posix
>>> mask = posix.sched_getaffinity(-1)
>>> mask
{0, 1, 2, 3, 4, 5, 6, 7}

Python 3.10.4 (main, Mar 25 2022, 00:00:00) [GCC 11.2.1 20220127 (Red Hat 11.2.1-9)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import posix
>>> mask = posix.sched_getaffinity(-1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ProcessLookupError: [Errno 3] No such process

FreeBSD always accepted -1 as denoting the current process, while Linux
does not. In fact, Linux uses 0 as current process pid alias.

For FreeBSD, both -1 and 0 works as getpid(), and I do not see it right
to break our traditional API conventions. I believe the test should be
fixed.

--------------------------------------------------------------

I am not sure what changed between 3.10 and main that changed/removed
ProcessLookupError as the return for posix.sched_{get|set}affinity
with -1 as the argument, but that appears to be the cause of the current test failure

[vstinner]

FreeBSD always accepted -1 as denoting the current process, while Linux does not. In fact, Linux uses 0 as current process pid alias.

So the current test is correct, I close the issue.