subprocess fails when used as init, vfork() returns EINVAL if PID=1

[socketpair]

BPO	47151
Nosy	@gpshead, @vstinner, @socketpair, @corona10, @miss-islington
PRs	bpo-47151: Fallback to fork when vfork fails in subprocess. #32186 [3.10] bpo-47151: Fallback to fork when vfork fails in subprocess. (GH-32186) #32219

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/gpshead'
closed_at = <Date 2022-03-31.23:12:07.704>
created_at = <Date 2022-03-29.10:54:10.950>
labels = ['extension-modules', 'type-bug', '3.10', '3.11']
title = 'subprocess fails when used as init, vfork() returns EINVAL if PID=1'
updated_at = <Date 2022-03-31.23:12:07.703>
user = 'https://github.com/socketpair'

bugs.python.org fields:

activity = <Date 2022-03-31.23:12:07.703>
actor = 'gregory.p.smith'
assignee = 'gregory.p.smith'
closed = True
closed_date = <Date 2022-03-31.23:12:07.704>
closer = 'gregory.p.smith'
components = ['Extension Modules']
creation = <Date 2022-03-29.10:54:10.950>
creator = 'socketpair'
dependencies = []
files = []
hgrepos = []
issue_num = 47151
keywords = ['patch']
message_count = 7.0
messages = ['416252', '416316', '416334', '416416', '416452', '416455', '416460']
nosy_count = 5.0
nosy_names = ['gregory.p.smith', 'vstinner', 'socketpair', 'corona10', 'miss-islington']
pr_nums = ['32186', '32219']
priority = 'normal'
resolution = 'fixed'
stage = 'commit review'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue47151'
versions = ['Python 3.10', 'Python 3.11']

[socketpair]

The bug introduced here: bpo-35823

If os.gepid() == 1, vfork() does not work and returns EINVAL.

Please add check for the current pid in condition where CPython chooses between fork() and vfork().

[gpshead]

Any possibility that you can test the attached PR as pid 1?

[vstinner]

Any possibility that you can test the attached PR as pid 1?

Python should be built as a static binary and can be used as the init process on the kernel command line, no? I'm not sure that "static binary" is a requirement, since I'm commonly using init=/usr/bin/bash to fix a broken Linux, and on my Fedora, this program is "dynamically linked".

Who uses Python as pid 1? I'm now curious :-)

[socketpair]

Well.

1. We use Python as PID 1. In PID namespace.
2. Further investigation gave information that vfork()+pid=1 actually WORKS. The problem is connected with another weird thing in kernel (undocumented unshare() flags).
3. The logic to try fork() after vfork() has failed is NICE. Please merge. I would also write a message to stderr in case vfork() failed.

P.S. Program may or may not be static in order to work as PID=1.

[gpshead]

Thanks. I had wondered if this was really a pid=1 restriction or not, but I could definitely imagine kernel scenarios where vfork is simply forbidden regardless. There are a variety of policy mechanisms in kernels, mainline Linux or not, that _could_ do that kind of thing.

As much as I'd like to expose that the fallback happened, emitting to stderr isn't friendly and using warnings from this code is complicated so I'm inclined to keep the silent fallback on failure simple as is until someone can demonstrate of it causing a practical problem.

Outside of unusual configurations, if this were ever happening when it is not expected, people observing low subprocess performance could strace and witness the vfork syscall failure.

I'll merge once our CI is happy.

[gpshead]

New changeset 4a08c4c by Gregory P. Smith in branch 'main':
bpo-47151: Fallback to fork when vfork fails in subprocess. (GH-32186)
4a08c4c

[miss-islington]

New changeset 9ed179b by Miss Islington (bot) in branch '3.10':
bpo-47151: Fallback to fork when vfork fails in subprocess. (GH-32186)
9ed179b