Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Subprocess returncode is not detected when running Gunicorn with aiohttp #6130

Closed
1 task done
harrylyx opened this issue Oct 25, 2021 · 0 comments · Fixed by #6134
Closed
1 task done

Subprocess returncode is not detected when running Gunicorn with aiohttp #6130

harrylyx opened this issue Oct 25, 2021 · 0 comments · Fixed by #6134
Labels

Comments

@harrylyx
Copy link
Contributor

Describe the bug

When starting Gunicorn with aiohttp worker(s), if the app uses subprocess to start other processes and captures the output, their returncode is in most cases 0, even if the actual exit code was 1.

To Reproduce

import asyncio
import subprocess
from aiohttp import web
from concurrent.futures import ProcessPoolExecutor

EXECUTOR = ProcessPoolExecutor(4)

def create():
    subprocess_run = subprocess.run(
        "pip install numpy==123",
        shell=True,
        stdout=subprocess.PIPE,
        stderr=subprocess.PIPE,
    )
    print(subprocess_run.returncode)
    print(subprocess_run.stderr.decode("utf-8"))
    return f"{subprocess_run.returncode}, {subprocess_run.stderr.decode('utf-8')}"


async def index(request):
    loop = asyncio.get_event_loop()
    return web.Response(text=await loop.run_in_executor(EXECUTOR, create))


async def my_web_app():
    app = web.Application()
    app.router.add_get("/", index)
    return app


if __name__ == "__main__":
    web.run_app(my_web_app())

Then run it with:

gunicorn start:my_web_app --bind localhost:8080 --worker-class aiohttp.GunicornWebWorker

Expected behavior

The detected returncode should always be 1, as the subprocess always exits with 1.

Logs/tracebacks

❯ gunicorn start:my_web_app --bind localhost:8080 --worker-class aiohttp.GunicornWebWorker
[2021-10-25 16:53:27 +0800] [4190] [INFO] Starting gunicorn 20.1.0
[2021-10-25 16:53:27 +0800] [4190] [INFO] Listening at: http://127.0.0.1:8080 (4190)
[2021-10-25 16:53:27 +0800] [4190] [INFO] Using worker: aiohttp.GunicornWebWorker
[2021-10-25 16:53:27 +0800] [4193] [INFO] Booting worker with pid: 4193
0
ERROR: Could not find a version that satisfies the requirement numpy==123 (from versions: 1.3.0, 1.4.1, 1.5.0, 1.5.1, 1.6.0, 1.6.1, 1.6.2, 1.7.0, 1.7.1, 1.7.2, 1.8.0, 1.8.1, 1.8.2, 1.9.0, 1.9.1, 1.9.2, 1.9.3, 1.10.0.post2, 1.10.1, 1.10.2, 1.10.4, 1.11.0, 1.11.1, 1.11.2, 1.11.3, 1.12.0, 1.12.1, 1.13.0rc1, 1.13.0rc2, 1.13.0, 1.13.1, 1.13.3, 1.14.0rc1, 1.14.0, 1.14.1, 1.14.2, 1.14.3, 1.14.4, 1.14.5, 1.14.6, 1.15.0rc1, 1.15.0rc2, 1.15.0, 1.15.1, 1.15.2, 1.15.3, 1.15.4, 1.16.0rc1, 1.16.0rc2, 1.16.0, 1.16.1, 1.16.2, 1.16.3, 1.16.4, 1.16.5, 1.16.6, 1.17.0rc1, 1.17.0rc2, 1.17.0, 1.17.1, 1.17.2, 1.17.3, 1.17.4, 1.17.5, 1.18.0rc1, 1.18.0, 1.18.1, 1.18.2, 1.18.3, 1.18.4, 1.18.5, 1.19.0rc1, 1.19.0rc2, 1.19.0, 1.19.1, 1.19.2, 1.19.3, 1.19.4, 1.19.5, 1.20.0rc1, 1.20.0rc2, 1.20.0, 1.20.1, 1.20.2, 1.20.3, 1.21.0rc1, 1.21.0rc2, 1.21.0, 1.21.1, 1.21.2, 1.21.3)
ERROR: No matching distribution found for numpy==123

Python Version

$ python --version
Python 3.7.10

aiohttp Version

$ python -m pip show aiohttp
Name: aiohttp
Version: 3.7.4.post0
Summary: Async http client/server framework (asyncio)
Home-page: https://github.com/aio-libs/aiohttp
Author: Nikolay Kim
Author-email: fafhrd91@gmail.com
License: Apache 2
Location: /Users/wj/tools/anaconda3/envs/py37/lib/python3.7/site-packages
Requires: async-timeout, attrs, chardet, multidict, typing-extensions, yarl
Required-by: pytest-aiohttp

multidict Version

$ python -m pip show multidict
Name: multidict
Version: 5.2.0
Summary: multidict implementation
Home-page: https://github.com/aio-libs/multidict
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache 2
Location: /Users/wj/tools/anaconda3/envs/py37/lib/python3.7/site-packages
Requires: 
Required-by: aiohttp, yarl

yarl Version

$ python -m pip show yarl
Name: yarl
Version: 1.7.0
Summary: Yet another URL library
Home-page: https://github.com/aio-libs/yarl/
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache 2
Location: /Users/wj/tools/anaconda3/envs/py37/lib/python3.7/site-packages
Requires: idna, multidict, typing-extensions
Required-by: aiohttp

OS

macOS Big Sur 11.5.2 (20G95)

Related component

Server, Client

Additional context

You can refer to the same question
encode/uvicorn#894
encode/uvicorn#895

I guess bug in https://github.com/aio-libs/aiohttp/blob/master/aiohttp/worker.py

Code of Conduct

  • I agree to follow the aio-libs Code of Conduct
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant