OSError: [Errno 9] Bad file descriptor #1877

leond08 · 2018-09-10T12:41:06Z

I'm having this issue in raspbian image

[2018-09-10 20:40:11 +0800] [21421] [CRITICAL] WORKER TIMEOUT (pid:21699)
[2018-09-10 20:40:11 +0800] [21699] [ERROR] Socket error processing request.
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/gunicorn/workers/async.py", line 62, in handle
six.reraise(*sys.exc_info())
File "/usr/lib/python3/dist-packages/gunicorn/six.py", line 625, in reraise
raise value
File "/usr/lib/python3/dist-packages/gunicorn/workers/async.py", line 35, in handle
listener_name = listener.getsockname()
OSError: [Errno 9] Bad file descriptor

benoitc · 2018-09-10T13:15:45Z

@leond08 Thanks for the ticket!

In order to understand how it is happening can you provide a little more info?

wich version of Gunicorn are you testing
which worker are you using
when does this happen?

leond08 · 2018-09-12T04:26:21Z

I'm using gunicorn3 latest version
I'm using eventlet and gevent for this
i'm running my flask application - Flask-SocketIO

I start my background task after a user clicked the button
My background task function is to listen to an event,
after clicking the "DONE" button the background task must stop
then send an emit message to all the users

yunstanford · 2018-09-14T04:19:29Z

Had the same issue with aiohttp + gunicorn, observe the same message each time when ctrl + c.

[INFO] Error while closing socket [Errno 9] Bad file descriptor

benoitc · 2018-10-09T05:47:02Z

I don't reproduce it. I suspect your application is closing some fds which create the issue above.

dpzx · 2019-01-10T16:12:25Z

We are experiencing the same issue, only thing is it's only happening on 1 out of 8 containers running in docker swarm.

georgexsh · 2020-04-01T12:38:56Z

We have experienced the same issue with 1 out of 9 containers, it looks like related to docker and python3 and gevent.

zgoda · 2020-05-23T19:54:22Z

gunicorn 20.0.4 + aiohttp 3.6.2

Gunicorn is running as dev server:

gunicorn --reload app:make_app --bind localhost:5000 --worker-class aiohttp.GunicornWebWorker --workers 2 --access-logfile -

Almost every Ctrl+C ends with

^C[2020-05-23 21:49:50 +0200] [38524] [INFO] Handling signal: int
Exception ignored when trying to write to the signal wakeup fd:
Exception ignored when trying to write to the signal wakeup fd:
Traceback (most recent call last):
  File "/usr/lib/python3.8/asyncio/unix_events.py", line 42, in _sighandler_noop
Traceback (most recent call last):
  File "/usr/lib/python3.8/asyncio/unix_events.py", line 42, in _sighandler_noop
    def _sighandler_noop(signum, frame):
    def _sighandler_noop(signum, frame):
OSError: [Errno 9] Bad file descriptor
OSError: [Errno 9] Bad file descriptor
[2020-05-23 21:49:50 +0200] [38526] [INFO] Worker exiting (pid: 38526)
[2020-05-23 21:49:50 +0200] [38528] [INFO] Worker exiting (pid: 38528)
[2020-05-23 21:49:50 +0200] [38524] [INFO] Shutting down: Master

It does not matter if application handled any request or not.

zgoda · 2020-05-26T11:26:26Z

With Sanic 20.3.0:

^C[2020-05-26 13:24:55 +0200] [27706] [INFO] Handling signal: int
[2020-05-26 13:24:55 +0200] [27769] [INFO] Stopping server: 27769, connections: 0
[2020-05-26 13:24:55 +0200] [27769] [INFO] Error while closing socket [Errno 9] Bad file descriptor
[2020-05-26 13:24:55 +0200] [27769] [INFO] Worker exiting (pid: 27769)
[2020-05-26 13:24:55 +0200] [27771] [INFO] Stopping server: 27771, connections: 0
[2020-05-26 13:24:55 +0200] [27771] [INFO] Error while closing socket [Errno 9] Bad file descriptor
[2020-05-26 13:24:55 +0200] [27771] [INFO] Worker exiting (pid: 27771)
[2020-05-26 13:24:55 +0200] [27706] [INFO] Shutting down: Master

Soberia · 2020-05-26T23:52:08Z

Same here with Gunicorn 20.0.4 + Uvicorn 0.11.5 worker class on every Ctrl+C

INFO:     [12621] [gunicorn.error] Handling signal: int
INFO:     [12635] [gunicorn.error] Error while closing socket [Errno 9] Bad file descriptor
INFO:     [12634] [gunicorn.error] Error while closing socket [Errno 9] Bad file descriptor
INFO:     [12635] [gunicorn.error] Worker exiting (pid: 12635)
INFO:     [12634] [gunicorn.error] Worker exiting (pid: 12634)
INFO:     [12621] [gunicorn.error] Shutting down: Master

benoitc · 2020-05-27T06:17:09Z

any example of application? Also which version of Python are we talking about?

zgoda · 2020-05-27T08:16:07Z

Ubuntu 20.04, system provided Python 3.8.2 in virtualenv

Example application: https://github.com/zgoda/newsloop-server/tree/d603a1c10c9e8be3d998f62ccc55dd73f4677115 (with aiohttp) or https://github.com/zgoda/newsloop-server/tree/b2a8a7f09fa9848d0384b51a3f6c0bb037fb459e (with Sanic). Exact gunicorn invocation in my earlier comment.

Differences in output between aiohttp and Sanic make me suspicious that there's something wrong involving workers.

Ronserruya · 2020-05-27T13:46:56Z

Same issue, python 3.8.0
sanic 19.12.2
gunicorn 20.0.4

Edit: this happens when i run locally on my Mac, but not when running inside a Linux docker, might help you

strelkov-a · 2020-06-01T08:30:08Z

Hello,
I guess this issue #2064 has the same reasons.
We have almost the same errors like in issue, but we use gunicorn - 19.9.0

fgimian · 2020-09-12T01:58:15Z

Im experiencing this too, FastAPI + the latest Gunicorn and uvicorn workers with Python 3.8.5

As soon as I stop using uvicorn (i.e. remove this line from my gunicorn config):

worker_class = "uvicorn.workers.UvicornWorker"

The errors dissappear.

As described above, this happens when stopping Gunicorn with Ctrl+C or sending a graceful kill signal to the PID.

[2020-09-12 11:56:37 +1000] [100390] [INFO] Starting gunicorn 20.0.4
[2020-09-12 11:56:37 +1000] [100390] [INFO] Listening at: http://0.0.0.0:6000 (100390)
[2020-09-12 11:56:37 +1000] [100390] [INFO] Using worker: uvicorn.workers.UvicornWorker
[2020-09-12 11:56:37 +1000] [100392] [INFO] Booting worker with pid: 100392
[2020-09-12 11:56:38 +1000] [100392] [INFO] Started server process [100392]
[2020-09-12 11:56:38 +1000] [100392] [INFO] Waiting for application startup.
[2020-09-12 11:56:38 +1000] [100392] [INFO] Application startup complete.
[2020-09-12 11:56:48 +1000] [100390] [INFO] Handling signal: term
[2020-09-12 11:56:48 +1000] [100392] [INFO] Shutting down
[2020-09-12 11:56:48 +1000] [100392] [INFO] Error while closing socket [Errno 9] Bad file descriptor
[2020-09-12 11:56:48 +1000] [100392] [INFO] Waiting for application shutdown.
[2020-09-12 11:56:48 +1000] [100392] [INFO] Application shutdown complete.
[2020-09-12 11:56:48 +1000] [100392] [INFO] Finished server process [100392]
[2020-09-12 11:56:48 +1000] [100392] [INFO] Worker exiting (pid: 100392)
[2020-09-12 11:56:48 +1000] [100390] [INFO] Shutting down: Master

fgimian · 2020-09-12T02:20:57Z

Here's an exact replication of the issue:

[fots@workstation testing]$ python3.8 -V
Python 3.8.5
[fots@workstation testing]$ python3.8 -m venv ~/.virtualenv/testing
[fots@workstation testing]$ source ~/.virtualenv/testing/bin/activate
(testing) [fots@workstation testing]$ pip install fastapi gunicorn uvicorn
Collecting fastapi
  Using cached fastapi-0.61.1-py3-none-any.whl (48 kB)
Collecting gunicorn
  Using cached gunicorn-20.0.4-py2.py3-none-any.whl (77 kB)
Collecting uvicorn
  Using cached uvicorn-0.11.8-py3-none-any.whl (43 kB)
Collecting pydantic<2.0.0,>=1.0.0
  Using cached pydantic-1.6.1-cp38-cp38-manylinux2014_x86_64.whl (11.5 MB)
Collecting starlette==0.13.6
  Using cached starlette-0.13.6-py3-none-any.whl (59 kB)
Requirement already satisfied: setuptools>=3.0 in /home/fots/.virtualenv/testing/lib/python3.8/site-packages (from gunicorn) (47.1.0)
Collecting h11<0.10,>=0.8
  Using cached h11-0.9.0-py2.py3-none-any.whl (53 kB)
Collecting websockets==8.*
  Using cached websockets-8.1-cp38-cp38-manylinux2010_x86_64.whl (78 kB)
Collecting httptools==0.1.*; sys_platform != "win32" and sys_platform != "cygwin" and platform_python_implementation != "PyPy"
  Using cached httptools-0.1.1-cp38-cp38-manylinux1_x86_64.whl (227 kB)
Collecting uvloop>=0.14.0; sys_platform != "win32" and sys_platform != "cygwin" and platform_python_implementation != "PyPy"
  Using cached uvloop-0.14.0-cp38-cp38-manylinux2010_x86_64.whl (4.7 MB)
Collecting click==7.*
  Using cached click-7.1.2-py2.py3-none-any.whl (82 kB)
Installing collected packages: pydantic, starlette, fastapi, gunicorn, h11, websockets, httptools, uvloop, click, uvicorn
Successfully installed click-7.1.2 fastapi-0.61.1 gunicorn-20.0.4 h11-0.9.0 httptools-0.1.1 pydantic-1.6.1 starlette-0.13.6 uvicorn-0.11.8 uvloop-0.14.0 websockets-8.1
WARNING: You are using pip version 20.1.1; however, version 20.2.3 is available.
You should consider upgrading via the '/home/fots/.virtualenv/testing/bin/python3.8 -m pip install --upgrade pip' command.
(testing) [fots@workstation testing]$ ls -l
total 4
-rw-rw-r-- 1 fots fots 117 Sep 12 12:13 main.py
(testing) [fots@workstation testing]$ cat main.py
from fastapi import FastAPI

app = FastAPI()


@app.get("/")
async def root():
    return {"message": "Hello World"}
(testing) [fots@workstation testing]$ gunicorn -k uvicorn.workers.UvicornWorker main:app
[2020-09-12 12:19:05 +1000] [105788] [INFO] Starting gunicorn 20.0.4
[2020-09-12 12:19:05 +1000] [105788] [INFO] Listening at: http://127.0.0.1:8000 (105788)
[2020-09-12 12:19:05 +1000] [105788] [INFO] Using worker: uvicorn.workers.UvicornWorker
[2020-09-12 12:19:05 +1000] [105790] [INFO] Booting worker with pid: 105790
[2020-09-12 12:19:05 +1000] [105790] [INFO] Started server process [105790]
[2020-09-12 12:19:05 +1000] [105790] [INFO] Waiting for application startup.
[2020-09-12 12:19:05 +1000] [105790] [INFO] Application startup complete.
^C[2020-09-12 12:19:06 +1000] [105788] [INFO] Handling signal: int
[2020-09-12 12:19:06 +1000] [105790] [INFO] Shutting down
[2020-09-12 12:19:06 +1000] [105790] [INFO] Error while closing socket [Errno 9] Bad file descriptor
[2020-09-12 12:19:06 +1000] [105790] [INFO] Waiting for application shutdown.
[2020-09-12 12:19:06 +1000] [105790] [INFO] Application shutdown complete.
[2020-09-12 12:19:06 +1000] [105790] [INFO] Finished server process [105790]
[2020-09-12 12:19:06 +1000] [105790] [INFO] Worker exiting (pid: 105790)
[2020-09-12 12:19:07 +1000] [105788] [INFO] Shutting down: Master

And here's the pip freeze output:

click==7.1.2
fastapi==0.61.1
gunicorn==20.0.4
h11==0.9.0
httptools==0.1.1
pydantic==1.6.1
starlette==0.13.6
uvicorn==0.11.8
uvloop==0.14.0
websockets==8.1

I attempted to install uvicorn and gunicorn from GitHub (master branch) to ensure I got the very latest fixes but the problem persisted.

Hope this helps
Fotis

gg349 · 2020-12-06T16:25:35Z

I am able to reproduce this log error with a setup with just gunicorn and uvicorn. This error message starts occurring with uvicorn==0.11.4 and not the previous version 0.11.3 (both on OSx and in a Linux container). This is consistent with the reports above with uvicorn, where the versions reported are alwasy greater than 0.11.4. Evidence at the end
The commit responsible for this error is this one. The problem lies with these few lines of the commit just mentioned. The commit changes only the order of two blocks of code. If I revert that change of order, the log error go away, while still passing the test suite of uvicorn
the same log error happens if one either: uses starlette and fastapi on top of the stack gunicorn+uvicorn as reported above; - runs the latest uvicorn version 12.X instead of 0.11.4; - runs gunicorn with more than just one uvicorn worker

Evidence. In a new folder on osx run the attached script test.sh (tested on osx). For testing in the Linux container, save both the script and the Dockerfile and then read the header of the Dockerfile. I also attach the log of the script.

@benoitc, what do you think of this commit in uvicorn that seems to introduce the bug? The problem seems to be at the interface between gunicorn and uvicorn. Can you comment on the ordering of the 2 blocks of code changed in the commit mentioned above in uvicorn? This may help finding out why this happens with the other cases too. So far this has been reported also with aiohttp, gevent, Flask-SocketIO sanic. I attach also the log of the script for ease.

log_test.log

file test.sh

#!/bin/bash
python3 -m venv venv
source venv/bin/activate
pip install gunicorn==20.0.04 uvicorn==0.11.4
# create simple uvicorn app
printf "async def app(scope, receive, send):\n    await send()\n" > example.py
# spin up gunicorn with 1 uvicorn worker, and then send TERM signal to gunicorn
gunicorn example:app -w 1 -k uvicorn.workers.UvicornWorker &
sleep 5 && pkill -f gunicorn
sleep 1
# you will see 1 log entry like this one:
# [XX] [YY] [INFO] Error while closing socket [Errno 9] Bad file descriptor

printf "\n\n[INFO] if you instead bump down uvicorn's version from 11.4 to 11.3 [Errno 9] goes away:\n\n"
pip install uvicorn==0.11.3
gunicorn example:app -w 1 -k uvicorn.workers.UvicornWorker &
sleep 5 && pkill -f gunicorn

file Dockerfile

# run with:
# docker run -it $(docker build -q .)
FROM python:3.8
COPY test.sh .
RUN chmod +x /test.sh
CMD /test.sh

ChrisXiaoShu · 2021-01-13T03:11:01Z

I had exactly same issue. Here is my case.

Brief : I'm trying to establish a test application for Django dwebsocket with gunicorn. When I trying to use websocket_client to test the result, after close the websocket this error happen everytime.

Environment :
Docker image : python:3.7
python version : python3.7.6
gunicorn : version = 20.0.4, work = gevent
Django version : Django==2.2
dwebsocket version : 0.5.12

Code:

view.py

from dwebsocket import accept_websocket

@accept_websocket
def my_ws(request):
    if request.is_websocket():
        ws = request.websocket
        while True:
            msg = ws.wait(timeout=15)
            if msg is None:
                print('get None message')
                break
            else:
                msg = b'echo :' + msg
                ws.send(msg)
                print('send ws seccess')
        print('websocket close')

urls.py

from websocketInfo.views import  my_ws
from django.conf.urls import url

urlpatterns = [
    url(r'my_ws/$', my_ws, name='my_ws')
]

websocket_client

from websocket import create_connection
ws = create_connection("ws://127.0.0.1:8080/my_ws/")
print("Sending 'Hello, World'...")
ws.send("Hello, World")
print("Receiving...")
result = ws.recv()
print(result)
ws.close()
print('ws close')

commad to run gunicorn server

gunicorn MyWebsocket.wsgi -b 0.0.0.0:8000 -w 3 -k gevent

error output

send ws seccess
get None message
websocket close
[2021-01-13 02:43:56 +0000] [101] [ERROR] Socket error processing request.
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/base_async.py", line 65, in handle
    util.reraise(*sys.exc_info())
  File "/usr/local/lib/python3.7/site-packages/gunicorn/util.py", line 625, in reraise
    raise value
  File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/base_async.py", line 55, in handle
    self.handle_request(listener_name, req, client, addr)
  File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/ggevent.py", line 143, in handle_request
    super().handle_request(listener_name, req, sock, addr)
  File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/base_async.py", line 128, in handle_request
    util.reraise(*sys.exc_info())
  File "/usr/local/lib/python3.7/site-packages/gunicorn/util.py", line 625, in reraise
    raise value
  File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/base_async.py", line 114, in handle_request
    resp.write(item)
  File "/usr/local/lib/python3.7/site-packages/gunicorn/http/wsgi.py", line 326, in write
    self.send_headers()
  File "/usr/local/lib/python3.7/site-packages/gunicorn/http/wsgi.py", line 322, in send_headers
    util.write(self.sock, util.to_bytestring(header_str, "latin-1"))
  File "/usr/local/lib/python3.7/site-packages/gunicorn/util.py", line 286, in write
    sock.sendall(data)
  File "/usr/local/lib/python3.7/site-packages/gevent/_socket3.py", line 523, in sendall
    return _socketcommon._sendall(self, data_memory, flags)
  File "/usr/local/lib/python3.7/site-packages/gevent/_socketcommon.py", line 367, in _sendall
    chunk_size = max(socket.getsockopt(SOL_SOCKET, SO_SNDBUF), 1024 * 1024)
  File "/usr/local/lib/python3.7/site-packages/gevent/_socket3.py", line 156, in __getattr__
    return getattr(self._sock, name)
  File "/usr/local/lib/python3.7/site-packages/gevent/_socket3.py", line 66, in _dummy
    raise OSError(EBADF, 'Bad file descriptor')
OSError: [Errno 9] Bad file descriptor

jamadden · 2021-01-13T11:19:12Z

@ChrisXiaoShu The stack trace you posted tells us that this specific socket object has been explicitly closed at the Python level (that's when gevent uses its _dummy to generate the same exceptions that the operating system does). This means some part of your application stack is closing the socket before returning the response to let gunicorn handle it; at the point the error occurs, gunicorn hadn't even sent HTTP response headers yet.

mawoka-myblock · 2021-06-07T14:21:58Z

Kind of the same problem in my case, with the difference that this error happens without doing anything. Sometimes after 5min, sometimes after 2hrs...

benoitc/gunicorn#1877 (comment) Signed-off-by: Cédric Foellmi <cedric@onekiloparsec.dev>

hardenchant · 2021-10-26T16:21:24Z

had same issue on that setup:

Docker image : python:3.7.10-slim-stretch
python version : python3.7.10
gunicorn : gunicorn = {version = "20.1.0", extras = ["eventlet"]}
Django version : Django = "2.2.18"

nazrulworld · 2021-11-29T14:46:57Z

I don’t know if the issue is still unresolved! I had a similar issue (FastAPI + uvicorn worker + gunicorn + supervisor). Gunicorn was unable to stop workers and I saw the above error in the log.

I give the unique process name in the gunicorn settings proc_name and it's working!!

jules-ch · 2021-12-03T13:15:08Z

Last time I had this error It was because of a print statement, it raises this error & kill the worker.

Removing the print statement & replacing it with logging works.
Hope it helps

mindflayer · 2022-01-10T13:47:48Z

I was trying to tackle this issue, since it's happening to us every time Gunicorn terminates an uvicorn worker which reached the max_requests limit.
At some point of my research, I ended up reading https://bugs.python.org/issue29343 and I realised that maybe we should just forget about logging.info() in case of a socket.EBADF (Bad file descriptor). It's not an issue per se and some core devs are even proposing to remove this failure, just like it was before.

mindflayer · 2022-01-10T13:57:29Z

A second approach would be logging it as DEBUG instead of INFO. Let me know if you like any of the above, I could even submit a PR for it.

nottrobin · 2022-01-21T16:17:52Z

Similar problem here, k8s, gunicorn with gevent, flask:


OSError: [Errno 9] Bad file descriptor
2022-01-21 14:10:31.189Z ERROR gunicorn.error "Socket error processing request." service=ubuntu.com pid=12 errno=EBADF strerror="Bad file descriptor"
Traceback (most recent call last):
  File "/root/.local/lib/python3.8/site-packages/gunicorn/workers/base_async.py", line 65, in handle
    util.reraise(*sys.exc_info())
  File "/root/.local/lib/python3.8/site-packages/gunicorn/util.py", line 626, in reraise
    raise value
  File "/root/.local/lib/python3.8/site-packages/gunicorn/workers/base_async.py", line 38, in handle
    listener_name = listener.getsockname()
  File "/root/.local/lib/python3.8/site-packages/gevent/_socketcommon.py", line 557, in getsockname
    return self._sock.getsockname()
OSError: [Errno 9] Bad file descriptor
2022-01-21 14:10:31.190Z ERROR gunicorn.error "Socket error processing request." service=ubuntu.com pid=12 errno=EBADF strerror="Bad file descriptor"
Traceback (most recent call last):
  File "/root/.local/lib/python3.8/site-packages/gunicorn/workers/base_async.py", line 65, in handle
    util.reraise(*sys.exc_info())
  File "/root/.local/lib/python3.8/site-packages/gunicorn/util.py", line 626, in reraise
    raise value
  File "/root/.local/lib/python3.8/site-packages/gunicorn/workers/base_async.py", line 38, in handle
    listener_name = listener.getsockname()
  File "/root/.local/lib/python3.8/site-packages/gevent/_socketcommon.py", line 557, in getsockname
    return self._sock.getsockname()
OSError: [Errno 9] Bad file descriptor
2022-01-21 14:10:31.191Z ERROR gunicorn.error "Socket error processing request." service=ubuntu.com pid=12 errno=EBADF strerror="Bad file descriptor"
Traceback (most recent call last):
  File "/root/.local/lib/python3.8/site-packages/gunicorn/workers/base_async.py", line 65, in handle
    util.reraise(*sys.exc_info())
  File "/root/.local/lib/python3.8/site-packages/gunicorn/util.py", line 626, in reraise
    raise value
  File "/root/.local/lib/python3.8/site-packages/gunicorn/workers/base_async.py", line 38, in handle
    listener_name = listener.getsockname()
  File "/root/.local/lib/python3.8/site-packages/gevent/_socketcommon.py", line 557, in getsockname
    return self._sock.getsockname()
OSError: [Errno 9] Bad file descriptor
2022-01-21 14:10:31.191Z ERROR gunicorn.error "Socket error processing request." service=ubuntu.com pid=12 errno=EBADF strerror="Bad file descriptor"
Traceback (most recent call last):
  File "/root/.local/lib/python3.8/site-packages/gunicorn/workers/base_async.py", line 65, in handle
    util.reraise(*sys.exc_info())
  File "/root/.local/lib/python3.8/site-packages/gunicorn/util.py", line 626, in reraise
    raise value
  File "/root/.local/lib/python3.8/site-packages/gunicorn/workers/base_async.py", line 38, in handle
    listener_name = listener.getsockname()
  File "/root/.local/lib/python3.8/site-packages/gevent/_socketcommon.py", line 557, in getsockname
    return self._sock.getsockname()
OSError: [Errno 9] Bad file descriptor
2022-01-21 14:10:31.195Z ERROR flask.app "Exception on /security/cve [GET]" service=ubuntu.com pid=12 request_id=e84ae25c8bc81046044c52901f176248
Traceback (most recent call last):
  File "/root/.local/lib/python3.8/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/root/.local/lib/python3.8/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/root/.local/lib/python3.8/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/root/.local/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/root/.local/lib/python3.8/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/root/.local/lib/python3.8/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/srv/webapp/security/views.py", line 735, in cve_index
    return flask.render_template(
  File "/root/.local/lib/python3.8/site-packages/flask/templating.py", line 136, in render_template
    ctx.app.update_template_context(context)
  File "/root/.local/lib/python3.8/site-packages/flask/app.py", line 838, in update_template_context
    context.update(func())
  File "/srv/webapp/app.py", line 272, in context
    "releases": releases(),
  File "/srv/webapp/context.py", line 39, in releases
    return yaml.load(releases, Loader=yaml.FullLoader)
  File "/root/.local/lib/python3.8/site-packages/yaml/__init__.py", line 114, in load
    return loader.get_single_data()
  File "/root/.local/lib/python3.8/site-packages/yaml/constructor.py", line 49, in get_single_data
    node = self.get_single_node()
  File "/root/.local/lib/python3.8/site-packages/yaml/composer.py", line 36, in get_single_node
    document = self.compose_document()
  File "/root/.local/lib/python3.8/site-packages/yaml/composer.py", line 55, in compose_document
    node = self.compose_node(None, None)
  File "/root/.local/lib/python3.8/site-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/root/.local/lib/python3.8/site-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/root/.local/lib/python3.8/site-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/root/.local/lib/python3.8/site-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/root/.local/lib/python3.8/site-packages/yaml/composer.py", line 71, in compose_node
    event = self.peek_event()
  File "/root/.local/lib/python3.8/site-packages/yaml/parser.py", line 107, in peek_event
    def peek_event(self):
  File "/root/.local/lib/python3.8/site-packages/gunicorn/workers/base.py", line 202, in handle_abort
    self.cfg.worker_abort(self)
  File "/root/.local/lib/python3.8/site-packages/talisker/gunicorn.py", line 123, in gunicorn_worker_abort
    raise talisker.wsgi.RequestTimeout(
talisker.wsgi.RequestTimeout: gunicorn worker timeout (pid: 12)
2022-01-21 14:10:31.997Z INFO talisker.wsgi "GET /security/cve?" method=GET path=/security/cve qs="package=libcurl4" status=500 view=webapp.security.views.cve_index duration_ms=9994.17 ip=91.189.88.181 proto=HTTP/1.1 length=9295 forwarded="2600:1900:2000:d9::2, 91.189.88.181" ua=Go-http-client/1.1 service=ubuntu.com pid=12 request_id=e84ae25c8bc81046044c52901f176248
2022-01-21 14:10:34.043Z INFO gunicorn.error "Worker exiting (pid: 12)" service=ubuntu.com pid=12
Sentry is attempting to send 126 pending error messages
Waiting up to 10 seconds
Press Ctrl-C to quit
2022-01-21 14:10:44.956Z INFO gunicorn.error "Booting worker with pid: 41"
2022-01-21 14:10:47.595Z INFO talisker.flask "updating raven config from flask app"

It sounds kind of like some people think this is a benign error that shouldn't be logged, but in our case it appears to be killing the worker - so not benign.

matteius · 2022-06-21T13:05:20Z

This seems to be biting me as well running saleor API deployed with gunicorn + uvicorn worker. The commit from @onekiloparsec kind of makes sense but that commit branch is old enough that its not compatible with my other dependencies when I try to install it. Forking the uvicorn repo to try and cherry pick that commit into my running version to see if it helps.

EDIT: Actually the 0.17.6 branch has the code already in the form of @onekiloparsec commit. I perhaps started seeing this only after adding --max-requests 2 argument to gunicorn.

Latest uvicorn logic:

        # Stop accepting new connections.
        for server in self.servers:
            server.close()
        for sock in sockets or []:
            sock.close()
        for server in self.servers:
            await server.wait_closed()

onekiloparsec · 2022-06-21T16:22:42Z

@matteius if you confirm running smoothly for a few days with 0.17.6, I may reconsider going back to the main branch!

benoitc · 2022-10-14T10:03:45Z

i am not reproucingi t myself. closing the issue

mstenta · 2022-10-14T11:45:53Z

I use this FastAPI Docker image, and this error occurs every time I restart my services: https://github.com/tiangolo/uvicorn-gunicorn-fastapi-docker

@benoitc Did you try to run the script that @gg349 posted above? #1877 (comment)

I just tested that and it replicates the error. Just create the Dockerfile and test.sh scripts with @gg349's code in the comment linked above, and run docker run -it $(docker build -q .)

The test.sh script runs two test cases, one with the error, and one without (after downgrading uvicorn).

Here is the relevant output which includes the error (fourth line from the bottom):

[2022-10-14 11:44:55 +0000] [25] [INFO] Starting gunicorn 20.0.4
[2022-10-14 11:44:55 +0000] [25] [INFO] Listening at: http://127.0.0.1:8000 (25)
[2022-10-14 11:44:55 +0000] [25] [INFO] Using worker: uvicorn.workers.UvicornWorker
[2022-10-14 11:44:55 +0000] [28] [INFO] Booting worker with pid: 28
[2022-10-14 11:44:55 +0000] [28] [INFO] Started server process [28]
[2022-10-14 11:44:55 +0000] [28] [INFO] Waiting for application startup.
[2022-10-14 11:44:55 +0000] [28] [INFO] ASGI 'lifespan' protocol appears unsupported.
[2022-10-14 11:44:55 +0000] [28] [INFO] Application startup complete.
[2022-10-14 11:45:00 +0000] [25] [INFO] Handling signal: term
[2022-10-14 11:45:00 +0000] [28] [INFO] Shutting down
[2022-10-14 11:45:00 +0000] [28] [INFO] Error while closing socket [Errno 9] Bad file descriptor
[2022-10-14 11:45:00 +0000] [28] [INFO] Finished server process [28]
[2022-10-14 11:45:00 +0000] [28] [INFO] Worker exiting (pid: 28)
[2022-10-14 11:45:00 +0000] [25] [INFO] Shutting down: Master

I think this issue should remain open.

muety · 2022-10-18T14:33:59Z

Still happening for me with FastAPI + Gunicorn + Uvicorn in Docker.

Kludex · 2022-10-31T08:24:02Z

@benoitc Maybe I can help you here...

The problem here is that the Arbiter on stop, calls the close_sockets:

gunicorn/gunicorn/arbiter.py

Line 382 in 69c508a

sock.close_sockets(self.LISTENERS, unlink)

The thing is that close_sockets will unlink them:

gunicorn/gunicorn/sock.py

Lines 211 to 212 in 69c508a

    
           if unlink and _sock_type(sock_name) is UnixSocket: 
        
               os.unlink(sock_name)

This decision was made on #1310. At the time, #1309 was closed. Maybe it shouldn't have been...

I'll merge encode/uvicorn#1710 on uvicorn before the next release, which will make this error appear more frequently. That will be merged to solve the forwarding of SIGQUIT instead of SIGINT, as we previously discussed on #2604.

tlinhart · 2023-02-07T12:00:30Z

This also happens for me when using Uvicorn worker class. In my case, I run nginx and Gunicorn in a Docker container where nginx server proxies requests to UNIX socket bound by Gunicorn. Both nginx and Gunicorn are managed by Supervisor. I also define a file watch for my app's config file (using watchfiles) in Supervisor's configuration which sends HUP signal to Gunicorn upon a file change. According to the documentation, this should be used for a graceful reload. It works well but exhibits the error with closing the socket.

Here's the components' version info:

Docker base image: python:3.10-bullseye
nginx: 1.18.0
Gunicorn: 20.1.0
Uvicorn: 0.20.0

Relevant part of supervisord.conf:

[program:gunicorn]
command = gunicorn -c ./gunicorn.conf.py
stdout_logfile = /dev/stdout
stdout_logfile_maxbytes = 0
stderr_logfile = /dev/stderr
stderr_logfile_maxbytes = 0

[program:config_watch]
command = watchfiles 'supervisorctl -c ./supervisord.conf signal HUP gunicorn' %(ENV_CONFIG_FILE)s
stdout_logfile = /dev/stdout
stdout_logfile_maxbytes = 0
stderr_logfile = /dev/stderr
stderr_logfile_maxbytes = 0

[program:nginx]
command = nginx -g 'daemon off;'
stopsignal = QUIT
stdout_logfile = /dev/stdout
stdout_logfile_maxbytes = 0
stderr_logfile = /dev/stderr
stderr_logfile_maxbytes = 0

CONFIG_FILE environment variable contains a path to my app's config file.

Relevant part of nginx.conf:

server {
    listen 80 default_server;
    server_name _;

    error_log stderr;
    access_log /dev/stdout;

    location / {
        gzip on;
        gzip_types application/json;
        gzip_min_length 1000;

        proxy_set_header Host $http_host;
        proxy_redirect off;
        proxy_pass http://unix:/tmp/gunicorn.socket;
    }
}

gunicorn.conf.py:

import multiprocessing

user = "www-data"
group = "www-data"
bind = "unix:/tmp/gunicorn.socket"

workers = multiprocessing.cpu_count() * 2 + 1
worker_class = "uvicorn.workers.UvicornWorker"
worker_tmp_dir = "/dev/shm"

timeout = 120
graceful_timeout = 120
keepalive = 5

accesslog = None
errorlog = "-"
loglevel = "info"

wsgi_app = "app:app"

Here's the Docker container log:

2023-02-07 11:58:11,874 INFO Set uid to user 0 succeeded
2023-02-07 11:58:11,931 INFO RPC interface 'supervisor' initialized
2023-02-07 11:58:11,931 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2023-02-07 11:58:11,932 INFO supervisord started with pid 1
2023-02-07 11:58:12,938 INFO spawned: 'config_watch' with pid 7
2023-02-07 11:58:12,944 INFO spawned: 'gunicorn' with pid 8
2023-02-07 11:58:12,950 INFO spawned: 'nginx' with pid 9
[11:58:13] watchfiles v0.18.1 👀  path="/srv/app/config.yaml" target="supervisorctl -c ./supervisord.conf signal HUP gunicorn" (command) filter=DefaultFilter...
gunicorn: signalled
2023-02-07 11:58:13,600 WARN exited: gunicorn (terminated by SIGHUP; not expected)
2023-02-07 11:58:14,601 INFO success: config_watch entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2023-02-07 11:58:14,605 INFO spawned: 'gunicorn' with pid 16
2023-02-07 11:58:14,606 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
[2023-02-07 11:58:15 +0000] [16] [INFO] Starting gunicorn 20.1.0
[2023-02-07 11:58:15 +0000] [16] [INFO] Listening at: unix:/tmp/gunicorn.socket (16)
[2023-02-07 11:58:15 +0000] [16] [INFO] Using worker: uvicorn.workers.UvicornWorker
[2023-02-07 11:58:15 +0000] [17] [INFO] Booting worker with pid: 17
[2023-02-07 11:58:15 +0000] [18] [INFO] Booting worker with pid: 18
[2023-02-07 11:58:15 +0000] [19] [INFO] Booting worker with pid: 19
[2023-02-07 11:58:15 +0000] [20] [INFO] Booting worker with pid: 20
[2023-02-07 11:58:15 +0000] [21] [INFO] Booting worker with pid: 21
[2023-02-07 11:58:15 +0000] [22] [INFO] Booting worker with pid: 22
2023-02-07 11:58:15,689 INFO success: gunicorn entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
[2023-02-07 11:58:15 +0000] [23] [INFO] Booting worker with pid: 23
[2023-02-07 11:58:15 +0000] [24] [INFO] Booting worker with pid: 24
[2023-02-07 11:58:15 +0000] [25] [INFO] Booting worker with pid: 25
[2023-02-07 11:58:16 +0000] [18] [INFO] Started server process [18]
[2023-02-07 11:58:16 +0000] [18] [INFO] Waiting for application startup.
[2023-02-07 11:58:16 +0000] [18] [INFO] Application startup complete.
[2023-02-07 11:58:17 +0000] [19] [INFO] Started server process [19]
[2023-02-07 11:58:17 +0000] [19] [INFO] Waiting for application startup.
[2023-02-07 11:58:17 +0000] [19] [INFO] Application startup complete.
[2023-02-07 11:58:17 +0000] [17] [INFO] Started server process [17]
[2023-02-07 11:58:17 +0000] [17] [INFO] Waiting for application startup.
[2023-02-07 11:58:17 +0000] [17] [INFO] Application startup complete.
[2023-02-07 11:58:17 +0000] [20] [INFO] Started server process [20]
[2023-02-07 11:58:17 +0000] [20] [INFO] Waiting for application startup.
[2023-02-07 11:58:17 +0000] [20] [INFO] Application startup complete.
[2023-02-07 11:58:17 +0000] [23] [INFO] Started server process [23]
[2023-02-07 11:58:17 +0000] [23] [INFO] Waiting for application startup.
[2023-02-07 11:58:17 +0000] [23] [INFO] Application startup complete.
[2023-02-07 11:58:17 +0000] [21] [INFO] Started server process [21]
[2023-02-07 11:58:17 +0000] [21] [INFO] Waiting for application startup.
[2023-02-07 11:58:17 +0000] [21] [INFO] Application startup complete.
[2023-02-07 11:58:17 +0000] [24] [INFO] Started server process [24]
[2023-02-07 11:58:17 +0000] [24] [INFO] Waiting for application startup.
[2023-02-07 11:58:17 +0000] [24] [INFO] Application startup complete.
[2023-02-07 11:58:17 +0000] [25] [INFO] Started server process [25]
[2023-02-07 11:58:17 +0000] [25] [INFO] Waiting for application startup.
[2023-02-07 11:58:17 +0000] [25] [INFO] Application startup complete.
[2023-02-07 11:58:18 +0000] [22] [INFO] Started server process [22]
[2023-02-07 11:58:18 +0000] [22] [INFO] Waiting for application startup.
[2023-02-07 11:58:18 +0000] [22] [INFO] Application startup complete.

...modifying the config file now...

[11:58:53] 1 change detected
[11:58:53] process already dead, exit code: 0
gunicorn: signalled
[2023-02-07 11:58:53 +0000] [16] [INFO] Handling signal: hup
[2023-02-07 11:58:53 +0000] [16] [INFO] Hang up: Master
[2023-02-07 11:58:53 +0000] [41] [INFO] Booting worker with pid: 41
[2023-02-07 11:58:53 +0000] [44] [INFO] Booting worker with pid: 44
[2023-02-07 11:58:53 +0000] [43] [INFO] Booting worker with pid: 43
[2023-02-07 11:58:53 +0000] [42] [INFO] Booting worker with pid: 42
[2023-02-07 11:58:53 +0000] [46] [INFO] Booting worker with pid: 46
[2023-02-07 11:58:53 +0000] [45] [INFO] Booting worker with pid: 45
[2023-02-07 11:58:53 +0000] [47] [INFO] Booting worker with pid: 47
[2023-02-07 11:58:53 +0000] [48] [INFO] Booting worker with pid: 48
[2023-02-07 11:58:53 +0000] [22] [INFO] Shutting down
[2023-02-07 11:58:53 +0000] [22] [INFO] Error while closing socket [Errno 9] Bad file descriptor
[2023-02-07 11:58:53 +0000] [17] [INFO] Shutting down
[2023-02-07 11:58:53 +0000] [17] [INFO] Error while closing socket [Errno 9] Bad file descriptor
[2023-02-07 11:58:53 +0000] [49] [INFO] Booting worker with pid: 49
[2023-02-07 11:58:53 +0000] [20] [INFO] Shutting down
[2023-02-07 11:58:53 +0000] [20] [INFO] Error while closing socket [Errno 9] Bad file descriptor
[2023-02-07 11:58:53 +0000] [21] [INFO] Shutting down
[2023-02-07 11:58:53 +0000] [21] [INFO] Error while closing socket [Errno 9] Bad file descriptor
[2023-02-07 11:58:53 +0000] [24] [INFO] Shutting down
[2023-02-07 11:58:53 +0000] [24] [INFO] Error while closing socket [Errno 9] Bad file descriptor
[2023-02-07 11:58:53 +0000] [25] [INFO] Shutting down
[2023-02-07 11:58:53 +0000] [25] [INFO] Error while closing socket [Errno 9] Bad file descriptor
[2023-02-07 11:58:53 +0000] [18] [INFO] Shutting down
[2023-02-07 11:58:53 +0000] [18] [INFO] Error while closing socket [Errno 9] Bad file descriptor
[2023-02-07 11:58:53 +0000] [23] [INFO] Shutting down
[2023-02-07 11:58:53 +0000] [23] [INFO] Error while closing socket [Errno 9] Bad file descriptor
[2023-02-07 11:58:53 +0000] [19] [INFO] Shutting down
[2023-02-07 11:58:53 +0000] [19] [INFO] Error while closing socket [Errno 9] Bad file descriptor
[2023-02-07 11:58:53 +0000] [22] [INFO] Waiting for application shutdown.
[2023-02-07 11:58:53 +0000] [22] [INFO] Application shutdown complete.
[2023-02-07 11:58:53 +0000] [22] [INFO] Finished server process [22]
[2023-02-07 11:58:53 +0000] [22] [INFO] Worker exiting (pid: 22)
[2023-02-07 11:58:53 +0000] [17] [INFO] Waiting for application shutdown.
[2023-02-07 11:58:53 +0000] [17] [INFO] Application shutdown complete.
[2023-02-07 11:58:53 +0000] [17] [INFO] Finished server process [17]
[2023-02-07 11:58:53 +0000] [17] [INFO] Worker exiting (pid: 17)
[2023-02-07 11:58:53 +0000] [20] [INFO] Waiting for application shutdown.
[2023-02-07 11:58:53 +0000] [20] [INFO] Application shutdown complete.
[2023-02-07 11:58:53 +0000] [20] [INFO] Finished server process [20]
[2023-02-07 11:58:53 +0000] [21] [INFO] Waiting for application shutdown.
[2023-02-07 11:58:53 +0000] [20] [INFO] Worker exiting (pid: 20)
[2023-02-07 11:58:53 +0000] [21] [INFO] Application shutdown complete.
[2023-02-07 11:58:53 +0000] [21] [INFO] Finished server process [21]
[2023-02-07 11:58:53 +0000] [21] [INFO] Worker exiting (pid: 21)
[2023-02-07 11:58:53 +0000] [24] [INFO] Waiting for application shutdown.
[2023-02-07 11:58:53 +0000] [24] [INFO] Application shutdown complete.
[2023-02-07 11:58:53 +0000] [24] [INFO] Finished server process [24]
[2023-02-07 11:58:53 +0000] [24] [INFO] Worker exiting (pid: 24)
[2023-02-07 11:58:53 +0000] [25] [INFO] Waiting for application shutdown.
[2023-02-07 11:58:53 +0000] [25] [INFO] Application shutdown complete.
[2023-02-07 11:58:53 +0000] [25] [INFO] Finished server process [25]
[2023-02-07 11:58:53 +0000] [25] [INFO] Worker exiting (pid: 25)
[2023-02-07 11:58:53 +0000] [18] [INFO] Waiting for application shutdown.
[2023-02-07 11:58:53 +0000] [18] [INFO] Application shutdown complete.
[2023-02-07 11:58:53 +0000] [18] [INFO] Finished server process [18]
[2023-02-07 11:58:53 +0000] [18] [INFO] Worker exiting (pid: 18)
[2023-02-07 11:58:53 +0000] [23] [INFO] Waiting for application shutdown.
[2023-02-07 11:58:53 +0000] [23] [INFO] Application shutdown complete.
[2023-02-07 11:58:53 +0000] [23] [INFO] Finished server process [23]
[2023-02-07 11:58:53 +0000] [23] [INFO] Worker exiting (pid: 23)
[2023-02-07 11:58:53 +0000] [19] [INFO] Waiting for application shutdown.
[2023-02-07 11:58:53 +0000] [19] [INFO] Application shutdown complete.
[2023-02-07 11:58:53 +0000] [19] [INFO] Finished server process [19]
[2023-02-07 11:58:53 +0000] [19] [INFO] Worker exiting (pid: 19)
[2023-02-07 11:58:54 +0000] [16] [WARNING] Worker with pid 20 was terminated due to signal 15
[2023-02-07 11:58:54 +0000] [16] [WARNING] Worker with pid 22 was terminated due to signal 15
[2023-02-07 11:58:54 +0000] [16] [WARNING] Worker with pid 23 was terminated due to signal 15
[2023-02-07 11:58:54 +0000] [16] [WARNING] Worker with pid 25 was terminated due to signal 15
[2023-02-07 11:58:54 +0000] [16] [WARNING] Worker with pid 21 was terminated due to signal 15
[2023-02-07 11:58:54 +0000] [16] [WARNING] Worker with pid 17 was terminated due to signal 15
[2023-02-07 11:58:54 +0000] [16] [WARNING] Worker with pid 18 was terminated due to signal 15
[2023-02-07 11:58:54 +0000] [16] [WARNING] Worker with pid 24 was terminated due to signal 15
[2023-02-07 11:58:55 +0000] [41] [INFO] Started server process [41]
[2023-02-07 11:58:55 +0000] [41] [INFO] Waiting for application startup.
[2023-02-07 11:58:55 +0000] [41] [INFO] Application startup complete.
[2023-02-07 11:58:56 +0000] [45] [INFO] Started server process [45]
[2023-02-07 11:58:56 +0000] [45] [INFO] Waiting for application startup.
[2023-02-07 11:58:56 +0000] [45] [INFO] Application startup complete.
[2023-02-07 11:58:56 +0000] [42] [INFO] Started server process [42]
[2023-02-07 11:58:56 +0000] [42] [INFO] Waiting for application startup.
[2023-02-07 11:58:56 +0000] [42] [INFO] Application startup complete.
[2023-02-07 11:58:56 +0000] [43] [INFO] Started server process [43]
[2023-02-07 11:58:56 +0000] [43] [INFO] Waiting for application startup.
[2023-02-07 11:58:56 +0000] [43] [INFO] Application startup complete.
[2023-02-07 11:58:56 +0000] [49] [INFO] Started server process [49]
[2023-02-07 11:58:56 +0000] [49] [INFO] Waiting for application startup.
[2023-02-07 11:58:56 +0000] [49] [INFO] Application startup complete.
[2023-02-07 11:58:56 +0000] [47] [INFO] Started server process [47]
[2023-02-07 11:58:56 +0000] [47] [INFO] Waiting for application startup.
[2023-02-07 11:58:56 +0000] [47] [INFO] Application startup complete.
[2023-02-07 11:58:56 +0000] [48] [INFO] Started server process [48]
[2023-02-07 11:58:56 +0000] [48] [INFO] Waiting for application startup.
[2023-02-07 11:58:56 +0000] [44] [INFO] Started server process [44]
[2023-02-07 11:58:56 +0000] [44] [INFO] Waiting for application startup.
[2023-02-07 11:58:56 +0000] [48] [INFO] Application startup complete.
[2023-02-07 11:58:56 +0000] [44] [INFO] Application startup complete.
[2023-02-07 11:58:56 +0000] [46] [INFO] Started server process [46]
[2023-02-07 11:58:56 +0000] [46] [INFO] Waiting for application startup.
[2023-02-07 11:58:56 +0000] [46] [INFO] Application startup complete.

beaugunderson · 2023-02-17T05:57:01Z

I also see this error on latest gunicorn+uvicorn every time max-requests triggers and reaps a worker.

alexgleason · 2023-02-17T06:45:52Z

For us the problem was our form was doing a POST /api while the route was on /api/. 🤦

The trailing slash was resulting in a 307 redirect and... making the file not write?

mohitbansal-gep · 2023-03-13T19:47:44Z

What is the solution to above issue?

artempronevskiy · 2023-06-14T20:52:17Z

Gunicorn (20.1.0) + Uvicorn (0.22.0) -> same issue

lvyaoo · 2023-06-27T04:03:23Z

This happens for me, too.

docker
gunicorn (20.1.0)
uvicorn (0.22.0)
fastapi (0.98.0)

Aniket-Singla · 2023-07-19T12:01:09Z

Happens to me too :

gunicorn 21.1.0 gthread workers with some threads
python 3.11
Django 4.2
Docker arm Architecture
If anyone has found solution to it, post it on this thread. Thanks

benoitc · 2023-07-19T12:10:53Z

@Aniket-Singla please upgrade to 21.2.0 this is fixing such issue. Thanks for the report, but next time can you open a new ticket or create a new one that link to a closed issue. This is described in the contributing document and helps me to track issues )

Aniket-Singla · 2023-07-19T12:30:10Z

@Aniket-Singla please upgrade to 21.2.0 this is fixing such issue. Thanks for the report, but next time can you open a new ticket or create a new one that link to a closed issue. This is described in the contributing document and helps me to track issues )

Thankyou so much for your reply and sorry for not reading the contributing Doc. Will upgrade

timkofu · 2023-08-22T11:49:08Z

It's a uvloop issue. Setting Uvicorn's loop to asyncio (by overriding uvicorn.workers.UvicornWorker.CONFIG_KWARGS) fixed it for me.

pplmx · 2023-10-26T06:42:25Z

The same issue to me, but it's only occurred on my k8s deployment.
Here is my repo, make k to deploy it by k8s to reproduce the issue.

KernelPryanic · 2023-11-03T21:41:29Z

The same for me with gunicorn == 21.2.0 and meinheld == 1.0.1.

jmugan · 2023-11-09T05:19:49Z

I'm getting this Error while closing socket [Errno 9] Bad file descriptor on 21.2.0. Was there any workaround or anything? I'm using FastAPI.

I'm running gunicorn in production with systemd. I periodically get [INFO] Handling signal: term and then Error while closing socket [Errno 9] Bad file descriptor and then it won't start up again saying Connection in use: ('127.0.0.1', 8000). This has been driving me nuts because prod goes down. Any help or pointers are appreciated.

And I checked memory, that's not it.

jsprieto10 · 2024-01-25T11:03:54Z

It's a uvloop issue. Setting Uvicorn's loop to asyncio (by overriding uvicorn.workers.UvicornWorker.CONFIG_KWARGS) fixed it for me.

can you explain more about this? my server is getting full of these logs

aaronsewall · 2024-01-25T23:04:40Z

It's a uvloop issue. Setting Uvicorn's loop to asyncio (by overriding uvicorn.workers.UvicornWorker.CONFIG_KWARGS) fixed it for me.

can you explain more about this? my server is getting full of these logs

So I just tried this and it did work for me. Here's a minimal example for FastAPI:

I put this into uvicorn_utils.py in my fast_api_app folder.

from uvicorn.workers import UvicornWorker

MyUvicornWorker = UvicornWorker
MyUvicornWorker.CONFIG_KWARGS = {"loop": "asyncio", "http": "auto"}

I use a gunicorn.conf.py file which then I invoke with gunicorn fast_api_app.main:app -c gunicorn.conf.py with the following

worker_class = "fast_api_app.uvicorn_utils.MyUvicornWorker"

Now I no longer get file descriptor warnings. I'm using gunicorn 20.1.0, uvicorn 0.24.0.post1 and fastapi 0.108.0.

benoitc added the Feedback Requested label Sep 10, 2018

benoitc self-assigned this Oct 9, 2018

benoitc added the Investigation label Jan 13, 2021

JaccoVE mentioned this issue Jul 31, 2021

Silence broken OS errors from the WebSocket layer (Fixes #12) miguelgrinberg/turbo-flask#13

Closed

onekiloparsec added a commit to onekiloparsec/uvicorn that referenced this issue Aug 6, 2021

Inverting lines according to issue report

6e336cf

benoitc/gunicorn#1877 (comment) Signed-off-by: Cédric Foellmi <cedric@onekiloparsec.dev>

benoitc closed this as completed Oct 14, 2022

pkucmus mentioned this issue Oct 22, 2022

The memory usage piles up over the time and leads to OOM fastapi/fastapi#1624

Closed

9 tasks

Kludex mentioned this issue Oct 31, 2022

Add SIGQUIT handler to UvicornWorker encode/uvicorn#1710

Merged

crankedguy mentioned this issue Jun 21, 2023

Bad file descriptor problem still exists as of today with uvicorn workers #3013

Open

PKizzle mentioned this issue Oct 17, 2023

Gunicorn throws error upon handling SIGINT goauthentik/authentik#7204

Open

Ishankoradia mentioned this issue Jul 22, 2024

Error while closing socket [Errno 9] Bad file descriptor DalgoT4D/prefect-proxy#144

Open

GGP1 mentioned this issue Jul 23, 2024

Comms API - Initial setup wazuh/wazuh#24616

Closed

10 tasks

GGP1 mentioned this issue Aug 9, 2024

Address technical debt for the new Comms API wazuh/wazuh#25121

Open

27 tasks

grinapo mentioned this issue Oct 25, 2024

self.handle_request(req, conn): OSError: [Errno 9] Bad file descriptor prometheus-pve/prometheus-pve-exporter#293

Open

OSError: [Errno 9] Bad file descriptor #1877

OSError: [Errno 9] Bad file descriptor #1877

Comments

leond08 commented Sep 10, 2018

benoitc commented Sep 10, 2018 • edited Loading

leond08 commented Sep 12, 2018

yunstanford commented Sep 14, 2018

benoitc commented Oct 9, 2018

dpzx commented Jan 10, 2019

georgexsh commented Apr 1, 2020

zgoda commented May 23, 2020 • edited Loading

zgoda commented May 26, 2020

Soberia commented May 26, 2020

benoitc commented May 27, 2020

zgoda commented May 27, 2020

Ronserruya commented May 27, 2020 • edited Loading

strelkov-a commented Jun 1, 2020

fgimian commented Sep 12, 2020 • edited Loading

fgimian commented Sep 12, 2020 • edited Loading

gg349 commented Dec 6, 2020 • edited Loading

ChrisXiaoShu commented Jan 13, 2021

jamadden commented Jan 13, 2021

mawoka-myblock commented Jun 7, 2021

hardenchant commented Oct 26, 2021

nazrulworld commented Nov 29, 2021

jules-ch commented Dec 3, 2021

mindflayer commented Jan 10, 2022 • edited Loading

mindflayer commented Jan 10, 2022 • edited Loading

nottrobin commented Jan 21, 2022

matteius commented Jun 21, 2022 • edited Loading

onekiloparsec commented Jun 21, 2022

benoitc commented Oct 14, 2022

mstenta commented Oct 14, 2022

muety commented Oct 18, 2022

Kludex commented Oct 31, 2022 • edited Loading

tlinhart commented Feb 7, 2023

beaugunderson commented Feb 17, 2023

alexgleason commented Feb 17, 2023

mohitbansal-gep commented Mar 13, 2023

artempronevskiy commented Jun 14, 2023

lvyaoo commented Jun 27, 2023

Aniket-Singla commented Jul 19, 2023 • edited Loading

benoitc commented Jul 19, 2023

Aniket-Singla commented Jul 19, 2023

timkofu commented Aug 22, 2023 • edited Loading

pplmx commented Oct 26, 2023 • edited Loading

KernelPryanic commented Nov 3, 2023

jmugan commented Nov 9, 2023 • edited Loading

jsprieto10 commented Jan 25, 2024

aaronsewall commented Jan 25, 2024

benoitc commented Sep 10, 2018 •

edited

Loading

zgoda commented May 23, 2020 •

edited

Loading

Ronserruya commented May 27, 2020 •

edited

Loading

fgimian commented Sep 12, 2020 •

edited

Loading

fgimian commented Sep 12, 2020 •

edited

Loading

gg349 commented Dec 6, 2020 •

edited

Loading

mindflayer commented Jan 10, 2022 •

edited

Loading

mindflayer commented Jan 10, 2022 •

edited

Loading

matteius commented Jun 21, 2022 •

edited

Loading

Kludex commented Oct 31, 2022 •

edited

Loading

Aniket-Singla commented Jul 19, 2023 •

edited

Loading

timkofu commented Aug 22, 2023 •

edited

Loading

pplmx commented Oct 26, 2023 •

edited

Loading

jmugan commented Nov 9, 2023 •

edited

Loading