OSError gunicorn.util in write Bad file descriptor guincorn v 21.0.1

[revanthmahesh]

 Traceback (most recent call last):

   File "/usr/local/lib/python3.8/dist-packages/gunicorn/workers/gthread.py", line 282, in handle

     keepalive = self.handle_request(req, conn)

   File "/usr/local/lib/python3.8/dist-packages/gunicorn/workers/gthread.py", line 354, in handle_request

     util.reraise(*sys.exc_info())

   File "/usr/local/lib/python3.8/dist-packages/gunicorn/util.py", line 641, in reraise

     raise value

   File "/usr/local/lib/python3.8/dist-packages/gunicorn/workers/gthread.py", line 340, in handle_request

     resp.write(item)

   File "/usr/local/lib/python3.8/dist-packages/gunicorn/http/wsgi.py", line 326, in write

     self.send_headers()

   File "/usr/local/lib/python3.8/dist-packages/gunicorn/http/wsgi.py", line 322, in send_headers

     util.write(self.sock, util.to_bytestring(header_str, "latin-1"))

   File "/usr/local/lib/python3.8/dist-packages/gunicorn/util.py", line 299, in write

     sock.sendall(data)

 OSError: [Errno 9] Bad file descriptor

This happening with latest gunicorn release, using previous versions is working fine

[benoitc]

@revanthmahesh how to reproduce it?

[GritWins]

We got the same issue today in Production and have reduced the version back to 20.1.0 to fix things.

[benoitc]

@SaravEasy I doin't reproduce it. Can you provide more info on how to reproduce it or in which context?

[GritWins]

@benoitc, let me try to reproduce this issue in a local environment and share more information shortly.

[GritWins]

Environment Details

Python Version : 3.6
OS architecture: AMD64
Flask: 0.12.1
Werkzeug: 0.12.1

Our API server started throwing the following exceptions intermittently and we noticed this issue and we reduced the version and it worked.

We did not take the open socket or file descriptors at that time or could not find the pattern.

Hope this helps you get started in the right direction. @benoitc

Stack Trace - Error

[2023-07-18 13:59:48 +0000] [18681] [ERROR] Exception in worker process
[2023-07-18 13:59:48 +0000] [18681] [INFO] Worker exiting (pid: 18681)
ValueError: Invalid file descriptor: -1
raise ValueError("Invalid file descriptor: {}".format(fd))
File "/usr/local/lib/python3.6/selectors.py", line 41, in _fileobj_to_fd
return _fileobj_to_fd(fileobj)
File "/usr/local/lib/python3.6/selectors.py", line 224, in _fileobj_lookup
key = self._fd_to_key.pop(self._fileobj_lookup(fileobj))
File "/usr/local/lib/python3.6/selectors.py", line 248, in unregister
key = super().unregister(fileobj)
File "/usr/local/lib/python3.6/selectors.py", line 419, in unregister
self.poller.unregister(conn.sock)
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/gthread.py", line 176, in murder_keepalived
self.murder_keepalived()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/gthread.py", line 232, in run
self.run()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 142, in init_process
super().init_process()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/gthread.py", line 95, in init_process
worker.init_process()
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 609, in spawn_worker
Traceback (most recent call last):
[2023-07-18 13:59:49 +0000] [16] [ERROR] Worker (pid:18681) exited with code 255.
[2023-07-18 13:59:49 +0000] [16] [ERROR] Worker (pid:18681) exited with code 255

[garikAsplund]

Similar thing in my project. Downgraded and it works fine.

ERROR 2023-07-18T20:37:26.355415Z Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/gunicorn/workers/gthread.py", line 285, in handle keepalive = self.handle_request(req, conn)
DEFAULT 2023-07-18T20:37:26.355419Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
DEFAULT 2023-07-18T20:37:26.355424Z File "/usr/local/lib/python3.11/site-packages/gunicorn/workers/gthread.py", line 357, in handle_request
DEFAULT 2023-07-18T20:37:26.355429Z util.reraise(*sys.exc_info())
DEFAULT 2023-07-18T20:37:26.355434Z File "/usr/local/lib/python3.11/site-packages/gunicorn/util.py", line 641, in reraise
DEFAULT 2023-07-18T20:37:26.355438Z raise value
DEFAULT 2023-07-18T20:37:26.355442Z File "/usr/local/lib/python3.11/site-packages/gunicorn/workers/gthread.py", line 343, in handle_request
DEFAULT 2023-07-18T20:37:26.355446Z resp.write(item)
DEFAULT 2023-07-18T20:37:26.355450Z File "/usr/local/lib/python3.11/site-packages/gunicorn/http/wsgi.py", line 326, in write
DEFAULT 2023-07-18T20:37:26.355454Z self.send_headers()
DEFAULT 2023-07-18T20:37:26.355459Z File "/usr/local/lib/python3.11/site-packages/gunicorn/http/wsgi.py", line 322, in send_headers
DEFAULT 2023-07-18T20:37:26.355464Z util.write(self.sock, util.to_bytestring(header_str, "latin-1"))
DEFAULT 2023-07-18T20:37:26.355470Z File "/usr/local/lib/python3.11/site-packages/gunicorn/util.py", line 299, in write
DEFAULT 2023-07-18T20:37:26.355473Z sock.sendall(data)
DEFAULT 2023-07-18T20:37:26.355908Z OSError: [Errno 9] Bad file descriptor
DEFAULT 2023-07-18T20:37:26.355914Z [2023-07-18 20:37:26 +0000] [2] [INFO] Worker exiting (pid: 2)

[benoitc]

Please test #3033 and let me know how it behaves after it.

[revanthmahesh]

Tested using the script below in docker with os

FROM ubuntu:20.04

RUN pip install git+https://github.com/benoitc/gunicorn.git@fix-gthread

Issue is fixed now @benoitc

[ahmetkurukose]

I also tested the fix-gthread branch with the Python 3.11 Debian Docker image, and I did not encounter any errors @benoitc

[jgehrcke]

Thank you @benoitc -- this also affected us.

[benoitc]

change applied in master. Thaks for the feedback!