Skip to content

gh-94609: Make test_ssl.ThreadedEchoServer exceptions appear in a main thread #92475

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 92 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
92 commits
Select commit Hold shift + click to select a range
ea5230f
Add a threaded server factory
arhadthedev May 19, 2022
c6e68e9
Add an alternative implementation of ThreadedEchoServer
arhadthedev May 19, 2022
47ac053
Port two simple cases as an example
arhadthedev May 8, 2022
05fd819
Port multiserver test_check_hostname
arhadthedev May 18, 2022
149d18c
Port a case where client connects twice
arhadthedev May 19, 2022
e97b07c
Port a case where client connects four times
arhadthedev May 8, 2022
4fa30f1
Fix a race between client connection and server launch
arhadthedev May 9, 2022
d5c4762
Add a temporary workaround for zero-byte SSL transfers
arhadthedev May 19, 2022
289e563
Prevent hanging in case of client overexpectation
arhadthedev May 21, 2022
9aa2bef
Fix more sporadic errors
arhadthedev May 21, 2022
bb750f3
Merge branch 'main' into threadedserver
arhadthedev Jul 5, 2022
d39fba9
Automatic thread management not only needs no manual leak detector, i…
arhadthedev Jul 6, 2022
a2f0b07
Use a named constant for a timeout
arhadthedev Jul 6, 2022
bbce667
Restore an unstable test
arhadthedev Jul 6, 2022
d10cc72
Restore the second unstable test
arhadthedev Jul 6, 2022
18b3c15
Remove accidentally included comment fragment
arhadthedev Jul 6, 2022
1c8a6f1
Add a NEWS entry
arhadthedev Jul 6, 2022
bbb7c6f
Fix a potential method name conflict between base and derived classes
arhadthedev Jul 6, 2022
beb1650
Remove a redundant parameter
arhadthedev Jul 6, 2022
4b4b4e5
Revert "Restore the second unstable test"
arhadthedev Jul 6, 2022
0f3663b
Forcefully release a thread after a module is over
arhadthedev Jul 6, 2022
a999eab
Account for a fact that assignments cannot be inside lambdas
arhadthedev Jul 6, 2022
5e570f2
Split test_check_hostname to discover which one fails
arhadthedev Jul 6, 2022
2ea2217
Teach the server to expect client errors on demand
arhadthedev Jul 7, 2022
8c78bb9
Make sure that a heavy context object with captures does not pass thr…
arhadthedev Jul 7, 2022
5dcea72
Teach the server to handle expectedly closed sockets
arhadthedev Jul 7, 2022
535c54c
Fit threading_helper into 79 columns
arhadthedev Jul 7, 2022
a39981f
Use single quotation because there won't be any chance to fix it later
arhadthedev Jul 7, 2022
2d7d843
Account that SSLError subclasses OSError
arhadthedev Jul 7, 2022
1418cc3
Debug output of whatever happens with SSLError
arhadthedev Jul 7, 2022
74bd397
(once again)
arhadthedev Jul 7, 2022
2b3cabf
(and again)
arhadthedev Jul 7, 2022
555bde6
SSLError seems to have nonzero errno - print it
arhadthedev Jul 7, 2022
8578b98
Properly separate exceptions
arhadthedev Jul 7, 2022
8858099
After all fixes, try to rely on natural garbage collection of thread …
arhadthedev Jul 7, 2022
59773f8
Another attempt at threadpool cleanup
arhadthedev Jul 7, 2022
29e6a6d
Make global threadpool explicit
arhadthedev Jul 7, 2022
c35630e
Another attempt
arhadthedev Jul 7, 2022
0a67199
Merge two setUpModule copies
arhadthedev Jul 7, 2022
02a1bc8
Explicit waiting on futures seem to destabilize a program
arhadthedev Jul 7, 2022
5c15509
Fix incorrect pass of normal results from a server to a client
arhadthedev Jul 7, 2022
0ce52ea
Merge branch 'main' into threadedserver
arhadthedev Jul 7, 2022
404d120
Simplify expected exception regex
arhadthedev Jul 7, 2022
a2a526e
Show what really is thrown on macOS runner
arhadthedev Jul 7, 2022
c7a3f13
typo
arhadthedev Jul 7, 2022
2caabcb
Context managers are not terminated after a `with` section
arhadthedev Jul 7, 2022
70f8f90
Temporarily disable expected exception handling
arhadthedev Jul 7, 2022
1f1da1e
Sort out discrepancy between macOS and others
arhadthedev Jul 8, 2022
23cf45d
Final touch-ups
arhadthedev Jul 8, 2022
2ee7b5a
Fix `SyntaxError: no binding for nonlocal '_thread_pool' found`
arhadthedev Jul 8, 2022
30fdbc7
Merge branch 'main' into threadedserver
arhadthedev Jul 8, 2022
a77d6e4
Stabilize tests by forcing TLS handshake with empty data packets
arhadthedev Jul 8, 2022
7e85ec8
Remove unnecessary wait_connection invocations
arhadthedev Jul 8, 2022
d167f1c
Temporarily show exact exception
arhadthedev Jul 8, 2022
910083b
Revert "Temporarily show exact exception"
arhadthedev Jul 8, 2022
458a0a6
An attempt to fix in-handshake connection abruption
arhadthedev Jul 9, 2022
1fa3f0c
Reorder imports
arhadthedev Jul 9, 2022
17c56dc
Fix potential infinite waiting on handshake enforcement
arhadthedev Jul 9, 2022
37e3470
Debug why runners hang
arhadthedev Jul 9, 2022
6b196f9
Try to remove server response waiting
arhadthedev Jul 9, 2022
2152234
Discriminate when we need to wait server response
arhadthedev Jul 9, 2022
f4406cf
TEMPORARILY force full-fledged logging
arhadthedev Jul 9, 2022
9f6e826
Add a socket timeout to not hang the whole test
arhadthedev Jul 9, 2022
19486c9
Fix server-side echo
arhadthedev Jul 9, 2022
cad138b
Fix THE REAL REASON: sockets do not implement context managers hence …
arhadthedev Jul 9, 2022
2bebeca
Merge branch 'main' into threadedserver
arhadthedev Jul 9, 2022
4ced1f8
Run patchcheck.py
arhadthedev Jul 10, 2022
78225c0
Reword a NEWS entry
arhadthedev Jul 10, 2022
eba78fa
Add socket timeout to all server-related tests
arhadthedev Jul 10, 2022
5287287
Restore code removed while debugging
arhadthedev Jul 10, 2022
71197b9
Restore more code
arhadthedev Jul 10, 2022
feb2b20
Try to replace server-side error swallow with client-side ignorance
arhadthedev Jul 10, 2022
e4c7fca
Clarify a purpose of wait_connection()
arhadthedev Jul 10, 2022
ed98010
A typo
arhadthedev Jul 10, 2022
cc1b400
Restore explicit server timeout; getdefaulttimeout does not propagate…
arhadthedev Jul 10, 2022
fd0468e
Try to fix the ENV CHANGED error probably caused by cleanup misorder
arhadthedev Jul 10, 2022
de27807
Add some debug scaffolding to track down a leaked thread
arhadthedev Jul 10, 2022
b410384
Avoid thread leaks by shutting down the pool
arhadthedev Jul 10, 2022
2a44b2a
Revert "Avoid thread leaks by shutting down the pool"
arhadthedev Jul 10, 2022
1f6378a
Remove threading control (a thread pool interferes with it)
arhadthedev Jul 10, 2022
4f7b0e9
Revert "Add some debug scaffolding to track down a leaked thread"
arhadthedev Jul 10, 2022
f0e7c8d
On MacOS server that lost a client still raises an exception
arhadthedev Jul 10, 2022
c67bb1a
Clarify terms in NEWS
arhadthedev Jul 10, 2022
cdb9137
Streamline comments
arhadthedev Jul 10, 2022
fbacdc8
Set a default socket timeout globally
arhadthedev Jul 10, 2022
d6940b9
Merge branch 'main' into threadedserver
arhadthedev Jul 10, 2022
e526af4
Use sockets directly, without contextlib.closing
arhadthedev Jul 10, 2022
78d725e
Merge branch 'main' into threadedserver
arhadthedev Jul 21, 2022
b8174ae
Put the news entry into a present tense
arhadthedev Oct 30, 2022
a551384
Merge branch 'main' into threadedserver
arhadthedev Oct 30, 2022
a3693f3
Merge branch 'main' into threadedserver
arhadthedev Nov 27, 2022
e4d35a5
Merge branch 'main' into threadedserver
arhadthedev Jan 14, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
102 changes: 102 additions & 0 deletions Lib/test/support/threading_helper.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,27 @@
import functools
import sys
import threading
from socket import socket
import time
import unittest
from concurrent.futures import ThreadPoolExecutor
from test.support.socket_helper import bind_port, HOST

from test import support

_thread_pool = None


def _release():
global _thread_pool
_thread_pool = None


def init():
global _thread_pool
_thread_pool = ThreadPoolExecutor()
unittest.addModuleCleanup(_release)
Comment on lines +17 to +25
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added init() to avoid a modified environment test failure due to a leaked object. If there's a better solution, I'll use that instead.



#=======================================================================
# Threading support to prevent reporting refleaks when running regrtest.py -R
Expand Down Expand Up @@ -240,3 +256,89 @@ def requires_working_threading(*, module=False):
raise unittest.SkipTest(msg)
else:
return unittest.skipUnless(can_start_thread, msg)


class Server:
"""A context manager for a blocking server in a thread pool.

The server is designed:

- for testing purposes so it serves a fixed count of clients, one by one
- to be one-pass, short-lived, and terminated by in-protocol means so no
stopper flag is used
- to be used where asyncio has no application

The server listens on an address returned from the ``with`` statement.

For each client connected, the server calls a user-supplied function and
preserves whatever the function returns or throws to pass it to a client
later.

When a client attempt to exit the context manager, it blocks until a server
stops processing all clients and exits.
"""

def __init__(self, client_func, *args, client_count=1, **kwargs):
"""Create and run the server.

The method blocks until the server is ready to accept clients.

After this constructor returns, the server:

1. Consequently waits for client_count clients
1. For each client:
a. Calls client_func for each of them
b. Closes client connection when the function returns
c. Collects returned values into Server.result list
5. Terminates a server
6. Allows a context manager to exit
7. Since ``with ... as`` section keeps its parameter alive,
Server.result field can be accessed outside of the section.

If client_func raises an exception, the server is stopped, all pending
clients are discarded and the context manager raises an exception.

Args:
client_func: a function called in a dedicated thread for each new
connected client. The function receives all argument passed to
the __init__ method excluding client_func and client_count.
args: positional arguments passed to client_func.
client_count: count of clients the server processes one by one
before stopping.
results: a reference to a list for collecting client_func
return values. Populated after the execution leaves a ``with``
blocks associated with the Server context manager.
kwargs: keyword arguments passed to client_func.
"""
server_socket = socket()
self._port = bind_port(server_socket)
server_socket.listen()
self._result = _thread_pool.submit(self._thread_func, server_socket,
client_func, client_count,
args, kwargs)

def _thread_func(self, server_socket, client_func, client_count,
args, kwargs):
with server_socket:
results = []
for i in range(client_count):
client, peer_address = server_socket.accept()
with client:
r = client_func(client, peer_address, *args, **kwargs)
results.append(r)
return results

def __enter__(self):
return HOST, self._port

def __exit__(self, etype, evalue, traceback):
peer_willingly_closed = isinstance(etype, ConnectionError)
# We find our client disappeared when our socket read() fails.
peer_disappeared = etype is OSError
if peer_willingly_closed or peer_disappeared:
if self._result.exception() is not None:
generic = RuntimeError('server-side error')
raise generic from self._result.exception()
return False
self.result = self._result.result()
return False
Loading