Skip to content

Commit

Permalink
Read smaller frames to workaround OpenSSL bug
Browse files Browse the repository at this point in the history
As older versions of OpenSSL (in particular 1.0.2) have limitations on
the size of buffers they can work with, take small views into our larger
buffer and read those in instead. This should keep the buffer sizes more
manageable for OpenSSL.
  • Loading branch information
jakirkham committed Jul 24, 2021
1 parent 9c30f38 commit 974daa5
Showing 1 changed file with 8 additions and 3 deletions.
11 changes: 8 additions & 3 deletions distributed/comm/tcp.py
Original file line number Diff line number Diff line change
Expand Up @@ -193,9 +193,14 @@ async def read(self, deserializers=None):
frames_nbytes = await stream.read_bytes(fmt_size)
(frames_nbytes,) = struct.unpack(fmt, frames_nbytes)

frames = bytearray(frames_nbytes)
n = await stream.read_into(frames)
assert n == frames_nbytes, (n, frames_nbytes)
frames = memoryview(bytearray(frames_nbytes))
# Workaround for OpenSSL 1.0.2 (can drop with OpenSSL 1.1.1)
MAX = 1_000
for i, j in sliding_window(range(0, frame_nbytes + MAX, MAX)):
chunk = frames[i:j]
chunk_nbytes = len(chunk)
n = await stream.read_into(chunk)
assert n == chunk_nbytes, (n, chunk_nbytes)
except StreamClosedError as e:
self.stream = None
self._closed = True
Expand Down

0 comments on commit 974daa5

Please sign in to comment.