Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix frame reader #436

Merged
merged 1 commit into from
Jul 27, 2021
Merged

Fix frame reader #436

merged 1 commit into from
Jul 27, 2021

Conversation

mdumandag
Copy link
Contributor

In the frame reader, we were storing the frame size including the
header (frame size + flags which is 6 bytes). Also, after reading
the frame size from the header, we were incrementing the bytes_read.

That approach works for small frames, that we can read with one go,
but was not working for frames that are large. For the same frame,
the next time we call read_frame, the length of the reader (the data
we haven't read from the buffer) was equal to frame size, but the
variable frame_size was equal to frame_size + HEADER_SIZE. So,
we were not reading the data that was actually there due to some
faulty length check.

We were able to read the data that is in the buffer after some long
time, when the response for the client ping request comes as it
was incrementing the length of the reader enough to pass the length
check.

To fix that, now, the frame_size field holds the frame size excluding
the header size.

closes #434

In the frame reader, we were storing the frame size including the
header (frame size + flags which is 6 bytes). Also, after reading
the frame size from the header, we were incrementing the bytes_read.

That approach works for small frames, that we can read with one go,
but was not working for frames that are large. For the same frame,
the next time we call read_frame, the length of the reader (the data
we haven't read from the buffer) was equal to frame size, but the
variable frame_size was equal to `frame_size + HEADER_SIZE`. So,
we were not reading the data that was actually there due to some
faulty length check.

We were able to read the data that is in the buffer after some long
time, when the response for the client ping request comes as it
was incrementing the length of the reader enough to pass the length
check.

To fix that, now, the frame_size field holds the frame size excluding
the header size.
@codecov-commenter
Copy link

Codecov Report

Merging #436 (37fbd20) into master (6e860c2) will decrease coverage by 0.18%.
The diff coverage is 88.88%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #436      +/-   ##
==========================================
- Coverage   86.61%   86.42%   -0.19%     
==========================================
  Files         336      336              
  Lines       17398    17398              
==========================================
- Hits        15070    15037      -33     
- Misses       2328     2361      +33     
Impacted Files Coverage Δ
hazelcast/connection.py 82.35% <88.88%> (-0.57%) ⬇️
hazelcast/reactor.py 77.51% <0.00%> (-6.23%) ⬇️
hazelcast/protocol/client_message.py 91.90% <0.00%> (-1.74%) ⬇️
hazelcast/invocation.py 80.52% <0.00%> (-0.75%) ⬇️
hazelcast/proxy/base.py 88.67% <0.00%> (-0.63%) ⬇️
hazelcast/listener.py 69.76% <0.00%> (+1.16%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6e860c2...37fbd20. Read the comment docs.

Copy link
Contributor

@yuce yuce left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@mdumandag
Copy link
Contributor Author

Thanks for the review Yüce

@mdumandag mdumandag merged commit 711f4a6 into hazelcast:master Jul 27, 2021
@mdumandag mdumandag deleted the frame-reader branch July 27, 2021 07:19
@mdumandag mdumandag modified the milestones: 5.0, 4.2.1 Jul 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Read in map extremely slow when dealing with large Payload
3 participants