Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update relative offset resolution to be saturating #1210

Closed
nicholastmosher opened this issue Jul 7, 2021 · 1 comment
Closed

Update relative offset resolution to be saturating #1210

nicholastmosher opened this issue Jul 7, 2021 · 1 comment
Assignees
Labels
bug Something isn't working Client consumer
Milestone

Comments

@nicholastmosher
Copy link
Contributor

Right now, using relative offsets Offset::from_beginning(X) or Offset::from_end(X) when fetching from a partition have undesirable behavior when X is greater than the number of records in that partition. When this happens, a stream request will just hang. Instead, I think that the relative offset resolution should be saturating, meaning that:

If n is the size of the partition, then

  • When we fetch using Offset::from_end(X) and X > n, then the fetch should resolve to absolute offset 0
  • When we fetch using Offset::from_beginning(X) and X > n, then the fetch should resolve to absolute offset X-1

The end result of this is that no call to fetch should ever cause the client to hang.

Related: #1007

@sehz sehz added this to the 0.8.5 milestone Jul 8, 2021
@sehz sehz added the bug Something isn't working label Jul 8, 2021
@sehz sehz removed this from the 0.8.5 milestone Jul 18, 2021
@sehz sehz added this to the 0.9.3 milestone Aug 16, 2021
@sehz
Copy link
Contributor

sehz commented Aug 16, 2021

find root cause

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Client consumer
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants