KAFKA-17498: Reduce the number of remote calls when serving LIST_OFFSETS request #17132
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kamalcph
force-pushed
the
KAFKA-17498
branch
2 times, most recently
from
57e9bd8
to
2daef27
Compare
clolov
approved these changes
Sep 20, 2024
kamalcph
force-pushed
the
KAFKA-17498
branch
from
2daef27
to
96f6b49
Compare
showuon
reviewed
Sep 25, 2024
showuon
approved these changes
Sep 26, 2024
kamalcph
force-pushed
the
KAFKA-17498
branch
from
7bf3026
to
de3cd25
Compare
kamalcph
force-pushed
the
KAFKA-17498
branch
from
de3cd25
to
c8f8130
Compare
satishd
reviewed
Sep 30, 2024
| @@ -698,14 +700,24 @@ public Optional<FileRecords.TimestampAndOffset> findOffsetByTimestamp(TopicParti | |||
| && rlsMetadata.endOffset() >= startingOffset | |||
| && isRemoteSegmentWithinLeaderEpochs(rlsMetadata, unifiedLog.logEndOffset(), epochWithOffsets) | |||
| && rlsMetadata.state().equals(RemoteLogSegmentState.COPY_SEGMENT_FINISHED)) { | |||
| return lookupTimestamp(rlsMetadata, timestamp, startingOffset); | |||
| // cache to avoid race conditions | |||
| List<LogSegment> segmentsCopy = new ArrayList<>(unifiedLog.logSegments()); | |||
There was a problem hiding this comment.
Raised an edge case here where we may miss returning the right segment while searching for the target timestamp. This can happen when the search operation moves from remote segments to local segments search but the local segment is moved to remote storage and deleted from local segments successfully before it is searched successfully.
Thanks @kamalcph for raising the KAFKA-17637 as we discussed offline. This can be taken up in a follow-up PR.
satishd
approved these changes
Sep 30, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
If the index to be fetched exists locally, then avoid fetching the remote indexes to serve the LIST_OFFSETS request.
Committer Checklist (excluded from commit message)