KAFKA-14618; Fix off by one error in snapshot id#13108
Merged
cmccabe merged 1 commit intoapache:trunkfrom Jan 13, 2023
Merged
Conversation
jsancio
commented
Jan 13, 2023
| public OffsetAndEpoch offsetAndEpoch() { | ||
| return new OffsetAndEpoch(offset, epoch); | ||
| public OffsetAndEpoch snapshotId() { | ||
| return new OffsetAndEpoch(lastContainedOffset + 1, lastContainedEpoch); |
Member
Author
There was a problem hiding this comment.
This is the most important change.
Contributor
There was a problem hiding this comment.
Can you add a comment about how the snapshot ID contains an offset one greater than the last contained offset? It could otherwise be easy to miss.
Probably include a reference to this JIRA as well.
Member
Author
There was a problem hiding this comment.
Yes. I'll do this when I implement https://issues.apache.org/jira/browse/KAFKA-14620.
cmccabe
approved these changes
Jan 13, 2023
Contributor
cmccabe
left a comment
cmccabe
left a comment
There was a problem hiding this comment.
LGTM.
Can you file a JIRA for creating a test that would have caught this?
cmccabe
pushed a commit
that referenced
this pull request
Jan 13, 2023
The KRaft client expects the offset of the snapshot id to be an end offset. End offsets are exclusive. The MetadataProvenance type was createing a snapshot id using the last contained offset which is inclusive. This change fixes that and renames some of the fields to make this difference more obvious. Reviewers: Colin P. McCabe <cmccabe@apache.org>
Contributor
|
I filed a follow-up jira here https://issues.apache.org/jira/browse/KAFKA-14622 |
Member
Author
Yes. When I implement https://issues.apache.org/jira/browse/KAFKA-14619, we can reliably catch this kind of issues in JUnit integration tests. |
ijuma
added a commit
to confluentinc/kafka
that referenced
this pull request
Jan 17, 2023
…master * apache-github/trunk: (23 commits) MINOR: Include the inner exception stack trace when re-throwing an exception (apache#12229) MINOR: Fix docs to state that sendfile implemented in `TransferableRecords` instead of `MessageSet` (apache#13109) Update ProducerConfig.java (apache#13115) KAFKA-14618; Fix off by one error in snapshot id (apache#13108) KAFKA-13709 (follow-up): Avoid mention of 'exactly-once delivery' or 'delivery guarantees' in Connect (apache#13106) KAFKA-14367; Add `TxnOffsetCommit` to the new `GroupCoordinator` interface (apache#12901) KAFKA-14568: Move FetchDataInfo and related to storage module (apache#13085) KAFKA-14612: Make sure to write a new topics ConfigRecords to metadata log iff the topic is created (apache#13104) KAFKA-14601: Improve exception handling in KafkaEventQueue apache#13089 KAFKA-14367; Add `OffsetCommit` to the new `GroupCoordinator` interface (apache#12886) KAFKA-14530: Check state updater more often (apache#13017) KAFKA-14304 Use boolean for ZK migrating brokers in RPC/record (apache#13103) KAFKA-14003 Kafka Streams JUnit4 to JUnit5 migration part 2 (apache#12301) KAFKA-14607: Move Scheduler/KafkaScheduler to server-common (apache#13092) KAFKA-14367; Add `OffsetFetch` to the new `GroupCoordinator` interface (apache#12870) KAFKA-14557; Lock metadata log dir (apache#13058) MINOR: Implement toString method for TopicAssignment and PartitionAssignment (apache#13101) KAFKA-12558: Do not prematurely mutate internal partition state in Mirror Maker 2 (apache#11818) KAFKA-14540: Fix DataOutputStreamWritable#writeByteBuffer (apache#13032) KAFKA-14600: Reduce flakiness in ProducerIdExpirationTest (apache#13087) ...
guozhangwang
pushed a commit
to guozhangwang/kafka
that referenced
this pull request
Jan 25, 2023
The KRaft client expects the offset of the snapshot id to be an end offset. End offsets are exclusive. The MetadataProvenance type was createing a snapshot id using the last contained offset which is inclusive. This change fixes that and renames some of the fields to make this difference more obvious. Reviewers: Colin P. McCabe <cmccabe@apache.org>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The KRaft client expects the offset of the snapshot id to be an end offset. End offsets are exclusive. The MetadataProvenance type was createing a snapshot id using the last contained offset which is inclusive. This change fixes that and renames some of the fields to make this difference more obvious.
Committer Checklist (excluded from commit message)