Skip to content

Conversation

@justin-chen
Copy link
Contributor

Similar to: #472
OffsetCommitValue V4: apache/kafka#19683

For compatibility with Kafka 4.1.0, add decodeOffsetValueV4. Fixes decoding errors such as:

{
  "level": "warn",
  "ts": 1759190908.5072575,
  "msg": "failed to decode",
  "type": "module",
  "coordinator": "consumer",
  "class": "kafka",
  "name": "kafka-staging-us-east1-consumers",
  "offset_topic": "__consumer_offsets",
  "offset_partition": 31,
  "offset_offset": 120351075,
  "message_type": "offset",
  "group": "kafka-client-go-test-staging",
  "topic": "kafka_client_go_test",
  "partition": 1,
  "offset": 4025888141,
  "timestamp": 0,
  "reason": "metadata"
}

By decoding offset value metadata using compact string, we determine string length by reading with uint8 rather than int16, which should be the default:

COMPACT_STRING | Represents a sequence of characters. First the length N + 1 is given as an UNSIGNED_VARINT . Then N bytes follow which are the UTF-8 encoding of the character sequence.

@justin-chen
Copy link
Contributor Author

Addressed lint errors by adding constants for the errors.

@bai
Copy link
Collaborator

bai commented Sep 30, 2025

You rock! Thanks for your contribution!

@bai bai merged commit f928239 into linkedin:master Sep 30, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants