Version 1.41.0 (2023-08-21)

What's Changed

🚨 Breaking Changes

Note: this version of Sarama has had a big overhaul in its adherence to the use of the right Kafka protocol versions for the given Config Version. It has also bumped the default Version set in Config (where one is not supplied) to 2.1.0. This is in preparation for Kafka 4.0 dropping support for protocol versions older than 2.1. If you are using Sarama against Kafka clusters older than v2.1.0, or using it against Azure EventHubs then you will likely have to change your application code to pin to the appropriate Version.

• chore(config): make DefaultVersion V2_0_0_0 by @dnwe in #2572
• chore(config): make DefaultVersion V2_1_0_0 by @dnwe in #2574

🎉 New Features / Improvements

• Implement resolve_canonical_bootstrap_servers_only by @gebn in #2156
• feat: sleep when throttled (KIP-219) by @hindessm in #2536
• feat: add isValidVersion to protocol types by @dnwe in #2538
• fix(consumer): use newer LeaveGroup as appropriate by @dnwe in #2544
• Add support for up to version 4 List Groups API by @prestona in #2541
• fix(producer): use newer ProduceReq as appropriate by @dnwe in #2546
• fix(proto): ensure req+resp requiredVersion match by @dnwe in #2548
• chore(proto): permit CreatePartitionsRequest V1 by @dnwe in #2549
• chore(proto): permit AlterConfigsRequest V1 by @dnwe in #2550
• chore(proto): permit DeleteGroupsRequest V1 by @dnwe in #2551
• fix(proto): correct JoinGroup usage for wider version range by @dnwe in #2553
• fix(consumer): use full range of FetchRequest vers by @dnwe in #2554
• fix(proto): use range of OffsetCommitRequest vers by @dnwe in #2555
• fix(proto): use full range of MetadataRequest by @dnwe in #2556
• fix(proto): use fuller ranges of supported proto by @dnwe in #2558
• fix(proto): use full range of SyncGroupRequest by @dnwe in #2565
• fix(proto): use full range of ListGroupsRequest by @dnwe in #2568
• feat(proto): support for Metadata V6-V10 by @dnwe in #2566
• fix(proto): use full ranges for remaining proto by @dnwe in #2570
• feat(proto): add remaining protocol for V2.1 by @dnwe in #2573
• feat: add new error for MockDeleteTopicsResponse by @javiercri in #2475
• feat(gzip): switch to klauspost/compress gzip by @dnwe in #2600

🐛 Fixes

• fix: correct unsupported version check by @hindessm in #2528
• fix: avoiding burning cpu if all partitions are paused by @napallday in #2532
• extend throttling metric scope by @hindessm in #2533
• Fix printing of final metrics by @prestona in #2545
• fix(consumer): cannot automatically fetch newly-added partitions unless restart by @napallday in #2563
• bug: implement unsigned modulus for partitioning with crc32 hashing by @csm8118 in #2560
• fix: avoid logging value of proxy.Dialer by @prestona in #2569
• fix(test): add missing closes to admin client tests by @dnwe in #2594
• fix(test): ensure some more clients are closed by @dnwe in #2595
• fix(examples): sync exactly_once and consumergroup by @dnwe in #2614
• fix(fvt): fresh metrics registry for each test by @dnwe in #2616
• fix(test): flaky test TestFuncOffsetManager by @napallday in #2609

📦 Dependency updates

• chore(deps): bump the golang-org-x group with 1 update by @dependabot in #2542
• chore(deps): bump the golang-org-x group with 1 update by @dependabot in #2561
• chore(deps): bump module github.com/pierrec/lz4/v4 to v4.1.18 by @dnwe in #2589
• chore(deps): bump module github.com/jcmturner/gokrb5/v8 to v8.4.4 by @dnwe in #2587
• chore(deps): bump github.com/eapache/go-xerial-snappy digest to c322873 by @dnwe in #2586
• chore(deps): bump module github.com/klauspost/compress to v1.16.7 by @dnwe in #2588
• chore(deps): bump github.com/eapache/go-resiliency from 1.3.0 to 1.4.0 by @dependabot in #2598

🔧 Maintenance

• fix(fvt): ensure fully-replicated at test start by @hindessm in #2531
• chore: rollup fvt kafka to latest three by @dnwe in #2537
• Merge the two CONTRIBUTING.md's by @prestona in #2543
• fix(test): test timing error by @hindessm in #2552
• chore(ci): tidyup and improve actions workflows by @dnwe in #2557
• fix(test): shutdown MockBroker by @dnwe in #2571
• chore(proto): match HeartbeatResponse version by @dnwe in #2576
• chore(test): ensure MockBroker closed within test by @dnwe in #2575
• chore(test): ensure all mockresponses use version by @dnwe in #2578
• chore(ci): use latest Go in actions by @dnwe in #2580
• chore(test): speedup some slow tests by @dnwe in #2579
• chore(test): use modern protocol versions in FVT by @dnwe in #2581
• chore(test): fix a couple of leaks by @dnwe in #2591
• feat(fvt): experiment with per-kafka-version image by @dnwe in #2592
• chore(ci): replace toxiproxy client dep by @dnwe in #2593
• feat(fvt): add healthcheck, depends_on and --wait by @dnwe in #2601
• fix(fvt): handle msgset vs batchset by @dnwe in #2603
• fix(fvt): Metadata version in ensureFullyReplicated by @dnwe in #2612
• fix(fvt): versioned cfg for invalid topic producer by @dnwe in #2613
• chore(fvt): tweak to work across more versions by @dnwe in #2615
• feat(fvt): test wider range of kafkas by @dnwe in #2605

📝 Documentation

• fix(example): check if msg channel is closed by @ioanzicu in #2479
• chore: use go install for installing sarama tools by @vigith in #2599

New Contributors

• @gebn made their first contribution in #2156
• @prestona made their first contribution in #2543
• @ioanzicu made their first contribution in #2479
• @csm8118 made their first contribution in #2560
• @javiercri made their first contribution in #2475
• @vigith made their first contribution in #2599

Full Changelog: v1.40.1...v1.41.0

Version 1.40.1 (2023-07-27)

What's Changed

🎉 New Features / Improvements

• Use buffer pools for decompression by @ronanh in #2484
• feat: support for Kerberos authentication with a credentials cache. by @mrogaski in #2457

🐛 Fixes

• Fix some retry issues by @hindessm in #2517
• fix: admin retry logic by @hindessm in #2519
• Add some retry logic to more admin client functions by @hindessm in #2520
• fix: concurrent issue on updateMetadataMs by @napallday in #2522
• fix(test): allow testing of skipped test without IsTransactional panic by @hindessm in #2525

📦 Dependency updates

• chore(deps): bump the golang-org-x group with 2 updates by @dependabot in #2509
• chore(deps): bump github.com/klauspost/compress from 1.15.14 to 1.16.6 by @dependabot in #2513
• chore(deps): bump github.com/stretchr/testify from 1.8.1 to 1.8.3 by @dependabot in #2512

🔧 Maintenance

• chore(ci): migrate probot-stale to actions/stale by @dnwe in #2496
• chore(ci): bump golangci version, cleanup, depguard config by @EladLeev in #2504
• Clean up some typos and docs/help mistakes by @hindessm in #2514

➕ Other Changes

• chore(ci): add simple apidiff workflow by @dnwe in #2497
• chore(ci): bump actions/setup-go from 3 to 4 by @dependabot in #2508
• fix(comments): PauseAll and ResumeAll by @napallday in #2523

New Contributors

• @EladLeev made their first contribution in #2504
• @hindessm made their first contribution in #2514
• @ronanh made their first contribution in #2484
• @mrogaski made their first contribution in #2457

Full Changelog: v1.40.0...v1.40.1

Version 1.40.0 (2023-07-17)

What's Changed

Note: this is the first release after the transition of Sarama ownership from Shopify to IBM in #2461

🚨 Breaking Changes

• chore: migrate module to github.com/IBM/sarama by @dnwe in #2492
• fix: restore (*OffsetCommitRequest) AddBlock func by @dnwe in #2494

🐛 Fixes

• fix(consumer): don't retry FindCoordinator forever by @dnwe in #2427
• fix(metrics): fix race condition when calling Broker.Open() twice by @vincentbernat in #2428
• fix: use version 4 of DescribeGroupsRequest only if kafka broker vers… …ion is >= 2.4 by @faillefer in #2451
• Fix HighWaterMarkOffset of mocks partition consumer by @gr8web in #2447
• fix: prevent data race in balance strategy by @napallday in #2453

📦 Dependency updates

• chore(deps): bump golang.org/x/net from 0.5.0 to 0.7.0 by @dependabot in #2452

🔧 Maintenance

• chore: add kafka 3.3.2 by @dnwe in #2434
• chore(ci): remove Shopify/shopify-cla-action by @dnwe in #2489
• chore: bytes.Equal instead bytes.Compare by @testwill in #2485

New Contributors

• @dependabot made their first contribution in #2452
• @gr8web made their first contribution in #2447
• @testwill made their first contribution in #2485

Full Changelog: v1.38.1...v1.40.0

Version 1.38.1 (2023-01-22)

What's Changed

🐛 Fixes

• fix(example): correct records-number param in txn producer readme by @diallo-han in #2420
• fix: use newConsumer method in newConsumerGroup method by @Lumotheninja in #2424

📦 Dependency updates

• chore(deps): bump module github.com/klauspost/compress to v1.15.14 by @dnwe in #2410
• chore(deps): bump module golang.org/x/net to v0.5.0 by @dnwe in #2413
• chore(deps): bump module github.com/stretchr/testify to v1.8.1 by @dnwe in #2411
• chore(deps): bump module github.com/xdg-go/scram to v1.1.2 by @dnwe in #2412
• chore(deps): bump module golang.org/x/sync to v0.1.0 by @dnwe in #2414
• chore(deps): bump github.com/eapache/go-xerial-snappy digest to bf00bc1 by @dnwe in #2418

New Contributors

• @diallo-han made their first contribution in #2420
• @Lumotheninja made their first contribution in #2424

Full Changelog: v1.38.0...v1.38.1

Version 1.38.0 (2023-01-08)

What's Changed

🎉 New Features / Improvements

• feat(producer): improve memory usage of zstd encoder by using our own pool management by @rtreffer in #2375
• feat(proto): implement and use MetadataRequest v7 by @dnwe in #2388
• feat(metrics): add protocol-requests-rate metric by @auntan in #2373

🐛 Fixes

• fix(proto): track and supply leader epoch to FetchRequest by @dnwe in #2389
• fix(example): improve arg name used for tls skip verify by @michaeljmarshall in #2385
• fix(zstd): default back to GOMAXPROCS concurrency by @bgreenlee in #2404
• fix(producer): add nil check while producer is retrying by @hsweif in #2387
• fix(producer): return errors for every message in retryBatch to avoid producer hang forever by @cch123 in #2378
• fix(metrics): fix race when accessing metric registry by @vincentbernat in #2409

📦 Dependency updates

• chore(deps): bump golang.org/x/net to v0.4.0 by @dnwe in #2403

🔧 Maintenance

• chore(ci): replace set-output command in GH Action by @dnwe in #2390
• chore(ci): include kafka 3.3.1 in testing matrix by @dnwe in #2406

New Contributors

• @michaeljmarshall made their first contribution in #2385
• @bgreenlee made their first contribution in #2404
• @hsweif made their first contribution in #2387
• @cch123 made their first contribution in #2378

Full Changelog: v1.37.2...v1.38.0

Version 1.37.2 (2022-10-04)

What's Changed

🐛 Fixes

• fix: ensure updateMetaDataMs is 64-bit aligned by @dnwe in #2356

➕ Other Changes

• fix: bump go.mod specification to go 1.17 by @dnwe in #2357

Full Changelog: v1.37.1...v1.37.2

Version 1.37.1 (2022-10-04)

What's Changed

🐛 Fixes

• fix: support existing deprecated Rebalance.Strategy field usage by @spongecaptain in #2352
• fix(test): consumer group rebalance strategy compatibility by @Jacob-bzx in #2353
• fix(producer): replace time.After with time.Timer to avoid high memory usage by @Jacob-bzx in #2355

New Contributors

• @spongecaptain made their first contribution in #2352

Full Changelog: v1.37.0...v1.37.1

Version 1.37.0 (2022-09-28)

What's Changed

🚨 Breaking Changes

• Due to a change in github.com/klauspost/compress v1.15.10, Sarama v1.37.0 requires Go 1.17 going forward, unfortunately due to an oversight this wasn't reflected in the go.mod declaration at time of release.

🎉 New Features / Improvements

• feat(consumer): support multiple balance strategies by @Jacob-bzx in #2339
• feat(producer): transactional API by @ryarnyah in #2295
• feat(mocks): support key in MockFetchResponse. by @Skandalik in #2328

🐛 Fixes

• fix: avoid panic when Metadata.RefreshFrequency is 0 by @Jacob-bzx in #2329
• fix(consumer): avoid pushing unrelated responses to paused children by @pkoutsovasilis in #2317
• fix: prevent metrics leak with cleanup by @auntan in #2340
• fix: race condition(may panic) when closing consumer group by @Jacob-bzx in #2331
• fix(consumer): default ResetInvalidOffsets to true by @dnwe in #2345
• Validate the Config when creating a mock producer/consumer by @joewreschnig in #2327

📦 Dependency updates

• chore(deps): bump module github.com/pierrec/lz4/v4 to v4.1.16 by @dnwe in #2335
• chore(deps): bump golang.org/x/net digest to bea034e by @dnwe in #2333
• chore(deps): bump golang.org/x/sync digest to 7f9b162 by @dnwe in #2334
• chore(deps): bump golang.org/x/net digest to f486391 by @dnwe in #2348
• chore(deps): bump module github.com/shopify/toxiproxy/v2 to v2.5.0 by @dnwe in #2336
• chore(deps): bump module github.com/klauspost/compress to v1.15.11 by @dnwe in #2349
• chore(deps): bump module github.com/pierrec/lz4/v4 to v4.1.17 by @dnwe in #2350

🔧 Maintenance

• chore(ci): bump kafka-versions to latest by @dnwe in #2346
• chore(ci): bump go-versions to N and N-1 by @dnwe in #2347

New Contributors

• @Jacob-bzx made their first contribution in #2329
• @pkoutsovasilis made their first contribution in #2317
• @Skandalik made their first contribution in #2328
• @auntan made their first contribution in #2340
• @ryarnyah made their first contribution in #2295

Full Changelog: v1.36.0...v1.37.0

Version 1.36.0 (2022-08-11)

What's Changed

🎉 New Features / Improvements

• feat: add option to propagate OffsetOutOfRange error by @dkolistratova in #2252
• feat(producer): expose ProducerMessage.byteSize() function by @k8scat in #2315
• feat(metrics): track consumer fetch request rates by @dnwe in #2299

🐛 Fixes

• fix(consumer): avoid submitting empty fetch requests when paused by @raulnegreiros in #2143

📦 Dependency updates

• chore(deps): bump module github.com/klauspost/compress to v1.15.9 by @dnwe in #2304
• chore(deps): bump golang.org/x/net digest to c7608f3 by @dnwe in #2301
• chore(deps): bump golangci/golangci-lint-action action to v3 by @dnwe in #2311
• chore(deps): bump golang.org/x/net digest to 07c6da5 by @dnwe in #2307
• chore(deps): bump github actions versions (major) by @dnwe in #2313
• chore(deps): bump module github.com/jcmturner/gofork to v1.7.6 by @dnwe in #2305
• chore(deps): bump golang.org/x/sync digest to 886fb93 by @dnwe in #2302
• chore(deps): bump module github.com/jcmturner/gokrb5/v8 to v8.4.3 by @dnwe in #2303

🔧 Maintenance

• chore: add kafka 3.1.1 to the version matrix by @dnwe in #2300

➕ Other Changes

• Migrate off probot-CLA to new GitHub Action by @cursedcoder in #2294
• Forgot to remove cla probot by @cursedcoder in #2297
• chore(lint): re-enable a small amount of go-critic by @dnwe in #2312

New Contributors

• @cursedcoder made their first contribution in #2294
• @dkolistratova made their first contribution in #2252
• @k8scat made their first contribution in #2315

Full Changelog: v1.35.0...v1.36.0

Version 1.35.0 (2022-07-22)

What's Changed

🐛 Fixes

• fix: fix metadata retry backoff invalid when get metadata failed by @Stephan14 in #2256
• fix(balance): sort and de-deplicate memberIDs by @dnwe in #2285
• fix: prevent DescribeLogDirs hang in admin client by @zerowidth in #2269
• fix: include assignment-less members in SyncGroup by @dnwe in #2292

📦 Dependency updates

• chore(deps): bump module github.com/stretchr/testify to v1.8.0 by @dnwe in #2284
• chore(deps): bump module github.com/eapache/go-resiliency to v1.3.0 by @dnwe in #2283
• chore(deps): bump golang.org/x/net digest to 1185a90 by @dnwe in #2279
• chore(deps): bump module github.com/pierrec/lz4/v4 to v4.1.15 by @dnwe in #2281
• chore(deps): bump module github.com/klauspost/compress to v1.15.8 by @dnwe in #2280

🔧 Maintenance

• chore: rename any func to avoid identifier by @dnwe in #2272
• chore: add and test against kafka 3.2.0 by @dnwe in #2288
• chore: document Fetch protocol fields by @dnwe in #2289

➕ Other Changes

• chore(ci): fix redirect with GITHUB_STEP_SUMMARY by @dnwe in #2286
• fix(test): permit ECONNRESET in TestInitProducerID by @dnwe in #2287
• fix: ensure empty or devel version valid by @dnwe in #2291

New Contributors

• @zerowidth made their first contribution in #2269

Full Changelog: v1.34.1...v1.35.0