Skip to content

Commit

Permalink
kgo: change default MaxBufferedRecords from unlimited to 10,000
Browse files Browse the repository at this point in the history
10k was the original limit. I'm not sure when this was bumped to
unlimited, but it is problematic when producing to topics that do not
exist. 10k is still a good limit.
  • Loading branch information
twmb committed Oct 11, 2021
1 parent c828190 commit b9b592e
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions pkg/kgo/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -451,7 +451,7 @@ func defaultCfg() cfg {
acks: AllISRAcks(),
compression: []CompressionCodec{SnappyCompression(), NoCompression()},
maxRecordBatchBytes: 1000000, // Kafka max.message.bytes default is 1000012
maxBufferedRecords: math.MaxInt64,
maxBufferedRecords: 10000,
produceTimeout: 30 * time.Second,
recordRetries: math.MaxInt64, // effectively unbounded
partitioner: StickyKeyPartitioner(nil), // default to how Kafka partitions
Expand Down Expand Up @@ -859,7 +859,7 @@ func ProducerBatchMaxBytes(v int32) ProducerOpt {

// MaxBufferedRecords sets the max amount of records the client will buffer,
// blocking produces until records are finished if this limit is reached.
// This overrides the unbounded default.
// This overrides the default of 10,000.
func MaxBufferedRecords(n int) ProducerOpt {
return producerOpt{func(cfg *cfg) { cfg.maxBufferedRecords = int64(n) }}
}
Expand Down

0 comments on commit b9b592e

Please sign in to comment.