v1.5.2

librdkafka v1.5.2

librdkafka v1.5.2 is a maintenance release.

Upgrade considerations

• The default value for the producer configuration property retries has
  been increased from 2 to infinity, effectively limiting Produce retries to
  only message.timeout.ms.
  As the reasons for the automatic internal retries vary (various broker error
  codes as well as transport layer issues), it doesn't make much sense to limit
  the number of retries for retriable errors, but instead only limit the
  retries based on the allowed time to produce a message.
• The default value for the producer configuration property
  request.timeout.ms has been increased from 5 to 30 seconds to match
  the Apache Kafka Java producer default.
  This change yields increased robustness for broker-side congestion.

Enhancements

• The generated CONFIGURATION.md (through rd_kafka_conf_properties_show()))
  now include all properties and values, regardless if they were included in
  the build, and setting a disabled property or value through
  rd_kafka_conf_set() now returns RD_KAFKA_CONF_INVALID and provides
  a more useful error string saying why the property can't be set.
• Consumer configs on producers and vice versa will now be logged with
  warning messages on client instantiation.

Fixes

Security fixes

• There was an incorrect call to zlib's inflateGetHeader() with
  unitialized memory pointers that could lead to the GZIP header of a fetched
  message batch to be copied to arbitrary memory.
  This function call has now been completely removed since the result was
  not used.
  Reported by Ilja van Sprundel.

General fixes

• rd_kafka_topic_opaque() (used by the C++ API) would cause object
  refcounting issues when used on light-weight (error-only) topic objects
  such as consumer errors (#2693).
• Handle name resolution failures when formatting IP addresses in error logs,
  and increase printed hostname limit to ~256 bytes (was ~60).
• Broker sockets would be closed twice (thus leading to potential race
  condition with fd-reuse in other threads) if a custom socket_cb would
  return error.

Consumer fixes

• The roundrobin partition.assignment.strategy could crash (assert)
  for certain combinations of members and partitions.
  This is a regression in v1.5.0. (#3024)
• The C++ KafkaConsumer destructor did not destroy the underlying
  C rd_kafka_t instance, causing a leak if close() was not used.
• Expose rich error strings for C++ Consumer Message->errstr().
• The consumer could get stuck if an outstanding commit failed during
  rebalancing (#2933).
• Topic authorization errors during fetching are now reported only once (#3072).

Producer fixes

• Topic authorization errors are now properly propagated for produced messages,
  both through delivery reports and as ERR_TOPIC_AUTHORIZATION_FAILED
  return value from produce*() (#2215)
• Treat cluster authentication failures as fatal in the transactional
  producer (#2994).
• The transactional producer code did not properly reference-count partition
  objects which could in very rare circumstances lead to a use-after-free bug
  if a topic was deleted from the cluster when a transaction was using it.
• ERR_KAFKA_STORAGE_ERROR is now correctly treated as a retriable
  produce error (#3026).
• Messages that timed out locally would not fail the ongoing transaction.
  If the application did not take action on failed messages in its delivery
  report callback and went on to commit the transaction, the transaction would
  be successfully committed, simply omitting the failed messages.
• EndTxnRequests (sent on commit/abort) are only retried in allowed
  states (#3041).
  Previously the transaction could hang on commit_transaction() if an abortable
  error was hit and the EndTxnRequest was to be retried.

Note: there was no v1.5.1 librdkafka release

Checksums

Release asset checksums:

• v1.5.2.zip SHA256 de70ebdb74c7ef8c913e9a555e6985bcd4b96eb0c8904572f3c578808e0992e1
• v1.5.2.tar.gz SHA256 ca3db90d04ef81ca791e55e9eed67e004b547b7adedf11df6c24ac377d4840c6