v4.38.0

For installation instructions check out the getting started guide.

Added

• Anonymous telemetry data is now sent by Connect instances after running for >5 mins. Details about which data is sent, when it is sent, and how to disable it can be found in the telemetry README. (@Jeffail)
• Field checksum_algorithm added to the aws_s3 output. (@dom-lee-naimuri)
• Field nkey added to nats, nats_jetstream, nats_kv and nats_stream components. (@ye11ow)
• Field private_key added to the snowflake_put output. (@mihaitodor)
• New azure_data_lake_gen2 output. (@ooesili)
• New timeplus output. (@ye11ow)

Fixed

• The elasticsearch output now performs retries for HTTP status code 429 (Too Many Requests). (@kahoowkh)
• The docs for the collection field of the mongodb output now specify support for interpolation functions. (@mihaitodor)

Changed

• All components with a default path field value (such as the aws_s3 output) containing the deprecated function count have now been changed to use the new function counter. This could potentially change behaviour in cases where multiple components are executing a mapping with a count function sharing the same of the old default count, and these counters need to cascade. This is an extremely unlikely scenario, but for all users of these components it is recommended that your path is defined explicitly, and in a future major version we will be removing the defaults.

The full change log can be found here.

v4.38.0-rc5

For installation instructions check out the getting started guide.

Added

• Anonymous telemetry data is now sent by Connect instances after running for >5 mins. Details about which data is sent, when it is sent, and how to disable it can be found in the telemetry README. (@Jeffail)
• Field checksum_algorithm added to the aws_s3 output. (@dom-lee-naimuri)
• Field nkey added to nats, nats_jetstream, nats_kv and nats_stream components. (@ye11ow)
• Field private_key added to the snowflake_put output. (@mihaitodor)
• New azure_data_lake_gen2 output. (@ooesili)
• New timeplus output. (@ye11ow)

Fixed

• The elasticsearch output now performs retries for HTTP status code 429 (Too Many Requests). (@kahoowkh)
• The docs for the collection field of the mongodb output now specify support for interpolation functions. (@mihaitodor)

Changed

• All components with a default path field value (such as the aws_s3 output) containing the deprecated function count have now been changed to use the new function counter. This could potentially change behaviour in cases where multiple components are executing a mapping with a count function sharing the same of the old default count, and these counters need to cascade. This is an extremely unlikely scenario, but for all users of these components it is recommended that your path is defined explicitly, and in a future major version we will be removing the defaults.

The full change log can be found here.

v4.38.0-rc4

For installation instructions check out the getting started guide.

Added

• Anonymous telemetry data is now sent by Connect instances after running for >5 mins. Details about which data is sent, when it is sent, and how to disable it can be found in the telemetry README. (@Jeffail)
• Field checksum_algorithm added to the aws_s3 output. (@dom-lee-naimuri)
• Field nkey added to nats, nats_jetstream, nats_kv and nats_stream components. (@ye11ow)
• Field private_key added to the snowflake_put output. (@mihaitodor)
• New azure_data_lake_gen2 output. (@ooesili)
• New timeplus output. (@ye11ow)

Fixed

• The elasticsearch output now performs retries for HTTP status code 429 (Too Many Requests). (@kahoowkh)
• The docs for the collection field of the mongodb output now specify support for interpolation functions. (@mihaitodor)

Changed

• All components with a default path field value (such as the aws_s3 output) containing the deprecated function count have now been changed to use the new function counter. This could potentially change behaviour in cases where multiple components are executing a mapping with a count function sharing the same of the old default count, and these counters need to cascade. This is an extremely unlikely scenario, but for all users of these components it is recommended that your path is defined explicitly, and in a future major version we will be removing the defaults.

The full change log can be found here.

v4.38.0-rc3

For installation instructions check out the getting started guide.

NOTE: This is a release candidate, you can download a binary from this page.

Added

• Field checksum_algorithm added to the aws_s3 output. (@dom-lee-naimuri)
• Field nkey added to nats, nats_jetstream, nats_kv and nats_stream components. (@ye11ow)
• Field private_key added to the snowflake_put output. (@mihaitodor)

Fixed

• The elasticsearch output now performs retries for HTTP status code 429 (Too Many Requests). (@kahoowkh)
• The docs for the collection field of the mongodb output now specify support for interpolation functions. (@mihaitodor)

The full change log can be found here.

v4.37.0

For installation instructions check out the getting started guide.

Added

• New experimental gcp_vertex_ai_embeddings processor. (@rockwotj)
• New experimental aws_bedrock_embeddings processor. (@rockwotj)
• New experimental cohere_chat and cohere_embeddings processors. (@rockwotj)
• New experimental questdb output. (@sklarsa)
• Field metadata_max_age added to the kafka_franz input. (@Scarjit)
• Field metadata_max_age added to the kafka_migrator input. (@mihaitodor)
• New experimental cypher output. (@rockwotj)
• New experimental couchbase output. (@rockwotj)
• Field fetch_in_order added to the schema_registry input. (@mihaitodor)

Fixed

• Fixed a bug with the input_resource field for the kafka_migrator output where new topics weren't created as expected. (@mihaitodor)
• Fixed a bug in the kafka_migrator input which could lead to extra duplicate messages during a consumer group rebalance. (@mihaitodor)
• kafka_migrator, kafka_migrator_offsets and kafka_migrator_bundle components renamed to redpanda_migrator, redpanda_migrator_offsets and redpanda_migrator_bundle (@mihaitodor)

Fixed

• Fixes a panic in the parquet_encode processor (@mihaitodor)

The full change log can be found here.

v4.37.0-rc2

For installation instructions check out the getting started guide.

Added

• New experimental gcp_vertex_ai_embeddings processor. (@rockwotj)
• New experimental aws_bedrock_embeddings processor. (@rockwotj)
• New experimental cohere_chat and cohere_embeddings processors. (@rockwotj)
• New experimental questdb output. (@sklarsa)
• Field metadata_max_age added to the kafka_franz input. (@Scarjit)
• Field metadata_max_age added to the kafka_migrator input. (@mihaitodor)
• New experimental cypher output. (@rockwotj)
• New experimental couchbase output. (@rockwotj)
• Field fetch_in_order added to the schema_registry input. (@mihaitodor)

Fixed

• Fixed a bug with the input_resource field for the kafka_migrator output where new topics weren't created as expected. (@mihaitodor)
• Fixed a bug in the kafka_migrator input which could lead to extra duplicate messages during a consumer group rebalance. (@mihaitodor)
• kafka_migrator, kafka_migrator_offsets and kafka_migrator_bundle components renamed to redpanda_migrator, redpanda_migrator_offsets and redpanda_migrator_bundle (@mihaitodor)

Fixed

• Fixes a panic in the parquet_encode processor (@mihaitodor)

The full change log can be found here.

v4.36.1-rc1

For installation instructions check out the getting started guide.

NOTE: This is a release candidate, you can download a binary from this page.

Added

• New experimental gcp_vertex_ai_embeddings processor. (@rockwotj)
• New experimental aws_bedrock_embeddings processor. (@rockwotj)
• New experimental cohere_chat and cohere_embeddings processors. (@rockwotj)
• New experimental questdb output. (@sklarsa)

The full change log can be found here.

v4.36.0

For installation instructions check out the getting started guide.

Added

• Fields replication_factor and replication_factor_override added to the kafka_migrator input and output. (@mihaitodor)

Fixed

• The schema_registry_encode and schema_registry_decode processors no longer unescape path separators in the schema name. (@Mizaro)
• (Benthos) The switch output metrics now emit the case id as part of their labels. This is a regression introduced in v4.25.0. (@mihaitodor)
• (Benthos) Fixed a bug where certain logs used the %w verb to print errors resulting in incorrect output. (@mihaitodor)
• (Benthos) The logger no longer tries to replace Go fmt verbs in log messages. (@mihaitodor)

The full change log can be found here.

v4.36.0-rc1

For installation instructions check out the getting started guide.

Added

• Fields replication_factor and replication_factor_override added to the kafka_migrator input and output. (@mihaitodor)

Fixed

• The schema_registry_encode and schema_registry_decode processors no longer unescape path separators in the schema name. (@Mizaro)
• (Benthos) The switch output metrics now emit the case id as part of their labels. This is a regression introduced in v4.25.0. (@mihaitodor)
• (Benthos) Fixed a bug where certain logs used the %w verb to print errors resulting in incorrect output. (@mihaitodor)
• (Benthos) The logger no longer tries to replace Go fmt verbs in log messages. (@mihaitodor)

The full change log can be found here.

v4.35.3

For installation instructions check out the getting started guide.

Added

• Azure and GCP components added to cloud builds. (@Jeffail)

Fixed

• The kafka_migrator_bundle input and output no longer require schema registry to be configured. (@mihaitodor)

The full change log can be found here.