Skip to content

http: default HTTP User-Agent header with version information#11434

Open
endersonmaia wants to merge 34 commits intofluent:masterfrom
endersonmaia:feat/add-http-agent-header
Open

http: default HTTP User-Agent header with version information#11434
endersonmaia wants to merge 34 commits intofluent:masterfrom
endersonmaia:feat/add-http-agent-header

Conversation

@endersonmaia
Copy link
Contributor

@endersonmaia endersonmaia commented Feb 5, 2026

This will implement a standard User-Agent header for all http requests.

For ex.:

User-Agent: Fluent-Bit/5.0.0 (Git commit: e4466f5114bdff387774d6998666d5f60fb801da)

Ideally, I'd like to have something like this with os/arch and plugin information:

User-Agent: Fluent-Bit/5.0.0 (Linux/amd64, input: in_prometheus_scrape, Git commit: e4466f5114bdff387774d6998666d5f60fb801da)

Tell me if I should work on something like that for this PR or another.

Fixes #8880

The issue was closed beacause it was stale. Please open it please.


Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

  • Example configuration file for the change
  • Debug log output from testing the change
  • Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

  • Run local packaging test showing all targets (including any new ones) build.
  • Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

  • Documentation required for this feature

Backporting

  • Backport to latest stable release.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

Summary by CodeRabbit

  • New Features

    • Standardized User-Agent header added to outgoing HTTP requests, including product version and git commit.
  • Bug Fixes

    • User-Agent is now applied consistently across inputs, outputs, and plugins so requests carry uniform client metadata.

@coderabbitai
Copy link

coderabbitai bot commented Feb 5, 2026

📝 Walkthrough

Walkthrough

Adds centralized User-Agent header macros and includes version info in the public HTTP client header; replaces ad-hoc "User-Agent"/"Fluent-Bit" literals across core HTTP client, AWS utils, many plugins, and tests. Some plugins now add an explicit User-Agent header where previously absent. (47 words)

Changes

Cohort / File(s) Summary
Header constants
include/fluent-bit/flb_http_client.h
Add FLB_HTTP_HEADER_USER_AGENT, FLB_HTTP_HEADER_USER_AGENT_DEFAULT, and include flb_version.h.
Core HTTP & AWS utils
src/flb_http_client.c, src/aws/flb_aws_util.c
Replace hard-coded "User-Agent"/"Fluent-Bit" with FLB_HTTP_HEADER_USER_AGENT / FLB_HTTP_HEADER_USER_AGENT_DEFAULT, using sizeof(...)-1 for lengths.
Input & filter plugins
plugins/in_*/*, plugins/filter_*/* e.g. plugins/in_prometheus_scrape/prom_scrape.c, plugins/in_nginx_exporter_metrics/nginx.c, plugins/filter_kubernetes/kube_meta.c, plugins/filter_ecs/ecs.c, plugins/filter_nightfall/nightfall_api.c
Swap literal User-Agent header usage for centralized macros; compute lengths with sizeof(...)-1.
Output plugins (bulk replacements)
plugins/out_*/* e.g. plugins/out_http/http.c, plugins/out_azure/*, plugins/out_s3/s3.c, plugins/out_opensearch/opensearch.c, plugins/out_bigquery/bigquery.c, plugins/out_prometheus_remote_write/remote_write.c, plugins/out_influxdb/influxdb.c
Replace literal User-Agent name/value with new macros and length calculations across many outputs.
Azure / Kusto / Blob specific
plugins/out_azure_kusto/..., plugins/out_azure_blob/..., plugins/out_azure/azure.c
Multiple call sites updated to use centralized User-Agent macros and sizeof-based lengths.
Plugins adding User-Agent where absent
plugins/in_calyptia_fleet/in_calyptia_fleet.c, plugins/out_td/td.c, plugins/out_websocket/websocket.c, plugins/in_nginx_exporter_metrics/nginx.c
Insert explicit User-Agent header into outgoing HTTP requests using the new macros.
Other multi-site replacements
plugins/out_chronicle/chronicle.c, plugins/out_datadog/datadog.c, plugins/out_splunk/splunk.c, plugins/out_logdna/logdna.c, plugins/out_loki/loki.c, plugins/out_nrlogs/newrelic.c, plugins/out_opentelemetry/opentelemetry.c, plugins/out_skywalking/skywalking.c, plugins/out_slack/slack.c, plugins/out_kafka_rest/kafka.c, plugins/out_stackdriver/*
Consistent replacement of literal User-Agent with FLB_HTTP_HEADER_USER_AGENT and default macro, adjusting lengths.
Tests
tests/internal/http_client.c
Update test to use FLB_HTTP_HEADER_USER_AGENT and length via sizeof(...)-1.

Sequence Diagram(s)

(omitted)

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Suggested reviewers

  • edsiper
  • cosmo0920
  • PettitWesley
  • braydonk

Poem

🐰 I hop through headers, neat and spry,
Replacing strings with macros high,
User-Agent now wears version and hash,
I nibble literals in a tidy dash,
Requests march out with a new badge — hooray!

🚥 Pre-merge checks | ✅ 4 | ❌ 1
❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 16.67% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title clearly and concisely summarizes the main change: adding a default HTTP User-Agent header with version information to all HTTP requests made by Fluent Bit.
Linked Issues check ✅ Passed The PR fully addresses the requirements from issue #8880: implementing a User-Agent header for all HTTP requests. The header includes version information as shown in the implementation (FLB_HTTP_HEADER_USER_AGENT_DEFAULT with Git commit details).
Out of Scope Changes check ✅ Passed All changes are in scope and directly support adding User-Agent headers: new header constants defined, header added to all HTTP client call sites across plugins, and tests updated. No unrelated refactoring or feature creep detected.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e4466f5114

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
plugins/out_websocket/websocket.c (1)

70-84: ⚠️ Potential issue | 🟠 Major

User‑configured User-Agent gets overridden.

Because flb_http_add_header de‑duplicates by key, adding the default header after config headers will replace any user‑provided User-Agent. This breaks override expectations and is inconsistent with other plugins.

♻️ Suggested ordering to allow user overrides
-    /* Append additional headers from configuration */
-    flb_config_map_foreach(head, mv, ctx->headers) {
-        key = mk_list_entry_first(mv->val.list, struct flb_slist_entry, _head);
-        val = mk_list_entry_last(mv->val.list, struct flb_slist_entry, _head);
-
-        flb_http_add_header(c,
-                            key->str, flb_sds_len(key->str),
-                            val->str, flb_sds_len(val->str));
-    }
-
-    /* User-Agent */
-    flb_http_add_header(c, FLB_HTTP_HEADER_USER_AGENT,
-                        sizeof(FLB_HTTP_HEADER_USER_AGENT) - 1,
-                        FLB_HTTP_HEADER_USER_AGENT_DEFAULT,
-                        sizeof(FLB_HTTP_HEADER_USER_AGENT_DEFAULT) - 1);
+    /* User-Agent (default; can be overridden by configured headers) */
+    flb_http_add_header(c, FLB_HTTP_HEADER_USER_AGENT,
+                        sizeof(FLB_HTTP_HEADER_USER_AGENT) - 1,
+                        FLB_HTTP_HEADER_USER_AGENT_DEFAULT,
+                        sizeof(FLB_HTTP_HEADER_USER_AGENT_DEFAULT) - 1);
+
+    /* Append additional headers from configuration */
+    flb_config_map_foreach(head, mv, ctx->headers) {
+        key = mk_list_entry_first(mv->val.list, struct flb_slist_entry, _head);
+        val = mk_list_entry_last(mv->val.list, struct flb_slist_entry, _head);
+
+        flb_http_add_header(c,
+                            key->str, flb_sds_len(key->str),
+                            val->str, flb_sds_len(val->str));
+    }
🤖 Fix all issues with AI agents
In `@plugins/in_nginx_exporter_metrics/nginx.c`:
- Around line 136-141: The call to flb_http_add_header(…,
FLB_HTTP_HEADER_USER_AGENT, …) must have its return value checked and handled;
if it fails (e.g., allocation error) abort building/sending the request instead
of continuing without the header. Update the code around the flb_http_add_header
invocation to test the return value, log an error, free/cleanup any
request/client resources as done on other failures, and return an
error/early-exit from the enclosing function (or propagate an error code) so the
request is not sent without the User-Agent header.

In `@plugins/out_http/http.c`:
- Around line 270-274: The default User-Agent header is being added after
append_headers which causes per-record User-Agent values to be replaced or
duplicated; move the flb_http_add_header call that adds
FLB_HTTP_HEADER_USER_AGENT_DEFAULT to occur before append_headers is invoked so
that any user-supplied User-Agent in append_headers can override it (ensure this
change respects existing allow_duplicated_headers logic and update any
surrounding comments accordingly).

In `@plugins/out_td/td.c`:
- Around line 180-184: Check the return value of flb_http_add_header when adding
FLB_HTTP_HEADER_USER_AGENT/FLB_HTTP_HEADER_USER_AGENT_DEFAULT; if it fails, log
an error mentioning the header insertion failed, perform necessary cleanup of
the HTTP request/context (free any allocated request object and related
resources), and short‑circuit by returning an error code from the surrounding
function instead of proceeding with the request.
🧹 Nitpick comments (1)
plugins/out_es/es.c (1)

65-69: Avoid magic length for the AWS UA value.
Consider using sizeof("aws-fluent-bit-plugin") - 1 to keep the length in sync with the literal.

Suggested tweak
 flb_http_add_header(c,
                     FLB_HTTP_HEADER_USER_AGENT,
                     sizeof(FLB_HTTP_HEADER_USER_AGENT) - 1,
-                    "aws-fluent-bit-plugin", 21);
+                    "aws-fluent-bit-plugin",
+                    sizeof("aws-fluent-bit-plugin") - 1);

Comment on lines +136 to +141
/* User-Agent */
flb_http_add_header(client,
FLB_HTTP_HEADER_USER_AGENT,
sizeof(FLB_HTTP_HEADER_USER_AGENT) - 1,
FLB_HTTP_HEADER_USER_AGENT_DEFAULT,
sizeof(FLB_HTTP_HEADER_USER_AGENT_DEFAULT) - 1);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Check the return value to avoid silently skipping the User-Agent.

If flb_http_add_header fails (e.g., allocation failure), the request proceeds without the header, which can reintroduce the 418 behavior this PR aims to fix. Consider aborting the request on failure.

Suggested fix
-    flb_http_add_header(client,
-                        FLB_HTTP_HEADER_USER_AGENT,
-                        sizeof(FLB_HTTP_HEADER_USER_AGENT) - 1,
-                        FLB_HTTP_HEADER_USER_AGENT_DEFAULT,
-                        sizeof(FLB_HTTP_HEADER_USER_AGENT_DEFAULT) - 1);
+    if (flb_http_add_header(client,
+                            FLB_HTTP_HEADER_USER_AGENT,
+                            sizeof(FLB_HTTP_HEADER_USER_AGENT) - 1,
+                            FLB_HTTP_HEADER_USER_AGENT_DEFAULT,
+                            sizeof(FLB_HTTP_HEADER_USER_AGENT_DEFAULT) - 1) < 0) {
+        flb_plg_error(ins, "failed to add User-Agent header");
+        goto http_error;
+    }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
/* User-Agent */
flb_http_add_header(client,
FLB_HTTP_HEADER_USER_AGENT,
sizeof(FLB_HTTP_HEADER_USER_AGENT) - 1,
FLB_HTTP_HEADER_USER_AGENT_DEFAULT,
sizeof(FLB_HTTP_HEADER_USER_AGENT_DEFAULT) - 1);
/* User-Agent */
if (flb_http_add_header(client,
FLB_HTTP_HEADER_USER_AGENT,
sizeof(FLB_HTTP_HEADER_USER_AGENT) - 1,
FLB_HTTP_HEADER_USER_AGENT_DEFAULT,
sizeof(FLB_HTTP_HEADER_USER_AGENT_DEFAULT) - 1) < 0) {
flb_plg_error(ins, "failed to add User-Agent header");
goto http_error;
}
🧰 Tools
🪛 Cppcheck (2.19.0)

[error] 140-140: There is an unknown macro here somewhere. Configuration is required. If FLB_VERSION_STR is a macro then please configure it.

(unknownMacro)

🤖 Prompt for AI Agents
In `@plugins/in_nginx_exporter_metrics/nginx.c` around lines 136 - 141, The call
to flb_http_add_header(…, FLB_HTTP_HEADER_USER_AGENT, …) must have its return
value checked and handled; if it fails (e.g., allocation error) abort
building/sending the request instead of continuing without the header. Update
the code around the flb_http_add_header invocation to test the return value, log
an error, free/cleanup any request/client resources as done on other failures,
and return an error/early-exit from the enclosing function (or propagate an
error code) so the request is not sent without the User-Agent header.

Comment on lines +180 to +184
/* User-Agent */
flb_http_add_header(c, FLB_HTTP_HEADER_USER_AGENT,
sizeof(FLB_HTTP_HEADER_USER_AGENT) - 1,
FLB_HTTP_HEADER_USER_AGENT_DEFAULT,
sizeof(FLB_HTTP_HEADER_USER_AGENT_DEFAULT) - 1);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Handle failure from flb_http_add_header instead of ignoring it.
If header insertion fails (e.g., OOM), the request proceeds without a User‑Agent and defeats the goal. Consider logging and short‑circuiting with cleanup.

✅ Suggested fix (error handling + cleanup)
     /* User-Agent */
-    flb_http_add_header(c, FLB_HTTP_HEADER_USER_AGENT,
-                        sizeof(FLB_HTTP_HEADER_USER_AGENT) - 1,
-                        FLB_HTTP_HEADER_USER_AGENT_DEFAULT,
-                        sizeof(FLB_HTTP_HEADER_USER_AGENT_DEFAULT) - 1);
+    ret = flb_http_add_header(c, FLB_HTTP_HEADER_USER_AGENT,
+                              sizeof(FLB_HTTP_HEADER_USER_AGENT) - 1,
+                              FLB_HTTP_HEADER_USER_AGENT_DEFAULT,
+                              sizeof(FLB_HTTP_HEADER_USER_AGENT_DEFAULT) - 1);
+    if (ret != 0) {
+        flb_plg_error(ctx->ins, "failed to add User-Agent header");
+        flb_free(pack);
+        flb_free(body);
+        flb_http_client_destroy(c);
+        flb_upstream_conn_release(u_conn);
+        FLB_OUTPUT_RETURN(FLB_RETRY);
+    }
🧰 Tools
🪛 Cppcheck (2.19.0)

[error] 183-183: There is an unknown macro here somewhere. Configuration is required. If FLB_VERSION_STR is a macro then please configure it.

(unknownMacro)

🤖 Prompt for AI Agents
In `@plugins/out_td/td.c` around lines 180 - 184, Check the return value of
flb_http_add_header when adding
FLB_HTTP_HEADER_USER_AGENT/FLB_HTTP_HEADER_USER_AGENT_DEFAULT; if it fails, log
an error mentioning the header insertion failed, perform necessary cleanup of
the HTTP request/context (free any allocated request object and related
resources), and short‑circuit by returning an error code from the surrounding
function instead of proceeding with the request.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Fix all issues with AI agents
In `@plugins/out_websocket/websocket.c`:
- Around line 80-85: The User-Agent default header is being added after the loop
that adds user-supplied headers (via the existing headers iteration in
websocket.c), which prevents users from overriding it when header deduplication
(allow_dup_headers) is disabled; move the flb_http_add_header(...) call that
adds FLB_HTTP_HEADER_USER_AGENT_DEFAULT so it executes before the
user-configured headers loop (i.e., place the flb_http_add_header(User-Agent...)
immediately before the code that iterates and adds user headers) to match the
ordering used in
loki.c/influxdb.c/opentelemetry.c/prometheus_remote_write.c/http.c/datadog.c and
allow user headers to override the default.

Signed-off-by: Enderson Maia <endersonmaia@gmail.com>
Signed-off-by: Enderson Maia <endersonmaia@gmail.com>
Signed-off-by: Enderson Maia <endersonmaia@gmail.com>
Signed-off-by: Enderson Maia <endersonmaia@gmail.com>
Signed-off-by: Enderson Maia <endersonmaia@gmail.com>
Signed-off-by: Enderson Maia <endersonmaia@gmail.com>
Signed-off-by: Enderson Maia <endersonmaia@gmail.com>
Signed-off-by: Enderson Maia <endersonmaia@gmail.com>
Signed-off-by: Enderson Maia <endersonmaia@gmail.com>
Signed-off-by: Enderson Maia <endersonmaia@gmail.com>
Signed-off-by: Enderson Maia <endersonmaia@gmail.com>
Signed-off-by: Enderson Maia <endersonmaia@gmail.com>
Signed-off-by: Enderson Maia <endersonmaia@gmail.com>
Signed-off-by: Enderson Maia <endersonmaia@gmail.com>
Signed-off-by: Enderson Maia <endersonmaia@gmail.com>
Signed-off-by: Enderson Maia <endersonmaia@gmail.com>
Signed-off-by: Enderson Maia <endersonmaia@gmail.com>
Signed-off-by: Enderson Maia <endersonmaia@gmail.com>
Signed-off-by: Enderson Maia <endersonmaia@gmail.com>
Signed-off-by: Enderson Maia <endersonmaia@gmail.com>
Copy link
Contributor

@braydonk braydonk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

out_stackdriver changes LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

HTTP requests should have an User-Agent header

2 participants

Comments