Skip to content

Conversation

@aletourneau
Copy link

@aletourneau aletourneau commented Oct 28, 2025

Fixes #10392

This patch is adding a new option add_remote_addr which appends a REMOTE_ADDR field to the record. The value of REMOTE_ADDR field is the client's IP address. The client's IP address is extracted from the X-Forwarded-For request header (as it was in fluentd)

For example:

X-Forwarded-For: host1, host2

The value of REMOTE_ADDR would be host1

Known Issues

If multiple X-Forwarded-For headers are specified in the request, which is used (last, or first) will depend on which http server is used (mk or ng):

  • mk: http2: false: The first XFF header will be used
  • ng: http2: true: The last XFF header will be used

Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

  • Example configuration file for the change
  • Debug log output from testing the change
  • Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

  • [N/A] Run local packaging test showing all targets (including any new ones) build.
  • [N/A] Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

  • Documentation required for this feature

Backporting

  • [N/A] Backport to latest stable release.

Configuration

service:
  daemon: Off
  flush: 1
  log_level: debug

pipeline:
  inputs:
     - name: http
       listen: 0.0.0.0
       port: 9880
       add_remote_addr: true

  outputs:
    - name: stdout
      match: "*"

Debug / Valgrind logs

curl command

curl  http://localhost:9880  -d '{"k":"v"}' -H "Content-Type: application/json" -H "X-Forwarded-For: 1.2.3.4, 5.6.7.8"

"REMOTE_ADDR"=>"1.2.3.4" is appended

[0] http.0: [[1761687337.855967665, {}], {"k"=>"v", "REMOTE_ADDR"=>"1.2.3.4"}]

==4070484== Memcheck, a memory error detector
==4070484== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==4070484== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info
==4070484== Command: ./bin/fluent-bit -c ./fluentbit.yaml
==4070484== 
Fluent Bit v4.2.0
* Copyright (C) 2015-2025 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

______ _                  _    ______ _ _             ___   __  
|  ___| |                | |   | ___ (_) |           /   | /  | 
| |_  | |_   _  ___ _ __ | |_  | |_/ /_| |_  __   __/ /| | `| | 
|  _| | | | | |/ _ \ '_ \| __| | ___ \ | __| \ \ / / /_| |  | | 
| |   | | |_| |  __/ | | | |_  | |_/ / | |_   \ V /\___  |__| |_
\_|   |_|\__,_|\___|_| |_|\__| \____/|_|\__|   \_/     |_(_)___/


[2025/10/28 17:39:54.683440115] [ info] Configuration:
[2025/10/28 17:39:54.698966888] [ info]  flush time     | 1.000000 seconds
[2025/10/28 17:39:54.704256903] [ info]  grace          | 5 seconds
[2025/10/28 17:39:54.704510289] [ info]  daemon         | 0
[2025/10/28 17:39:54.704733293] [ info] ___________
[2025/10/28 17:39:54.704947428] [ info]  inputs:
[2025/10/28 17:39:54.705262274] [ info]      http
[2025/10/28 17:39:54.705523272] [ info] ___________
[2025/10/28 17:39:54.705737267] [ info]  filters:
[2025/10/28 17:39:54.705996728] [ info] ___________
[2025/10/28 17:39:54.706204856] [ info]  outputs:
[2025/10/28 17:39:54.706513417] [ info]      stdout.0
[2025/10/28 17:39:54.706778117] [ info] ___________
[2025/10/28 17:39:54.706997000] [ info]  collectors:
[2025/10/28 17:39:54.782106092] [ info] [fluent bit] version=4.2.0, commit=6345fd16ba, pid=4070484
[2025/10/28 17:39:54.798297478] [debug] [engine] coroutine stack size: 24576 bytes (24.0K)
[2025/10/28 17:39:54.805791346] [ info] [storage] ver=1.5.3, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2025/10/28 17:39:54.897690158] [ info] [output:stdout:stdout.0] worker #0 started
[2025/10/28 17:39:54.806267736] [ info] [simd    ] disabled
[2025/10/28 17:39:54.806625046] [ info] [cmetrics] version=1.0.5
[2025/10/28 17:39:54.806958470] [ info] [ctraces ] version=0.6.6
[2025/10/28 17:39:54.823119196] [ info] [input:http:http.0] initializing
[2025/10/28 17:39:54.823797847] [ info] [input:http:http.0] storage_strategy='memory' (memory only)
[2025/10/28 17:39:54.824941504] [debug] [http:http.0] created event channels: read=25 write=26
[2025/10/28 17:39:54.840517725] [debug] [downstream] listening on 0.0.0.0:9880
[2025/10/28 17:39:54.843567989] [debug] [stdout:stdout.0] created event channels: read=28 write=29
[2025/10/28 17:39:54.876344319] [ info] [sp] stream processor started
[2025/10/28 17:39:54.877986785] [ info] [engine] Shutdown Grace Period=5, Shutdown Input Grace Period=2
[2025/10/28 17:40:04.887836946] [debug] [task] created task=0x6303550 id=0 OK
[0] http.0: [[1761687604.082926903, {}], {"k"=>"v", "REMOTE_ADDR"=>"1.2.3.4"}]
[2025/10/28 17:40:04.890196685] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
[2025/10/28 17:40:04.919828745] [debug] [out flush] cb_destroy coro_id=0
[2025/10/28 17:40:04.936131040] [debug] [task] destroy task=0x6303550 (task_id=0)
^C[2025/10/28 17:40:09] [engine] caught signal (SIGINT)
[2025/10/28 17:40:09.475472495] [ warn] [engine] service will shutdown in max 5 seconds
[2025/10/28 17:40:09.477024027] [ info] [engine] pausing all inputs..
[2025/10/28 17:40:09.879831748] [ info] [engine] service has stopped (0 pending tasks)
[2025/10/28 17:40:09.886188874] [ info] [output:stdout:stdout.0] thread worker #0 stopping...
[2025/10/28 17:40:09.897929043] [ info] [output:stdout:stdout.0] thread worker #0 stopped
==4070484== 
==4070484== HEAP SUMMARY:
==4070484==     in use at exit: 0 bytes in 0 blocks
==4070484==   total heap usage: 3,385 allocs, 3,385 frees, 1,414,591 bytes allocated
==4070484== 
==4070484== All heap blocks were freed -- no leaks are possible
==4070484== 
==4070484== For lists of detected and suppressed errors, rerun with: -s
==4070484== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

Valgrind output for tests

$ valgrind --leak-check=full ./bin/flb-rt-in_http
==4071595== Memcheck, a memory error detector
==4071595== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==4071595== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info
==4071595== Command: ./bin/flb-rt-in_http
==4071595== 
Test http...                                    ==4071595== Warning: invalid file descriptor -1 in syscall close()
[ OK ]
Test successful_response_code_200...            ==4071595== Warning: invalid file descriptor -1 in syscall close()
==4071595== Warning: invalid file descriptor -1 in syscall close()
[ OK ]
Test successful_response_code_204...            ==4071595== Warning: invalid file descriptor -1 in syscall close()
==4071595== Warning: invalid file descriptor -1 in syscall close()
[ OK ]
Test failure_response_code_400_bad_json...      ==4071595== Warning: invalid file descriptor -1 in syscall close()
[ OK ]
Test failure_response_code_400_bad_disk_write... [/home/alexandre-local/work/ext/github/fluent-bit/lib/chunkio/src/cio_file_unix.c:410 errno=2] No such file or directory
[2025/10/28 17:43:12.953527805] [error] [input chunk] could not create chunk file: http.0:4071595-1761687792.945368626.flb
[2025/10/28 17:43:12.972434144] [error] [input chunk] no available chunk
[2025/10/28 17:43:12.973184034] [error] [input:http:http.0] Error encoding record : -1
==4071595== Warning: invalid file descriptor -1 in syscall close()
[ OK ]
Test tag_key_with_map_input...                  ==4071595== Warning: invalid file descriptor -1 in syscall close()
[ OK ]
Test tag_key_with_array_input...                ==4071595== Warning: invalid file descriptor -1 in syscall close()
[ OK ]
Test add_remote_addr_map_ng...                  ==4071595== Warning: invalid file descriptor -1 in syscall close()
[ OK ]
Test add_remote_addr_array_ng...                ==4071595== Warning: invalid file descriptor -1 in syscall close()
[ OK ]
Test add_remote_addr_map...                     ==4071595== Warning: invalid file descriptor -1 in syscall close()
[ OK ]
Test add_remote_addr_array...                   ==4071595== Warning: invalid file descriptor -1 in syscall close()
[ OK ]
SUCCESS: All unit tests have passed.
==4071595== 
==4071595== HEAP SUMMARY:
==4071595==     in use at exit: 0 bytes in 0 blocks
==4071595==   total heap usage: 35,527 allocs, 35,527 frees, 12,927,663 bytes allocated
==4071595== 
==4071595== All heap blocks were freed -- no leaks are possible
==4071595== 
==4071595== For lists of detected and suppressed errors, rerun with: -s
==4071595== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

Summary by CodeRabbit

  • New Features

    • Option to include the client's remote address in HTTP input records (disabled by default).
    • Configurable field name for the remote address (default: REMOTE_ADDR).
    • Remote address is extracted (including from X-Forwarded-For) and applied for both HTTP/1.1 and HTTP/2 server paths.
  • Tests

    • Added tests validating REMOTE_ADDR propagation for map and array payloads on both legacy and NextGen HTTP servers.

@coderabbitai
Copy link

coderabbitai bot commented Oct 28, 2025

Walkthrough

Adds optional extraction of client IP into HTTP input events: new config options and struct fields, a REMOTE_ADDR macro, header/remote-address helpers, payload parsing/packing changes to append REMOTE_ADDR into MsgPack records, and tests covering legacy and NextGen HTTP paths. (≤50 words)

Changes

Cohort / File(s) Summary
Config & Types
plugins/in_http/http.h, plugins/in_http/http.c
Added REMOTE_ADDR_KEY macro; extended struct flb_http with add_remote_addr and remote_addr_key; added add_remote_addr (bool) and remote_addr_key (string) entries to the plugin config_map.
Protocol & Processing Logic
plugins/in_http/http_prot.c
Added http_header_lookup(), get_remote_addr(), and append_remote_addr() helpers. Updated JSON and urlencoded payload parsing and pack/NG functions to accept a request parameter and conditionally append REMOTE_ADDR to records. Adjusted memory management and error paths and updated function signatures to propagate request.
Tests
tests/runtime/in_http.c
Added test_http_add_remote_addr() helper plus four tests (*_map/_array for NG and legacy) and extended TEST_LIST to validate REMOTE_ADDR behavior with/without X-Forwarded-For.

Sequence Diagram(s)

sequenceDiagram
    autonumber
    participant Client
    participant HTTPServer
    participant Handler
    participant HeaderHelper
    participant PayloadProcessor
    participant Output

    Client->>HTTPServer: POST (body + optional X-Forwarded-For)
    HTTPServer->>Handler: deliver request (includes version/context)
    Handler->>HeaderHelper: http_header_lookup / get_remote_addr(request)
    HeaderHelper-->>Handler: remote_addr (sanitized)
    Handler->>PayloadProcessor: parse_payload_*(payload, request)
    PayloadProcessor->>PayloadProcessor: process_pack(..., request)
    alt add_remote_addr enabled
        PayloadProcessor->>PayloadProcessor: append_remote_addr(record, remote_addr)
    end
    PayloadProcessor->>Output: emit MsgPack record(s) (may include REMOTE_ADDR)
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~40 minutes

  • Areas needing extra attention:
    • Propagation and signature changes for the request parameter across NextGen and legacy paths.
    • Correctness and sanitization of get_remote_addr() (X-Forwarded-For parsing and fallback behavior).
    • MsgPack unpack/append flow and lifecycle for appended_result / appended_sbuf.
    • Memory cleanup on all error and iterative paths.

Poem

🐰 I sniff the headers, light and spry,
I pull the IP from XFF sky,
REMOTE_ADDR hops into the pack,
Each record gets a tiny track,
I thump my feet — the logs reply!

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 30.00% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check ✅ Passed The pull request title "in_http: add fluentd like add_remote_addr functionality" clearly and concisely describes the main change: adding a new add_remote_addr feature to the in_http input plugin. The title is specific enough that a teammate reviewing the commit history would immediately understand the primary objective without being vague or overly broad. It directly references both the component being modified (in_http) and the feature being added (add_remote_addr functionality).
Linked Issues Check ✅ Passed All coding requirements from linked issue #10392 are met by this implementation. The PR adds the add_remote_addr configuration option [http.c] with configurable field naming via remote_addr_key [http.c, http.h], implements X-Forwarded-For header extraction via get_remote_addr() [http_prot.c], injects the remote address into records via append_remote_addr() [http_prot.c], supports both HTTP/1.1 and HTTP/2.0 paths via http_header_lookup() [http_prot.c], and provides comprehensive test coverage for both map and array payloads with both legacy and NextGen server variants [in_http.c]. The feature defaults to false, matching fluentd's behavior as documented in the PR.
Out of Scope Changes Check ✅ Passed All changes are directly scoped to implementing the add_remote_addr feature requested in #10392. The modifications to process_pack(), parse_payload_json(), and parse_payload_urlencoded() signatures are necessary to thread request context through the payload processing pipeline. The addition of remote_addr_key as a configurable option (beyond the base requirement) is a reasonable extension that provides flexibility consistent with fluentd's approach. The new helper functions (http_header_lookup, get_remote_addr, append_remote_addr) and comprehensive tests are all directly supporting the core feature implementation with no unrelated changes detected.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
plugins/in_http/http_prot.c (1)

413-562: Critical: HTTP version mismatch causes undefined behavior—confirmed.

Line 432 hardcodes HTTP_PROTOCOL_VERSION_11 when calling get_remote_addr, but HTTP/2.0 requests using urlencoded payloads reach this code. The HTTP/2.0 code path at line 1406 calls parse_payload_urlencoded, which then invokes process_pack (line 697) with the hardcoded HTTP/1.1 version. The request object is a struct flb_http_request* (HTTP/2.0), but get_remote_addr is called with version 11, causing http_header_lookup to incorrectly cast it as struct mk_http_request* (HTTP/1.1).

For comparison, JSON payloads have a separate parse_payload_json_ng path that correctly calls process_pack_ng (line 1191), which uses HTTP_PROTOCOL_VERSION_20. Urlencoded payloads lack this alternative, creating an asymmetry.

Solution: Create parse_payload_urlencoded_ng that calls process_pack_ng with the correct HTTP/2.0 version, or pass the HTTP version as a parameter to parse_payload_urlencoded.

🧹 Nitpick comments (1)
plugins/in_http/http.c (1)

221-221: Remove trailing whitespace.

Line 221 contains only whitespace.

Apply this diff:

-    
+
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 6345fd1 and 5e60653.

📒 Files selected for processing (4)
  • plugins/in_http/http.c (1 hunks)
  • plugins/in_http/http.h (2 hunks)
  • plugins/in_http/http_prot.c (14 hunks)
  • tests/runtime/in_http.c (2 hunks)
🧰 Additional context used
🧬 Code graph analysis (2)
tests/runtime/in_http.c (3)
src/flb_lib.c (3)
  • flb_input_set (300-330)
  • flb_output_set (515-546)
  • flb_start (914-925)
src/flb_http_client.c (4)
  • flb_http_client (814-859)
  • flb_http_add_header (963-995)
  • flb_http_do (1572-1632)
  • flb_http_client_destroy (1688-1695)
src/flb_upstream.c (1)
  • flb_upstream_conn_release (862-947)
plugins/in_http/http_prot.c (4)
src/flb_http_common.c (1)
  • flb_http_request_get_header (230-257)
src/flb_sds.c (3)
  • flb_sds_create_len (58-76)
  • flb_sds_trim (148-202)
  • flb_sds_destroy (389-399)
include/fluent-bit/flb_mem.h (1)
  • flb_free (126-128)
lib/cfl/src/cfl_sds.c (1)
  • cfl_sds_len (88-91)
🔇 Additional comments (13)
plugins/in_http/http.c (1)

210-220: LGTM! Configuration options are well-defined.

The new configuration options for add_remote_addr and remote_addr_key are properly structured with appropriate types, defaults, and descriptions. The default of true for add_remote_addr makes the feature opt-out, which aligns with the PR objectives.

plugins/in_http/http_prot.c (7)

218-283: Well-structured version-aware header lookup.

The http_header_lookup function properly abstracts header retrieval across HTTP/1.1 (Monkey parser) and HTTP/2.0 (NextGen server) backends. The case-insensitive comparison for HTTP/1.1 headers and the parameter validation are appropriate.


285-319: Remote address extraction logic is sound.

The get_remote_addr function correctly:

  • Extracts the X-Forwarded-For header using the version-aware lookup
  • Handles comma-separated proxy chains by taking only the first IP (the client)
  • Trims whitespace from the result
  • Properly manages memory by destroying on error paths

The empty-string check on line 313 is appropriate, as flb_sds_trim returns the final length (0 for empty, -1 for error).


321-361: MessagePack augmentation is correctly implemented.

The append_remote_addr function properly:

  • Repacks the existing map with an additional entry
  • Copies all existing key-value pairs
  • Appends the REMOTE_ADDR field with the configurable key name
  • Unpacks the result for the caller to use

484-492: Memory management for array elements is correct.

The code properly handles the appended_result lifecycle:

  • Skips destruction on first iteration (i=0) since nothing was allocated yet
  • Destroys the previous iteration's result before creating a new one (i>0)
  • The final iteration's result is cleaned up at lines 540-543

564-599: JSON payload parsing correctly propagates the request.

The function signature now includes the request parameter, which is properly passed through to process_pack for remote address extraction.


1172-1322: NextGen processing uses the correct HTTP version.

Unlike process_pack, the process_pack_ng function correctly calls get_remote_addr with HTTP_PROTOCOL_VERSION_20 (line 1191), ensuring proper header parsing for HTTP/2.0 requests. The memory management and record augmentation logic mirrors the non-ng variant.


1324-1365: NextGen JSON parsing correctly uses process_pack_ng.

The parse_payload_json_ng function properly calls process_pack_ng (line 1361), maintaining separation between HTTP/1.1 and HTTP/2.0 processing paths.

plugins/in_http/http.h (2)

35-35: LGTM! Macro definition for default remote address key.

The REMOTE_ADDR_KEY macro provides a clean default value that can be referenced consistently across the codebase.


51-52: LGTM! Struct fields properly support the new feature.

The add_remote_addr and remote_addr_key fields are correctly typed and positioned within the flb_http structure, matching the configuration options defined in http.c.

tests/runtime/in_http.c (3)

674-764: Well-structured test helper for remote address validation.

The test_http_add_remote_addr function provides comprehensive validation:

  • Configures add_remote_addr feature and HTTP version
  • Sends HTTP POST with X-Forwarded-For header
  • Validates that REMOTE_ADDR appears in the output with the expected IP
  • Properly handles both NextGen (HTTP/2.0) and legacy (HTTP/1.1) server variants

797-800: Test list properly updated.

The four new test cases are correctly registered in the TEST_LIST for execution.


767-786: Remove unsubstantiated "critical bug" claim; urlencoded test suggestion is valid.

The four test cases shown correctly cover JSON payloads across both HTTP/2.0 and HTTP/1.1 paths with proper remote address extraction validation. However, the reference to "the critical bug identified in http_prot.c" cannot be substantiated—no bugs were found in the HTTP/2.0 remote address handling, which implements identical logic to HTTP/1.1. The suggestion to add urlencoded tests is valid: the HTTP input plugin supports urlencoded payloads (confirmed in http_prot.c lines 897-898 for HTTP/1.1 and 1384-1386 for HTTP/2.0), but no such tests currently exist in the suite.

Copy link
Contributor

@cosmo0920 cosmo0920 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the contributor document, we need to use C style of comments even if one line comments.

@aletourneau aletourneau force-pushed the in_http/add_remote_addr branch 2 times, most recently from 9924cf3 to 71c1a00 Compare October 29, 2025 13:47
@aletourneau
Copy link
Author

  • Fixed the comments style
  • Made "false" the default value for the add_remote_addr in_http config (this was the default in fluentd)

Also: Thank you for looking at this PR, and for the comments, it is appreciated.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
plugins/in_http/http_prot.c (2)

437-473: Fix memory leak: destroy appended_result between outer loop iterations in MAP case.

When processing multiple maps in the outer while loop (line 436), appended_result is reinitialized in append_remote_addr without destroying the previous iteration's data. The ARRAY case correctly handles this on lines 485-487.

Apply this diff to destroy the previous result before reuse:

         if (result.data.type == MSGPACK_OBJECT_MAP) {
             obj = &result.data;
             if (remote_addr != NULL && strlen(remote_addr) > 0) {
                 if (!appended_initialized) {
                     /* doing this only once, since it can be cleared and reused */
                     msgpack_sbuffer_init(&appended_sbuf);
                     appended_initialized = 1;
                 }
+                else {
+                    /* destroy previous unpacked result before reuse */
+                    msgpack_unpacked_destroy(&appended_result);
+                }
 
                 /* if we fail to append, we just continue with the original object */
                 if (append_remote_addr(obj, &appended_result, &appended_sbuf, ctx, remote_addr) == 0) {

1196-1230: Fix memory leak: destroy appended_result between outer loop iterations in MAP case.

Same issue as in process_pack: when processing multiple maps in the outer while loop (line 1195), appended_result is reinitialized without destroying the previous iteration's data.

Apply this diff:

         if (result.data.type == MSGPACK_OBJECT_MAP) {
             obj = &result.data;
             if (remote_addr != NULL && strlen(remote_addr) > 0) {
                 if (!appended_initialized) {
                     /* doing this only once, since it can be cleared and reused */
                     msgpack_sbuffer_init(&appended_sbuf);
                     appended_initialized = 1;
                 }
+                else {
+                    /* destroy previous unpacked result before reuse */
+                    msgpack_unpacked_destroy(&appended_result);
+                }
                 if (append_remote_addr(obj, &appended_result, &appended_sbuf, ctx, remote_addr) == 0) {
🧹 Nitpick comments (3)
plugins/in_http/http_prot.c (3)

439-439: Consider using flb_sds_len for consistency.

Since remote_addr is an flb_sds_t type, prefer flb_sds_len(remote_addr) over strlen(remote_addr) for consistency with the codebase and potential performance benefits (O(1) vs O(n)).

Apply this diff:

-            if (remote_addr != NULL && strlen(remote_addr) > 0) {
+            if (remote_addr != NULL && flb_sds_len(remote_addr) > 0) {

And similarly on line 478:

-                if (remote_addr != NULL && strlen(remote_addr) > 0) {
+                if (remote_addr != NULL && flb_sds_len(remote_addr) > 0) {

Also applies to: 478-478


1198-1198: Consider using flb_sds_len for consistency.

Same as in process_pack: prefer flb_sds_len(remote_addr) over strlen(remote_addr) for consistency and efficiency.

Apply this diff on line 1198:

-            if (remote_addr != NULL && strlen(remote_addr) > 0) {
+            if (remote_addr != NULL && flb_sds_len(remote_addr) > 0) {

And on line 1237:

-                if (remote_addr != NULL && strlen(remote_addr) > 0) {
+                if (remote_addr != NULL && flb_sds_len(remote_addr) > 0) {

Also applies to: 1237-1237


1320-1320: Optional: Remove trailing whitespace.

Line 1320 appears to have trailing whitespace. Consider removing it for consistency.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 5e60653 and 71c1a00.

📒 Files selected for processing (4)
  • plugins/in_http/http.c (1 hunks)
  • plugins/in_http/http.h (2 hunks)
  • plugins/in_http/http_prot.c (14 hunks)
  • tests/runtime/in_http.c (2 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • tests/runtime/in_http.c
🧰 Additional context used
🧬 Code graph analysis (1)
plugins/in_http/http_prot.c (4)
src/flb_http_common.c (1)
  • flb_http_request_get_header (230-257)
src/flb_sds.c (3)
  • flb_sds_create_len (58-76)
  • flb_sds_trim (148-202)
  • flb_sds_destroy (389-399)
include/fluent-bit/flb_mem.h (1)
  • flb_free (126-128)
lib/cfl/src/cfl_sds.c (1)
  • cfl_sds_len (88-91)
🔇 Additional comments (7)
plugins/in_http/http.h (1)

35-35: LGTM! Clean additions for remote address configuration.

The macro and struct fields are well-defined and properly positioned within the existing structure.

Also applies to: 51-52

plugins/in_http/http.c (1)

210-220: LGTM! Configuration entries are well-structured.

The new options are properly defined with appropriate defaults and clear descriptions.

plugins/in_http/http_prot.c (5)

218-283: LGTM! Well-structured header lookup supporting both HTTP versions.

The function correctly handles both HTTP/1.x (using mk_http_session parser) and HTTP/2.0 (using flb_http_request_get_header) with appropriate null checks.


476-522: LGTM! Array case correctly manages msgpack unpacked results.

The cleanup logic properly destroys previous results when iterating (line 486) and the final result after the loop (line 541).


531-559: LGTM! Comprehensive resource cleanup on all exit paths.

Both normal and error paths properly clean up remote_addr, appended_result, and appended_sbuf.


564-599: LGTM! Request parameter correctly threaded through payload parsing.

The request parameter is properly added to both parsing functions and passed to process_pack.

Also applies to: 601-716


1324-1365: LGTM! NextGen payload processing correctly integrates request parameter.

The request parameter is properly threaded through the NextGen HTTP server path.

Also applies to: 1406-1406

@aletourneau aletourneau force-pushed the in_http/add_remote_addr branch from 71c1a00 to 2227f64 Compare October 29, 2025 14:12
@aletourneau
Copy link
Author

  • Addressed the 2 issues reported by coderabbitai (and also the nitpick items)

@aletourneau aletourneau force-pushed the in_http/add_remote_addr branch from 2227f64 to e8d9673 Compare October 29, 2025 14:17
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (3)
tests/runtime/in_http.c (1)

674-764: Clean test implementation with minor whitespace nit.

The test helper function is well-structured and covers the essential scenarios. It properly validates that REMOTE_ADDR is injected with the expected IP from the X-Forwarded-For header.

Line 697 appears to have trailing whitespace after the semicolon. Consider removing it for consistency.

plugins/in_http/http_prot.c (2)

413-562: Address code style inconsistencies.

The remote address augmentation logic is correct and memory management is solid. However, there are minor code style issues:

Apply this diff to fix spacing:

-    
+
 	flb_sds_t remote_addr = NULL;
-                    if(i > 0) {
+                    if (i > 0) {
                         msgpack_unpacked_destroy(&appended_result);
                     }
 
-                    if(append_remote_addr(&record, &appended_result, &appended_sbuf, ctx, remote_addr) == 0) {
+                    if (append_remote_addr(&record, &appended_result, &appended_sbuf, ctx, remote_addr) == 0) {
                         record = appended_result.data;
                     }

1172-1322: Address code style inconsistencies.

The NextGen remote address augmentation mirrors the legacy path correctly. However, the same code style issues appear:

Apply this diff to fix spacing:

-                    if (i > 0) {
+                    if (i > 0) {
                         msgpack_unpacked_destroy(&appended_result);
                     }
 
-                    if(append_remote_addr(&record, &appended_result, &appended_sbuf, ctx, remote_addr) == 0) {
+                    if (append_remote_addr(&record, &appended_result, &appended_sbuf, ctx, remote_addr) == 0) {
                         record = appended_result.data;
                     }

And remove trailing whitespace at line 1320:

-    
+
     return -1;
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 71c1a00 and 2227f64.

📒 Files selected for processing (4)
  • plugins/in_http/http.c (1 hunks)
  • plugins/in_http/http.h (2 hunks)
  • plugins/in_http/http_prot.c (14 hunks)
  • tests/runtime/in_http.c (2 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • plugins/in_http/http.c
🧰 Additional context used
🧬 Code graph analysis (2)
tests/runtime/in_http.c (3)
src/flb_lib.c (3)
  • flb_input_set (300-330)
  • flb_output_set (515-546)
  • flb_start (914-925)
src/flb_http_client.c (4)
  • flb_http_client (814-859)
  • flb_http_add_header (963-995)
  • flb_http_do (1572-1632)
  • flb_http_client_destroy (1688-1695)
src/flb_upstream.c (1)
  • flb_upstream_conn_release (862-947)
plugins/in_http/http_prot.c (4)
src/flb_http_common.c (1)
  • flb_http_request_get_header (230-257)
src/flb_sds.c (3)
  • flb_sds_create_len (58-76)
  • flb_sds_trim (148-202)
  • flb_sds_destroy (389-399)
include/fluent-bit/flb_mem.h (1)
  • flb_free (126-128)
lib/cfl/src/cfl_sds.c (1)
  • cfl_sds_len (88-91)
🔇 Additional comments (10)
plugins/in_http/http.h (2)

35-35: LGTM!

The macro definition follows the existing pattern and provides a clear constant for the remote address field name.


51-52: LGTM!

The struct fields are well-typed and integrate cleanly with the existing structure. The use of const char * for remote_addr_key is appropriate and matches the actual usage with strlen in the implementation.

tests/runtime/in_http.c (2)

767-786: LGTM!

Excellent test coverage across all combinations: NextGen/Legacy HTTP server paths and Map/Array payload formats. The test wrappers properly validate that the first IP address in the X-Forwarded-For header list is extracted.


797-800: LGTM!

The new tests are properly registered in the TEST_LIST with clear, descriptive names.

plugins/in_http/http_prot.c (6)

218-283: LGTM!

The version-aware header lookup function correctly handles both HTTP/1.1 (Monkey parser) and HTTP/2.0 (flb_http_request) backends. Case-insensitive comparison is appropriate for HTTP headers, and error handling is sound.


285-319: LGTM!

The function correctly extracts the first IP address from the X-Forwarded-For header. The trim validation check at line 313 has been properly fixed to handle both error cases (-1) and empty strings (0).


321-361: LGTM!

The function properly augments msgpack map objects with the REMOTE_ADDR field. Line 347 now correctly uses strlen for the const char * field, fixing the previous type cast issue.


564-599: LGTM!

The payload parsing functions correctly thread the request context through to process_pack, enabling remote address extraction without altering the existing payload processing logic.

Also applies to: 601-716


947-947: LGTM!

The process_payload function correctly passes the request context to the payload parsing functions.

Also applies to: 950-950


1406-1406: LGTM!

The NextGen process_payload function correctly passes the request context to parse_payload_urlencoded.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
tests/runtime/in_http.c (2)

691-695: Remove redundant TEST_CHECK call.

Line 691 calls TEST_CHECK(ret > 0) standalone, then line 692 checks the same condition in an if statement. The standalone call is redundant.

Apply this diff:

-    TEST_CHECK(ret > 0);
     if(!TEST_CHECK(ret > 0)) {
         TEST_MSG("snprintf failed");
         exit(EXIT_FAILURE);
     }

697-697: Remove trailing whitespace.

Apply this diff:

-    cb_data.data = expected_buffer;
-    
+    cb_data.data = expected_buffer;
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 2227f64 and e8d9673.

📒 Files selected for processing (4)
  • plugins/in_http/http.c (1 hunks)
  • plugins/in_http/http.h (2 hunks)
  • plugins/in_http/http_prot.c (14 hunks)
  • tests/runtime/in_http.c (2 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
  • plugins/in_http/http.h
  • plugins/in_http/http.c
🧰 Additional context used
🧬 Code graph analysis (2)
tests/runtime/in_http.c (3)
src/flb_lib.c (3)
  • flb_input_set (300-330)
  • flb_output_set (515-546)
  • flb_start (914-925)
src/flb_http_client.c (4)
  • flb_http_client (814-859)
  • flb_http_add_header (963-995)
  • flb_http_do (1572-1632)
  • flb_http_client_destroy (1688-1695)
src/flb_upstream.c (1)
  • flb_upstream_conn_release (862-947)
plugins/in_http/http_prot.c (4)
src/flb_http_common.c (1)
  • flb_http_request_get_header (230-257)
src/flb_sds.c (3)
  • flb_sds_create_len (58-76)
  • flb_sds_trim (148-202)
  • flb_sds_destroy (389-399)
include/fluent-bit/flb_mem.h (1)
  • flb_free (126-128)
lib/cfl/src/cfl_sds.c (1)
  • cfl_sds_len (88-91)
🔇 Additional comments (6)
tests/runtime/in_http.c (1)

674-764: Test implementation looks solid.

The test function properly validates the add_remote_addr functionality by:

  • Configuring the input with add_remote_addr enabled
  • Adding an X-Forwarded-For header with comma-separated IPs
  • Verifying REMOTE_ADDR is correctly extracted and appended to records
  • Testing both NextGen and legacy server paths with map and array inputs
plugins/in_http/http_prot.c (5)

218-283: LGTM: Header lookup implementation is correct.

The function properly handles both HTTP/1.1 (using Monkey parser) and HTTP/2.0 (using flb_http_request_get_header) with appropriate null checks and case-insensitive comparisons.


321-361: LGTM: Remote address appending logic is correct.

The function properly:

  • Creates a new map with increased size
  • Copies existing key-value pairs
  • Appends the REMOTE_ADDR field using the configured key
  • Correctly uses strlen for the const char* key

413-562: LGTM: Memory management is correct.

The function properly handles resource cleanup:

  • Initializes appended_sbuffer only once when needed
  • For arrays, destroys the previous appended_result before creating a new one (line 485-487)
  • Cleans up all resources on both success and error paths
  • The appended_initialized flag ensures proper cleanup

1172-1322: LGTM: NextGen variant follows the same correct patterns.

The NextGen implementation mirrors the legacy variant with proper:

  • HTTP/2.0 protocol version for header lookup
  • Memory management for appended buffers
  • Resource cleanup on all exit paths

564-565: LGTM: Function signatures updated consistently.

The request parameter is properly threaded through all payload parsing functions, enabling remote address extraction at the right points in the call chain.

Also applies to: 601-602

Signed-off-by: Alexandre Létourneau <letourneau.alexandre@gmail.com>
@aletourneau aletourneau force-pushed the in_http/add_remote_addr branch from e8d9673 to 5fa7f6d Compare October 29, 2025 14:30
@aletourneau
Copy link
Author

  • Addressed the issue raised by coderabbitai (and nitpick items)

@aletourneau
Copy link
Author

@coderabbitai review

@coderabbitai
Copy link

coderabbitai bot commented Oct 29, 2025

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
tests/runtime/in_http.c (1)

765-784: LGTM! Comprehensive test coverage for both server paths.

The four test wrappers provide good coverage by exercising:

  • NextGen HTTP/2 server vs. legacy Monkey server
  • Map input ({}) vs. array input ([{}])

All tests consistently validate extraction of the first IP from the X-Forwarded-For header.

Consider adding optional edge-case tests for:

  • Single IP in XFF (e.g., "1.2.3.4")
  • Missing or empty XFF header
  • Malformed XFF values

These would strengthen robustness validation but are not essential for the initial implementation.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between e8d9673 and 5fa7f6d.

📒 Files selected for processing (4)
  • plugins/in_http/http.c (1 hunks)
  • plugins/in_http/http.h (2 hunks)
  • plugins/in_http/http_prot.c (14 hunks)
  • tests/runtime/in_http.c (2 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • plugins/in_http/http_prot.c
🔇 Additional comments (4)
plugins/in_http/http.h (1)

35-35: LGTM! Clean implementation.

The macro definition and struct field additions are well-designed and follow the existing code patterns. The use of a macro for the default key name ensures consistency across the codebase.

Also applies to: 51-52

plugins/in_http/http.c (1)

210-220: LGTM! Configuration options properly defined.

The new configuration options follow the established pattern and integrate smoothly into the existing config_map. The default value of false for add_remote_addr correctly matches Fluentd's behavior (opt-in), as noted in the PR objectives.

tests/runtime/in_http.c (2)

674-762: LGTM! Well-structured test implementation.

The test helper function properly validates the add_remote_addr feature by:

  • Configuring the plugin with add_remote_addr=true
  • Injecting an X-Forwarded-For header
  • Verifying the expected IP appears in the REMOTE_ADDR field

The implementation follows the existing test patterns and includes appropriate error checking.


795-798: LGTM! Test registration is correct.

The four new test cases are properly registered in the TEST_LIST with descriptive names that clearly indicate what each test validates.

@aletourneau aletourneau requested a review from cosmo0920 October 29, 2025 14:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

add_remote_addr to http input like fluentd has

2 participants