Skip to content

Conversation

@cosmo0920
Copy link
Contributor

@cosmo0920 cosmo0920 commented Oct 31, 2025

This is because RHEL10 or other distributions which use zstd compression inside of systemd, the symbols could be collided and we need to unify the using zstd symbols.
This is just for changing the way to linking libzstd libraries. No need to change for building rules in packaging.


Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

  • Example configuration file for the change
  • Debug log output from testing the change
  • Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

  • Run local packaging test showing all targets (including any new ones) build.
  • Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

  • Documentation required for this feature

Backporting

  • Backport to latest stable release.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

Summary by CodeRabbit

  • Chores
    • Improved build system for compression library detection: discovery now runs later in the configure flow, prefers system-provided versions on Linux/systemd, performs quieter checks, logs the detected system version when used, and falls back to the bundled implementation when needed for more reliable builds.

@coderabbitai
Copy link

coderabbitai bot commented Oct 31, 2025

Note

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.

Walkthrough

Zstd detection in CMakeLists.txt was moved to after the JOURNALD block and reworked to prefer the system zstd on Linux/systemd where configured, using quiet pkg-config checks, a two-step system lookup (LIBZSTD then zstd), and falling back to the internal cmake/zstd.cmake if not found.

Changes

Cohort / File(s) Summary
Zstd CMake Configuration Reorganization
CMakeLists.txt
Moved Zstd handling to post-JOURNALD; changed condition to FLB_PREFER_SYSTEM_LIB_ZSTD OR (FLB_SYSTEM_LINUX AND FLB_IN_SYSTEMD); use pkg_check_modules in QUIET mode to try LIBZSTD then zstd; set LIBZSTD_INCLUDE_DIRS/LIBZSTD_LIBRARY_DIRS when found; otherwise include cmake/zstd.cmake. Removed previous standalone block.

Sequence Diagram(s)

sequenceDiagram
    participant CMake as CMakeLists.txt
    participant JOURNALD as JOURNALD checks
    participant PKG as pkg_check_modules (QUIET)
    participant FIND as find_package zstd (fallback)
    participant INTERNAL as include cmake/zstd.cmake

    CMake->>JOURNALD: run JOURNALD-related checks
    JOURNALD-->>CMake: continue
    CMake->>CMake: evaluate condition\n(FLB_PREFER_SYSTEM_LIB_ZSTD OR\n(FLB_SYSTEM_LINUX AND FLB_IN_SYSTEMD))
    alt condition true
        CMake->>PKG: pkg_check_modules(LIBZSTD) (QUIET)
        alt LIBZSTD found
            PKG-->>CMake: set LIBZSTD_INCLUDE_DIRS / LIBZSTD_LIBRARY_DIRS
        else LIBZSTD not found
            CMake->>FIND: attempt zstd discovery
            alt zstd found
                FIND-->>CMake: set include/link dirs
            else zstd not found
                CMake->>INTERNAL: include cmake/zstd.cmake (fallback)
            end
        end
    else condition false
        CMake->>INTERNAL: include cmake/zstd.cmake (legacy/internal only)
    end
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

  • Review the combined conditional for correct operator grouping and intended scope.
  • Confirm pkg_check_modules(... QUIET) behavior and that LIBZSTD_* variable names match later usage.
  • Verify fallback order (LIBZSTD → zstd → internal) and that moving the block after JOURNALD doesn't introduce ordering issues.

Suggested reviewers

  • patrick-stephens
  • niedbalski
  • fujimotos
  • edsiper

Poem

🐇 With tiny hops through CMake's glade I pry,

System zstd first under systemd's sky,
If pkg is silent and treasures hide away,
I fetch the fallback to keep builds in play,
A rabbit's tidy patch—hoppity hooray! ✨

Pre-merge checks and finishing touches

✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title directly describes the main change: linking system libzstd when in_systemd is enabled, which matches the CMakeLists.txt modification that moves Zstd handling to be triggered by FLB_SYSTEM_LINUX AND FLB_IN_SYSTEMD.
✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch cosmo0920-link-system-zstd-when-enabling-in_systemd

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between cd6eefd and 60e849e.

📒 Files selected for processing (1)
  • CMakeLists.txt (1 hunks)
🧰 Additional context used
🧠 Learnings (9)
📓 Common learnings
Learnt from: shadowshot-x
Repo: fluent/fluent-bit PR: 10794
File: src/aws/flb_aws_compress.c:26-26
Timestamp: 2025-08-29T06:24:44.797Z
Learning: In Fluent Bit, ZSTD support is always available and enabled by default. The build system automatically detects and uses either the system libzstd library or builds the bundled ZSTD version. Unlike other optional dependencies like Arrow which use conditional compilation guards (e.g., FLB_HAVE_ARROW), ZSTD does not require conditional includes or build flags.
Learnt from: shadowshot-x
Repo: fluent/fluent-bit PR: 10794
File: tests/internal/aws_compress.c:93-107
Timestamp: 2025-08-29T06:25:27.250Z
Learning: In Fluent Bit, ZSTD compression is enabled by default and is treated as a core dependency, not requiring conditional compilation guards like `#ifdef FLB_HAVE_ZSTD`. Unlike some other optional components such as ARROW/PARQUET (which use `#ifdef FLB_HAVE_ARROW` guards), ZSTD support is always available and doesn't need build-time conditionals. ZSTD headers are included directly without guards across multiple plugins and core components.
Learnt from: shadowshot-x
Repo: fluent/fluent-bit PR: 10794
File: tests/internal/aws_compress.c:7-7
Timestamp: 2025-08-29T06:25:02.561Z
Learning: In Fluent Bit, ZSTD (zstandard) compression library is bundled directly in the source tree at `lib/zstd-1.5.7` and is built unconditionally as a static library. Unlike optional external dependencies, ZSTD does not use conditional compilation guards like `FLB_HAVE_ZSTD` and is always available. Headers like `<fluent-bit/flb_zstd.h>` can be included directly without guards.
Learnt from: shadowshot-x
Repo: fluent/fluent-bit PR: 10794
File: tests/internal/aws_compress.c:93-107
Timestamp: 2025-08-29T06:25:27.250Z
Learning: In Fluent Bit, ZSTD compression is enabled by default and is treated as a core dependency, not requiring conditional compilation guards like `#ifdef FLB_HAVE_ZSTD`. Unlike some other optional components, ZSTD support is always available and doesn't need build-time conditionals.
Learnt from: shadowshot-x
Repo: fluent/fluent-bit PR: 10794
File: src/aws/flb_aws_compress.c:52-56
Timestamp: 2025-08-29T06:24:55.855Z
Learning: ZSTD compression is always available in Fluent Bit and does not require conditional compilation guards. Unlike Arrow/Parquet which use #ifdef FLB_HAVE_ARROW guards, ZSTD is built unconditionally with flb_zstd.c included directly in src/CMakeLists.txt and a bundled ZSTD library at lib/zstd-1.5.7/.
Learnt from: ThomasDevoogdt
Repo: fluent/fluent-bit PR: 9277
File: .github/workflows/pr-compile-check.yaml:147-151
Timestamp: 2025-08-31T12:46:11.940Z
Learning: In fluent-bit CMakeLists.txt, the system library preference flags are defined as FLB_PREFER_SYSTEM_LIB_ZSTD and FLB_PREFER_SYSTEM_LIB_KAFKA with the FLB_ prefix.
Learnt from: shadowshot-x
Repo: fluent/fluent-bit PR: 10794
File: tests/internal/aws_compress.c:39-42
Timestamp: 2025-08-29T06:24:26.170Z
Learning: In Fluent Bit, ZSTD compression support is enabled by default and does not require conditional compilation guards (like #ifdef FLB_HAVE_ZSTD) around ZSTD-related code declarations and implementations.
📚 Learning: 2025-08-31T12:46:11.940Z
Learnt from: ThomasDevoogdt
Repo: fluent/fluent-bit PR: 9277
File: .github/workflows/pr-compile-check.yaml:147-151
Timestamp: 2025-08-31T12:46:11.940Z
Learning: In fluent-bit CMakeLists.txt, the system library preference flags are defined as FLB_PREFER_SYSTEM_LIB_ZSTD and FLB_PREFER_SYSTEM_LIB_KAFKA with the FLB_ prefix.

Applied to files:

  • CMakeLists.txt
📚 Learning: 2025-08-29T06:24:55.855Z
Learnt from: shadowshot-x
Repo: fluent/fluent-bit PR: 10794
File: src/aws/flb_aws_compress.c:52-56
Timestamp: 2025-08-29T06:24:55.855Z
Learning: ZSTD compression is always available in Fluent Bit and does not require conditional compilation guards. Unlike Arrow/Parquet which use #ifdef FLB_HAVE_ARROW guards, ZSTD is built unconditionally with flb_zstd.c included directly in src/CMakeLists.txt and a bundled ZSTD library at lib/zstd-1.5.7/.

Applied to files:

  • CMakeLists.txt
📚 Learning: 2025-08-29T06:25:02.561Z
Learnt from: shadowshot-x
Repo: fluent/fluent-bit PR: 10794
File: tests/internal/aws_compress.c:7-7
Timestamp: 2025-08-29T06:25:02.561Z
Learning: In Fluent Bit, ZSTD (zstandard) compression library is bundled directly in the source tree at `lib/zstd-1.5.7` and is built unconditionally as a static library. Unlike optional external dependencies, ZSTD does not use conditional compilation guards like `FLB_HAVE_ZSTD` and is always available. Headers like `<fluent-bit/flb_zstd.h>` can be included directly without guards.

Applied to files:

  • CMakeLists.txt
📚 Learning: 2025-08-29T06:25:27.250Z
Learnt from: shadowshot-x
Repo: fluent/fluent-bit PR: 10794
File: tests/internal/aws_compress.c:93-107
Timestamp: 2025-08-29T06:25:27.250Z
Learning: In Fluent Bit, ZSTD compression is enabled by default and is treated as a core dependency, not requiring conditional compilation guards like `#ifdef FLB_HAVE_ZSTD`. Unlike some other optional components such as ARROW/PARQUET (which use `#ifdef FLB_HAVE_ARROW` guards), ZSTD support is always available and doesn't need build-time conditionals. ZSTD headers are included directly without guards across multiple plugins and core components.

Applied to files:

  • CMakeLists.txt
📚 Learning: 2025-08-29T06:24:44.797Z
Learnt from: shadowshot-x
Repo: fluent/fluent-bit PR: 10794
File: src/aws/flb_aws_compress.c:26-26
Timestamp: 2025-08-29T06:24:44.797Z
Learning: In Fluent Bit, ZSTD support is always available and enabled by default. The build system automatically detects and uses either the system libzstd library or builds the bundled ZSTD version. Unlike other optional dependencies like Arrow which use conditional compilation guards (e.g., FLB_HAVE_ARROW), ZSTD does not require conditional includes or build flags.

Applied to files:

  • CMakeLists.txt
📚 Learning: 2025-08-29T06:24:26.170Z
Learnt from: shadowshot-x
Repo: fluent/fluent-bit PR: 10794
File: tests/internal/aws_compress.c:39-42
Timestamp: 2025-08-29T06:24:26.170Z
Learning: In Fluent Bit, ZSTD compression support is enabled by default and does not require conditional compilation guards (like #ifdef FLB_HAVE_ZSTD) around ZSTD-related code declarations and implementations.

Applied to files:

  • CMakeLists.txt
📚 Learning: 2025-08-29T06:25:27.250Z
Learnt from: shadowshot-x
Repo: fluent/fluent-bit PR: 10794
File: tests/internal/aws_compress.c:93-107
Timestamp: 2025-08-29T06:25:27.250Z
Learning: In Fluent Bit, ZSTD compression is enabled by default and is treated as a core dependency, not requiring conditional compilation guards like `#ifdef FLB_HAVE_ZSTD`. Unlike some other optional components, ZSTD support is always available and doesn't need build-time conditionals.

Applied to files:

  • CMakeLists.txt
📚 Learning: 2025-08-31T12:46:11.940Z
Learnt from: ThomasDevoogdt
Repo: fluent/fluent-bit PR: 9277
File: .github/workflows/pr-compile-check.yaml:147-151
Timestamp: 2025-08-31T12:46:11.940Z
Learning: In fluent-bit, the correct CMake flag for using system librdkafka is `FLB_PREFER_SYSTEM_LIB_KAFKA=ON`.

Applied to files:

  • CMakeLists.txt
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (76)
  • GitHub Check: PR - container builds / Windows container images (2025)
  • GitHub Check: PR - container builds / Windows container images (2022)
  • GitHub Check: PR - packages build Linux / ubuntu/24.04.arm64v8 package build and stage to S3
  • GitHub Check: PR - packages build Linux / raspbian/bookworm package build and stage to S3
  • GitHub Check: PR - packages build Linux / debian/buster.arm64v8 package build and stage to S3
  • GitHub Check: PR - packages build Linux / ubuntu/24.04 package build and stage to S3
  • GitHub Check: PR - packages build Linux / ubuntu/22.04.arm64v8 package build and stage to S3
  • GitHub Check: PR - packages build Linux / debian/trixie.arm64v8 package build and stage to S3
  • GitHub Check: PR - packages build Linux / ubuntu/22.04 package build and stage to S3
  • GitHub Check: PR - packages build Linux / debian/buster package build and stage to S3
  • GitHub Check: PR - packages build Linux / debian/bullseye.arm64v8 package build and stage to S3
  • GitHub Check: PR - packages build Linux / debian/trixie package build and stage to S3
  • GitHub Check: PR - packages build Linux / debian/bookworm.arm64v8 package build and stage to S3
  • GitHub Check: PR - packages build Linux / rockylinux/10.arm64v8 package build and stage to S3
  • GitHub Check: PR - packages build Linux / debian/bullseye package build and stage to S3
  • GitHub Check: PR - packages build Linux / rockylinux/9 package build and stage to S3
  • GitHub Check: PR - packages build Linux / rockylinux/10 package build and stage to S3
  • GitHub Check: PR - packages build Linux / rockylinux/8.arm64v8 package build and stage to S3
  • GitHub Check: PR - packages build Linux / almalinux/10 package build and stage to S3
  • GitHub Check: PR - packages build Linux / centos/10 package build and stage to S3
  • GitHub Check: PR - packages build Linux / debian/bookworm package build and stage to S3
  • GitHub Check: PR - packages build Linux / almalinux/9.arm64v8 package build and stage to S3
  • GitHub Check: PR - packages build Linux / rockylinux/9.arm64v8 package build and stage to S3
  • GitHub Check: PR - packages build Linux / almalinux/8.arm64v8 package build and stage to S3
  • GitHub Check: PR - packages build Linux / almalinux/10.arm64v8 package build and stage to S3
  • GitHub Check: PR - packages build Linux / almalinux/8 package build and stage to S3
  • GitHub Check: PR - packages build Linux / almalinux/9 package build and stage to S3
  • GitHub Check: PR - packages build Linux / amazonlinux/2023.arm64v8 package build and stage to S3
  • GitHub Check: PR - packages build Linux / rockylinux/8 package build and stage to S3
  • GitHub Check: PR - packages build Linux / centos/10.arm64v8 package build and stage to S3
  • GitHub Check: PR - packages build Linux / amazonlinux/2.arm64v8 package build and stage to S3
  • GitHub Check: PR - packages build Linux / centos/7 package build and stage to S3
  • GitHub Check: PR - packages build Linux / centos/9 package build and stage to S3
  • GitHub Check: PR - packages build Linux / amazonlinux/2 package build and stage to S3
  • GitHub Check: PR - packages build Linux / centos/9.arm64v8 package build and stage to S3
  • GitHub Check: PR - packages build Linux / centos/8.arm64v8 package build and stage to S3
  • GitHub Check: PR - packages build Linux / centos/8 package build and stage to S3
  • GitHub Check: PR - packages build Linux / centos/7.arm64v8 package build and stage to S3
  • GitHub Check: PR - packages build Linux / amazonlinux/2023 package build and stage to S3
  • GitHub Check: PR - container builds / arm/v7/production container image build
  • GitHub Check: PR - container builds / arm/v7/debug container image build
  • GitHub Check: PR - container builds / arm64/production container image build
  • GitHub Check: PR - container builds / arm64/debug container image build
  • GitHub Check: PR - container builds / amd64/production container image build
  • GitHub Check: PR - container builds / amd64/debug container image build
  • GitHub Check: PR - packages build MacOS / call-build-macos-package (Intel macOS runner, macos-14-large, 3.31.6)
  • GitHub Check: PR - packages build MacOS / call-build-macos-package (Apple Silicon macOS runner, macos-14, 3.31.6)
  • GitHub Check: PR - packages build Windows / call-build-windows-package (Windows 64bit, x64, x64-windows-static, 3.31.6)
  • GitHub Check: PR - packages build Windows / call-build-windows-package (Windows 64bit (Arm64), amd64_arm64, -DCMAKE_SYSTEM_NAME=Windows -DCMA...
  • GitHub Check: PR - packages build Windows / call-build-windows-package (Windows 32bit, x86, x86-windows-static, 3.31.6)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=Off, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_ARROW=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=Off, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_THREAD=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_THREAD=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_MEMORY=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_MEMORY=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_COVERAGE=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_UNDEFINED=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_ADDRESS=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_ADDRESS=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_UNDEFINED=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SMALL=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SMALL=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=Off, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=Off, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=On, 3.31.6, gcc, g++)
  • GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, clang, clang++, ubuntu-24.04, clang-14)
  • GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, gcc, g++, ubuntu-22.04, clang-12)
  • GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, clang, clang++, ubuntu-22.04, clang-12)
  • GitHub Check: pr-compile-without-cxx (3.31.6)
  • GitHub Check: pr-compile-centos-7
  • GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, gcc, g++, ubuntu-24.04, clang-14)

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@cosmo0920 cosmo0920 added the ok-package-test Run PR packaging tests label Oct 31, 2025
Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@edsiper
Copy link
Member

edsiper commented Nov 1, 2025

interesting approach.

This is because RHEL10 or other distributions which use zstd compression
inside of systemd, the symbols could be collided and we need to unify
the using zstd symbols.
This is just for changing the way to linking libzstd libraries.
No need to change for building rules in packaging.

Signed-off-by: Hiroshi Hatake <hiroshi@chronosphere.io>
@cosmo0920 cosmo0920 force-pushed the cosmo0920-link-system-zstd-when-enabling-in_systemd branch from cd6eefd to 60e849e Compare November 4, 2025 05:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs-required ok-package-test Run PR packaging tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants