GH-48248: [C++] Use FetchContent for bundled gRPC #48250

raulcd · 2025-11-25T11:04:08Z

Rationale for this change

As a follow up of requiring a minimum CMake version >= 3.25 we discussed moving our dependencies from ExternalProject to FetchContent. This can simplify our third party dependency management.

What changes are included in this PR?

The general change is moving gRPC from ExternalProject to FetchContent.

We can clean up previously required installation on RE2 and C-Ares. We can't do it for Abseil or protobuf as those are also used on other dependencies. We will be able to clean those later.

Are these changes tested?

Yes, the changes are tested locally and on CI.

Are there any user-facing changes?

No

GitHub Issue: [C++] Use FetchContent for bundled gRPC #48248

raulcd · 2025-11-25T12:20:39Z

@github-actions crossbow submit -g cpp

github-actions · 2025-11-25T12:24:07Z

Revision: 9386a28

Submitted crossbow builds: ursacomputing/crossbow @ actions-bbf70f60fe

Task	Status
example-cpp-minimal-build-static
example-cpp-minimal-build-static-system-dependency
example-cpp-tutorial
test-build-cpp-fuzz
test-conda-cpp
test-conda-cpp-valgrind
test-cuda-cpp-ubuntu-22.04-cuda-11.7.1
test-debian-12-cpp-amd64
test-debian-12-cpp-i386
test-fedora-42-cpp
test-ubuntu-22.04-cpp
test-ubuntu-22.04-cpp-20
test-ubuntu-22.04-cpp-bundled
test-ubuntu-22.04-cpp-emscripten
test-ubuntu-22.04-cpp-no-threading
test-ubuntu-24.04-cpp
test-ubuntu-24.04-cpp-bundled-offline
test-ubuntu-24.04-cpp-gcc-13-bundled
test-ubuntu-24.04-cpp-gcc-14
test-ubuntu-24.04-cpp-minimal-with-formats
test-ubuntu-24.04-cpp-thread-sanitizer

raulcd · 2025-11-25T13:21:39Z

@github-actions crossbow submit -g r

github-actions · 2025-11-25T13:24:34Z

Revision: 9386a28

Submitted crossbow builds: ursacomputing/crossbow @ actions-b3bd179582

Task	Status
r-binary-packages
r-recheck-most
test-r-arrow-backwards-compatibility
test-r-depsource-bundled
test-r-depsource-system
test-r-dev-duckdb
test-r-devdocs
test-r-extra-packages
test-r-gcc-11
test-r-gcc-12
test-r-install-local
test-r-install-local-minsizerel
test-r-linux-as-cran
test-r-linux-rchk
test-r-linux-sanitizers
test-r-linux-valgrind
test-r-m1-san
test-r-macos-as-cran
test-r-minimal-build
test-r-offline-maximal
test-r-offline-minimal
test-r-rhub-debian-gcc-devel-lto-latest
test-r-rhub-ubuntu-gcc12-custom-ccache
test-r-rhub-ubuntu-release-latest
test-r-rocker-r-ver-latest
test-r-rstudio-r-base-4.1-focal
test-r-rstudio-r-base-4.2-focal
test-r-ubuntu-22.04
test-r-versions

raulcd · 2025-11-26T13:57:10Z

@github-actions crossbow submit -g cpp

github-actions · 2025-11-26T13:59:41Z

Revision: 1144f25

Submitted crossbow builds: ursacomputing/crossbow @ actions-c9de41cc83

Task	Status
example-cpp-minimal-build-static
example-cpp-minimal-build-static-system-dependency
example-cpp-tutorial
test-build-cpp-fuzz
test-conda-cpp
test-conda-cpp-valgrind
test-cuda-cpp-ubuntu-22.04-cuda-11.7.1
test-debian-12-cpp-amd64
test-debian-12-cpp-i386
test-fedora-42-cpp
test-ubuntu-22.04-cpp
test-ubuntu-22.04-cpp-20
test-ubuntu-22.04-cpp-bundled
test-ubuntu-22.04-cpp-emscripten
test-ubuntu-22.04-cpp-no-threading
test-ubuntu-24.04-cpp
test-ubuntu-24.04-cpp-bundled-offline
test-ubuntu-24.04-cpp-gcc-13-bundled
test-ubuntu-24.04-cpp-gcc-14
test-ubuntu-24.04-cpp-minimal-with-formats
test-ubuntu-24.04-cpp-thread-sanitizer

raulcd · 2025-11-26T15:46:49Z

@github-actions crossbow submit test-ubuntu-22.04-cpp-bundled

github-actions · 2025-11-26T15:49:05Z

Revision: 9bddf67

Submitted crossbow builds: ursacomputing/crossbow @ actions-de489c8583

Task	Status
test-ubuntu-22.04-cpp-bundled

raulcd · 2025-11-26T15:58:25Z

@github-actions crossbow submit -g r

github-actions · 2025-11-26T16:01:38Z

Revision: 9bddf67

Submitted crossbow builds: ursacomputing/crossbow @ actions-3f45b50a5c

Task	Status
r-binary-packages
r-recheck-most
test-r-arrow-backwards-compatibility
test-r-depsource-bundled
test-r-depsource-system
test-r-dev-duckdb
test-r-devdocs
test-r-extra-packages
test-r-gcc-11
test-r-gcc-12
test-r-install-local
test-r-install-local-minsizerel
test-r-linux-as-cran
test-r-linux-rchk
test-r-linux-sanitizers
test-r-linux-valgrind
test-r-m1-san
test-r-macos-as-cran
test-r-minimal-build
test-r-offline-maximal
test-r-offline-minimal
test-r-rhub-debian-gcc-devel-lto-latest
test-r-rhub-ubuntu-gcc12-custom-ccache
test-r-rhub-ubuntu-release-latest
test-r-rocker-r-ver-latest
test-r-rstudio-r-base-4.1-focal
test-r-rstudio-r-base-4.2-focal
test-r-ubuntu-22.04
test-r-versions

raulcd · 2025-11-26T17:26:31Z

@github-actions crossbow submit -g cpp

github-actions · 2025-11-26T17:29:06Z

Revision: 9bddf67

Submitted crossbow builds: ursacomputing/crossbow @ actions-0e07cbb9ef

Task	Status
example-cpp-minimal-build-static
example-cpp-minimal-build-static-system-dependency
example-cpp-tutorial
test-build-cpp-fuzz
test-conda-cpp
test-conda-cpp-valgrind
test-cuda-cpp-ubuntu-22.04-cuda-11.7.1
test-debian-12-cpp-amd64
test-debian-12-cpp-i386
test-fedora-42-cpp
test-ubuntu-22.04-cpp
test-ubuntu-22.04-cpp-20
test-ubuntu-22.04-cpp-bundled
test-ubuntu-22.04-cpp-emscripten
test-ubuntu-22.04-cpp-no-threading
test-ubuntu-24.04-cpp
test-ubuntu-24.04-cpp-bundled-offline
test-ubuntu-24.04-cpp-gcc-13-bundled
test-ubuntu-24.04-cpp-gcc-14
test-ubuntu-24.04-cpp-minimal-with-formats
test-ubuntu-24.04-cpp-thread-sanitizer

…t and make config conditional to vendored third parties

…ile_options

… be a submodule of grpc

…override

raulcd · 2025-11-28T11:59:47Z

@github-actions crossbow submit -g cpp

github-actions · 2025-11-28T12:02:24Z

Revision: a219eff

Submitted crossbow builds: ursacomputing/crossbow @ actions-b3962c3cec

Task	Status
example-cpp-minimal-build-static
example-cpp-minimal-build-static-system-dependency
example-cpp-tutorial
test-build-cpp-fuzz
test-conda-cpp
test-conda-cpp-valgrind
test-cuda-cpp-ubuntu-22.04-cuda-11.7.1
test-cuda-cpp-ubuntu-24.04-cuda-13.0.2
test-debian-12-cpp-amd64
test-debian-12-cpp-i386
test-fedora-42-cpp
test-ubuntu-22.04-cpp
test-ubuntu-22.04-cpp-20
test-ubuntu-22.04-cpp-bundled
test-ubuntu-22.04-cpp-emscripten
test-ubuntu-22.04-cpp-no-threading
test-ubuntu-24.04-cpp
test-ubuntu-24.04-cpp-bundled-offline
test-ubuntu-24.04-cpp-gcc-13-bundled
test-ubuntu-24.04-cpp-gcc-14
test-ubuntu-24.04-cpp-minimal-with-formats
test-ubuntu-24.04-cpp-thread-sanitizer

raulcd · 2025-11-28T12:39:01Z

@github-actions crossbow submit -g r

github-actions · 2025-11-28T12:41:55Z

Revision: a219eff

Submitted crossbow builds: ursacomputing/crossbow @ actions-ada8c5b28b

Task	Status
r-binary-packages
r-recheck-most
test-r-arrow-backwards-compatibility
test-r-depsource-bundled
test-r-depsource-system
test-r-dev-duckdb
test-r-devdocs
test-r-extra-packages
test-r-gcc-11
test-r-gcc-12
test-r-install-local
test-r-install-local-minsizerel
test-r-linux-as-cran
test-r-linux-rchk
test-r-linux-sanitizers
test-r-linux-valgrind
test-r-m1-san
test-r-macos-as-cran
test-r-minimal-build
test-r-offline-maximal
test-r-offline-minimal
test-r-rhub-debian-gcc-devel-lto-latest
test-r-rhub-ubuntu-gcc12-custom-ccache
test-r-rhub-ubuntu-release-latest
test-r-rocker-r-ver-latest
test-r-rstudio-r-base-4.1-focal
test-r-rstudio-r-base-4.2-focal
test-r-ubuntu-22.04
test-r-versions

kou · 2025-11-29T02:14:27Z

cpp/cmake_modules/ThirdpartyToolchain.cmake

                       URL ${RE2_SOURCE_URL}
-                       URL_HASH "SHA256=${ARROW_RE2_BUILD_SHA256_CHECKSUM}")
+                       URL_HASH "SHA256=${ARROW_RE2_BUILD_SHA256_CHECKSUM}"
+                       EXCLUDE_FROM_ALL)


Could you use ${FC_DECLARE_COMMON_OPTIONS} instead like

arrow/cpp/cmake_modules/ThirdpartyToolchain.cmake

Lines 1770 to 1774 in 5aa7dd1

fetchcontent_declare(thrift

${FC_DECLARE_COMMON_OPTIONS}

PATCH_COMMAND ${THRIFT_PATCH_COMMAND}

URL ${THRIFT_SOURCE_URL}

URL_HASH "SHA256=${ARROW_THRIFT_BUILD_SHA256_CHECKSUM}")

?

kou · 2025-11-29T02:15:05Z

cpp/cmake_modules/ThirdpartyToolchain.cmake

+  # Disable install rules for RE2 so it is not installed on our Linux-packages.
+  set(CMAKE_SKIP_INSTALL_RULES ON)


Can we use

arrow/cpp/cmake_modules/ThirdpartyToolchain.cmake

Lines 1818 to 1820 in 5aa7dd1

if(CMAKE_VERSION VERSION_LESS 3.28)

set_property(DIRECTORY ${thrift_SOURCE_DIR} PROPERTY EXCLUDE_FROM_ALL TRUE)

endif()

instead of this?

kou · 2025-11-29T02:20:33Z

cpp/cmake_modules/ThirdpartyToolchain.cmake

+  if(PROTOBUF_VENDORED)
+    set(_gRPC_PROTOBUF_LIBRARIES
+        "protobuf::libprotobuf"
+        CACHE STRING "" FORCE)


Can we use normal variable not cache variable for all gRPC related variables?

github-actions bot added Component: C++ Component: FlightRPC awaiting committer review Awaiting committer review labels Nov 25, 2025

github-actions bot removed the Component: FlightRPC label Nov 26, 2025

raulcd force-pushed the GH-48248 branch from d458a43 to 9bddf67 Compare November 26, 2025 15:42

raulcd added the CI: Extra Run extra CI label Nov 26, 2025

raulcd added 9 commits November 28, 2025 11:06

apacheGH-48248: [C++] Use FetchContent for bundled gRPC

b3af2bf

Some more cleaning after initial tests

de072aa

Some more cleanup

e91fcaa

Move configuration after fetchcontent_declare and prepare_fetchconten…

169ddd4

…t and make config conditional to vendored third parties

Move the '-Wno-error=deprecated-declarations' to a common target_comp…

ee02044

…ile_options

Add package providers for cares and re2 if not vendored as they won't…

43b2e02

… be a submodule of grpc

Always use None as RE and C-Ares provider

4b0f7cb

Add back grpc fix for ar -M for bundling dependencies together

d9946a2

Do not install RE2 and force cache variables so linux packages don't …

6f1ddbc

…override

raulcd force-pushed the GH-48248 branch from f98b026 to 84f2f87 Compare November 28, 2025 10:07

raulcd added 2 commits November 28, 2025 12:08

Do not include headers on install to avoid adding them on RPMs

c737ee7

Remove unnecessary EXCLUDE_FROM_ALL on cares fetchcontent declare

a219eff

raulcd force-pushed the GH-48248 branch from 84f2f87 to a219eff Compare November 28, 2025 11:30

raulcd marked this pull request as ready for review November 28, 2025 13:52

raulcd requested a review from kou November 28, 2025 13:52

kou reviewed Nov 29, 2025

View reviewed changes

github-actions bot added awaiting changes Awaiting changes and removed awaiting committer review Awaiting committer review labels Nov 29, 2025

	fetchcontent_declare(thrift
	${FC_DECLARE_COMMON_OPTIONS}
	PATCH_COMMAND ${THRIFT_PATCH_COMMAND}
	URL ${THRIFT_SOURCE_URL}
	URL_HASH "SHA256=${ARROW_THRIFT_BUILD_SHA256_CHECKSUM}")

		# Disable install rules for RE2 so it is not installed on our Linux-packages.
		set(CMAKE_SKIP_INSTALL_RULES ON)

	if(CMAKE_VERSION VERSION_LESS 3.28)
	set_property(DIRECTORY ${thrift_SOURCE_DIR} PROPERTY EXCLUDE_FROM_ALL TRUE)
	endif()

GH-48248: [C++] Use FetchContent for bundled gRPC #48250

Are you sure you want to change the base?

GH-48248: [C++] Use FetchContent for bundled gRPC #48250

Conversation

raulcd commented Nov 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

raulcd commented Nov 25, 2025

Uh oh!

github-actions bot commented Nov 25, 2025

Uh oh!

raulcd commented Nov 25, 2025

Uh oh!

github-actions bot commented Nov 25, 2025

Uh oh!

raulcd commented Nov 26, 2025

Uh oh!

github-actions bot commented Nov 26, 2025

Uh oh!

raulcd commented Nov 26, 2025

Uh oh!

github-actions bot commented Nov 26, 2025

Uh oh!

raulcd commented Nov 26, 2025

Uh oh!

github-actions bot commented Nov 26, 2025

Uh oh!

raulcd commented Nov 26, 2025

Uh oh!

github-actions bot commented Nov 26, 2025

Uh oh!

raulcd commented Nov 28, 2025

Uh oh!

github-actions bot commented Nov 28, 2025

Uh oh!

raulcd commented Nov 28, 2025

Uh oh!

github-actions bot commented Nov 28, 2025

Uh oh!

kou Nov 29, 2025

Choose a reason for hiding this comment

Uh oh!

kou Nov 29, 2025

Choose a reason for hiding this comment

Uh oh!

kou Nov 29, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

raulcd commented Nov 25, 2025 •

edited

Loading