[core] Don't build cpp api on pip install #50499

dayshah · 2025-02-12T23:12:20Z

Why are these changes needed?

Don't want to build the cpp api when using the pip install -e . --verbose command

Related issue number

Checks

I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
I've run scripts/format.sh to lint the changes in this PR.
I've included any doc changes needed for https://docs.ray.io/en/master/.
- I've added any new APIs to the API Reference. For example, if I added a
  method in Tune, I've added it in doc/source/tune/api/ under the
  corresponding .rst file.
I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
Testing Strategy
- Unit tests
- Release tests
- This PR is not tested :(

aslonnie · 2025-02-13T00:13:05Z

python/setup.py

@@ -33,6 +33,7 @@

 ROOT_DIR = os.path.dirname(__file__)
 BUILD_JAVA = os.getenv("RAY_INSTALL_JAVA") == "1"
+BUILD_CPP = os.getenv("RAY_INSTALL_CPP") == "1"


I think this will break release wheel building? at least this is a behavior change.

ray/python/setup.py

Lines 123 to 124 in 02d3f3c

if os.getenv("RAY_INSTALL_CPP") == "1":

# "ray-cpp" wheel package.

wouldn't cpp wheel building go down this path anyways and RAY_INSTALL_CPP has to be true

yeah.. you are right that when we are building for releases, we always set RAY_INSTALL_CPP=1 already.

yeah.. you are right that when we are building for releases, we always set RAY_INSTALL_CPP=1 already.

well, no, actually, RAY_INSTALL_CPP is used to toggle if to build the ray package or the ray-cpp package. it is not for controlling the cpp header files are built.

what cpp header files? we should only build the cpp api for the ray-cpp package right

what cpp header files?

whatever //cpp:ray_cpp_pkg is building.

what I meant is that, for example, before this change:

python setup.py bdist_wheel RAY_INSTALL_CPP=1 SKIP_BAZEL_BUILD=1 python setup.py bdist_wheel

this can correctly generate both the related ray and ray-cpp wheel

after your change, it can no longer generate the right ray-cpp wheel.

The way we actually build the wheels is with:

python setup.py bdist_wheel RAY_INSTALL_CPP=1 python setup.py bdist_wheel

if there are any file that is generated by //cpp:ray_cpp_pkg but packaged in ray wheel, and if it is critical for some reason, the file will be lost now.

ray/java/BUILD.bazel

Lines 204 to 205 in a8e217d

"//cpp:counter.so",

"//cpp:plus.so",

ray/java/BUILD.bazel

Line 108 in a8e217d

visibility = ["//cpp:__subpackages__"],

this is really the only place i see something from //cpp mentioned outside. If there's any files generated outside we'd see build failures or something anyways right

And there's no python api/utility or anything exposed from inside cpp/

aslonnie

talked to @dayshah , will check if this change affects wheel building result . let me know.

Signed-off-by: dayshah <dhyey2019@gmail.com>

dayshah · 2025-03-04T20:03:03Z

talked to @dayshah , will check if this change affects wheel building result . let me know.

@aslonnie tried it out doesn't affect the standard wheel building result for the standard non-cpp ray wheel built on manylinux from directions here https://github.com/ray-project/ray/blob/a692d9bc601e800b758424bf94aee44cb8082955/python/README-building-wheels.md

this is the result for with this change vs. without this change

Filenames only in /home/ubuntu/ray_no_cpp_wheel:
  ray-3.0.0.dev0/ray/_private/runtime_env/agent/thirdparty_files/aiohappyeyeballs-2.4.6.dist-info/INSTALLER
  ray-3.0.0.dev0/ray/_private/runtime_env/agent/thirdparty_files/aiohappyeyeballs-2.4.6.dist-info/LICENSE
  ray-3.0.0.dev0/ray/_private/runtime_env/agent/thirdparty_files/aiohappyeyeballs-2.4.6.dist-info/METADATA
  ray-3.0.0.dev0/ray/_private/runtime_env/agent/thirdparty_files/aiohappyeyeballs-2.4.6.dist-info/RECORD
  ray-3.0.0.dev0/ray/_private/runtime_env/agent/thirdparty_files/aiohappyeyeballs-2.4.6.dist-info/WHEEL

Filenames only in /home/ubuntu/ray_with_cpp_wheel:
  ray-3.0.0.dev0/ray/_private/runtime_env/agent/thirdparty_files/aiohappyeyeballs-2.4.8.dist-info/INSTALLER
  ray-3.0.0.dev0/ray/_private/runtime_env/agent/thirdparty_files/aiohappyeyeballs-2.4.8.dist-info/LICENSE
  ray-3.0.0.dev0/ray/_private/runtime_env/agent/thirdparty_files/aiohappyeyeballs-2.4.8.dist-info/METADATA
  ray-3.0.0.dev0/ray/_private/runtime_env/agent/thirdparty_files/aiohappyeyeballs-2.4.8.dist-info/RECORD
  ray-3.0.0.dev0/ray/_private/runtime_env/agent/thirdparty_files/aiohappyeyeballs-2.4.8.dist-info/WHEEL

aslonnie · 2025-03-04T20:35:01Z

aiohappyeyeballs-2.4.8.dist-info

why it affects the version of this package?

looks like it is because they just released a new version recently..

Don't want to build the cpp api when using the pip install -e . --verbose command --------- Signed-off-by: dayshah <dhyey2019@gmail.com> Signed-off-by: Abrar Sheikh <abrar@anyscale.com>

dayshah requested review from dentiny and aslonnie February 12, 2025 23:12

dayshah assigned dentiny and aslonnie Feb 12, 2025

dayshah requested review from richardliaw and edoakes as code owners February 12, 2025 23:12

dayshah added the go add ONLY when ready to merge, run all tests label Feb 12, 2025

dayshah changed the title ~~[core] dont build cpp api on pip install~~ [core] Don't build cpp api on pip install Feb 12, 2025

dentiny approved these changes Feb 12, 2025

View reviewed changes

aslonnie reviewed Feb 13, 2025

View reviewed changes

aslonnie self-requested a review February 19, 2025 01:30

aslonnie reviewed Feb 20, 2025

View reviewed changes

dayshah force-pushed the cpp-build-pip branch from 18af1fe to b071aa8 Compare March 4, 2025 01:22

dayshah added 2 commits March 3, 2025 17:46

dont build cpp api on pip install

4be0865

Signed-off-by: dayshah <dhyey2019@gmail.com>

build cpp if java or cpp

dd45bd3

Signed-off-by: dayshah <dhyey2019@gmail.com>

dayshah force-pushed the cpp-build-pip branch from b071aa8 to dd45bd3 Compare March 4, 2025 01:46

aslonnie approved these changes Mar 4, 2025

View reviewed changes

aslonnie merged commit 8571381 into ray-project:master Mar 4, 2025
5 checks passed

dayshah deleted the cpp-build-pip branch March 4, 2025 20:41

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[core] Don't build cpp api on pip install #50499

[core] Don't build cpp api on pip install #50499

dayshah commented Feb 12, 2025

aslonnie Feb 13, 2025

dayshah Feb 13, 2025 •

edited

Loading

aslonnie Feb 13, 2025

aslonnie Feb 13, 2025

dayshah Feb 13, 2025

aslonnie Feb 13, 2025 •

edited

Loading

dayshah Feb 18, 2025

aslonnie left a comment

dayshah commented Mar 4, 2025

aslonnie commented Mar 4, 2025

	if os.getenv("RAY_INSTALL_CPP") == "1":
	# "ray-cpp" wheel package.

[core] Don't build cpp api on pip install #50499

[core] Don't build cpp api on pip install #50499

Conversation

dayshah commented Feb 12, 2025

Why are these changes needed?

Related issue number

Checks

aslonnie Feb 13, 2025

Choose a reason for hiding this comment

dayshah Feb 13, 2025 • edited Loading

Choose a reason for hiding this comment

aslonnie Feb 13, 2025

Choose a reason for hiding this comment

aslonnie Feb 13, 2025

Choose a reason for hiding this comment

dayshah Feb 13, 2025

Choose a reason for hiding this comment

aslonnie Feb 13, 2025 • edited Loading

Choose a reason for hiding this comment

dayshah Feb 18, 2025

Choose a reason for hiding this comment

aslonnie left a comment

Choose a reason for hiding this comment

dayshah commented Mar 4, 2025

aslonnie commented Mar 4, 2025

dayshah Feb 13, 2025 •

edited

Loading

aslonnie Feb 13, 2025 •

edited

Loading