Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ray-packages v2.3.0 #94

Merged
merged 5 commits into from
Feb 26, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ray-patching.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ Initial setup:
* apply all the patches from the current feedstock (which are known to be good);
since they're sorted in the `patches/` folder, you can even use `xargs`, e.g.
```sh
ls -d ../path/to/feedstock/recipe/patches/ | xargs git am
find ../path/to/feedstock/recipe/patches/ | xargs git am
```

Rebase to new version or ray:
Expand Down
10 changes: 5 additions & 5 deletions recipe/meta.yaml
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
{% set version = "2.2.0" %}
{% set version = "2.3.0" %}

package:
name: ray-packages
version: {{ version }}

source:
url: https://github.com/ray-project/ray/archive/ray-{{ version }}.tar.gz
sha256: 1379fa18d9df2d07f501f326a194140b853b676a4dd211bf093c8e1cf3be4878
sha256: 0554b8711db70fae4c0acc8e01674901a84b81405f5247ed933b12ed047ff353
patches:
- patches/0001-Redis-deps-now-build-but-do-not-link.patch
- patches/0002-Disable-making-non-core-entry-scripts.patch
Expand All @@ -16,7 +16,7 @@ source:


build:
number: 2
number: 0
# skip on MacOS, needs macos 10.15
skip: true # [osx]

Expand Down Expand Up @@ -172,7 +172,7 @@ outputs:
- colorful
- gpustat >=1.0.0
- opencensus
- prometheus_client >=0.7.1, <0.14.0
- prometheus_client >=0.7.1
- pydantic
- requests
- smart_open
Expand Down Expand Up @@ -212,7 +212,7 @@ outputs:
- python
- {{ pin_subpackage('ray-tune', exact=True) }}
- dm-tree
- gym >=0.21.0, <0.24
- gymnasium ==0.26.3
- lz4
- matplotlib-base !=3.4.3
- pyyaml
Expand Down
14 changes: 7 additions & 7 deletions recipe/patches/0001-Redis-deps-now-build-but-do-not-link.patch
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
From a439f3419ebff6e7850c04c9161d773b9032a97d Mon Sep 17 00:00:00 2001
From e255a749a292fa1ea13815b82b0108d150359640 Mon Sep 17 00:00:00 2001
From: Vasily Litvinov <vasilij.n.litvinov@intel.com>
Date: Fri, 3 Dec 2021 10:46:36 -0800
Subject: [PATCH 1/5] Redis deps now build but do not link
Expand All @@ -15,7 +15,7 @@ Signed-off-by: H. Vetinari <h.vetinari@gmx.com>
rename thirdparty/patches/{redis-quiet.patch => 0001-keep-redis-quiet-and-override-AR.patch} (53%)

diff --git a/bazel/BUILD.redis b/bazel/BUILD.redis
index 9edccf5780..60e070028f 100644
index 9edccf578..60e070028 100644
--- a/bazel/BUILD.redis
+++ b/bazel/BUILD.redis
@@ -42,13 +42,17 @@ make(
Expand Down Expand Up @@ -50,12 +50,12 @@ index 9edccf5780..60e070028f 100644
tags = ["local"],
)
diff --git a/bazel/ray_deps_setup.bzl b/bazel/ray_deps_setup.bzl
index 31d5f06b95..24c6badfe1 100644
index 916ca4883..b100e7f56 100644
--- a/bazel/ray_deps_setup.bzl
+++ b/bazel/ray_deps_setup.bzl
@@ -102,7 +102,7 @@ def ray_deps_setup():
url = "https://github.com/redis/redis/archive/refs/tags/7.0.5.tar.gz",
sha256 = "40827fcaf188456ad9b3be8e27a4f403c43672b6bb6201192dc15756af6f1eae",
url = "https://github.com/redis/redis/archive/refs/tags/7.0.8.tar.gz",
sha256 = "0e439cbc19f6db5a4c63d355519ab73bf6ac2ecd47df806c14b19564b3d0c593",
patches = [
- "@com_github_ray_project_ray//thirdparty/patches:redis-quiet.patch",
+ "@com_github_ray_project_ray//thirdparty/patches:0001-keep-redis-quiet-and-override-AR.patch",
Expand All @@ -66,7 +66,7 @@ diff --git a/thirdparty/patches/redis-quiet.patch b/thirdparty/patches/0001-keep
similarity index 53%
rename from thirdparty/patches/redis-quiet.patch
rename to thirdparty/patches/0001-keep-redis-quiet-and-override-AR.patch
index d034387596..27c01310ed 100644
index d03438759..27c01310e 100644
--- a/thirdparty/patches/redis-quiet.patch
+++ b/thirdparty/patches/0001-keep-redis-quiet-and-override-AR.patch
@@ -1,14 +1,45 @@
Expand Down Expand Up @@ -199,5 +199,5 @@ index d034387596..27c01310ed 100644
+2.38.1.windows.1
+
--
2.25.1
2.38.1.windows.1

Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
From 9ffdee26e7ba1f2df8eccf9140a0663666cf1663 Mon Sep 17 00:00:00 2001
From 2dc928f645f28ddcd394c61e4a5587a728b36ab1 Mon Sep 17 00:00:00 2001
From: Vasily Litvinov <vasilij.n.litvinov@intel.com>
Date: Tue, 10 Nov 2020 23:26:35 +0300
Subject: [PATCH 2/5] Disable making non-core entry scripts
Expand All @@ -8,7 +8,7 @@ Subject: [PATCH 2/5] Disable making non-core entry scripts
1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/python/setup.py b/python/setup.py
index 0773027a30..e7d838a0c7 100644
index c9d9ae574..6c60acbc5 100644
--- a/python/setup.py
+++ b/python/setup.py
@@ -780,9 +780,9 @@ setuptools.setup(
Expand All @@ -25,5 +25,5 @@ index 0773027a30..e7d838a0c7 100644
},
package_data={
--
2.25.1
2.38.1.windows.1

34 changes: 17 additions & 17 deletions recipe/patches/0003-Remove-all-dependencies-from-setup.py.patch
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
From 9617d0d44f432b2d4081a2af4d1f623c7a04102c Mon Sep 17 00:00:00 2001
From cef3150908795045cc870326b598ac3778a4efc9 Mon Sep 17 00:00:00 2001
From: Kai Fricke <kai@anyscale.com>
Date: Fri, 3 Dec 2021 10:55:23 -0800
Subject: [PATCH 3/5] Remove all dependencies from setup.py
Expand All @@ -10,10 +10,10 @@ Signed-off-by: Matti Picus <matti.picus@gmail.com>
1 file changed, 11 insertions(+), 96 deletions(-)

diff --git a/python/setup.py b/python/setup.py
index e7d838a0c7..83a7171d83 100644
index 6c60acbc5..aa63c8d7d 100644
--- a/python/setup.py
+++ b/python/setup.py
@@ -209,85 +209,21 @@ ray_files += [
@@ -209,82 +209,21 @@ ray_files += [
# also update the matching section of requirements/requirements.txt
# in this directory
if setup_spec.type == SetupType.RAY:
Expand All @@ -26,7 +26,7 @@ index e7d838a0c7..83a7171d83 100644
- # NumPy dropped python 3.6 support in 1.20.
- numpy_dep = "numpy >= 1.19"
- if sys.version_info >= (3, 7) and sys.platform != "win32":
- pyarrow_dep = "pyarrow >= 6.0.1, < 8.0.0"
- pyarrow_dep = "pyarrow >= 6.0.1"
- else:
- # pyarrow dropped python 3.6 support in 7.0.0.
- # Serialization workaround for pyarrow 7.0.0+ doesn't work for Windows.
Expand All @@ -49,16 +49,16 @@ index e7d838a0c7..83a7171d83 100644
- "gpustat >= 1.0.0", # for windows
- "opencensus",
- "pydantic",
- "prometheus_client >= 0.7.1, < 0.14.0",
- "prometheus_client >= 0.7.1",
- "smart_open",
- ],
- "serve": ["uvicorn", "requests", "starlette", "fastapi", "aiorwlock"],
- "tune": ["pandas", "tabulate", "tensorboardX>=1.9", "requests"],
- "k8s": ["kubernetes", "urllib3"],
- "observability": [
- "opentelemetry-api==1.1.0",
- "opentelemetry-sdk==1.1.0",
- "opentelemetry-exporter-otlp==1.1.0",
- "opentelemetry-api",
- "opentelemetry-sdk",
- "opentelemetry-exporter-otlp",
- ],
+ "data": [],
+ "default": [],
Expand All @@ -81,11 +81,8 @@ index e7d838a0c7..83a7171d83 100644

- setup_spec.extras["rllib"] = setup_spec.extras["tune"] + [
- "dm_tree",
- "gym>=0.21.0,<0.24.0",
- "gymnasium==0.26.3",
- "lz4",
- # matplotlib (dependency of scikit-image) 3.4.3 breaks docker build
- # Todo: Remove this when safe?
- "matplotlib!=3.4.3",
- "scikit-image",
- "pyyaml",
- "scipy",
Expand All @@ -108,7 +105,7 @@ index e7d838a0c7..83a7171d83 100644
setup_spec.extras["all"] = list(
set(chain.from_iterable(setup_spec.extras.values()))
)
@@ -300,28 +236,7 @@ if setup_spec.type == SetupType.RAY:
@@ -297,31 +236,7 @@ if setup_spec.type == SetupType.RAY:
# install-core-prerelease-dependencies.sh so we can test
# new releases candidates.
if setup_spec.type == SetupType.RAY:
Expand All @@ -117,8 +114,11 @@ index e7d838a0c7..83a7171d83 100644
- "click >= 7.0",
- "dataclasses; python_version < '3.7'",
- "filelock",
- "grpcio >= 1.32.0; python_version < '3.10'",
- "grpcio >= 1.42.0; python_version >= '3.10'",
- # Tracking issue: https://github.com/ray-project/ray/issues/30984
- "grpcio >= 1.32.0, <= 1.49.1; python_version < '3.10' and sys_platform == 'darwin'", # noqa
- "grpcio >= 1.42.0, <= 1.49.1; python_version >= '3.10' and sys_platform == 'darwin'", # noqa
- "grpcio >= 1.32.0; python_version < '3.10' and sys_platform != 'darwin'",
- "grpcio >= 1.42.0; python_version >= '3.10' and sys_platform != 'darwin'",
- "jsonschema",
- "msgpack >= 1.0.0, < 2.0.0",
- "numpy >= 1.16; python_version < '3.9'",
Expand All @@ -142,11 +142,11 @@ index e7d838a0c7..83a7171d83 100644
# The BinaryDistribution argument triggers build_ext.
distclass=BinaryDistribution,
install_requires=setup_spec.install_requires,
- setup_requires=["cython >= 0.29.26", "wheel"],
- setup_requires=["cython >= 0.29.32", "wheel"],
+ setup_requires=[],
extras_require=setup_spec.extras,
entry_points={
"console_scripts": [
--
2.25.1
2.38.1.windows.1

Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
From 555c2e1fa70bcb2a30645cd2fc4a9c7063155371 Mon Sep 17 00:00:00 2001
From 74e4d4fd2b90427e842a5cd6710530234dff09a8 Mon Sep 17 00:00:00 2001
From: Vasily Litvinov <vasilij.n.litvinov@intel.com>
Date: Fri, 3 Dec 2021 10:59:10 -0800
Subject: [PATCH 4/5] Ignore warnings in event.cc and logging.cc
Expand All @@ -10,7 +10,7 @@ Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com>
1 file changed, 3 insertions(+)

diff --git a/.bazelrc b/.bazelrc
index 1c4b9f296b..bf429439a0 100644
index 1b98b8f72..a04690969 100644
--- a/.bazelrc
+++ b/.bazelrc
@@ -41,6 +41,9 @@ build:clang-cl --per_file_copt="-\\.(asm|S)$@-Werror"
Expand All @@ -24,5 +24,5 @@ index 1c4b9f296b..bf429439a0 100644
#build --per_file_copt="external/.*@-Wno-unused-result"
# Ignore minor warnings for host tools, which we generally can't control
--
2.25.1
2.38.1.windows.1

6 changes: 3 additions & 3 deletions recipe/patches/0005-Add-bazel-linkopts-libs.patch
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
From e42938fdc68489e578a1898df71213db3d1729a9 Mon Sep 17 00:00:00 2001
From 02836d1970b581888d8ad74b97d47c30273e69e1 Mon Sep 17 00:00:00 2001
From: Kai Fricke <kai@anyscale.com>
Date: Fri, 3 Dec 2021 11:04:33 -0800
Subject: [PATCH 5/5] Add bazel linkopts/libs
Expand All @@ -9,7 +9,7 @@ Signed-off-by: Jiajun Yao <jeromeyjj@gmail.com>
1 file changed, 2 insertions(+)

diff --git a/python/setup.py b/python/setup.py
index 83a7171d83..dcd9ee8789 100644
index aa63c8d7d..821dd75ff 100644
--- a/python/setup.py
+++ b/python/setup.py
@@ -414,6 +414,8 @@ def build(build_python, build_java, build_cpp):
Expand All @@ -22,5 +22,5 @@ index 83a7171d83..dcd9ee8789 100644
if is_native_windows_or_msys():
SHELL = bazel_env.get("SHELL")
--
2.25.1
2.38.1.windows.1