Skip to content

Conversation

@jameslamb
Copy link
Member

@jameslamb jameslamb commented Aug 6, 2025

Fixes https://github.com/rapidsai/ops/issues/4044 (private issue, sorry)

Contributes to rapidsai/build-planning#197

  • introduces a new section called "Inactive Projects" at the bottom of https://docs.rapids.ai/api/, to host docs links for projects that are no longer maintained.
  • moves cuproj, cusignal, and cuspatial docs links there
  • updates code for copying docs from S3, to account for these "inactive" projects with old/different versions

Notes for Reviewers

How I tested this

The preview page looks right to me: https://deploy-preview-654--docs-rapids-ai.netlify.app/api/

I see:

  • no formatting issues
  • correct versions for all the active projects
  • correct versions for the inactive projects

We'll have to merge and deploy this to test the S3 changes, but I did a dry run and the output looks correct.

patch to remove AWS API calls (click me)
diff --git a/ci/download_from_s3.sh b/ci/download_from_s3.sh
index 4b72289..61abecd 100755
--- a/ci/download_from_s3.sh
+++ b/ci/download_from_s3.sh
@@ -50,17 +50,17 @@ aws_cp() {
   SRC=$1
   DST=$2
 
-  if ! aws s3 ls "${SRC}" > /dev/null; then
-    echo "No files found in ${SRC}. Exiting."
-    exit 1
-  fi
+  # if ! aws s3 ls "${SRC}" > /dev/null; then
+  #   echo "No files found in ${SRC}. Exiting."
+  #   exit 1
+  # fi
 
   echo "Copying ${SRC} to ${DST}"
-  aws s3 cp \
-    --only-show-errors \
-    --recursive \
-    "${SRC}" \
-    "${DST}"
+  # aws s3 cp \
+  #   --only-show-errors \
+  #   --recursive \
+  #   "${SRC}" \
+  #   "${DST}"
 }
 
 # Downloads the RAPIDS libraries' documentation files from S3 and places them
@@ -152,6 +152,6 @@ download_deployment_docs() {
   done
 }
 
-check_dirs
+# check_dirs
 download_lib_docs
 download_deployment_docs

With that patch applied, ran:

./ci/download_from_s3.sh

Saw the expected output... correct versions, nightlies skipped based on configuration in _data/docs.yml, no duplicate libraries.

logs (click me)
--- processing active RAPIDS libraries ---
Copying s3://rapidsai-docs/cudf/html/25.04/ to _site/api/cudf/25.04/
Copying s3://rapidsai-docs/cudf/html/25.08/ to _site/api/cudf/25.08/
Copying s3://rapidsai-docs/cudf/html/25.06/ to _site/api/cudf/25.06/
Copying s3://rapidsai-docs/dask-cudf/html/25.04/ to _site/api/dask-cudf/25.04/
Copying s3://rapidsai-docs/dask-cudf/html/25.08/ to _site/api/dask-cudf/25.08/
Copying s3://rapidsai-docs/dask-cudf/html/25.06/ to _site/api/dask-cudf/25.06/
Copying s3://rapidsai-docs/cuml/html/25.04/ to _site/api/cuml/25.04/
Copying s3://rapidsai-docs/cuml/html/25.08/ to _site/api/cuml/25.08/
Copying s3://rapidsai-docs/cuml/html/25.06/ to _site/api/cuml/25.06/
Copying s3://rapidsai-docs/cugraph/html/25.04/ to _site/api/cugraph/25.04/
Copying s3://rapidsai-docs/cugraph/html/25.08/ to _site/api/cugraph/25.08/
Copying s3://rapidsai-docs/cugraph/html/25.06/ to _site/api/cugraph/25.06/
Copying s3://rapidsai-docs/cuxfilter/html/25.04/ to _site/api/cuxfilter/25.04/
Copying s3://rapidsai-docs/cuxfilter/html/25.08/ to _site/api/cuxfilter/25.08/
Copying s3://rapidsai-docs/cuxfilter/html/25.06/ to _site/api/cuxfilter/25.06/
Copying s3://rapidsai-docs/cudf-java/html/25.04/ to _site/api/cudf-java/25.04/
Skipping: cudf-java | nightly | 25.08
Copying s3://rapidsai-docs/cudf-java/html/25.06/ to _site/api/cudf-java/25.06/
Copying s3://rapidsai-docs/cucim/html/25.04/ to _site/api/cucim/25.04/
Copying s3://rapidsai-docs/cucim/html/25.08/ to _site/api/cucim/25.08/
Copying s3://rapidsai-docs/cucim/html/25.06/ to _site/api/cucim/25.06/
Copying s3://rapidsai-docs/cuvs/html/25.04/ to _site/api/cuvs/25.04/
Copying s3://rapidsai-docs/cuvs/html/25.08/ to _site/api/cuvs/25.08/
Copying s3://rapidsai-docs/cuvs/html/25.06/ to _site/api/cuvs/25.06/
Copying s3://rapidsai-docs/kvikio/html/25.04/ to _site/api/kvikio/25.04/
Copying s3://rapidsai-docs/kvikio/html/25.08/ to _site/api/kvikio/25.08/
Copying s3://rapidsai-docs/kvikio/html/25.06/ to _site/api/kvikio/25.06/
Copying s3://rapidsai-docs/raft/html/25.04/ to _site/api/raft/25.04/
Copying s3://rapidsai-docs/raft/html/25.08/ to _site/api/raft/25.08/
Copying s3://rapidsai-docs/raft/html/25.06/ to _site/api/raft/25.06/
Copying s3://rapidsai-docs/dask-cuda/html/25.04/ to _site/api/dask-cuda/25.04/
Copying s3://rapidsai-docs/dask-cuda/html/25.08/ to _site/api/dask-cuda/25.08/
Copying s3://rapidsai-docs/dask-cuda/html/25.06/ to _site/api/dask-cuda/25.06/
Copying s3://rapidsai-docs/rmm/html/25.04/ to _site/api/rmm/25.04/
Copying s3://rapidsai-docs/rmm/html/25.08/ to _site/api/rmm/25.08/
Copying s3://rapidsai-docs/rmm/html/25.06/ to _site/api/rmm/25.06/
Skipping: rapidsmpf | legacy | 25.04
Copying s3://rapidsai-docs/rapidsmpf/html/25.08/ to _site/api/rapidsmpf/25.08/
Skipping: rapidsmpf | stable | 25.06
Copying s3://rapidsai-docs/librmm/html/25.04/ to _site/api/librmm/25.04/
Copying s3://rapidsai-docs/librmm/html/25.08/ to _site/api/librmm/25.08/
Copying s3://rapidsai-docs/librmm/html/25.06/ to _site/api/librmm/25.06/
Copying s3://rapidsai-docs/libcudf/html/25.04/ to _site/api/libcudf/25.04/
Copying s3://rapidsai-docs/libcudf/html/25.08/ to _site/api/libcudf/25.08/
Copying s3://rapidsai-docs/libcudf/html/25.06/ to _site/api/libcudf/25.06/
Copying s3://rapidsai-docs/libcuml/html/25.04/ to _site/api/libcuml/25.04/
Copying s3://rapidsai-docs/libcuml/html/25.08/ to _site/api/libcuml/25.08/
Copying s3://rapidsai-docs/libcuml/html/25.06/ to _site/api/libcuml/25.06/
Copying s3://rapidsai-docs/libkvikio/html/25.04/ to _site/api/libkvikio/25.04/
Copying s3://rapidsai-docs/libkvikio/html/25.08/ to _site/api/libkvikio/25.08/
Copying s3://rapidsai-docs/libkvikio/html/25.06/ to _site/api/libkvikio/25.06/
Copying s3://rapidsai-docs/libucxx/html/0.43/ to _site/api/libucxx/0.43/
Copying s3://rapidsai-docs/libucxx/html/0.45/ to _site/api/libucxx/0.45/
Copying s3://rapidsai-docs/libucxx/html/0.44/ to _site/api/libucxx/0.44/
Copying s3://rapidsai-docs/rapids-cmake/html/25.04/ to _site/api/rapids-cmake/25.04/
Copying s3://rapidsai-docs/rapids-cmake/html/25.08/ to _site/api/rapids-cmake/25.08/
Copying s3://rapidsai-docs/rapids-cmake/html/25.06/ to _site/api/rapids-cmake/25.06/
--- processing inactive projects ---
Copying s3://rapidsai-docs/cuproj/html/25.02/ to _site/api/cuproj/25.02/
Skipping: cuproj | nightly
Copying s3://rapidsai-docs/cuproj/html/25.04/ to _site/api/cuproj/25.04/
Skipping: cusignal | legacy
Skipping: cusignal | nightly
Skipping: cusignal | stable
Copying s3://rapidsai-docs/cuspatial/html/25.02/ to _site/api/cuspatial/25.02/
Skipping: cuspatial | nightly
Copying s3://rapidsai-docs/cuspatial/html/25.04/ to _site/api/cuspatial/25.04/
Copying s3://rapidsai-docs/libcuproj/html/25.02/ to _site/api/libcuproj/25.02/
Skipping: libcuproj | nightly
Copying s3://rapidsai-docs/libcuproj/html/25.04/ to _site/api/libcuproj/25.04/
Copying s3://rapidsai-docs/libcuspatial/html/25.02/ to _site/api/libcuspatial/25.02/
Skipping: libcuspatial | nightly
Copying s3://rapidsai-docs/libcuspatial/html/25.04/ to _site/api/libcuspatial/25.04/
--- processing deployment docs ---
Copying s3://rapidsai-docs/deployment/html/nightly/ to _site/deployment/nightly/
Copying s3://rapidsai-docs/deployment/html/stable/ to _site/deployment/stable/

Before you review: Liquid is not Jinja!

I learned this doing this PR, maybe other reviewers will already be aware... the templating in this repo looks like Jinja2 (https://jinja.palletsprojects.com/en/stable), but it's actually Liquid (https://shopify.github.io/liquid).

These will be especially useful in understanding this PR's changes:

@jameslamb jameslamb added the doc label Aug 6, 2025
@netlify
Copy link

netlify bot commented Aug 6, 2025

Deploy Preview for docs-rapids-ai ready!

Name Link
🔨 Latest commit 0a752ef
🔍 Latest deploy log https://app.netlify.com/projects/docs-rapids-ai/deploys/6893c806b981a900089d4150
😎 Deploy Preview https://deploy-preview-654--docs-rapids-ai.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@jameslamb jameslamb changed the title WIP: freeze cuspatial and cuproject docs under 'inactive projects' WIP: move cuspatial, cuproj docs under 'inactive projects' Aug 6, 2025
Comment on lines +12 to +13
{%- if api.version-overrides -%}
**[{{ version_name }} ({{ api.version-overrides[version_name] }})](/api/{{ api.path }}/{{ version_name }})**
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These 2 lines are the only new logic. Basically, "if the entry in _data/docs.yml explicitly specifies versions, just use those directly instead of looking up versions in the data from _data/releases.json".

The rest of the diff here is just expanding this out to multiple lines with indentation, to make it easier to read. This doesn't introduce any visual artifacts like weird spacing or line breaks because I've also introduced a lot more whitespace stripping, via {%- and -%}.

For details on how that works in Liquid, see https://shopify.github.io/liquid/basics/whitespace/

for VERSION_NAME in $(jq -r 'keys | .[]' <<< "$VERSION_MAP"); do
for PROJECT in $(yq -r 'keys | .[]' <<< "$PROJECT_MAP"); do
for PROJECT in $(yq -r 'keys | .[]' <<< "$PROJECT_MAP"); do
for VERSION_NAME in $(jq -r 'keys | .[]' <<< "$VERSION_MAP"); do
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Switching the order of these so that in the logs, all the updates are grouped by library. I personally think it's more common to look at these logs with a question like "which files were copied for cudf?" than "which files were copied for all of the nightlies?".

Happy to switch this back if reviewers disagree, I don't feel strongly.

@jameslamb jameslamb changed the title WIP: move cuspatial, cuproj docs under 'inactive projects' move cuspatial, cuproj docs under 'inactive projects' Aug 6, 2025
@jameslamb jameslamb marked this pull request as ready for review August 6, 2025 21:37
@jameslamb jameslamb requested a review from a team as a code owner August 6, 2025 21:37
Copy link
Contributor

@bdice bdice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One possible typo, otherwise LGTM!

#### DOCS {% for version_name in versions -%}
{%- if api.version-overrides -%}
**[{{ version_name }} ({{ api.version-overrides[version_name] }})](/api/{{ api.path }}/{{ version_name }})**
{%- elsif api.name == "libucxx" -%}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check this? elsif seems unusual.

Suggested change
{%- elsif api.name == "libucxx" -%}
{%- elif api.name == "libucxx" -%}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was as surprised as you are to learn that in Liquid, it's literally spelled elsif 😅

https://shopify.github.io/liquid/tags/control-flow/

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's... not Jinja. My bad.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ha no problem, I only just learned that recently through this work.

@jameslamb jameslamb merged commit 6d39343 into main Aug 7, 2025
6 checks passed
@jameslamb jameslamb deleted the inactive-projects branch August 7, 2025 15:51
@jameslamb
Copy link
Member Author

Checked the build after this PR... this is not working yet.

Build link: https://github.com/rapidsai/docs/actions/runs/16807658454/job/47604282737

Noticed this in the logs in the "Post-process docs" section

_site/api/cuproj/--------
  - legacy points to 25.04
---------------


_site/api/cuspatial/--------
  - legacy points to 25.04
---------------

_site/api/libcuproj/--------
  - legacy points to 25.04
---------------


_site/api/libcuspatial/--------
  - legacy points to 25.04
---------------

(build link)

That's not correct. And sure enough, the links at https://docs.rapids.ai/api/ are not doing what we want.

They look right:

image

But for example:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants