Skip to content

Conversation

@jameslamb
Copy link
Member

After merging #663, the post-processing "customization" process that updates things like drop-down version selectors is not processing all of the files it should be.

See #663 (comment) for details.

This fixes that.

Notes for Reviewers

Speed?

In #663, I said that I saw a post-processing process that's been taking around 1 hour in CI take around 4 minutes for me locally. That was an over-estimate influenced by this accidentally stopping early 😭

Following the testing steps added in #659, with the fixes in this PR it looks like this process will now take more like 20 minutes.

So still noticably faster, even if not as fast as I'd though before 😅

How was this not caught in testing?

In the logs I see in the build linked from #663 (comment), raft is the first library processed and it immediately hits this issue.

When I run this locally, several other libraries are processed before that, and don't hit this issue.

Locally this prints thousands of lines of logs and exited with 0, so I just assumed it was working correctly and didn't carefully check those logs.

I suspect the difference in order is a result of different Python versions... no guarantees are made about the ordering of dictionary keys, and this code is driven by some YAML/JSON config files converted to Python dictionaries.

@jameslamb jameslamb requested a review from a team as a code owner August 11, 2025 15:59
@netlify
Copy link

netlify bot commented Aug 11, 2025

Deploy Preview for docs-rapids-ai ready!

Built without sensitive environment variables

Name Link
🔨 Latest commit e529933
🔍 Latest deploy log https://app.netlify.com/projects/docs-rapids-ai/deploys/689a178d1d7804000856760e
😎 Deploy Preview https://deploy-preview-664--docs-rapids-ai.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@AyodeAwe
Copy link
Contributor

@jameslamb What do you think about a README.md in the ci directory that describes the end-to-end processing logic that occurs in this repo?

I believe this information would be helpful moving forward for everyone who will do similar work here.


echo "Customizing $(wc -l < ${MANIFEST_FILE} | tr -d ' ') HTML files"
python ${SCRIPT_SRC_FOLDER}/customize_doc.py "${MANIFEST_FILE}"
python -u ${SCRIPT_SRC_FOLDER}/customize_doc.py "${MANIFEST_FILE}"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For those finding this PR from search... adding -u ensures that Python stdout / stderr are printed immediately ("unbuffered").

I found that the warning about unrecognized sphinx themes, which goes to stderr, was not being printed immediately after the log messages about that same file printed to stdout.

Removing buffering fixes that.

@jameslamb
Copy link
Member Author

Thanks for the quick review!

What do you think about a README.md in the ci directory that describes the end-to-end processing logic that occurs in this repo?

There is already https://github.com/rapidsai/docs/blob/main/ci/customization/README.md for the "customization" part, but it depends on other things not in that customization/ directory, like ci/update_symlinks.sh.

Sure! I'll update / add docs in a follow-up PR.

I'll merge this so we can get feedback from another deployment.

@jameslamb jameslamb merged commit 0999adf into rapidsai:main Aug 11, 2025
6 checks passed
@jameslamb jameslamb deleted the fix/post-processing branch August 11, 2025 16:19
@jameslamb
Copy link
Member Author

jameslamb commented Aug 11, 2025

Ok yep this worked!

Build link: https://github.com/rapidsai/docs/actions/runs/16885825806/job/47833203056

I see several places where this warning was emitted but didn't stop the build, for example:

...
--- _site/api/raft/25.10/_static/webpack-macros.html ---
Couldn't identify _site/api/raft/25.10/_static/webpack-macros.html as a supported theme type. Skipping file.
--- _site/api/raft/25.08/genindex/index.html ---
default lib: raft
default version: stable
...

(logs link)

and

...
--- _site/api/dask-cudf/25.08/index.html ---
default lib: dask-cudf
default version: stable
--- _site/api/dask-cudf/25.08/_static/webpack-macros.html ---
Couldn't identify _site/api/dask-cudf/25.08/_static/webpack-macros.html as a supported theme type. Skipping file.
--- _site/api/dask-cudf/25.06/genindex/index.html ---
default lib: dask-cudf
default version: legacy
...

(logs link)

And the drop-down selectors and other things look correct (checked after clearing my browser cache, to be sure):

The post-processing took about 20 minutes and the entire deployment took about 30 minutes... so this still cut the deployment time by 30-40 minutes relative to what it was before this set of PRs 😁

@ajschmidt8
Copy link
Member

The post-processing took about 20 minutes and the entire deployment took about 30 minutes... so this still cut the deployment time by 30-40 minutes relative to what it was before this set of PRs 😁

Great work!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants