Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A better solution for skipping CI steps for docs-only changes. #16053

Merged
merged 1 commit into from
Jul 5, 2022

Conversation

benjyw
Copy link
Contributor

@benjyw benjyw commented Jul 5, 2022

The previous solution involved some complexity in extracting the filenames
from JSON, and was not robust to spaces in path names (we do have such
paths under docs/).

Instead we use a different GHA marketplace action. This one is more featureful, and
allows us to specify a subset of files to look at, so we don't need to do the filtering
ourselves. We can just check if the filtered set (everything except docs/**)
has any changes in it.

@benjyw benjyw force-pushed the better_docs_only_ci_fix branch from 38cdd48 to ae2f071 Compare July 5, 2022 00:07
@benjyw benjyw added the category:internal CI, fixes for not-yet-released features, etc. label Jul 5, 2022
@benjyw benjyw requested a review from Eric-Arellano July 5, 2022 00:07
@@ -5,7 +5,7 @@

jobs:
audit:
if: ${{ github.repository_owner == 'pantsbuild' }}
if: github.repository_owner == 'pantsbuild'
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ${{ }} are unnecessary in an if - GHA always evaluates the contents of if as an expression - and removing them makes it easier to add other conditions later by string manipulation.

if: (${{ github.repository_owner == 'pantsbuild' }}) && needs.docs_only_check.outputs.docs_only
== 0
if: (github.repository_owner == 'pantsbuild') && (needs.docs_only_check.outputs.docs_only
!= 'DOCS_ONLY')
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With this new change, we only set the output at all if this is in fact a docs-only change. In other cases we get an empty string. So rather than setting it to "1" or "true", or something that would imply that "0" or "false" are also possible values, we set it to some sentinel string, and we check the inverse with != <the value> instead of == <the inverse value>

with:
format: json
files_ignore: docs/**
files_ignore_separator: '|'
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is future-proofing, in case we have other paths to ignore (see the generator script for details).

- id: docs_only_check
if: steps.files.outputs.any_changed != 'true'
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is how the marketplace action indicates that no files in the set we care about (everything not under docs/) were changed

},
]
)
if get_commit_msg:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The job now requires us to do the checkout, but it doesn't care about the commit message, so we allow fetching it to be skipped.

@benjyw benjyw force-pushed the better_docs_only_ci_fix branch from ae2f071 to 9c03af1 Compare July 5, 2022 01:03

def is_docs_only() -> Jobs:
"""Check if this change only involves docs."""
linux_x86_64_helper = Helper(Platform.LINUX_X86_64)
docs_files = ["docs/**"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense to circle back now that the action can handle something like **/*.md to add this generic pattern to catch / encourage markdown in the source tree along with markdown edits?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I think that could make a lot of sense! Probably after we move off readme.com, since their syncing logic expects a certain directory structure. But if we switch to something where we get to control the layout mapping, we can probably have the best of both worlds.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was actually thinking its good and fine to have docs that are not even part of a doc site. Generally notes to people who actually hack on Pants itself.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That does make sense!

@benjyw benjyw merged commit 04a9db0 into pantsbuild:main Jul 5, 2022
@benjyw benjyw deleted the better_docs_only_ci_fix branch July 5, 2022 03:32
benjyw added a commit to benjyw/pants that referenced this pull request Jul 5, 2022
…build#16053)

The previous solution involved some complexity in extracting the filenames
from JSON, and was not robust to spaces in path names (we do have such
paths under docs/).

Instead we use a different GHA marketplace action. This one is more featureful, and
allows us to specify a subset of files to look at, so we don't need to do the filtering
ourselves. We can just check if the filtered set (everything except docs/**)
has any changes in it.
benjyw added a commit that referenced this pull request Jul 5, 2022
Copy link
Member

@stuhood stuhood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot!

@@ -897,7 +891,7 @@ def merge_ok(needs: list[str], docs_only: bool) -> Jobs:
def generate() -> dict[Path, str]:
"""Generate all YAML configs with repo-relative paths."""

docs_only_cond = "needs.docs_only_check.outputs.docs_only == 0"
not_docs_only = _docs_only_cond(docs_only=False)
pr_jobs = test_workflow_jobs([PYTHON37_VERSION], cron=False)
pr_jobs.update(**is_docs_only())
for key, val in pr_jobs.items():
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like it is nearly time for Job to become a proper class.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category:internal CI, fixes for not-yet-released features, etc.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants