-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Makefile performance improvements #1151
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Create a `.make` directory for tracking targets which don't generate a single file output. This should be ignored by git. For targets which either don't generate a single file output, or the output file is committed, we use a "sentinel" file within `.make/` to track the staleness of the target and only rebuild when needed. For each phony target, we create an internal target with the same name, but prefixed with `.make/` where the work is performed. At the end of each internal target we run `@touch $@` to update the file which is the name of the target.
- Depend on the binary rather than the phoney target so tfgen_no_deps can be up-to-date if we make it a file target too. - Group tfgen .PHONY targets near where they're declared.
danielrbradley
force-pushed
the
file-targets
branch
from
November 22, 2024 08:55
a321731
to
2609752
Compare
danielrbradley
force-pushed
the
file-targets
branch
2 times, most recently
from
November 22, 2024 09:47
5f5aab3
to
9d6bc68
Compare
- Upstream depends on the patches and upstream HEAD - Remove confusing whitespace around `XrunUpstreamTools` block. - Move .PHONY entry adjacent to target.
danielrbradley
force-pushed
the
file-targets
branch
from
November 22, 2024 11:03
9d6bc68
to
bf6d6d6
Compare
Add dependency on sources files (including the upstream HEAD) so we rebuild when they're updated. Add an alias of "schema" which is more generic that "tfgen" and can be standardised for all other providers too.
If the target it running but we're already on the right version, we still need to bump the timestamp to avoid re-running this whole check.
- Remove "cleanup" so we only have "clean" for consistency. - The schema.go hasn't existed for a while now so can be removed.
It doesn't output anything useful. More comments & better structure would probably be more helpful.
- Add dependency on bin/$(TFGEN).
If we make the file target depend on _no_deps, it will never be up-to-date. If we make _no_deps depend on the file target it will pull in all dependencies. Instead we extract the build provider command to a variable so it's re-used from both the file target and _no_deps versions.
danielrbradley
force-pushed
the
file-targets
branch
from
November 22, 2024 11:25
bf6d6d6
to
912f23e
Compare
- Separate generate from build for an option for faster local iteration. We don't actually need to run the full SDK build to be able to commit and push to CI. - Extract env vars to a makefile variable to reduce duplication and improve visibility in the logs. Don't include the PATH in the logs though as that's too noisy.
- Auto-generate generate and install_sdks aliases from supported languages.
- Group all install_*_sdk targets together after the build_* targets.
- Move .PHONY declaration next to the target definition.
Let's keep this consistent and make it not the first thing you read in each block.
blampe
added a commit
to pulumi/pulumi-artifactory
that referenced
this pull request
Dec 17, 2024
After pulumi/ci-mgmt#1151 our `make` targets should now all correctly call `make upstream` if they need it. This updates `.ci-mgmt.yaml` and re-runs `make ci-mgmt`.
blampe
added a commit
to pulumi/pulumi-aws
that referenced
this pull request
Dec 17, 2024
After pulumi/ci-mgmt#1151 our `make` targets should now all correctly call `make upstream` if they need it. This PR removes the `make upstream` step from `preTest` and updates `examples/webserver-go/go.mod` to no longer break `make ci-mgmt`.
blampe
added a commit
to pulumi/pulumi-tailscale
that referenced
this pull request
Dec 17, 2024
After pulumi/ci-mgmt#1151 our `make` targets should now all correctly call `make upstream` if they need it. This updates `.ci-mgmt.yaml` and re-runs `make ci-mgmt`.
blampe
added a commit
to pulumi/pulumi-sdwan
that referenced
this pull request
Dec 17, 2024
After pulumi/ci-mgmt#1151 our `make` targets should now all correctly call `make upstream` if they need it. This updates `.ci-mgmt.yaml` and re-runs `make ci-mgmt`.
blampe
added a commit
to pulumi/pulumi-hcloud
that referenced
this pull request
Dec 17, 2024
After pulumi/ci-mgmt#1151 our `make` targets should now all correctly call `make upstream` if they need it. This updates `.ci-mgmt.yaml` and re-runs `make ci-mgmt`.
blampe
added a commit
to pulumi/pulumi-azuredevops
that referenced
this pull request
Dec 17, 2024
After pulumi/ci-mgmt#1151 our `make` targets should now all correctly call `make upstream` if they need it. This updates `.ci-mgmt.yaml` and re-runs `make ci-mgmt`.
blampe
added a commit
to pulumi/pulumi-harness
that referenced
this pull request
Dec 17, 2024
After pulumi/ci-mgmt#1151 our `make` targets should now all correctly call `make upstream` if they need it. This updates `.ci-mgmt.yaml` and re-runs `make ci-mgmt`.
blampe
added a commit
to pulumi/pulumi-digitalocean
that referenced
this pull request
Dec 17, 2024
After pulumi/ci-mgmt#1151 our `make` targets should now all correctly call `make upstream` if they need it. This updates `.ci-mgmt.yaml` and re-runs `make ci-mgmt`.
blampe
added a commit
to pulumi/pulumi-datadog
that referenced
this pull request
Dec 17, 2024
After pulumi/ci-mgmt#1151 our `make` targets should now all correctly call `make upstream` if they need it. This updates `.ci-mgmt.yaml` and re-runs `make ci-mgmt`.
blampe
added a commit
to pulumi/pulumi-meraki
that referenced
this pull request
Dec 17, 2024
After pulumi/ci-mgmt#1151 our `make` targets should now all correctly call `make upstream` if they need it. This updates `.ci-mgmt.yaml` and re-runs `make ci-mgmt`.
blampe
added a commit
to pulumi/pulumi-postgresql
that referenced
this pull request
Dec 17, 2024
After pulumi/ci-mgmt#1151 our `make` targets should now all correctly call `make upstream` if they need it. This updates `.ci-mgmt.yaml` and re-runs `make ci-mgmt`.
blampe
added a commit
to pulumi/pulumi-gitlab
that referenced
this pull request
Dec 17, 2024
After pulumi/ci-mgmt#1151 our `make` targets should now all correctly call `make upstream` if they need it. This updates `.ci-mgmt.yaml` and re-runs `make ci-mgmt`.
blampe
added a commit
to pulumi/pulumi-cloudflare
that referenced
this pull request
Dec 17, 2024
After pulumi/ci-mgmt#1151 our `make` targets should now all correctly call `make upstream` if they need it. This updates `.ci-mgmt.yaml` and re-runs `make ci-mgmt`.
blampe
added a commit
to pulumi/pulumi-dbtcloud
that referenced
this pull request
Dec 17, 2024
After pulumi/ci-mgmt#1151 our `make` targets should now all correctly call `make upstream` if they need it. This updates `.ci-mgmt.yaml` and re-runs `make ci-mgmt`.
blampe
added a commit
to pulumi/pulumi-okta
that referenced
this pull request
Dec 17, 2024
After pulumi/ci-mgmt#1151 our `make` targets should now all correctly call `make upstream` if they need it. This updates `.ci-mgmt.yaml` and re-runs `make ci-mgmt`.
blampe
added a commit
to pulumi/pulumi-ise
that referenced
this pull request
Dec 17, 2024
After pulumi/ci-mgmt#1151 our `make` targets should now all correctly call `make upstream` if they need it. This updates `.ci-mgmt.yaml` and re-runs `make ci-mgmt`.
blampe
added a commit
to pulumi/pulumi-cloudngfwaws
that referenced
this pull request
Dec 17, 2024
After pulumi/ci-mgmt#1151 our `make` targets should now all correctly call `make upstream` if they need it. This updates `.ci-mgmt.yaml` and re-runs `make ci-mgmt`.
blampe
added a commit
to pulumi/pulumi-newrelic
that referenced
this pull request
Dec 17, 2024
After pulumi/ci-mgmt#1151 our `make` targets should now all correctly call `make upstream` if they need it. This updates `.ci-mgmt.yaml` and re-runs `make ci-mgmt`.
blampe
added a commit
to pulumi/pulumi-random
that referenced
this pull request
Dec 17, 2024
After pulumi/ci-mgmt#1151 our `make` targets should now all correctly call `make upstream` if they need it. This updates `.ci-mgmt.yaml` and re-runs `make ci-mgmt`.
blampe
added a commit
to pulumi/pulumi-scm
that referenced
this pull request
Dec 17, 2024
After pulumi/ci-mgmt#1151 our `make` targets should now all correctly call `make upstream` if they need it. This updates `.ci-mgmt.yaml` and re-runs `make ci-mgmt`.
blampe
added a commit
to pulumi/pulumi-pagerduty
that referenced
this pull request
Dec 17, 2024
After pulumi/ci-mgmt#1151 our `make` targets should now all correctly call `make upstream` if they need it. This updates `.ci-mgmt.yaml` and re-runs `make ci-mgmt`.
blampe
added a commit
to pulumi/pulumi-auth0
that referenced
this pull request
Dec 17, 2024
After pulumi/ci-mgmt#1151 our `make` targets should now all correctly call `make upstream` if they need it. This updates `.ci-mgmt.yaml` and re-runs `make ci-mgmt`.
blampe
added a commit
to pulumi/pulumi-azure
that referenced
this pull request
Dec 17, 2024
After pulumi/ci-mgmt#1151 our `make` targets should now all correctly call `make upstream` if they need it. This updates `.ci-mgmt.yaml` and re-runs `make ci-mgmt`.
blampe
added a commit
to pulumi/pulumi-snowflake
that referenced
this pull request
Dec 17, 2024
After pulumi/ci-mgmt#1151 our `make` targets should now all correctly call `make upstream` if they need it. This updates `.ci-mgmt.yaml` and re-runs `make ci-mgmt`.
blampe
added a commit
to pulumi/pulumi-gcp
that referenced
this pull request
Dec 17, 2024
After pulumi/ci-mgmt#1151 our `make` targets should now all correctly call `make upstream` if they need it. This updates `.ci-mgmt.yaml` and re-runs `make ci-mgmt`.
blampe
added a commit
to pulumi/pulumi-vault
that referenced
this pull request
Dec 17, 2024
After pulumi/ci-mgmt#1151 our `make` targets should now all correctly call `make upstream` if they need it. This updates `.ci-mgmt.yaml` and re-runs `make ci-mgmt`.
blampe
added a commit
to pulumi/pulumi-ise
that referenced
this pull request
Dec 17, 2024
After pulumi/ci-mgmt#1151 our `make` targets should now all correctly call `make upstream` if they need it. This updates `.ci-mgmt.yaml` and re-runs `make ci-mgmt`.
blampe
added a commit
to pulumi/pulumi-artifactory
that referenced
this pull request
Dec 18, 2024
After pulumi/ci-mgmt#1151 our `make` targets should now all correctly call `make upstream` if they need it. This updates `.ci-mgmt.yaml` and re-runs `make ci-mgmt`.
blampe
added a commit
to pulumi/pulumi-junipermist
that referenced
this pull request
Dec 18, 2024
After pulumi/ci-mgmt#1151 our `make` targets should now all correctly call `make upstream` if they need it. This updates `.ci-mgmt.yaml` and re-runs `make ci-mgmt`.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Allow targets to be up-to-date by using the "sentinel file" pattern where each unit of work is tracked via a file in the
.make
directory to indicate when it was last successfully completed. Replace dependencies on phony targets with these sentinel files.Other improvements:
make schema
target as a generic replacement formake tfgen
.make generate
target for generating all things which need to be committed without having to wait on build steps.make prepare_local_workspace
which currently encompases installing plugins and preparing upstream..PHONY
declarations near the definition of the phony target rather than in one very long line. This allows us to see if we're missing declarations for specific targets more easily..pulumi/version
getting out of sync with the installed files.clean
target to include thebin
and.make
directories (remove the duplicatecleanup
target alongside).help
target which hasn't been functional in many years AFAIK.Commits are structued to address one target at a time for review and history.
Preview of new help text: