Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow make target customization #1131

Open
blampe opened this issue Nov 7, 2024 · 1 comment
Open

Allow make target customization #1131

blampe opened this issue Nov 7, 2024 · 1 comment
Labels
kind/engineering Work that is not visible to an external user

Comments

@blampe
Copy link
Contributor

blampe commented Nov 7, 2024

Currently, we either ship a fully generated Makefile to providers (bridged) or we leave it entirely up to the provider to implement their own Makefile (native).

There are a number of problems with this:

  • Manually maintained Makefiles can break or become stale as the underlying workflows change or evolve. For example as we look to roll out more generalized test sharding (Invert build tags to improve DX #676) we would need to update all native Makefiles to take advantage.
  • Our generated Makefiles don't expose any entry points for customization. As a result, we've baked a number options into ci-mgmt.yml which in many cases only apply to one or two providers. This isn't scalable -- it's difficult to maintain and it creates a tight coupling between providers and ci-mgmt.

Instead, ci-mgmt should make some default make targets available to each provider while also giving providers the ability to override those targets should they need to. For example, pulumi-eks should be able to use standard build_go, build_python, build_dotnet targets but also be able to define its own build_nodejs since it's a component provider with unique node requirements.

I've prototyped one approach which seems viable:

  • ci-mgmt manages a .mk directory for the provider. This directory includes defaults.mk for default targets, and vars.mk for default variable like PACK=eks.
  • Default targets take the form of something like default.build_nodejs:. The defaults.mk file includes a catch-all rule like so:
    include .mk/vars.mk
    
    default.build_nodejs:
       ...
    
    # Delegate any undefined target "foo" to its respective "default.foo" target,
    # if it exists.
    %:
        @$(MAKE) -f defaults.mk "default.$@"
  • A provider's Makefile can then contain only the following to automatically pick up any default variables and targets. This should be completely sufficient for the boilerplate to participate in CI workflows:
    include .mk/defaults.mk
  • A provider that wished to define custom behavior for itself can then do so as needed:
    include .mk/defaults.mk
    
    lint: # Override default lint target.
        ...
@pulumi-bot pulumi-bot added the needs-triage Needs attention from the triage team label Nov 7, 2024
@guineveresaenger guineveresaenger removed the needs-triage Needs attention from the triage team label Nov 7, 2024
@mikhailshilkov mikhailshilkov added the kind/engineering Work that is not visible to an external user label Nov 18, 2024
@t0yv0
Copy link
Member

t0yv0 commented Dec 18, 2024

We use a simple form of this in pulumi-aws.

https://github.com/pulumi/pulumi-aws/blob/master/.mk/minimal_schema.mk

It's very finnicky though to get the customized targets run at the right time either "before or after" the normal targets.

Perhaps if the default makefile had before_build_nodejs and after_build_nodejs etc targets then these could be used to add behavior in .mk overrides at the precise phase.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/engineering Work that is not visible to an external user
Projects
None yet
Development

No branches or pull requests

5 participants