Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kustomize localize command to pull all remote data #3980

Closed
monopole opened this issue Jun 10, 2021 · 14 comments
Closed

kustomize localize command to pull all remote data #3980

monopole opened this issue Jun 10, 2021 · 14 comments
Assignees
Labels
kind/feature Categorizes issue or PR as related to a new feature. lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Milestone

Comments

@monopole
Copy link
Contributor

Some kustomize uses cases preclude network use, so it would be nice to capture the raw material from remote sources but still retain the option of performing different builds.

So perhaps

kustomize localize {target} {newDir}

would create a new directory (failing if it already existed), containing a localized, copied version of {target}, such that

kustomize build {target}
kustomize build {newDir}

would produce the same output, but the latter wouldn’t need the network to run.

The kustomization files under {newDir} would have their resources: fields rewritten to point to directories under {newDir}, named to not collide with any copies of {target} subdirectories. The contents of {newDir} would be the result of a graph traversal rooted at {target}.

Likewise, the helm directive would be rewritten to download and use a local chart.
See https://github.com/kubernetes-sigs/kustomize/blob/master/api/types/helmchartargs.go

This feature would benefit from first implementing #3979

@monopole monopole added the kind/feature Categorizes issue or PR as related to a new feature. label Jun 10, 2021
@k8s-ci-robot k8s-ci-robot added the needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. label Jun 10, 2021
@bgrant0607
Copy link
Member

+1. Also could simplify access to private repos (requiring credentials) in some scenarios.

@natasha41575 natasha41575 added the triage/accepted Indicates an issue or PR is ready to be actively worked on. label Jun 10, 2021
@k8s-ci-robot k8s-ci-robot removed the needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. label Jun 10, 2021
@natasha41575 natasha41575 self-assigned this Jun 10, 2021
@marshall007
Copy link

@monopole I think this would be a fantastic addition to kustomize. We currently use kpt pkg commands to achieve roughly the same behavior. There are some friction points with our current approach because kpt is totally unopinionated in terms of the resources it manages, has no concept of package structure (as defined by a kustomization.yaml), and defines its own "package entrypoint" (i.e. a Kptfile vs Krmfile).

All that to say, a kustomize-aware alternative to kpt for package management would be tremendously useful to us. In addition to the use cases you outlined above, I think another very compelling reason to manage packages this way is that it makes it possible to consume/apply setters defined in remote packages.

I think a key piece of functionality necessary to make this feature sucessful would be the ability to intelligently merge upstream changes with changes made in the local package (when the remote is a git repo, that is). This is something that kpt pkg update does quite well and is super valuable. We use this extensively.

@bgrant0607
Copy link
Member

bgrant0607 commented Jul 7, 2021

If setters were changed to an on-the-fly transformer as discussed in #3953, then localize wouldn't be necessary in order to apply setter values to remote resources and merging on update wouldn't be necessary. I suggest that localize should just fetch, not merge, consistent with the existing fetch behavior during build.

@marshall007
Copy link

@bgrant0607 I definitely agree that applying setters on-the-fly is necessary as there should be no distinction between how setters (or any other functionality) behaves when referencing local vs remote resources.

That being said, I did not mean to conflate this with our desire for kustomize-aware package management utilities in general. We have found that it is simply easier to build and maintain kustomizations on top of resources that are declared locally. Also, because remote references are inherently mutable in most cases, we find explicitly requiring a human in the loop when pulling upstream changes to be a desirable property.

Ideally, something like kustomize localize as described (no 3-way merging) would be enough and an engineer would have no reason to modify the local copy of the remote package. In practice, however, it can be significantly easier to maintain minor manual edits to the remote resources that facilitate writing kustomizations/patches. This can include things like:

  • adding a kustomization.yaml and populating resources: if one was not already present in the remote package
  • adding setters that were not originally defined in the remote package
  • updating hard-coded command flags to have their values sourced from env variables (more easily targeted by replacements)
  • replacing static ConfigMap resources in favor of maintaining them exclusively as configMapGenerators

I see all of these use cases I'm describing as an extension of your statement in #3953:

I think of setters as an ad hoc transformer for cases where a dedicated transformer (like images, replicas) hasn't been built yet.

I don't want to go off on too much of a tangent here, but I would like to summarize by adding that we are not super pleased with the direction kpt has started to take as of v1, and hope to see standardization of a "universal package entrypoint" for tooling based on cli-utils. See:

This is our primary motivation for wanting to see the kpt pkg subcommands maintained in cli-utils (and exposed in kustomize).

@bgrant0607
Copy link
Member

We experimented with integrating cli-utils actuation commands into kustomize, but the command architecture didn't make sense from an experience point of view. Integrating cli-utils into kubectl as originally planned (kubernetes/enhancements#810) encountered other challenges.

In the meantime, tools like Helm, Terraform, Pulumi, and dozens of others have already implemented their own mechanisms, as discussed in kubernetes-sigs/cli-utils#364 (comment), so the ship has kind of sailed.

As for "package entrypoint", packaging is out of scope for Kustomize and for Kubernetes:
https://github.com/kubernetes/community/blob/master/contributors/design-proposals/architecture/scope.md#examples-of-projects-and-areas-not-in-scope

@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle stale
  • Mark this issue or PR as rotten with /lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Oct 8, 2021
@natasha41575
Copy link
Contributor

/remove-lifecycle stale

@k8s-ci-robot k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Oct 14, 2021
@k8s-triage-robot
Copy link

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle stale
  • Mark this issue or PR as rotten with /lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jan 12, 2022
@natasha41575
Copy link
Contributor

/remove-lifecycle stale
/lifecycle frozen

@k8s-ci-robot k8s-ci-robot added lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Jan 29, 2022
@annasong20
Copy link
Contributor

/assign

@natasha41575
Copy link
Contributor

@annasong20 and I are working internally on a proposal for this issue. I'm hopeful that we will have it ready in the next couple of weeks.

@bgrant0607
Copy link
Member

Another reason localize may be useful: Remote bases may not work in some embedded uses of kustomize due to the git requirement:
https://github.com/kubernetes-sigs/kustomize/blob/master/api/internal/git/gitrunner.go#L25

@natasha41575
Copy link
Contributor

Linking it here so we don't forget: for alpha, we should define what it means to "not handle" the helm fields: #4933 (comment)

@natasha41575
Copy link
Contributor

Alpha version is done and will be available in the next release. Closing this now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/feature Categorizes issue or PR as related to a new feature. lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness. triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
None yet
Development

No branches or pull requests

8 participants