Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[design proposal] Add config option 'deploy.config.transformableAllowList' #6236

Merged
merged 2 commits into from
Aug 10, 2021

Conversation

shawnzhu
Copy link
Contributor

@shawnzhu shawnzhu commented Jul 16, 2021

Supports: #4081
Related: Relevant tracking issues, for context
Merge before/after: Dependent or prerequisite PRs

Description
This is important for working with CRDs that needs transform by skaffold.

User facing changes (remove if N/A)

It introduces new schema change so user may need to update the skaffold.yaml to pick up this new feature

@shawnzhu shawnzhu requested a review from a team as a code owner July 16, 2021 21:25
@shawnzhu shawnzhu requested a review from aaron-prindle July 16, 2021 21:25
@google-cla google-cla bot added the cla: yes label Jul 16, 2021
@codecov
Copy link

codecov bot commented Jul 16, 2021

Codecov Report

Merging #6236 (6a048ee) into main (27c3822) will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##             main    #6236   +/-   ##
=======================================
  Coverage   70.80%   70.80%           
=======================================
  Files         494      494           
  Lines       22327    22327           
=======================================
  Hits        15808    15808           
  Misses       5499     5499           
  Partials     1020     1020           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 27c3822...6a048ee. Read the comment docs.

Copy link
Contributor

@tejal29 tejal29 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @viglesiasce for design input.

@google-cla
Copy link

google-cla bot commented Jul 20, 2021

All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter.

We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only @googlebot I consent. in this pull request.

Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the cla label to yes (if enabled on your project).

ℹ️ Googlers: Go here for more info.

@google-cla google-cla bot added cla: no and removed cla: yes labels Jul 20, 2021
@shawnzhu shawnzhu changed the title [design proposal] Add config option 'transformableAllowList' [design proposal] Add config option 'deploy.config.transformableAllowList' Jul 20, 2021
@shawnzhu shawnzhu requested a review from tejal29 July 20, 2021 19:31
@shawnzhu
Copy link
Contributor Author

@tejal29 would you please review again?

@tejal29
Copy link
Contributor

tejal29 commented Jul 21, 2021

@tejal29 would you please review again?

Thanks @shawnzhu. Bringing this in today's team meeting. Will update my review based on discussion there.

@tejal29
Copy link
Contributor

tejal29 commented Jul 22, 2021

We missed covering this in Wed's meeting. I will update the DD soon. Thanks!

Copy link
Member

@briandealwis briandealwis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this proposal! One comment below.

Comment on lines 40 to 41
- Group: example.com
Kind: Application
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice if this could be more compact. And we should also be able to specify versions here too as they may have dramatically different shapes. In Kubernetes, these objects are referred to as resource types, and they have versions.

Then I think we should allow some configurability to determine whether (and where) we can perform certain transforms. Currently Skaffold:

  • applies labels: Skaffold looks for any field named metadata and add a labels block if missing. This action breaks CRDs, which is why we blocked this behind an allowlist.
  • rewrite images: Skaffold looks for any field named image

So I suggest something like the following for the simple cases (rewrite everything based on field names):

- type: pod # no group, implicitly all versions
- type: batch/Job # group, implicitly all versions
- type: extensions/v1beta1/Deployment  # specific version

And we might try something like adding images and labels fields that provide a list of JSON-path-like paths. Maybe with * support?

- type: openfaas.com/v1/Function
  images: [spec.image]
  labels: [spec.metadata.labels, spec.labels]    # https://www.openfaas.com/blog/manage-functions-with-kubectl/
- type: apps/v1/Deployment
  images: [spec.template.spec.initContainers.*.image, spec.template.spec.containers.*.image]
  labels: [spec.metadata.labels, spec.template.metadata.labels]

Nit: these fields should be lower-cased in keeping with the rest (k8s yaml uses lower case)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like such compact configuration and the explict specification for images and labels.

How about this:

  1. adopt the compact configuration
  2. implement applies labels so it only rewrite allowed labels of resource types in allowlist
  3. only rewrite image of resource types in allowlist

Any guidance on the implementation like which module to change would also be appreicated!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shawnzhu shawnzhu requested a review from briandealwis July 28, 2021 21:45
@tejal29 tejal29 added the triage/discuss Items for discussion label Aug 9, 2021
@tejal29
Copy link
Contributor

tejal29 commented Aug 10, 2021

@shawnzhu Sorry for the delay! LGTM!

Do you want to start working on it?

@tejal29 tejal29 merged commit a453306 into GoogleContainerTools:main Aug 10, 2021
@shawnzhu shawnzhu deleted the issue-4081 branch August 11, 2021 02:33
@shawnzhu
Copy link
Contributor Author

@tejal29 Thanks! and yes, let me try.

@tejal29 tejal29 removed the triage/discuss Items for discussion label Aug 17, 2021
@briandealwis
Copy link
Member

Hi @shawnzhu — do you have the bandwidth to continue on with this effort? Can I help?

@shawnzhu
Copy link
Contributor Author

shawnzhu commented Sep 29, 2021

@briandealwis I'm glad you offer help. I was learning how to parse 3 different expressions of resource types with k8s API, but haven't figure out a clue yet. if you can provide pointers or start work on one expression of resource type, that would be awesome.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants