Skip to content
This repository has been archived by the owner on Oct 10, 2023. It is now read-only.

[WIP]Implement packages installation APIs for cluster essential #4503

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

Vandy-P
Copy link
Contributor

@Vandy-P Vandy-P commented Mar 19, 2023

What this PR does / why we need it

Implementing APIs for cluster essential installation

Which issue(s) this PR fixes

Fixes #

Describe testing done for PR

Release note


Additional information

Special notes for your reviewer

@Vandy-P Vandy-P requested a review from a team as a code owner March 19, 2023 16:33
cluster-essentials/clusterinit.go Outdated Show resolved Hide resolved
cluster-essentials/clusterinit.go Outdated Show resolved Hide resolved
cluster-essentials/clusterinit.go Outdated Show resolved Hide resolved
cluster-essentials/clusterinit.go Outdated Show resolved Hide resolved
@codecov
Copy link

codecov bot commented Mar 21, 2023

Codecov Report

Merging #4503 (40e6a97) into main (bb07e03) will decrease coverage by 0.92%.
The diff coverage is n/a.

❗ Current head 40e6a97 differs from pull request most recent head 734920f. Consider uploading reports for the commit 734920f to get more accurate results

@@            Coverage Diff             @@
##             main    #4503      +/-   ##
==========================================
- Coverage   49.77%   48.86%   -0.92%     
==========================================
  Files         453      483      +30     
  Lines       45424    47544    +2120     
==========================================
+ Hits        22612    23230     +618     
- Misses      20652    22099    +1447     
- Partials     2160     2215      +55     

see 35 files with indirect coverage changes

Copy link
Contributor

@codegold79 codegold79 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems pretty straight forward to me. I had a couple questions and minor, optional feedback. None block my approval, so I approve.

cluster-essentials/clusterinit.go Outdated Show resolved Hide resolved
cluster-essentials/clusterinit.go Outdated Show resolved Hide resolved
util/carvelhelpers/kbld.go Outdated Show resolved Hide resolved
util/carvelhelpers/package.go Outdated Show resolved Hide resolved
util/carvelhelpers/package.go Outdated Show resolved Hide resolved
// Also creates missing directories if any
func SaveFile(filePath string, data []byte) error {
dirName := filepath.Dir(filePath)
if _, serr := os.Stat(dirName); serr != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't tell, but looks like the serr information doesn't get passed along. Do you think that's all right? Maybe people don't need to know what serr is because you pass along merr later? Kind of strange, but I guess it can work.

@Vandy-P Vandy-P requested review from a team and prkalle as code owners March 23, 2023 06:06
@github-actions
Copy link

Cluster Generation A/B Results:
https://storage.googleapis.com/tkg-clustergen/4503/20230323062110/clustergen.diff.txt
Author/reviewers:
Please review to verify that the effects on the generated cluster configurations are exactly what the PR intended, and give a thumbs-up if so.

@github-actions
Copy link

Cluster Generation A/B Results:
https://storage.googleapis.com/tkg-clustergen/4503/20230323065946/clustergen.diff.txt
Author/reviewers:
Please review to verify that the effects on the generated cluster configurations are exactly what the PR intended, and give a thumbs-up if so.

@github-actions
Copy link

Cluster Generation A/B Results:
https://storage.googleapis.com/tkg-clustergen/4503/20230323114346/clustergen.diff.txt
Author/reviewers:
Please review to verify that the effects on the generated cluster configurations are exactly what the PR intended, and give a thumbs-up if so.

@github-actions
Copy link

Cluster Generation A/B Results:
https://storage.googleapis.com/tkg-clustergen/4503/20230323170259/clustergen.diff.txt
Author/reviewers:
Please review to verify that the effects on the generated cluster configurations are exactly what the PR intended, and give a thumbs-up if so.

@github-actions
Copy link

Cluster Generation A/B Results:
https://storage.googleapis.com/tkg-clustergen/4503/20230323190717/clustergen.diff.txt
Author/reviewers:
Please review to verify that the effects on the generated cluster configurations are exactly what the PR intended, and give a thumbs-up if so.

@github-actions
Copy link

Cluster Generation A/B Results:
https://storage.googleapis.com/tkg-clustergen/4503/20230324063347/clustergen.diff.txt
Author/reviewers:
Please review to verify that the effects on the generated cluster configurations are exactly what the PR intended, and give a thumbs-up if so.

@github-actions
Copy link

Cluster Generation A/B Results:
https://storage.googleapis.com/tkg-clustergen/4503/20230326174544/clustergen.diff.txt
Author/reviewers:
Please review to verify that the effects on the generated cluster configurations are exactly what the PR intended, and give a thumbs-up if so.

@github-actions
Copy link

Cluster Generation A/B Results:
https://storage.googleapis.com/tkg-clustergen/4503/20230326181447/clustergen.diff.txt
Author/reviewers:
Please review to verify that the effects on the generated cluster configurations are exactly what the PR intended, and give a thumbs-up if so.

@github-actions
Copy link

Cluster Generation A/B Results:
https://storage.googleapis.com/tkg-clustergen/4503/20230326183605/clustergen.diff.txt
Author/reviewers:
Please review to verify that the effects on the generated cluster configurations are exactly what the PR intended, and give a thumbs-up if so.

@danniel1205
Copy link
Contributor

Question: Why do we have go mod change in other modules ? (Those modules seem unrelated to your change)

@Vandy-P
Copy link
Contributor Author

Vandy-P commented Mar 27, 2023

@danniel1205 We created kube utility function [util/kube/kube_util.go ] which needs to use client-go v0.25.0 hence all the dependent packages which are using util package got updated with client-go version v0.25.0.

Copy link
Contributor

@avi-08 avi-08 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@rajathagasthya rajathagasthya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested some minor changes.

cluster-essentials/pkg/client/fakes.go Outdated Show resolved Hide resolved
cluster-essentials/pkg/client/helpers.go Outdated Show resolved Hide resolved
cluster-essentials/pkg/client/helpers.go Outdated Show resolved Hide resolved
cluster-essentials/pkg/client/helpers.go Outdated Show resolved Hide resolved
cluster-essentials/pkg/client/install.go Outdated Show resolved Hide resolved
cluster-essentials/pkg/client/install.go Outdated Show resolved Hide resolved
@sathyanarays
Copy link
Contributor

Should this be merged with main or feature/runtime-core?

cluster-essentials/pkg/client/install.go Outdated Show resolved Hide resolved
cluster-essentials/pkg/client/README.md Outdated Show resolved Hide resolved
cluster-essentials/pkg/client/install.go Show resolved Hide resolved
}
var generatedManifestBytes []byte
for _, item := range packages {
packageDir := filepath.Join(bundleDir, item.Name())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i'm confused, you are saying this Install function, expects a imgpkg bundle path, but then here you say each package contains a .imgpkg directory, how can a bundle contain multiple .imgpkg directories? how does the structure of imgpkg bundle look like? may be tree structure in the documentation helps

Copy link
Contributor Author

@Vandy-P Vandy-P Apr 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

├── kapp-controller
│   ├── config
│   │   ├── overlays
│   │   │   ├── change-namespace.yaml
│   │   │   ├── update-clusterrole-with-psp.yaml
│   │   │   ├── update-configmap.yaml
│   │   │   ├── update-crds.yaml
│   │   │   ├── update-deployment.yaml
│   │   │   └── update-strategy-overlay.yaml
│   │   ├── upstream
│   │   │   └── kapp-controller.yaml
│   │   ├── values.star
│   │   └── values.yaml
│   └── .imgpkg
│   └── imgpkg.yml
└── secretgen-controller
├── config
│   └── secretgen-controller.yaml
└── .imgpkg
└── imgpkg.yml

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yharish991 each package is bundle itself

cluster-essentials/pkg/client/install.go Outdated Show resolved Hide resolved
cluster-essentials/pkg/client/install.go Outdated Show resolved Hide resolved
cluster-essentials/pkg/client/install.go Show resolved Hide resolved
cluster-essentials/pkg/client/install.go Outdated Show resolved Hide resolved
cluster-essentials/pkg/client/install.go Outdated Show resolved Hide resolved
cluster-essentials/pkg/client/install.go Outdated Show resolved Hide resolved
cluster-essentials/pkg/client/install.go Outdated Show resolved Hide resolved
@Vandy-P Vandy-P requested a review from a team as a code owner May 1, 2023 20:36
cluster-essentials/pkg/client/install.go Outdated Show resolved Hide resolved
// This function attempts to install packages from the given imgpkg bundle path
// and waits until install is complete or the given timeout is reached, whichever
// occurs first.
func Install(ctx context.Context, config *rest.Config, clusterEssentialRepo, clusterEssentialVersion string, timeout time.Duration) error {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any reason why we want to call this clusterEssentialRepo, it is still a package bundle with some config in it, calling it package bundle makes more sense

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yharish991 , Its not exactly a package bundle but the repo where cluster essential package reside
Eg:

clusterEssentialRepo := "public.ecr.aws/abcd/cluster-essentials"
clusterEssentialVersion := "v0.0.1"

cluster-essentials/pkg/client/helpers.go Outdated Show resolved Hide resolved
if err != nil {
return err
}
err = validatePreInstall(ctx, clientSet, clusterEssentialVersion)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we first do pre-install validation and then download the bundle?

cluster-essentials/pkg/client/helpers.go Outdated Show resolved Hide resolved
cluster-essentials/pkg/client/install.go Outdated Show resolved Hide resolved
if !ok {
return fmt.Errorf("pre validation check failed for package %s, package is not a part of cluster essential", packageName)
}
// Get the existing cluster essential version
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how would you handle the non-cluster-essentials installed kapp/secretgen-controller?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for non-cluster-essential , prevalidation will fail at line 65
_, ok := deployment.Annotations[packageNameAnnotation]
// If annotation is not present then return error
if !ok {
return fmt.Errorf("pre validation check failed for package %s, package is not a part of cluster essential", packageName)
}

Signed-off-by: Vandana Pathak <pathakv@sjc-ubu-eng-127.vmware.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants