Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert manifests + BSL api client to kubebuilder #2561

Merged
merged 34 commits into from
Jun 24, 2020

Conversation

carlisia
Copy link
Contributor

@carlisia carlisia commented May 22, 2020

  • All CRD manifests, and generation of manifests generated by kubebuilder (same but different)
  • BSL api client code generation ported from go-client to kubebuilder
  • Substituted all BSL usage to the kubebuilder api client
  • Tied in CRD/code generation with existing scripts

Closes: #2604

TODO:

  • basic CRUD tests for backup and backup storage location
  • more manual tests, esp. with restic

https://www.loom.com/share/b96ce9e9cd3846428ef4243ed9a1828a

@carlisia carlisia changed the title Convert CRDs/manifests to kubebuilder Convert manifests + BSL api client to kubebuilder May 22, 2020
@carlisia carlisia force-pushed the c-kb branch 3 times, most recently from b25c7b6 to d8b786d Compare May 22, 2020 15:11
@carlisia carlisia requested review from skriss, nrb and ashish-amarnath and removed request for skriss May 22, 2020 16:09
@carlisia carlisia self-assigned this May 22, 2020
@carlisia
Copy link
Contributor Author

Rebasing this PR is not fun @vmware-tanzu/velero-maintainers!

Copy link
Contributor

@skriss skriss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@carlisia thanks for getting this up! I did a quick first pass and added some high-level questions/comments. Still need to do more in-depth review and playing around locally too.

api/v1/backupstoragelocation_types.go Outdated Show resolved Hide resolved
hack/update-generated-crd-code.sh Outdated Show resolved Hide resolved
main.go Outdated Show resolved Hide resolved
pkg/backup/request.go Outdated Show resolved Hide resolved
pkg/cmd/cli/backup/create.go Outdated Show resolved Hide resolved
pkg/cmd/cli/restic/server.go Show resolved Hide resolved
pkg/controller/suite_test.go Outdated Show resolved Hide resolved
Makefile Outdated Show resolved Hide resolved
Copy link
Contributor

@nrb nrb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly questions right now, before I weight in on proceeding.

api/v1/backupstoragelocation_types.go Outdated Show resolved Hide resolved
api/v1/backupstoragelocation_types.go Outdated Show resolved Hide resolved
config/crd/bases/velero.io_backups.yaml Outdated Show resolved Hide resolved
config/default/kustomization.yaml Outdated Show resolved Hide resolved
config/prometheus/kustomization.yaml Outdated Show resolved Hide resolved
pkg/cmd/server/server.go Outdated Show resolved Hide resolved
@carlisia carlisia mentioned this pull request Jun 1, 2020
21 tasks
@carlisia carlisia force-pushed the c-kb branch 4 times, most recently from 5880d10 to d2ce999 Compare June 5, 2020 22:15
@carlisia
Copy link
Contributor Author

carlisia commented Jun 5, 2020

Moved the BSL api into the old package. Cleaned up makefile and the update crd code script.

Deleting the entire generated directory and running make update
works. Modifying an api and running make verify works as expected.

Pending: addressing questions and maybe a couple clean ups as per the reviews so far, but no big change expected.

@carlisia
Copy link
Contributor Author

carlisia commented Jun 8, 2020

@vmware-tanzu/velero-maintainers PTAL when you have a chance, it's ready for review.

There are 3 //todo(carlisia) to call attention to some questions I have about how to best refactor those code sections.

All other reviews were addressed, minus one question for Ashish (see above).

Carlisia added 9 commits June 19, 2020 14:37
Signed-off-by: Carlisia <carlisia@vmware.com>
This will allow for proper cache management.

Signed-off-by: Carlisia <carlisia@vmware.com>
Signed-off-by: Carlisia <carlisia@vmware.com>
Signed-off-by: Carlisia <carlisia@vmware.com>
Signed-off-by: Carlisia <carlisia@vmware.com>
Signed-off-by: Carlisia <carlisia@vmware.com>
Signed-off-by: Carlisia <carlisia@vmware.com>
Signed-off-by: Carlisia <carlisia@vmware.com>
Signed-off-by: Carlisia <carlisia@vmware.com>
Signed-off-by: Carlisia <carlisia@vmware.com>
nrb
nrb previously approved these changes Jun 22, 2020
Copy link
Contributor

@nrb nrb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your hard work in getting this done Carlisia! Definitely appreciate your patience and willingness to tackle this technical debt!

@ashish-amarnath If you're comfortable with this code, let's squash it rather than doing a plain merge.

@nrb nrb dismissed skriss’s stale review June 22, 2020 17:35

Steve's comments have since been addressed, he just hasn't come back to the PR.

@nrb nrb mentioned this pull request Jun 22, 2020
Copy link
Contributor

@alaypatel07 alaypatel07 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added some questions/nits, as the first pass. This is a huge PR, apologies, was not able to finish looking at all portions, looking to reconvene sometime tomorrow. Thanks

cmd.CheckError(err)

var locations *api.BackupStorageLocationList
locations := new(velerov1api.BackupStorageLocationList)
Copy link
Contributor

@alaypatel07 alaypatel07 Jun 23, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

quick question, I wonder if there is a specific reason for using the new keyword rather than velerov1api.BackupStorageLocationList{}?

The latter seems to be used couple lines later and it would be good to have the same pattern.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we want to create a new list object and not a slice of backupstoragelocations.
This, IMO, is correct.

go func() {
select {
case <-parentCh:
cancel()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wonder if this go func should return here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if you mean instead of cancel() otherwise there's no need to add a return, that's the default behavior.

Copy link
Contributor Author

@carlisia carlisia Jun 24, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean, it returns from the switch but then falls down to return ctx, cancel in the end. I don't think it's needed to add that here, this seems fine to me. If I misunderstood please let me know.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no need for an explicit return - if this case is hit it will cancel the child context, exit the select statement, and then exit the goroutine.

select {
case <-parentCh:
cancel()
case <-ctx.Done():
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

out of curiosity, if this case is a no-op, would it make sense for it to be removed?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is this case a no-op? The goroutine's mission is to either close the child context if the parent is closed, or exit when only the child is cancelled. The child can be cancelled without the parent being closed. Without the second case - the goroutine would leak.

pkg/controller/backup_controller.go Outdated Show resolved Hide resolved
pkg/controller/backup_controller_test.go Outdated Show resolved Hide resolved
pkg/restic/common.go Outdated Show resolved Hide resolved
Copy link
Member

@ashish-amarnath ashish-amarnath left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This LGTM!
There are a couple of double imports that needs to be addressed.
We can :shipit: after that!

Signed-off-by: Carlisia <carlisia@vmware.com>
Signed-off-by: Carlisia <carlisia@vmware.com>
@carlisia carlisia requested a review from nrb June 24, 2020 15:09
Signed-off-by: Carlisia <carlisia@vmware.com>
Copy link
Member

@ashish-amarnath ashish-amarnath left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for addressing all the comments!
🚀

@nrb nrb merged commit 4048c02 into vmware-tanzu:master Jun 24, 2020
@carlisia carlisia deleted the c-kb branch June 24, 2020 16:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add kubebuilder framework + convert BSL resource
7 participants