Cleaned enable pvc datasource #76913

j-griffith · 2019-04-22T22:53:54Z

This enables the ability to specify and existing PVC as a DataSource in
a new PVC Spec (eg "clone" and existing volume).

What type of PR is this?

/kind feature

What this PR does / why we need it:

Currently the only valid input Kind for PVC DataSource is "VolumeSnapshot", this feature adds the ability to also specify a DataSource Kind of "PersistentVolumeClaim".

This results in the ability for a user to specify intent to clone and existing PVC, it does NOT implement cloning functionality in Kubernetes; that's left to the external csi provisioner and plugins. It does however provide a mechanism to request a clone operation to the csi-provisioner and ultimately the csi plugin that the original PVC is allocated from.

Currently (as of 1.14) a user is able to enter a DataSource with any information they wish, however if it's not of the type "VolumeSnapshot" and the "VolumeSnapshotDataSource" feature gate is not enabled, the input is nilled and silently ignored. This change follows the same pattern, via the "VolumeDataSource" feature gate.

Which issue(s) this PR fixes:

Implements enhancement/KEP 642
Issue kubernetes/enhancements#989

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

This change enables a user to specify a DataSource/Kind of type "PersistentVolumeClaim" in their PVC spec.  This can then be detected by the external csi-provisioner and plugins if capable.

j-griffith · 2019-04-22T22:56:07Z

/assign @msau42 @jsafrane

j-griffith · 2019-04-22T22:56:54Z

This supersedes #76913

fejta-bot · 2019-04-22T23:06:31Z

This PR may require API review.

If so, when the changes are ready, complete the pre-review checklist and request an API review.

Status of requested reviews is tracked in the API Review project.

fejta-bot · 2019-04-23T01:45:28Z

Unknown CLA label state. Rechecking for CLA labels.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/check-cla

xing-yang · 2019-04-23T03:05:56Z

pkg/features/kube_features.go

@@ -504,6 +510,7 @@ var defaultKubernetesFeatureGates = map[utilfeature.Feature]utilfeature.FeatureS
 	NodeLease:                                   {Default: true, PreRelease: utilfeature.Beta},
 	SCTPSupport:                                 {Default: false, PreRelease: utilfeature.Alpha},
 	VolumeSnapshotDataSource:                    {Default: false, PreRelease: utilfeature.Alpha},
+	VolumePVCDataSource:                         {Default: false, PreRelease: utilfeature.Alpha},


On a second thought, I think "PVCDataSource" is better. We can't drop "Volume" from "VolumeSnapshotDataSource" because a snapshot could be for something else such as a database.

Yeah, that naming gets ugly 👍

xing-yang · 2019-04-23T21:06:03Z

My comment is addressed. LGTM

jsafrane · 2019-04-30T13:16:40Z

LGTM from my side.

There is a small change in types.go, an already existing field is extended and its documentation is updated. Let's start formal review, it can be always shortened.

/label api-review
/priority important-soon

msau42 · 2019-05-03T01:44:20Z

/assign @saad-ali

humblec · 2019-05-03T06:58:09Z

lgtm from my side . Thanks @j-griffith !

thockin

What bad things happen if we let users put whatever they want in this field and have the provisioners be responsible for checking the flag gate and eventing "not supported" ?

thockin · 2019-05-30T22:28:46Z

pkg/api/persistentvolumeclaim/util.go

@@ -52,3 +52,20 @@ func volumeSnapshotDataSourceInUse(oldPVCSpec *core.PersistentVolumeClaimSpec) b
 	}
 	return false
 }
+
+func dataSourceIsEnabled(pvcSpec *core.PersistentVolumeClaimSpec) bool {
+	if pvcSpec.DataSource != nil {


Do we really need this level of checking? I mean, is it possible that external controllers could consume this with other types that we don't know about here?

IMO no, in fact I thought it seemed redundant and had initially just removed the restrictions on the datasource field with a new feature gate and called it done. Yes, my thought was this was designed for external controllers and I'd love to be able to use it as such. During conversations I was asked to add the checks back in.

I'm probably not helping myself out here.

thockin · 2019-05-30T22:29:27Z

pkg/apis/core/types.go

-	// If the provisioner does not support VolumeSnapshot data source, volume will
+	// This field can be used to specify either:
+	// * An existing VolumeSnapshot object (snapshot.storage.k8s.io/VolumeSnapshot)
+	// * An existing PVC (""/PersistentVolumeClaim)


whats with teh quotes?

Oops, fixed.

j-griffith · 2019-05-30T22:41:31Z

What bad things happen if we let users put whatever they want in this field and have the provisioners be responsible for checking the flag gate and eventing "not supported" ?

@thockin Not much really, either it does "nothing" because there isn't another controller looking for it or acting upon it, or said entry is valid and it's acted upon (or rejected) by the interested controller.

thockin · 2019-05-30T22:50:54Z

I'm fine with this for alpha, but it seems like the final state is that the provisioner is responsible for evaluating this and deciding whether it supports the source or not (and in fact, already not all provisioners will support all sources or even support any source at all, right)?

I leave it to you and SIG (@saad-ali and @msau42 assigned here) to decide, but I will approve as-is.

/approve

k8s-ci-robot · 2019-05-30T22:51:37Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: j-griffith, saad-ali, thockin

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~pkg/api/OWNERS~~ [thockin]
~~pkg/apis/core/OWNERS~~ [thockin]
~~pkg/features/OWNERS~~ [saad-ali,thockin]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

saad-ali · 2019-05-30T22:53:33Z

What bad things happen if we let users put whatever they want in this field and have the provisioners be responsible for checking the flag gate and eventing "not supported" ?

The idea was to start off strict and loosen over time: only allow specific datasources initially, and loosen over time to eventually allow arbitrary datasources. In order to support aribitrary datasources we want to formalize the idea of "populators" independent of "provisioners". We haven't gotten that far yet, so for now we're just slightly loosening, instead of allowing anything.

k8s-ci-robot · 2019-05-30T22:54:50Z

@j-griffith: you cannot LGTM your own PR.

In response to this:

/LGTM

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

saad-ali · 2019-05-31T00:59:46Z

/lgtm

This enables the ability to specify and existing PVC as a DataSource in a new PVC Spec (eg "clone" and existing volume).

Update the unit tests to include checks for incorrect APIGroup type in PVC DataSource and change the name of the feature gate to be more clear: s/VolumeDataSource/VolumePVCDataSource/

This enables the ability to specify and existing PVC as a DataSource in a new PVC Spec (eg "clone" and existing volume).

j-griffith · 2019-05-31T12:10:09Z

@saad-ali I had to rebase; shouldn't be anything else added if you could reapply lgtm

jsafrane · 2019-05-31T12:40:04Z

/lgtm
just restoring after rebase

k8s-ci-robot requested review from david-mcmahon and lavalamp April 22, 2019 22:54

j-griffith mentioned this pull request Apr 22, 2019

Enable PVC as DataSource for PVC creation #76459

Closed

k8s-ci-robot assigned msau42 Apr 22, 2019

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Apr 23, 2019

xing-yang reviewed Apr 23, 2019

View reviewed changes

j-griffith force-pushed the cleaned_enable_pvc_datasource branch from 2199b6f to 8f2b9a9 Compare April 23, 2019 16:18

k8s-ci-robot added priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. and removed needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Apr 30, 2019

k8s-ci-robot assigned saad-ali May 3, 2019

jsafrane added the api-review Categorizes an issue or PR as actively needing an API review. label May 9, 2019

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 9, 2019

k8s-ci-robot assigned thockin May 30, 2019

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 30, 2019

thockin reviewed May 30, 2019

View reviewed changes

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 30, 2019

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 31, 2019

j-griffith added 6 commits May 31, 2019 06:06

Enable PVC as DataSource for PVC creation

123f1ba

This enables the ability to specify and existing PVC as a DataSource in a new PVC Spec (eg "clone" and existing volume).

Update unit tests and feature name

a66bb3c

Update the unit tests to include checks for incorrect APIGroup type in PVC DataSource and change the name of the feature gate to be more clear: s/VolumeDataSource/VolumePVCDataSource/

Enable PVC as DataSource for PVC creation

ae4c2a1

This enables the ability to specify and existing PVC as a DataSource in a new PVC Spec (eg "clone" and existing volume).

rework pvc datasource filter tests

54154f8

add comments to validation testcases, and use const in util.go

60d991e

fix typo in types.go comment

62a4861

j-griffith force-pushed the cleaned_enable_pvc_datasource branch from f57bcbb to 62a4861 Compare May 31, 2019 12:08

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 31, 2019

k8s-ci-robot assigned jsafrane May 31, 2019

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 31, 2019

k8s-ci-robot merged commit 82bfa66 into kubernetes:master May 31, 2019

janetkuo added sig/storage Categorizes an issue or PR as relevant to SIG Storage. and removed sig/apps Categorizes an issue or PR as relevant to SIG Apps. labels Jun 6, 2019

liggitt added this to the v1.15 milestone Jun 11, 2019

j-griffith mentioned this pull request Jul 12, 2019

UPSTREAM: <carry>: PVC DataSource support openshift/origin#23241

Closed

liggitt mentioned this pull request May 11, 2021

Split Volume Populators KEP kubernetes/enhancements#2552

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cleaned enable pvc datasource #76913

Cleaned enable pvc datasource #76913

j-griffith commented Apr 22, 2019 •

edited

Loading

j-griffith commented Apr 22, 2019 •

edited

Loading

j-griffith commented Apr 22, 2019

fejta-bot commented Apr 22, 2019

fejta-bot commented Apr 23, 2019

xing-yang Apr 23, 2019

j-griffith Apr 23, 2019

xing-yang commented Apr 23, 2019

jsafrane commented Apr 30, 2019

msau42 commented May 3, 2019

humblec commented May 3, 2019 •

edited

Loading

thockin left a comment

thockin May 30, 2019

j-griffith May 30, 2019 •

edited

Loading

thockin May 30, 2019

j-griffith May 30, 2019 •

edited

Loading

j-griffith commented May 30, 2019

thockin commented May 30, 2019

k8s-ci-robot commented May 30, 2019

saad-ali commented May 30, 2019

k8s-ci-robot commented May 30, 2019

saad-ali commented May 31, 2019

j-griffith commented May 31, 2019 •

edited

Loading

jsafrane commented May 31, 2019

Cleaned enable pvc datasource #76913

Cleaned enable pvc datasource #76913

Conversation

j-griffith commented Apr 22, 2019 • edited Loading

j-griffith commented Apr 22, 2019 • edited Loading

j-griffith commented Apr 22, 2019

fejta-bot commented Apr 22, 2019

fejta-bot commented Apr 23, 2019

xing-yang Apr 23, 2019

Choose a reason for hiding this comment

j-griffith Apr 23, 2019

Choose a reason for hiding this comment

xing-yang commented Apr 23, 2019

jsafrane commented Apr 30, 2019

msau42 commented May 3, 2019

humblec commented May 3, 2019 • edited Loading

thockin left a comment

Choose a reason for hiding this comment

thockin May 30, 2019

Choose a reason for hiding this comment

j-griffith May 30, 2019 • edited Loading

Choose a reason for hiding this comment

thockin May 30, 2019

Choose a reason for hiding this comment

j-griffith May 30, 2019 • edited Loading

Choose a reason for hiding this comment

j-griffith commented May 30, 2019

thockin commented May 30, 2019

k8s-ci-robot commented May 30, 2019

saad-ali commented May 30, 2019

k8s-ci-robot commented May 30, 2019

saad-ali commented May 31, 2019

j-griffith commented May 31, 2019 • edited Loading

jsafrane commented May 31, 2019

j-griffith commented Apr 22, 2019 •

edited

Loading

j-griffith commented Apr 22, 2019 •

edited

Loading

humblec commented May 3, 2019 •

edited

Loading

j-griffith May 30, 2019 •

edited

Loading

j-griffith May 30, 2019 •

edited

Loading

j-griffith commented May 31, 2019 •

edited

Loading