Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: PVC configuration and impl #4750

Merged
merged 12 commits into from
Nov 13, 2024
42 changes: 39 additions & 3 deletions infra/feast-operator/api/v1alpha1/featurestore_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,8 @@ type OfflineStorePersistence struct {
// OfflineStorePersistence configures the file-based persistence for the offline store service
type OfflineStoreFilePersistence struct {
// +kubebuilder:validation:Enum=dask;duckdb
Type string `json:"type,omitempty"`
Type string `json:"type,omitempty"`
PvcConfig *PvcConfig `json:"pvc,omitempty"`
}

var ValidOfflineStoreFilePersistenceTypes = []string{
Expand All @@ -102,8 +103,11 @@ type OnlineStorePersistence struct {
}

// OnlineStoreFilePersistence configures the file-based persistence for the offline store service
// +kubebuilder:validation:XValidation:rule="(!has(self.pvc) && has(self.path)) ? self.path.startsWith('/') : true",message="Ephemeral stores must have absolute paths."
// +kubebuilder:validation:XValidation:rule="(has(self.pvc) && has(self.path)) ? !self.path.startsWith('/') : true",message="PVC path must be a file name only, with no slashes."
type OnlineStoreFilePersistence struct {
Path string `json:"path,omitempty"`
Path string `json:"path,omitempty"`
PvcConfig *PvcConfig `json:"pvc,omitempty"`
}

// LocalRegistryConfig configures the deployed registry service
Expand All @@ -118,8 +122,40 @@ type RegistryPersistence struct {
}

// RegistryFilePersistence configures the file-based persistence for the registry service
// +kubebuilder:validation:XValidation:rule="(!has(self.pvc) && has(self.path)) ? self.path.startsWith('/') : true",message="Ephemeral stores must have absolute paths."
// +kubebuilder:validation:XValidation:rule="(has(self.pvc) && has(self.path)) ? !self.path.startsWith('/') : true",message="PVC path must be a file name only, with no slashes."
type RegistryFilePersistence struct {
Path string `json:"path,omitempty"`
Path string `json:"path,omitempty"`
PvcConfig *PvcConfig `json:"pvc,omitempty"`
}
Comment on lines 127 to +130
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OnlineStoreFilePersistence and RegistryFilePersistence are identical. Any reason we shouldn't make a single FilePersistence struct instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Registry also supports the s3_additional_kwargs options (which will go under the Registry.Persistence.File section). Will we add it to a new PR dedicated to object store settings.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, so this is in prep for future work... got it, thanks


// PvcConfig defines the settings for a persistent file store based on PVCs.
// We can refer to an existing PVC using the `Ref` field, or create a new one using the `Create` field.
// +kubebuilder:validation:XValidation:rule="[has(self.ref), has(self.create)].exists_one(c, c)",message="One selection is required between ref and create."
// +kubebuilder:validation:XValidation:rule="self.mountPath.matches('^/[^:]*$')",message="Mount path must start with '/' and must not contain ':'"
type PvcConfig struct {
// Reference to an existing field
Ref *corev1.LocalObjectReference `json:"ref,omitempty"`
// Settings for creating a new PVC
Create *PvcCreate `json:"create,omitempty"`
// MountPath within the container at which the volume should be mounted.
// Must start by "/" and cannot contain ':'.
MountPath string `json:"mountPath,omitempty"`
}

// PvcCreate defines the immutable settings to create a new PVC mounted at the given path.
// The PVC name is the same as the associated deployment name.
// +kubebuilder:validation:XValidation:rule="self == oldSelf",message="PvcCreate is immutable"
type PvcCreate struct {
// StorageClassName is the name of an existing StorageClass to which this persistent volume belongs. Empty value
// means that this volume does not belong to any StorageClass and the cluster default will be used.
StorageClassName *string `json:"storageClassName,omitempty"`
// Resources describes the storage resource requirements for a volume.
// Default requested storage size depends on the associated service:
// - 10Gi for offline store
// - 5Gi for online store
// - 5Gi for registry
Resources corev1.VolumeResourceRequirements `json:"resources,omitempty"`
}

// Registry configures the registry service. One selection is required. Local is the default setting.
Expand Down
67 changes: 64 additions & 3 deletions infra/feast-operator/api/v1alpha1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading