Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: refactor 1.0 parameter api #8441

Open
wants to merge 52 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
5ce7472
chore: add parametersdefinition
sophon-zt Oct 22, 2024
a8aa895
chore: add ParameterDrivenConfigRender
sophon-zt Oct 22, 2024
a3c4d28
chore: add ComponentParameter
sophon-zt Oct 22, 2024
fec6c0f
chore: add Parameter
sophon-zt Oct 22, 2024
7e7bb15
add controller
sophon-zt Oct 22, 2024
8f7a823
update operator for parameters
sophon-zt Oct 23, 2024
34842d1
chore: component parameter reconcile
sophon-zt Oct 25, 2024
7edae77
update cluster controller
sophon-zt Oct 29, 2024
c631e29
chore: refine reloadAction transformer
sophon-zt Oct 29, 2024
358f6b6
chore: refactor config controller
sophon-zt Oct 31, 2024
89d67ce
chore: removed Rendere on pod
sophon-zt Nov 1, 2024
27bc828
chore: remove Configuration
sophon-zt Nov 1, 2024
8f9a7fc
fix ut
sophon-zt Nov 1, 2024
eb5e660
chore: update crd doc
sophon-zt Nov 5, 2024
5948a1b
fix
sophon-zt Nov 6, 2024
fa5aea2
fix lint
sophon-zt Nov 6, 2024
ae834f5
fix npe
sophon-zt Nov 6, 2024
6e7bde5
chore: refine code
sophon-zt Nov 7, 2024
b263edc
chore: remove useless code
sophon-zt Nov 7, 2024
c0a7a1e
chore: update api-reference doc
sophon-zt Nov 7, 2024
a8a6d0a
fix lint fail
sophon-zt Nov 7, 2024
6acd520
chore: update config-manager
sophon-zt Nov 7, 2024
f7e077f
fix reconfigure reconcile
sophon-zt Nov 7, 2024
a083b9a
suppport reconfiure
sophon-zt Nov 11, 2024
adc3469
fix bug
sophon-zt Nov 11, 2024
bead517
chore: adjust reconfigure ops
sophon-zt Nov 11, 2024
58575cc
update crd
sophon-zt Nov 12, 2024
b5a7709
fix ut
sophon-zt Nov 13, 2024
d5e6feb
chore: remove ComponentTemplateSpec
sophon-zt Nov 13, 2024
3b6fa81
chore: parameter-related reconcile is migrated from apps controller t…
sophon-zt Nov 13, 2024
74a2956
chore: refine
sophon-zt Nov 13, 2024
022df45
chore: refine ut
sophon-zt Nov 14, 2024
0a32383
fix ut
sophon-zt Nov 15, 2024
fa5e3d6
refine tls rerender
sophon-zt Nov 15, 2024
a29766a
chore: add ut for parameters
sophon-zt Nov 15, 2024
29e535b
ut
sophon-zt Nov 15, 2024
60b1cbd
ut
sophon-zt Nov 15, 2024
5a12c2f
fix lint
sophon-zt Nov 15, 2024
353a5fd
chore: rebase main
sophon-zt Nov 15, 2024
9e0446b
chore: update ut
sophon-zt Nov 18, 2024
6064ff8
chore: fix lint fails
sophon-zt Nov 18, 2024
5c441af
fix ut fail
sophon-zt Nov 19, 2024
34118fe
chore: rename functions
sophon-zt Nov 19, 2024
a5d2001
rename functions and args
sophon-zt Nov 19, 2024
2db9e9f
chore: Adjusted the code according to ai suggestion
sophon-zt Nov 20, 2024
952191f
chore: removed parameter-related depend from apps
sophon-zt Nov 20, 2024
97addde
split template render into separate package
sophon-zt Nov 20, 2024
d9d5027
add ut
sophon-zt Nov 20, 2024
2f8e80a
add ut for template_render
sophon-zt Nov 20, 2024
eaa4ea0
refine render package
sophon-zt Nov 21, 2024
949fa18
rebase main
sophon-zt Nov 21, 2024
58a161a
fix lint
sophon-zt Nov 21, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions .golangci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,20 @@ run:

timeout: 30m

skip-files:
# skip-files:
# - "^zz_generated.*"

issues:
exclude-files:
- "^zz_generated.*"

# build-tags:
# - containers_image_openpgp

output:
# colored-line-number|line-number|json|tab|checkstyle|code-climate, default is "colored-line-number"
format: colored-line-number
formats:
format: colored-line-number

# print lines of code with issue, default is true
print-issued-lines: true
Expand Down
34 changes: 34 additions & 0 deletions PROJECT
Original file line number Diff line number Diff line change
Expand Up @@ -316,4 +316,38 @@ resources:
kind: SidecarDefinition
path: github.com/apecloud/kubeblocks/apis/apps/v1
version: v1
- api:
crdVersion: v1
controller: true
domain: kubeblocks.io
group: parameters
kind: ParametersDefinition
path: github.com/apecloud/kubeblocks/apis/parameters/v1alpha1
version: v1alpha1
- api:
crdVersion: v1
controller: true
domain: kubeblocks.io
group: parameters
kind: ParameterDrivenConfigRender
path: github.com/apecloud/kubeblocks/apis/parameters/v1alpha1
version: v1alpha1
- api:
crdVersion: v1
namespaced: true
controller: true
domain: kubeblocks.io
group: parameters
kind: ComponentParameter
path: github.com/apecloud/kubeblocks/apis/parameters/v1alpha1
version: v1alpha1
- api:
crdVersion: v1
namespaced: true
controller: true
domain: kubeblocks.io
group: parameters
kind: Parameter
path: github.com/apecloud/kubeblocks/apis/parameters/v1alpha1
version: v1alpha1
version: "3"
5 changes: 5 additions & 0 deletions apis/apps/v1/cluster_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -499,6 +499,11 @@ type ClusterComponentSpec struct {
//
// +optional
Stop *bool `json:"stop,omitempty"`

// Specifies the initialization parameters.
//
// +optional
InitParameters ComponentParameters `json:"initParameters,omitempty"`
Copy link
Contributor

@leon-inf leon-inf Nov 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be added into spec.components.configs, and be used as init values to render config templates.

}

type ClusterComponentService struct {
Expand Down
7 changes: 7 additions & 0 deletions apis/apps/v1/component_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -289,6 +289,11 @@ type ComponentSpec struct {
//
// +optional
Sidecars []Sidecar `json:"sidecars,omitempty"`

// Specifies the initialization parameters.
//
// +optional
InitParameters ComponentParameters `json:"initParameters,omitempty"`
}

// ComponentStatus represents the observed state of a Component within the Cluster.
Expand Down Expand Up @@ -378,3 +383,5 @@ const (
// FailedComponentPhase indicates that there are some pods of the component not in a 'Running' state.
FailedComponentPhase ComponentPhase = "Failed"
)

type ComponentParameters map[string]*string
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// Better with {name:, value:}, perhaps a parameter will come from a secret or config map?

99 changes: 1 addition & 98 deletions apis/apps/v1/componentdefinition_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -318,7 +318,7 @@ type ComponentDefinitionSpec struct {
// +listType=map
// +listMapKey=name
// +optional
Configs []ComponentConfigSpec `json:"configs,omitempty"`
Configs []ComponentTemplateSpec `json:"configs,omitempty"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ComponentConfigSpec could be deleted from this package?


// Defines the types of logs generated by instances of the Component and their corresponding file paths.
// These logs can be collected for further analysis and monitoring.
Expand Down Expand Up @@ -1031,103 +1031,6 @@ type ComponentTemplateSpec struct {
DefaultMode *int32 `json:"defaultMode,omitempty" protobuf:"varint,3,opt,name=defaultMode"`
}

type ComponentConfigSpec struct {
ComponentTemplateSpec `json:",inline"`

// Specifies the configuration files within the ConfigMap that support dynamic updates.
//
// A configuration template (provided in the form of a ConfigMap) may contain templates for multiple
// configuration files.
// Each configuration file corresponds to a key in the ConfigMap.
// Some of these configuration files may support dynamic modification and reloading without requiring
// a pod restart.
//
// If empty or omitted, all configuration files in the ConfigMap are assumed to support dynamic updates,
// and ConfigConstraint applies to all keys.
//
// +listType=set
// +optional
Keys []string `json:"keys,omitempty"`

// Specifies the secondary rendered config spec for pod-specific customization.
//
// The template is rendered inside the pod (by the "config-manager" sidecar container) and merged with the main
// template's render result to generate the final configuration file.
//
// This field is intended to handle scenarios where different pods within the same Component have
// varying configurations. It allows for pod-specific customization of the configuration.
//
// Note: This field will be deprecated in future versions, and the functionality will be moved to
// `cluster.spec.componentSpecs[*].instances[*]`.
//
// +kubebuilder:deprecatedversion:warning="This field has been deprecated since 0.9.0 and will be removed in 0.10.0"
// +optional
LegacyRenderedConfigSpec *LegacyRenderedTemplateSpec `json:"legacyRenderedConfigSpec,omitempty"`

// Specifies the name of the referenced configuration constraints object.
//
// +kubebuilder:validation:MaxLength=63
// +kubebuilder:validation:Pattern:=`^[a-z0-9]([a-z0-9\.\-]*[a-z0-9])?$`
// +optional
ConfigConstraintRef string `json:"constraintRef,omitempty"`

// Specifies the containers to inject the ConfigMap parameters as environment variables.
//
// This is useful when application images accept parameters through environment variables and
// generate the final configuration file in the startup script based on these variables.
//
// This field allows users to specify a list of container names, and KubeBlocks will inject the environment
// variables converted from the ConfigMap into these designated containers. This provides a flexible way to
// pass the configuration items from the ConfigMap to the container without modifying the image.
//
// Deprecated: `asEnvFrom` has been deprecated since 0.9.0 and will be removed in 0.10.0.
// Use `injectEnvTo` instead.
//
// +kubebuilder:deprecatedversion:warning="This field has been deprecated since 0.9.0 and will be removed in 0.10.0"
// +listType=set
// +optional
AsEnvFrom []string `json:"asEnvFrom,omitempty"`

// Specifies the containers to inject the ConfigMap parameters as environment variables.
//
// This is useful when application images accept parameters through environment variables and
// generate the final configuration file in the startup script based on these variables.
//
// This field allows users to specify a list of container names, and KubeBlocks will inject the environment
// variables converted from the ConfigMap into these designated containers. This provides a flexible way to
// pass the configuration items from the ConfigMap to the container without modifying the image.
//
//
// +listType=set
// +optional
InjectEnvTo []string `json:"injectEnvTo,omitempty"`

// Specifies whether the configuration needs to be re-rendered after v-scale or h-scale operations to reflect changes.
//
// In some scenarios, the configuration may need to be updated to reflect the changes in resource allocation
// or cluster topology. Examples:
//
// - Redis: adjust maxmemory after v-scale operation.
// - MySQL: increase max connections after v-scale operation.
// - Zookeeper: update zoo.cfg with new node addresses after h-scale operation.
//
// +listType=set
// +optional
ReRenderResourceTypes []RerenderResourceType `json:"reRenderResourceTypes,omitempty"`

// Whether to store the final rendered parameters as a secret.
//
// +optional
AsSecret *bool `json:"asSecret,omitempty"`
}

// LegacyRenderedTemplateSpec describes the configuration extension for the lazy rendered template.
// Deprecated: LegacyRenderedTemplateSpec has been deprecated since 0.9.0 and will be removed in 0.10.0
type LegacyRenderedTemplateSpec struct {
// Extends the configuration template.
ConfigTemplateExtension `json:",inline"`
}

type ConfigTemplateExtension struct {
// Specifies the name of the referenced configuration template ConfigMap object.
//
Expand Down
126 changes: 63 additions & 63 deletions apis/apps/v1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 3 additions & 4 deletions apis/operations/v1alpha1/opsrequest_conditions.go
Original file line number Diff line number Diff line change
Expand Up @@ -298,15 +298,14 @@ func NewReconfigureCondition(ops *OpsRequest) *metav1.Condition {
}

// NewReconfigureRunningCondition creates a condition that the OpsRequest reconfigure workflow
func NewReconfigureRunningCondition(ops *OpsRequest, conditionType string, configSpecName string, info ...string) *metav1.Condition {
func NewReconfigureRunningCondition(ops *OpsRequest, conditionType string, info ...string) *metav1.Condition {
status := metav1.ConditionTrue
if conditionType == appsv1alpha1.ReasonReconfigureFailed {
status = metav1.ConditionFalse
}
message := fmt.Sprintf("Reconfiguring in Cluster: %s, Component: %s, ConfigSpec: %s",
message := fmt.Sprintf("Reconfiguring in Cluster: %s, Component: %s",
ops.Spec.GetClusterName(),
getComponentName(ops.Spec),
configSpecName)
getComponentName(ops.Spec))
if len(info) > 0 {
message = message + ", info: " + info[0]
}
Expand Down
Loading
Loading