Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

scale down observer #174

Merged
merged 19 commits into from
Jun 18, 2024
Merged

Conversation

catpineapple
Copy link
Contributor

No description provided.

@catpineapple catpineapple changed the title scale out observer scale in observer May 28, 2024
@catpineapple catpineapple changed the title scale in observer scale down observer May 30, 2024
Copy link
Contributor

@intelligentfu intelligentfu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check

api/doris/v1/types.go Outdated Show resolved Hide resolved
Copy link
Contributor

@intelligentfu intelligentfu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check

@catpineapple
Copy link
Contributor Author

not close pvc resource


const (
PHASE_INITIALIZING Phase = "initializing"
PHASE_OPERABLE Phase = "operable"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not phase name as operable

@@ -386,6 +419,7 @@ const (
// +kubebuilder:subresource:status
// +kubebuilder:resource:shortName=dcr
// +kubebuilder:subresource:status
// +kubebuilder:printcolumn:name="ClusterPhase",type=string,JSONPath=`.status.clusterPhase.phase`
// +kubebuilder:printcolumn:name="FeStatus",type=string,JSONPath=`.status.feStatus.componentCondition.phase`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not needed.

return &svc, nil
}

func GetPods(ctx context.Context, k8sclient client.Client, targetDCR dorisv1.DorisCluster) (corev1.PodList, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GetPods is a common function, so why use fe filter in the function?

}

for _, pod := range pods.Items {
fmt.Printf("pod --- Name: %s, pod: %s \n", pod.GetName(), pod.Status.PodIP)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this log format should not appear.

}

// recycleBEResources pvc resource for be recycle
func (d *SubDefaultController) recycleBEResources(ctx context.Context, dcr *dorisv1.DorisCluster) error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is very very dangerous!!!!!! I have told you don't clear be resources.

Copy link
Contributor

@intelligentfu intelligentfu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check

return err
}
if scaleNumber != -1 {
klog.Info("controlClusterPhaseAndPreOperation scale down observer task is not completed, %d tasks are left. ", -scaleNumber-1)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

出现了负号?????

var mergeError error
for ; baseOrdinal > int(replicas); baseOrdinal-- {
pvc := resource.BuildPVC(volume, selector, dcr.Namespace, stsName, strconv.Itoa(baseOrdinal-1))
if err := k8s.DeletePVC(ctx, d.K8sclient, dcr.Namespace, pvc.Name); err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not right, first get next delete.

Copy link
Contributor

@intelligentfu intelligentfu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check

Copy link
Contributor

@intelligentfu intelligentfu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check

}

// sort fe by index and return top scaleNumber
func getTopFrontends(frontendMap map[int]*mysql.Frontend, scaleNumber int32) []*mysql.Frontend {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

findNeedDeleteFrontends

}

// deletePVCs will Loop to remove excess pvc
func (d *SubDefaultController) deletePVCs(ctx context.Context, dcr *dorisv1.DorisCluster, selector map[string]string,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where used pvcs?

Phase Phase `json:"phase,omitempty"`

// describe change status task of the doris cluster need retry
Retry RetrySign `json:"retry,omitempty"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cluster spec =每一个服务的spec,每一个pod的状态满足预期。

return errors.New(fmt.Sprintf("There is a conflict in crd operation. currently, cluster Phase is %+v ", phase.Phase))
}

if phase.Phase == v1.PHASE_OPERABLE || phase.Retry == v1.RETRY_OPERATOR_FE {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

statefulset的pod的数量和所有pod的状态。

)

// ControlClusterPhaseAndPreOperation means Pre-operation and status control on the client side
func (fc *Controller) controlClusterPhaseAndPreOperation(ctx context.Context, cluster *v1.DorisCluster) error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

判断是否可以执行 缩容

  1. statefulset的replicas 和 pod 数量。
  2. 所有pod的状态。

switch componentType {
case dorisv1.Component_FE:
return d.recycleFEResources(ctx, dcr)
case dorisv1.Component_BE:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

case fe:
xxxx
default:
return

@intelligentfu intelligentfu merged commit d772294 into apache:master Jun 18, 2024
@catpineapple catpineapple deleted the scale_out_observer branch June 24, 2024 06:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants