Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: graceful-eviction still works after an unexpected restart of karmada-controller-manager #3475

Merged
merged 1 commit into from
May 4, 2023

Conversation

chaunceyjiang
Copy link
Member

@chaunceyjiang chaunceyjiang commented Apr 27, 2023

Signed-off-by: chaunceyjiang chaunceyjiang@gmail.com

What type of PR is this?
/kind bug

What this PR does / why we need it:

When there are tasks in the GracefulEvictionTasks queue, graceful-eviction-controller will not work after restarting karmada-controller-manager.

Which issue(s) this PR fixes:
Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

`karmada-controller-manager`: Fixed a corner case that when there are tasks in the GracefulEvictionTasks queue, graceful-eviction-controller will not work after restarting karmada-controller-manager.

@karmada-bot karmada-bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. kind/bug Categorizes issue or PR as related to a bug. labels Apr 27, 2023
@karmada-bot karmada-bot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Apr 27, 2023
@codecov-commenter
Copy link

codecov-commenter commented Apr 27, 2023

Codecov Report

Merging #3475 (ec0b677) into master (0f84dd7) will decrease coverage by 0.18%.
The diff coverage is 0.00%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@            Coverage Diff             @@
##           master    #3475      +/-   ##
==========================================
- Coverage   53.76%   53.59%   -0.18%     
==========================================
  Files         210      210              
  Lines       19123    19192      +69     
==========================================
+ Hits        10281    10285       +4     
- Misses       8290     8355      +65     
  Partials      552      552              
Flag Coverage Δ
unittests 53.59% <0.00%> (-0.18%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...acefuleviction/crb_graceful_eviction_controller.go 0.00% <0.00%> (ø)
pkg/controllers/gracefuleviction/evictiontask.go 95.65% <ø> (-0.30%) ⬇️
...racefuleviction/rb_graceful_eviction_controller.go 0.00% <0.00%> (ø)

... and 4 files with indirect coverage changes

@chaunceyjiang chaunceyjiang marked this pull request as ready for review April 27, 2023 05:45
@karmada-bot karmada-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 27, 2023
@RainbowMango
Copy link
Member

@chaunceyjiang Please add a release-note.

/assign @Poor12

Copy link
Member

@XiShanYongYe-Chang XiShanYongYe-Chang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I think this patch is reasonable.
LGTM

@XiShanYongYe-Chang
Copy link
Member

BTY, I think we can add some comments to explain why we need to care about the create event.

@RainbowMango
Copy link
Member

BTY, I think we can add some comments to explain why we need to care about the create event.

Good point!

@XiShanYongYe-Chang
Copy link
Member

XiShanYongYe-Chang commented Apr 28, 2023

Hi @chaunceyjiang. I guess we may have missed a little bit of our analysis.
After the karmada-controller-manager restarts, the spec of the rb/crb changes, and the generation increases by 1.

It can be obtained from the analysis of:
#3455 (comment)

This triggers the karmada-scheduler to scheduler the rb/crb again and updates the status. graceful controller will watch the update event and continues to process the event.
So, there will be no bug here.

@chaunceyjiang
Copy link
Member Author

After the karmada-controller-manager restarts, the spec of the rb/crb changes,

Why does the spec change?

@chaunceyjiang
Copy link
Member Author

chaunceyjiang commented Apr 28, 2023

It can be obtained from the analysis of:
#3455 (comment)

Note: This is caused by another bug.

Normally, if the resources remain unchanged and the status of the member cluster remains unchanged, then rb will not change.

It can be obtained from the analysis of:
#3455 (comment)
After the status is aggregated into the resource template, the resourceVersion in rb.spec.resource is inconsistent with that in the actual resource template.

If this resource does not have a 'status' field, then it will not trigger a change in the resourceVersion of the resource

…mada-controller-manager

Signed-off-by: chaunceyjiang <chaunceyjiang@gmail.com>
@XiShanYongYe-Chang
Copy link
Member

thanks
/lgtm

@karmada-bot karmada-bot added the lgtm Indicates that a PR is ready to be merged. label May 4, 2023
Copy link
Member

@RainbowMango RainbowMango left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve

@karmada-bot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: RainbowMango

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@karmada-bot karmada-bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 4, 2023
@karmada-bot karmada-bot merged commit 03d1bc2 into karmada-io:master May 4, 2023
@XiShanYongYe-Chang
Copy link
Member

Hi @chaunceyjiang, maybe we need to cherry-pick this patch to the previous branch, can you help do it?

calvin0327 pushed a commit to calvin0327/karmada that referenced this pull request May 4, 2023
fix: graceful-eviction still works after an unexpected restart of karmada-controller-manager
karmada-bot added a commit that referenced this pull request May 4, 2023
…#3475-origin-release-1.5

Automated cherry pick of #3475: fix: graceful-eviction still works after an unexpected
calvin0327 pushed a commit to calvin0327/karmada that referenced this pull request May 4, 2023
fix: graceful-eviction still works after an unexpected restart of karmada-controller-manager
@chaunceyjiang chaunceyjiang deleted the graceful branch June 26, 2023 10:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. kind/bug Categorizes issue or PR as related to a bug. lgtm Indicates that a PR is ready to be merged. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants