Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reconciler/managed: avoid temporary data loss to managed on annotation update #526

Conversation

sttts
Copy link
Contributor

@sttts sttts commented Aug 23, 2023

Description of your changes

When storing critical annotations, the retry loop always dropped changes to the managed resource object, i.e.
part of the reconcile loop's effects on managed were just lost, always. This PR changes this to try one update
of the resource without reset (aka client get call), and only after that fall back into best-effort code path. In effect, this
might save a number of (expensive) reconciles.
I have:

  • Read and followed Crossplane's contribution process.
  • Run make reviewable test to ensure this PR is ready for review.

How has this code been tested

@sttts sttts requested review from a team as code owners August 23, 2023 13:12
@sttts sttts requested review from turkenh and nullable-eth August 23, 2023 13:12
@sttts sttts changed the title <!-- Thank you for helping to improve Crossplane! reconciler/managed: avoid temporary data loss to managed on annotation update Aug 23, 2023
Copy link
Member

@negz negz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, nice optimization.

@@ -172,12 +172,18 @@ func NewRetryingCriticalAnnotationUpdater(c client.Client) *RetryingCriticalAnno
// reset to their current state according to the API server.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps change this comment from "pending changes are reset" to "pending changes may be reset".

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@negz
Copy link
Member

negz commented Aug 23, 2023

Do you think it's worth "E2E" testing this? It would have to be manual at the moment, unfortunately. (i.e. Build a provider using the updated reconciler.)

@sttts
Copy link
Contributor Author

sttts commented Aug 23, 2023

Do you think it's worth "E2E" testing this?

That would be a tricky e2e. Certainly doable, but on the borderline of a test belonging into e2e.

@sttts
Copy link
Contributor Author

sttts commented Aug 23, 2023

Actually, we should only retry with a Get on conflict. Let me change that before merging. On any other error we can just keep the old object and retry.

@sttts sttts force-pushed the sttts-managed-reconcile-avoid-temporary-data-loss branch from f6a2f41 to e2684ab Compare August 24, 2023 07:21
@sttts
Copy link
Contributor Author

sttts commented Aug 24, 2023

Have extended the unit tests and only get on conflict.

Comment on lines 378 to 379
want error
wantGetCalled bool
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Typically once we want more than one thing we nest them into a want struct (similar to args).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Comment on lines 427 to 441
getCalled := tc.args.o.GetAnnotations()["getcalled"] == "true"
if getCalled != tc.wantGetCalled {
t.Errorf("\n%s\nu.UpdateCriticalAnnotations(...) calling get: -want %v, +got %v", tc.reason, tc.wantGetCalled, getCalled)
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

			// tc.want.o replaces tc.wantGetCalled
			if diff := cmp.Diff(tc.want.o, tc.args.o,); diff != "" {
				t.Errorf("\n%s\nu.UpdateCriticalAnnotations(...): -want, +got:\n%s", tc.reason, diff)
			}

Nit: Typically in this case where we're testing for a mutation of the supplied object we'd use cmp.Diff on the whole object as above. A little overkill here, but more extensible if we ever want to test more things about the object we're mutating.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, I like this pattern of adding an annotation to indicate that the fake client operated on the object. Could be worth generalizing and adopting elsewhere.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@sttts sttts force-pushed the sttts-managed-reconcile-avoid-temporary-data-loss branch from e2684ab to 50c75a1 Compare August 24, 2023 17:50
…n update

Signed-off-by: Dr. Stefan Schimanski <stefan.schimanski@upbound.io>
@sttts sttts force-pushed the sttts-managed-reconcile-avoid-temporary-data-loss branch from 50c75a1 to 5b4ebc1 Compare August 24, 2023 17:55
Copy link
Member

@negz negz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work, thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants