Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error: the object has been modified; please apply your changes to the latest version and try again #571

Closed
1 task done
dansd opened this issue Dec 8, 2022 · 1 comment · Fixed by #660
Closed
1 task done

Comments

@dansd
Copy link

dansd commented Dec 8, 2022

Describe the bug

Hello,

we are receiving the following error on our clusters with flux. It does not seem to affecting anything it all, the deployments work just fine.

reconciliation failed: Operation cannot be fulfilled on helmreleases.helm.toolkit.fluxcd.io "our-service": the object has been modified; please apply your changes to the latest version and try again

followed by many entries of

reconciliation failed: upgrade retries exhausted

A proposed fix was to suspend and resume the helm release, however this is not working in this case and the error message reappears.

I have found many issues in regard to this, but not many reactions or proposed solutions. I also created a Slack thread (https://cloud-native.slack.com/archives/CLAJ40HV3/p1666857141221079) myself. It seems like this issue is impacting multiple people, hence why I am opening this bug issue.

Threads with presumably the same issue:
https://cloud-native.slack.com/archives/CLAJ40HV3/p1670412735540889
https://cloud-native.slack.com/archives/CLAJ40HV3/p1663834852950639
https://cloud-native.slack.com/archives/CLAJ40HV3/p1654256065632129
https://cloud-native.slack.com/archives/CLAJ40HV3/p1650521514561729
https://cloud-native.slack.com/archives/CLAJ40HV3/p1643019745040000

Is it a mistake in configuration? Although flux works fine, the error messages spam makes it hard to follow the logs for any actual issues.

Steps to reproduce

We just had to install Flux and apply some helm releases. It appears on all of our 3 environments.

Expected behavior

No error message :)

Screenshots and recordings

No response

OS / Distro

EKS

Flux version

v0.37.0

Flux check

$ flux check
► checking prerequisites
✗ flux 0.35.0 <0.37.0 (new version is available, please upgrade)
✔ Kubernetes 1.23.13-eks-fb459a0 >=1.20.6-0
► checking controllers
✔ helm-controller: deployment ready
► ghcr.io/fluxcd/helm-controller:v0.27.0
✔ image-automation-controller: deployment ready
► ghcr.io/fluxcd/image-automation-controller:v0.27.0
✔ image-reflector-controller: deployment ready
► ghcr.io/fluxcd/image-reflector-controller:v0.23.0
✔ kustomize-controller: deployment ready
► ghcr.io/fluxcd/kustomize-controller:v0.31.0
✔ notification-controller: deployment ready
► ghcr.io/fluxcd/notification-controller:v0.29.0
✔ source-controller: deployment ready
► ghcr.io/fluxcd/source-controller:v0.32.1
► checking crds
✔ alerts.notification.toolkit.fluxcd.io/v1beta1
✔ buckets.source.toolkit.fluxcd.io/v1beta1
✔ gitrepositories.source.toolkit.fluxcd.io/v1beta1
✔ helmcharts.source.toolkit.fluxcd.io/v1beta1
✔ helmreleases.helm.toolkit.fluxcd.io/v2beta1
✔ helmrepositories.source.toolkit.fluxcd.io/v1beta1
✔ imagepolicies.image.toolkit.fluxcd.io/v1beta1
✔ imagerepositories.image.toolkit.fluxcd.io/v1beta1
✔ imageupdateautomations.image.toolkit.fluxcd.io/v1beta1
✔ kustomizations.kustomize.toolkit.fluxcd.io/v1beta1
✔ ocirepositories.source.toolkit.fluxcd.io/v1beta2
✔ providers.notification.toolkit.fluxcd.io/v1beta1
✔ receivers.notification.toolkit.fluxcd.io/v1beta1
✔ all checks passed

Git provider

No response

Container Registry provider

No response

Additional context

No response

Code of Conduct

  • I agree to follow this project's Code of Conduct
@stefanprodan stefanprodan transferred this issue from fluxcd/flux2 Dec 8, 2022
@dansd
Copy link
Author

dansd commented Mar 24, 2023

The issue is still current, as our logs are still spammed with these messages on all of our clusters. Is there any way I can assist in finding the bug, or provide useful information?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant