Renovate integration: call-out somehow in docs

[kingdonb]

Renovate is a paid SaaS product with a (slightly limited) free-use tier.

fluxcd/flux2#2293 (reply in thread) mentioned that Renovate has a specific, current Flux manager which integrates such useful features as "automatically opening a PR for HelmRelease chart version upgrades when they are available" and even upgrading Flux itself.

Since this first feature is not available anywhere in Flux, I thought we should call this out. I'm not sure of the disposition of Flux toward mentioning freemium products by companies who do not contribute directly to Flux, but I am sure it could be at the very least added to the Integrations page: https://fluxcd.io/integrations/

Given this is a commonly requested feature though, I was hoping to mention it specifically in the docs, somewhere a bit more embedded/accessible and discoverable with reference to the specific capabilities it offers. WDYT?

[kingdonb]

Ongoing discussion already:

• Document Renovate Flux integration flux2#2394

Also related:

• Expand List of "Integrations, Extensions" to accommodate ancillary utility tools #558

[dholbach]

So we call out to renovatebot under https://fluxcd.io/integrations now and link off to its own documentation on renovatebot.com.

I'm not quite sure how best to integrate this more tightly into our docs without copying and pasting the renovatebot docs. Any thoughts?

[kingdonb]

Before we advertise this, we should make sure this issue gets addressed:

• improve flux manager with custom flags renovatebot/renovate#13952

Users who add Renovate to their repos are already reporting that it removes components-extra and can't apparently be configured to retain the extra controllers for now, ideally it will read the manifest and intuit which controllers should be used.

[dholbach]

Thanks for highlighting this - I'll back it out of /integrations for now.

[kingdonb]

I may also have mischaracterized Renovate, according to their docs:

Renovate documentation¶
Renovate (often referred to as "Renovate Bot") is an Open Source tool to automate:

• Detecting dependencies in a repository (Open Source and private/closed source)
• Checking if there are newer versions which can be updated to
• Creating commits and Merge/Pull Requests to apply such changes, and show the Release Notes (if available)

If it is really open source, then there may be no qualms about including it in our docs in a first-class way. There is definitely a paid offering but (I am sure we don't) have any prohibition on mentioning projects that are related to other products with a paid offering in our docs.

I made mention on this page for the Renovate maintainers to be aware of the issue: renovatebot/renovate#13952 (comment)

[onedr0p]

If it is really open source, then there may be no qualms about including it in our docs in a first-class way.

Renovate is fully open-source, but not the code for their SaaS as far as I am aware. You can clone the renovate repo and run it locally against any github/gitlab/etc repo you own and provide a personal access token to.

I run it in a GitHub action, https://github.com/onedr0p/home-ops/blob/main/.github/workflows/schedule-renovate.yaml which uses https://github.com/renovatebot/github-action action, and at a previous job I was running it in Gitlab with a Gitlab runner, fully self-hosted.

For me and others I've introduced Renovate to, Renovate and Flux work wonderfully together, in having a GitOps workflow for opening PRs on new helm chart updates, container image updates, and just overall managing lots of other deps in a repository like pre-commit hook versions, github action versions, terraform module versions and much much more.

Renovate has almost made the image automation controller not needed 😉

[danports]

Wow, thanks for the rapid & vigorous discussion here! I wasn't expecting such a flurry of activity. 🚀 😅

I agree with @onedr0p's assessment that Renovate is fully open source. In fact, when I reviewed the Whitesource product page for Renovate again just now, I couldn't find any mention of purchasing, nor any mention of a closed source/paid version of Renovate (e.g. an enterprise version with additional features). Whitesource does have a paid "Full Solution" offering that apparently integrates several of their open source products with some additional functionality into a web UI, but I don't think that impacts our considerations here.

I think it would be helpful if the Flux docs more clearly outlined day 1/day 2 operations, the GitOps workflow, and how Renovate fits into that. Right now, the Guides and the Use Cases sections are like cookbooks with a mish-mash of various recipes, which is a little bit confusing. I'd suggest reorganizing the contents of those sections into two new sections: Installation/Configuration (i.e. day 1) and Operations (i.e. day 2). Then I'd add an Operations subsection to discuss manual and automated upgrades via flux bootstrap, the GitHub Action, Renovate, etc. I'd probably put the Prometheus monitoring guide and migrations bits in the Operations section too.

[dholbach]

Thanks @danports for your feedback.

I think it would be helpful if the Flux docs more clearly outlined day 1/day 2 operations, the GitOps workflow, and how Renovate fits into that. Right now, the Guides and the Use Cases sections are like cookbooks with a mish-mash of various recipes, which is a little bit confusing. I'd suggest reorganizing the contents of those sections into two new sections: Installation/Configuration (i.e. day 1) and Operations (i.e. day 2). Then I'd add an Operations subsection to discuss manual and automated upgrades via flux bootstrap, the GitHub Action, Renovate, etc. I'd probably put the Prometheus monitoring guide and migrations bits in the Operations section too.

I'll copy this to #717

[danports]

I've partially addressed renovatebot/renovate#13952 in renovatebot/renovate#14332 by adding support for --components when generating Flux system manifests. Would you folks be open to a Flux PR to add the options used to generate the system manifest to the manifest header as discussed here? That way we could easily grab the install options in Renovate without resorting to potentially fragile regexes to extract the options from the contents of the manifest or duplicating the options in the Renovate configuration.

[kingdonb]

I think that's a great idea after our conversation before, I think everyone will agree that Renovate should not try to parse the manifests and infer Flux's bootstrap configuration based on what is found or missing there.

Flux team, how do we feel about adding more information to the "DO NOT EDIT" section at the top of gotk-components.yaml? I think our design is that, any bootstrap (or install) option which leaves a footprint on the manifests, should leave enough metadata behind for upgraders to reproduce them on future versions. This idea can have some thought and iteration, but in principle what do we think?

I think only install should suffice as Renovate should not need to run bootstrap again, only install --export (I think the list in renovatebot/renovate#13952 (comment) looks correct at a glance.)

[dholbach]

Can this be resolved?

[kingdonb]

I think so. The missing parts are not related to the core issue, which is that "we can't really advertise renovate if it's going to remove image controllers when people use it to upgrade Flux" – it's still true that renovate will wipe out things like --network-policy=false but those options can all be worked around by customizing Flux in the prescribed way.

And now, we are calling out renovate in the docs:

https://fluxcd.io/ecosystem/#ancillary-tools

We already merged the note adding this back to ecosystem tools, after @danports added those fixes to Renovatebot. It's not searchable in our docs for some reason, (but still that is not the same issue.)