Suggestion: idempotent install

[andreas-baus]

I would like to suggest a feature: "idempotent" installations. To illustrate what I mean: the company I work for is building a system that involves deploying a set of microservices to a kubernetes cluster, and we want to automate the deployment process. We don't want the deployment pipeline to need information about which services are already deployed and in which version; so it should just (re-)install every service whenever a new version of the system is to be deployed, even if not all of them have actually changed. This is accomplished by doing a 'porter install' using your mixin with the upsert flag (so it does not matter if the service is already installed or not).

However, this still has a small flaw: the mixin always installs the charts, leading to them getting a new revision number, even if nothing about the installation has changed (either the chart itself or its parameters). We would prefer if there was a way (i.e. via by an additional flag) to tell the mixin to compare the chart already installed on the cluster with the one about to be installed and not actually perform the installation if the 'new' installation is in fact identical to the current one.

I already implemented logic that accomplishes this (by essentially invoking a "helm get manifest" command to get the manifest of the installed release, then using the "helm template" command to render the manifest for the chart about to be installed, comparing the two and skipping the installation if they are identical) in a fork of your code for our internal use, but I've been wondering if you might be interested in including this in your project.

[carolynvs]

@andreas-baus Can you share your fork? I'm curious how you are handling when the chart is the same but the specified values are different (perhaps because the user repeated install and changed a parameter).

[andreas-baus]

Essentially, we retrieve the manifests of the installation already present (by invoking a “helm3 get manifest” command), then ask helm to render the templates as it would install them with the current parameters (with a “helm3 template” command”), parse the yaml output from both commands and then just do a deep compare of the resulting object trees for equality. If they come up as equal we assume there’s no relevant change and we can just skip the installation. Otherwise, the installation proceeds as usual. The code can be found at https://gitlab.com/centros.one/packaging/porter-helm3-centros, the relevant commit is https://gitlab.com/centros.one/packaging/porter-helm3-centros/-/commit/1cb0f93b980ea5a7b029b2925d87331232ebd794 ​​ ​Mit freundlichen Grüßen / Kind regards ​ Andreas Baus Senior Developer ​ DDG AG Standort Kaiserslautern Trippstadter Str. 110 67663 Kaiserslautern ​ Jetzt DDG Newsletter abonnieren! ​ T : +49 (0) 631 - 343 591 66 F : +49 (0) 631 - 343 591 69 W: www.ddg.ag ​__________________________________________________________________________________________________ ​ DDG AG | Vorstand: Alexander Fridhi | Aufsichtsrat: Nikolaus Reuter (Vorsitzender), Dr.-Ing. Sven J. Körner, Dr. Stefan Wess | Sitz der Gesellschaft: Taunusanlage 8, 60329 Frankfurt am Main | Registergericht Frankfurt am Main | Register-Nr.: HRB 119966 | Standorte: Berlin, Frankfurt a. M., Kaiserslautern, Luxemburg, Mainz, Mannheim | Tel.: 0800 866 111 1 | Mail: ***@***.*** From: Carolyn Van Slyck ***@***.***> Sent: Tuesday, September 14, 2021 10:55 PM To: MChorfa/porter-helm3 ***@***.***> Cc: Andreas Baus / DDG - THE AI ECOSYSTEM ***@***.***>; Mention ***@***.***> Subject: Re: [MChorfa/porter-helm3] Suggestion: idempotent install (#17) @andreas-baus<https://github.com/andreas-baus> Can you share your fork? I'm curious how you are handling when the chart is the same but the specified values are different (perhaps because the user repeated install and changed a parameter). — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#17 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AS4XBXWZPZSIUTQJIC3QFNDUB6ZCZANCNFSM4X2JANKQ>. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[carolynvs]

That's great! 💯 Do you do that just for install or also for upgrade too? It almost seems like it should be done for both, unless someone uses a flag to force it.

[andreas-baus]

As it is currently implemented, this is only done for the install command. That’s mainly because we adapted the mixin for our specific use case where that’s the only one we use; since we wanted some kind of “fire-and-forget” installation procedure for automatic deployment which does not have to find out whether there’s a release already present to decide whether to use ‘install’ or ‘upgrade’. But I see no reason why the same logic should not be applicable in the ‘upgrade’ case as well. ​​ ​Mit freundlichen Grüßen / Kind regards ​ Andreas Baus Senior Developer ​ DDG AG Standort Kaiserslautern Trippstadter Str. 110 67663 Kaiserslautern ​ Jetzt DDG Newsletter abonnieren! ​ T : +49 (0) 631 - 343 591 66 F : +49 (0) 631 - 343 591 69 W: www.ddg.ag ​__________________________________________________________________________________________________ ​ DDG AG | Vorstand: Alexander Fridhi | Aufsichtsrat: Nikolaus Reuter (Vorsitzender), Dr.-Ing. Sven J. Körner, Dr. Stefan Wess | Sitz der Gesellschaft: Taunusanlage 8, 60329 Frankfurt am Main | Registergericht Frankfurt am Main | Register-Nr.: HRB 119966 | Standorte: Berlin, Frankfurt a. M., Kaiserslautern, Luxemburg, Mainz, Mannheim | Tel.: 0800 866 111 1 | Mail: ***@***.*** From: Carolyn Van Slyck ***@***.***> Sent: Thursday, September 16, 2021 4:22 PM To: MChorfa/porter-helm3 ***@***.***> Cc: Andreas Baus / DDG - THE AI ECOSYSTEM ***@***.***>; Mention ***@***.***> Subject: Re: [MChorfa/porter-helm3] Suggestion: idempotent install (#17) That's great! 💯 Do you do that just for install or also for upgrade too? It almost seems like it should be done for both, unless someone uses a flag to force it. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#17 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AS4XBXWWJHRHER6AW3W5SMDUCH4QDANCNFSM4X2JANKQ>. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[carolynvs]

Oh then this new feature in the upcoming porter v1 alpha may be useful to you. I am adding additional commands to help manage an installation using GitOps (and a Porter Kubernetes Operator).

porter installation apply my-installation.yaml

The definition of what you want the installation to look like is represented in a file, and you can pass that file into porter and it is reconciled against the current state of the installation. If the installation doesn't exist, it is installed. If it does exist, porter checks whether the bundle has changed, or any parameters/credentials that you have specified to use with the installation. If anything has changed, the installation is upgraded.

Here's an example of what that will look like

schemaVersion: 1.0.0
name: demo
namespace: demo
bundleRepository: carolynvs/tabbycat-demo
bundleVersion: 0.2.1
parameters:
  logLevel: 11
parameterSets:
  - devEnv
credentialSets:
  - myCloudCreds