Regarding MSI builds and their maintenance

[mapkon]

Context

PR 766 by Brian

Short version

In summary - the PR adds gh action that build windows specific msi binaries during the release. However, we already build windows binaries, and adding MSI builds means:

• More maintenance
• More things that can break during a release

TL;DR

Brian has done a stellar job of implementing the ability to build and release MSI artifacts. Incidentally, we do have a build that is/was maintained @gliptak that ships to choco (a windows package system) - the corresponding config is here- @gliptak is still active on the project (judging from the contributions that he is submitting), and I hazard he will continue to be.

As discussed in the PR, MSI builds add benefits such as:

• Standard packing format that natively supports install/remove/repair
• Works interactively or unattended
• Works with many software management platforms in enterprises
• Works with Group Policy

However, the project is being maintained by people offering their free time -- the majority of whom have spent less than a month on the project -- and increasing the surface area for supported architecture increases the maintenance burden.

Suggestion

There are some middle grounds we could explore:

• Document the MSI generation build process in a readme so that anyone can reference it if they prefer (this might cause some consternation since it could be construed as omitting a major platform) -

Thoughts?

[mapkon]

@wolfeidau I would appreciate your input here!

[gliptak]

There might be a benefit to continue leaning on goreleaser only and it doesn't natively support MSI

goreleaser/goreleaser#1295

With that said, this new additional build path would likely require limited maintenance.

Implementing Choco upload directly from goreleaser also simplify

https://goreleaser.com/customization/chocolatey/

[briantist]

Hi @gliptak indeed the MSI build job I've added is unlikely to require much maintenance at all.

While it's true that goreleaser does not support MSI, I am not looking to replace goreleaser.

The workflow I've put in place requires the current build process to complete, and uses the native Windows binary built from it.

---

I am also not trying to replace the chocolately package. I haven't looked at how it's being created at the moment, but the MSI could possibly simplify it, by having the chocolately installer use the MSI.

That could be done by either embedding the MSI in the chocolately package, or having the chocolately installer download the MSI.

The advantage of this is that it would unify the installation mechanism on Windows, so whether installed via chocolately or via direct MSI there would be unity between:

• the path where it's installed (standard Program Files location)
• the ability to repair/remove it via standard mechanisms
  • add/remove programs UI
  • command line (including existing choco commands)
  • various APIs
• a single UUID representing the "product" (product code), which relates to the above as far as standard registry entries and things (we don't touch these, it all gets handled by the package management system)

---

Anyway, I'm sure we wouldn't do this now, but it would be a possibility in the future, if desired (but first we'd have to accept the MSI build at all).

[pwmcintyre]

I don't have opinions on MSI's as I don't use them — but I'm interested to understand how much value is provided by shifting the MSI creation into this repo, versus companies packaging it themselves from the binaries.

I assume (and again I don't know much about MSI's) that companies who are savvy with MSI's and distributing them within an enterprise, would be familiar with the process and better equipped to handle it.

Meanwhile, I'm unsure the maintainers of this repo (ie. think Golang and Identity) are equally equipped?

Seeking opinions / clarification.

[briantist]

@pwmcintyre that's a great question! As someone who has spent a lot of time supporting companies who use Windows (ranging from 1 to over 1000 employees), I can say that everyone who uses Windows uses MSIs, but the vast majority do not know how to create them, and do not create their own packages.

The ease of use and the support from pretty much every tool and config management software that comes from using the platform's native packaging system is unmatched.

That includes free/OSS tools (Ansible/Puppet/Chef/Chocolately/etc.) and commercial/enterprise tools (SCCM/Active Directory Group Policy/Citrix/etc.).

Assuming an enterprise can and wants to customize their own installer, they still can (and would be better off starting from an existing installer, but I don't think that's a wide enough use case to support adding it here).

---

I'm unsure the maintainers of this repo (ie. think Golang and Identity) are equally equipped?

The single XML file in the PR is what defines the package (28 lines)
The shell script in the PR is enough to build it (3 lines if you count the shebang)

The GitHub Actions workflow added in the PR is mostly stuff around downloading the existing Windows binary from the release; the single line build step to create the package calls the shell script mentioned above.

One of the reasons I used a docker image that already has all the Wix tools is specifically so that this could be built locally by other maintainers, without needing to have a Windows environment.

I can change the GitHub Actions workflow to use a Windows runner (which has the Wix framework pre-installed!), but I felt that providing a workflow that doesn't rely on Windows nor on installing the Wix framework was more compelling for maintainers.

[mapkon]

@pwmcintyre

Like you, I don't have a particular opinion on MSI builds since I hardly use them. My concern stems from maintenance aspects of it, and the overall value it provides to the community as whole, in lieu of other options. If the existing channels of releasing MSI builds are frictionless, then the need to bundle it into the main project is negated.

However, like all things, the community's voice should prevail!

[briantist]

Hi @mapkon I am definitely sensitive to maintenance concerns being a maintainer of several open source projects myself.

I understand it can be difficult to see the value if you don't have heavy exposure to the administrative side of Windows-based enterprises; I can't really overstate how much more useful a standalone MSI is than something like chocolately for central administration and governance.

That being said, let's see if I can help ease the maintenance concerns somewhat.

First, I can say that I don't intend to drop this here and disappear; if something breaks or needs updating I intend to fix it. But this is open source, and even with good intentions, situations change, so I understand that that isn't necessarily compelling.

I have structured the PR in a way that I think 1) makes it unlikely to break and 2) severely limits the blast radius and effect on the project if it does break.

For limiting breakage:

• the build process does not rely on anything specific about the project, its code, or its current build process: the only thing it depends on from the project is:
  • a Windows binary is already built and added to the release (this is handled entirely by goreleaser now, but the MSI build does not depend on that detail)
• a container image that contains the Wix tools means not having to fiddle with versions in the workflow

For limiting the blast radius of a breakage:

• the (separate) MSI build job depends on the current release job, but not the other way around
  • so if the MSI job fails, the project still has a successful release, with all of the existing release artifacts still included; this will not be an emergency to address or prevent a release
  • the workflow_dispatch trigger added means that the MSI build can be run after the fact, so when the job is repaired, it's easy to run it against an existing release without having to delete releases or tags, and it will create the MSI and upload it to the existing release, without touching any other aspect of the release

I wrote some additional stuff in this reply about why I went with the container-based build even though GHA Windows runners already include the Wix framework; I would recommend giving a local build a shot if that would help inspire confidence.

---

All that being said, if it would also be helpful for me to add more in the way of documentation to the PR, I can do that, in the form of more comments within the files, a README with instructions and troubleshooting, etc. Let me know.

---

Finally, if there's still uncertainty after the above, I might suggest a trial run? Something like:

• accept the PR
• add the MSIs to a few previous releases
• update the README to say that MSIs are being added on a strictly experimental basis
• give it until at least the next natural release (or two? if the time is very short)
• if anything comes up during this time, I hope I can demonstrate my willingness to address those issues
• after some time, re-evaluate if there are any additional concerns, and go from there?

[tzvifriedman]

Our engineers are heavy saml2aws users and, put simply, having an MSI option for managing the install across a large fleet of Windows hosts would be absolutely invaluable for the overall engineering experience.

While documentation for an MSI build would be nice, the folks managing Windows application deployments to endusers aren't always going to have the wherewithal to actually manage that build process, and the lack of an easy MSI might be a non-starter in those shops.

[lowlydba]

the folks managing Windows application deployments to endusers aren't always going to have the wherewithal to actually manage that build process, and the lack of an easy MSI might be a non-starter in those shops.

💯 agree with this point. Even with the knowledge to do so, I've never personally packaged MSIs for third party content, or seen them done in-house, even at even Fortune 500 scale companies. The burden to maintain that build process in-house could greatly vary depending on security requirements, budgets, time constraints, etc. which are unfortunate barriers for something relatively easily solved upstream.

[kaypeter87]

Having many options and ignoring a native installer for Windows is an inconvenience for shops that do not use chocolately for reasons (security, company policy etc). Having this implemented would help with adoption and improve QoL immensely.

[mapkon]

Vox populi, Vox dei. I will merge the PR and organize to cut the next release with it.

Thank you all for your thoughtful responses!

[kaypeter87]

Thank you for being active and responsive! Appreciate all your hard work! Especially @briantist for the PR.

[briantist]

Thank you for being active and responsive! Appreciate all your hard work!

@mapkon I'll second that, it's great to see new activity here and community interaction, love to see it!

Looking forward to the PR merging; might recommend running it once manually with the workflow_dispatch trigger against the last release before doing a new one.

To do that (after merging, the option won't be there before then), go to Actions > (left side) "release" > (right side) "Run workflow" and enter the tag name of the release to run it against

[mapkon]

Thank you for appreciating guys. However, I can't take all the credit. @wolfeidau has been doing a lot of hand-holding in the background given I am new to this project. So most the credit should go to him.

My aim is to merge most of the over 50 PRs and get those contributions out so that we can focus on the issues. We are not doing as much as I would like, but we are slowly getting there.

Re: MSI Release

@briantist I like your suggestion. Would it make sense to cut a release without it and make it the main focus of the next release? I ask that because I have been deferring the release to get this PR in, however, given the caveats you mentioned, it might be wiser to delay it in lieu of more testing runs.

Let me know your thoughts on that!

[briantist]

@mapkon going to use the term "release" to refer to both a specific version of the product (compiled executable), and a GitHub Release (the entries on the releases page), so I'll try to make them clear.

The PR and a release of the product itself don't depend on each other, so you should be able to do them in either order. Since the PR does not modify any of the application's code, it can be merged at any time and will not affect the contents of any binaries that get released before or after that.

And since it contains the ability to add the MSI to previous GitHub releases (by adding an additional release artifact, without changing any of the existing ones), we can update a recently released release (oof, GitHub release!) to include an MSI after the fact.

That post-release process is what I described above with the screenshot.

So in summary:

• there's no reason to delay a release for this
• no need to make it a "focus" of the upcoming or next release, since it won't affect the product itself
• but also there's no need to delay it

If it's merged now, even if it's broken, it will not interrupt your current normal release workflow, because the MSI steps run after that (since they depend on it).

And merging now gives you the opportunity to run it against the last release or three before having it run automatically on the next release (but again it's not strictly necessary to delay either).

I hope this helps clear it up and unblocks you :)

[mapkon]

Hello folks!

By way of an update, this has been merged. This should conclude this matter.

[briantist]

@mapkon that's great, thank you! Could you run the workflow against v2.36.5 as described here?

That would add an MSI to that release (and confirm the workflow is working correctly, so that I have time to fix it before the next release in case anything is wrong).

[mapkon]

Yes! I merged it - will try to run it later in the day or tomorrow.

[mapkon]

@briantist

I did run it - let me know if all is well in your camp: https://github.com/Versent/saml2aws/actions/runs/4495930896/jobs/7910086016

[briantist]

looks great! the MSI was added to the release:
https://github.com/Versent/saml2aws/releases/tag/v2.36.5

thank you!