Keep main stable and PRs small

[lkingland]

It seems to me that we have two goals which are currently at odds with each other:

1. Small PRs
2. Stable main branch

It is often the case that a change necessitates additional changes for it to be fully stable. For example, adding the Created timestamp to a new Function required that we have Function migrations to support old versions, to refactor Function initialization to allow for tests to work without the old implementation of .Initialized(), and for the inclusion of the required --language argument to be logically consistent. This is a lot of change for one PR! But by breaking it into smaller pieces and merging them one at a time, the main branch is unstable (at least for a short time).

We can either ignore this issue and say that the only version of func which is expected to be stable is the last tagged release, or we can attempt to keep the main branch stable. I lean toward the latter, because if we assume main is unstable, we can not hotfix bugs and request users "update to the latest main to get the fix"... because main may be unstable.

Alternately, we can discard our goal of having PRs be small, targeted changes and instead expect them to include all changes necessary to make the catalyzing change stable. I'd prefer not to do this, however, because I agree that smaller PRs are best, for reasons discussed ad nauseam elsewhere.

Or finally, we can do a hybrid approach where we do assume main is unstable, but streamline the release process such that creating point releases is quite simple. In the event of a bugfix, we would have to of course verify that main is not unstable and then issue a point release. This is perhaps easiest, but it does present the problem that larger PRs (like the initial example) could perhaps need to be reverted in the event of a critical bug where then "stabilizing" PRs are still WIP.

The solution I would propose to these conflicting goals is to institute the concept of an "unstable" branch which includes changes from recent PRs which may not be entirely completed (read: stable). The library-level equivalent to integration testing, this also allows PRs to see if they also function along side any related PRs and PRs from development toward unrelated goals. This is often called the develop branch. PRs are merged to this branch first, and when the branch is proved stable, it is merged to main and tagged with a release version in the same commit. I believe this discussion is apropos considering our release process is currently a topic of discussion.

Of course this solution is not without its challenges. Chief among these is the tendency for the develop branch to grow without ever being merged to main (released). This should be addressed at a process level. Put simply: An issue is not completed until the fix is in main.

I would love to hear what others think about this situation, and especially if there are better ideas about how it can be solved.

[julz]

FWIW I'd personally push for avoiding a develop branch and sticking with Trunk Based Development (similar to what serving/eventing etc do). It means you have to make more effort when landing PRs to feature flag things or otherwise ensure that pre-requisites land first (and more effort on testing to ensure bugs are caught before merging so main stays stable), but I think that's a (very) good trade-off for avoiding the extra machinery, overhead and management of develop, and promotes good habits (good tests, small safe PRs, feature flags, frequent integration etc) anyway. We already have the buffer of a 6-weekly release cycle so people have a choice between main and release versions, and we can land more risky changes early in the release to give them time to bake where that makes sense.

To put it another way, I think the question assumes there are two alternatives: develop branch, or main is unstable; that's not the case, you can keep main stable and use small PRs by feature flagging, having good integration testing/automation, splitting out pre-requisites PRs etc (and releases can be particularly stable by putting more risky changes early in a release, when you want extra testing for things). I'd point to the fact that this is what we do on the rest of the project, and both main and releases have pretty rarely had breaking bugs (not that there are never bugs that make it to main/release, but I'm personally convinced we'd have more rather than less with a develop branch approach).

tldr the trunk-based/main approach isn't without downsides but has worked well elsewhere in the project and avoids the automation, process and contributor-understandability overhead of a develop branch, so that'd be my personal vote fwiw.

[lkingland]

the trunk-based/main approach isn't without downsides but has worked well elsewhere

Works for me; and barring other opinions, will be what I stick with supporting. Thanks for the input.

[salaboy]

@julz @lkingland yes please.. let's use trunk-based development + small PRs which add enough unit and integration testing to keep main stable.