ci: Automated Y-Stream Releases

[courtneypacheco]

This dev-doc describes a design we can utilize to automate y-stream release processes for each library within the instructlab GitHub org.

Z-stream releases can use the base logic defined in this dev-doc, but will require additional logic to handle backporting, manual rebasing, etc.

[kami619]

@courtneypacheco Would this automated-release-config.yml file be per y stream release ? Is there a way we could add a checkpoint to verify in the beginning of the workflow to make sure this file exists and if it does, make sure it exists with the necessary details and format ? We could skip the secondary validation, if the file doesn't exist, which seems to be a possibility based on your workflow diagram.

[kami619]

Also would it be a good idea to version this file within the y stream release, if things change on us ? In other words, do we want this file to be immutable once the release process kicks off.

[courtneypacheco]

For now, per y-stream release.

I definitely do plan on adding a check to see if the file exists. :) If it the file doesn't exist, then the code will resort to built-in default values where applicable.

Since dependency capping is the only "configurable" component right now, the default value for the configurable list of components would be an empty list.

[kami619]

Thank you.

[kami619]

Would we want to make this also to be available to run outside of this cron schedule for any reason, by exposing the inputs for the workflow ?

[courtneypacheco]

Yes, as mentioned near the bottom of the document, there is an option to add your own trigger conditions, like workflow_dispatch. When using workflow_dispatch, users can manually kick off a release process.

The trigger conditions is defined at the Git workflow level, but not at the create-automated-release level.

[kami619]

@courtneypacheco Thank you for the PR, I think overall intent is on point and the workflow diagram really lays it out to understand it better. Just few comments, but I get those are nit picks.

[ktdreyer]

Looks good to me.

This is not a blocker, just a thought. enabled: true looks superfluous to me - like if a developer wants to disable dependency_caps, they could specify an empty dict, or delete the dependency_caps section altogether. Or if you want to keep it that feature, maybe we could default enabled: true so users don't have to copy that line around to all repos.

Either way, great feature.

[nathan-weinberg]

I feel the commas and "and" aren't really necessary in a numbered list

[nathan-weinberg]

Suggested change

Y-stream (minor) releases have historically been handled differently from Z-stream releases. Z-stream releases oftentimes involve backports for bugfixes and may require manual code rebasing to get those backports merged into the appropriate existing release branch. Therefore, we can think of Y-stream release logic as the "basis" for Z-stream release logic, which takes the Y-stream logic and builds upon it to account for backporting and other desirable actions.

Y-stream (minor) releases have historically been handled differently from Z-stream releases. Z-stream releases often times involve backports for bugfixes and may require manual code rebasing to get those backports merged into the appropriate existing release branch. Therefore, we can think of Y-stream release logic as the "basis" for Z-stream release logic, which takes the Y-stream logic and builds upon it to account for backporting and other desirable actions.

[nathan-weinberg]

Suggested change

	As mentioned above, there are configurable components within this release process automation. The diagram in the next section references two configurable components:
	As mentioned above, there are configurable components within this release process automation. The diagram in the next section references two configurable components

[nathan-weinberg]

Suggestion here, lmk what you think

Suggested change

	packages:
	instructlab-sdg: "+0.1.0"
	instructlab-eval: "+0.2.0"
	instructlab-training: "+1.0.0"
	packages:
	- name: instructlab-sdg
	greater_than: 0.1.0
	- name: instructlab-eval
	greater_than: 0.2.0
	- name: instructlab-training
	greater_than: 1.0.0

You could have similar fields equals or less_than when applicable

[nathan-weinberg]

Suggested change

	# Allow manual dispatch, too
	# Allow manual dispatch too

[booxter]

I have my reservations about automation of capping. We should just stop capping.

[booxter]

Where is this requirement to cap coming from? Aren't we asked to not cap?

Only apply "caps" to dependencies (using <) when that dependency has established a pattern of producing new releases with breaking changes.

Without this, steps 3-5 are no longer necessary.

[courtneypacheco]

We have been capping certain dependencies in the core repo since October 2024: https://github.com/instructlab/instructlab/pulls?q=is%3Apr+%22deps%3A+cap%22+is%3Aclosed+

To elaborate, we have been capping our own InstructLab library dependencies to ensure that if we create a new release from the core repo, that new release won't automatically consume potential breaking changes from one of our own libraries.

If we don't cap certain dependencies, then we'll need to address those breaking changes in one or more pull requests and follow up by publishing a Z-stream release so that end users can consume the fixes. So I think for end users in particular, it can be extremely frustrating to pull the latest InstructLab release from 1-7 days ago, only to find out that the InstructLab release doesn't work because it's pulling an incompatible package.

[courtneypacheco]

Also, to be clear, the "automatic capping" feature is not required to be used by anybody. Maintainers can ignore the feature if desired. And since the automation described here will create a pull request that someone has to manually approve and review, the dependency capping changes will not be merged automatically. 😃

[booxter]

See above. Capping is not something we should automate. It's an exceptional situation, not a matter of course.