Update release documentation for stable branch releases

[jodh-intel]

We need docs explaining:

• - what stable branches are.
• - how to create them.
• - how to populate them.
• -how to test them.

That will require updates to ateast the following, but ideally a new document that is references the below (and is itself referenced by them):

• - https://github.com/kata-containers/documentation/blob/master/Releases.md
• - https://github.com/kata-containers/documentation/blob/master/Release-Checklist.md

[jodh-intel]

Ah - and we also need to ensure that the checklist file mentions updating the VERSION file both in the stable branch and in master to ensure that:

• the stable branch has the expected semver version
• the master branch version is newer than the newest stable branch
  (without this, it will be very hard to identify what release a user is running :)

/cc @egernst, @jcvenegas, @grahamwhaley, @bergwolf.

[jcvenegas]

@jodh-intel we also need update our documents on how to install packages.

We now have multiples repos for each distro , supporting different architectures and different branches.

See https://build.opensuse.org/project/subprojects/home:katacontainers

https://build.opensuse.org/project/show/home:katacontainers:releases:${architecture}:${branch}

We will have a repository per branch : stable-1.1, stable-1.2, master

For users that wants the latest and greatest all the time they should follow the master repository, else ping to a stable branch repository and upgrade to a new repository when they decide.

[egernst]

closing #231 as duplicate of this, since @jodh-intel has gone much further in this issue already!

[egernst]

@jodh-intel:

the master branch version is newer than the newest stable branch
(without this, it will be very hard to identify what release a user is running :)

This isn't the case right now but I've been thinking on this today. Do you think after we make a MINOR release and create a new stable branch for that release that we should update MASTER to be .rc0?
For example, today our stable is at 1.1.1 and 1.2.1. I would consider updating master's version to be 1.3.rc0. We could then release master on a weekly cadence with the stable branches, only name it 1.3.rcX. Or, suggestions here?

[marcov]

In #238 its stated that we will ensure compatibility among minor releases.

Is there already any plan of how this is actually done (e.g. new tests suite, automated / manual QA, etc...)?

[jodh-intel]

This isn't the case right now

We might be talking at crossed purposes? Looking at runtime for example:

$ for branch in stable-1.1 stable-1.2 master; do git checkout  $branch >/dev/null 2>&1 && git rev-parse --abbrev-ref HEAD && cat ./VERSION
stable-1.1
1.1.1
stable-1.2
1.2.1
master
1.2.0

Note that master is at 1.2.0, which is "older" than the version in stable-1.2 (version 1.2.1).

So once we create a stable release, I think we'll have to update ./VERSION in master to atleast something like 1.2.2-rc0 to avoid any confusion between the branches. But that could also accommodate 1.2.2-rc0-weekly-2018-09-03 or something.

[jodh-intel]

Hi @marcov - good question! This is referred to in kata-containers/ci#10 but I agree that we need to be explicit about this.

Any thoughts @egernst, @chavafg, @jcvenegas?

[jcvenegas]

1.2.2-rc0 sounds good, about tests we should have and test to install the latest kata release in a branch and test changes of a component against the last packaged version.

Lets say
PR101 for runtime in branch stable-1.1
Install last packages for branch stable-1.1 check the test work as expected.

[chavafg]

We currently have the corresponding stable branches on the tests repository. These branches have fewer tests than master as some of the features were not supported on stable-1.1 or stable-1.2.
When there is a PR opened to a stable branch, we use the corresponding stable branch on tests to ensure that we are testing the right features.
We still need to enhance our testing for the created packages on OBS and for using those packages as part of our CI (we are currently building everything from sources).

[marcov]

@jcvenegas @chavafg: I see. My concern is whether we need to test against all the released versions for a given major N in order to ensure compatibility. Doing a MINOR x PATCH x <component> can generate quite a lot of tests.