diff --git a/doc/internal/Overview.md b/doc/internal/README.md similarity index 86% rename from doc/internal/Overview.md rename to doc/internal/README.md index e1cb74e4ddd..4cb81d7cda2 100644 --- a/doc/internal/Overview.md +++ b/doc/internal/README.md @@ -3,3 +3,5 @@ The documents in this category document internal processes which are taken care of by the Vitess Team e.g. re-publishing the website [vitess.io](https://vitess.io) or creating a new release. We have put them here to increase transparency and make it easy for others to follow and improve processes. + +- [**Release Instructions**](./release/README.md) \ No newline at end of file diff --git a/doc/internal/.images/post-release-01.png b/doc/internal/release/.images/post-release-01.png similarity index 100% rename from doc/internal/.images/post-release-01.png rename to doc/internal/release/.images/post-release-01.png diff --git a/doc/internal/.images/release-01.png b/doc/internal/release/.images/release-01.png similarity index 100% rename from doc/internal/.images/release-01.png rename to doc/internal/release/.images/release-01.png diff --git a/doc/internal/.images/release-02.png b/doc/internal/release/.images/release-02.png similarity index 100% rename from doc/internal/.images/release-02.png rename to doc/internal/release/.images/release-02.png diff --git a/doc/internal/.images/release-03.png b/doc/internal/release/.images/release-03.png similarity index 100% rename from doc/internal/.images/release-03.png rename to doc/internal/release/.images/release-03.png diff --git a/doc/internal/.images/release-04.png b/doc/internal/release/.images/release-04.png similarity index 100% rename from doc/internal/.images/release-04.png rename to doc/internal/release/.images/release-04.png diff --git a/doc/internal/release/README.md b/doc/internal/release/README.md new file mode 100644 index 00000000000..18e668fd20c --- /dev/null +++ b/doc/internal/release/README.md @@ -0,0 +1,13 @@ +# Release Instructions + +This page describes the steps for cutting a new [open source release](https://github.com/vitessio/vitess/releases). + +### Summary + +- [Versioning](./versioning.md) +- [Release Branches](./release-branches.md) +- [Release Tags](./release-tags.md) +- [Docker Images](./docker-images.md) +- [Java Packages](./java-packages.md) +- [Release Cutover](./release-cutover.md) +- [End Of Life Process](./eol-process.md) diff --git a/doc/internal/release/docker-images.md b/doc/internal/release/docker-images.md new file mode 100644 index 00000000000..75941ca6309 --- /dev/null +++ b/doc/internal/release/docker-images.md @@ -0,0 +1,3 @@ +# Docker Images + +Docker images built automatically on DockerHub and can be found [here](https://hub.docker.com/repository/docker/vitess/lite/). diff --git a/doc/internal/release/eol-process.md b/doc/internal/release/eol-process.md new file mode 100644 index 00000000000..f1d2a343d0f --- /dev/null +++ b/doc/internal/release/eol-process.md @@ -0,0 +1,12 @@ +# End-of-Life Process + +The lifespan of a major version is one year long, after that time, the version has reached its end-of-life. +To properly deprecate a major of Vitess follow the following steps: + +- **Update the website documentation** + > - In the ['Releases' documentation](https://vitess.io/docs/releases/), the EOL version must be moved under the ['Archived Releases' section](https://vitess.io/docs/releases/#archived-releases). + > - The sidebar of the website must be changed. We need to remove the EOL version from it. To do so, we move the version folder onto the `archive` folder. +- **Delete the `Backport To: ...` label** + > - Delete the corresponding label for the EOL version, we do not want to motivate anymore backport to the EOL release branch. +- **Make proper announcement on Slack** + > - Notify the community of this deprecation. \ No newline at end of file diff --git a/doc/internal/release/java-packages.md b/doc/internal/release/java-packages.md new file mode 100644 index 00000000000..3b3d2a38472 --- /dev/null +++ b/doc/internal/release/java-packages.md @@ -0,0 +1,27 @@ +# Java Packages + +We publish binary packages for our [JDBC driver and Java client on Maven Central](https://search.maven.org/#search|ga|1|g:"io.vitess"). + +To do so, we use the http://oss.sonatype.org/ repository. +New packages must be uploaded there ("deployed") and will be automatically published ("released"). +Once they are released there, they will be automatically synchronized with Maven Central. +The synchronization takes only several minutes, but the update on http://search.maven.org may take up to two hours. + +## Access to oss.sonatype.org + +[Sign up here.](https://issues.sonatype.org/secure/Signup!default.jspa) +Then you must be added as member to our `io.vitess` namespace. +Therefore, file a JIRA ticket with Sonatype to get added ([example for a different namespace](https://issues.sonatype.org/browse/OSSRH-30604)). + +## One-time setup + +### Set up GPG + +Follow [Sonatype's GPG instructions](https://central.sonatype.org/pages/working-with-pgp-signatures.html). + +Install `gpg-agent` (needed below) e.g. on Ubuntu via: `sudo apt-get install gnupg-agent`. +for Mac you need to install 'gnupg' via 'brew install gnupg' + +### Login configuration + +Create the `settings.xml` in the `$HOME/.m2/` directory as described in their [instructions](https://central.sonatype.org/pages/apache-maven.html). diff --git a/doc/internal/release/release-branches.md b/doc/internal/release/release-branches.md new file mode 100644 index 00000000000..876ec9070d3 --- /dev/null +++ b/doc/internal/release/release-branches.md @@ -0,0 +1,17 @@ +# Release Branches + +Each major and minor releases (X.Y) should have a [release branch](https://github.com/vitessio/vitess/branches/all?query=release) named +`release-X.Y`. This branch should diverge from `main` when the release +is declared, after which point only bugfix PRs should be cherry-picked onto the branch. +All other activity on `main` will go out with a subsequent major or minor release. + +```shell +git checkout main +git pull --ff-only upstream main + +git checkout -b release-X.Y +git push upstream release-X.Y +``` + +The branches are named `release-X.Y` to distinguish them from point-in-time +tags, which are named `vX.Y.Z`. \ No newline at end of file diff --git a/doc/internal/ReleaseInstructions.md b/doc/internal/release/release-cutover.md similarity index 71% rename from doc/internal/ReleaseInstructions.md rename to doc/internal/release/release-cutover.md index 80ad0d38458..5836a7353ca 100644 --- a/doc/internal/ReleaseInstructions.md +++ b/doc/internal/release/release-cutover.md @@ -1,155 +1,53 @@ -# Release Instructions +# Release Cutover -This page describes the steps for cutting a new [open source release](https://github.com/vitessio/vitess/releases). +In this section we describe our current release process. Below is a summary of this document. -### Summary +- [**Pre-requisite for the release team**](#pre-requisites) +- [**0verview**](#overview) +- [**Pre-Release**](#pre-release) +- [**Release**](#release) +- [**Post-Release**](#post-release) +- [**How To prepare the release of Vitess**](#how-to-prepare-the-release-of-vitess) +- [**How To Release Vitess**](#how-to-release-vitess) +- [**How To Code Freeze**](#how-to-code-freeze) +- [**How To Merge During Code Freeze**](#how-to-merge-during-code-freeze) +- [**Java Packages: Deploy & Release**](#java-packages-deploy--release) -- [Versioning](#versioning) -- [Release Branches](#release-branches) -- [Release Tags](#release-tags) -- [Docker Images](#docker-images) -- [Java Packages](#java-packages) -- [Release Cutover](#release-cutover) -------- +----- -## Versioning - -Our versioning strategy is based on [VEP5](https://github.com/vitessio/enhancements/blob/main/veps/vep-5.md). - -### Major Release (vX) - -A new major release is needed when the public API changes in a -backward-incompatible way -- for example, when removing deprecated interfaces. - -Our public API includes (but is not limited to): - -* The VTGate [RPC interfaces](https://github.com/vitessio/vitess/tree/main/proto). -* The interfaces exposed by the VTGate client library in each language. - -Care must also be taken when changing the format of any data stored by a live -system, such as topology data or Vitess-internal tables (used for sequences, -distributed transactions, etc.). Although this data is considered as internal to -Vitess, if any change breaks the upgrade path for a live system (for example, -requiring that it be shut down and reinitialized from scratch), then it must be -considered as a breaking change. - -### Minor Release (vX.Y) - -A new minor release indicates that functionality has been added or changed in a -backward-compatible way. This should be the majority of normal releases. - -### Patch Release (vX.Y.Z) - -A patch release indicates that only a select set of bugfixes have been -cherry-picked onto the associated minor release. The expectation is that -upgrading by a patch release should be painless (not requiring any config -changes) and safe (isolated from active development on `main`). - -### Pre-Release Labels (vX.Y.Z-labelN) - -Pre-release versions should be labeled with a suffix like `-beta2` or `-rc1`. - -------- - -## Release Branches - -Each major and minor releases (X.Y) should have a [release branch](https://github.com/vitessio/vitess/branches/all?query=release) named -`release-X.Y`. This branch should diverge from `main` when the release -is declared, after which point only bugfix PRs should be cherry-picked onto the branch. -All other activity on `main` will go out with a subsequent major or minor release. - -```shell -git checkout main -git pull --ff-only upstream main - -git checkout -b release-X.Y -git push upstream release-X.Y -``` - -The branches are named `release-X.Y` to distinguish them from point-in-time -tags, which are named `vX.Y.Z`. - -------- - -## Release Tags - -While the release branch is a moving target, release tags mark point-in-time -snapshots of the repository. Essentially, a tag assigns a human-readable name to -a specific Git commit hash. Although it's technically possible to reassign a tag -name to a different hash, we must never do this. - -------- - -## Docker Images - -Docker images built automatically on DockerHub and can be found [here](https://hub.docker.com/repository/docker/vitess/lite/). - -------- - -## Java Packages - -We publish binary packages for our [JDBC driver and Java client on Maven Central](https://search.maven.org/#search|ga|1|g:"io.vitess"). - -To do so, we use the http://oss.sonatype.org/ repository. -New packages must be uploaded there ("deployed") and will be automatically published ("released"). -Once they are released there, they will be automatically synchronized with Maven Central. -The synchronization takes only several minutes, but the update on http://search.maven.org may take up to two hours. - -### Access to oss.sonatype.org - -[Sign up here.](https://issues.sonatype.org/secure/Signup!default.jspa) -Then you must be added as member to our `io.vitess` namespace. -Therefore, file a JIRA ticket with Sonatype to get added ([example for a different namespace](https://issues.sonatype.org/browse/OSSRH-30604)). - -### One-time setup - -#### Set up GPG - -Follow [Sonatype's GPG instructions](https://central.sonatype.org/pages/working-with-pgp-signatures.html). - -Install `gpg-agent` (needed below) e.g. on Ubuntu via: `sudo apt-get install gnupg-agent`. -for Mac you need to install 'gnupg' via 'brew install gnupg' - -#### Login configuration - -Create the `settings.xml` in the `$HOME/.m2/` directory as described in their [instructions](https://central.sonatype.org/pages/apache-maven.html). - -------- - -## Release Cutover - -In this section we describe our current release process. We begin with a list of [**pre-requisite for the release team**](#pre-requisites) and with a short [**overview**](#overview). -The release process is divided into three parts: [**Pre-Release**](#pre-release), [**Release**](#release), [**Post-Release**](#post-release), which are detailed after the overview. - -### Pre-Requisites +## Pre-Requisites This section highlights the different pre-requisites the release team has to meet before releasing. - The tool `gh` must be installed locally and ready to be used. -- You must have access to the Java release, more information in the [**Java Packages**](#java-packages) section. +- You must have access to the Java release, more information in the [**Java Packages**](./java-packages.md) section. - You must be able to create branches and have admin right on the `vitessio/vitess` and `planetscale/vitess-operator` repositories. -### Overview +----- + +## Overview -#### Schedule +### Schedule A new major version of Vitess is released every four months. For each major version there is at least one release candidate, which we release three weeks before the GA version. We usually create the RC1 during the first week of the month, and the GA version three weeks later. -#### Code Freeze +### Code Freeze Before creating RC1, there is a code freeze. Assuming the release of RC1 happens on a Tuesday, the release branch will be frozen Friday of the previous week. This allows us to test that the release branch can be released and avoid discovering unwanted events during the release day. Once the RC1 is released, there are three more weeks to backport bug fixes into the release branches. However, we also proceed to a code freeze the Friday before the GA release. (Assuming GA is on a Tuesday) Regarding patch releases, no code freeze is planned. -#### Tracking Issue for each Release +### Tracking Issue for each Release For each release, it is recommended to create an issue like [this one](https://github.com/vitessio/vitess/issues/10476) to track the current and past progress of a release. It also allows us to document what happened during a release. -### Pre-Release +----- + +## Pre-Release This step happens a few weeks before the actual release (whether it is an RC, GA or a patch release). The main goal of this step is to make sure everything is ready to be released for the release day. @@ -158,7 +56,7 @@ That includes: > - All the Pull Requests that need to be in the release must be reviewed and merged before the code freeze. > - The code freeze usually happens a few days before the release. - **Making sure the people doing the release have access to all the tools and infrastructure needed to do the release.** - > - This includes write access to the Vitess repository and to the Maven repository. + > - This includes write access to the Vitess repository and to the Maven repository. - **Preparing and cleaning the release notes summary.** > - One or more Pull Requests have to be submitted in advance to create and update the release summary. > - The summary files are located in: `./changelog/*.0/*.*.*/summary.md`. @@ -186,14 +84,16 @@ That includes: - **Update the website documentation.** > - We want to open a preparatory **draft** Pull Request to update the documentation. > - There are several pages we want to update: - > - [The releases page](https://vitess.io/docs/releases/), we must add the new release to the list with all its information and link. The links can be broken (404 error) while we are preparing for the release, this is fine. + > - [The releases page](https://vitess.io/docs/releases/), we must add the new release to the list with all its information and link. The links can be broken (404 error) while we are preparing for the release, this is fine. > - [The local install page](https://vitess.io/docs/get-started/local/), we must use the proper version increment for this guide and the proper SHA. The SHA will have to be modified once the Release Pull Request and the release is tagged is merged. > - If we are doing a GA or RC release follow the instructions below: - > - There are two scripts in the website repository in `./tools/{ga|rc}_release.sh`, use them to update the website documentation. The scripts automate: - > - For an RC, we need to create a new version in the sidebar and mark the current version as RC. + > - There are two scripts in the website repository in `./tools/{ga|rc}_release.sh`, use them to update the website documentation. The scripts automate: + > - For an RC, we need to create a new version in the sidebar and mark the current version as RC. > - For a GA, we need to mark the version we are releasing as "Stable" and the next one as "Development". -### Release +----- + +## Release On the release day, there are several things to do: @@ -224,18 +124,22 @@ On the release day, there are several things to do: > - This step is even more important for GA releases as we often include a link to _arewefastyet_ in the blog post. > - The benchmarks need to complete before announcing the blog posts or before they get cross-posted. - **Go back to dev mode on the release branch.** - > - The version constants across the codebase must be updated to `SNAPSHOT`. + > - The version constants across the codebase must be updated to `SNAPSHOT`. - **Build k8s Docker images and publish them** > - The docker image for `base`, `lite`, etc are built automatically by DockerHub. The k8s images however are dependent on these images and are required to be built manually. > - These images should be built after the `base` image has been built and available on DockerHub. > - To build and publish these images, run `./release.sh` from the directory `vitess/docker`. -### Post-Release +----- + +## Post-Release Once the release is over, we need to announce it on both Slack and Twitter. We also want to make sure the blog post was cross-posted, if applicable. We need to verify that _arewefastyet_ has finished the benchmark too. -### How to prepare the release of Vitess +----- + +## How to prepare the release of Vitess > In this example our current version is `v14.0.3` and we release the version `v15.0.0`. > Alongside Vitess' release, we also release a new version of the operator. @@ -254,8 +158,8 @@ We need to verify that _arewefastyet_ has finished the benchmark too. ``` 2. Creation of the Release Pull Request. - > This step will create the Release Pull Request that will then be reviewed ahead of the release day. - > The merge commit of that Pull Request will be used during the release day to tag the release. + > This step will create the Release Pull Request that will then be reviewed ahead of the release day. + > The merge commit of that Pull Request will be used during the release day to tag the release. 1. Run the `create_release` script using the Makefile: 1. Release Candidate: ```shell @@ -277,9 +181,9 @@ We need to verify that _arewefastyet_ has finished the benchmark too. ```shell go run ./go/tools/release-notes --from "v14.0.3" --to "HEAD" --version "v15.0.0" --summary "./changelog/15.0/15.0.0/summary.md" [--threads=[0-9.]] ``` - - > Important note: The release note generation fetches a lot of data from the GitHub API. You might reach the API request limit. - In which case you should use the `--threads=` flag and set an integer value lower than 10 (the default). + + > Important note: The release note generation fetches a lot of data from the GitHub API. You might reach the API request limit. + In which case you should use the `--threads=` flag and set an integer value lower than 10 (the default). This command will generate the release notes by looking at all the commits between the tag `v14.0.3` and the reference `HEAD`. It will also use the file located in `./changelog/15.0/15.0.0/summary.md` to prefix the release notes with a text that the maintainers wrote before the release. @@ -290,25 +194,27 @@ We need to verify that _arewefastyet_ has finished the benchmark too. 4. If we are doing an RC release it means we created a new branch from `main`. We need to update `main` with the next SNAPSHOT version. If `main` was on `15.0.0-SNAPSHOT`, we need to update it to `16.0.0-SNAPSHOT`. A simple find and replace in the IDE is sufficient, there only a handful of files that must be changed: `version.go` and several java files. -### How To Release Vitess +----- + +## How To Release Vitess This section is divided into two parts: - [Creation of the tags and release notes](#creation-of-the-tags-and-release-notes). - [Creating Release or Release Candidate on the GitHub UI](#creating-release-or-release-candidate-on-the-github-ui) -#### Creation of the tags and release notes +### Creation of the tags and release notes > This step implies that you have created a [Release Pull Request](#how-to-prepare-the-release-of-vitess) beforehand and that it has been reviewed. > The merge commit of this Release Pull Request will be used to tag the release. -> +> > In this example our current version is `v14.0.3` and we release the version `v15.0.0`. > Alongside Vitess' release, we also release a new version of the operator. > Since we are releasing a release candidate here, the new version of the operator will also be a release candidate. > In this example, the new operator version is `2.8.0`. -> +> > It is important to note that before the RC, there is a code freeze during which we create the release branch. > > The release branch in this example is `release-15.0`. -> +> > The example also assumes that `origin` is the `vitessio/vitess` remote. 1. Fetch `github.com/vitessio/vitess`'s remote. @@ -330,26 +236,26 @@ This section is divided into two parts: make BASE_BRANCH="release-15.0" BASE_REMOTE="origin" RELEASE_VERSION="15.0.0-rc1" DEV_VERSION="15.0.0-SNAPSHOT" back_to_dev_mode ``` > You will then need to follow the instructions given by the output of the back_to_dev_mode Makefile command. You will need to push the newly created branch and open a Pull Request. - + 6. Release the tag on GitHub UI as explained in the following section. -#### Creating Release or Release Candidate on the GitHub UI +### Creating Release or Release Candidate on the GitHub UI > In the below steps, we use `v8.0.0` and `v9.0.0` as an example. -##### 1. Open the releases page +#### 1. Open the releases page On Vitess' GitHub repository main page, click on Code -> [Releases](https://github.com/vitessio/vitess/releases). ![alt text](.images/release-01.png) -##### 2. Draft a new release +#### 2. Draft a new release On the Releases page, click on `Draft a new release`. ![alt text](.images/release-02.png) -##### 3. Tag a new release +#### 3. Tag a new release When drafting a new release, we are asked to choose the release's tag and branch. We format the tag this way: `v9.0.0`. We append `-rcN` to the tag name for release candidates, @@ -357,7 +263,7 @@ with `N` being the increment of the release candidate. ![alt text](.images/release-03.png) -##### 4. Add release notes and release +#### 4. Add release notes and release Copy/paste the previously built Release Notes into the description of the release. @@ -367,7 +273,9 @@ And finally, click on `Publish release`. ![alt text](.images/release-04.png) -### How To Code Freeze +----- + +## How To Code Freeze In this example we are going to do a code freeze on the `release-15.0` branch. If we are doing a release candidate, there won't be a branch yet, hence we need to create it. @@ -397,10 +305,12 @@ The script will prompt the command that will allow you to push the code freeze c Remember, you should also disable the Launchable integration from the newly created release branch. -### How To Merge During Code Freeze +----- + +## How To Merge During Code Freeze > **Warning:** It is not advised to merge a PR during code-freeze. If it is deemed absolutely necessary, then the following steps can be followed. - + The PR that needs to be merged will be failing on the `Code Freeze` CI. To merge this PR, we'll have to mark this CI action as not required. You will need administrator privileges on the vitess repository to be able to make this change. @@ -411,15 +321,17 @@ You will need administrator privileges on the vitess repository to be able to ma 5. Within this list find `Code Freeze` and click on the cross next to it to remove it from this list. 6. Save your changes on the bottom of the page. 7. Refresh the page of the PR, and you should be able to merge it. -8. After merging the PR, you need to do 2 more things - - 1. Add `Code Freeze` back as a required check. - 2. Check if the release PR has any merge conflicts. If it does, fix them and push. +8. After merging the PR, you need to do 2 more things - + 1. Add `Code Freeze` back as a required check. + 2. Check if the release PR has any merge conflicts. If it does, fix them and push. + +----- -### Java Packages: Deploy & Release +## Java Packages: Deploy & Release > **Warning:** This section's steps need to be executed only when releasing a new major version of Vitess, > or if the Java packages changed from one minor/patch version to another. -> +> > For this example, we assume we juste released `v12.0.0`. 1. Checkout to the release commit. @@ -443,7 +355,7 @@ You will need administrator privileges on the vitess repository to be able to ma 4. Deploy (upload) the Java code to the oss.sonatype.org repository: - > **Warning:** After the deployment, the Java packages will be automatically released. Once released, you cannot delete them. The only option is to upload a newer version (e.g. increment the patch level).

+ > **Warning:** After the deployment, the Java packages will be automatically released. Once released, you cannot delete them. The only option is to upload a newer version (e.g. increment the patch level).

```bash cd ./java/ diff --git a/doc/internal/release/release-tags.md b/doc/internal/release/release-tags.md new file mode 100644 index 00000000000..4136df1bbb9 --- /dev/null +++ b/doc/internal/release/release-tags.md @@ -0,0 +1,6 @@ +# Release Tags + +While the release branch is a moving target, release tags mark point-in-time +snapshots of the repository. Essentially, a tag assigns a human-readable name to +a specific Git commit hash. Although it's technically possible to reassign a tag +name to a different hash, we must never do this. \ No newline at end of file diff --git a/doc/internal/release/versioning.md b/doc/internal/release/versioning.md new file mode 100644 index 00000000000..b760e32d1b5 --- /dev/null +++ b/doc/internal/release/versioning.md @@ -0,0 +1,36 @@ +# Versioning + +Our versioning strategy is based on [VEP5](https://github.com/vitessio/enhancements/blob/main/veps/vep-5.md). + +## Major Release (vX) + +A new major release is needed when the public API changes in a +backward-incompatible way -- for example, when removing deprecated interfaces. + +Our public API includes (but is not limited to): + +* The VTGate [RPC interfaces](https://github.com/vitessio/vitess/tree/main/proto). +* The interfaces exposed by the VTGate client library in each language. + +Care must also be taken when changing the format of any data stored by a live +system, such as topology data or Vitess-internal tables (used for sequences, +distributed transactions, etc.). Although this data is considered as internal to +Vitess, if any change breaks the upgrade path for a live system (for example, +requiring that it be shut down and reinitialized from scratch), then it must be +considered as a breaking change. + +## Minor Release (vX.Y) + +A new minor release indicates that functionality has been added or changed in a +backward-compatible way. This should be the majority of normal releases. + +## Patch Release (vX.Y.Z) + +A patch release indicates that only a select set of bugfixes have been +cherry-picked onto the associated minor release. The expectation is that +upgrading by a patch release should be painless (not requiring any config +changes) and safe (isolated from active development on `main`). + +## Pre-Release Labels (vX.Y.Z-labelN) + +Pre-release versions should be labeled with a suffix like `-beta2` or `-rc1`. \ No newline at end of file