diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 01ef4a7f..a5ecba17 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -21,7 +21,7 @@ on: branches: ["main"] push: tags: ["*-rc*"] - branches: ["main"] + branches: ["branch-*"] jobs: generate-license: diff --git a/dev/release/README.md b/dev/release/README.md index aa2a10e3..aed0e29c 100644 --- a/dev/release/README.md +++ b/dev/release/README.md @@ -19,22 +19,51 @@ # DataFusion Python Release Process -## Update Version +Development happens on the `main` branch, and most of the time, we depend on DataFusion using GitHub dependencies +rather than using an official release from crates.io. This allows us to pick up new features and bug fixes frequently +by creating PRs to move to a later revision of the code. It also means we can incrementally make updates that are +required due to changes in DataFusion rather than having a large amount of work to do when the next official release +is available. -The version number in Cargo.toml should be increased, according to semver. +When there is a new official release of DataFusion, we update the `main` branch to point to that, update the version +number, and create a new release branch, such as `branch-0.8`. Once this branch is created, we switch the `main` branch +back to using GitHub dependencies. The release activity (such as generating the changelog) can then happen on the +release branch without blocking ongoing development in the `main` branch. -## Update CHANGELOG.md +We can cherry-pick commits from the `main` branch into `branch-0.8` as needed and then create new patch releases +from that branch. -Define release branch (e.g. `main`), base version tag (e.g. `0.6.0`) and future version tag (e.g. `0.7.0`). Commits -between the base version tag and the release branch will be used to populate the changelog content. +## Detailed Guide + +### Pre-requisites -You will need a GitHub Personal Access Token for the following steps. Follow +Releases can currently only be created by PMC members due to the permissions needed. + +You will need a GitHub Personal Access Token. Follow [these instructions](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token) to generate one if you do not already have one. +You will need a PyPI API token. Create one at https://test.pypi.org/manage/account/#api-tokens, setting the “Scope” to +“Entire account”. + +You will also need access to the [datafusion](https://test.pypi.org/project/datafusion/) project on testpypi. + +### Preparing the `main` Branch + +Before creating a new release: + +- We need to ensure that the main branch does not have any GitHub dependencies +- a PR should be created and merged to update the major version number of the project +- A new release branch should be created, such as `branch-0.8` + +### Update CHANGELOG.md + +Define release branch (e.g. `branch-0.8`), base version tag (e.g. `0.7.0`) and future version tag (e.g. `0.9.0`). Commits +between the base version tag and the release branch will be used to populate the changelog content. + ```bash # create the changelog -CHANGELOG_GITHUB_TOKEN= ./dev/release/update_change_log-datafusion-python.sh main 0.7.0 0.6.0 +CHANGELOG_GITHUB_TOKEN= ./dev/release/update_change_log-datafusion-python.sh main 0.8.0 0.7.0 # review change log / edit issues and labels if needed, rerun until you are happy with the result git commit -a -m 'Create changelog for release' ``` @@ -46,24 +75,23 @@ value of the `--cpus` argument in the `update_change_log.sh` script._ You can add `invalid` or `development-process` label to exclude items from release notes. -Send a PR to get these changes merged into `main` branch. If new commits that -could change the change log content landed in the `main` branch before you -could merge the PR, you need to rerun the changelog update script to regenerate -the changelog and update the PR accordingly. +Send a PR to get these changes merged into the release branch (e.g. `branch-0.8`). If new commits that could change the +change log content landed in the release branch before you could merge the PR, you need to rerun the changelog update +script to regenerate the changelog and update the PR accordingly. -## Preparing a Release Candidate +### Preparing a Release Candidate ### Tag the Repository ```bash -git tag 0.7.0-rc1 -git push apache 0.7.0-rc1 +git tag 0.8.0-rc1 +git push apache 0.8.0-rc1 ``` ### Create a source release ```bash -./dev/release/create_tarball 0.7.0 1 +./dev/release/create_tarball 0.8.0 1 ``` This will also create the email template to send to the mailing list. Here is an example: @@ -107,17 +135,12 @@ Create a draft email using this content, but do not send until after completing ### Publish Python Artifacts to testpypi -To securely upload your project, you’ll need a PyPI API token. Create one at -https://test.pypi.org/manage/account/#api-tokens, setting the “Scope” to “Entire account”. - -You will also need access to the [datafusion](https://test.pypi.org/project/datafusion/) project on testpypi. - -This section assumes some familiary with publishing Python packages to PyPi. For more information, refer to \ +This section assumes some familiarity with publishing Python packages to PyPi. For more information, refer to \ [this tutorial](https://packaging.python.org/en/latest/tutorials/packaging-projects/#uploading-the-distribution-archives). #### Publish Python Wheels to testpypi -Pushing an `rc` tag to main will cause a GitHub Workflow to run that will build the Python wheels. +Pushing an `rc` tag to the release branch will cause a GitHub Workflow to run that will build the Python wheels. Go to https://github.com/apache/arrow-datafusion-python/actions and look for an action named "Python Release Build" that has run against the pushed tag. @@ -195,7 +218,7 @@ Once the vote passes, we can publish the release. Create the source release tarball: ```bash -./dev/release/release-tarball.sh 0.7.0 1 +./dev/release/release-tarball.sh 0.8.0 1 ``` ### Publishing Python Artifacts @@ -211,7 +234,7 @@ twine upload --repository pypi dist-release/* ### Push the Release Tag ```bash -git checkout 0.7.0-rc1 -git tag 0.7.0 -git push apache 0.7.0 +git checkout 0.8.0-rc1 +git tag 0.8.0 +git push apache 0.8.0 ```