-
-
Notifications
You must be signed in to change notification settings - Fork 210
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Automate release process #325
Merged
Merged
Changes from 76 commits
Commits
Show all changes
84 commits
Select commit
Hold shift + click to select a range
c645011
Draft release functions
ekcorso 9803abc
Add build verification part b
ekcorso 89bd0fa
Add pause for manual build verification
ekcorso c615013
Bring in tested release script from the other repo
ekcorso ea3c1f6
Fix references to dist folder location
ekcorso e5972cb
Create release notes
ekcorso 27d5d5e
Revise regexs + minor linting
ekcorso 9ed8ebb
Create github release draft
ekcorso f2e334c
Refactor build verification logic
ekcorso 083b827
Update tag push logic, remove exit
ekcorso 7abfb1a
Refactor release draft url check
ekcorso 728382f
Clean up style and improve logging
ekcorso 6417210
Add check for test release
ekcorso ecc3767
List dependencies for release
john-kurkowski 4621470
Check scripts
john-kurkowski e88e423
Move basic setup (input) inside main
ekcorso bac1248
Refactor releasing to pypi
ekcorso a5824fa
Merge remote-tracking branch 'upstream/master' into add-release-script
john-kurkowski 6fce0f1
Add docstring
ekcorso 0c9bd7e
Remove unused import
ekcorso 97334ad
Sort imports
ekcorso be3c801
Rename ambiguous dir variable
ekcorso 27c9048
Remove unecessary params from open()
ekcorso d74891f
Remove unnecessary try/ except blocks
ekcorso d3ecd31
Streamline logging
ekcorso 82cf56f
Fix regex and update it's documentation
ekcorso 885a46b
Fix type errors
ekcorso b17d244
Parameterize use of version number (remove global)
ekcorso 720c5af
Merge remote-tracking branch 'upstream/master' into add-release-script
john-kurkowski 1496107
Parameterize is_test
ekcorso 29c618a
Pass GITHUB_TOKEN in as parameter
ekcorso 56659e6
Use pathlib for dist paths throughout
ekcorso 3590279
Dedupe twine test/ reg upload logic
ekcorso a0b8e5c
Use requests lib instead of curl
ekcorso 3dd1fbe
Fix lint errors
ekcorso dc40d6f
Add type annotation for upload command
ekcorso 4d375f0
Allow newer syntax on older Python
john-kurkowski f060e86
Scaffold test for release script
john-kurkowski c74da79
Allow typechecking non-package code
john-kurkowski af9b799
Simplify upload to Pypi
ekcorso d089ba7
Remove comments
ekcorso 1a42543
Refactor error handling for requests
ekcorso d9d0e5a
Refactor imput validation for is_test check
ekcorso a4fb608
Merge branch 'add-release-script' into test-release-script
john-kurkowski 7f64e21
Reduce calls
john-kurkowski 25f03a4
Prefer f-strings to concatenation
john-kurkowski d6f177c
Document prerequisites
ekcorso 8535304
Cleanup main and extract env setup checks to helpers
ekcorso 1b74692
Test release script happy path
john-kurkowski 502f357
Merge branch 'add-release-script' into test-release-script
john-kurkowski e1be439
Check if version tag already exists
ekcorso a231fe1
Simplify
john-kurkowski d7f63dc
Merge branch 'add-release-script' into test-release-script
john-kurkowski 3d99f1c
Revert "Check if version tag already exists"
ekcorso 72141cd
Replace test repo url
ekcorso 32b4c9a
Expect Windows snapshot test to fail
john-kurkowski 978913b
fixup! Expect Windows snapshot test to fail
john-kurkowski aa29a97
Merge pull request #1 from ekcorso/add-release-script
ekcorso 83e706b
Revert "Check if version tag already exists"
ekcorso dbb2bd1
Replace test repo url
ekcorso 939e9ba
Update snapshot
ekcorso d6a3946
Merge pull request #2 from ekcorso/test-release-script
ekcorso ac81f79
Merge remote-tracking branch 'remote_fork/master'
ekcorso 6454199
Scaffold test for release script
john-kurkowski bbd4e6d
Allow typechecking non-package code
john-kurkowski 99ac9ef
Reduce calls
john-kurkowski 8bc06ee
Prefer f-strings to concatenation
john-kurkowski 78baee7
Test release script happy path
john-kurkowski 7bd161d
Simplify
john-kurkowski 4153e83
Expect Windows snapshot test to fail
john-kurkowski aa22335
fixup! Expect Windows snapshot test to fail
john-kurkowski ffea698
Update snapshot
ekcorso 5492d66
Temp commit for test
ekcorso a80899c
Merge remote-tracking branch 'origin/add-release-script'
ekcorso d8601ff
Revert "Temp commit for test"
ekcorso 10424f6
Update snapshot
john-kurkowski f14f8e7
Document
john-kurkowski 9c7bce9
Update pre req documentation
ekcorso 7ed31c1
Snapshot all mock calls
john-kurkowski 19e7b5b
DRY
john-kurkowski a8ae3bf
Fix missing `tar` test
john-kurkowski 7f5936c
fixup! DRY
john-kurkowski 9f6a530
Unskip tests on Windows
john-kurkowski e9d64d6
Revert "Unskip tests on Windows"
john-kurkowski File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,231 @@ | ||
""" | ||
This script automates the release process for a Python package. | ||
|
||
It will: | ||
- Add a git tag for the given version. | ||
- Remove the previous dist folder. | ||
- Create a build. | ||
- Ask the user to verify the build. | ||
- Upload the build to PyPI. | ||
- Push all git tags to the remote. | ||
- Create a draft release on GitHub using the version notes in CHANGELOG.md. | ||
|
||
Prerequisites: | ||
- This must be run from the root of the repository. | ||
- The repo must have a clean git working tree. | ||
- The user must have the GITHUB_TOKEN environment variable set to a valid GitHub personal access token. | ||
- The CHANGELOG.md file must already contain an entry for the version being released. | ||
- Install requirements with: pip install --upgrade --editable '.[release]' | ||
|
||
""" | ||
|
||
from __future__ import annotations | ||
|
||
import os | ||
import re | ||
import subprocess | ||
import sys | ||
from pathlib import Path | ||
|
||
import requests | ||
|
||
|
||
def add_git_tag_for_version(version: str) -> None: | ||
"""Add a git tag for the given version.""" | ||
subprocess.run(["git", "tag", "-a", version, "-m", version], check=True) | ||
print(f"Version {version} tag added successfully.") | ||
|
||
|
||
def remove_previous_dist() -> None: | ||
"""Check for dist folder, and if it exists, remove it.""" | ||
subprocess.run(["rm", "-rf", Path("dist")], check=True) | ||
print("Previous dist folder removed successfully.") | ||
|
||
|
||
def create_build() -> None: | ||
"""Create a build.""" | ||
subprocess.run(["python", "-m", "build"], check=True) | ||
print("Build created successfully.") | ||
|
||
|
||
def verify_build(is_test: str) -> None: | ||
"""Verify the build.""" | ||
build_files = os.listdir("dist") | ||
if len(build_files) != 2: | ||
print( | ||
"WARNING: dist folder contains incorrect number of files.", file=sys.stderr | ||
) | ||
print("Contents of dist folder:") | ||
subprocess.run(["ls", "-l", Path("dist")], check=True) | ||
print("Contents of tar files in dist folder:") | ||
for build_file in build_files: | ||
subprocess.run(["tar", "tvf", Path("dist") / build_file], check=True) | ||
confirmation = input("Does the build look correct? (y/n): ") | ||
if confirmation == "y": | ||
print("Build verified successfully.") | ||
upload_build_to_pypi(is_test) | ||
push_git_tags() | ||
else: | ||
raise Exception("Could not verify. Build was not uploaded.") | ||
|
||
|
||
def generate_github_release_notes_body(token: str, version: str) -> str: | ||
"""Generate and grab release notes URL from Github.""" | ||
response = requests.post( | ||
"https://api.github.com/repos/john-kurkowski/tldextract/releases/generate-notes", | ||
headers={ | ||
"Accept": "application/vnd.github+json", | ||
"Authorization": f"Bearer {token}", | ||
"X-GitHub-Api-Version": "2022-11-28", | ||
}, | ||
json={"tag_name": version}, | ||
) | ||
|
||
try: | ||
response.raise_for_status() | ||
except requests.exceptions.HTTPError as err: | ||
print( | ||
f"WARNING: Failed to generate release notes from Github: {err}", | ||
file=sys.stderr, | ||
) | ||
return "" | ||
return str(response.json()["body"]) | ||
|
||
|
||
def get_release_notes_url(body: str) -> str: | ||
"""Parse the release notes content to get the changelog URL.""" | ||
url_pattern = re.compile(r"\*\*Full Changelog\*\*: (.*)$") | ||
match = url_pattern.search(body) | ||
if match: | ||
return match.group(1) | ||
else: | ||
print( | ||
"WARNING: Failed to parse release notes URL from GitHub response.", | ||
file=sys.stderr, | ||
) | ||
return "" | ||
|
||
|
||
def get_changelog_release_notes(release_notes_url: str, version: str) -> str: | ||
"""Get the changelog release notes. | ||
|
||
Uses a regex starting on a heading beginning with the version number literal, and matching until the next heading. Using regex to match markup is brittle. Consider a Markdown-parsing library instead. | ||
""" | ||
with open("CHANGELOG.md") as file: | ||
changelog_text = file.read() | ||
pattern = re.compile(rf"## {re.escape(version)}[^\n]*(.*?)## ", re.DOTALL) | ||
match = pattern.search(changelog_text) | ||
if match: | ||
return str(match.group(1)).strip() | ||
else: | ||
print( | ||
f"WARNING: Failed to parse changelog release notes. Manually copy this version's notes from the CHANGELOG.md file to {release_notes_url}.", | ||
file=sys.stderr, | ||
) | ||
return "" | ||
|
||
|
||
def create_release_notes_body(token: str, version: str) -> str: | ||
"""Compile the release notes.""" | ||
github_release_body = generate_github_release_notes_body(token, version) | ||
release_notes_url = get_release_notes_url(github_release_body) | ||
changelog_notes = get_changelog_release_notes(release_notes_url, version) | ||
full_release_notes = f"{changelog_notes}\n\n**Full Changelog**: {release_notes_url}" | ||
return full_release_notes | ||
|
||
|
||
def create_github_release_draft(token: str, version: str) -> None: | ||
"""Create a release on GitHub.""" | ||
release_body = create_release_notes_body(token, version) | ||
response = requests.post( | ||
"https://api.github.com/repos/john-kurkowski/tldextract/releases", | ||
headers={ | ||
"Accept": "application/vnd.github+json", | ||
"Authorization": f"Bearer {token}", | ||
"X-GitHub-Api-Version": "2022-11-28", | ||
}, | ||
json={ | ||
"tag_name": version, | ||
"name": version, | ||
"body": release_body, | ||
"draft": True, | ||
"prerelease": False, | ||
}, | ||
) | ||
|
||
try: | ||
response.raise_for_status() | ||
except requests.exceptions.HTTPError as err: | ||
print( | ||
f"WARNING: Failed to create release on Github: {err}", | ||
file=sys.stderr, | ||
) | ||
return | ||
print(f'Release created successfully: {response.json()["html_url"]}') | ||
|
||
|
||
def upload_build_to_pypi(is_test: str) -> None: | ||
"""Upload the build to PyPI.""" | ||
repository: list[str | Path] = ( | ||
[] if is_test == "n" else ["--repository", "testpypi"] | ||
) | ||
upload_command = ["twine", "upload", *repository, Path("dist") / "*"] | ||
subprocess.run( | ||
upload_command, | ||
check=True, | ||
) | ||
|
||
|
||
def push_git_tags() -> None: | ||
"""Push all git tags to the remote.""" | ||
subprocess.run(["git", "push", "--tags", "origin", "master"], check=True) | ||
|
||
|
||
def check_for_clean_working_tree() -> None: | ||
"""Check for a clean git working tree.""" | ||
git_status = subprocess.run( | ||
["git", "status", "--porcelain"], capture_output=True, text=True | ||
) | ||
if git_status.stdout: | ||
print( | ||
"Git working tree is not clean. Please commit or stash changes.", | ||
file=sys.stderr, | ||
) | ||
sys.exit(1) | ||
|
||
|
||
def get_env_github_token() -> str: | ||
"""Check for the GITHUB_TOKEN environment variable.""" | ||
github_token = os.environ.get("GITHUB_TOKEN") | ||
if not github_token: | ||
print("GITHUB_TOKEN environment variable not set.", file=sys.stderr) | ||
sys.exit(1) | ||
return github_token | ||
|
||
|
||
def get_is_test_response() -> str: | ||
"""Ask the user if this is a test release.""" | ||
while True: | ||
is_test = input("Is this a test release? (y/n): ") | ||
if is_test in ["y", "n"]: | ||
return is_test | ||
else: | ||
print("Invalid input. Please enter 'y' or 'n.'") | ||
|
||
|
||
def main() -> None: | ||
"""Run the main program.""" | ||
check_for_clean_working_tree() | ||
github_token = get_env_github_token() | ||
is_test = get_is_test_response() | ||
version_number = input("Enter the version number: ") | ||
|
||
add_git_tag_for_version(version_number) | ||
remove_previous_dist() | ||
create_build() | ||
verify_build(is_test) | ||
create_github_release_draft(github_token, version_number) | ||
|
||
|
||
if __name__ == "__main__": | ||
main() |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you add a bullet for PyPI credentials? During the script, users will need to paste in a token from their password manager?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No problem. Done!