-
Notifications
You must be signed in to change notification settings - Fork 325
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge branch 'main' into static/dataset-event
- Loading branch information
Showing
18 changed files
with
208 additions
and
58 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -39,3 +39,6 @@ metadata.json | |
|
|
||
| # Dependent Helm charts | ||
| chart/charts/ | ||
|
|
||
| # Dev | ||
| changes.txt | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,29 +1,23 @@ | ||
| ### Using `get_changes` | ||
|
|
||
| The `get_changes.sh` script uses a fork of saadmk11/changelog-ci to get all | ||
| merged changes between two specified releases. To get all changes since the latest | ||
| release, set `END_RELEASE_VERSION` to the planned next release. | ||
|
|
||
| The changes will appear at the top of CHANGELOG.md. | ||
| The changes will appear in ../CHANGELOG.md. | ||
|
|
||
| #### Requirements | ||
|
|
||
| Python 3.10 or newer is required. | ||
| See the requirements.txt file for required dependencies. | ||
|
|
||
| The script also requires that the following environment variables be set: | ||
|
|
||
| `END_RELEASE_VERSION`\ | ||
| `START_RELEASE_VERSION`\ | ||
| `INPUT_GITHUB_TOKEN` | ||
| Install the required dependencies in requirements.txt. | ||
|
|
||
| For example: `export END_RELEASE_VERSION=0.21.0`. | ||
|
|
||
| Use the planned next release for the end release version. | ||
| The script also requires a GitHub token. | ||
|
|
||
| For instructions on creating a GitHub personal access token to use the GitHub API, | ||
| see: https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token. | ||
|
|
||
| #### Running the script | ||
|
|
||
| Make the script executable (the first time), then run it with `./get_changes.sh`. | ||
| Run the script from the root directory: | ||
|
|
||
| ```sh | ||
| python3 dev/get_changes.py --github_token token --previous 0.42.0 --current 0.43.0 --path /local/path/to/CHANGELOG.md | ||
| ``` | ||
|
|
||
| If you get a `command not found` or similar error, make sure you have made the | ||
| script an executable (e.g., `chmod u+x ./dev/get_changes.py`). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,135 @@ | ||
| #!/bin/bash | ||
| # | ||
| # Copyright 2018-2023 contributors to the Marquez project | ||
| # SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| from github import Github | ||
| import rich_click as click | ||
| from datetime import date | ||
| from typing import TYPE_CHECKING | ||
| if TYPE_CHECKING: | ||
| from github.PullRequest import PullRequest | ||
|
|
||
| class GetChanges: | ||
|
|
||
| def __init__(self, github_token: str, previous: str, current: str, path: str): | ||
| self.github_token = github_token | ||
| self.previous = previous | ||
| self.current = current | ||
| self.path = path | ||
| self.pulls: list[PullRequest] = [] | ||
| self.rel_title_str: str = '' | ||
| self.text: list[str] = [] | ||
| self.new_contributors: dict[str:str] = {} | ||
|
|
||
| def get_pulls(self): | ||
| print('Working on it...') | ||
| g = Github(self.github_token) | ||
| repo = g.get_repo("MarquezProject/marquez") | ||
| prev_date = repo.get_release(self.previous).created_at | ||
| commits = repo.get_commits(since=prev_date) | ||
| self.pulls = [pull for commit in commits for pull in commit.get_pulls()] | ||
|
|
||
| def write_title(self): | ||
| self.rel_title_str = f'## [{self.current}](https://github.com/MarquezProject/marquez/compare/{self.previous}...{self.current}) - {date.today()}' | ||
|
|
||
| def describe_changes(self): | ||
| for pull in self.pulls: | ||
|
|
||
| """ Assembles change description with PR and user URLs """ | ||
| entry = [] | ||
| if pull.user.login != 'dependabot[bot]': | ||
| labels = [] | ||
| for label in pull.labels: | ||
| if label.name != 'documentation': | ||
| labels.append(label.name) | ||
| change_str = f'* **{labels[0]}: {pull.title}** [`#{pull.number}`]({pull.html_url}) [@{pull.user.login}]({pull.user.html_url}) ' | ||
|
|
||
| """ Extracts one-line description if present """ | ||
| beg = pull.body.find('One-line summary:') + 18 | ||
| if beg == 17: | ||
| change_descrip_str = ' **' | ||
| else: | ||
| test = pull.body.find('### Checklist') | ||
| if test == -1: | ||
| end = beg + 75 | ||
| else: | ||
| end = test - 1 | ||
| descrip = pull.body[beg:end].split() | ||
| descrip_str = ' '.join(descrip) | ||
| change_descrip_str = f' *{descrip_str}*' | ||
|
|
||
| entry.append(change_str + '\n') | ||
| entry.append(change_descrip_str + '\n') | ||
| self.text.append(entry) | ||
|
|
||
| def get_new_contributors(self): | ||
| for pull in self.pulls: | ||
| comments = pull.get_issue_comments() | ||
| for comment in comments: | ||
| if 'Thanks for opening your' in comment.body: | ||
| self.new_contributors[pull.user.login] = pull.user.url | ||
| if self.new_contributors: | ||
| print('New contributors:') | ||
| for k, v in self.new_contributors.items(): | ||
| print(f'@{k}: {v}') | ||
| else: | ||
| print('Note: no new contributors were found.') | ||
|
|
||
| def update_changelog(self): | ||
| f = open('changes.txt', 'w+') | ||
| f = open('changes.txt', 'a') | ||
| f.write(self.rel_title_str + '\n') | ||
| for entry in self.text: | ||
| for line in entry: | ||
| f.write(line) | ||
| f.close() | ||
|
|
||
| with open('changes.txt', 'r+') as f: | ||
| new_changes = f.read() | ||
| with open(self.path, 'r') as contents: | ||
| save = contents.read() | ||
| with open(self.path, 'w') as contents: | ||
| contents.write(new_changes) | ||
| with open(self.path, 'a') as contents: | ||
| contents.write(save) | ||
|
|
||
| @click.command() | ||
| @click.option( | ||
| '--github_token', type=str, default='' | ||
| ) | ||
| @click.option( | ||
| "--previous", type=str, default='' | ||
| ) | ||
| @click.option( | ||
| "--current", type=str, default='' | ||
| ) | ||
| @click.option( | ||
| "--path", | ||
| type=str, | ||
| default='', | ||
| help='absolute path to changelog', | ||
| ) | ||
|
|
||
| def main( | ||
| github_token: str, | ||
| previous: str, | ||
| current: str, | ||
| path: str, | ||
| ): | ||
| c = GetChanges( | ||
| github_token=github_token, | ||
| previous=previous, | ||
| current=current, | ||
| path=path | ||
| ) | ||
| c.get_pulls() | ||
| c.describe_changes() | ||
| c.write_title() | ||
| c.update_changelog() | ||
| c.get_new_contributors() | ||
| print('...done!') | ||
|
|
||
| if __name__ == "__main__": | ||
| main() | ||
|
|
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,35 @@ | ||
| PyYAML~=5.4.1 | ||
| requests~=2.25.1 | ||
| github-action-utils~=1.1.0 | ||
| aiohttp==3.8.1 | ||
| aiosignal==1.3.1 | ||
| async-timeout==4.0.2 | ||
| attrs==22.2.0 | ||
| bump2version==1.0.1 | ||
| certifi==2022.12.7 | ||
| cffi==1.15.1 | ||
| chardet==4.0.0 | ||
| charset-normalizer==2.1.1 | ||
| click==8.1.3 | ||
| Deprecated==1.2.13 | ||
| frozenlist==1.3.3 | ||
| github==1.2.7 | ||
| github-action-utils==1.1.0 | ||
| idna==2.10 | ||
| markdown-it-py==2.1.0 | ||
| mdurl==0.1.2 | ||
| multidict==6.0.4 | ||
| pendulum==2.1.2 | ||
| pycparser==2.21 | ||
| PyGithub==1.57 | ||
| Pygments==2.14.0 | ||
| PyJWT==2.6.0 | ||
| PyNaCl==1.5.0 | ||
| python-dateutil==2.8.2 | ||
| pytzdata==2020.1 | ||
| PyYAML==5.4.1 | ||
| requests==2.25.1 | ||
| rich==13.3.1 | ||
| rich-click==1.6.1 | ||
| six==1.16.0 | ||
| typing_extensions==4.4.0 | ||
| urllib3==1.26.14 | ||
| wrapt==1.14.1 | ||
| yarl==1.8.2 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.