Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛 Source GitHub: fix bug with IssueEvents stream and add handling for rate limiting #4708

Merged
merged 14 commits into from
Jul 15, 2021
Merged

🐛 Source GitHub: fix bug with IssueEvents stream and add handling for rate limiting #4708

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
315c8c5
Few updates for GitHub source
Zirochkaa Jul 12, 2021
3837336
Update `github.md` file
Zirochkaa Jul 12, 2021
0e6e00a
Implement change request
Zirochkaa Jul 14, 2021
8981064
Merge branch 'master' of github.com:airbytehq/airbyte into oleh/4612-…
Zirochkaa Jul 14, 2021
7e99263
Implement requested changes
Zirochkaa Jul 14, 2021
bb549f1
apply suggestions from @keu
eugene-kulak Jul 14, 2021
badafc4
format
eugene-kulak Jul 14, 2021
b404164
Update airbyte-integrations/connectors/source-github/source_github/st…
sherifnada Jul 14, 2021
d153724
re-think the decision about direction of the read
eugene-kulak Jul 14, 2021
3361a42
groom
eugene-kulak Jul 14, 2021
ce34018
fix VCR cache
eugene-kulak Jul 14, 2021
9441eae
format
eugene-kulak Jul 14, 2021
5a51fd7
fix file removal for python 3.7
eugene-kulak Jul 15, 2021
3cd57be
bumping version
eugene-kulak Jul 15, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions ...ain/resources/config/STANDARD_SOURCE_DEFINITION/ef69ef6e-aa7f-4af1-a01d-ef775033524e.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"sourceDefinitionId": "ef69ef6e-aa7f-4af1-a01d-ef775033524e",
"name": "GitHub",
"dockerRepository": "airbyte/source-github",
"dockerImageTag": "0.1.1",
"documentationUrl": "https://hub.docker.com/r/airbyte/source-github",
"dockerImageTag": "0.1.2",
"documentationUrl": "https://docs.airbyte.io/integrations/sources/github",
"icon": "github.svg"
}
4 changes: 2 additions & 2 deletions airbyte-config/init/src/main/resources/seed/source_definitions.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,8 @@
- sourceDefinitionId: ef69ef6e-aa7f-4af1-a01d-ef775033524e
name: GitHub
dockerRepository: airbyte/source-github
dockerImageTag: 0.1.1
documentationUrl: https://hub.docker.com/r/airbyte/source-github
dockerImageTag: 0.1.2
documentationUrl: https://docs.airbyte.io/integrations/sources/github
icon: github.svg
- sourceDefinitionId: b5ea17b1-f170-46dc-bc31-cc744ca984c1
name: Microsoft SQL Server (MSSQL)
Expand Down
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-github/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,5 @@ RUN pip install .
ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]

LABEL io.airbyte.version=0.1.1
LABEL io.airbyte.version=0.1.2
LABEL io.airbyte.name=airbyte/source-github
23 changes: 12 additions & 11 deletions airbyte-integrations/connectors/source-github/source_github/source.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,22 +64,23 @@ def check_connection(self, logger: AirbyteLogger, config: Mapping[str, Any]) ->
def streams(self, config: Mapping[str, Any]) -> List[Stream]:
authenticator = TokenAuthenticator(token=config["access_token"], auth_method="token")
full_refresh_args = {"authenticator": authenticator, "repository": config["repository"]}
incremental_args = {"authenticator": authenticator, "repository": config["repository"], "start_date": config["start_date"]}
incremental_args = {**full_refresh_args, "start_date": config["start_date"]}

return [
Assignees(**full_refresh_args),
Reviews(**full_refresh_args),
Collaborators(**full_refresh_args),
Teams(**full_refresh_args),
IssueLabels(**full_refresh_args),
Releases(**incremental_args),
Events(**incremental_args),
Comments(**incremental_args),
PullRequests(**incremental_args),
CommitComments(**incremental_args),
IssueMilestones(**incremental_args),
Commits(**incremental_args),
Stargazers(**incremental_args),
Projects(**incremental_args),
Issues(**incremental_args),
Events(**incremental_args),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍🏼 ty for sorting

IssueEvents(**incremental_args),
IssueLabels(**full_refresh_args),
IssueMilestones(**incremental_args),
Issues(**incremental_args),
Projects(**incremental_args),
PullRequests(**incremental_args),
Releases(**incremental_args),
Reviews(**full_refresh_args),
Stargazers(**incremental_args),
Teams(**full_refresh_args),
]
Loading