Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adding GitHub provider #21076

Merged
merged 22 commits into from
Jan 28, 2022
Merged

Adding GitHub provider #21076

merged 22 commits into from
Jan 28, 2022

Conversation

pateash
Copy link
Contributor

@pateash pateash commented Jan 24, 2022

closes: #20300

Description

Adding Github provider based on Python SDK https://github.com/PyGithub/PyGithub
The SDK has many objects which can be viewed at https://pygithub.readthedocs.io/en/latest/github_objects.html

Users can create their own custom operators leveraging the PyGithub SDK, either by leveraging Hook directly or building their operator on GithubOperator by providing method and method_args from PyGithub
results can be further processed by implementing result_processor method, please look into GithubTagSensor for reference.

github_operator = GithubOperator(
    task_id='github-test',
    github_method="get_repo",
    github_method_args={'full_name_or_id': 'apache/airflow'},
    result_processor=lambda r: r.full_name,
    dag=self.dag,
)

@pateash pateash changed the title 20300 add GitHub provider [WIP]: 20300 add GitHub provider Jan 24, 2022
@pateash pateash changed the title [WIP]: 20300 add GitHub provider [WIP]: 20300- Adding GitHub provider Jan 24, 2022
@potiuk potiuk changed the title [WIP]: 20300- Adding GitHub provider [WIP]: Adding GitHub provider Jan 25, 2022
@potiuk potiuk changed the title [WIP]: Adding GitHub provider [WIP]: Add GitHub provider Jan 25, 2022
potiuk
potiuk requested changes Jan 25, 2022
View reviewed changes
Copy link
Member

@potiuk potiuk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice one!

I left a few comments, but it looks great already!

@pateash pateash requested a review from potiuk January 26, 2022 15:44
@pateash pateash changed the title [WIP]: Add GitHub provider Adding GitHub provider Jan 26, 2022
@pateash
Copy link
Contributor Author

pateash commented Jan 26, 2022

image

image

@pateash
Copy link
Contributor Author

pateash commented Jan 26, 2022

image

@pateash pateash force-pushed the 20300-Add-Github-provider branch from 3fcacda to 58e3f99 Compare January 26, 2022 15:49
@pateash
Copy link
Contributor Author

pateash commented Jan 26, 2022

Hi @potiuk,
thanks for the comments, Please let me know if anything else needed.

@pateash pateash changed the title Adding GitHub provider WIP: Adding GitHub provider Jan 26, 2022
@pateash pateash requested a review from eladkal January 26, 2022 16:55
pateash added 22 commits January 28, 2022 12:24
@pateash
20300 - initial files added
a774d51
@pateash
20300 - test files added
a742f34
@pateash
20300 - ui behaviour added
1224ebf
@pateash
20300 - fixed depedency version
73b7f91
@pateash
20300 - fixed the provider in UI
c4a305b
@pateash
20300 - files refactored to allow execution of parameterized methods
fa2b56e
@pateash
20300 - tag sensor implemented
d78e150
@pateash
20300 - github tag sensor working in local env
25304ae
@pateash
20300 - github tag sensor test working
85bf561
@pateash
20300 - github hook test added
8b3361d
@pateash
20300 - github operator test added
d18eddc
@pateash
20300 - precommit changes
bb33893
@pateash
20300 - github Enterprise connection added
47f8fc1
@pateash
20300 - PR comments resolved
19b684a
@pateash
20300 - adding docs
929636e
@pateash
20300 - adding docs
5b1d5a7
@pateash
20300 - adding docs for sensors
3868ab2
@pateash
20300 - adding docs done
0c38beb
@pateash
20300 - fixing unintented docs changes
a5071f3
@pateash
20300 - more information added in docs
8401865
@pateash
20300 - fixing static checks error
aa35f33
@pateash
20300 - fixing static checks error
8369d9f
@pateash pateash force-pushed the 20300-Add-Github-provider branch from 94c689a to 8369d9f Compare January 28, 2022 12:24
eladkal
eladkal approved these changes Jan 28, 2022
View reviewed changes
Copy link
Contributor

@eladkal eladkal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@potiuk any further comments?

@potiuk potiuk merged commit 5d23fbe into apache:main Jan 28, 2022
@potiuk
Copy link
Member

potiuk commented Jan 28, 2022

Just in time for the January wave of Providers :)

@eladkal eladkal mentioned this pull request Feb 11, 2022
Closed
74 tasks
@pateash pateash deleted the 20300-Add-Github-provider branch February 15, 2022 13:49
@jedcunningham jedcunningham added the changelog:skip Changes that should be skipped from the changelog (CI, tests, etc..) label Mar 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@potiuk potiuk potiuk approved these changes

@eladkal eladkal eladkal approved these changes

@kaxil kaxil Awaiting requested review from kaxil

@mik-laj mik-laj Awaiting requested review from mik-laj

@vikramkoka vikramkoka Awaiting requested review from vikramkoka

Assignees
No one assigned
Labels
area:dev-tools area:providers changelog:skip Changes that should be skipped from the changelog (CI, tests, etc..) kind:documentation
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add Github provider
4 participants
@pateash @eladkal @potiuk @jedcunningham