Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Update GitHub functions #368

Merged
merged 19 commits into from
Feb 1, 2024
Merged

Conversation

eric-nguyen-cs
Copy link
Contributor

@eric-nguyen-cs eric-nguyen-cs commented Jan 31, 2024

What

  • Replace PyGithub with Githubkit for async HTTP requests
  • 7c4db4b Use run_in_threadpool to run I/O-bound synchronous functions inside of asynchronous path operations/background tasks (if FastAPI detects that a path operation/background task is a co-routine, it directly runs the function in the main thread, instead of in a worker thread, which can block the server from responding if the co-routine contains blocking synchronous code - like the parsing function on big taxonomies) -> Fixes Github export, by using AsyncTransactionContext in BackgroundTasks (this also allows for controllers code reuse)
  • Export from the correct SHA when creating a new PR (used new project_controllers and project_models)
  • Add a new commit when exporting an exported taxonomy (instead of raising an GtihubBranchAlreadyExists error)

Fixes bug(s)

Fixes #349

Copy link
Member

@alexgarel alexgarel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great PR.

Globally ok, if we can keep tests, it would be better !

backend/editor/entries.py Outdated Show resolved Hide resolved
backend/editor/entries.py Show resolved Hide resolved
backend/editor/entries.py Outdated Show resolved Hide resolved
backend/editor/entries.py Outdated Show resolved Hide resolved
backend/editor/github_functions.py Outdated Show resolved Hide resolved
backend/pyproject.toml Show resolved Hide resolved
Comment on lines -44 to -55
def test_import_from_github(client, github_mock, mocker):
# We mock the TaxonomyGraph.import_from_github method,
# which downloads the taxonomy file from a Github URL
mocker.patch("editor.api.TaxonomyGraph.import_from_github", return_value=True)

response = client.post(
"/test/testing_branch/import",
json={"description": "test_description"},
)

assert response.status_code == 200
assert response.json() is True
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find this test useful, at least we will see when it break.

Of course this test should have more checks:

  • is project updated etc. 
  • eventually which methods were called on the mock

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really sure what value this tests brings, as the only difference with the /upload endpoint is that we have to access Github, which should be mocked

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hear the first part of the video you mentioned in other comment. I'm ok with what is said in the video. I also value more integration test than unit test if functions are just assembling stuff.

But here, this is no unit test, this is integration test, that's why I think it's useful.

Comment on lines -70 to -81
def test_add_taxonomy_duplicate_branch_name(client, github_mock):
github_mock.return_value.get_repo.return_value.get_branches.return_value[0].name = "test_branch"

with open("tests/data/test.txt", "rb") as f:
response = client.post(
"/test_taxonomy_2/test_branch/upload",
files={"file": f},
data={"description": "test_description"},
)

assert response.status_code == 409
assert response.json() == {"detail": "branch_name: Branch name should be unique!"}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here also test is useful I think, here again we might add more checks.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, I believe this checks if the mock function works correctly and if we check for duplicate branches by using the get_all_branches endpoint of Github (that is not the case anymore, we check if the branch we want to create already exists or not).
My philosophy on tests follows quite closely what this engineer describes in this video: https://www.youtube.com/watch?v=MbU-PKukdMw
If that's ok, I'll merge the PR and if you want to add the test back, I'll do that in another PR

Copy link
Contributor

@perierc perierc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good work, happy with the new models and controller, a few nitpicks

backend/pyproject.toml Show resolved Hide resolved
backend/editor/github_functions.py Outdated Show resolved Hide resolved
backend/editor/github_functions.py Outdated Show resolved Hide resolved
backend/editor/github_functions.py Outdated Show resolved Hide resolved
backend/editor/entries.py Outdated Show resolved Hide resolved
@eric-nguyen-cs eric-nguyen-cs merged commit 3fefb1d into main Feb 1, 2024
7 checks passed
@eric-nguyen-cs eric-nguyen-cs deleted the ericn/update-github-functions branch February 1, 2024 21:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

Update Github API calls
3 participants