Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add async rest transport call methods #2140

Merged

Conversation

ohmayr
Copy link
Contributor

@ohmayr ohmayr commented Sep 11, 2024

This PR adds asynchronous rest transport call methods which when called raise a NotImplementedError. Follow up PRs will implement this method class.

As of this PR, asynchronous rest transport does not support calling any type of methods.

This PR should be reviewed and merged after: #2139.

@ohmayr ohmayr marked this pull request as ready for review September 11, 2024 00:21
@ohmayr ohmayr requested a review from a team as a code owner September 11, 2024 00:21
@product-auto-label product-auto-label bot added the size: l Pull request size is large. label Sep 11, 2024
Copy link
Contributor

@parthea parthea left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added non-blocking comments. Feel free to follow up in a later PR

{# TODO (https://github.com/googleapis/gapic-generator-python/issues/2128): Update `rest_version` to include the transport dependency version. #}
DEFAULT_CLIENT_INFO = gapic_v1.client_info.ClientInfo(
gapic_version=BASE_DEFAULT_CLIENT_INFO.gapic_version,
grpc_version=None,
rest_version=None,
)

{# TODO: Add an `_interceptor` property once implemented #}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please include the TODO bug to follow up

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's fine to ignore this one since this comment is addressed and removed in a follow up PR: #2151.

@property
def {{method.transport_safe_name|snake_case}}(self) -> Callable[
[{{method.input.ident}}],
{{method.output.ident}}]:

return # type: ignore
return self._{{method.name}}(self._session, self._host) # type: ignore
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we still need the # type: ignore ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. My initial thought was that it's probably a mismatch since I wasn't returning a Callable. However, I later realised that we're doing the same in rest.py.

I haven't really looked into why we've been ignoring the type in rest.py but I'm following whatever we're doing there. Probably because the Callables for client streaming which raise NotImplmentedError do not return anything so mypy isn't happy about it? Regardless, I've filed an issue to investigate and remove this as a follow up.

@ohmayr ohmayr force-pushed the wrap-async-methods-with-error-handling-logic branch from 79cc77d to 23cb93e Compare September 12, 2024 20:37
Copy link
Contributor

@vchudnov-g vchudnov-g left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good, but I'll have another look after this is rebased on top of the previous PRs.

return hash("Async{{service.name}}RestTransport.{{method.name}}")

async def __call__(self,
request: {{method.input.ident}}, *,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does the asterisk mean here?

Copy link
Contributor Author

@ohmayr ohmayr Sep 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

* enforces that arguments following request must be passed in as keyword arguments.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, it's a Python thing. Because it was on the same line, I assumed it was part of the type hint and glossed over the comma.

I suggest having the * in its own line, like the other parameters. It gets lost here.

@property
def {{method.transport_safe_name|snake_case}}(self) -> Callable[
[{{method.input.ident}}],
{{method.output.ident}}]:

return # type: ignore
return self._{{method.name}}(self._session, self._host) # type: ignore
Copy link
Contributor

@vchudnov-g vchudnov-g Sep 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So each of these properties is a callable that invokes the appropriate inner class. When do the self._wrapped_methods get called? (And which of your PRs will ensure that?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wrapping happens during transport initialization / construction via self._prep_wrapped_messages(client_info). If you look at the implementation for this helper, it creates a dictionary called _wrapped_methods which is something like {method : wrapped_method, ... }.

At the client level instead of calling method(), we call self._wrapped_methods[method](). However, we're not doing this for the following methods:

  • set_iam_policy
  • get_iam_policy
  • test_iam_permissions

FYI these are mixin methods. We don't pre-wrap mixin methods which is why this logic isn't added. We probably should do that or at least pass down the kind property to avoid gRPC code path in a rest call at the very least.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is important. I've filed #2156 to ensure that we do this as a follow up since this PR doesn't touch the client layer.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the client level instead of calling method(), we call self._wrapped_methods[method]()

OK, this is what I was looking for.

Once we have self._wrapped_methods, do we ever call a method() directly? If we don't, then the most elegant thing would arguably be to overwrite the properties with their wrapped versions, and just called each (now-wrapped) method() directly. Does that make sense? If so, it would be a good code simplification to do later.

Base automatically changed from wrap-async-methods-with-error-handling-logic to async-rest-support-in-gapics September 12, 2024 23:01
@ohmayr ohmayr force-pushed the implement-async-rest-call-method-class branch from 6444826 to c562932 Compare September 12, 2024 23:20
@ohmayr ohmayr force-pushed the implement-async-rest-call-method-class branch from c562932 to 8fd829c Compare September 13, 2024 00:49
Copy link
Contributor

@vchudnov-g vchudnov-g left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but PTAL @ my previous comments before merging.

@ohmayr ohmayr merged commit 5c32e9e into async-rest-support-in-gapics Sep 13, 2024
71 checks passed
@ohmayr ohmayr deleted the implement-async-rest-call-method-class branch September 13, 2024 15:51
ohmayr added a commit that referenced this pull request Sep 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size: l Pull request size is large.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants