-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat/river schema v2 compatibility #106
base: main
Are you sure you want to change the base?
Conversation
a59a4df
to
dadffb8
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you wanna set up a different base branch in case we need to send a patch against main?
if x['{discriminator_name}'] | ||
== '{discriminator_value}' | ||
if x[{repr(discriminator_name)}] | ||
== {repr(discriminator_value)} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
huh, how did this even work lol
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's equivalent in the happy path but repr
will surface bugs in the not happy path
init: InitType, | ||
request: Optional[AsyncIterable[RequestType]], | ||
init_serializer: Callable[[InitType], Any], | ||
request_serializer: Optional[Callable[[RequestType], Any]], |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ideally the type is if request
exists then request_serializer
should not be optional.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe it should be request: Optional[ tuple[ {request_type}, {request_serializer_type} ] ]
procedure_name=procedure_name, | ||
payload=init_serializer(init), | ||
) | ||
first_message = False |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
first_message
should be no longer needed since we always send a STREAM_OPEN_BIT
and we can remove the bit from send_close_stream
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Based on how the code worked previously, if we had an exception during sending the first message, then STREAM_OPEN_BIT
was sent in the send_close_stream
-- was that not intentional?
control_flags = 0 | ||
await self.send_message( | ||
stream_id=stream_id, | ||
service_name=service_name, | ||
procedure_name=procedure_name, | ||
control_flags=control_flags, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
control_flags = 0 | |
await self.send_message( | |
stream_id=stream_id, | |
service_name=service_name, | |
procedure_name=procedure_name, | |
control_flags=control_flags, | |
await self.send_message( | |
stream_id=stream_id, | |
service_name=service_name, | |
procedure_name=procedure_name, | |
control_flags=0, |
'{name}', | ||
input, | ||
{render_input_method}, | ||
return await self.client.send_rpc( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
all this stuff is hard to review (and I'm sure iterate on), we should set up some simple tests. Snapshot testing is probably good enough https://pypi.org/project/pytest-snapshot/.
I did this for river js replit/river#261 using https://vitest.dev/guide/snapshot
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The best way I have found to iterate on this is to just run the codegen against different production schemas and verify that the set of changes is what I expect. I'm happy to wait until after I land the babel changes before merging this.
The codegen currently isn't compositional enough to make snapshot testing manageable. The more I work with it though I've been trying to come up with strategies to make it less imperative without increasing cognitive load too much, I'll land snapshot testing then.
] | ||
# Non-oneof fields. | ||
oneofs: DefaultDict[int, List[descriptor_pb2.FieldDescriptorProto]] = ( | ||
oneofs: DefaultDict[int, list[descriptor_pb2.FieldDescriptorProto]] = ( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's the difference?
I wonder if it's like the stupid typescript thing where String
(object) is different from string
(literal/primitive) 😬
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Python 3.8 and below did not permit the builtin types to be subscripted, typing.List
and typing.Dict
were added as future-compatible shims during the migration. Python 3.8 is EOL as of the end of Spooky Season, so now we can 🚀.
dadffb8
to
4c0e071
Compare
If we need to patch, we should do feature patches off previous release tags. Since we're reserving 1.x for GA, I can follow the 0.200.0 strategy in this repo for schema-v2 releases. Doing that though would suggest that we do at least a halfway decent pass at the v2 server codegen as well though. Given what we discussed supporting v1/v2 using a naive shim doesn't seem too bad, probably can get that done this week. Wdyt? |
4c0e071
to
b71a809
Compare
b71a809
to
717cc68
Compare
Why
River Schema v2 compatibility so we can start getting rid of the compatibility layer.
This PR is just the first, it does not introduce the half-close semantics yet.
What changed
init
is now required for all method types,input
is now optional. This simplifiesrpc
andstream
codegen.init
was being used ininput
position and vice versa.f"'{foo}'"
encoding forf"{repr(foo)}"
, giving greater safety post-generation (to avoid situations wheref"'{None}'"
gets rendered as"None"
.f"{repr(None)}"
would render asNone
which would fail typechecks.)return
List
andDict
tolist
anddict
encode_...
lambdas fordef
, to placate both line length as well as lint suggestions (and increase readability)Test plan
Manually ran codegen against v2 generated schemas, everything typechecks.