diff --git a/.github/package.json b/.github/package.json index ddad85d9..e9011ce7 100644 --- a/.github/package.json +++ b/.github/package.json @@ -2,6 +2,6 @@ "name": "fake_package_json_for_github_action_to_read_version", "private": true, "devDependencies": { - "pyright": "1.1.293" + "pyright": "1.1.319" } } diff --git a/CHANGELOG.md b/CHANGELOG.md index b2350625..e0c1f12e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,8 @@ ## Upcoming +- Add gRPC aio stub and servicer generation (#489) +- Bump tested dependencies to pyright==1.1.319, mypy==1.4.1, protobuf==4.23.4, grpcio-tools==1.56.2 + ## 3.4.0 - Mark messages as @typing.final @@ -10,9 +13,9 @@ ## 3.3.0 - Prefer (mypy_protobuf.options).casttype to (mypy_protobuf.casttype) - - Allows us to use a single extension number - - Applies to casttype,keytype,valuetype - - Deprecate (but still support) the old-style extension + - Allows us to use a single extension number + - Applies to casttype,keytype,valuetype + - Deprecate (but still support) the old-style extension - Prefer importing from `typing` over `typing_extensions` on new enough python versions - Support emitting module docstrings - Make generated code flake8 compatible @@ -43,8 +46,8 @@ - Require protobuf 3.19.1 - Change `EnumTypeWrapper.V` to `EnumTypeWrapper.ValueType` per https://github.com/protocolbuffers/protobuf/pull/8182. -Will allow for unquoted annotations starting with protobuf 3.20.0. `.V` will continue to work for the foreseeable -future for backward compatibility. + Will allow for unquoted annotations starting with protobuf 3.20.0. `.V` will continue to work for the foreseeable + future for backward compatibility. - suppress pyright warning reportSelfClsParameterName when a proto field is named `self` - Allow optional constructor keywords for primitive field types in proto3, following this [chart](https://github.com/protocolbuffers/protobuf/blob/master/docs/field_presence.md#presence-in-proto3-apis). - Reorder Enum helper classes to eliminate pycharm errors @@ -84,11 +87,13 @@ future for backward compatibility. ## 2.7 - Fix [#244](https://github.com/nipunn1313/mypy-protobuf/issues/244) - support extensions defined at module scope with proper types, matching extensions defined within Messages. See [`_ExtensionDict`](https://github.com/python/typeshed/blob/4765978f6ceeb24e10bdf93c0d4b72dfb35836d4/stubs/protobuf/google/protobuf/internal/extension_dict.pyi#L9) + ```proto extend google.protobuf.MessageOptions { string test_message_option = 51234; } ``` + ```python # Used to generate test_message_option: google.protobuf.descriptor.FieldDescriptor = ... @@ -96,28 +101,33 @@ test_message_option: google.protobuf.descriptor.FieldDescriptor = ... # Now generates test_message_option: google.protobuf.internal.extension_dict._ExtensionFieldDescriptor[google.protobuf.descriptor_pb2.MessageOptions, typing.Text] = ... ``` + Fix repeated extensions as well - to generate RepeatedScalarFieldContainer and RepeatedCompositeFieldContainer + - Now requires [types-protobuf](https://pypi.org/project/types-protobuf/) 3.17.3 - Fix [#238](https://github.com/nipunn1313/mypy-protobuf/issues/238) - handling enum variants that name conflict with EnumTypeWrapper methods - Improve mypy-protobuf testsuite expected-errors file to make insertions/deletions easier to code review - Fix [#227](https://github.com/nipunn1313/mypy-protobuf/issues/227) - improve support for messages/enums with python reserved keyword names - Order fields within a message in original .proto file order Previously, they were grouped by scalar/nonscalar. Remove -that grouping to make it a bit easier to correlate .proto files to .pyi files. + that grouping to make it a bit easier to correlate .proto files to .pyi files. ## 2.6 - Bump protoc support to 3.17.3 - Use latest python versions in tests (3.6.14 3.7.11 3.8.11 3.9.6) - Support reserved names for message types. Previously generated invalid mypy. + ```proto message M { message None {} None none = 1; } ``` + - Support `protoc-gen-mypy -V` and `protoc-gen-mypy --version` to print version number - Return `Optional[Literal[...]]` instead of `Literal[...]` from WhichOneof to support -cases in which none of the fields of the WhichOneof are set. See the following example. + cases in which none of the fields of the WhichOneof are set. See the following example. + ```python def hello(name: str) -> None: ... n = proto.WhichOneof("name") @@ -125,6 +135,7 @@ hello(n) # Will now result in a mypy error. assert n is not None hello(n) # Should work ok ``` + - Bump mypy version to 0.910, utilizing stubs types-protobuf==0.1.14. See https://mypy-lang.blogspot.com/2021/05/the-upcoming-switch-to-modular-typeshed.html - Bump grpcio version tested to 1.38.1 and grpc-stubs to 1.24.6 - Generate a `# type: ignore` for enum generated stubs to avoid circular dependency described in #214. Bandaid solution. @@ -156,13 +167,15 @@ hello(n) # Should work ok ## 2.0 Non Backward Compatible Changes + - Dropping support for running mypy-protobuf in python <= 3.5. Note you can still generate stubs target-compatible to python2 - Type proto Enum values for as `MyEnum.V` rather than `MyEnumValue` for import ergonomics, -allowing the caller to import `MyEnum` rather than conditionally importing `MyEnumValue` + allowing the caller to import `MyEnum` rather than conditionally importing `MyEnumValue` - Default disallow `None` as argument for primitive fields of constructors in proto3. -Provided `relax_strict_optional_primitives` flag to relax this strictness if you prefer. + Provided `relax_strict_optional_primitives` flag to relax this strictness if you prefer. New Features + - Support for `grpcio` stubs generation - Allow `mypy_protobuf.py` to be run directly as a script - Add support for proto's [`well_known_types`](https://developers.google.com/protocol-buffers/docs/reference/python-generated#wkt) @@ -172,16 +185,18 @@ New Features - Add support for `import public` proto imports - by reexporting in generated code Output Format + - Generate fully qualified references rather than mangling - Import builtins library rather than mangling builtins - Use fully qualified names rather than mangling imports - Only mangle-alias top-level identifiers w/ `global___` to avoid conflict w/ fields of same name [previously was mangling inner messages as well] - Add support for `readable_stubs` parameter for mangle-free output code w/o fully-qualified references to message. -(in many cases this is good enough and easier to read) + (in many cases this is good enough and easier to read) - Generate `arg: Optional[type] = ...` instead of `arg: Optional[type] = None` - Avoid importing google.protobuf.message.Message unless it's needed Internal Improvements + - Add support for python 3.9 to CI - Update mypy-protobuf CI to target 3.8 rather than 3.5 - Inline mypy annotations, eliminate six, and remove `__future__` import in `mypy_protobuf_lib.py` @@ -200,46 +215,55 @@ Internal Improvements - Support `Message.HasExtension` and `Message.ClearExtension` - Bump python-protobuf from 3.11.3 to 3.13.0 - Add support for optional proto3 fields -- Support ScalarMap and MessageMap generated types for map types in proto. This will allow us to support `get_or_create` +- Support ScalarMap and MessageMap generated types for map types in proto. This will allow us to support `get_or_create` ```proto message Message { map map_message = 17 } ``` + and + ```python message.map_message.get_or_create(0) ``` Before (1.23) + ``` main.py:4: error: "MutableMapping[str, Nested]" has no attribute "get_or_create" [attr-defined] ``` + After (1.24) - there is no error ## 1.23 - Inherit FromString from superclass Message - rather than re-generating here. Fixes bug -in python2 usage `google/protobuf/type_pb2.pyi:92: error: Argument 1 of "FromString" is incompatible with supertype "Message"; supertype defines the argument type as "ByteString" [override]` + in python2 usage `google/protobuf/type_pb2.pyi:92: error: Argument 1 of "FromString" is incompatible with supertype "Message"; supertype defines the argument type as "ByteString" [override]` ## 1.22 - Update tested/required mypy version to 0.780 (picks up new typeshed annotations). Includes improved typing/error messages on Message. Before (mypy < 0.780): + ``` test_negative/negative.py:26: error: Argument 1 to "CopyFrom" of "Message" has incompatible type "str"; expected "Message" ``` + After (mypy >= 0.780: + ``` test_negative/negative.py:26: error: Argument 1 to "CopyFrom" of "Message" has incompatible type "str"; expected "Simple1" ``` + - Update generated EnumTypeWrapper to be instances of EnumTypeWrapper - for more consistency -with generated python code. Most caller code should not require mypy type changes. Egh -`ProtoEnum.Value('first')` should work either way. + with generated python code. Most caller code should not require mypy type changes. Egh + `ProtoEnum.Value('first')` should work either way. Generated Before (in 1.21) + ```python class ProtoEnum(object): @classmethod @@ -247,11 +271,13 @@ class ProtoEnum(object): ``` Generated After (in 1.22) + ```python ProtoEnum: _ProtoEnum class _ProtoEnum(google.protobuf.EnumTypeWrapper): def Value(self, name: str) -> ProtoEnumValue ``` + - Remove autogenerated EnumTypeWrapper methods that are redundant to the typeshed parent class. Added testing for these. ## 1.21 @@ -259,9 +285,10 @@ class _ProtoEnum(google.protobuf.EnumTypeWrapper): - Support for module descriptor. - Update mangling from `global__` to `message__` - Fix bug in message typing for nested enums. Split EnumValue from EnumTypeWrapper. Enforces that constructing -an enum value must happen via a NewType wrapper to the int. + an enum value must happen via a NewType wrapper to the int. Example: + ```proto enum ProtoEnum { FIRST = 1; @@ -272,7 +299,9 @@ mesage ProtoMsg { ProtoEnum enum = 1; } ``` + Generated Before (in 1.20): + ```python class ProtoEnum(object): @classmethod @@ -281,7 +310,9 @@ class ProtoEnum(object): class ProtoMsg(Message): def __init__(self, enum: ProtoEnum) -> None ``` + Generated After (in 1.21): + ```python ProtoEnumValue = NewType('ProtoEnumValue', int) class ProtoEnum(object): @@ -291,9 +322,11 @@ class ProtoEnum(object): class ProtoMsg(Message): def __init__(self, enum: ProtoEnumValue) -> None ``` + Migration Guide (with example calling code) Before (with 1.20) + ```python from msg_pb2 import ProtoEnum, ProtoMsg @@ -301,7 +334,9 @@ def make_proto_msg(enum: ProtoEnum) -> ProtoMsg: return ProtoMsg(enum) make_proto_msg(ProtoMsg.FIRST) ``` + After (with 1.21) + ```python from msg_pb2 import ProtoEnum, ProtoMsg @@ -346,5 +381,4 @@ make_proto_msg(ProtoMsg.FIRST) - Add `Message.DESCRIPTOR` - ## Older changelogs not available. Check git log if you need them! diff --git a/README.md b/README.md index 273a2fea..ea2c4e1b 100644 --- a/README.md +++ b/README.md @@ -12,22 +12,28 @@ Built originally with love at [Dropbox](https://github.com/dropbox) See [Changelog](CHANGELOG.md) for recent changes. ## Requirements to run mypy-protobuf + Earlier releases might work, but aren't tested -- [protoc >= 4.21.8](https://github.com/protocolbuffers/protobuf/releases) -- [python-protobuf >= 4.21.8](https://pypi.org/project/protobuf/) - matching protoc release + +- [protoc >= 23.4](https://github.com/protocolbuffers/protobuf/releases) +- [python-protobuf >= 4.23.4](https://pypi.org/project/protobuf/) - matching protoc release - [python >= 3.7](https://www.python.org/downloads/source/) - for running mypy-protobuf plugin. ## Requirements to run typecheckers on stubs generated by mypy-protobuf + Earlier releases might work, but aren't tested -- [mypy >= v0.941](https://pypi.org/project/mypy) or [pyright >= 1.1.206](https://github.com/microsoft/pyright) -- [python-protobuf >= 4.21.8](https://pypi.org/project/protobuf/) - matching protoc release -- [types-protobuf >= 3.20.4](https://pypi.org/project/types-protobuf/) - for stubs from the google.protobuf library + +- [mypy >= v1.4.1](https://pypi.org/project/mypy) or [pyright >= 1.1.206](https://github.com/microsoft/pyright) +- [python-protobuf >= 4.23.4](https://pypi.org/project/protobuf/) - matching protoc release +- [types-protobuf >= 4.23.0.2](https://pypi.org/project/types-protobuf/) - for stubs from the google.protobuf library ### To run typecheckers on code generated with grpc plugin - you'll additionally need + Earlier releases might work, but aren't tested -- [grpcio>=1.50.0](https://pypi.org/project/grpcio/) -- [grpcio-tools>=1.50.0](https://pypi.org/project/grpcio-tools/) -- [grpc-stubs>=1.24.9](https://pypi.org/project/grpc-stubs/) + +- [grpcio>=1.56.2](https://pypi.org/project/grpcio/) +- [grpcio-tools>=1.56.2](https://pypi.org/project/grpcio-tools/) +- [grpc-stubs>=1.53.0.2](https://pypi.org/project/grpc-stubs/) Other configurations may work, but are not continuously tested currently. We would be open to expanding this list - file an issue on the issue tracker. @@ -35,10 +41,13 @@ We would be open to expanding this list - file an issue on the issue tracker. ## Installation The plugin can be installed with + ``` pip3 install mypy-protobuf ``` + To install unreleased + ``` REV=main # or whichever unreleased git rev you'd like pip3 install git+https://github.com/nipunn1313/mypy-protobuf.git@$REV @@ -48,6 +57,7 @@ pip3 install git+https://github.com/nipunn1313/mypy-protobuf.git@$REV#subdirecto ``` In order to run mypy on the generated code, you'll need to install + ``` pip3 install mypy>=0.910 types-protobuf>=0.1.14 ``` @@ -56,14 +66,19 @@ pip3 install mypy>=0.910 types-protobuf>=0.1.14 On posix, protoc-gen-mypy is installed to python's executable bin. Assuming that's on your $PATH, you can run + ``` protoc --python_out=output/location --mypy_out=output/location ``` + Alternately, you can explicitly provide the path: + ``` protoc --plugin=protoc-gen-mypy=path/to/protoc-gen-mypy --python_out=output/location --mypy_out=output/location ``` + Check the version number with + ``` > protoc-gen-mypy --version ``` @@ -85,18 +100,21 @@ will appear as docstrings in .pyi files. Useful in IDEs for showing completions ### Types enum int values more strongly Enum int values produce stubs which wrap the int values in NewType + ```proto enum MyEnum { HELLO = 0; WORLD = 1; } ``` + Will yield an [enum type wrapper](https://github.com/python/typeshed/blob/16ae4c61201cd8b96b8b22cdfb2ab9e89ba5bcf2/stubs/protobuf/google/protobuf/internal/enum_type_wrapper.pyi) whose methods type to `MyEnum.ValueType` (a `NewType(int)` rather than `int`. This allows mypy to catch bugs where the wrong enum value is being used. Calling code may be typed as follows. In python >= 3.7 + ```python # May need [PEP 563](https://www.python.org/dev/peps/pep-0563/) to postpone evaluation of annotations # from __future__ import annotations # Not needed with python>=3.11 or protobuf>=3.20.0 @@ -108,11 +126,13 @@ f(MyEnum.Value("HELLO")) With protobuf <= 3.20.0, for usages of cast, the type of `x` must be quoted After protobuf >= 3.20.0 - `ValueType` exists in the python code and quotes aren't needed until [upstream protobuf](https://github.com/protocolbuffers/protobuf/pull/8182) includes `ValueType` + ```python cast('MyEnum.ValueType', x) ``` Similarly, for type aliases with protobuf < 3.20.0, you must either quote the type or hide it behind `TYPE_CHECKING` + ```python from typing import Tuple, TYPE_CHECKING HELLO = Tuple['MyEnum.ValueType', 'MyEnum.ValueType'] @@ -122,7 +142,7 @@ if TYPE_CHECKING: #### Enum int impl details -mypy-protobuf autogenerates an instance of the EnumTypeWrapper as follows. +mypy-protobuf autogenerates an instance of the EnumTypeWrapper as follows. ```python class _MyEnum: @@ -141,14 +161,14 @@ WORLD: MyEnum.ValueType # 1 `_MyEnumEnumTypeWrapper` extends the EnumTypeWrapper to take/return MyEnum.ValueType rather than int `MyEnum` is an instance of the `EnumTypeWrapper`. + - Use `_MyEnum` and of metaclass is an implementation detail to make MyEnum.ValueType a valid type w/o a circular dependency - `V` is supported as an alias of `ValueType` for backward compatibility - - ### Supports generating type wrappers for fields and maps M.proto + ```proto message M { uint32 user_id = 1 [(mypy_protobuf.options).casttype="mymod.UserId"]; @@ -158,17 +178,21 @@ message M { ]; } ``` + mymod.py + ```python UserId = NewType("UserId", int) Email = NewType("Email", Text) ``` ### `py_generic_services` + If `py_generic_services` is set in your proto file, then mypy-protobuf will generate service stubs. If you want GRPC stubs instead - use the GRPC instructions. ### `readable_stubs` + If `readable_stubs` is set, mypy-protobuf will generate easier-to-read stubs. The downside to this approach - is that it's possible to generate stubs which do not pass mypy - particularly in the case of name collisions. mypy-protobuf defaults to generating stubs with fully qualified @@ -176,6 +200,7 @@ imports and mangled global-level identifiers to defend against name collisions b identifiers and field names. If you're ok with this risk, try it out! + ``` protoc --python_out=output/location --mypy_out=readable_stubs:output/location ``` @@ -195,7 +220,9 @@ protoc --python_out=output/location --mypy_out=relax_strict_optional_primitives: ``` ### Output suppression + To suppress output, you can run + ``` protoc --python_out=output/location --mypy_out=quiet:output/location ``` @@ -203,6 +230,7 @@ protoc --python_out=output/location --mypy_out=quiet:output/location ### GRPC This plugin provides stubs generation for grpcio generated code. + ``` protoc \ --python_out=output/location \ @@ -212,21 +240,23 @@ protoc \ ``` Note that generated code for grpc will work only together with code for python and locations should be the same. -If you need stubs for grpc internal code we suggest using this package https://github.com/shabbyrobe/grpc-stubs +If you need stubs for grpc internal code we suggest using this package https://github.com/shabbyrobe/grpc-stubs ### Targeting python2 support mypy-protobuf's drops support for targeting python2 with version 3.0. If you still need python2 support - + ``` python3 -m pip install mypy_protobuf==2.10 protoc --python_out=output/location --mypy_out=output/location mypy --target-version=2.7 {files} ``` - ## Contributing + Contributions to the implementation are welcome. Please run tests using `./run_test.sh`. Ensure code is formatted using black. + ``` pip3 install black black . @@ -272,8 +302,7 @@ black . - [@Alphadelta14](https://github.com/Alphadelta14) - [@fergyfresh](https://github.com/fergyfresh) -Licence etc. ------------- +## Licence etc. 1. License: Apache 2.0. 2. Copyright attribution: Copyright (c) 2022 Nipunn Koorapati diff --git a/mypy_protobuf/extensions_pb2.py b/mypy_protobuf/extensions_pb2.py index 301e7f0d..a638958a 100644 --- a/mypy_protobuf/extensions_pb2.py +++ b/mypy_protobuf/extensions_pb2.py @@ -2,10 +2,10 @@ # Generated by the protocol buffer compiler. DO NOT EDIT! # source: mypy_protobuf/extensions.proto """Generated protocol buffer code.""" -from google.protobuf.internal import builder as _builder from google.protobuf import descriptor as _descriptor from google.protobuf import descriptor_pool as _descriptor_pool from google.protobuf import symbol_database as _symbol_database +from google.protobuf.internal import builder as _builder # @@protoc_insertion_point(imports) _sym_db = _symbol_database.Default() @@ -16,8 +16,9 @@ DESCRIPTOR = _descriptor_pool.Default().AddSerializedFile(b'\n\x1emypy_protobuf/extensions.proto\x12\rmypy_protobuf\x1a google/protobuf/descriptor.proto\"D\n\x0c\x46ieldOptions\x12\x10\n\x08\x63\x61sttype\x18\x01 \x01(\t\x12\x0f\n\x07keytype\x18\x02 \x01(\t\x12\x11\n\tvaluetype\x18\x03 \x01(\t:L\n\x07options\x12\x1d.google.protobuf.FieldOptions\x18\x82\t \x01(\x0b\x32\x1b.mypy_protobuf.FieldOptions:4\n\x08\x63\x61sttype\x12\x1d.google.protobuf.FieldOptions\x18\xff\x08 \x01(\tB\x02\x18\x01:3\n\x07keytype\x12\x1d.google.protobuf.FieldOptions\x18\x80\t \x01(\tB\x02\x18\x01:5\n\tvaluetype\x12\x1d.google.protobuf.FieldOptions\x18\x81\t \x01(\tB\x02\x18\x01\x62\x06proto3') -_builder.BuildMessageAndEnumDescriptors(DESCRIPTOR, globals()) -_builder.BuildTopDescriptorsAndMessages(DESCRIPTOR, 'mypy_protobuf.extensions_pb2', globals()) +_globals = globals() +_builder.BuildMessageAndEnumDescriptors(DESCRIPTOR, _globals) +_builder.BuildTopDescriptorsAndMessages(DESCRIPTOR, 'mypy_protobuf.extensions_pb2', _globals) if _descriptor._USE_C_DESCRIPTORS == False: google_dot_protobuf_dot_descriptor__pb2.FieldOptions.RegisterExtension(options) google_dot_protobuf_dot_descriptor__pb2.FieldOptions.RegisterExtension(casttype) @@ -31,6 +32,6 @@ keytype._serialized_options = b'\030\001' valuetype._options = None valuetype._serialized_options = b'\030\001' - _FIELDOPTIONS._serialized_start=83 - _FIELDOPTIONS._serialized_end=151 + _globals['_FIELDOPTIONS']._serialized_start=83 + _globals['_FIELDOPTIONS']._serialized_end=151 # @@protoc_insertion_point(module_scope) diff --git a/mypy_protobuf/main.py b/mypy_protobuf/main.py index 02f1bda2..5d2cc9bb 100644 --- a/mypy_protobuf/main.py +++ b/mypy_protobuf/main.py @@ -346,12 +346,11 @@ def write_enums( wl("V: {} = ValueType", self._import("typing_extensions", "TypeAlias")) wl("") wl( - "class {}({}[{}], {}):{}", + "class {}({}[{}], {}):", etw_helper_class, self._import("google.protobuf.internal.enum_type_wrapper", "_EnumTypeWrapper"), value_type_helper_fq, self._builtin("type"), - " # noqa: F821" if prefix else "", ) with self._indent(): ed = self._import("google.protobuf.descriptor", "EnumDescriptor") @@ -460,8 +459,7 @@ def write_messages( wl("def __init__(") with self._indent(): if any(f.name == "self" for f in desc.field): - wl("# pyright: reportSelfClsParameterName=false") - wl("self_,") + wl("self_, # pyright: ignore[reportSelfClsParameterName]") else: wl("self,") with self._indent(): @@ -577,7 +575,7 @@ def write_methods( wl("@{}", self._import("abc", "abstractmethod")) wl(f"def {method.name}(") with self._indent(): - wl(f"inst: {class_name},") + wl(f"inst: {class_name}, # pyright: ignore[reportSelfClsParameterName]") wl( "rpc_controller: {},", self._import("google.protobuf.service", "RpcController"), diff --git a/mypy_requirements.txt b/mypy_requirements.txt index 9005eee8..d59a3261 100644 --- a/mypy_requirements.txt +++ b/mypy_requirements.txt @@ -1,2 +1,2 @@ # Requirements to run mypy itself. Mypy executable exists in a separate venv. -mypy==0.991 +mypy==1.4.1 diff --git a/run_test.sh b/run_test.sh index 5c6d32cd..ba677baa 100755 --- a/run_test.sh +++ b/run_test.sh @@ -33,7 +33,7 @@ if [[ -z $SKIP_CLEAN ]] || [[ ! -e $PROTOC_DIR ]]; then unzip "$PROTOC_DIR/$PROTOC_FILENAME" -d "$PROTOC_DIR/protoc_install" fi PROTOC="$PROTOC_DIR/protoc_install/bin/protoc" -if [[ $($PROTOC --version) != "libprotoc 3.$PROTOBUF_VERSION" ]]; then +if [[ $($PROTOC --version) != "libprotoc $PROTOBUF_VERSION" ]]; then echo -e "${RED}Wrong protoc installed?" exit 1 fi diff --git a/test/generated/testproto/nested/nested_pb2.pyi b/test/generated/testproto/nested/nested_pb2.pyi index 4670a335..bc83b8e1 100644 --- a/test/generated/testproto/nested/nested_pb2.pyi +++ b/test/generated/testproto/nested/nested_pb2.pyi @@ -40,7 +40,7 @@ class AnotherNested(google.protobuf.message.Message): ValueType = typing.NewType("ValueType", builtins.int) V: typing_extensions.TypeAlias = ValueType - class _NestedEnumEnumTypeWrapper(google.protobuf.internal.enum_type_wrapper._EnumTypeWrapper[AnotherNested._NestedEnum.ValueType], builtins.type): # noqa: F821 + class _NestedEnumEnumTypeWrapper(google.protobuf.internal.enum_type_wrapper._EnumTypeWrapper[AnotherNested._NestedEnum.ValueType], builtins.type): DESCRIPTOR: google.protobuf.descriptor.EnumDescriptor INVALID: AnotherNested._NestedEnum.ValueType # 0 ONE: AnotherNested._NestedEnum.ValueType # 1 @@ -59,7 +59,7 @@ class AnotherNested(google.protobuf.message.Message): ValueType = typing.NewType("ValueType", builtins.int) V: typing_extensions.TypeAlias = ValueType - class _NestedEnum2EnumTypeWrapper(google.protobuf.internal.enum_type_wrapper._EnumTypeWrapper[AnotherNested.NestedMessage._NestedEnum2.ValueType], builtins.type): # noqa: F821 + class _NestedEnum2EnumTypeWrapper(google.protobuf.internal.enum_type_wrapper._EnumTypeWrapper[AnotherNested.NestedMessage._NestedEnum2.ValueType], builtins.type): DESCRIPTOR: google.protobuf.descriptor.EnumDescriptor UNDEFINED: AnotherNested.NestedMessage._NestedEnum2.ValueType # 0 NESTED_ENUM1: AnotherNested.NestedMessage._NestedEnum2.ValueType # 1 diff --git a/test/generated/testproto/test3_pb2.pyi b/test/generated/testproto/test3_pb2.pyi index 9b7d14b3..9571d1ad 100644 --- a/test/generated/testproto/test3_pb2.pyi +++ b/test/generated/testproto/test3_pb2.pyi @@ -59,7 +59,7 @@ class SimpleProto3(google.protobuf.message.Message): ValueType = typing.NewType("ValueType", builtins.int) V: typing_extensions.TypeAlias = ValueType - class _InnerEnumEnumTypeWrapper(google.protobuf.internal.enum_type_wrapper._EnumTypeWrapper[SimpleProto3._InnerEnum.ValueType], builtins.type): # noqa: F821 + class _InnerEnumEnumTypeWrapper(google.protobuf.internal.enum_type_wrapper._EnumTypeWrapper[SimpleProto3._InnerEnum.ValueType], builtins.type): DESCRIPTOR: google.protobuf.descriptor.EnumDescriptor INNER1: SimpleProto3._InnerEnum.ValueType # 0 INNER2: SimpleProto3._InnerEnum.ValueType # 1 diff --git a/test/generated/testproto/test_pb2.pyi b/test/generated/testproto/test_pb2.pyi index 86d9ade8..673320db 100644 --- a/test/generated/testproto/test_pb2.pyi +++ b/test/generated/testproto/test_pb2.pyi @@ -78,7 +78,7 @@ class Simple1(google.protobuf.message.Message): ValueType = typing.NewType("ValueType", builtins.int) V: typing_extensions.TypeAlias = ValueType - class _InnerEnumEnumTypeWrapper(google.protobuf.internal.enum_type_wrapper._EnumTypeWrapper[Simple1._InnerEnum.ValueType], builtins.type): # noqa: F821 + class _InnerEnumEnumTypeWrapper(google.protobuf.internal.enum_type_wrapper._EnumTypeWrapper[Simple1._InnerEnum.ValueType], builtins.type): DESCRIPTOR: google.protobuf.descriptor.EnumDescriptor INNER1: Simple1._InnerEnum.ValueType # 1 """INNER1""" @@ -285,7 +285,7 @@ class PythonReservedKeywords(google.protobuf.message.Message): ValueType = typing.NewType("ValueType", builtins.int) V: typing_extensions.TypeAlias = ValueType - class _finallyEnumTypeWrapper(google.protobuf.internal.enum_type_wrapper._EnumTypeWrapper[PythonReservedKeywords._finally.ValueType], builtins.type): # noqa: F821 + class _finallyEnumTypeWrapper(google.protobuf.internal.enum_type_wrapper._EnumTypeWrapper[PythonReservedKeywords._finally.ValueType], builtins.type): DESCRIPTOR: google.protobuf.descriptor.EnumDescriptor valid_in_finally: PythonReservedKeywords._finally.ValueType # 2 @@ -375,8 +375,7 @@ class SelfField(google.protobuf.message.Message): self: builtins.int """Field self -> must generate an __init__ method w/ different name""" def __init__( - # pyright: reportSelfClsParameterName=false - self_, + self_, # pyright: ignore[reportSelfClsParameterName] *, self: builtins.int | None = ..., ) -> None: ... @@ -391,7 +390,7 @@ class PythonReservedKeywordsService(google.protobuf.service.Service, metaclass=a DESCRIPTOR: google.protobuf.descriptor.ServiceDescriptor @abc.abstractmethod def valid_method_name1( - inst: PythonReservedKeywordsService, + inst: PythonReservedKeywordsService, # pyright: ignore[reportSelfClsParameterName] rpc_controller: google.protobuf.service.RpcController, request: global___Simple1, callback: collections.abc.Callable[[global____r_None], None] | None, @@ -399,7 +398,7 @@ class PythonReservedKeywordsService(google.protobuf.service.Service, metaclass=a """valid_method_name1""" @abc.abstractmethod def valid_method_name2( - inst: PythonReservedKeywordsService, + inst: PythonReservedKeywordsService, # pyright: ignore[reportSelfClsParameterName] rpc_controller: google.protobuf.service.RpcController, request: global___Simple1, callback: collections.abc.Callable[[global___PythonReservedKeywords._r_lambda], None] | None, @@ -412,14 +411,14 @@ class PythonReservedKeywordsService_Stub(PythonReservedKeywordsService): def __init__(self, rpc_channel: google.protobuf.service.RpcChannel) -> None: ... DESCRIPTOR: google.protobuf.descriptor.ServiceDescriptor def valid_method_name1( - inst: PythonReservedKeywordsService_Stub, + inst: PythonReservedKeywordsService_Stub, # pyright: ignore[reportSelfClsParameterName] rpc_controller: google.protobuf.service.RpcController, request: global___Simple1, callback: collections.abc.Callable[[global____r_None], None] | None = ..., ) -> concurrent.futures.Future[global____r_None]: """valid_method_name1""" def valid_method_name2( - inst: PythonReservedKeywordsService_Stub, + inst: PythonReservedKeywordsService_Stub, # pyright: ignore[reportSelfClsParameterName] rpc_controller: google.protobuf.service.RpcController, request: global___Simple1, callback: collections.abc.Callable[[global___PythonReservedKeywords._r_lambda], None] | None = ..., @@ -430,7 +429,7 @@ class ATestService(google.protobuf.service.Service, metaclass=abc.ABCMeta): DESCRIPTOR: google.protobuf.descriptor.ServiceDescriptor @abc.abstractmethod def Echo( - inst: ATestService, + inst: ATestService, # pyright: ignore[reportSelfClsParameterName] rpc_controller: google.protobuf.service.RpcController, request: global___Simple1, callback: collections.abc.Callable[[global___Simple2], None] | None, @@ -440,7 +439,7 @@ class ATestService_Stub(ATestService): def __init__(self, rpc_channel: google.protobuf.service.RpcChannel) -> None: ... DESCRIPTOR: google.protobuf.descriptor.ServiceDescriptor def Echo( - inst: ATestService_Stub, + inst: ATestService_Stub, # pyright: ignore[reportSelfClsParameterName] rpc_controller: google.protobuf.service.RpcController, request: global___Simple1, callback: collections.abc.Callable[[global___Simple2], None] | None = ..., diff --git a/test/test_generated_mypy.py b/test/test_generated_mypy.py index c0f2b1ee..13d393b4 100644 --- a/test/test_generated_mypy.py +++ b/test/test_generated_mypy.py @@ -113,7 +113,7 @@ def grab_expectations(filename: str, marker: str) -> Generator[Tuple[str, int], assert errors_38 == expected_errors_38 # Some sanity checks to make sure we don't mess this up. Please update as necessary. - assert len(errors_38) == 74 + assert len(errors_38) == 77 def test_func() -> None: diff --git a/test_negative/negative.py b/test_negative/negative.py index be697849..1923cfa2 100644 --- a/test_negative/negative.py +++ b/test_negative/negative.py @@ -252,7 +252,7 @@ def StreamStream( class BadServicer(DummyServiceServicer): def UnaryUnary( # E:3.8 self, - request: Iterator[DummyRequest], + request: Iterator[DummyRequest], # E:3.8 context: grpc.ServicerContext, ) -> Iterator[DummyReply]: for _data in request: @@ -260,7 +260,7 @@ def UnaryUnary( # E:3.8 def UnaryStream( # E:3.8 self, - request: Iterator[DummyRequest], + request: Iterator[DummyRequest], # E:3.8 context: grpc.ServicerContext, ) -> DummyReply: for _data in request: @@ -269,7 +269,7 @@ def UnaryStream( # E:3.8 def StreamUnary( # E:3.8 self, - request: DummyRequest, + request: DummyRequest, # E:3.8 context: grpc.ServicerContext, ) -> Iterator[DummyReply]: yield DummyReply() diff --git a/test_negative/output.expected.3.8 b/test_negative/output.expected.3.8 index 9cf4d93c..29a5eac8 100644 --- a/test_negative/output.expected.3.8 +++ b/test_negative/output.expected.3.8 @@ -47,14 +47,14 @@ test_negative/negative.py:135: error: Argument 1 to "Name" of "_EnumTypeWrapper" test_negative/negative.py:139: error: "ScalarMap[int, str]" has no attribute "get_or_create" [attr-defined] test_negative/negative.py:141: error: No overload variant of "get" of "ScalarMap" matches argument type "str" [call-overload] test_negative/negative.py:141: note: Possible overload variants: -test_negative/negative.py:141: note: def get(self, key: int, default: None = ...) -> str +test_negative/negative.py:141: note: def get(self, key: int, default: None = ...) -> Optional[str] test_negative/negative.py:141: note: def [_T] get(self, key: int, default: Union[str, _T]) -> Union[str, _T] test_negative/negative.py:142: error: No overload variant of "get" of "MessageMap" matches argument type "str" [call-overload] test_negative/negative.py:142: note: Possible overload variants: -test_negative/negative.py:142: note: def get(self, key: int, default: None = ...) -> OuterMessage3 +test_negative/negative.py:142: note: def get(self, key: int, default: None = ...) -> Optional[OuterMessage3] test_negative/negative.py:142: note: def [_T] get(self, key: int, default: Union[OuterMessage3, _T]) -> Union[OuterMessage3, _T] -test_negative/negative.py:145: error: Incompatible types in assignment (expression has type "str", variable has type "int") [assignment] -test_negative/negative.py:146: error: Incompatible types in assignment (expression has type "OuterMessage3", variable has type "int") [assignment] +test_negative/negative.py:145: error: Incompatible types in assignment (expression has type "Optional[str]", variable has type "int") [assignment] +test_negative/negative.py:146: error: Incompatible types in assignment (expression has type "Optional[OuterMessage3]", variable has type "int") [assignment] test_negative/negative.py:148: error: Dict entry 0 has incompatible type "str": "int"; expected "int": "str" [dict-item] test_negative/negative.py:148: error: Dict entry 0 has incompatible type "str": "str"; expected "int": "OuterMessage3" [dict-item] test_negative/negative.py:152: error: Incompatible types in assignment (expression has type "int", variable has type "UserId") [assignment] @@ -82,16 +82,16 @@ test_negative/negative.py:209: error: "DummyReply" has no attribute "__iter__" ( test_negative/negative.py:209: error: Argument 1 to "__call__" of "StreamUnaryMultiCallable" has incompatible type "DummyRequest"; expected "Iterator[DummyRequest]" [arg-type] test_negative/negative.py:212: error: Argument 1 to "__call__" of "StreamStreamMultiCallable" has incompatible type "DummyRequest"; expected "Iterator[DummyRequest]" [arg-type] test_negative/negative.py:216: error: "DummyReply" has no attribute "not_exists" [attr-defined] -test_negative/negative.py:253: error: Argument 1 of "UnaryUnary" is incompatible with supertype "DummyServiceServicer"; supertype defines the argument type as "DummyRequest" [override] -test_negative/negative.py:253: note: This violates the Liskov substitution principle -test_negative/negative.py:253: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides test_negative/negative.py:253: error: Return type "Iterator[DummyReply]" of "UnaryUnary" incompatible with return type "Union[DummyReply, Awaitable[DummyReply]]" in supertype "DummyServiceServicer" [override] -test_negative/negative.py:261: error: Argument 1 of "UnaryStream" is incompatible with supertype "DummyServiceServicer"; supertype defines the argument type as "DummyRequest" [override] -test_negative/negative.py:261: note: This violates the Liskov substitution principle -test_negative/negative.py:261: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides +test_negative/negative.py:255: error: Argument 1 of "UnaryUnary" is incompatible with supertype "DummyServiceServicer"; supertype defines the argument type as "DummyRequest" [override] +test_negative/negative.py:255: note: This violates the Liskov substitution principle +test_negative/negative.py:255: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides test_negative/negative.py:261: error: Return type "DummyReply" of "UnaryStream" incompatible with return type "Union[Iterator[DummyReply], AsyncIterator[DummyReply]]" in supertype "DummyServiceServicer" [override] -test_negative/negative.py:270: error: Argument 1 of "StreamUnary" is incompatible with supertype "DummyServiceServicer"; supertype defines the argument type as "_MaybeAsyncIterator[DummyRequest]" [override] -test_negative/negative.py:270: note: This violates the Liskov substitution principle -test_negative/negative.py:270: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides +test_negative/negative.py:263: error: Argument 1 of "UnaryStream" is incompatible with supertype "DummyServiceServicer"; supertype defines the argument type as "DummyRequest" [override] +test_negative/negative.py:263: note: This violates the Liskov substitution principle +test_negative/negative.py:263: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides test_negative/negative.py:270: error: Return type "Iterator[DummyReply]" of "StreamUnary" incompatible with return type "Union[DummyReply, Awaitable[DummyReply]]" in supertype "DummyServiceServicer" [override] +test_negative/negative.py:272: error: Argument 1 of "StreamUnary" is incompatible with supertype "DummyServiceServicer"; supertype defines the argument type as "_MaybeAsyncIterator[DummyRequest]" [override] +test_negative/negative.py:272: note: This violates the Liskov substitution principle +test_negative/negative.py:272: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides Found 80 errors in 1 file (checked 4 source files) diff --git a/test_negative/output.expected.3.8.omit_linenos b/test_negative/output.expected.3.8.omit_linenos index 2d17a89a..5df0d985 100644 --- a/test_negative/output.expected.3.8.omit_linenos +++ b/test_negative/output.expected.3.8.omit_linenos @@ -47,14 +47,14 @@ test_negative/negative.py: error: Argument 1 to "Name" of "_EnumTypeWrapper" has test_negative/negative.py: error: "ScalarMap[int, str]" has no attribute "get_or_create" [attr-defined] test_negative/negative.py: error: No overload variant of "get" of "ScalarMap" matches argument type "str" [call-overload] test_negative/negative.py: note: Possible overload variants: -test_negative/negative.py: note: def get(self, key: int, default: None = ...) -> str +test_negative/negative.py: note: def get(self, key: int, default: None = ...) -> Optional[str] test_negative/negative.py: note: def [_T] get(self, key: int, default: Union[str, _T]) -> Union[str, _T] test_negative/negative.py: error: No overload variant of "get" of "MessageMap" matches argument type "str" [call-overload] test_negative/negative.py: note: Possible overload variants: -test_negative/negative.py: note: def get(self, key: int, default: None = ...) -> OuterMessage3 +test_negative/negative.py: note: def get(self, key: int, default: None = ...) -> Optional[OuterMessage3] test_negative/negative.py: note: def [_T] get(self, key: int, default: Union[OuterMessage3, _T]) -> Union[OuterMessage3, _T] -test_negative/negative.py: error: Incompatible types in assignment (expression has type "str", variable has type "int") [assignment] -test_negative/negative.py: error: Incompatible types in assignment (expression has type "OuterMessage3", variable has type "int") [assignment] +test_negative/negative.py: error: Incompatible types in assignment (expression has type "Optional[str]", variable has type "int") [assignment] +test_negative/negative.py: error: Incompatible types in assignment (expression has type "Optional[OuterMessage3]", variable has type "int") [assignment] test_negative/negative.py: error: Dict entry 0 has incompatible type "str": "int"; expected "int": "str" [dict-item] test_negative/negative.py: error: Dict entry 0 has incompatible type "str": "str"; expected "int": "OuterMessage3" [dict-item] test_negative/negative.py: error: Incompatible types in assignment (expression has type "int", variable has type "UserId") [assignment] @@ -82,16 +82,16 @@ test_negative/negative.py: error: "DummyReply" has no attribute "__iter__" (not test_negative/negative.py: error: Argument 1 to "__call__" of "StreamUnaryMultiCallable" has incompatible type "DummyRequest"; expected "Iterator[DummyRequest]" [arg-type] test_negative/negative.py: error: Argument 1 to "__call__" of "StreamStreamMultiCallable" has incompatible type "DummyRequest"; expected "Iterator[DummyRequest]" [arg-type] test_negative/negative.py: error: "DummyReply" has no attribute "not_exists" [attr-defined] +test_negative/negative.py: error: Return type "Iterator[DummyReply]" of "UnaryUnary" incompatible with return type "Union[DummyReply, Awaitable[DummyReply]]" in supertype "DummyServiceServicer" [override] test_negative/negative.py: error: Argument 1 of "UnaryUnary" is incompatible with supertype "DummyServiceServicer"; supertype defines the argument type as "DummyRequest" [override] test_negative/negative.py: note: This violates the Liskov substitution principle test_negative/negative.py: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides -test_negative/negative.py: error: Return type "Iterator[DummyReply]" of "UnaryUnary" incompatible with return type "Union[DummyReply, Awaitable[DummyReply]]" in supertype "DummyServiceServicer" [override] +test_negative/negative.py: error: Return type "DummyReply" of "UnaryStream" incompatible with return type "Union[Iterator[DummyReply], AsyncIterator[DummyReply]]" in supertype "DummyServiceServicer" [override] test_negative/negative.py: error: Argument 1 of "UnaryStream" is incompatible with supertype "DummyServiceServicer"; supertype defines the argument type as "DummyRequest" [override] test_negative/negative.py: note: This violates the Liskov substitution principle test_negative/negative.py: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides -test_negative/negative.py: error: Return type "DummyReply" of "UnaryStream" incompatible with return type "Union[Iterator[DummyReply], AsyncIterator[DummyReply]]" in supertype "DummyServiceServicer" [override] +test_negative/negative.py: error: Return type "Iterator[DummyReply]" of "StreamUnary" incompatible with return type "Union[DummyReply, Awaitable[DummyReply]]" in supertype "DummyServiceServicer" [override] test_negative/negative.py: error: Argument 1 of "StreamUnary" is incompatible with supertype "DummyServiceServicer"; supertype defines the argument type as "_MaybeAsyncIterator[DummyRequest]" [override] test_negative/negative.py: note: This violates the Liskov substitution principle test_negative/negative.py: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#incompatible-overrides -test_negative/negative.py: error: Return type "Iterator[DummyReply]" of "StreamUnary" incompatible with return type "Union[DummyReply, Awaitable[DummyReply]]" in supertype "DummyServiceServicer" [override] Found 80 errors in 1 file (checked 4 source files) diff --git a/test_requirements.txt b/test_requirements.txt index 43f0e5ea..974106c8 100644 --- a/test_requirements.txt +++ b/test_requirements.txt @@ -1,8 +1,8 @@ # Requirements to run unit tests. Tests import from # generated code. -protobuf==4.21.12 +protobuf==4.23.4 pytest==7.3.1 pytest-asyncio==0.20.3 -grpc-stubs==1.24.12.1 -grpcio-tools==1.51.1 -types-protobuf==4.21.0.5 +grpc-stubs==1.53.0.2 +grpcio-tools==1.56.2 +types-protobuf==4.23.0.2