Buf Generate Error

[prodbyola]

Hi, I just installed buf using Homebrew.

Anytime I try to buf generate, I get a long list of error like below. I have removed previously installed protoc on my PC but I still get the same error.

This is my config:

version: v1beta1

plugins:
  - name: js
    out: /front/src/proto
    opt: import_style=commonjs,binary

  - name: grpc
    out: /front/src/proto
    opt: grpc_js
    path: grpc_tools_node_protocol_plugin

  - name: ts
    out: /front/src/proto
    opt: grpc_js

Thanks for your help.

Error:

front/node_modules/grpc-tools/bin/google/protobuf/descriptor.proto:607:17:symbol "google.protobuf.FieldOptions.lazy" already defined at google/protobuf/descriptor.proto:612:17
front/node_modules/grpc-tools/bin/google/protobuf/descriptor.proto:613:17:symbol "google.protobuf.FieldOptions.deprecated" already defined at google/protobuf/descriptor.proto:623:17
front/node_modules/grpc-tools/bin/google/protobuf/descriptor.proto:616:17:symbol "google.protobuf.FieldOptions.weak" already defined at google/protobuf/descriptor.proto:626:17
front/node_modules/grpc-tools/bin/google/protobuf/descriptor.proto:620:32:symbol "google.protobuf.FieldOptions.uninterpreted_option" already defined at google/protobuf/descriptor.proto:630:32
front/node_modules/grpc-tools/bin/google/protobuf/descriptor.proto:628:9:symbol "google.protobuf.OneofOptions" already defined at google/protobuf/descriptor.proto:638:9
front/node_modules/grpc-tools/bin/google/protobuf/descriptor.proto:630:32:symbol "google.protobuf.OneofOptions.uninterpreted_option" already defined at google/protobuf/descriptor.proto:640:32
front/node_modules/grpc-tools/bin/google/protobuf/descriptor.proto:636:9:symbol "google.protobuf.EnumOptions" already defined at google/protobuf/descriptor.proto:646:9
front/node_modules/grpc-tools/bin/google/protobuf/descriptor.proto:640:17:symbol "google.protobuf.EnumOptions.allow_alias" already defined at google/protobuf/descriptor.proto:650:17
front/node_modules/grpc-tools/bin/google/protobuf/descriptor.proto:646:17:symbol "google.protobuf.EnumOptions.deprecated" already defined at google/protobuf/descriptor.proto:656:17
front/node_modules/grpc-tools/bin/google/protobuf/descriptor.proto:651:32:symbol "google.protobuf.EnumOptions.uninterpreted_option" already defined at google/protobuf/descriptor.proto:661:32
front/node_modules/grpc-tools/bin/google/protobuf/descriptor.proto:657:9:symbol "google.protobuf.EnumValueOptions" already defined at google/protobuf/descriptor.proto:667:9
front/node_modules/grpc-tools/bin/google/protobuf/descriptor.proto:662:17:symbol "google.protobuf.EnumValueOptions.deprecated" already defined at google/protobuf/descriptor.proto:672:17
front/node_modules/grpc-tools/bin/google/protobuf/descriptor.proto:665:32:symbol "google.protobuf.EnumValueOptions.uninterpreted_option" already defined at google/protobuf/descriptor.proto:675:32
front/node_modules/grpc-tools/bin/google/protobuf/descriptor.proto:671:9:symbol "google.protobuf.ServiceOptions" already defined at google/protobuf/descriptor.proto:681:9
front/node_modules/grpc-tools/bin/google/protobuf/descriptor.proto:682:17:symbol "google.protobuf.ServiceOptions.deprecated" already defined at google/protobuf/descriptor.proto:692:17
front/node_modules/grpc-tools/bin/google/protobuf/descriptor.proto:685:32:symbol "google.protobuf.ServiceOptions.uninterpreted_option" already defined at google/protobuf/descriptor.proto:695:32
front/node_modules/grpc-tools/bin/google/protobuf/descriptor.proto:691:9:symbol "google.protobuf.MethodOptions" already defined at google/protobuf/descriptor.proto:701:9
front/node_modules/grpc-tools/bin/google/protobuf/descriptor.proto:702:17:symbol "google.protobuf.MethodOptions.deprecated" already defined at google/protobuf/descriptor.proto:712:17
front/node_modules/grpc-tools/bin/google/protobuf/descriptor.proto:707:8:symbol "google.protobuf.MethodOptions.IdempotencyLevel" already defined at google/protobuf/descriptor.proto:717:8
front/node_modules/grpc-tools/bin/google/protobuf/descriptor.proto:708:5:symbol "google.protobuf.MethodOptions.IDEMPOTENCY_UNKNOWN" already defined at google/protobuf/descriptor.proto:718:5; protobuf uses C++ scoping rules for enum values, so they exist in the scope enclosing the enum

[timostamm]

Hey @prodbyola, if you run any of the buf commands without specifying an input explicitly, buf takes the current working directory as an input, and recursively looks up proto files.

But the npm package grpc-tools ships with its own definitions of the well-known types, and because buf - as any protobuf compiler - already includes the well-known types, you see the error of a duplicate definition.

You have two options to avoid this issue: 1. use the "excludes" option:

# buf.yaml file at the root, next to your buf.gen.yaml
version: v1
build:
  excludes: 
    - front/node_modules

Or 2. - move your proto files into a proto directory, and run buf generate proto instead of buf generate.

Let us know if that resolves the issue for you.

[prodbyola]

Hi @timostamm,
Thanks for the help! The second option did the trick for me. I really appreciate.

Although I'm facing another issue now but I don't think it's related to bufbuild itself. I guess it might be protoc-gen-ts plugin and Typescript conflict.

This is the error I'm getting now.

DeprecationWarning: 'createPropertyDeclaration' has been deprecated since v4.8.0. Decorators have been combined with modifiers. Callers should switch to an overload that does not accept a 'decorators' parameter.
DeprecationWarning: 'createParameterDeclaration' has been deprecated since v4.8.0. Decorators have been combined with modifiers. Callers should switch to an overload that does not accept a 'decorators' parameter.
DeprecationWarning: 'createConstructorDeclaration' has been deprecated since v4.8.0. Decorators are no longer supported for this function. Callers should switch to an overload that does not accept a 'decorators' parameter.
DeprecationWarning: 'createGetAccessorDeclaration' has been deprecated since v4.8.0. Decorators have been combined with modifiers. Callers should switch to an overload that does not accept a 'decorators' parameter.
DeprecationWarning: 'createSetAccessorDeclaration' has been deprecated since v4.8.0. Decorators have been combined with modifiers. Callers should switch to an overload that does not accept a 'decorators' parameter.
DeprecationWarning: 'createMethodDeclaration' has been deprecated since v4.8.0. Decorators have been combined with modifiers. Callers should switch to an overload that does not accept a 'decorators' parameter.
DeprecationWarning: 'createClassDeclaration' has been deprecated since v4.8.0. Decorators have been combined with modifiers. Callers should switch to an overload that does not accept a 'decorators' parameter.
DeprecationWarning: 'createImportDeclaration' has been deprecated since v4.8.0. Decorators are no longer supported for this function. Callers should switch to an overload that does not accept a 'decorators' parameter.
/home/prodbyola/grpc-tutorials/front/node_modules/typescript/lib/typescript.js:172134
            throw new TypeError("Invalid arguments");
            ^

TypeError: Invalid arguments
    at Object.createParameterDeclaration (/home/prodbyola/grpc-tutorials/front/node_modules/typescript/lib/typescript.js:172134:19)
    at createParameter (/home/prodbyola/grpc-tutorials/front/node_modules/protoc-gen-ts/src/rpc.js:154:23)
    at Object.createGrpcInterfaceType (/home/prodbyola/grpc-tutorials/front/node_modules/protoc-gen-ts/src/rpc.js:161:31)
    at Object.<anonymous> (/home/prodbyola/grpc-tutorials/front/node_modules/protoc-gen-ts/src/index.js:55:32)
    at Module._compile (node:internal/modules/cjs/loader:1159:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1213:10)
    at Module.load (node:internal/modules/cjs/loader:1037:32)
    at Module._load (node:internal/modules/cjs/loader:878:12)
    at Module.require (node:internal/modules/cjs/loader:1061:19)
    at require (node:internal/modules/cjs/helpers:103:18)

Node.js v18.12.1
Failure: plugin ts: exit status 1

[prodbyola]

The second is an issue with protoc-gen-ts thesayyn/protoc-gen-ts#170.

Thanks for your help @timostamm.

[timostamm]

That's correct, @prodbyola. I'm going to close this issue, because there is nothing we can do about this issue in protoc-gen-ts.

If you are looking for a protobuf implementation in TypeScript, you might also want to take a look at our protobuf-es. Similar to protoc-gen-ts, it provides protobuf fields as plain properties, but it also passes the conformance tests, implements the JSON format correctly, and comes with a simple framework to write your own plugins.

We don't have a package yet to use it with @grpc/grpc-js, but you have two options for this use case:

• Use the code snippets mentioned in Outputting service for @grpc/grpc-js protobuf-es#252
• Use our @bufbuild/connect-node, which implements gRPC

[prodbyola]

That's correct, @prodbyola. I'm going to close this issue, because there is nothing we can do about this issue in protoc-gen-ts.

If you are looking for a protobuf implementation in TypeScript, you might also want to take a look at our protobuf-es. Similar to protoc-gen-ts, it provides protobuf fields as plain properties, but it also passes the conformance tests, implements the JSON format correctly, and comes with a simple framework to write your own plugins.

We don't have a package yet to use it with @grpc/grpc-js, but you have two options for this use case:

• Use the code snippets mentioned in Outputting service for @grpc/grpc-js protobuf-es#252
• Use our @bufbuild/connect-node, which implements gRPC

Hi @timostamm,
Thanks for the help so far. As I've been encountering some issues using @grpc/grpc-js, I decided to go with the second option @bufbuild/connect-node. However, I encounter this error in the console. I can't really tell what's the cause.

node_modules/@bufbuild/connect-node/dist/esm/compression.js:15:9: ERROR: No matching export in "browser-external:util" for import "promisify"

Thanks for your help.
NOTE: I'm trying to add this to a vue (quasar-cli/vite in SSR mode) project.

[timostamm]

@prodbyola, this looks like an issue with vite trying to bundle Node.js code for the browser. @grpc/grpc-js is also using Node.js code, so I wouldn't expect an issue here from using @bufbuild/connect-node instead of @grpc/grpc-js.

Are you actually using @grpc/grpc-js? I just assumed you are doing that because of the grpc-tools package in the original issue description. If you think there is a problem, can I ask you to open an issue on the connect repository?

You could also start a discussion there, or in our Slack channel. But it's likely that you need to provide a bit more information about what servers you want to call.