How to add Custom vanity to grpc-gateway

[Tommy-42]

Hello guys,

I am currently using gogoproto with a custom vanity
CustomNameID same one as swarm-kit

What does this CustomNameID? it basically change my_id to myID instead of myId.

so our protoc-gen-custom cmd looks like :

package main

import (
    "github.com/gogo/protobuf/protoc-gen-gogo/descriptor"
    "github.com/gogo/protobuf/vanity"
    "github.com/gogo/protobuf/vanity/command"

    // custom govalidator import
    validator_plugin "myrepo/govalidator/validator"
)

func main() {
    req := command.Read()
    files := req.GetProtoFile()
    files = vanity.FilterFiles(files, vanity.NotGoogleProtobufDescriptorProto)

    req.ProtoFile = files

    for _, opt := range []func(*descriptor.FileDescriptorProto){
        vanity.TurnOffGoGettersAll,
        vanity.TurnOffGoStringerAll,
        vanity.TurnOnMarshalerAll,
        vanity.TurnOnStringerAll,
        vanity.TurnOnUnmarshalerAll,
        vanity.TurnOnSizerAll,
        CustomNameID, // custom added here
    } {
        vanity.ForEachFile(files, opt)
    }

    // generate default models
    resp := command.Generate(req)
    command.Write(resp)
    
    // generate go validator plugin
    resp = command.GeneratePlugin(req, validator_plugin.NewPlugin(), ".validator.pb.go")
    command.Write(resp)
}    

my problem is, I am currently trying to add grpc-gateway to my project but grpc-gateway output field without our CustomNameID.

Does anyone has ever encountered this problem ?

Is there a way to add grpc-gateway to this custom protoc-gen-custom to be able to use vanity on it, like we used with our govalidator ?

Thanks you all

[awalterschulze]

I don't see how adding grpc-gateway to the plugin list is going to solve your problem, but I also think I don't fully understand your problem yet.

Could you show an example snippet of how grpc-gateway outputs the field without the CustomNameID and why that breaks things?

[Tommy-42]

Yes sure,

Output with protoc + our customname we have

`OwnerID:` + fmt.Sprintf("%v", this.OwnerID) + `,`,

With grpc-gateway we have :

`OwnerId:` + fmt.Sprintf("%v", this.OwnerId) + `,`,

I am currently trying to use in my proto

import "github.com/gogo/protobuf/gogoproto/gogo.proto";

message foo {
  string owner_id = 1 [(gogoproto.customname) = "OwnerID"];
}

( without succes yet, still digging what did I forgot again )

[awalterschulze]

Where is this output happening?

[Tommy-42]

for the first one into the pb.go

the second one into the pb.gw.go

[awalterschulze]

I really need more info.
Please help.

I don't have a lot of time to spend on these issues. Please try and make it as easy for me as possible to help you.

[Tommy-42]

Sorry, I have been kind busy since my answer.

Ok so :

I have a proto file with a message, let say :

message Contact {
  string contact_id = 1;
}

If I generate the code using protoc the output will be something like ( pb.go) :

func (this *Contact) String() string {
	if this == nil {
		return "nil"
	}
	s := strings.Join([]string{`&Contact{`,
		`ContactId:` + fmt.Sprintf("%v", this.ContactId) + `,`,
		`}`,
	}, "")
	return s
}

Note that ContactId I uppercase d lowercase

so what we do is :

We have a custom generator :

package main

import (
    "github.com/gogo/protobuf/protoc-gen-gogo/descriptor"
    "github.com/gogo/protobuf/vanity"
    "github.com/gogo/protobuf/vanity/command"

    // custom govalidator import
    validator_plugin "myrepo/govalidator/validator"
)

func main() {
    req := command.Read()
    files := req.GetProtoFile()
    files = vanity.FilterFiles(files, vanity.NotGoogleProtobufDescriptorProto)

    req.ProtoFile = files

    for _, opt := range []func(*descriptor.FileDescriptorProto){
        vanity.TurnOffGoGettersAll,
        vanity.TurnOffGoStringerAll,
        vanity.TurnOnMarshalerAll,
        vanity.TurnOnStringerAll,
        vanity.TurnOnUnmarshalerAll,
        vanity.TurnOnSizerAll,
        CustomNameID, // custom added here
    } {
        vanity.ForEachFile(files, opt)
    }

    // generate default models
    resp := command.Generate(req)
    command.Write(resp)
    
    // generate go validator plugin
    resp = command.GeneratePlugin(req, validator_plugin.NewPlugin(), ".validator.pb.go")
    command.Write(resp)
}   

we added one more custom vanity check here : CustomNameID

Which basically search for *Id and replace them to *ID

It work fine when generating the code ( pb.go) but ...

If we want to use grpc-gateway ( pb.gw.go ) like :
protoc .... --grpc-gateway_out= .....

the vanity check doesn't apply and we have on one hand :

pb.go : ContactID

and on the other hand :

pb.gw.go : ContactId

Resulting in error where the variable are not found.

thanks for your time and your patience

[awalterschulze]

Could you paste a little code showing the output in pb.gw.go where ContactId is referenced?

[Tommy-42]

func request_GetContact_0(ctx context.Context, marshaler runtime.Marshaler, client Client, req *http.Request, pathParams map[string]string) (proto.Message, runtime.ServerMetadata, error) {
	var protoReq GetContactRequest
	var metadata runtime.ServerMetadata

	var (
		val string
		ok  bool
		err error
		_   = err
	)

	val, ok = pathParams["id"]
	if !ok {
		return nil, metadata, status.Errorf(codes.InvalidArgument, "missing parameter %s", "id")
	}

	// HERE
	protoReq.Id, err = runtime.String(val)

	if err != nil {
		return nil, metadata, status.Errorf(codes.InvalidArgument, "type mismatch, parameter: %s, error: %v", "id", err)
	}

	msg, err := client.GetContact(ctx, &protoReq, grpc.Header(&metadata.HeaderMD), grpc.Trailer(&metadata.TrailerMD))
	return msg, metadata, err

}

[awalterschulze]

Ok so that is just Id and not ContactId, but I think I get it now.

I assume swarmkit is not using grpc-gateway?

Could you generate your grpc-gateway as a separate service? without CustomNameID, because I don't think grpc-gateway is going to support that. But then you should still be able to generate a separate service with your normal gRPC code that can interact on a binary level with your gateway using your vanity binary.

Otherwise you could also wait for golang/protobuf#555
which should deprecate gogoproto.customname and should be supported by grpc-gateway in some future.

[Tommy-42]

yes not quite the same but you got the idea :) !( contact_id is no more generated and I couldnt get you the exact output )

No swarm-kit does not use it :( , I think I'll have to wait golang/protobuf#555

I will just remove the vanity filter and go back to Id instead of ID.

If I found something interesting, as a workaround or something. I will try to share it :)

Thanks for the help

[awalterschulze]

How about having your grpc-gateway as a separate service with fieldname Id that communicates over the wire with your grpc service where the fieldname is generated as ID. That should work?

[awalterschulze]

Can I close this?

[Tommy-42]

Yes, sorry for the delay