bug: URL Query parameter not bind to Optional proto message field

[afifurrohman-id]

🐛 Bug Report

When using optional field as request message and will be expected to bind with HTTP url query parameter the optional field always nil

To Reproduce

• buf.gen.yaml

version: v2
plugins:
  - local: protoc-gen-go
    out: .
    opt: paths=source_relative
  - local: protoc-gen-go-grpc
    out: .
    opt: paths=source_relative
  - local: protoc-gen-grpc-gateway
    out: .
    opt: paths=source_relative

inputs:
  - directory: .

• buf.yaml

version: v2
name: example.com/foo
deps:
  - buf.build/googleapis/googleapis

• svc.proto

syntax = "proto3";

import "google/api/annotations.proto";
import "google/protobuf/empty.proto";

message DeleteFooRequest {
    string foo_id = 1;
    optional bool soft_delete = 2;
}

service ProductService {

    rpc DeleteFoo(DeleteFooRequest) returns (google.protobuf.Empty) {
        option (google.api.http) = {
            delete: "/foo/{foo_id}"
        };
    }

}

• Generate stub

buf generate

• Create normal grpc gateway server
• Test using curl with url?soft_delete=true or url?soft_delete=false

Expected behavior

it should either false or true

Actual Behavior

Soft delete it will always nil

Your Environment

Linux GRPC-GATEWAY 5.15.153.1-microsoft-standard-WSL2 #1 SMP Fri Mar 29 23:14:13 UTC 2024 x86_64 GNU/Linux

[johanbrandhorst]

Hi. Thanks for the example. Does it work if you don't use optional?

[afifurrohman-id]

Hi. Thanks for the example. Does it work if you don't use optional?

Yes, it works as expected

[johanbrandhorst]

Ah, very cool! We haven't yet put time into supporting the optional keyword, so this sounds like a bug. The code you want to look at is in

grpc-gateway/runtime/query.go

Lines 48 to 65 in ffc4120

	func (*DefaultQueryParser) Parse(msg proto.Message, values url.Values, filter *utilities.DoubleArray) error {
	for key, values := range values {
	if match := valuesKeyRegexp.FindStringSubmatch(key); len(match) == 3 {
	key = match[1]
	values = append([]string{match[2]}, values...)
	}
	msgValue := msg.ProtoReflect()
	fieldPath := normalizeFieldPath(msgValue, strings.Split(key, "."))
	if filter.HasCommonPrefix(fieldPath) {
	continue
	}
	if err := populateFieldValueFromPath(msgValue, fieldPath, values); err != nil {
	return err
	}
	}
	return nil
	}

. I would start by adding a failing test case to https://github.com/grpc-ecosystem/grpc-gateway/blob/main/runtime/query_test.go. Contributions accepted!

[lemonnn-8]

@johanbrandhorst if no one is working on this I can take this up :)

[johanbrandhorst]

Please go for it!

[lemonnn-8]

Hey @johanbrandhorst @afifurrohman-id

So I did some investigation and created a demo as per as bug reported. But it was working fine for me. I am attaching repo, so you guys can let me know if something is wrong from my side.

https://github.com/lemonnn-8/grpc-gateway-demo

looking forward to your reply :)

[afifurrohman-id]

So I did some investigation and created a demo as per as bug reported. But it was working fine for me. I am attaching repo, so you guys can let me know if something is wrong from my side.

I can confirm now this working, i also re-generate the stubs output.
But, why it works, when grpc-gateway itself doesn't put a release and i didn't find update in googleapis that fixed specific http.proto ?

i can see that buf registry update the googleapis dep: https://buf.build/googleapis/googleapis/commits/commit/e7f8d366f5264595bcc4cd4139af9973 but is not related to http.proto

but i think googleapis http.proto is not the problem, the problem is the parser itself.

[lemonnn-8]

@johanbrandhorst I think we can close this issue then

[afifurrohman-id]

Update, after long debugging i found where the problem comes.

The gateway and service is separate program, and i forgot to restart the gateway after re-generate the stubs that used by service and gateway.