Using google.protobuf.Empty as response results in a bad gRPC response

[ArcaneTSGK]

When writing a response builder with google.protobuf.Empty the mock gRPC client will return StatusCode=Unimplemented.

Expected behavior:

It should be possible to use googles 'WellKnown' types as the protobuf body in the response builder.

Test to reproduce

1. Proto file greet.proto

syntax = "proto3"

import "google/protobuf/empty.proto";

package organization.greet.api.v1;

message HelloRequest {
  string name = 1;
}

service Greeter {
  rpc SayHello (HelloRequest) returns (google.protobuf.Empty);
}

2. The test

[Fact]
public async Task SayHello_ShouldReturnEmptyResponse_WhenInvoked()
{
  using var server = WireMockServer.Start(new WireMockServerSettings
  {
    Urls = ["grpcs://localhost:5000"],
    UseHttp2 = true
  });

  var protoFile = await File.ReadAllTextAsync(@"C:\projects\GrpcIntegrationTests\GrpcIntegrationTests\Protos\greet.proto");

  var protoDefinitionid = "GrpcGreet";

  server.AddProtoDefinition(protoDefinitionId, protoFile)
    .Given(Request.Create()
      .UsingPost()
      .WithHttpVersion("2")
      .WithPath("/organization.greet.api.v1.Greeter/SayHello")
      .WithBodyAsProtoBud("organization.greet.api.v1.HelloRequest"))
    .WithProtoDefinition(protoDefinitionId)
    .RespondWith(Response.Create()
      .WithHeader("Content-Type", "application/grpc")
      .WithTrailingHeader("grpc-status", "0")
      .WithBodyAsProtoBuf("google.protobuf.Empty", new {})
      .WithTransformer());

  var channel = GrpcChannel.ForAddress(server.Url!);

  var client = new Greeter.GreeterClient(channel);

  var reply = await client.SayHelloAsync(new HelloRequest { Name = "Test" });

  reply.Should.BeOfType<Empty>();

  server.Stop();
}

I have tried with and without the .WithTransformer(), I have also tried with new Empty() new Empty {} and new() but none of those match.

Other related info

I am able to get the tests to work if the response is a type that I create in my proto, for example lets say that the HelloReply is in the proto file with a message HelloReply { string reply_message = 1; }

.WithBodyAsProtoBuf("organization.greet.api.v1.HelloReply", new { reply_message = "Hello" })

I have also tried creating my own message EmptyResponse { } and this also did not work

.WithBodyAsProtoBuf("organization.greet.api.v1.EmptyResponse", new {})

It only works when the response contains something like

I have also tried creating my own message EmptyResponse { string status = 1; }

.WithBodyAsProtoBuf("organization.greet.api.v1.EmptyResponse", new { status = "accepted" })

Please can someone either point me in the direction of what I am doing wrong to match an empty response, or confirm that this is a bug

Thanks

[StefH]

@ArcaneTSGK Thanks for noticing this. This is a new scenario I never tested.
Probably I need to fix this here: StefH/ProtoBufJsonConverter#15

I'll keep you informed on the progress...

[StefH]

@ArcaneTSGK
I was thinking, are there any other predefined types in google?

And what should actually be the behavior of WireMock.Net if a google.protobuf.Empty is a return value?
Is this just an empty response?

[ArcaneTSGK]

@StefH There are a number of them yes,

For Empty return value it's Empty {} it's because protobuf does not allow you to request/response with null / no body like a typical REST API would.

Converting Empty to JSON would represent an empty JSON object {}

Here are all of Googles Well Known types:

https://protobuf.dev/reference/protobuf/google.protobuf/

https://protobuf.dev/reference/protobuf/google.protobuf/#empty

Returning any of the scalar types in protobuf within a custom response that maps to a C# type is fine except for 'Enum'. I've not tried the 'Any' well-known type as my application doesn't have use for it, but I imagine that one might cause some issues, also when working with nullable types you use 'OneOf', again I haven't had to use these so I do not know if they'll work, but this is what a proto file would look like using those:

Usage of 'Any'

syntax = "proto3";

package tutorial;

import "include/google/protobuf/any.proto"

message Animal {
	string name=1;
	int32 age=2;
	google.protobuf.Any care_giver=3;
}

message Owner {
	int32 id=1;
	string first_name=2;
	string last_name=3;
}

message Foster {
	int32 id=1;
	string address=2;
}

In that example a care give can be any pre-defined message in the proto, which could be Owner or the Foster, so that might be a test case for Wiremock to see if Any maps back, and lastly OneOf for nullables where you'd need to be able to allow a null return in the response:

And this is OneOf

import "google/protobuf/struct.proto";

package tutorial;

option csharp_namespace = "MyExample.Dog";

message Dog {
	google.protobuf.Int32Value id=1;
	string name=2;
	NullableField profile_picture=3;
}

message NullableField {
	oneof kind {
		google.protobuf.NullValue null=1;
		google.protobuf.StringValue value=2;
	}
}

[StefH]

Currently I only have an easy way to support:

• google.protobuf.Empty
• google.protobuf.Duration
• google.protobuf.Timestamp

[StefH]

StefH/ProtoBufJsonConverter#18

[StefH]

@ArcaneTSGK
If you have time, you can use this preview version: 1.5.62-ci-19066

[StefH]

@ArcaneTSGK
Can you please test preview 1.5.62-ci-19067 , this version supports:

• google.protobuf.Empty
• google.protobuf.Duration
• google.protobuf.Timestamp
• google/protobuf/wrappers
• google/protobuf/any
• google/protobuf/struct
• Enum
• oneof

[ArcaneTSGK]

@ArcaneTSGK Can you please test preview 1.5.62-ci-19067 , this version supports:

• google.protobuf.Empty
• google.protobuf.Duration
• google.protobuf.Timestamp
• google/protobuf/wrappers
• google/protobuf/any
• google/protobuf/struct
• Enum
• oneof

I will have some time tomorrow to take a look, I'll let you know my results, thanks @StefH

[StefH]

@ArcaneTSGK
Did you have some time to check?

[StefH]

@ArcaneTSGK
Did you have some time to check?

[ArcaneTSGK]

Hi @StefH sorry for the delay,

Unfortunately, I can't install your prerelease version 1.5.62-ci-19067 as it does not resolve, are your pre-releases publicly available?

I only have the option (with prereleases enabled) to install 1.5.62 or 1.60 / 1.61

[StefH]

Preview versions are defined on MyGet.
See https://github.com/WireMock-Net/WireMock.Net/wiki/MyGet-preview-versions

But it could be that that specific version is automatically deleted because only x versions are kept on MyGet.

I will take a look and maybe I need to build a new preview.

[ArcaneTSGK]

Preview versions are defined on MyGet. See https://github.com/WireMock-Net/WireMock.Net/wiki/MyGet-preview-versions

But it could be that that specific version is automatically deleted because only x versions are kept on MyGet.

I will take a look and maybe I need to build a new preview.

No worries, I've added MyGet as a feed and will await the specific version with the gRPC fix to test

[StefH]

@ArcaneTSGK
Preview version on MyGet for this fix should be : 1.6.1-ci-19109

[ArcaneTSGK]

@StefH

So I attempted to use this build and I get the following exception:

(Status(StatusCode="Internal", Detail="Failed to deserialize response message.")

The proto file looks like this

syntax = "proto3";

import "google/protobuf/empty.proto";

package communication.api.v1;

message SendEmailRequest {
  string email_address = 1;
}

service CommunicationService {
  rpc SendEmail(SendEmailRequest) returns (google.protobuf.Empty);
}

The test is setup as follows:

public void SetupSendEmail()
{
    _server!.AddProtoDefinition(ProtoDefinitionId, CommunicationProtoFile)
        .Given(Request.Create()
            .UsingPost()
            .WithHttpVersion("2")
            .WithPath("/communication.api.v1.CommunicationService/SendEmail")
            .WithBodyAsProtoBuf("communication.api.v1.SendEmailRequest"))
        .WithProtoDefinition(ProtoDefinitionId)
        .RespondWith(Response.Create()
            .WithHeader("Content-Type", "application/grpc")
            .WithTrailingHeader("grpc-status", "0")
            .WithBodyAsProtoBuf(
                "google.protobuf.Empty",
                new { })
            .WithTransformer());
}

I did try Duration and Timestamp aswell but those were returning unimplemented for me when I was using google.protobuf.Timestamp and google.protobuf.Duration respectively as return types

Let me know if I'm missing anything

I also tried the fully qualified name of Google.Protobuf.WellKnownTypes.Empty as the message type to no avail.

Thanks

[StefH]

Strange, I did add a unit test https://github.com/WireMock-Net/WireMock.Net/blob/bug/1144-protobuf/test/WireMock.Net.Tests/Grpc/WireMockServerTests.Grpc.cs#L104

proto

private const string ProtoDefinitionWithWellKnownTypes = @"
syntax = ""proto3"";

import ""google/protobuf/empty.proto"";
import ""google/protobuf/timestamp.proto"";
import ""google/protobuf/duration.proto"";

service Greeter {
    rpc SayNothing (google.protobuf.Empty) returns (google.protobuf.Empty);
}

message MyMessageTimestamp {
    google.protobuf.Timestamp ts = 1;
}

message MyMessageDuration {
    google.protobuf.Duration du = 1;
}
";

Unit test code:

// Arrange
        var bytes = Convert.FromBase64String("CgRzdGVm");

        using var server = WireMockServer.Start();

        server
            .Given(Request.Create()
                .UsingPost()
                .WithPath("/grpc/Greeter/SayNothing")
                .WithBody(new NotNullOrEmptyMatcher())
            )
            .RespondWith(Response.Create()
                .WithBodyAsProtoBuf(ProtoDefinitionWithWellKnownTypes, "google.protobuf.Empty",
                    new { }
                )
                .WithTrailingHeader("grpc-status", "0")
                .WithTransformer()
            );

        // Act
        var protoBuf = new ByteArrayContent(bytes);
        protoBuf.Headers.ContentType = new MediaTypeHeaderValue("application/grpc-web");

        var client = server.CreateClient();
        var response = await client.PostAsync("/grpc/Greeter/SayNothing", protoBuf);

        // Assert
        response.StatusCode.Should().Be(HttpStatusCode.OK);
        var responseBytes = await response.Content.ReadAsByteArrayAsync();

        Convert.ToBase64String(responseBytes).Should().Be("");

        server.Stop();

What is the difference?

[StefH]

Also added another test: https://github.com/WireMock-Net/WireMock.Net/blob/bug/1144-protobuf/test/WireMock.Net.Tests/Grpc/WireMockServerTests.Grpc.cs#L144

[ArcaneTSGK]

I will have to provide a minimal reproduction solution when I have time, I've tried a few different things and for .Empty I always get the unable to deserialize response, and for duration/timestamp I'm getting the Unimplemented error.

I'll provide a link to the repository and throw something together by the end of the week

[StefH]

#1153

[StefH]

Using https://protobuf.heyenrath.nl/ with your message and google.protobuf.Empty does return an empty byte array

[StefH]

Hello @ArcaneTSGK, could you make an example project to demonstrate this issue?

[StefH]

Hello @ArcaneTSGK, could you make an example project to demonstrate this issue?

[carlos-pilao-deltatre]

Hello,
Just confirmed the reported issue (using the latest version: 1.6.3):

1. If you have import google/protobuf/empty.proto in your proto file and you are not referencing Empty anywhere the response will be always 404 Unimplemented
2. If you have import google/protobuf/empty.proto in your proto file and you are referencing Empty in the contracts then it works
3. If you have import google/protobuf/struct.proto in your proto file then the response will be always 404 Unimplemented
   cc @StefH

[StefH]

@carlos-pilao-deltatre
Can you please test preview version 1.6.2-ci-19188?

[carlos-pilao-deltatre]

@StefH
Cannot find that release anymore. Tried with the latest preview (1.6.3-ci-19196) and cant compile my solution with that one:

• Getting ambiguous reference on Google.Protobuf.WellKnownTypes

[StefH]

You did check MyGet?

[carlos-pilao-deltatre]

yes

[StefH]

@carlos-pilao-deltatre

try this
1.6.3-ci-19198