Serialization API - efficiency

[JamesNK]

Consider providing an extension point allowing users to configure serialization libraries other than Google.Protobuf.

Grpc.Core already offers an abstraction layer that we can use.

Issue is for discussing improving it and Google.Protobuf (default serialization layer)

Start from #30 (comment)

[jtattermusch]

Consider providing an extension point allowing users to configure serialization libraries other than Google.Protobuf.

Right now Google.Protobuf is used for:

• Determining the methods via its service descriptor

As noted in #21 (comment) this would not work when using other serialization format.

• Serialization/deserialization between .NET messages and protobuf wire format

Two design questions:

• Do we want Grpc.AspNetCore.* to not have a reference to Google.Protobuf. Right now we use types from it related to the descriptor on a grpc service to discover the methods on a service. Changing this is possible but would require changes to what Grpc.Tools generates. See BindService should not require a concrete instance of the GRPC service implementation  #21

I think Grpc.AspNetCore should not have a dependency on Google.Protobuf as
gRPC's philosophy is to be serialization format agnostic. We do provide easy-to-use Google.Protobuf support as the default choice, but in most implementations we do not take a direct dependency on Google.Protobuf. Support for other serialization formats (Flatbuffers, Bond, Thrift and many others) should be provided as add-on packages (provided by the community) and should not be part of the core libraries as there is many serialization formats out there and each one has its specifics.

• Do we want Grpc.AspNetCore.* to support serialization libraries other than Google.Protobuf? For example, a C# version of FlatBuffers. Providing an interface like SignalR's IHubProtocol would allow Google.Protobuf's serialization logic to be hidden behind an interface, and let users configure their own implementation.

We should make it possible to use other serialization formats and expect users to do that at some point. I'd follow the pattern used by Grpc.Core: The only thing that Grpc.Core currently assumes is that a serialized message will be an instance of a class and that one defines a "Marshaller" that knows how to serialize/deserialize (invoking the protobuf-specific serialization/deserialization is part of the generated code right now: https://github.com/grpc/grpc/blob/b6d8957b0d78f768b858698ba6a79296db448bb2/src/csharp/Grpc.Examples/MathGrpc.cs#L30)
To use a different serialization format, one needs to write their own codegen and possibly their own version of Grpc.Tools package for build integration.

[JamesNK]

https://github.com/grpc/grpc/blob/master/src/csharp/Grpc.Core/Marshaller.cs

An issue with the current Marshaller pattern is it operates on byte arrays. We'd want to have a pattern that doesn't require allocating an array and buffering to memory. For best performance we'd ideally read directly from the request and write directly to the response.

For serialization extensibility, do you expect compatible serializer libraries to always generate a BindService method? Would you recommend BindService as the way Grpc.AspNetCore.Server discovers services and aquires marshallers?

https://github.com/grpc/grpc/blob/b6d8957b0d78f768b858698ba6a79296db448bb2/src/csharp/Grpc.Examples/MathGrpc.cs#L279-L300

If the answer is yes, I think there are two changes we would want to make:

1. Update Grpc.Tools to output a BindService method that doesn't take an instance and add a new AddMethod overload, ServiceBinderBase.AddMethod<TRequest, TResponse>(Method<TRequest, TResponse>), that it would call. This relates to BindService should not require a concrete instance of the GRPC service implementation  #21

   Once we can use BindService then we should only depend on Grpc.Core types. Swapping the serialization would only involve changing the marshaller in generated code.

/// <summary>Register service method implementations with a service binder. Useful when customizing the service binding logic.
/// Note: this method is part of an experimental API that can change or be removed without any prior notice.</summary>
/// <param name="serviceBinder">Service methods will be bound by calling <c>AddMethod</c> on this object.</param></param>
public static void BindService(grpc::ServiceBinderBase serviceBinder)
{
  serviceBinder.AddMethod(__Method_Div);
  serviceBinder.AddMethod(__Method_DivMany);
  serviceBinder.AddMethod(__Method_Fib);
  serviceBinder.AddMethod(__Method_Sum);
}

2. Update Marshaller with Serializer/Deserializer properties that allow message data to be written and read without allocating byte arrays.

(1) is something to figure out now. It will be the main way Grpc.AspNetCore.Server integrates with Grpc.Core so it is something to lock down.

(2) is largely an API detail, and is something that doesn't need to be figured out right away. I'd like to learn more about how gRPC and ASP.NET Core interact before deciding what the best signature would be. ASP.NET Core will soon expose System.IO.Pipelines in HTTP2 and it will provide the best opportunity to maximize performance.

[jtattermusch]

https://github.com/grpc/grpc/blob/master/src/csharp/Grpc.Core/Marshaller.cs

An issue with the current Marshaller pattern is it operates on byte arrays. We'd want to have a pattern that doesn't require allocating an array and buffering to memory. For best performance we'd ideally read directly from the request and write directly to the response.

I am aware of that limitation and I've had a solution in progress for a while:
The idea is to switch the "simple marshaller" (which converts class into byte array and back). Into a "contextual marshaller" which also takes context which allow multiple ways of accessing the payload (e.g. accessing ReadOnlySpan with data slice by slice etc or using a rented buffer).

Grpc.Core already uses contextual serializers internally (but they delegate to the old-style byte array conversions for now).
grpc/grpc#16367
grpc/grpc#17167

Serialization/DeserializationContext is not populated yet with the advanced members, but I have a PR in progress. A taste of new DeserializationContext (not yet merged):
https://github.com/grpc/grpc/pull/16371/files#diff-22501e836d94b70164ce50d8b0c0c4e5

Btw, this effort started before we came up with the plan for Grpc.AspNetCore, so it would be good to review if this design works well with the planned APIs for Asp.NET Core pipeline access to requests/responses - let's create a separate issue for this review?

For serialization extensibility, do you expect compatible serializer libraries to always generate a BindService method? Would you recommend BindService as the way Grpc.AspNetCore.Server discovers services and aquires marshallers?

https://github.com/grpc/grpc/blob/b6d8957b0d78f768b858698ba6a79296db448bb2/src/csharp/Grpc.Examples/MathGrpc.cs#L279-L300

If the answer is yes, I think there are two changes we would want to make:

I think the answer is Yes.

1. Update Grpc.Tools to output a BindService method that doesn't take an instance and add a new AddMethod overload, ServiceBinderBase.AddMethod<TRequest, TResponse>(Method<TRequest, TResponse>), that it would call. This relates to BindService should not require a concrete instance of the GRPC service implementation  #21
   Once we can use BindService then we should only depend on Grpc.Core types. Swapping the serialization would only involve changing the marshaller in generated code.

/// <summary>Register service method implementations with a service binder. Useful when customizing the service binding logic.
/// Note: this method is part of an experimental API that can change or be removed without any prior notice.</summary>
/// <param name="serviceBinder">Service methods will be bound by calling <c>AddMethod</c> on this object.</param></param>
public static void BindService(grpc::ServiceBinderBase serviceBinder)
{
  serviceBinder.AddMethod(__Method_Div);
  serviceBinder.AddMethod(__Method_DivMany);
  serviceBinder.AddMethod(__Method_Fib);
  serviceBinder.AddMethod(__Method_Sum);
}

1. Update Marshaller with Serializer/Deserializer properties that allow message data to be written and read without allocating byte arrays.

(1) is something to figure out now. It will be the main way Grpc.AspNetCore.Server integrates with Grpc.Core so it is something to lock down.

I think we can add that soon. If you want, you can also create a PR yourself, here is an example PR that added ServiceBinderBase: https://github.com/grpc/grpc/pull/17157/commits

(2) is largely an API detail, and is something that doesn't need to be figured out right away. I'd like to learn more about how gRPC and ASP.NET Core interact before deciding what the best signature would be. ASP.NET Core will soon expose System.IO.Pipelines in HTTP2 and it will provide the best opportunity to maximize performance.

This is what ContextualSerializer/ContextualDeserializer can help with (see above). Let's review in a separate issue to make sure the (De)SerializerContext has methods that play well with both Grpc.Core and Grpc.AspNetCore (which will use System.IO.Pipelines).

[JamesNK]

I think we can add that soon. If you want, you can also create a PR yourself, here is an example PR that added ServiceBinderBase: grpc/grpc/pull/17157/commits

Sure, we'll create a PR for it.

This is what ContextualSerializer/ContextualDeserializer can help with (see above). Let's review in a separate issue to make sure the (De)SerializerContext has methods that play well with both Grpc.Core and Grpc.AspNetCore (which will use System.IO.Pipelines).

Great. ASP.NET Core currently doesn't expose pipelines, but it will come in the next preview of ASP.NET Core 3.0. That shouldn't be too far away. We'll look at whether the contextual serializers are the right shape then.

FYI @davidfowl

[davidfowl]

We'll need another implementation to vet the abstraction. We can try using the other popular .NET protobuf library as a test https://www.nuget.org/packages/protobuf-net/.

[jtattermusch]

I'm not sure that Protobuf.NET supports proto3 (which is the recommended version of .proto format to use and the only one supported by Google.Protobuf for C#).
An alternative would be to try with Flatbuffers https://github.com/google/flatbuffers/tree/master/net/FlatBuffers that have the advantage of zero-overhead serialization/deserialization so could be useful for performance testing.

But it terms of vetting the abstraction, I think just the ability to expose a "generic" service (e.g. a method handler that gives access to the actual binary payloads instead of serialized/deserialized messages would be good enough). E.g. here: https://github.com/grpc/grpc/blob/1064c6f663a2735fa2f4230dd8d278ae66d344b7/src/csharp/Grpc.IntegrationTesting/ServerRunners.cs#L92

[JamesNK]

I've renamed this issue. Grpc.AspNetCore should use Grpc.Core's abstraction.

This is what ContextualSerializer/ContextualDeserializer can help with (see above). Let's review in a separate issue to make sure the (De)SerializerContext has methods that play well with both Grpc.Core and Grpc.AspNetCore (which will use System.IO.Pipelines).

I think we should have a meeting about this: @jtattermusch, @davidfowl, @JunTaoLuo and I.

My high-level thoughts after some research:

1. Google.Protobuf serialization is based around Stream. Unless we want to make large changes to that code base, or switch to another serializer, then the Grpc.Core API will need logic to interop with streams
2. Grpc.Server's most efficient way to write: write directly to Response.PipeBody, an IBufferWriter<byte>.
   • It will be simple to create a Stream implementation that writes to a IBufferWriter<byte>.
   • Getting the message length will need to be part of the marshaller API. It needs to be written first.
   • Google.Protobuf writing is sync. An async flush can happen after the message has been written to the buffer writer.
3. Grpc.Server's most efficient way to read: Request.BodyPipe.ReadAsync() to get the message header with the message length, then read the complete message into a ReadOnlySequence<byte>.
   • It will be simple to create a Stream implementation that wraps a IBufferWriter<byte>.
   • Google.Protobuf reading is sync from the stream wrapping ReadOnlySequence<byte>. Async happens when reading the request into the sequence.
4. Updating Grpc.Tools output needs to be updated
   • Today ContextualSerializer/ContextualDeserializer use the array methods.

[jtattermusch]

I've renamed this issue. Grpc.AspNetCore should use Grpc.Core's abstraction.

This is what ContextualSerializer/ContextualDeserializer can help with (see above). Let's review in a separate issue to make sure the (De)SerializerContext has methods that play well with both Grpc.Core and Grpc.AspNetCore (which will use System.IO.Pipelines).

I think we should have a meeting about this: @jtattermusch, @davidfowl, @JunTaoLuo and I.

My high-level thoughts after some research:

1. Google.Protobuf serialization is based around Stream. Unless we want to make large changes to that code base, or switch to another serializer, then the Grpc.Core API will need logic to interop
   with streams

A bit more detail here:
The two main classes are https://github.com/protocolbuffers/protobuf/blob/master/csharp/src/Google.Protobuf/CodedInputStream.cs and CodedOutputStream. All the operations are actually implemented over a byte array (including when reading/writing to from stream), the streams are only used to read/write the next chunk of data once the buffer gets full. Using a stream is not necessary, one can provide her own buffer directly (doesn't really help us in our situation, but it's good to know). The problem with using Span<> instead of a byte array here is that all the parsing state is kept in the CodedInputStream instance (and Span<> can't be stored there).

2. Grpc.Server's most efficient way to write: write directly to Response.PipeBody, an IBufferWriter<byte>.

   • It will be simple to create a Stream implementation that writes to a IBufferWriter<byte>.
   • Getting the message length will need to be part of the marshaller API. It needs to be written first.
   • Google.Protobuf writing is sync. An async flush can happen after the message has been written to the buffer writer.

3. Grpc.Server's most efficient way to read: Request.BodyPipe.ReadAsync() to get the message header with the message length, then read the complete message into a ReadOnlySequence<byte>.

   • It will be simple to create a Stream implementation that wraps a IBufferWriter<byte>.
   • Google.Protobuf reading is sync from the stream wrapping ReadOnlySequence<byte>. Async happens when reading the request into the sequence.

4. Updating Grpc.Tools output needs to be updated

   • Today ContextualSerializer/ContextualDeserializer use the array methods.

[JunTaoLuo]

Let's go over this in our next weekly sync.