Feat/http transcoding (Continuation) (#351)

* minimal working example * return correct content-type when codec is JSON * support default rpc options arg * add custom protoc plugin * include api extensions * include http options in helloworld example * dont pack message if the method was called as a transcode req * wip! build route from HttpRule * update helloworld example with HttpRule options * rename mod to transcode * fix! match rpc options in stub * encode segments to path * expose routes from server Routes are fetched from server and compiled into a dispatch conf. Instead of finding a matching server in the handler the path is matched and the handler is called with the correct server. * init http/json transcode integration tests * merge path bindings with body * http rule template lexer / parse rename literal -> identifier * fix warnings * fix! merge request params before decoding into struct * update example curl requests * Fix typo in filenames * minimal working example * return correct content-type when codec is JSON * support default rpc options arg * add custom protoc plugin * include api extensions * include http options in helloworld example * dont pack message if the method was called as a transcode req * wip! build route from HttpRule * update helloworld example with HttpRule options * rename mod to transcode * fix! match rpc options in stub * encode segments to path * expose routes from server Routes are fetched from server and compiled into a dispatch conf. Instead of finding a matching server in the handler the path is matched and the handler is called with the correct server. * init http/json transcode integration tests * merge path bindings with body * http rule template lexer / parse rename literal -> identifier * fix warnings * fix! merge request params before decoding into struct * update example curl requests * include http method in __call_rpc__ This enables to differentiate to transcoded calls which has the same URL path but different request methods * add query decoder * include query string in request mapping * map request using HttpRule.body * comply to request mapping rules by checking HttpRule.body param * support newline delimeted json for transcoded server streams * base generator on protobuf 0.11.0 * map transcoded response using HttpRule.response_body * support nested field paths in http paths * Fix! parse sub-paths for variable assignment * support calling grpc method with json if http_transcode == true * remove params from template parsing * add GRPC.Server transcode example * get requests can use accept header to find codec * remove reference to HttpRule type * rm protoc plugin in favour of protobuf_generate * add new cowboy router middleware The default routing in cowboy is not capable enough to express 'complex' routes on the form `/v1/{a=shelves/*}/books/**`. The added router is derived from `:cowboy_router` but implements new matching to allow to match on the above statement. * move routing specific func. to GRPC.Server.Router * don't generate protos before tests * reset helloworld example * add helloworld_transcode example * include google api protos in tests only * map gRPC code -> http status * Minor adjusts * Adjusts * Fix. Assert payload * Remove deprecated and fix some tests * Fix. status handler test * Update lib/grpc/codec/json.ex * Update lib/grpc/codec/json.ex * Update lib/grpc/server.ex * Update lib/grpc/transport/http2.ex * Update lib/grpc/server/transcode.ex * chore: remove prometheus stray deps --------- Co-authored-by: Simon Thörnqvist <simon.thornqvist@erlang-solutions.com> Co-authored-by: Paulo Valente <16843419+polvalente@users.noreply.github.com>
elixir-grpc · Mar 25, 2024 · 8aaf3d3 · 8aaf3d3
1 parent ab18c93
commit 8aaf3d3
Show file tree

Hide file tree

Showing 80 changed files with 5,925 additions and 215 deletions.
diff --git a/examples/helloworld/README.md b/examples/helloworld/README.md
@@ -17,6 +17,16 @@ $ mix run --no-halt
 $ mix run priv/client.exs
 ```
 
+## HTTP Transcoding
+
+``` shell
+# Say hello
+curl -H 'Content-type: application/json' http://localhost:50051/v1/greeter/test
+
+# Say hello from
+curl -XPOST -H 'Content-type: application/json' -d '{"name": "test", "from": "anon"}' http://localhost:50051/v1/greeter
+```
+
 ## Regenerate Elixir code from proto
 
 1. Modify the proto `priv/protos/helloworld.proto`
@@ -26,8 +36,10 @@ $ mix run priv/client.exs
 mix escript.install hex protobuf
 ```
 4. Generate the code:
+
 ```shell
-$ protoc -I priv/protos --elixir_out=plugins=grpc:./lib/ priv/protos/helloworld.proto
+$ (cd ../../; mix build_protobuf_escript && mix escript.build)
+$ protoc -I priv/protos --elixir_out=:./lib/ --grpc_elixir_out=./lib --plugin="../../deps/protobuf/protoc-gen-elixir" --plugin="../../protoc-gen-grpc_elixir" priv/protos/helloworld.proto
 ```
 
 Refer to [protobuf-elixir](https://github.com/tony612/protobuf-elixir#usage) for more information.

diff --git a/examples/helloworld/lib/helloworld.pb.ex b/examples/helloworld/lib/helloworld.pb.ex
@@ -5,6 +5,14 @@ defmodule Helloworld.HelloRequest do
   field :name, 1, type: :string
 end
 
+defmodule Helloworld.HelloRequestFrom do
+  @moduledoc false
+  use Protobuf, protoc_gen_elixir_version: "0.11.0", syntax: :proto3
+
+  field :name, 1, type: :string
+  field :from, 2, type: :string
+end
+
 defmodule Helloworld.HelloReply do
   @moduledoc false
   use Protobuf, protoc_gen_elixir_version: "0.11.0", syntax: :proto3
@@ -13,14 +21,16 @@ defmodule Helloworld.HelloReply do
   field :today, 2, type: Google.Protobuf.Timestamp
 end
 
-defmodule Helloworld.Greeter.Service do
+defmodule Helloworld.GetMessageRequest do
   @moduledoc false
-  use GRPC.Service, name: "helloworld.Greeter", protoc_gen_elixir_version: "0.11.0"
+  use Protobuf, protoc_gen_elixir_version: "0.11.0", syntax: :proto3
 
-  rpc :SayHello, Helloworld.HelloRequest, Helloworld.HelloReply
+  field :name, 1, type: :string
 end
 
-defmodule Helloworld.Greeter.Stub do
+defmodule Helloworld.Message do
   @moduledoc false
-  use GRPC.Stub, service: Helloworld.Greeter.Service
+  use Protobuf, protoc_gen_elixir_version: "0.11.0", syntax: :proto3
+
+  field :text, 1, type: :string
 end
diff --git a/examples/helloworld/lib/helloworld.svc.ex b/examples/helloworld/lib/helloworld.svc.ex
@@ -0,0 +1,65 @@
+defmodule Helloworld.Greeter.Service do
+  @moduledoc false
+
+  use GRPC.Service, name: "helloworld.Greeter", protoc_gen_elixir_version: "0.11.0"
+
+  rpc(:SayHello, Helloworld.HelloRequest, Helloworld.HelloReply, %{
+    http: %{
+      type: Google.Api.PbExtension,
+      value: %Google.Api.HttpRule{
+        __unknown_fields__: [],
+        additional_bindings: [],
+        body: "",
+        pattern: {:get, "/v1/greeter/{name}"},
+        response_body: "",
+        selector: ""
+      }
+    }
+  })
+
+  rpc(:SayHelloFrom, Helloworld.HelloRequestFrom, Helloworld.HelloReply, %{
+    http: %{
+      type: Google.Api.PbExtension,
+      value: %Google.Api.HttpRule{
+        __unknown_fields__: [],
+        additional_bindings: [],
+        body: "*",
+        pattern: {:post, "/v1/greeter"},
+        response_body: "",
+        selector: ""
+      }
+    }
+  })
+end
+
+defmodule Helloworld.Greeter.Stub do
+  @moduledoc false
+
+  use GRPC.Stub, service: Helloworld.Greeter.Service
+end
+
+defmodule Helloworld.Messaging.Service do
+  @moduledoc false
+
+  use GRPC.Service, name: "helloworld.Messaging", protoc_gen_elixir_version: "0.11.0"
+
+  rpc(:GetMessage, Helloworld.GetMessageRequest, Helloworld.Message, %{
+    http: %{
+      type: Google.Api.PbExtension,
+      value: %Google.Api.HttpRule{
+        __unknown_fields__: [],
+        additional_bindings: [],
+        body: "",
+        pattern: {:get, "/v1/{name=messages/*}"},
+        response_body: "",
+        selector: ""
+      }
+    }
+  })
+end
+
+defmodule Helloworld.Messaging.Stub do
+  @moduledoc false
+
+  use GRPC.Stub, service: Helloworld.Messaging.Service
+end
diff --git a/examples/helloworld/lib/server.ex b/examples/helloworld/lib/server.ex
@@ -1,16 +1,31 @@
 defmodule Helloworld.Greeter.Server do
-  use GRPC.Server, service: Helloworld.Greeter.Service
+  use GRPC.Server,
+    service: Helloworld.Greeter.Service,
+    http_transcode: true
 
   @spec say_hello(Helloworld.HelloRequest.t(), GRPC.Server.Stream.t()) ::
           Helloworld.HelloReply.t()
   def say_hello(request, _stream) do
+    Helloworld.HelloReply.new(
+      message: "Hello #{request.name}",
+      today: today()
+    )
+  end
+
+  @spec say_hello_from(Helloworld.HelloFromRequest.t(), GRPC.Server.Stream.t()) ::
+          Helloworld.HelloReply.t()
+  def say_hello_from(request, _stream) do
+    Helloworld.HelloReply.new(
+      message: "Hello #{request.name}. From #{request.from}",
+      today: today()
+    )
+  end
+
+  defp today do
     nanos_epoch = System.system_time() |> System.convert_time_unit(:native, :nanosecond)
     seconds = div(nanos_epoch, 1_000_000_000)
     nanos = nanos_epoch - seconds * 1_000_000_000
 
-    Helloworld.HelloReply.new(
-      message: "Hello #{request.name}",
-      today: %Google.Protobuf.Timestamp{seconds: seconds, nanos: nanos}
-    )
+    %Google.Protobuf.Timestamp{seconds: seconds, nanos: nanos}
   end
 end
diff --git a/examples/helloworld/mix.exs b/examples/helloworld/mix.exs
@@ -19,6 +19,7 @@ defmodule Helloworld.Mixfile do
   defp deps do
     [
       {:grpc, path: "../../"},
+      {:jason, "~> 1.3.0"},
       {:protobuf, "~> 0.11"},
       {:google_protos, "~> 0.3.0"},
       {:dialyxir, "~> 1.1", only: [:dev, :test], runtime: false}

diff --git a/examples/helloworld/mix.lock b/examples/helloworld/mix.lock
@@ -5,6 +5,7 @@
   "erlex": {:hex, :erlex, "0.2.6", "c7987d15e899c7a2f34f5420d2a2ea0d659682c06ac607572df55a43753aa12e", [:mix], [], "hexpm", "2ed2e25711feb44d52b17d2780eabf998452f6efda104877a3881c2f8c0c0c75"},
   "google_protos": {:hex, :google_protos, "0.3.0", "15faf44dce678ac028c289668ff56548806e313e4959a3aaf4f6e1ebe8db83f4", [:mix], [{:protobuf, "~> 0.10", [hex: :protobuf, repo: "hexpm", optional: false]}], "hexpm", "1f6b7fb20371f72f418b98e5e48dae3e022a9a6de1858d4b254ac5a5d0b4035f"},
   "gun": {:hex, :grpc_gun, "2.0.1", "221b792df3a93e8fead96f697cbaf920120deacced85c6cd3329d2e67f0871f8", [:rebar3], [{:cowlib, "~> 2.11", [hex: :cowlib, repo: "hexpm", optional: false]}], "hexpm", "795a65eb9d0ba16697e6b0e1886009ce024799e43bb42753f0c59b029f592831"},
+  "jason": {:hex, :jason, "1.3.0", "fa6b82a934feb176263ad2df0dbd91bf633d4a46ebfdffea0c8ae82953714946", [:mix], [{:decimal, "~> 1.0 or ~> 2.0", [hex: :decimal, repo: "hexpm", optional: true]}], "hexpm", "53fc1f51255390e0ec7e50f9cb41e751c260d065dcba2bf0d08dc51a4002c2ac"},
   "protobuf": {:hex, :protobuf, "0.11.0", "58d5531abadea3f71135e97bd214da53b21adcdb5b1420aee63f4be8173ec927", [:mix], [{:jason, "~> 1.2", [hex: :jason, repo: "hexpm", optional: true]}], "hexpm", "30ad9a867a5c5a0616cac9765c4d2c2b7b0030fa81ea6d0c14c2eb5affb6ac52"},
   "ranch": {:hex, :ranch, "1.8.0", "8c7a100a139fd57f17327b6413e4167ac559fbc04ca7448e9be9057311597a1d", [:make, :rebar3], [], "hexpm", "49fbcfd3682fab1f5d109351b61257676da1a2fdbe295904176d5e521a2ddfe5"},
 }
diff --git a/examples/helloworld/priv/protos/google/api/annotations.proto b/examples/helloworld/priv/protos/google/api/annotations.proto
@@ -0,0 +1,31 @@
+// Copyright 2015 Google LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+syntax = "proto3";
+
+package google.api;
+
+import "google/api/http.proto";
+import "google/protobuf/descriptor.proto";
+
+option go_package = "google.golang.org/genproto/googleapis/api/annotations;annotations";
+option java_multiple_files = true;
+option java_outer_classname = "AnnotationsProto";
+option java_package = "com.google.api";
+option objc_class_prefix = "GAPI";
+
+extend google.protobuf.MethodOptions {
+  // See `HttpRule`.
+  HttpRule http = 72295728;
+}