Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GSoD 2020] Added tutorials directory on gRPC-Gateway #1829

Merged
merged 19 commits into from
Dec 1, 2020
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/docs/faq.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
layout: default
title: FAQ
nav_order: 7
nav_order: 8
---

# FAQ
Expand Down
122 changes: 122 additions & 0 deletions docs/docs/tutorials/adding_annotations.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
---
layout: default
title: Adding the grpc-gateway annotations to an existing protobuf file
parent: Tutorials
nav_order: 5
---

## Adding the grpc-gateway annotations to an existing protobuf file
johanbrandhorst marked this conversation as resolved.
Show resolved Hide resolved

Now that we've got a working Go gRPC server, we need to add the grpc-gateway annotations:

```proto
syntax = "proto3";

package helloworld;

import "google/api/annotations.proto";

// Here is the overall greeting service definition where we define all our endpoints
service Greeter {
// Sends a greeting
rpc SayHello (HelloRequest) returns (HelloReply) {
option (google.api.http) = {
post: "/v1/example/echo"
body: "*"
};
}
}

// The request message containing the user's name.
message HelloRequest {
string name = 1;
}

// The response message containing the greetings
message HelloReply {
string message = 1;
}
```

johanbrandhorst marked this conversation as resolved.
Show resolved Hide resolved
iamrajiv marked this conversation as resolved.
Show resolved Hide resolved
Also, See [a_bit_of_everything.proto](https://github.com/grpc-ecosystem/grpc-gateway/blob/master/examples/internal/proto/examplepb/a_bit_of_everything.proto) for examples of more annotations you can add to customize gateway behavior and generated OpenAPI output.

### Generating the grpc-gateway stubs

Now that we've got the grpc-gateway annotations added to the proto file, we need to use the grpc-gateway generator to generate the stubs.

Before we can do that, we need to copy some dependencies into our protofile structure. Copy the `third_party/googleapis` folder from the grpc-gateway repository to your local protofile structure. It should look like this afterwards:

```
proto/
helloworld/
hello_world.proto
google/
api/
http.proto
annotations.proto
iamrajiv marked this conversation as resolved.
Show resolved Hide resolved
```

#### Using buf

We'll need to add the grpc-gateway generator to the generation configuration:

```yml
plugins:
- name: go
out: proto
opt: paths=source_relative
- name: go-grpc
out: proto
opt: paths=source_relative
- name: grpc-gateway
out: proto
opt: paths=source_relative
```

And that's it! Now if you run:

```sh
buf generate
```

It should produce a `*.gw.pb.go` file.

#### Using protoc

Now we need to add the grpc-gateway generator to the protoc invocation:

```
protoc -I ./proto \
... other plugins ...
--grpc-gateway_out ./proto --grpc-gateway_opt paths=source_relative
./proto/helloworld/hello_world.proto
```
iamrajiv marked this conversation as resolved.
Show resolved Hide resolved

```sh
protoc -I ./proto \
--go_out ./proto --go_opt paths=source_relative \
--go-grpc_out ./proto --go-grpc_opt paths=source_relative \
./proto/helloworld/hello_world.proto
--grpc-gateway_out ./proto --grpc-gateway_opt paths=source_relative
./proto/helloworld/hello_world.proto
iamrajiv marked this conversation as resolved.
Show resolved Hide resolved
```

This should generate a `*.gw.pb.go` file.

johanbrandhorst marked this conversation as resolved.
Show resolved Hide resolved
### Testing the grpc-gateway
iamrajiv marked this conversation as resolved.
Show resolved Hide resolved

Then we use curl to send http requests:

```sh
curl -X POST -k http://localhost:8080/v1/example/echo -d '{"name": " Hello"}'
```

```
{"message":"Hello World"}
```

The process is as follows:
iamrajiv marked this conversation as resolved.
Show resolved Hide resolved

`curl` sends a request to the gateway with the post, gateway as proxy forwards the request to greeter_server through grpc, greeter_server returns the result through grpc, the gateway receives the result, and json returns to the front end.

In this way, the transformation process from http json to internal grpc is completed through grpc-gateway.
74 changes: 74 additions & 0 deletions docs/docs/tutorials/creating_main.go.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
---
layout: default
title: Creating main.go
parent: Tutorials
nav_order: 4
---

## Creating main.go

```go
package main

import (
"context"
"io/ioutil"
"log"
"net"
"os"

"google.golang.org/grpc"
"google.golang.org/grpc/credentials"
"google.golang.org/grpc/grpclog"

// Static files

"github.com/hello/hello_world/gateway"
"github.com/hello/hello_world/insecure"
iamrajiv marked this conversation as resolved.
Show resolved Hide resolved
hello "github.com/hello/hello_world/proto"
iamrajiv marked this conversation as resolved.
Show resolved Hide resolved
)

type server struct{}

func NewServer() *server {
return &server{}
}

func (s *server) SayHello(ctx context.Context, in *hello.HelloRequest) (*hello.HelloReply, error) {

iamrajiv marked this conversation as resolved.
Show resolved Hide resolved
log.Println("request: ", in.Name)
iamrajiv marked this conversation as resolved.
Show resolved Hide resolved
return &hello.HelloReply{Message: in.Name + " World"}, nil
}

func main() {

// Adds gRPC internal logs. This is quite verbose, so adjust as desired!
log := grpclog.NewLoggerV2(os.Stdout, ioutil.Discard, ioutil.Discard)
grpclog.SetLoggerV2(log)
iamrajiv marked this conversation as resolved.
Show resolved Hide resolved

addr := "0.0.0.0:10000"
lis, err := net.Listen("tcp", addr)
if err != nil {
log.Fatalln("Failed to listen:", err)
}

s := grpc.NewServer(
// TODO: Replace with your own certificate!
grpc.Creds(credentials.NewServerTLSFromCert(&insecure.Cert)),
)
hello.RegisterGreeterServer(s, &server{})

// Serve gRPC Server
log.Info("Serving gRPC on https://", addr)
go func() {
log.Fatal(s.Serve(lis))
}()

err = gateway.Run("dns:///" + addr)
log.Fatalln(err)
}
iamrajiv marked this conversation as resolved.
Show resolved Hide resolved
```

### Read More

For more refer to this boilerplate repository [grpc-gateway-boilerplate Template](https://github.com/johanbrandhorst/grpc-gateway-boilerplate).
iamrajiv marked this conversation as resolved.
Show resolved Hide resolved
7 changes: 7 additions & 0 deletions docs/docs/tutorials/generating_stubs/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
layout: default
title: Generating stubs
parent: Tutorials
nav_order: 3
has_children: true
---
iamrajiv marked this conversation as resolved.
Show resolved Hide resolved
46 changes: 46 additions & 0 deletions docs/docs/tutorials/generating_stubs/using_buf.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
---
layout: default
title: Generating stubs using buf
parent: Generating stubs
grand_parent: Tutorials
nav_order: 1
---

## Generating stubs using buf

[Buf](https://github.com/bufbuild/buf) is a tool that provides various protobuf utilities such as linting, breaking change detection and generation. Please find installation instructions on https://github.com/bufbuild/buf.

It is configured through a `buf.yaml` file that should be checked in to the root of your repository. Buf will automatically read this file if present. Configuration can also be provided via the command-line flag `--config`, which accepts a path to a `.json` or `.yaml` file, or direct JSON or YAML data.

All Buf operations that use your local `.proto` files as input rely on a valid build configuration. This configuration tells Buf where to search for `.proto` files, and how to handle imports. As opposed to protoc, where all `.proto` files are manually specified on the command-line, buf operates by recursively discovering all `.proto` files under configuration and building them.
iamrajiv marked this conversation as resolved.
Show resolved Hide resolved

The following is an example of a valid configuration, assuming you have your `.proto` files rooted in the `proto` folder relative to the root of your repository.

```yml
version: v1beta1
build:
roots:
- proto
```

To generate type and gRPC stubs for Go, create the file `buf.gen.yaml` at the root of the repository:

```yml
plugins:
- name: go
out: proto
opt: paths=source_relative
- name: go-grpc
out: proto
opt: paths=source_relative
```

We use the `go` and `go-grpc` plugins to generate Go types and gRPC service definitions. We're outputting the generated files relative to the `proto` folder, and we're using the `paths=source_relative` option, which means that the generated files will appear in the same directory as the source `.proto` file.

Then run

```sh
buf generate
```

This will have generated a `*.pb.go` and a `*_grpc.pb.go` file for each protobuf package in our `proto` file hierarchy.
26 changes: 26 additions & 0 deletions docs/docs/tutorials/generating_stubs/using_protoc.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
---
layout: default
title: Generating stubs using protoc
parent: Generating stubs
grand_parent: Tutorials
nav_order: 2
---

## Generating stubs using protoc

Generate Go type and gRPC stubs:

This step generates the Go types and gRPC stubs that you can use to implement the service and consume from clients.
iamrajiv marked this conversation as resolved.
Show resolved Hide resolved

Here's an example of what a `protoc` command might look like to generate Go stubs, assuming that you're at the root of your repository and you have your proto files in a directory called `proto`:

```sh
protoc -I ./proto \
--go_out ./proto --go_opt paths=source_relative \
--go-grpc_out ./proto --go-grpc_opt paths=source_relative \
./proto/helloworld/hello_world.proto
```

We use the `go` and `go-grpc` plugins to generate Go types and gRPC service definitions. We're outputting the generated files relative to the `proto` folder, and we're using the `paths=source_relative` option, which means that the generated files will appear in the same directory as the source `.proto` file.

This will have generated a `*.pb.go` and a `*_grpc.pb.go` file for each protobuf package in our `proto` file hierarchy.
iamrajiv marked this conversation as resolved.
Show resolved Hide resolved
6 changes: 6 additions & 0 deletions docs/docs/tutorials/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
layout: default
title: Tutorials
nav_order: 7
has_children: true
---
36 changes: 36 additions & 0 deletions docs/docs/tutorials/introduction.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
---
layout: default
title: Introduction about gRPC-Gateway
iamrajiv marked this conversation as resolved.
Show resolved Hide resolved
parent: Tutorials
nav_order: 1
---

## Introduction

We all know that gRPC is not a tool for everything. There are cases where we still want to provide a traditional RESTful JSON API. The reasons can range from maintaining backwards-compatibility to supporting programming languages or clients not well supported by gRPC. But coding another API for REST is quite a time consuming and tedious.
iamrajiv marked this conversation as resolved.
Show resolved Hide resolved

So is there any way to code just once, but can provide APIs in both gRPC and REST at the same time?
iamrajiv marked this conversation as resolved.
Show resolved Hide resolved

The answer is Yes.

The grpc-gateway is a plugin of the Google protocol buffers compiler [protoc](https://github.com/protocolbuffers/protobuf). It reads protobuf service definitions and generates a reverse-proxy server which translates a RESTful HTTP API into gRPC. This server is generated according to the [`google.api.http`](https://github.com/googleapis/googleapis/blob/master/google/api/http.proto#L46) annotations in your service definitions.

This helps you provide your APIs in both gRPC and RESTful style at the same time.
iamrajiv marked this conversation as resolved.
Show resolved Hide resolved

![architecture introduction diagram](https://docs.google.com/drawings/d/12hp4CPqrNPFhattL_cIoJptFvlAqm5wLQ0ggqI5mkCg/pub?w=749&h=370)

### Prerequisites

Before start coding, we have to install some tools and need to do small amount of configuration in the service.
iamrajiv marked this conversation as resolved.
Show resolved Hide resolved

We will be using a Go gRPC server in the examples, so please install Go first from [https://golang.org/dl/](https://golang.org/dl/).

After installing Go, use `go get` to download the following packages:

```sh
go get github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-grpc-gateway
go get google.golang.org/protobuf/cmd/protoc-gen-go
go get google.golang.org/grpc/cmd/protoc-gen-go-grpc
```

This installs the protoc generator plugins we need to generate the stubs. Make sure to add `$GOPATH/bin` to your `$PATH` so that executables installed via `go get` are available on your `$PATH`.
iamrajiv marked this conversation as resolved.
Show resolved Hide resolved
40 changes: 40 additions & 0 deletions docs/docs/tutorials/learn_more.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
---
layout: default
title: Learn More
parent: Tutorials
nav_order: 6
---

## Learn More

#### google.api.http

GRPC transcoding is a conversion function between the gRPC method and one or more HTTP REST endpoints. This allows developers to create a single API service that supports both the gRPC API and the REST API. Many systems, including the API Google, Cloud Endpoints, gRPC Gateway, and the Envoy proxy server support this feature and use it for large-scale production services.

The grcp-gateway the server is created according to the `google.api.http` annotations in your service definitions.

HttpRule defines the gRPC / REST mapping scheme. The mapping defines how different parts of a gRPC request message are mapped to the URL path, URL request parameters, and HTTP request body. It also controls how the gRPC response message is displayed in the HTTP response body. HttpRule is usually specified as a `google.api.http` annotation in the gRPC method.

Each mapping defines a URL path template and an HTTP method. A path template can refer to one or more fields in a gRPC request message if each field is a non-repeating field with a primitive type. The path template controls how the request message fields are mapped to the URL path.

```proto
import "google/api/annotations.proto";
import "google/protobuf/empty.proto";
import "google/protobuf/timestamp.proto";
message StatusResponse {
google.protobuf.Timestamp current_time = 1;
}
service MyService {
rpc Status(google.protobuf.Empty)
returns (StatusResponse) {
option (google.api.http) = {
get: "/status"
};
}
}
}
```

You will need to provide the necessary third-party `protobuf` files to the `protoc` compiler. They have included in the `grpc-gateway` repository in the `[third_party/googleapis](https://github.com/grpc-ecosystem/grpc-gateway/tree/master/third_party/googleapis)` folder, and we recommend copying them to the project file structure.
iamrajiv marked this conversation as resolved.
Show resolved Hide resolved

Read more about HTTP and gRPC Transcoding on https://google.aip.dev/127.
Loading