Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(generator): generate new methods for existing APIs #784

Merged
merged 14 commits into from
Sep 27, 2022
30 changes: 30 additions & 0 deletions cmd/layotto_multiple_api/advanced_queue/advanced_queue.proto
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
syntax = "proto3";

option go_package = "github.com/mosn/layotto/cmd/layotto_multiple_api/advanced_queue;advanced_queue";

package cmd.layotto_multiple_api.advanced_queue;

/* @exclude skip sdk_generator */
/* @exclude extends pub_subs */
// AdvancedQueue is advanced pubsub API
service AdvancedQueue {

// Publish a transactional message
rpc PublishTransactionalMessage(TransactionalMessageRequest) returns (TransactionalMessageResponse);

}

// TransactionalMessageRequest
message TransactionalMessageRequest {
// Required
string store_name = 1;
// Required
string content = 2;
}

// TransactionalMessageResponse
message TransactionalMessageResponse {

// message_id is identifier of a message
string message_id = 1;
}
4 changes: 3 additions & 1 deletion docs/_sidebar.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,9 @@
- [Hello World](en/start/rpc/helloworld.md)
- [Dubbo JSON RPC](en/start/rpc/dubbo_json_rpc.md)
- [Use OSS API](en/start/oss/start.md)
- [API plugin: register your own API](en/start/api_plugin/helloworld.md)
- API plugin
- [Register your own API](en/start/api_plugin/helloworld.md)
- [Generate API plugin automatically](en/start/api_plugin/generate.md)
<!--quickstart_generator-->
- [(Under construction) Use phone API](en/start/phone/start)
- [(Under construction) Use email API](en/start/email/start)
Expand Down
2 changes: 0 additions & 2 deletions docs/en/api_reference/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,6 @@ These protos define Layotto's runtime API, including:

In addition to this, Layotto also provides some extension APIs, including:



email: [spec/proto/extension/v1/email](https://mosn.io/layotto/api/v1/email.html)

phone: [spec/proto/extension/v1/phone](https://mosn.io/layotto/api/v1/phone.html)
Expand Down
10 changes: 6 additions & 4 deletions docs/en/api_reference/how_to_generate_api_doc.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,21 +39,23 @@ service EmailService2 {
// different message types......
```

- If you don't want to generate the quickstart docs for the proto, add a comment `/* @exclude quickstart generator */` .
- If you don't want to generate the sdk & sidecar code for the proto, add a comment `/* @exclude code generator */` .
- If you don't want to generate the quickstart docs for the proto, add a comment `/* @exclude skip quickstart_generator */` .
- If you don't want to generate the sdk & sidecar code for the proto, add a comment `/* @exclude skip code_generator */` .

You can take the `spec/proto/extension/v1/s3/oss.proto` as an example:

```protobuf
/* @exclude quickstart generator */
/* @exclude code generator */
/* @exclude skip quickstart_generator */
/* @exclude skip code_generator */
// ObjectStorageService is an abstraction for blob storage or so called "object storage", such as alibaba cloud OSS, such as AWS S3.
// You invoke ObjectStorageService API to do some CRUD operations on your binary file, e.g. query my file, delete my file, etc.
service ObjectStorageService{
//......
}
```

These special comments are called "Master's commands". There are many other commands, and you can check [the doc](https://github.com/seeflood/protoc-gen-p6#masters-commands) for more details.

## step 2. Check the environment
To run the generator, you need:
- Go version >=1.16
Expand Down
50 changes: 50 additions & 0 deletions docs/en/start/api_plugin/generate.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# Generate API plugin automatically

Writing the API plugin yourself is boring. You can use code generator to generate all the code.

Let's say you want to add a `PublishTransactionalMessage` method to existing pubsub API. You write a new proto file `cmd/layotto_multiple_api/advanced_queue/advanced_queue.proto` :

```protobuf
// ......
/* @exclude extends pub_subs */
// AdvancedQueue is advanced pubsub API
service AdvancedQueue {

rpc PublishTransactionalMessage(TransactionalMessageRequest) returns (TransactionalMessageResponse);

}

message TransactionalMessageRequest {
string store_name = 1;

string content = 2;
}

message TransactionalMessageResponse {
string message_id = 1;
}

```

and run the generator:

```protobuf
protoc -I . \
--go_out . --go_opt=paths=source_relative \
--go-grpc_out=. \
--go-grpc_opt=require_unimplemented_servers=false,paths=source_relative \
--p6_out ./cmd/layotto_multiple_api/advanced_queue --p6_opt=paths=source_relative \
cmd/layotto_multiple_api/advanced_queue/advanced_queue.proto
```

then you get the code:

<img src="https://user-images.githubusercontent.com/26001097/189822603-c4c9d0c6-86a1-4a66-bba8-3d01758808e7.png" width="30%" height="30%" />

Fix the path error and then you can register this API plugin in you `main`.

## Reference

[How to generate code and documentation from the .proto files](en/api_reference/how_to_generate_api_doc)

[protoc-gen-p6](https://github.com/seeflood/protoc-gen-p6)
4 changes: 3 additions & 1 deletion docs/zh/_sidebar.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,9 @@
- [(建设中)使用 phone API](zh/start/phone/start)
- [(建设中)使用 email API](zh/start/email/start)
- [使用 lifecycle API](zh/start/lifecycle/start)
- [API插件:注册您自己的API](zh/start/api_plugin/helloworld.md)
- API插件
- [注册您自己的API](zh/start/api_plugin/helloworld.md)
- [自动生成 API 插件](zh/start/api_plugin/generate.md)
- 作为 Istio 的数据面
- [集成 Istio 1.10.6 演示](zh/start/istio/)
- [集成 Istio 1.5.x 演示](zh/start/istio/start.md)
Expand Down
2 changes: 0 additions & 2 deletions docs/zh/api_reference/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,6 @@ Layotto 有多个 gRPC proto 文件, 对应的接口文档在:

除此之外,Layotto 还提供了一些扩展 API,包括:



email: [spec/proto/extension/v1/email](https://mosn.io/layotto/api/v1/email.html)

phone: [spec/proto/extension/v1/phone](https://mosn.io/layotto/api/v1/phone.html)
Expand Down
10 changes: 6 additions & 4 deletions docs/zh/api_reference/how_to_generate_api_doc.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,21 +39,23 @@ service EmailService2 {
// different message types......
```

- If you don't want to generate the quickstart docs for the proto, add a comment `/* @exclude quickstart generator */` .
- If you don't want to generate the sdk & sidecar code for the proto, add a comment `/* @exclude code generator */` .
- If you don't want to generate the quickstart docs for the proto, add a comment `/* @exclude skip quickstart_generator */` .
- If you don't want to generate the sdk & sidecar code for the proto, add a comment `/* @exclude skip code_generator */` .

You can take the `spec/proto/extension/v1/s3/oss.proto` as an example:

```protobuf
/* @exclude quickstart generator */
/* @exclude code generator */
/* @exclude skip quickstart_generator */
/* @exclude skip code_generator */
// ObjectStorageService is an abstraction for blob storage or so called "object storage", such as alibaba cloud OSS, such as AWS S3.
// You invoke ObjectStorageService API to do some CRUD operations on your binary file, e.g. query my file, delete my file, etc.
service ObjectStorageService{
//......
}
```

These special comments are called "Master's commands". There are many other commands, and you can check [the doc](https://github.com/seeflood/protoc-gen-p6#masters-commands) for more details.

## step 2. Check the environment
To run the generator, you need:
- Go version >=1.16
Expand Down
50 changes: 50 additions & 0 deletions docs/zh/start/api_plugin/generate.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# 自动生成 API 插件

Writing the API plugin yourself is boring. You can use code generator to generate all the code.

Let's say you want to add a `PublishTransactionalMessage` method to existing pubsub API. You write a new proto file `cmd/layotto_multiple_api/advanced_queue/advanced_queue.proto` :

```protobuf
// ......
/* @exclude extends pub_subs */
// AdvancedQueue is advanced pubsub API
service AdvancedQueue {

rpc PublishTransactionalMessage(TransactionalMessageRequest) returns (TransactionalMessageResponse);

}

message TransactionalMessageRequest {
string store_name = 1;

string content = 2;
}

message TransactionalMessageResponse {
string message_id = 1;
}

```

and run the generator:

```protobuf
protoc -I . \
--go_out . --go_opt=paths=source_relative \
--go-grpc_out=. \
--go-grpc_opt=require_unimplemented_servers=false,paths=source_relative \
--p6_out ./cmd/layotto_multiple_api/advanced_queue --p6_opt=paths=source_relative \
cmd/layotto_multiple_api/advanced_queue/advanced_queue.proto
```

then you get the code:

<img src="https://user-images.githubusercontent.com/26001097/189822603-c4c9d0c6-86a1-4a66-bba8-3d01758808e7.png" width="30%" height="30%" />

Fix the path error and then you can register this API plugin in you `main`.

## Reference

[How to generate code and documentation from the .proto files](zh/api_reference/how_to_generate_api_doc)

[protoc-gen-p6](https://github.com/seeflood/protoc-gen-p6)
4 changes: 3 additions & 1 deletion docs/zh/start/api_plugin/helloworld.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,4 +59,6 @@ Greeting: Hello world

您可以参考演示的代码来实现你自己的API。快来试试吧!

想要了解更多的详情,您可以参考[设计文档](zh/design/api_plugin/design.md)
想要了解更多的详情,您可以参考[设计文档](zh/design/api_plugin/design.md)

为了简化 API 插件的开发,Layotto 社区提供了一套代码生成器,可以基于 proto 文件生成 API 插件相关代码,见 [文档](zh/start/api_plugin/generate.md)
2 changes: 1 addition & 1 deletion etc/script/generate-code.sh
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ needGenerate() {
file=$1

# check no `@exclude` tag
if [ $(grep "@exclude code generator" $file | wc -l) -eq 0 ]; then
if [ $(grep "@exclude skip code_generator" $file | wc -l) -eq 0 ]; then
# check if there's a gRPC service in it
if [ $(grep "service " $file | wc -l) -gt 0 ]; then
return $true
Expand Down
2 changes: 1 addition & 1 deletion etc/script/generate-doc.sh
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ needGenerateQuickstart() {
file=$1

# check no `@exclude` tag
if [ $(grep "@exclude quickstart generator" $file | wc -l) -eq 0 ]; then
if [ $(grep "@exclude skip quickstart_generator" $file | wc -l) -eq 0 ]; then
# check if there's a gRPC service in it
if [ $(grep "service " $file | wc -l) -gt 0 ]; then
return $true
Expand Down
30 changes: 30 additions & 0 deletions pkg/runtime/extension_api_generated.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

11 changes: 0 additions & 11 deletions pkg/runtime/options_generated.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions spec/proto/extension/v1/s3/oss.proto
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ option go_package = "mosn.io/layotto/spec/proto/extension/v1/s3;s3";
option java_outer_classname = "ObjectStorageProto";
option java_package = "spec.proto.extension.v1.s3";

/* @exclude quickstart generator */
/* @exclude code generator */
/* @exclude skip quickstart_generator */
/* @exclude skip code_generator */
// ObjectStorageService is an abstraction for blob storage or so called "object storage", such as alibaba cloud OSS, such as AWS S3.
// You invoke ObjectStorageService API to do some CRUD operations on your binary file, e.g. query my file, delete my file, etc.
service ObjectStorageService{
Expand Down
2 changes: 1 addition & 1 deletion spec/proto/runtime/v1/appcallback.proto
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ option go_package = "mosn.io/layotto/spec/proto/runtime/v1;runtime";
option java_outer_classname = "AppCallbackProto";
option java_package = "spec.proto.runtime.v1";

/* @exclude quickstart generator */
/* @exclude skip quickstart_generator */
// AppCallback V1 allows user application to interact with runtime.
// User application needs to implement AppCallback service if it needs to
// receive message from runtime.
Expand Down
2 changes: 1 addition & 1 deletion spec/proto/runtime/v1/runtime.proto
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ option go_package = "mosn.io/layotto/spec/proto/runtime/v1;runtime";
option java_outer_classname = "RuntimeProto";
option java_package = "spec.proto.runtime.v1";

/* @exclude quickstart generator */
/* @exclude skip quickstart_generator */
// Runtime encapsulates variours Runtime APIs(such as Configuration API, Pub/Sub API, etc)
service Runtime {
//SayHello used for test
Expand Down