Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add mongodb #28

Merged
merged 4 commits into from
May 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions .github/workflows/mongodb.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
name: Mongodb Pull Request Check
on:
pull_request:
paths:
- 'mongodb/**'
- '.github/workflows/mongodb.yml'
push:
paths:
- 'mongodb/**'
- '.github/workflows/mongodb.yml'
workflow_dispatch:
permissions:
contents: read
jobs:
build:
name: build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-go@v4
with:
go-version: 1.19
- name: Build
run: cd mongodb && go build -v ./...
- name: Test
run: cd mongodb && go test -v -coverprofile=coverage.out -gcflags=all=-l -run Unit ./...
- name: Upload coverage reports to Codecov
uses: codecov/codecov-action@v3
with:
files: ./mongodb/coverage.out
flags: mongodb
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
6 changes: 4 additions & 2 deletions LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,9 @@ Source code of this software can be obtained from: github.com/golang/mock
Copyright 2018 by David A. Golden. All rights reserved.
Source code of this software can be obtained from: github.com/xdg-go/scram

3. mongo-driver
Copyright (C) MongoDB, Inc. 2017-present.
Source code of this software can be obtained from: github.com/mongodb/mongo-go-driver

Terms of the Apache License Version 2.0:
--------------------------------------------------------------------
Expand Down Expand Up @@ -157,7 +160,6 @@ Copyright (c) supermonkey original author and authors
6. cos-go-sdk-v5
Copyright (c) 2017 mozillazg


7. miniredis
Copyright (c) 2014 Harmen

Expand Down Expand Up @@ -187,7 +189,7 @@ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLI



Open Source Software Licensed under the MIT and Apache 2.0:
Open Source Software Licensed under the MIT and Apache 2.0:
--------------------------------------------------------------------
1. yaml.v3
Copyright 2011-2016 Canonical Ltd.
Expand Down
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,11 @@ Considering that tRPC-Go provides a variety of plugins for naming routing, monit
| bigcache | Wraps the open-source local caching database [Bigcache](https://github.com/allegro/bigcache) |
| clickhouse | Wraps the open-source database [Clickhouse SDK](https://github.com/ClickHouse/clickhouse-go) |
| cos | Wraps Tencent Cloud Object Storage [COS SDK](https://github.com/tencentyun/cos-go-sdk-v5) |
| goes | Wraps the open-source official Go [ElasticSearch client](https://github.com/elastic/go-elasticsearch) |
| goredis | Wraps the in-memory database [Redis SDK](https://github.com/redis/go-redis) |
| gorm | Wraps the Golang ORM library [GORM](https://github.com/go-gorm/gorm) |
| hbase | Wraps the open-source database [HBase SDK](https://github.com/tsuna/gohbase) |
| kafka | Wraps the open-source Kafka message queue SDK [Sarama](https://github.com/IBM/sarama) |
| mongodb | Wraps the open-source database [MongoDB Driver](https://go.mongodb.org/mongo-driver/mongo) |
| mysql | Wraps the open-source database [MySQL Driver](https://github.com/go-sql-driver/mysql) |
| goes | Wraps the open-source official go [ElasticSearch client](https://github.com/elastic/go-elasticsearch) |
| timer | Local/distributed timer functionality |
2 changes: 2 additions & 0 deletions README.zh_CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,11 @@
| bigcache | 封装开源本地缓存数据库 [Bigcache](https://github.com/allegro/bigcache) |
| clickhouse | 封装开源数据库 [Clickhouse SDK](https://github.com/ClickHouse/clickhouse-go) |
| cos | 封装腾讯云对象存储 [COS SDK](https://github.com/tencentyun/cos-go-sdk-v5) |
| goes | 封装开源官方 Go [ElasticSearch client](https://github.com/elastic/go-elasticsearch) |
| goredis | 封装内存数据库 [Redis SDK](https://github.com/redis/go-redis) |
| gorm | 封装 Golang ORM 库 [GORM](https://github.com/go-gorm/gorm) |
| hbase | 封装开源数据库 [HBase SDK](https://github.com/tsuna/gohbase) |
| kafka | 封装开源消息队列 Kafka SDK [Sarama](https://github.com/IBM/sarama) |
| mongodb | 封装开源数据库 [MongoDB Driver](https://go.mongodb.org/mongo-driver/mongo) |
| mysql | 封装开源数据库 [Mysql Driver](https://github.com/go-sql-driver/mysql) |
| timer | 本地/分布式定时器 |
1 change: 1 addition & 0 deletions mongodb/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# Change Log
97 changes: 97 additions & 0 deletions mongodb/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
English | [中文](README.zh_CN.md)

# tRPC-Go mongodb plugin
[![BK Pipelines Status](https://api.bkdevops.qq.com/process/api/external/pipelines/projects/pcgtrpcproject/p-d7b163d3830a429e976bf77e2409c6d3/badge?X-DEVOPS-PROJECT-ID=pcgtrpcproject)](http://devops.oa.com/ms/process/api-html/user/builds/projects/pcgtrpcproject/pipelines/p-d7b163d3830a429e976bf77e2409c6d3/latestFinished?X-DEVOPS-PROJECT-ID=pcgtrpcproject)

Base on community [mongo](https://go.mongodb.org/mongo-driver/mongo), used with trpc.

## mongodb client
```yaml
client: # Backend configuration for client calls.
service: # Configuration for the backend.
- name: trpc.mongodb.xxx.xxx
target: mongodb://user:passwd@vip:port # mongodb standard uri:mongodb://[username:password@]host1[:port1][,host2[:port2],...[,hostN[:portN]]][/[database][?options]]
timeout: 800 # The maximum processing time of the current request.
- name: trpc.mongodb.xxx.xxx1
target: mongodb+polaris://user:passwd@polaris_name # mongodb+polaris means that the host in the mongodb uri will perform Polaris analysis.
timeout: 800 # The maximum processing time of the current request.
```
```go
package main

import (
"context"

"go.mongodb.org/mongo-driver/bson"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
"trpc.group/trpc-go/trpc-database/mongodb"
"trpc.group/trpc-go/trpc-go/log"
)

// BattleFlow is battle information.
type BattleInfo struct {
Id string `bson:"_id,omitempty"`
Ctime uint32 `bson:"ctime,omitempty" json:"ctime,omitempty"`
}

func (s *server) SayHello(ctx context.Context, req *pb.ReqBody, rsp *pb.RspBody) (err error) {
proxy := mongodb.NewClientProxy("trpc.mongodb.xxx.xxx") // Your custom service name,used for monitoring, reporting and mapping configuration.

// mongodb insert
_, err = proxy.InsertOne(sc, "database", "table", bson.M{"_id": "key2", "value": "v2"})

// mongodb ReplaceOne
opts := options.Replace().SetUpsert(true)
filter := bson.D{{"_id", "key1"}}
_, err := proxy.ReplaceOne(ctx, "database", "table", filter, &BattleInfo{}, opts)
if err != nil {
log.Errorf("err=%v, data=%v", err, *battleInfo)
return err
}

// mongodb FindOne
rst := proxy.FindOne(ctx, "database", "table", bson.D{{"_id", "key1"}})
battleInfo = &BattleInfo{}
err = rst.Decode(battleInfo)
if err != nil {
return nil, err
}

// mongodb transaction
err = proxy.Transaction(ctx, func(sc mongo.SessionContext) error {
// The same proxy instance needs to be used during transaction execution.
_, tErr := proxy.InsertOne(sc, "database", "table", bson.M{"_id": "key1", "value": "v1"})
if tErr != nil {
return tErr
}
_, tErr = proxy.InsertOne(sc, "database", "table", bson.M{"_id": "key2", "value": "v2"})
if tErr != nil {
return tErr
}
return nil
}, nil)

// mongodb RunCommand
cmdDB := bson.D{}
cmdDB = append(cmdDB, bson.E{Key: "enableSharding", Value: "dbName"})
err = proxy.RunCommand(ctx, "admin", cmdDB).Err()
if err != nil {
return nil, err
}

cmdColl := bson.D{}
cmdColl = append(cmdColl, bson.E{Key: "shardCollection", Value: "dbName.collectionName"})
cmdColl = append(cmdColl, bson.E{Key: "key", Value: bson.D{{"openId", "hashed"}}})
cmdColl = append(cmdColl, bson.E{Key: "unique", Value: false})
cmdColl = append(cmdColl, bson.E{Key: "numInitialChunks", Value: 10})
err = proxy.RunCommand(ctx, "admin", cmdColl).Err()
if err != nil {
return nil, err
}
// Business logic.
}
```
## Frequently Asked Questions (FAQs)
- Q1: How to configure ClientOptions:
- A1: When creating a Transport, you can use WithOptionInterceptor to configure ClientOptions. You can refer to options_test.go for more information.
94 changes: 94 additions & 0 deletions mongodb/README.zh_CN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
# tRPC-Go mongodb 插件
[![BK Pipelines Status](https://api.bkdevops.qq.com/process/api/external/pipelines/projects/pcgtrpcproject/p-d7b163d3830a429e976bf77e2409c6d3/badge?X-DEVOPS-PROJECT-ID=pcgtrpcproject)](http://devops.oa.com/ms/process/api-html/user/builds/projects/pcgtrpcproject/pipelines/p-d7b163d3830a429e976bf77e2409c6d3/latestFinished?X-DEVOPS-PROJECT-ID=pcgtrpcproject)

封装社区的 [mongo](https://go.mongodb.org/mongo-driver/mongo) ,配合 trpc 使用。

## mongodb client
```yaml
client: #客户端调用的后端配置
service: #针对后端的配置
- name: trpc.mongodb.xxx.xxx
target: mongodb://user:passwd@vip:port #mongodb 标准uri:mongodb://[username:password@]host1[:port1][,host2[:port2],...[,hostN[:portN]]][/[database][?options]]
timeout: 800 #当前这个请求最长处理时间
- name: trpc.mongodb.xxx.xxx1
target: mongodb+polaris://user:passwd@polaris_name # mongodb+polaris表示mongodb uri中的host会进行北极星解析
timeout: 800 # 当前这个请求最长处理时间
```
```go
package main

import (
"time"
"context"

"trpc.group/trpc-go/trpc-database/mongodb"
"trpc.group/trpc-go/trpc-go/client"
)

// BattleFlow 对局信息
type BattleInfo struct {
Id string `bson:"_id,omitempty" `
Ctime uint32 `bson:"ctime,omitempty" json:"ctime,omitempty"`
}

func (s *server) SayHello(ctx context.Context, req *pb.ReqBody, rsp *pb.RspBody) (err error) {
proxy := mongodb.NewClientProxy("trpc.mongodb.xxx.xxx") // service name自己随便填,主要用于监控上报和寻址配置项

// mongodb insert
_, err = proxy.InsertOne(sc, "database", "table", bson.M{"_id": "key2", "value": "v2"})

// mongodb ReplaceOne
opts := options.Replace().SetUpsert(true)
filter := bson.D{{"_id", "key1"}}
_, err := proxy.ReplaceOne(ctx, "database", "table", filter, &BattleInfo{}, opts)
if err != nil {
log.Errorf("err=%v, data=%v", err, *battleInfo)
return err
}

// mongodb FindOne
rst := proxy.FindOne(ctx, "database", "table", bson.D{{"_id", "key1"}})
battleInfo = &BattleInfo{}
err = rst.Decode(battleInfo)
if err != nil {
return nil, err
}

// mongodb transaction
err = proxy.Transaction(ctx, func(sc mongo.SessionContext) error {
//事务执行过程中需要使用同一proxy实例执行
_, tErr := proxy.InsertOne(sc, "database", "table", bson.M{"_id": "key1", "value": "v1"})
if tErr != nil {
return tErr
}
_, tErr = proxy.InsertOne(sc, "database", "table", bson.M{"_id": "key2", "value": "v2"})
if tErr != nil {
return tErr
}
return nil
}, nil)

// mongodb RunCommand
cmdDB := bson.D{}
cmdDB = append(cmdDB, bson.E{Key: "enableSharding", Value: "dbName"})
err = proxy.RunCommand(ctx, "admin", cmdDB).Err()
if err != nil {
return nil, err
}

cmdColl := bson.D{}
cmdColl = append(cmdColl, bson.E{Key: "shardCollection", Value: "dbName.collectionName"})
cmdColl = append(cmdColl, bson.E{Key: "key", Value: bson.D{{"openId", "hashed"}}})
cmdColl = append(cmdColl, bson.E{Key: "unique", Value: false})
cmdColl = append(cmdColl, bson.E{Key: "numInitialChunks", Value: 10})
err = proxy.RunCommand(ctx, "admin", cmdColl).Err()
if err != nil {
return nil, err
}
// 业务逻辑
}
```
## 常见问题

- Q1: 如何配置 ClientOptions
- A1: 创建Transport时可以使用WithOptionInterceptor对ClientOptions进行配置,可以参考options_test.go
Loading
Loading