Skip to content

Commit

Permalink
添加自定义vpc额外公网网卡功能介绍 (#134)
Browse files Browse the repository at this point in the history 
* 添加自定义vpc额外公网网卡功能介绍

Signed-off-by: zcq98 <zhaocongqi_yewu@cmss.chinamobile.com>

* add ovn-eip-fip-snat english doc

Signed-off-by: zcq98 <zhaocongqi_yewu@cmss.chinamobile.com>

---------

Signed-off-by: zcq98 <zhaocongqi_yewu@cmss.chinamobile.com>
Co-authored-by: zcq98 <zhaocongqi_yewu@cmss.chinamobile.com>
  • Loading branch information
@zhaocongqi
zhaocongqi and zhaocongqi authored Nov 8, 2023
1 parent ecb90cb commit a79e8ed
Show file tree
Hide file tree
Showing 2 changed files with 202 additions and 3 deletions.
103 changes: 101 additions & 2 deletions docs/advance/ovn-eip-fip-snat.en.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,7 @@ The CRD supported by this function is basically the same as the iptable nat gw p

## 1. Deployment

Currently allows all vpcs to share the same provider vlan subnet resources, similar to neutron ovn mode.
Compatible with previous scenarios [default VPC EIP/SNAT](../guide/eip-snat.en.md).
Currently allows all vpcs to share the same default provider vlan subnet resources, custom vpcs support extending provider vlan subnet to enable the use of multiple public networks. similar to neutron ovn mode. Compatible with previous scenarios [default VPC EIP/SNAT](../guide/eip-snat.en.md).

During the deployment phase, you may need to specify a default public network logical switch based on actual conditions.
If no vlan is in use (vlan 0 is used), the following startup parameters do not need to be configured.
Expand Down Expand Up @@ -130,6 +129,13 @@ Of course, you can also manually create the lrp type ovn eip in advance.

### 1.3 Custom vpc enable eip snat fip function

Clusters generally require multiple gateway nodes to achieve high availability. The configuration is as follows:

```bash
# First specify external-gw-nodes by adding label
kubectl label nodes pc-node-1 pc-node-2 pc-node-3 ovn.kubernetes.io/external-gw=true
```

``` bash
# cat 00-ns.yml

Expand Down Expand Up @@ -202,6 +208,91 @@ Route Table <main>:
# The route currently supports automatic maintenance
```

### 1.4 Use additional public network

#### 1.4.1 Create additional underlay public network

Additional public network functions will be enabled after the default eip snat fip function is enabled. If there is only 1 public network card, please use the default eip snat fip function.

```yaml
# provider-network, vlan, subnet
# cat 01-extra-provider-network.yaml
apiVersion: kubeovn.io/v1
kind: ProviderNetwork
metadata:
name: extra
spec:
defaultInterface: vlan
# cat 02-extra-vlan.yaml
apiVersion: kubeovn.io/v1
kind: Vlan
metadata:
name: vlan0
spec:
id: 0
provider: extra
# cat 03-extra-vlan-subnet.yaml
apiVersion: kubeovn.io/v1
kind: Subnet
metadata:
name: extra
spec:
protocol: IPv4
cidrBlock: 10.10.204.0/24
gateway: 10.10.204.254
vlan: vlan0
excludeIps:
- 10.10.204.1..10.10.204.100
```
#### 1.4.2 Custom vpc configuration
```yaml
apiVersion: kubeovn.io/v1
kind: Vpc
metadata:
name: vpc1
spec:
namespaces:
- vpc1
staticRoutes: # configure routing rules: Which additional public network routes a subnet under the vpc needs to be based on needs to be added manually. The following example is for reference only. Users need to configure it according to their actual situation.
- cidr: 192.168.0.1/28
nextHopIP: 10.10.204.254
policy: policySrc
enableExternal: true # vpc enableExternal will automatically create an lrp association to the public network specified above
addExternalSubnets: # configure addExternalSubnets to support connecting multiple additional public networks
- extra
```
After the above template is applied, you should see the following resources exist
```yaml
# k ko nbctl show vpc1
router 87ad06fd-71d5-4ff8-a1f0-54fa3bba1a7f (vpc1)
port vpc1-vpc1-subnet1
mac: "00:00:00:ED:8E:C7"
networks: ["192.168.0.1/24"]
port vpc1-external204
mac: "00:00:00:EF:05:C7"
networks: ["10.5.204.105/24"]
gateway chassis: [7cedd14f-265b-42e5-ac17-e03e7a1f2342 276baccb-fe9c-4476-b41d-05872a94976d fd9f140c-c45d-43db-a6c0-0d4f8ea298dd]
port vpc1-extra
mac: "00:00:00:EF:6A:C7"
networks: ["10.10.204.105/24"]
gateway chassis: [7cedd14f-265b-42e5-ac17-e03e7a1f2342 276baccb-fe9c-4476-b41d-05872a94976d fd9f140c-c45d-43db-a6c0-0d4f8ea298dd]
```
```bash
# k ko nbctl lr-route-list vpc1
IPv4 Routes
Route Table <main>:
192.168.0.1/28 10.10.204.254 src-ip
0.0.0.0/0 10.5.204.254 dst-ip
# The route currently supports automatic maintenance
# Additional public networks require manual routing configuration in the vpc. In the above example, the source IP address is 192.168.0.1/28 and will be forwarded to the additional public network.
# Users can manually configure routing rules according to the situation
```

## 2. ovn-eip

This function is designed and used in the same way as iptables-eip, ovn-eip currently has three types
Expand All @@ -224,6 +315,8 @@ spec:
# Dynamically allocate an eip resource that is reserved for fip dnat_and_snat scenarios
```

When an additional public network is configured, you can specify the public network that needs to be expanded through externalSubnet. In the above configuration, external204 and extra are optional.

### 2.1 Create an fip for pod

``` bash
Expand Down Expand Up @@ -450,6 +543,8 @@ spec:

```

When an additional public network is configured, you can specify the public network that needs to be expanded through externalSubnet. In the above configuration, external204 and extra are optional.

### 3.2 ovn-snat corresponds to a pod IP

This feature is designed and used in much the same way as iptables-snat
Expand Down Expand Up @@ -488,6 +583,8 @@ spec:

```

When an additional public network is configured, you can specify the public network that needs to be expanded through externalSubnet. In the above configuration, external204 and extra are optional.

After the above resources are created, you can see the following resources that the snat public network feature depends on.

``` bash
Expand Down Expand Up @@ -613,6 +710,8 @@ spec:

```

When an additional public network is configured, you can specify the public network that needs to be expanded through externalSubnet. In the above configuration, external204 and extra are optional.

The configuration of OvnDnatRule is similar to that of IptablesDnatRule.

```bash
Expand Down
102 changes: 101 additions & 1 deletion docs/advance/ovn-eip-fip-snat.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ Pod 基于分布式网关 FIP (dnat_and_snat) 出公网的大致流程，最后

## 1. 部署

目前允许所有（默认以及自定义）vpc 使用同一个 provider vlan subnet 资源，同时兼容[默认 VPC EIP/SNAT](../guide/eip-snat.md)的场景。
目前允许所有（默认以及自定义） vpc 使用同一个默认 provider vlan subnet 资源，同时自定义 vpc 支持扩展 provider vlan subnet 从而实现使用多个公网，兼容[默认 VPC EIP/SNAT](../guide/eip-snat.md)的场景。

类似 neutron ovn，服务启动配置中需要指定 provider network 相关的配置，下述的启动参数也是为了兼容 VPC EIP/SNAT 的实现。

Expand Down Expand Up @@ -125,6 +125,13 @@ data:

### 1.3 自定义 vpc 启用 eip snat fip 功能

集群一般需要多个网关 node 来实现高可用，配置如下：

```bash
# 首先通过添加标签指定 external-gw-nodes
kubectl label nodes pc-node-1 pc-node-2 pc-node-3 ovn.kubernetes.io/external-gw=true
```

``` bash
# cat 00-ns.yml

Expand Down Expand Up @@ -199,6 +206,91 @@ Route Table <main>:
# 目前该路由已自动维护
```

### 1.4 使用额外的公网网络

#### 1.4.1 准备额外 underlay 公网网络

额外的公网网络功能在启动默认 eip snat fip 功能后才会启用，若只有 1 个公网网卡，请使用默认 eip snat fip 功能

```yaml
# 准备 provider-network， vlan， subnet
# cat 01-extra-provider-network.yaml
apiVersion: kubeovn.io/v1
kind: ProviderNetwork
metadata:
name: extra
spec:
defaultInterface: vlan
# cat 02-extra-vlan.yaml
apiVersion: kubeovn.io/v1
kind: Vlan
metadata:
name: vlan0
spec:
id: 0
provider: extra
# cat 03-extra-vlan-subnet.yaml
apiVersion: kubeovn.io/v1
kind: Subnet
metadata:
name: extra
spec:
protocol: IPv4
cidrBlock: 10.10.204.0/24
gateway: 10.10.204.254
vlan: vlan0
excludeIps:
- 10.10.204.1..10.10.204.100
```
#### 1.4.2 自定义 vpc 配置
```yaml
apiVersion: kubeovn.io/v1
kind: Vpc
metadata:
name: vpc1
spec:
namespaces:
- vpc1
staticRoutes: # 配置路由规则：vpc下的某个子网需要基于哪一个额外的公网网络的路由需要手动添加，以下示例仅供参考，用户需根据自己的实际情况进行配置
- cidr: 192.168.0.1/28
nextHopIP: 10.10.204.254
policy: policySrc
enableExternal: true # 开启enableExternal后vpc会自动连接名为external的ls
addExternalSubnets: # 配置addExternalSubnets支持连接多个额外的公网网络
- extra
```
以上模板应用后，应该可以看到如下资源存在
```bash
# k ko nbctl show vpc1
router 87ad06fd-71d5-4ff8-a1f0-54fa3bba1a7f (vpc1)
port vpc1-vpc1-subnet1
mac: "00:00:00:ED:8E:C7"
networks: ["192.168.0.1/24"]
port vpc1-external204
mac: "00:00:00:EF:05:C7"
networks: ["10.5.204.105/24"]
gateway chassis: [7cedd14f-265b-42e5-ac17-e03e7a1f2342 276baccb-fe9c-4476-b41d-05872a94976d fd9f140c-c45d-43db-a6c0-0d4f8ea298dd]
port vpc1-extra
mac: "00:00:00:EF:6A:C7"
networks: ["10.10.204.105/24"]
gateway chassis: [7cedd14f-265b-42e5-ac17-e03e7a1f2342 276baccb-fe9c-4476-b41d-05872a94976d fd9f140c-c45d-43db-a6c0-0d4f8ea298dd]
```
```bash
# k ko nbctl lr-route-list vpc1
IPv4 Routes
Route Table <main>:
192.168.0.1/28 10.10.204.254 src-ip
0.0.0.0/0 10.5.204.254 dst-ip
# 目前会为默认公网网络配置默认路由
# 额外公网网络需要在vpc手动配置路由，上述实例中源IP地址为192.168.0.1/28会转发至额外公网网络
# 用户可根据情况手动配置路由规则
```

## 2. ovn-eip

该功能和 iptables-eip 设计和使用方式基本一致，ovn-eip 目前有三种 type
Expand All @@ -220,6 +312,8 @@ spec:
# 动态分配一个 eip 资源，该资源预留用于 fip 场景
```

当配置了额外公网网络时，可以通过 externalSubnet 指定需要扩展使用的公网网络，在上述配置中，可选 external204 和 extra 两个公网网络

### 2.1 ovn-fip 为 pod 绑定一个 fip

``` bash
Expand Down Expand Up @@ -443,6 +537,8 @@ spec:

```

当配置了额外公网网络时，可以通过 externalSubnet 指定需要扩展使用的公网网络，在上述配置中，可选 external204 和 extra 两个公网网络

### 3.2 ovn-snat 对应到一个 pod ip

该功能和 iptables-snat 设计和使用方式基本一致
Expand Down Expand Up @@ -482,6 +578,8 @@ spec:

```

当配置了额外公网网络时，可以通过 externalSubnet 指定需要扩展使用的公网网络，在上述配置中，可选 external204 和 extra 两个公网网络。

以上资源创建后，可以看到 snat 公网功能依赖的如下资源。

``` bash
Expand Down Expand Up @@ -608,6 +706,8 @@ spec:

```

当配置了额外公网网络时，可以通过 externalSubnet 指定需要扩展使用的公网网络，在上述配置中，可选 external204 和 extra 两个公网网络

OvnDnatRule 的配置与 IptablesDnatRule 类似

```bash
Expand Down

0 comments on commit a79e8ed

Please sign in to comment.