Skip to content

Commit

Permalink
Add more remote vlan examples (#4433) (#4465)
Browse files Browse the repository at this point in the history
- The existing suite was broken into two parts:
  - internal
  - vlan breaktrough
- Added OVS-forwarder example
- Added multiple NS example

Signed-off-by: Laszlo Kiraly <laszlo.kiraly@est.tech>

Co-authored-by: Szilard Vincze <szilard@est.home>
  • Loading branch information
ljkiraly and Szilard Vincze authored Feb 28, 2022
1 parent df76a41 commit 17c1f2f
Show file tree
Hide file tree
Showing 18 changed files with 971 additions and 139 deletions.
9 changes: 0 additions & 9 deletions apps/nse-remote-vlan/nse-remote-vlan.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -21,22 +21,13 @@ spec:
imagePullPolicy: IfNotPresent
ports:
- containerPort: 5003
hostPort: 5003
env:
- name: SPIFFE_ENDPOINT_SOCKET
value: unix:///run/spire/sockets/agent.sock
- name: NSM_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: NSM_CONNECT_TO
value: "registry:5002"
- name: NSM_SERVICES
value: "finance-bridge { vlan: 100; via: gw1}"
- name: NSM_CIDR_PREFIX
value: "172.10.0.0/24"
- name: NSM_IPV6_PREFIX
value: "100:200::/64"
- name: NSM_POINT2POINT
value: "False"
- name: NSM_REGISTER_SERVICE
Expand Down
149 changes: 88 additions & 61 deletions examples/remotevlan/README.md
Original file line number Diff line number Diff line change
@@ -1,80 +1,107 @@
# NSM Remote Vlan Examples

This setup can be used to check remote vlan mechanism. Contain basic setup for NSM that includes `nsmgr`, `registry-k8s`, `admission-webhook-k8s`, `nse-remote-vlan`. The `nse-remote-vlan` belongs to the nsm-system since does not have role in data-plane connection.
This setup can be used to check remote vlan mechanism with both OVS and VPP forwarder. Contain basic setup for NSM that includes `nsmgr`, `registry-k8s`, `admission-webhook-k8s`, `nse-remote-vlan`. The `nse-remote-vlan` belongs to the nsm-system since does not have role in data-plane connection.

## Requires

- [spire](../spire)

## Includes

- [Remote VLAN mechanism using forwarder-ovs](./rvlanovs)
- [Remote VLAN mechanism using forwarder-vpp](./rvlanvpp)

## Run

1. Create secondary bridge network and connect kind-worker nodes:

```bash
docker network create bridge-2
docker network connect bridge-2 kind-worker
docker network connect bridge-2 kind-worker2
```

2. Rename the newly generated interface to eth1 in both kind-workers:

```bash
ifw1=$(echo $(docker exec kind-worker ip link | tail -2 | head -1) | cut -f1 -d"@" | cut -f2 -d" ")
docker exec kind-worker ip link set $ifw1 down
docker exec kind-worker ip link set $ifw1 name eth1
docker exec kind-worker ip link set eth1 up
ifw2=$(echo $(docker exec kind-worker2 ip link | tail -2 | head -1) | cut -f1 -d"@" | cut -f2 -d" ")
docker exec kind-worker2 ip link set $ifw2 down
docker exec kind-worker2 ip link set $ifw2 name eth1
docker exec kind-worker2 ip link set eth1 up
```

3. Create ns for deployments:

```bash
kubectl create ns nsm-system
```

4. Apply NSM resources for basic tests:

```bash
kubectl apply -k .
```

5. Wait for NSE application:

```bash
kubectl -n nsm-system wait --for=condition=ready --timeout=2m pod -l app=nse-remote-vlan
```

6. Wait for admission-webhook-k8s:

```bash
WH=$(kubectl get pods -l app=admission-webhook-k8s -n nsm-system --template '{{range .items}}{{.metadata.name}}{{"\n"}}{{end}}')
kubectl wait --for=condition=ready --timeout=1m pod ${WH} -n nsm-system
```
Create secondary bridge network and connect kind-worker nodes:

```bash
docker network create bridge-2
docker network connect bridge-2 kind-worker
docker network connect bridge-2 kind-worker2
```

Rename the newly generated interface to eth1 in both kind-workers:

```bash
ifw1=$(echo $(docker exec kind-worker ip link | tail -2 | head -1) | cut -f1 -d"@" | cut -f2 -d" ")
docker exec kind-worker ip link set $ifw1 down
docker exec kind-worker ip link set $ifw1 name eth1
docker exec kind-worker ip link set eth1 up
ifw2=$(echo $(docker exec kind-worker2 ip link | tail -2 | head -1) | cut -f1 -d"@" | cut -f2 -d" ")
docker exec kind-worker2 ip link set $ifw2 down
docker exec kind-worker2 ip link set $ifw2 name eth1
docker exec kind-worker2 ip link set eth1 up
```

Create ns for deployments:

```bash
kubectl create ns nsm-system
```

Create NSE patch:

```bash
cat > patch-nse.yaml <<EOF
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nse-remote-vlan
spec:
template:
spec:
containers:
- name: nse
env:
- name: NSM_CONNECT_TO
value: "registry:5002"
- name: NSM_SERVICES
value: "finance-bridge { vlan: 100; via: gw1}"
- name: NSM_CIDR_PREFIX
value: "172.10.0.0/24"
- name: NSM_IPV6_PREFIX
value: "100:200::/64"
EOF
```

Apply NSM resources for basic tests:

```bash
kubectl apply -k .
```

Wait for NSE application:

```bash
kubectl -n nsm-system wait --for=condition=ready --timeout=2m pod -l app=nse-remote-vlan
```

Wait for admission-webhook-k8s:

```bash
WH=$(kubectl get pods -l app=admission-webhook-k8s -n nsm-system --template '{{range .items}}{{.metadata.name}}{{"\n"}}{{end}}')
kubectl wait --for=condition=ready --timeout=1m pod ${WH} -n nsm-system
```

## Cleanup

1. To free resources follow the next command:
To free resources follow the next command:

```bash
WH=$(kubectl get pods -l app=admission-webhook-k8s -n nsm-system --template '{{range .items}}{{.metadata.name}}{{"\n"}}{{end}}')
kubectl delete mutatingwebhookconfiguration ${WH}
kubectl delete ns nsm-system
```
```bash
WH=$(kubectl get pods -l app=admission-webhook-k8s -n nsm-system --template '{{range .items}}{{.metadata.name}}{{"\n"}}{{end}}')
kubectl delete mutatingwebhookconfiguration ${WH}
kubectl delete ns nsm-system
```

2. Delete secondary network and kind-worker node connections:
Delete secondary network and kind-worker node connections:

```bash
docker network disconnect bridge-2 kind-worker
docker network disconnect bridge-2 kind-worker2
docker network rm bridge-2
docker exec kind-worker ip link del eth1
docker exec kind-worker2 ip link del eth1
true
```
```bash
docker network disconnect bridge-2 kind-worker
docker network disconnect bridge-2 kind-worker2
docker network rm bridge-2
docker exec kind-worker ip link del eth1
docker exec kind-worker2 ip link del eth1
true
```
11 changes: 7 additions & 4 deletions examples/remotevlan/kustomization.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,10 @@ kind: Kustomization
namespace: nsm-system

bases:
- ../../apps/nsmgr
- ../../apps/registry-k8s
- ../../apps/nse-remote-vlan
- ../../apps/admission-webhook-k8s
- ../../apps/nsmgr
- ../../apps/registry-k8s
- ../../apps/nse-remote-vlan
- ../../apps/admission-webhook-k8s

patchesStrategicMerge:
- patch-nse.yaml
35 changes: 35 additions & 0 deletions examples/remotevlan/rvlanovs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# NSM Remote Vlan OVS Forwarder

Contains setup for `forwarder-ovs` and device configuration file for remote vlan mechanism.

## Requires

Make sure that you have completed steps from [remotevlan](../../remotevlan) setup.

## Includes

- [Kernel2RVlanInternal](../../use-cases/Kernel2RVlanInternal)
- [Kernel2RVlanBreakout](../../use-cases/Kernel2RVlanBreakout)
- [Kernel2RVlanMultiNS](../../use-cases/Kernel2RVlanMultiNS)

## Run

Deploy the forwarder:

```bash
kubectl apply -k .
```

Wait forwarder to start:

```bash
kubectl -n nsm-system wait --for=condition=ready --timeout=2m pod -l app=forwarder-ovs
```

## Cleanup

Delete the forwarder:

```bash
kubectl delete -k .
```
16 changes: 16 additions & 0 deletions examples/remotevlan/rvlanovs/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

namespace: nsm-system

bases:
- ../../../apps/forwarder-ovs

configMapGenerator:
- name: device-selector
files:
- selector

patchesStrategicMerge:
- patch-forwarder-ovs.yaml
24 changes: 24 additions & 0 deletions examples/remotevlan/rvlanovs/patch-forwarder-ovs.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: forwarder-ovs
spec:
template:
spec:
containers:
- name: forwarder-ovs
env:
- name: NSM_L2_RESOURCE_SELECTOR_FILE
value: /var/lib/networkservicemesh/device-selector.yaml
volumeMounts:
- name: devsel-vol
mountPath: /var/lib/networkservicemesh/device-selector.yaml
subPath: device-selector.yaml
volumes:
- name: devsel-vol
configMap:
name: device-selector
items:
- key: selector
path: device-selector.yaml
7 changes: 7 additions & 0 deletions examples/remotevlan/rvlanovs/selector
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
interfaces:
- name: eth1
bridge: br0
matches:
- labelSelector:
- via: gw1
18 changes: 17 additions & 1 deletion examples/remotevlan/rvlanvpp/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,9 @@ Make sure that you have completed steps from [remotevlan](../../remotevlan) setu

## Includes

- [Kernel2RVlan](../../use-cases/Kernel2RVlan)
- [Kernel2RVlanInternal](../../use-cases/Kernel2RVlanInternal)
- [Kernel2RVlanBreakout](../../use-cases/Kernel2RVlanBreakout)
- [Kernel2RVlanMultiNS](../../use-cases/Kernel2RVlanMultiNS)

## Run

Expand All @@ -17,3 +19,17 @@ Deploy the forwarder:
```bash
kubectl apply -k .
```

Wait forwarder to start:

```bash
kubectl -n nsm-system wait --for=condition=ready --timeout=2m pod -l app=forwarder-vpp
```

## Cleanup

Delete the forwarder:

```bash
kubectl delete -k .
```
9 changes: 0 additions & 9 deletions examples/remotevlan/rvlanvpp/device-selector.yaml

This file was deleted.

7 changes: 4 additions & 3 deletions examples/remotevlan/rvlanvpp/kustomization.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,9 @@ namespace: nsm-system
bases:
- ../../../apps/forwarder-vpp

resources:
- device-selector.yaml

configMapGenerator:
- name: device-selector
files:
- selector
patchesStrategicMerge:
- patch-forwarder-vpp.yaml
6 changes: 6 additions & 0 deletions examples/remotevlan/rvlanvpp/selector
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
interfaces:
- name: eth1
matches:
- labelSelector:
- via: gw1
File renamed without changes.
Loading

0 comments on commit 17c1f2f

Please sign in to comment.