Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add more remote vlan examples #4465

Merged
merged 1 commit into from
Feb 28, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 0 additions & 9 deletions apps/nse-remote-vlan/nse-remote-vlan.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -21,22 +21,13 @@ spec:
imagePullPolicy: IfNotPresent
ports:
- containerPort: 5003
hostPort: 5003
env:
- name: SPIFFE_ENDPOINT_SOCKET
value: unix:///run/spire/sockets/agent.sock
- name: NSM_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: NSM_CONNECT_TO
value: "registry:5002"
- name: NSM_SERVICES
value: "finance-bridge { vlan: 100; via: gw1}"
- name: NSM_CIDR_PREFIX
value: "172.10.0.0/24"
- name: NSM_IPV6_PREFIX
value: "100:200::/64"
- name: NSM_POINT2POINT
value: "False"
- name: NSM_REGISTER_SERVICE
Expand Down
149 changes: 88 additions & 61 deletions examples/remotevlan/README.md
Original file line number Diff line number Diff line change
@@ -1,80 +1,107 @@
# NSM Remote Vlan Examples

This setup can be used to check remote vlan mechanism. Contain basic setup for NSM that includes `nsmgr`, `registry-k8s`, `admission-webhook-k8s`, `nse-remote-vlan`. The `nse-remote-vlan` belongs to the nsm-system since does not have role in data-plane connection.
This setup can be used to check remote vlan mechanism with both OVS and VPP forwarder. Contain basic setup for NSM that includes `nsmgr`, `registry-k8s`, `admission-webhook-k8s`, `nse-remote-vlan`. The `nse-remote-vlan` belongs to the nsm-system since does not have role in data-plane connection.

## Requires

- [spire](../spire)

## Includes

- [Remote VLAN mechanism using forwarder-ovs](./rvlanovs)
- [Remote VLAN mechanism using forwarder-vpp](./rvlanvpp)

## Run

1. Create secondary bridge network and connect kind-worker nodes:

```bash
docker network create bridge-2
docker network connect bridge-2 kind-worker
docker network connect bridge-2 kind-worker2
```

2. Rename the newly generated interface to eth1 in both kind-workers:

```bash
ifw1=$(echo $(docker exec kind-worker ip link | tail -2 | head -1) | cut -f1 -d"@" | cut -f2 -d" ")
docker exec kind-worker ip link set $ifw1 down
docker exec kind-worker ip link set $ifw1 name eth1
docker exec kind-worker ip link set eth1 up
ifw2=$(echo $(docker exec kind-worker2 ip link | tail -2 | head -1) | cut -f1 -d"@" | cut -f2 -d" ")
docker exec kind-worker2 ip link set $ifw2 down
docker exec kind-worker2 ip link set $ifw2 name eth1
docker exec kind-worker2 ip link set eth1 up
```

3. Create ns for deployments:

```bash
kubectl create ns nsm-system
```

4. Apply NSM resources for basic tests:

```bash
kubectl apply -k .
```

5. Wait for NSE application:

```bash
kubectl -n nsm-system wait --for=condition=ready --timeout=2m pod -l app=nse-remote-vlan
```

6. Wait for admission-webhook-k8s:

```bash
WH=$(kubectl get pods -l app=admission-webhook-k8s -n nsm-system --template '{{range .items}}{{.metadata.name}}{{"\n"}}{{end}}')
kubectl wait --for=condition=ready --timeout=1m pod ${WH} -n nsm-system
```
Create secondary bridge network and connect kind-worker nodes:

```bash
docker network create bridge-2
docker network connect bridge-2 kind-worker
docker network connect bridge-2 kind-worker2
```

Rename the newly generated interface to eth1 in both kind-workers:

```bash
ifw1=$(echo $(docker exec kind-worker ip link | tail -2 | head -1) | cut -f1 -d"@" | cut -f2 -d" ")
docker exec kind-worker ip link set $ifw1 down
docker exec kind-worker ip link set $ifw1 name eth1
docker exec kind-worker ip link set eth1 up
ifw2=$(echo $(docker exec kind-worker2 ip link | tail -2 | head -1) | cut -f1 -d"@" | cut -f2 -d" ")
docker exec kind-worker2 ip link set $ifw2 down
docker exec kind-worker2 ip link set $ifw2 name eth1
docker exec kind-worker2 ip link set eth1 up
```

Create ns for deployments:

```bash
kubectl create ns nsm-system
```

Create NSE patch:

```bash
cat > patch-nse.yaml <<EOF
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nse-remote-vlan
spec:
template:
spec:
containers:
- name: nse
env:
- name: NSM_CONNECT_TO
value: "registry:5002"
- name: NSM_SERVICES
value: "finance-bridge { vlan: 100; via: gw1}"
- name: NSM_CIDR_PREFIX
value: "172.10.0.0/24"
- name: NSM_IPV6_PREFIX
value: "100:200::/64"
EOF
```

Apply NSM resources for basic tests:

```bash
kubectl apply -k .
```

Wait for NSE application:

```bash
kubectl -n nsm-system wait --for=condition=ready --timeout=2m pod -l app=nse-remote-vlan
```

Wait for admission-webhook-k8s:

```bash
WH=$(kubectl get pods -l app=admission-webhook-k8s -n nsm-system --template '{{range .items}}{{.metadata.name}}{{"\n"}}{{end}}')
kubectl wait --for=condition=ready --timeout=1m pod ${WH} -n nsm-system
```

## Cleanup

1. To free resources follow the next command:
To free resources follow the next command:

```bash
WH=$(kubectl get pods -l app=admission-webhook-k8s -n nsm-system --template '{{range .items}}{{.metadata.name}}{{"\n"}}{{end}}')
kubectl delete mutatingwebhookconfiguration ${WH}
kubectl delete ns nsm-system
```
```bash
WH=$(kubectl get pods -l app=admission-webhook-k8s -n nsm-system --template '{{range .items}}{{.metadata.name}}{{"\n"}}{{end}}')
kubectl delete mutatingwebhookconfiguration ${WH}
kubectl delete ns nsm-system
```

2. Delete secondary network and kind-worker node connections:
Delete secondary network and kind-worker node connections:

```bash
docker network disconnect bridge-2 kind-worker
docker network disconnect bridge-2 kind-worker2
docker network rm bridge-2
docker exec kind-worker ip link del eth1
docker exec kind-worker2 ip link del eth1
true
```
```bash
docker network disconnect bridge-2 kind-worker
docker network disconnect bridge-2 kind-worker2
docker network rm bridge-2
docker exec kind-worker ip link del eth1
docker exec kind-worker2 ip link del eth1
true
```
11 changes: 7 additions & 4 deletions examples/remotevlan/kustomization.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,10 @@ kind: Kustomization
namespace: nsm-system

bases:
- ../../apps/nsmgr
- ../../apps/registry-k8s
- ../../apps/nse-remote-vlan
- ../../apps/admission-webhook-k8s
- ../../apps/nsmgr
- ../../apps/registry-k8s
- ../../apps/nse-remote-vlan
- ../../apps/admission-webhook-k8s

patchesStrategicMerge:
- patch-nse.yaml
35 changes: 35 additions & 0 deletions examples/remotevlan/rvlanovs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# NSM Remote Vlan OVS Forwarder

Contains setup for `forwarder-ovs` and device configuration file for remote vlan mechanism.

## Requires

Make sure that you have completed steps from [remotevlan](../../remotevlan) setup.

## Includes

- [Kernel2RVlanInternal](../../use-cases/Kernel2RVlanInternal)
- [Kernel2RVlanBreakout](../../use-cases/Kernel2RVlanBreakout)
- [Kernel2RVlanMultiNS](../../use-cases/Kernel2RVlanMultiNS)

## Run

Deploy the forwarder:

```bash
kubectl apply -k .
```

Wait forwarder to start:

```bash
kubectl -n nsm-system wait --for=condition=ready --timeout=2m pod -l app=forwarder-ovs
```

## Cleanup

Delete the forwarder:

```bash
kubectl delete -k .
```
16 changes: 16 additions & 0 deletions examples/remotevlan/rvlanovs/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

namespace: nsm-system

bases:
- ../../../apps/forwarder-ovs

configMapGenerator:
- name: device-selector
files:
- selector

patchesStrategicMerge:
- patch-forwarder-ovs.yaml
24 changes: 24 additions & 0 deletions examples/remotevlan/rvlanovs/patch-forwarder-ovs.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: forwarder-ovs
spec:
template:
spec:
containers:
- name: forwarder-ovs
env:
- name: NSM_L2_RESOURCE_SELECTOR_FILE
value: /var/lib/networkservicemesh/device-selector.yaml
volumeMounts:
- name: devsel-vol
mountPath: /var/lib/networkservicemesh/device-selector.yaml
subPath: device-selector.yaml
volumes:
- name: devsel-vol
configMap:
name: device-selector
items:
- key: selector
path: device-selector.yaml
7 changes: 7 additions & 0 deletions examples/remotevlan/rvlanovs/selector
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
interfaces:
- name: eth1
bridge: br0
matches:
- labelSelector:
- via: gw1
18 changes: 17 additions & 1 deletion examples/remotevlan/rvlanvpp/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,9 @@ Make sure that you have completed steps from [remotevlan](../../remotevlan) setu

## Includes

- [Kernel2RVlan](../../use-cases/Kernel2RVlan)
- [Kernel2RVlanInternal](../../use-cases/Kernel2RVlanInternal)
- [Kernel2RVlanBreakout](../../use-cases/Kernel2RVlanBreakout)
- [Kernel2RVlanMultiNS](../../use-cases/Kernel2RVlanMultiNS)

## Run

Expand All @@ -17,3 +19,17 @@ Deploy the forwarder:
```bash
kubectl apply -k .
```

Wait forwarder to start:

```bash
kubectl -n nsm-system wait --for=condition=ready --timeout=2m pod -l app=forwarder-vpp
```

## Cleanup

Delete the forwarder:

```bash
kubectl delete -k .
```
9 changes: 0 additions & 9 deletions examples/remotevlan/rvlanvpp/device-selector.yaml

This file was deleted.

7 changes: 4 additions & 3 deletions examples/remotevlan/rvlanvpp/kustomization.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,9 @@ namespace: nsm-system
bases:
- ../../../apps/forwarder-vpp

resources:
- device-selector.yaml

configMapGenerator:
- name: device-selector
files:
- selector
patchesStrategicMerge:
- patch-forwarder-vpp.yaml
6 changes: 6 additions & 0 deletions examples/remotevlan/rvlanvpp/selector
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
interfaces:
- name: eth1
matches:
- labelSelector:
- via: gw1
Loading