Floating vl3 fix (#7054)

* Fix floating vl3

Signed-off-by: Artem Glazychev <artem.glazychev@xored.com>

* Replace to refs

Signed-off-by: Artem Glazychev <artem.glazychev@xored.com>

* Apply comments

Signed-off-by: Artem Glazychev <artem.glazychev@xored.com>

Signed-off-by: Artem Glazychev <artem.glazychev@xored.com>

examples/floating_interdomain/README.md

@@ -25,6 +25,7 @@ Interdomain tests can be on two clusters, for thus tests scheme of request will
 - [Kernel to VXLAN to Kernel Connection via floating registry](./usecases/FloatingKernel2Vxlan2Kernel)
 - [Kernel to WIREGUARD to Kernel Connection](./usecases/Kernel2Wireguard2Kernel)
 - [Kernel to WIREGUARD to Kernel Connection via floating registry](./usecases/FloatingKernel2Wireguard2Kernel)
+- [Floating VL3](./usecases/FloatingVl3)
 
 ## Run
 

examples/floating_interdomain/usecases/FloatingVl3/README.md

@@ -17,7 +17,7 @@ flowchart TB
     subgraph cluster2
     nsc2---nsm2---nse-vl3-vpp2
     end
-    subgraph floading domain
+    subgraph floating domain
     vl3-ipam
     registry
     end
@@ -36,51 +36,109 @@ Make sure that you have completed steps from [interdomain](../../)
 export KUBECONFIG=$KUBECONFIG3
 ```
 
-1.2. Start **vl3 ipam** and register **vl3 network service** in the *floaing domain*.
+1.2. Start **vl3 ipam** and register **vl3 network service** in the *floating domain*.
 
 
 Note: *By default we're using ipam prefix is `169.254.0.0/16` and client prefix len is `24`. We also have two vl3 nses in this example. So we are expect to have a two vl3 addresses: `169.254.0.0` and `169.254.1.0` that should be accessible by each client.*
 
 
 ```bash
-kubectl apply -k ./cluster3
+kubectl apply -k https://github.com/networkservicemesh/deployments-k8s/examples/floating_interdomain/usecases/FloatingVl3/cluster3?ref=d8a3d2c5481d66092b877043573c268afd5c3f85
 ```
 
-1.3 Switch context to the *cluster1*.
+1.3. Switch context to the *cluster1*.
 
 ```bash
 export KUBECONFIG=$KUBECONFIG1
 ```
 
+1.4. Prepare a patch with **vl3 ipam URL**:
+```bash
+cat > kustomization.yaml <<EOF
+---
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+
+resources:
+-  https://github.com/networkservicemesh/deployments-k8s/examples/floating_interdomain/usecases/FloatingVl3/cluster1?ref=d8a3d2c5481d66092b877043573c268afd5c3f85
+
+patchesStrategicMerge:
+- patch-vl3-nse.yaml
+EOF
+```
 
-1.4. Start **vl3 nse** and client in the *cluster1*.
+```bash
+cat > patch-vl3-nse.yaml <<EOF
+---
+apiVersion: v1
+kind: Pod
+metadata:
+  name: nse-vl3-vpp-1
+spec:
+  containers:
+    - name: nse
+      env:
+        - name: NSM_PREFIX_SERVER_URL
+          value: "tcp://vl3-ipam.nsm-system.my.cluster3:5006"
+EOF
+```
+
+1.5. Start **vl3 nse** and client in the *cluster1*.
 
 ```bash
-kubectl apply -k ./cluster1
+kubectl apply -k .
 ```
 
-1.5. Switch context to the *cluster2*.
+1.6. Switch context to the *cluster2*.
 
 ```bash
 export KUBECONFIG=$KUBECONFIG2
 ```
 
+1.7. Prepare a patch with **vl3 ipam URL**:
+```bash
+cat > kustomization.yaml <<EOF
+---
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
 
-1.6. Start **vl3 nse** and client in the *cluster2*.
+resources:
+- https://github.com/networkservicemesh/deployments-k8s/examples/floating_interdomain/usecases/FloatingVl3/cluster2?ref=d8a3d2c5481d66092b877043573c268afd5c3f85
+
+patchesStrategicMerge:
+- patch-vl3-nse.yaml
+EOF
+```
 
 ```bash
-kubectl apply -k ./cluster2
+cat > patch-vl3-nse.yaml <<EOF
+---
+apiVersion: v1
+kind: Pod
+metadata:
+  name: nse-vl3-vpp-2
+spec:
+  containers:
+    - name: nse
+      env:
+        - name: NSM_PREFIX_SERVER_URL
+          value: "tcp://vl3-ipam.nsm-system.my.cluster3:5006"
+EOF
+```
+
+1.8. Start **vl3 nse** and client in the *cluster2*.
+
+```bash
+kubectl apply -k .
 ```
 
 
 **2. Get assigned IP addresses**
 
-2.1. Get assigned IP address from vl3-nse for the client from the *cluster2*
+2.1. Find NSC in the *cluster2*:
 
 ```bash
 nsc2=$(kubectl get pods -l app=nsc-kernel -n ns-vl3-interdomain --template '{{range .items}}{{.metadata.name}}{{"\n"}}{{end}}')
-ipAddr2=$(kubectl exec -n ns-vl3-interdomain  $nsc2 -- ifconfig nsm-1)
-ipAddr2=$(echo $ipAddr | grep -Eo 'inet addr:[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}'| cut -c 11-)
 ```
 
 2.2. Switch context to the *cluster1*.
@@ -89,18 +147,18 @@ ipAddr2=$(echo $ipAddr | grep -Eo 'inet addr:[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\
 export KUBECONFIG=$KUBECONFIG1
 ```
 
-2.3. Get assigned IP addres from vl3-nse for the client from the *cluster1*
+2.3. Find NSC in the *cluster1*:
 
 ```bash
 nsc1=$(kubectl get pods -l app=nsc-kernel -n ns-vl3-interdomain --template '{{range .items}}{{.metadata.name}}{{"\n"}}{{end}}')
-ipAddr1=$(kubectl exec -n ns-vl3-interdomain $nsc1 -- ifconfig nsm-1)
-ipAddr1=$(echo $ipAddr | grep -Eo 'inet addr:[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}'| cut -c 11-)
 ```
 
 **3. Check connectivity**
 
-3.1. Ping remote client:
+3.1. Get assigned IP address from the vl3-NSE for the NSC2 and ping the remote client (NSC1):
 ```bash
+ipAddr2=$(kubectl --kubeconfig=$KUBECONFIG2 exec -n ns-vl3-interdomain $nsc2 -- ifconfig nsm-1)
+ipAddr2=$(echo $ipAddr2 | grep -Eo 'inet addr:[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}'| cut -c 11-)
 kubectl exec $nsc1 -n ns-vl3-interdomain -- ping -c 4 $ipAddr2
 ```
 
@@ -116,9 +174,10 @@ kubectl exec $nsc1 -n ns-vl3-interdomain -- ping -c 4 169.254.1.0
 export KUBECONFIG=$KUBECONFIG2
 ```
 
-
-3.4. Ping remote client:
+3.4. Get assigned IP address from the vl3-NSE for the NSC1 and ping the remote client (NSC2):
 ```bash
+ipAddr1=$(kubectl --kubeconfig=$KUBECONFIG1 exec -n ns-vl3-interdomain $nsc1 -- ifconfig nsm-1)
+ipAddr1=$(echo $ipAddr1 | grep -Eo 'inet addr:[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}'| cut -c 11-)
 kubectl exec $nsc2 -n ns-vl3-interdomain -- ping -c 4 $ipAddr1
 ```
 
@@ -133,17 +192,17 @@ kubectl exec $nsc2 -n ns-vl3-interdomain -- ping -c 4 169.254.1.0
 1. Cleanup floating domain:
 
 ```bash
-export KUBECONFIG=$KUBECONFIG3 kubectl delete -k ./cluster3
+export KUBECONFIG=$KUBECONFIG3 && kubectl delete -k https://github.com/networkservicemesh/deployments-k8s/examples/floating_interdomain/usecases/FloatingVl3/cluster3?ref=d8a3d2c5481d66092b877043573c268afd5c3f85
 ```
 
 2. Cleanup cluster2 domain:
 
 ```bash
-export KUBECONFIG=$KUBECONFIG2 kubectl delete -k ./cluster2
+export KUBECONFIG=$KUBECONFIG2 && kubectl delete -k https://github.com/networkservicemesh/deployments-k8s/examples/floating_interdomain/usecases/FloatingVl3/cluster2?ref=d8a3d2c5481d66092b877043573c268afd5c3f85
 ```
 
 3. Cleanup cluster1 domain:
 
 ```bash
-export KUBECONFIG=$KUBECONFIG1 kubectl delete -k ./cluster1
+export KUBECONFIG=$KUBECONFIG1 && kubectl delete -k https://github.com/networkservicemesh/deployments-k8s/examples/floating_interdomain/usecases/FloatingVl3/cluster1?ref=d8a3d2c5481d66092b877043573c268afd5c3f85
 ```

examples/floating_interdomain/usecases/FloatingVl3/cluster1/kustomization.yaml

@@ -7,9 +7,7 @@ namespace: ns-vl3-interdomain
 resources:
   - namespace.yaml
   - nse-vl3-vpp.yaml
-
-bases:
-- ../../../../../apps/nsc-kernel
+  - ../../../../../apps/nsc-kernel
 
 patchesStrategicMerge:
-- nsc-patch.yaml
+  - nsc-patch.yaml

examples/floating_interdomain/usecases/FloatingVl3/cluster1/nsc-patch.yaml

@@ -12,4 +12,4 @@ spec:
         - name: nsc
           env:
             - name: NSM_NETWORK_SERVICES
-              value: kernel://my-interdomain-vl3-network/nsm-1
+              value: kernel://my-interdomain-vl3-network@my.cluster3/nsm-1

examples/floating_interdomain/usecases/FloatingVl3/cluster1/nse-vl3-vpp.yaml

@@ -11,7 +11,7 @@ metadata:
 spec:
   containers:
     - name: nse
-      image: ghcr.io/networkservicemesh/cmd-nse-vl3-vpp:e9e706e
+      image: ghcr.io/networkservicemesh/ci/cmd-nse-vl3-vpp:e9e706e
       imagePullPolicy: IfNotPresent
       env:
         - name: SPIFFE_ENDPOINT_SOCKET
@@ -22,8 +22,6 @@ spec:
           value: "false"
         - name: NSM_SERVICE_NAMES
           value: "my-interdomain-vl3-network@my.cluster3"
-        - name: NSM_PREFIXSERVERURL
-          value: "tcp://172.18.3.131:5006"
         - name: NSM_LOG_LEVEL
           value: TRACE
       volumeMounts:

examples/floating_interdomain/usecases/FloatingVl3/cluster2/kustomization.yaml

@@ -7,9 +7,7 @@ namespace: ns-vl3-interdomain
 resources:
   - namespace.yaml
   - nse-vl3-vpp.yaml
-
-bases:
-- ../../../../../apps/nsc-kernel
+  - ../../../../../apps/nsc-kernel
 
 patchesStrategicMerge:
 - nsc-patch.yaml

examples/floating_interdomain/usecases/FloatingVl3/cluster2/nse-vl3-vpp.yaml

@@ -11,17 +11,17 @@ metadata:
 spec:
   containers:
     - name: nse
-      image: ghcr.io/networkservicemesh/cmd-nse-vl3-vpp:e9e706e
+      image: ghcr.io/networkservicemesh/ci/cmd-nse-vl3-vpp:e9e706e
       imagePullPolicy: IfNotPresent
       env:
         - name: SPIFFE_ENDPOINT_SOCKET
           value: unix:///run/spire/sockets/agent.sock
         - name: NSM_NAME
           value: "nse-vl3-vpp-2@my.cluster3"
+        - name: NSM_REGISTER_SERVICE
+          value: "false"
         - name: NSM_SERVICE_NAMES
           value: "my-interdomain-vl3-network@my.cluster3"
-        - name: NSM_PREFIXSERVERURL
-          value: "tcp://172.18.3.131:5006"
         - name: NSM_LOG_LEVEL
           value: TRACE
       volumeMounts:

examples/floating_interdomain/usecases/FloatingVl3/cluster3/kustomization.yaml

@@ -6,9 +6,7 @@ namespace: nsm-system
 
 resources:
   - networkservice.yaml
-
-bases:
-- ../../../../../apps/vl3-ipam
+  - ../../../../../apps/vl3-ipam
 
 patchesStrategicMerge:
 - patch-vl3-ipam.yaml

examples/floating_interdomain/usecases/FloatingVl3/cluster3/patch-vl3-ipam.yaml

@@ -0,0 +1,10 @@
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: vl3-ipam
+spec:
+  template:
+    metadata:
+      annotations:
+        spiffe.io/federatesWith: nsm.cluster1,nsm.cluster2