How can i make composition of endpoints?

[isZams]

Hello everyone, I would like to understand how I can manage the composition of a chain of security functions. More specifically, what I need is the creation of a kernel interface in the various pods in the middle of the chain. Taking a look at the code used to implement "cmd-nse-icmp-responder" and ""cmd-nse-icmp-responder-vpp," I saw how the related "kernel" and "memif" interfaces are created. However, I did not understand if it is possible to use the same approach, or something similar, for creating the kernel interface within "cmd-nse-firewall-vpp," that is, the image used in the "nse-composition" example within the various "passthroughs" . In the end, what I want to achieve is an identical result to this example, only that the pods in the middle of the chain are composed of 2 containers: one container that implements the actual security function, and another container useful for creating interface (a kernel one) that will then allow me to work with NSM.
I asked for the specific case of "cmd-nse-firewall-vpp" because I would like the traffic to flow, considering also a certain order, within the chain, but any other approach that works is fine as well, and I don't know if "cmd-nse-icmp-responder" would then allow me to handle anything beyond a simple ping.

Also, I wanted to ask where some documentation could be found so that I can get a clearer view of the actions and mechanisms that need to be implemented.
I thank you in advance for any responses.

EDIT
I have already looked at the various issues and I wanted to ask if the suggestion provided in this one, should actually be done like this:

The added packages are:

• https://github.com/networkservicemesh/sdk-vpp/tree/main/pkg/networkservice/xconnect/l2xconnect
• https://github.com/networkservicemesh/sdk-vpp/tree/main/pkg/networkservice/mechanisms/kernel/kerneltap

[isZams]

I was able to modify the code of cmd-nse-icmp-responder-vpp so that it also creates a kernel interface. I tested it by replacing the image inside the "Memif2Memif" example and it seems to work. Now what I am wondering is if there is a way to do the same thing within the cmd-nse-firewall-vpp image.
From what I understand (correct me if I'm wrong), when an NSC requires a network service with a specific type of mechanism (kernel or memif), let's say the NSE also has a set of supported mechanisms. The interfaces that are created, however, refer to those that are used to "pass through" the forwarder. So in the "nse-composition" example, since the client requests the endpoint at the end of the chain, the various pods in between (i.e., pass-throughs) are not registered to work with the forwarder, right? And If so, how do I create a kernel interface always exploiting the cmd-nse-firewall-vpp image so that it is available for the pod?

[denis-tingaikin]

Hello!

From what I understand (correct me if I'm wrong), when an NSC requires a network service with a specific type of mechanism (kernel or memif), let's say the NSE also has a set of supported mechanisms. The interfaces that are created, however, refer to those that are used to "pass through" the forwarder. So in the "nse-composition" example, since the client requests the endpoint at the end of the chain, the various pods in between (i.e., pass-throughs) are not registered to work with the forwarder, right? And If so, how do I create a kernel interface always exploiting the cmd-nse-firewall-vpp image so that it is available for the pod?

We're using a simple trick in cmd-nse-vl3-vpp that might be what you want. Have a look at https://github.com/networkservicemesh/cmd-nse-vl3-vpp/blob/main/main.go#L424

I think we could consider this problem within our technical backlog on Bug fixing week (from 12th to 19th Sept)

Added on the project board.

We'll post any info as soon as we get started work with the ticket.

[isZams]

Thanks for the reply @denis-tingaikin Maybe I didn't explain well what i wanted to achieve. Do you have any suggestions to map a memif interface (the one present in cmd-nse-firewall -vpp) into a kernel one?(using kerneltap and l2xconnect).

[glazychev-art]

Hello @isZams ,
cmd-nse-firewall-vpp applies ACL rules for interfaces in the VPP space. Do I understand correctly that you want to add security rules, but in the kernel?
Can you please tell what utility/feature you want to use to protect the kernel interface?

[isZams]

Hi @glazychev-art Thanks for the reply. Actually what I would like is to have a container to use as a sidecar inside a pod to interact with NSM. This container would have to implement two kernel interfaces (so it would work at L2), one for incoming and one for outgoing traffic.
In this way, I would then compose a chain of security functions to be placed in the pods alongside the sidecar described above.
Just to be clear, I would like to be able to do, for example, packet filtering with a container inside a pod where there is another container that allows me to have the kernel interfaces available to work with NSM. This pod should then be placed before the pod(NSE) where the service,required by NSC, is actually provided.

[isZams]

If it helps, @edwarnicke suggested to start from the code of cmd-nse-icmp-responder and then add the part where the cmd-nsc request is created https://github.com/networkservicemesh/cmd-nsc/blob/main/main.go#L190

[glazychev-art]

@isZams
Ok, thank you! Got it.
I don't think cmd-nse-firewall-vpp suits you. For this reason:
cmd-nse-firewall-vpp has a cross-connect between two memif interfaces:
[...] ==== [(memif) x (memif)] ==== [...]
These interfaces do not have IP addresses, it’s just like 2 connected pipes - what comes to one is immediately transferred to the other.
We will not be able to add another (third) interface to this cross-connect.

You need exactly the same application as cmd-nse-firewall-vpp but without VPP, in the OS space. Then there will be these 2 kernel interfaces.

I think @edwarnicke is mainly right, but the data will not be transmitted in this case. Because cmd-nse-icmp-responder and cmd-nsc have their IP-addresses. We need to add routing between.

Perhaps the simplest way to check is to define NSC and NSE containers in the same pod (need to check how this will work). The main thing is not to get confused in the services and set the CIDRs correctly.
And then manually (for now) do the routing inside this pod. It will look like:

NSC(actual) ===== [NSE: CIDR 172.16.0.0/31(kernel 172.16.0.0) ------ manual routing------ NSC (kernel 172.16.1.1)] ===== NSE(actual) (CIDR 172.16.1.0/31)

[isZams]

@glazychev-art i followed your suggestions but i still have a problem. The kernel interface(172.16.0.0) is up and works but the other one (kernel 172.16.1.1) not. These are the files that i'm using:
nse-kernel-composition.zip
I included also the logs of cmd-nsc container within the pod in the middle.
Do you have any advice or is there something i did wrong?

[glazychev-art]

@isZams
For the passthrough-1 cmd-nsc:

            - name: NSM_NETWORK_SERVICES 
              value: kernel://nse-composition/kernelint2?app=passthrough-1

kernelint2 instead of kernelinterface2 because a linux network interface name has 15 length max.
app=passthrough-1 - allows you to choose the right nse

[isZams]

@glazychev-art
Thanks again for the help. Now with these fixes it works.
Can you give me a motivation of why it is used app=passthrough-1?
And also, Are there some hints that i have to keep in mind for the routing within the pod?
I'm trying to use this configuration within nse container:

        apk add --no-cache iptables

        INTERFACE=$(ip -o -4 addr list | grep "172.16.0.0/32" | awk '{print $2}')
        iptables -A FORWARD -i $INTERFACE -o kernelint2 -p icmp -d 172.16.1.0 -j ACCEPT 
        iptables -A FORWARD -i kernelint2 -o $INTERFACE -p icmp -s 172.16.1.0 -j ACCEPT

[glazychev-art]

@isZams
Keep in mind, that we work within the same passthrough-1 pod for both nse-icmp-responder and nsc.
We have selectors inside nse-composition-ns.yaml:

  matches:
    - source_selector:
        app: passthrough-1
      routes:
        - destination_selector:
            app: gateway
    - routes:
        - destination_selector:
            app: passthrough-1

It says that "if the source (cmd-nsc of the passthrough-1 in our case) has app: passthrough-1 label, NSM has to choose the endpoint that has app: gateway"
And the real endpoint actually has that label (see NSM_LABELS of patch-nse.yaml)

Regarding routing, have you tried to use a simple ip route command? I thought we have to run something like "ip route add 172.16.1.0 via kernelint2" and "ip route add 172.16.0.1 via $INTERFACE". Need to check it, I'm not sure if it will work properly..

[isZams]

@glazychev-art
Thanks again for your reply.
I would like to ask if it is possible to place another container (not configured to work with NSM) within the pod in the middle of the chain.
For example, consider this scenario:
NSC ==== PT === NSE
The NSC wants to connect to NSE where a service is provided. Considering nse-composition-ns.yaml, the traffic has to be routed through PT(Passthrough) which is composed by 2 containers:

• cmd-nse-icmp-responder

• a security function container (SFC), that, for instance, performs packet filtering

The first container it is used create a kernel interface within the POD, for the incoming traffic, while the second container has to be attached to the previous interface in order to perform the packet filtering. When SFC performs what it has to do, it will forward the traffic to the next endpoint of the chain (so the actual service requested by the NSC)
Here i have 3 questions:

1. Is this something that could work?
2. If it works, it is just thanks to the nse-composition-ns.yaml or is there another component that handles this situation?
3. Is an interface configured with NSM needed to forward the traffic to the next pod in the chain?

[glazychev-art]

@isZams
As I understand it, we decided that for your issue the PT already consists of 2 containers:

• cmd-nse-icmp-responder (kernel1)
• cmd-nsc (kernel2)

You can add a third container (SFC) there without any problems:

• cmd-nse-icmp-responder (kernel1)
• cmd-nsc (kernel2)
• SFC (protects kernel1 or kernel2)

They are all located in the same network space - i.e. we see all network interfaces from any container of the Pod.

2. I don't quite understand the question... We can assume that the Pod is a regular host that has 2 interfaces, one of which is protected by SFC. nse-composition-ns.yaml is responsible for the correct choice of NetworkService and NSE. We have already done this, if you see 2 kernel interfaces in PT

3. This is what needs to be done manually (I was talking about manual routing). Once again, the schema looks like this:
   NSC ==== [kernel1(protected by SFC) ---- kernel2] === NSE

You were right here, it is possible to do this via iptables, the rules could be like this:

INTERFACE=$(ip -o -4 addr list | grep "172.16.0.0/32" | awk '{print $2}')
iptables -t nat -A PREROUTING -i $INTERFACE -j DNAT --to 172.16.1.0
iptables -t nat -A POSTROUTING -o kernelint2 -j SNAT --to 172.16.1.1

Now ping 172.16.0.0 from the client reaches the endpoint

[isZams]

Thank you very much again for your help @glazychev-art , I just wanted to better understand the mechanisms of NSM. Thanks again.

[glazychev-art]

@isZams
It is always welcomed!
Ping if you have any more questions

[isZams]

Sorry again @glazychev-art but I have another doubt.
Considering a scenario, where I want to use an external image container that is not configured to work with NSM ,, if i wanted to make a chain like this:
NSC === POD1 === NSE
In order for the container in question (the one used within POD1) , and thus the pod, to be considered as NSM endpoint, what is the best approach?
What is important to me is to be able to use any type of container( and therefore any image), and have the pod where this container is running, configured as an endpoint of NSM.
In this specific context I am not interested in having a certain type of interface, but I would like to understand how to make any POD (so with images that have not been configured to work with NSM) an NSM endpoint.
I hope I have explained myself sufficiently well.

[glazychev-art]

It doesn’t matter what container you want to run in your pod - be it a packet filter, or any http server. Being in the NSM network is ensured by a sidecar container (in your case, 2 sidecar containers - cmd-nsc and cmd-nse).

Have you tried running a pod like this (with three containers)? Have you faced any problems?

[isZams]

@glazychev-art , yes I tried to add a 3rd alpine container that would control passing traffic on one of the 2 kernel interfaces (just as simple example) . The problem is that the pod remains in the "CrashLoopBackOff" state.
Here there is the yaml file
passthrough-1.zip

EDIT
There were 2 errors in the commands for the container and now it works.

[glazychev-art]

I think the original problem has been resolved, so we can close this issue.
Feel free to open new issues if you need help!