Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP][Windows]Antrea Proxy NodePort Service Support #1934

Closed
ruicao93 opened this issue Mar 3, 2021 · 2 comments
Closed

[WIP][Windows]Antrea Proxy NodePort Service Support #1934

ruicao93 opened this issue Mar 3, 2021 · 2 comments
Assignees
Labels
area/OS/windows Issues or PRs related to the Windows operating system. kind/feature Categorizes issue or PR as related to a new feature.

Comments

@ruicao93
Copy link
Contributor

ruicao93 commented Mar 3, 2021

Problem

We has already implemented ClusterIP Service support (Pod) with antrea-proxy, but Kube-Proxy is still needed for NodePort services.

On Windows there're no native kit like Netfilter + Iptables which can help to filter/redirect/nat packets in kernel. Use 3rd tools will need to install additional network drivers.

As we know, kube-proxy on Windows currently runs in user space which is inefficient and will cause high latency. We intent to implement the NodePort service in Antrea by leveraging OVS to handle the kind of traffic in kernel space.

kube-proxy

Design

antrea-proxy will handle the NodePort service after removing kube-proxy. And ClusterIP Endpoint selection flows in OVS are reused for NodePort service.

3 main cases of NodePort service according to the traffic source:

  • From external host.
  • From local host.
  • From local Pod.

Flow tables needed:

  • UplinkTable: Redirect to NodePortFilterTables to check if the packets are sent to NodePort service.
  • (New) NodePortFilterTable1: Filter NodePort IPs
  • (New) NodePortFilterTable2: Filter NodePort Ports
  • (New) DnatTables (& PostDNAT): Set dst IP to VIP
  • (New) SnatTables (& PostSNAT): For externalTrafficPolicy: "cluster", do SNAT

1. From external host

source_external_arch

The packets of NodePort service traffic from external host enter OVS pipeline through uplink port. Before enter existing Service Endpoint selection pipeline, the packets need to be DNATed and SNATed:

  • DNAT (dst=$VIP):

    The Service Endpoint selection pipeline (Antrea Proxy NodePort Service Support #1463) uses a VirtualIP to match the flow DSTed to NodePort service. So DNAT need to be applied.

  • SNAT (src=$GwIP): SNAT is required when externalTrafficPolicy is Cluster.

Then the Service Endpoint selection pipeline will loadbalance the access to backend endpoints as ClusterIP service.

Because only the NodePort service traffic is needed to apply above NATs. So we introduce several tables to filter the traffic from uplink. And only redirect NodePort service packets to NATs. The pipeline will look like:

source_external_tables

  • 0x1 in REG8: request
  • 0x2 in REG8: reply

Note: "from" of a request is uplink and "out" of a reply is "antrea-gw0". antrea-gw0 --> host --> br-int --> uplink. Better to keep them same.

2. From local host

Need:

  • A userspace proxy (fullNAT) to start the new traffic with (GwIP, vIP) after receiving traffic from Local Process.
  • A new route so such traffic will be routed to antrea-gw0

Then reuse the flow pipeline already implemented in OVS.

source_localhost_arch

source_localhost_tables

3. From local Pod

It reuses most of the local host case. The Pod sends traffic via antrea-gw0 to the host with the help of OVS.
Then the pipeline is the same as local host case.

source_pod_arch

@ruicao93 ruicao93 added the kind/feature Categorizes issue or PR as related to a new feature. label Mar 3, 2021
@ruicao93 ruicao93 added the area/OS/windows Issues or PRs related to the Windows operating system. label Mar 14, 2021
@github-actions
Copy link
Contributor

This issue is stale because it has been open 180 days with no activity. Remove stale label or comment, or this will be closed in 180 days

@github-actions github-actions bot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Sep 13, 2021
@lzhecheng lzhecheng removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Sep 13, 2021
@lzhecheng
Copy link
Contributor

NodePort support is implemented in this PR: #2863

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/OS/windows Issues or PRs related to the Windows operating system. kind/feature Categorizes issue or PR as related to a new feature.
Projects
None yet
Development

No branches or pull requests

2 participants