Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

portmap: fix nftables backend #1116

Merged
merged 1 commit into from
Nov 18, 2024
Merged

portmap: fix nftables backend #1116

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 10 additions & 11 deletions plugins/meta/portmap/portmap_nftables.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ import (
"context"
"fmt"
"net"
"strconv"

"sigs.k8s.io/knftables"
)
Expand Down Expand Up @@ -110,23 +111,23 @@ func (pmNFT *portMapperNFTables) forwardPorts(config *PortMapConf, containerNet
})

tx.Add(&knftables.Chain{
Name: "input",
Name: "prerouting",
Type: knftables.PtrTo(knftables.NATType),
Hook: knftables.PtrTo(knftables.InputHook),
Hook: knftables.PtrTo(knftables.PreroutingHook),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I think I know what I was confused about: in prerouting it hasn't yet decided where it's going to route the packet to, so you can't use oifname. I was thinking that meant you couldn't use fib daddr type either, but that's wrong; fib daddr type answers "what does the routing table say we should do with this packet?", not "what are we actually going to do with this packet?", so it doesn't depend on the routing decision.

Priority: knftables.PtrTo(knftables.DNATPriority),
})
tx.Flush(&knftables.Chain{
Name: "input",
Name: "prerouting",
})
tx.Add(&knftables.Rule{
Chain: "input",
Chain: "prerouting",
Rule: knftables.Concat(
conditions,
"jump", hostIPHostPortsChain,
),
})
tx.Add(&knftables.Rule{
Chain: "input",
Chain: "prerouting",
Rule: knftables.Concat(
conditions,
"jump", hostPortsChain,
Expand Down Expand Up @@ -187,19 +188,17 @@ func (pmNFT *portMapperNFTables) forwardPorts(config *PortMapConf, containerNet
Chain: hostIPHostPortsChain,
Rule: knftables.Concat(
ipX, "daddr", e.HostIP,
ipX, "protocol", e.Protocol,
"th dport", e.HostPort,
"dnat", ipX, "addr . port", "to", containerNet.IP, ".", e.ContainerPort,
e.Protocol, "dport", e.HostPort,
"dnat to", net.JoinHostPort(containerNet.IP.String(), strconv.Itoa(e.ContainerPort)),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(yeah, this rule was complicated because it's derived from a kube-proxy rule where we look up the "addr . protocol . port" in a map rather than just having a static rule)

),
Comment: &config.ContainerID,
})
} else {
tx.Add(&knftables.Rule{
Chain: hostPortsChain,
Rule: knftables.Concat(
ipX, "protocol", e.Protocol,
"th dport", e.HostPort,
"dnat", ipX, "addr . port", "to", containerNet.IP, ".", e.ContainerPort,
e.Protocol, "dport", e.HostPort,
"dnat to", net.JoinHostPort(containerNet.IP.String(), strconv.Itoa(e.ContainerPort)),
),
Comment: &config.ContainerID,
})
Expand Down
36 changes: 18 additions & 18 deletions plugins/meta/portmap/portmap_nftables_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,21 +82,21 @@ var _ = Describe("portmapping configuration (nftables)", func() {
add table ip cni_hostport { comment "CNI portmap plugin" ; }
add chain ip cni_hostport hostip_hostports
add chain ip cni_hostport hostports
add chain ip cni_hostport input { type nat hook input priority -100 ; }
add chain ip cni_hostport masquerading { type nat hook postrouting priority 100 ; }
add chain ip cni_hostport output { type nat hook output priority -100 ; }
add rule ip cni_hostport hostip_hostports ip daddr 192.168.0.2 ip protocol tcp th dport 8083 dnat ip addr . port to 10.0.0.2 . 83 comment "icee6giejonei6so"
add rule ip cni_hostport hostports ip protocol tcp th dport 8080 dnat ip addr . port to 10.0.0.2 . 80 comment "icee6giejonei6so"
add rule ip cni_hostport hostports ip protocol tcp th dport 8081 dnat ip addr . port to 10.0.0.2 . 80 comment "icee6giejonei6so"
add rule ip cni_hostport hostports ip protocol udp th dport 8080 dnat ip addr . port to 10.0.0.2 . 81 comment "icee6giejonei6so"
add rule ip cni_hostport hostports ip protocol udp th dport 8082 dnat ip addr . port to 10.0.0.2 . 82 comment "icee6giejonei6so"
add rule ip cni_hostport hostports ip protocol tcp th dport 8084 dnat ip addr . port to 10.0.0.2 . 84 comment "icee6giejonei6so"
add rule ip cni_hostport input a b jump hostip_hostports
add rule ip cni_hostport input a b jump hostports
add chain ip cni_hostport prerouting { type nat hook prerouting priority -100 ; }
add rule ip cni_hostport hostip_hostports ip daddr 192.168.0.2 tcp dport 8083 dnat to 10.0.0.2:83 comment "icee6giejonei6so"
add rule ip cni_hostport hostports tcp dport 8080 dnat to 10.0.0.2:80 comment "icee6giejonei6so"
add rule ip cni_hostport hostports tcp dport 8081 dnat to 10.0.0.2:80 comment "icee6giejonei6so"
add rule ip cni_hostport hostports udp dport 8080 dnat to 10.0.0.2:81 comment "icee6giejonei6so"
add rule ip cni_hostport hostports udp dport 8082 dnat to 10.0.0.2:82 comment "icee6giejonei6so"
add rule ip cni_hostport hostports tcp dport 8084 dnat to 10.0.0.2:84 comment "icee6giejonei6so"
add rule ip cni_hostport masquerading ip saddr 10.0.0.2 ip daddr 10.0.0.2 masquerade comment "icee6giejonei6so"
add rule ip cni_hostport masquerading ip saddr 127.0.0.1 ip daddr 10.0.0.2 masquerade comment "icee6giejonei6so"
add rule ip cni_hostport output a b jump hostip_hostports
add rule ip cni_hostport output a b fib daddr type local jump hostports
add rule ip cni_hostport prerouting a b jump hostip_hostports
add rule ip cni_hostport prerouting a b jump hostports
`)
actualRules := strings.TrimSpace(ipv4Fake.Dump())
Expect(actualRules).To(Equal(expectedRules))
Expand All @@ -113,18 +113,18 @@ add rule ip cni_hostport output a b fib daddr type local jump hostports
add table ip6 cni_hostport { comment "CNI portmap plugin" ; }
add chain ip6 cni_hostport hostip_hostports
add chain ip6 cni_hostport hostports
add chain ip6 cni_hostport input { type nat hook input priority -100 ; }
add chain ip6 cni_hostport output { type nat hook output priority -100 ; }
add rule ip6 cni_hostport hostip_hostports ip6 daddr 2001:db8:a::1 ip6 protocol tcp th dport 8085 dnat ip6 addr . port to 2001:db8::2 . 85 comment "icee6giejonei6so"
add rule ip6 cni_hostport hostports ip6 protocol tcp th dport 8080 dnat ip6 addr . port to 2001:db8::2 . 80 comment "icee6giejonei6so"
add rule ip6 cni_hostport hostports ip6 protocol tcp th dport 8081 dnat ip6 addr . port to 2001:db8::2 . 80 comment "icee6giejonei6so"
add rule ip6 cni_hostport hostports ip6 protocol udp th dport 8080 dnat ip6 addr . port to 2001:db8::2 . 81 comment "icee6giejonei6so"
add rule ip6 cni_hostport hostports ip6 protocol udp th dport 8082 dnat ip6 addr . port to 2001:db8::2 . 82 comment "icee6giejonei6so"
add rule ip6 cni_hostport hostports ip6 protocol tcp th dport 8086 dnat ip6 addr . port to 2001:db8::2 . 86 comment "icee6giejonei6so"
add rule ip6 cni_hostport input c d jump hostip_hostports
add rule ip6 cni_hostport input c d jump hostports
add chain ip6 cni_hostport prerouting { type nat hook prerouting priority -100 ; }
add rule ip6 cni_hostport hostip_hostports ip6 daddr 2001:db8:a::1 tcp dport 8085 dnat to [2001:db8::2]:85 comment "icee6giejonei6so"
add rule ip6 cni_hostport hostports tcp dport 8080 dnat to [2001:db8::2]:80 comment "icee6giejonei6so"
add rule ip6 cni_hostport hostports tcp dport 8081 dnat to [2001:db8::2]:80 comment "icee6giejonei6so"
add rule ip6 cni_hostport hostports udp dport 8080 dnat to [2001:db8::2]:81 comment "icee6giejonei6so"
add rule ip6 cni_hostport hostports udp dport 8082 dnat to [2001:db8::2]:82 comment "icee6giejonei6so"
add rule ip6 cni_hostport hostports tcp dport 8086 dnat to [2001:db8::2]:86 comment "icee6giejonei6so"
add rule ip6 cni_hostport output c d jump hostip_hostports
add rule ip6 cni_hostport output c d fib daddr type local jump hostports
add rule ip6 cni_hostport prerouting c d jump hostip_hostports
add rule ip6 cni_hostport prerouting c d jump hostports
`)
actualRules = strings.TrimSpace(ipv6Fake.Dump())
Expect(actualRules).To(Equal(expectedRules))
Expand Down