Add Nftables support

[yboaron]

Epic Description

Nftables, abbreviated as "netfilter tables," represents a contemporary replacement for iptables, specifically crafted to align with the requirements of modern networking environments.

Starting with iptables version 1.8.0, it supports a new mode (a.k.a iptables-nft) that uses nftables APIs of the kernel while preserving the same original iptables user-facing API. Most of the linux distributions now-a-days allow us to use iptables-nft or iptables-legacy but default to iptables-nft.

This epic captures the changes necessary in various Submariner components for supporting Nftables based platforms.

Acceptance Criteria

Definition of Done (Checklist)

• Code complete
• Relevant metrics added
• The acceptance criteria met
• Unit/e2e test added & pass
• CI jobs pass
• Deployed using cloud-prepare+subctl
• Deployed on supported platforms (for e.g kind, OCP on AWS, OCP on GCP)
• Run subctl verify, diagnose and gather
• Uninstall
• Troubleshooting (gather/diagnose) added
• Documentation added
• Release notes added

Work Items

• Document covering proposed high level design for Nftables support
• Refactor iptables package under packetfilter
• Refactor ipset package under packetfilter
• Auto-detection of Iptables/Nftables
• Deliver Submariner (no Nftables support) with the new packetfilter package
• Extend packetfilter package to support Nftables
• Upgrade

[maayanf24]

@yboaron - Are you preparing an ep for this epic?

[yboaron]

High Level Design document

[yboaron]

@maayanf24 Nope, we are not preparing ep for this epic, all the relevant details covered in this doc