Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use iptables-wrapper in Antrea container #3276

Merged

Conversation

antoninbas
Copy link
Contributor

Instead of iptables directly.
Antrea uses a Ubuntu 20.04 base container image, for which the default iptables
mode is "legacy". This may not match the iptables mode for the Node OS, which in
turn can create issues:

  • Other K8s components (kubelet, kube-proxy) will create rules using the
    default iptables mode for the Node. Assumptions about evaluation order
    between these rules and the Antrea rules may break.
  • The required kernel module for the "legacy" mode (ip_tables) may not be
    available on the Node.

The iptables-wrapper is meant to address these issues:
https://github.com/kubernetes-sigs/iptables-wrappers. We install it in the
Antrea container image. The first time Antrea invokes iptables, the wrapper will
determine the underlying iptables mode (for the Node OS) and adjust the iptables
symlinks in the container.

Fixes #3243
Fixes #3274

Signed-off-by: Antonin Bas abas@vmware.com

@antoninbas
Copy link
Contributor Author

This was tested on Ubuntu 20.04 (default is "legacy") and Ubuntu 21.04 (default is "nft").

@antoninbas antoninbas added action/backport Indicates a PR that requires backports. action/release-note Indicates a PR that should be included in release notes. kind/bug Categorizes issue or PR as related to a bug. labels Jan 31, 2022
@codecov-commenter
Copy link

codecov-commenter commented Feb 1, 2022

Codecov Report

Merging #3276 (c9ca8d6) into main (5ccd3d1) will increase coverage by 1.04%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #3276      +/-   ##
==========================================
+ Coverage   59.04%   60.08%   +1.04%     
==========================================
  Files         331      331              
  Lines       28420    28420              
==========================================
+ Hits        16781    17077     +296     
+ Misses       9798     9478     -320     
- Partials     1841     1865      +24     
Flag Coverage Δ
kind-e2e-tests 47.88% <ø> (+1.19%) ⬆️
unit-tests 41.85% <ø> (+0.05%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
pkg/agent/nodeportlocal/k8s/annotations.go 83.87% <0.00%> (-16.13%) ⬇️
...agent/flowexporter/connections/deny_connections.go 86.15% <0.00%> (-3.08%) ⬇️
pkg/controller/networkpolicy/status_controller.go 65.89% <0.00%> (-0.58%) ⬇️
pkg/agent/agent.go 51.43% <0.00%> (ø)
pkg/agent/controller/networkpolicy/reconciler.go 77.19% <0.00%> (+0.20%) ⬆️
pkg/agent/openflow/client.go 54.24% <0.00%> (+0.62%) ⬆️
...ntroller/networkpolicy/networkpolicy_controller.go 71.34% <0.00%> (+0.91%) ⬆️
pkg/ipam/poolallocator/allocator.go 51.15% <0.00%> (+1.15%) ⬆️
...kg/agent/flowexporter/connections/conntrack_ovs.go 77.57% <0.00%> (+1.21%) ⬆️
pkg/ovs/openflow/ofctrl_action.go 69.58% <0.00%> (+1.66%) ⬆️
... and 9 more

rm -rf /var/lib/apt/lists/* && \
chmod +x /iptables-wrapper-installer.sh && \
/iptables-wrapper-installer.sh && \
rm -rf /iptables-wrapper-installer.sh
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I saw the installer has the following:

# Cleanup
rm -f "$0"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, I removed the rm command

build/images/base/Dockerfile Show resolved Hide resolved
@antoninbas antoninbas force-pushed the use-iptables-wrapper-in-antrea-container branch from 377b766 to 4754d44 Compare February 8, 2022 17:57
tnqn
tnqn previously approved these changes Feb 9, 2022
build/images/base/Dockerfile Show resolved Hide resolved
&& rm -rf /var/lib/apt/lists/*
# See https://github.com/kubernetes-sigs/iptables-wrappers
# /iptables-wrapper-installer.sh will have permissions of 600.
# --chmod=644 doesn't work with older versions of Docker and requires DOCKER_BUILDKIT=1, so we use
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean --chmod=755?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I meant 700, just need the owner to be able to execute the script. Let me fix it.

Instead of iptables directly.
Antrea uses a Ubuntu 20.04 base container image, for which the default iptables
mode is "legacy". This may not match the iptables mode for the Node OS, which in
turn can create issues:
 * Other K8s components (kubelet, kube-proxy) will create rules using the
   default iptables mode for the Node. Assumptions about evaluation order
   between these rules and the Antrea rules may break.
 * The required kernel module for the "legacy" mode (ip_tables) may not be
   available on the Node.

The iptables-wrapper is meant to address these issues:
https://github.com/kubernetes-sigs/iptables-wrappers. We install it in the
Antrea container image. The first time Antrea invokes iptables, the wrapper will
determine the underlying iptables mode (for the Node OS) and adjust the iptables
symlinks in the container.

Fixes antrea-io#3243
Fixes antrea-io#3274

Signed-off-by: Antonin Bas <abas@vmware.com>
@antoninbas
Copy link
Contributor Author

/test-all

@antoninbas antoninbas added this to the Antrea v1.6 release milestone Feb 10, 2022
@antoninbas
Copy link
Contributor Author

/test-e2e
/test-integration

@tnqn
Copy link
Member

tnqn commented Feb 11, 2022

/test-e2e

@tnqn
Copy link
Member

tnqn commented Feb 11, 2022

/test-e2e unrelated failure: #3307. Maybe we could skip it if it still fails.

@antoninbas
Copy link
Contributor Author

Another unrelated failure:

--- FAIL: TestBandwidth/testPodTrafficShaping/limited_by_egress_bandwidth (90.02s)

I'll merge now

@antoninbas antoninbas merged commit f384995 into antrea-io:main Feb 11, 2022
@antoninbas antoninbas deleted the use-iptables-wrapper-in-antrea-container branch February 11, 2022 18:02
yanjunz97 pushed a commit to yanjunz97/antrea that referenced this pull request Feb 14, 2022
Instead of iptables directly.
Antrea uses a Ubuntu 20.04 base container image, for which the default iptables
mode is "legacy". This may not match the iptables mode for the Node OS, which in
turn can create issues:
 * Other K8s components (kubelet, kube-proxy) will create rules using the
   default iptables mode for the Node. Assumptions about evaluation order
   between these rules and the Antrea rules may break.
 * The required kernel module for the "legacy" mode (ip_tables) may not be
   available on the Node.

The iptables-wrapper is meant to address these issues:
https://github.com/kubernetes-sigs/iptables-wrappers. We install it in the
Antrea container image. The first time Antrea invokes iptables, the wrapper will
determine the underlying iptables mode (for the Node OS) and adjust the iptables
symlinks in the container.

Fixes antrea-io#3243
Fixes antrea-io#3274

Signed-off-by: Antonin Bas <abas@vmware.com>
bangqipropel pushed a commit to bangqipropel/antrea that referenced this pull request Mar 2, 2022
Instead of iptables directly.
Antrea uses a Ubuntu 20.04 base container image, for which the default iptables
mode is "legacy". This may not match the iptables mode for the Node OS, which in
turn can create issues:
 * Other K8s components (kubelet, kube-proxy) will create rules using the
   default iptables mode for the Node. Assumptions about evaluation order
   between these rules and the Antrea rules may break.
 * The required kernel module for the "legacy" mode (ip_tables) may not be
   available on the Node.

The iptables-wrapper is meant to address these issues:
https://github.com/kubernetes-sigs/iptables-wrappers. We install it in the
Antrea container image. The first time Antrea invokes iptables, the wrapper will
determine the underlying iptables mode (for the Node OS) and adjust the iptables
symlinks in the container.

Fixes antrea-io#3243
Fixes antrea-io#3274

Signed-off-by: Antonin Bas <abas@vmware.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
action/backport Indicates a PR that requires backports. action/release-note Indicates a PR that should be included in release notes. kind/bug Categorizes issue or PR as related to a bug.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Compatibility with iptables (nf_tables) Agent fails on AlmaLinux8/CentOS8
3 participants