Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] Interfaces show DOWN #202

Closed
havok4u opened this issue Aug 2, 2019 · 18 comments
Closed

[BUG] Interfaces show DOWN #202

havok4u opened this issue Aug 2, 2019 · 18 comments
Labels
bug Something isn't working good first issue Good for newcomers

Comments

@havok4u
Copy link

havok4u commented Aug 2, 2019

Describe the bug

When adding interfaces to polycube, they show down and connectivity doesn't work

To Reproduce

sudo ip netns add myapp0
sudo ip netns add myapp1

Create interfaces and up the links

sudo ip link add veth1 type veth peer name veth2
sudo ip link add veth3 type veth peer name veth4
for i in 1 2 3 4
do
sudo ip link set veth$i up
done
sudo ip link set veth2 netns myapp0
sudo ip link set veth4 netns myapp1

Add to Polycube

polycubectl br0 ports add veth1
polycubectl br0 ports add veth3

Set IP address to namespaces

sudo ip netns exec myapp0 ip addr add 10.1.1.2/24 dev veth2
sudo ip netns exec myapp1 ip addr add 10.1.1.3/24 dev veth4

Up interfaces in namespaces if required

sudo ip netns exec myapp0 sudo ip link set veth2 up
sudo ip netns exec myapp0 ip addr show
1: lo: mtu 65536 qdisc noop state DOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
14: veth2@if15: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 2e:a9:da:40:ec:af brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 10.1.1.2/24 scope global veth2
valid_lft forever preferred_lft forever
inet6 fe80::2ca9:daff:fe40:ecaf/64 scope link
valid_lft forever preferred_lft forever
sudo ip netns exec myapp1 sudo ip link set veth4 up
sudo ip netns exec myapp1 ip addr show
1: lo: mtu 65536 qdisc noop state DOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
16: veth4@if17: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 3a:49:95:d0:5a:bf brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 10.1.1.3/24 scope global veth4
valid_lft forever preferred_lft forever
inet6 fe80::3849:95ff:fed0:5abf/64 scope link
valid_lft forever preferred_lft forever

Verify veth1 and veth3

ip link show veth1
15: veth1@if14: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
link/ether 86:83:02:57:b4:1b brd ff:ff:ff:ff:ff:ff link-netnsid 0

ip link show veth3
17: veth3@if16: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
link/ether e2:75:0c:f1:47:1c brd ff:ff:ff:ff:ff:ff link-netnsid 1

Ping

sudo ip netns exec myapp1 ping 10.1.1.2 -c 5
PING 10.1.1.2 (10.1.1.2) 56(84) bytes of data.

--- 10.1.1.2 ping statistics ---
5 packets transmitted, 0 received, 100% packet loss, time 4100ms

Show Bridge

polycubectl br0 show
name: br0
uuid: 91081d7c-b164-4487-a161-3876467d338d
service-name: bridge
type: TC
loglevel: INFO
shadow: false
span: false
stp-enabled: false
mac: f2:f6:ae:20:3a:dc
fdb:
aging-time: 300

ports:
name uuid status peer mac mode
veth1 14720cdf-8ae2-45aa-8dad-8420e1432943 DOWN ae:8f:a6:15:f3:a3 access
veth3 ef0e0094-12aa-4fdd-96be-09326317f2d0 DOWN 36:18:34:87:f3:97 access
@havok4u havok4u added the bug Something isn't working label Aug 2, 2019
@mauriciovasquezbernal
Copy link
Contributor

Ì think you are missing to set the peer.

polycubectl br0 ports add veth1 peer=veth1
polycubectl br0 ports add veth3 peer=veth3

@havok4u
Copy link
Author

havok4u commented Aug 2, 2019

I guess I would be curious to know why would you do that when the port is already added? Regardless, it did not work.

$ polycubectl br0 ports add veth1 peer=veth1
Port veth1 already exists
$ polycubectl br0 ports add veth3 peer=veth3
Port veth3 already exists

$ polycubectl br0 show
name: br0
uuid: 91081d7c-b164-4487-a161-3876467d338d
service-name: bridge
type: TC
loglevel: INFO
shadow: false
span: false
stp-enabled: false
mac: f2:f6:ae:20:3a:dc
fdb:
aging-time: 300

ports:
name uuid status peer mac mode
veth1 14720cdf-8ae2-45aa-8dad-8420e1432943 DOWN ae:8f:a6:15:f3:a3 access
veth3 ef0e0094-12aa-4fdd-96be-09326317f2d0 DOWN 36:18:34:87:f3:97 access

@havok4u
Copy link
Author

havok4u commented Aug 2, 2019

Some missing information here, this is running in a KVM on a machine. The KVM OS Info is as follows:
$ uname -a
Linux xdptest1 4.15.0-55-generic #60-Ubuntu SMP Tue Jul 2 18:22:20 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

$ cat /etc/os-release
NAME="Ubuntu"
VERSION="18.04.2 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.2 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic

$ kvm-ok
INFO: /dev/kvm exists
KVM acceleration can be used

The host machine OS is as follows:
$ uname -a
Linux node41 4.4.0-154-generic #181-Ubuntu SMP Tue Jun 25 05:29:03 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

$ cat /etc/os-release
NAME="Ubuntu"
VERSION="16.04.6 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.6 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial

@havok4u
Copy link
Author

havok4u commented Aug 5, 2019

BTW, Just to reiterate, I tried what @mauriciovasquezbernal suggested, it did not work. Also wanted to show that the two interfaces from the OS level are up, but obviously down from the polycube standpoint:
$ ip link show veth1
15: veth1@if14: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
link/ether 86:83:02:57:b4:1b brd ff:ff:ff:ff:ff:ff link-netnsid 0
$ ip link show veth1
15: veth1@if14: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
link/ether 86:83:02:57:b4:1b brd ff:ff:ff:ff:ff:ff link-netnsid 0

@havok4u
Copy link
Author

havok4u commented Aug 5, 2019

@mauriciovasquezbernal Sorry, I didn't remove the interfaces before I performed your commands. I deleted the interfaces and added them back in the way you instructed and they are up.
$ polycubectl br0 ports show
mac mode name peer status uuid
aa:f0:84:c6:db:42 access veth1 veth1 UP 6f1f1a31-3bbe-46ed-ad2c-183d0feae485
8a:72:c0:80:75:a8 access veth3 veth3 UP 222a1540-7c84-4b54-b552-dd429ad80b40

I think we are ok to close this case, but I would like to understand why we would need to add the peer for just and interface? Also, if this is the case, can we get a fail and error message if we try to add the interface without defining a peer. Thanks

@goldenrye
Copy link
Contributor

$ polycubectl br0 ports add veth1 peer=veth1
The first "veth1" is the name of port -- an attribute of bridge br0, and the second "veth1" refers to the veth1 interface in the host, to avoid the confusion I think it is better to configure like the following command:
$ polycubectl br0 ports add p1 peer=veth1

We cannot fail the configuration without peer parameter because the port not necessary has a interface peer, for example if a bridge connects to a router, either the bridge or the router's port has to be configured without a peer because the other peer may not configure yet.

@acloudiator acloudiator added the good first issue Good for newcomers label Aug 7, 2019
@mauriciovasquezbernal
Copy link
Contributor

@havok4u I think this https://polycube.readthedocs.io/en/latest/cubes.html can clarify your doubts about the peer parameter in the ports.

I am closing this as this is really not an issue.

btw, @acloudiator I think the good first issue label should only be added when the issue is confirmed, otherwise a newcomer could get confused trying to solve something that is already working.

@havok4u
Copy link
Author

havok4u commented Aug 23, 2019 via email

@frisso
Copy link
Contributor

frisso commented Aug 23, 2019

@sebymiano I remember that you tested the bridge in XDP mode. Did you note any issues? Suggestions?

@havok4u
Copy link
Author

havok4u commented Aug 23, 2019 via email

@sebymiano
Copy link
Collaborator

@frisso Unfortunately, I have always tested the simplebridge service in either the TC or XDP_DRV mode, but not in the XDP_SKB mode.
@havok4u Could you please try the same test with the simplebridge? Just to understand in the problem is within the service or something else.

@havok4u
Copy link
Author

havok4u commented Aug 24, 2019 via email

@havok4u
Copy link
Author

havok4u commented Aug 24, 2019 via email

@mauriciovasquezbernal
Copy link
Contributor

mauriciovasquezbernal commented Aug 25, 2019 •
edited
Loading

I am not able to go through the full logs you provided, these are some clarifications:

  • There is an issue in XDP_SKB mode (also called Generic XDP) that causes XDP programs to drop TCP packets. It has already been discussed by @mbertrone in [1]
  • XDP_DRV (also called Native XDP) does not work with veth interfaces in the way we use it (the packet originates on a namespace and should be forwarded to another namespace). The native support for XDP in veth was created with other use cases in mind, in a nutshell, when a packet comes from the external world (i.e, received through a physical interface) it could be delivered very fast to a namespace (container) that has an XDP program in its network interface. Many more details are available at [2]. To clarify, we have use the different services with XDP_DRV and physical interfaces and they work.
  • XDP programs are executed quite early (even before allocating the SKB), for this reason tcpdump is not able to capture packets on interfaces that have XDP programs and they perform a redirect action.

Just as a summary:

  • TC mode works fine with physical and virtual interfaces.
  • XDP_SKB mode is mostly intended for debugging and developing purposes.
  • XDP_DRV mode is the fastest one and should be only used with physical interfaces that support it.

[1] https://www.spinics.net/lists/xdp-newbies/msg00440.html
[2] https://www.netdevconf.org/0x13/session.html?talk-veth-xdp

@havok4u
Copy link
Author

havok4u commented Aug 26, 2019 via email

@mauriciovasquezbernal
Copy link
Contributor

Using an XDP_DRV bridge would only make sense if there are XDP programs attached to the veth interfaces inside the containers. This would allow to pass the frame from the physical NIC to the veth in the container without performing an SKB allocation.

If you have standard applications in the container (by standard I mean any non XDP program) the XDP_DRV mode would not make sense because the SKB would have to be allocated anyway, so there is not any performance advantage in this case, hence a TC bridge could be used.

As a last comment, this is not possible to connect ports on cubes of different type, so unfortunately your idea will not work in this case.

@havok4u
Copy link
Author

havok4u commented Aug 28, 2019 via email

@mauriciovasquezbernal
Copy link
Contributor

I didn't say this explicitly but you're right, the performance of XDP_SKB and TC should be similar, XDP_SKB is mainly intended for testing, that's the reason for keeping it.

In production only TX or XDP_DRV modes should be used.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working good first issue Good for newcomers
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@havok4u @sebymiano @frisso @mauriciovasquezbernal @goldenrye @acloudiator