Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support BESS protocol (for UML) #83

Merged
merged 1 commit into from
Jan 6, 2022
Merged

Conversation

AkihiroSuda
Copy link
Contributor

BESS protocol transferrs L2 packets as AF_UNIX SOCK_SEQPACKET .
BESS protocol has been used by the vector network interfaces of User Mode Linux (UML).

(terminal 1) $ bin/gvproxy -debug -listen unix:///tmp/network.sock -listen-bess unixpacket:///tmp/bess.sock
(terminal 2) $ linux.uml vec0:transport=bess,dst=/tmp/bess.sock,depth=128,gro=1,mac=5a:94:ef:e4:0c:ee root=/dev/root rootfstype=hostfs init=/bin/bash mem=2G
(terminal 2: UML)$ ip addr add 192.168.127.2/24 dev vec0
(terminal 2: UML)$ ip link set vec0 up
(terminal 2: UML)$ ip route add default via 192.168.127.254

More docs about the User Mode Linux with BESS socket transport: https://www.kernel.org/doc/html/latest/virt/uml/user_mode_linux_howto_v2.html#bess-socket-transport

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jan 5, 2022

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: AkihiroSuda

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@AkihiroSuda AkihiroSuda force-pushed the bess branch 2 times, most recently from 87dbcf2 to 229ca40 Compare January 5, 2022 07:15
BESS protocol transferrs L2 packets as AF_UNIX SOCK_SEQPACKET .
BESS protocol has been used by the vector network interfaces of User Mode Linux (UML).

```
(terminal 1) $ bin/gvproxy -debug -listen unix:///tmp/network.sock -listen-bess unixpacket:///tmp/bess.sock
(terminal 2) $ linux.uml vec0:transport=bess,dst=/tmp/bess.sock,depth=128,gro=1,mac=5a:94:ef:e4:0c:ee root=/dev/root rootfstype=hostfs init=/bin/bash mem=2G
(terminal 2: UML)$ ip addr add 192.168.127.2/24 dev vec0
(terminal 2: UML)$ ip link set vec0 up
(terminal 2: UML)$ ip route add default via 192.168.127.254
```

More docs about the User Mode Linux with BESS socket transport: https://www.kernel.org/doc/html/latest/virt/uml/user_mode_linux_howto_v2.html#bess-socket-transport

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
@AkihiroSuda
Copy link
Contributor Author

CI failure is probably unrelated.
The main branch has been failing too: https://github.com/containers/gvisor-tap-vsock/runs/4706874516?check_suite_focus=true

@guillaumerose
Copy link
Contributor

Yes I am looking at the CI. I don't understand for the moment why it hangs :/

Thanks for the patch, looks good. Do you use or plan to use gvproxy in one of your project?

@guillaumerose
Copy link
Contributor

Tests are passing on my machine. lgtm.

Maybe in future we could add e2e tests for this feature.

@guillaumerose guillaumerose merged commit 50daff3 into containers:main Jan 6, 2022
@AkihiroSuda
Copy link
Contributor Author

Do you use or plan to use gvproxy in one of your project?

Maybe, still under an early experiment.

@AkihiroSuda

This comment has been minimized.

@guillaumerose
Copy link
Contributor

Interesting! What happens if you change the MTU? It should be better.

For stability, maybe check the stats API. There is maybe a clue.

@guillaumerose
Copy link
Contributor

guillaumerose commented Jan 6, 2022

Retr column seems really bad. Maybe you can compare in stats API the number of Retransmit you see with the Retr column.

Maybe there is something to do/change in the way the listener in Go is configured ? Some packets are dropped ?
I had issues with conn.Read with Qemu. At full speed, some packets were lost. io.ReadFull saved me.

@AkihiroSuda
Copy link
Contributor Author

My guest kernel config was unoptimized, tested again with a new config
dot.config.txt

slirp4netns

root@(none):/# iperf3 -c 10.0.2.2
Connecting to host 10.0.2.2, port 5201
[  5] local 10.0.2.15 port 47866 connected to 10.0.2.2 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   116 MBytes   968 Mbits/sec    0    104 KBytes       
[  5]   1.00-2.00   sec   114 MBytes   962 Mbits/sec    0    104 KBytes       
[  5]   2.00-3.00   sec   112 MBytes   938 Mbits/sec    0    104 KBytes       
[  5]   3.00-4.00   sec   112 MBytes   938 Mbits/sec    0    104 KBytes       
[  5]   4.00-5.00   sec   110 MBytes   923 Mbits/sec    0    104 KBytes       
[  5]   5.00-6.00   sec   108 MBytes   905 Mbits/sec    0    104 KBytes       
[  5]   6.00-7.00   sec   109 MBytes   914 Mbits/sec    0    104 KBytes       
[  5]   7.00-8.00   sec   109 MBytes   914 Mbits/sec    0    104 KBytes       
[  5]   8.00-9.00   sec   109 MBytes   919 Mbits/sec    0    104 KBytes       
[  5]   9.00-10.00  sec   108 MBytes   910 Mbits/sec    0    104 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  1.08 GBytes   929 Mbits/sec    0             sender
[  5]   0.00-10.00  sec  1.08 GBytes   929 Mbits/sec                  receiver

iperf Done.

gvproxy

root@(none):/# iperf3 -c 192.168.127.254
Connecting to host 192.168.127.254, port 5201
[  5] local 192.168.127.2 port 50452 connected to 192.168.127.254 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  37.1 MBytes   311 Mbits/sec  515    272 KBytes       
[  5]   1.00-2.00   sec  30.8 MBytes   258 Mbits/sec  791    327 KBytes       
[  5]   2.00-3.00   sec  18.0 MBytes   151 Mbits/sec  400    346 KBytes       
[  5]   3.00-4.00   sec  10.2 MBytes  85.4 Mbits/sec  403    250 KBytes       
[  5]   4.00-5.00   sec  16.5 MBytes   138 Mbits/sec  1035    262 KBytes       
[  5]   5.00-6.00   sec  16.7 MBytes   140 Mbits/sec  692    163 KBytes       
[  5]   6.00-7.00   sec  22.4 MBytes   188 Mbits/sec  1133    236 KBytes       
[  5]   7.00-8.00   sec  45.1 MBytes   378 Mbits/sec  473    399 KBytes       
[  5]   8.00-9.00   sec  15.8 MBytes   132 Mbits/sec  483    113 KBytes       
[  5]   9.00-10.00  sec  47.7 MBytes   400 Mbits/sec  947    175 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   260 MBytes   218 Mbits/sec  6872             sender
[  5]   0.00-10.00  sec  0.00 Bytes  0.00 bits/sec                  receiver
iperf3: error - control socket has closed unexpectedly

Slirp4netns provides an acceptable throughput, but something is obviously broken in gvproxy.
Taking a look.

Interesting! What happens if you change the MTU? It should be better.

UML doesn't seem to support MTU > 1500.

@trevor403
Copy link

Great news!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants