[workspacekit] Use veth instead of slirp4netns #8106
Comments
csweichel
added
component: ws-daemon
aspect: performance
|
Did some research into how a veth pair can be provided. The assumption is that you have some shell running with The propagation unchanged part is import as unshare by default will mount the rootfs as private in the new mount namespace, so mount events in the host mount namespace will not propagate to the new mount namespace. This means that after moving one end of the veth pair in the net namespace it will show "peer reference not found" when listing the network interfaces with I recommend saving your iptables rules with iptables-save or using a virtual machine to avoid messing up your system. |
|
I feel that perhaps @Furisto's method would work with IWS and nsinsider. (I will make a sequence diagram at a later date for more details) @csweichel @Furisto |
|
Bypass4netns was a hot contender. We can't just drop it in and use it though because we already use seccomp-notify. We'd need to integrate it with workspacekit. It does solve a different problem though. We do have a privileged process we can externalise operations to, unlike what bypass4netns was built for. |
|
I had not come up with a good way to go about this either. I was thinking that if there is a way to do it, it would be the best option.
|
|
Manually created a pair of veth and successfully communicated with the outside world. ##### IN HOST ######
# Enable forwarding
echo 1 > /proc/sys/net/ipv4/ip_forward
# enter into the target pod pid
nsenter -t 212137 -n -m -p bash
##### IN POD'S NAMESPACE ######
# Set the values
INSTANCE_ID=w2
WORKSPACE_PID=57
# Create veth pair
ip link add veth-$INSTANCE_ID type veth peer name ceth-$INSTANCE_ID
# Turn anonymous network namespace into named namespace and
# attach one end of veth pair to network namespace
# ip netns attach netns-<instanceId> <pid>
mkdir -p /var/run/netns
ln -s /proc/$WORKSPACE_PID/ns/net /var/run/netns/netns-$INSTANCE_ID
ip link set ceth-$INSTANCE_ID netns netns-$INSTANCE_ID
# Assign IP addresses to both ends of veth pair
ip addr add 10.0.5.1/24 dev veth-$INSTANCE_ID
ip netns exec netns-$INSTANCE_ID ip addr add 10.0.5.2/24 dev ceth-$INSTANCE_ID
# Bring up network interfaces
ip link set veth-$INSTANCE_ID up
ip netns exec netns-$INSTANCE_ID ip link set ceth-$INSTANCE_ID up
ip netns exec netns-$INSTANCE_ID ip link set lo up
# Add iptables rules
iptables -A FORWARD -o eth0 -i veth-$INSTANCE_ID -j ACCEPT
iptables -A FORWARD -i eth0 -o veth-$INSTANCE_ID -j ACCEPT
iptables -t nat -A POSTROUTING -s 10.0.5.0/24 -o eth0 -j MASQUERADE
# Setup DNS
mkdir -p /etc/netns/netns-$INSTANCE_ID
ln -s /etc/resolv.conf /etc/netns/netns-$INSTANCE_ID/resolv.conf
# Set default gateway
ip netns exec netns-$INSTANCE_ID ip route replace default via 10.0.5.1### in workspace's namespace ###
root@gfe3de62b9ab20671cf9d9f:/# traceroute 8.8.8.8
traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets
1 10.0.5.1 (10.0.5.1) 0.036 ms 0.005 ms 0.004 ms
2 10-132-0-67.kubernetes.default.svc.cluster.local (10.132.0.67) 0.091 ms 0.063 ms 0.040 ms
3 * * *
...
12 * * *
13 dns.google (8.8.8.8) 1.023 ms 0.786 ms 0.338 msHowever, when I changed the default gateway, the workspace kept reconnecting. I am not sure if this is because I manually changed the settings of a workspace that has finished starting, or if the original default gateway (10.0.0.2) has a special meaning. |
|
@utam0k Can reproduce, even switching the default route back to the tap device does not help. |
|
@Furisto Thanks for your challenge! But, in my environment, reverting to 10.0.2.2 restored it properly. |
|
@csweichel @Furisto Here is in ring2. Preview environment is here. Logs is here |
|
Now I understand why. We needed the ability to dynamically expose ports. |
|
(Updated 2022-03-31)
If you are not familiar with this field(me too), this article would be very helpful. previous picture |
|
A deployment plan that ensures that the IDE and workspace teams do not influence each other. @utam0k will create all PRs we need. Each checklist was assigned a person in charge of marking the checklist. This is to avoid miscommunication by marking the checklist when deployment is complete. The person in charge should mark the checklist when the item is completely finished.
|
|
Hi, @iQQBot, Was no3 finished? Can we go to the next step? |
|
@utam0k We've deployed the new version of supervisor and can move on to the next step |
Is your feature request related to a problem? Please describe
Today we use slirp4netns to connect the network namespace of our workspaces. While convenient and easy to use, this comes at a runtime cost because we make networking a userland issue.
Describe the behaviour you'd like
Instead, we should defer to the ws-dameon IWS to create
vethpairs between ring1 and the workspace pod.Also, if there's an opportunity to improve the logging, that would be great.
Describe alternatives you've considered
Maybe IPVLAN or MACVLAN work nicer because we don't have to setup routing in the workspace pod - but the interaction with CNI is unclear.
The text was updated successfully, but these errors were encountered: