Allow setting up "socket" network for VM-to-VM communication

[afbjorklund]

Description

Currently lima sets up a "user" network, so that each VM gets host and internet access.

It would be possible to set up a second "socket" network, so that they could reach eachother.

https://wiki.qemu.org/Documentation/Networking

I'm not sure what would be the best YAML configuration, but made a small hack PoC to set it up.

Basically I added one boolean to say if it is the listen or the connect, and also the port number.

diff --git a/pkg/limayaml/limayaml.go b/pkg/limayaml/limayaml.go
index 47ac646..d91f002 100644
--- a/pkg/limayaml/limayaml.go
+++ b/pkg/limayaml/limayaml.go
@@ -22,6 +22,8 @@ type LimaYAML struct {
        Probes            []Probe           `yaml:"probes,omitempty" json:"probes,omitempty"`
        PortForwards      []PortForward     `yaml:"portForwards,omitempty" json:"portForwards,omitempty"`
        Message           string            `yaml:"message,omitempty" json:"message,omitempty"`
+       SocketPort        *int              `yaml:"socketPort,omitempty" json:"socketPort,omitempty"`
+       SocketMain        *bool             `yaml:"socketMain,omitempty" json:"socketMain,omitempty"`
        Networks          []Network         `yaml:"networks,omitempty" json:"networks,omitempty"`
        Network           NetworkDeprecated `yaml:"network,omitempty" json:"network,omitempty"` // DEPRECATED, use `networks` instead
        Env               map[string]string `yaml:"env,omitempty" json:"env,omitempty"`
diff --git a/pkg/qemu/qemu.go b/pkg/qemu/qemu.go
index d681400..8f582a7 100644
--- a/pkg/qemu/qemu.go
+++ b/pkg/qemu/qemu.go
@@ -277,6 +277,15 @@ func Cmdline(cfg Config) (string, []string, error) {
        args = append(args, "-netdev", fmt.Sprintf("user,id=net0,net=%s,dhcpstart=%s,hostfwd=tcp:127.0.0.1:%d-:22",
                qemu.SlirpNetwork, qemu.SlirpIPAddress, cfg.SSHLocalPort))
        args = append(args, "-device", "virtio-net-pci,netdev=net0,mac="+limayaml.MACAddress(cfg.InstanceDir))
+       if y.SocketPort != nil && *y.SocketPort != 0 {
+               port := *y.SocketPort
+               if y.SocketMain != nil && *y.SocketMain == true {
+                       args = append(args, "-netdev", fmt.Sprintf("socket,id=net1,listen=127.0.0.1:%d", port))
+               } else {
+                       args = append(args, "-netdev", fmt.Sprintf("socket,id=net1,connect=127.0.0.1:%d", port))
+               }
+               args = append(args, "-device", "virtio-net-pci,netdev=net1,mac="+limayaml.MACAddress(cfg.InstanceDir + strconv.Itoa(port)))
+       }
        if len(y.Networks) > 0 && !strings.Contains(string(features.NetdevHelp), "vde") {
                return "", nil, fmt.Errorf("netdev \"vde\" is not supported by %s ( Hint: recompile QEMU with `configure --enable-vde` )", exe)
        }

So the first VM goes

socketMain: true
socketPort: 1234

And the second goes

socketMain: false
socketPort: 1234

And they each can configure eth1, to form a mutual network.

eth0      Link encap:Ethernet  HWaddr 52:55:55:3F:BC:6F  
          inet addr:192.168.5.15  Bcast:0.0.0.0  Mask:255.255.255.0
          inet6 addr: fe80::5055:55ff:fe3f:bc6f/64 Scope:Link
          inet6 addr: fec0::5055:55ff:fe3f:bc6f/64 Scope:Site
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:3692 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1304 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:3619251 (3.4 MiB)  TX bytes:131459 (128.3 KiB)

eth1      Link encap:Ethernet  HWaddr 52:55:55:6D:2C:78  
          inet addr:10.0.2.100  Bcast:10.0.2.255  Mask:255.255.255.0
          inet6 addr: fe80::5055:55ff:fe6d:2c78/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:25 errors:0 dropped:0 overruns:0 frame:0
          TX packets:37 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:3261 (3.1 KiB)  TX bytes:4517 (4.4 KiB)

eth0      Link encap:Ethernet  HWaddr 52:55:55:C7:56:92  
          inet addr:192.168.5.15  Bcast:0.0.0.0  Mask:255.255.255.0
          inet6 addr: fe80::5055:55ff:fec7:5692/64 Scope:Link
          inet6 addr: fec0::5055:55ff:fec7:5692/64 Scope:Site
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:3134 errors:0 dropped:0 overruns:0 frame:0
          TX packets:986 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:3122371 (2.9 MiB)  TX bytes:92152 (89.9 KiB)

eth1      Link encap:Ethernet  HWaddr 52:55:55:EA:F8:43  
          inet addr:10.0.2.101  Bcast:10.0.2.255  Mask:255.255.255.0
          inet6 addr: fe80::5055:55ff:feea:f843/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:23 errors:0 dropped:0 overruns:0 frame:0
          TX packets:39 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:3441 (3.3 KiB)  TX bytes:4337 (4.2 KiB)

[afbjorklund]

Here is a nice variant, using multicast for the second interface and showing how to run a DHCP server on the host:

https://gist.github.com/mcastelino/88195a7d99811a177f5e643d1465e19e

-netdev socket,id=mynet0,listen=:1234
-netdev socket,id=mynet0,connect=:1234

-netdev socket,id=mynet0,mcast=230.0.0.1:1234
-netdev socket,id=mynet0,mcast=230.0.0.1:1234

Alternatively one could use static network addresses*, or run the DHCP server in one of the VMs (e.g. the "listen")

* that is what I did, in the example above

[afbjorklund]

More advanced external approach:

• Enable guest-to-guest networking by default, with (gvisor-based) usermode networking #1222