Unable to listen and connect to a port on the same stack

[MagnusS]

I'm unable to get the TCP/IP stack to connect to its own listening ports. The unikernel seems to try to resolve its own IP in ARP, but not sure if that's the only problem. I've written a test that currently fails in PR #132

[yomimono]

Dropping some logic into ipv4 that automatically resolves "our own" IP to the correct MAC reveals another problem, at least for tests: vnetif doesn't send traffic where the source is equal to the destination:

https://github.com/MagnusS/mirage-vnetif/blob/master/lib/basic_backend.ml#L86

Generally, I'm not sure whether we can count on a network interface to give us back traffic that we sent in the first place -- perhaps you have a better sense of this?

[MagnusS]

Yes, that check should probably be more specific. Right now it doesn't inspect the frame, it only relays the payload to every other connected node (except the sender) - so if the dst mac belongs to the sender the frame is lost. I've added an issue here.

[mor1]

FWIW this is something that occurs in OpenFlow too -- by default, frames are not forwarded out of the interface on which they arrived, though you can set an explicit flag that disable this check. (Of note when, eg., using Open vSwitch to manage wireless interfaces, where it is quite common that incoming and outgoing interfaces will be the same.)

Making it explicitly controllable might be something worth doing here too?

[MagnusS]

Hm - so the TCP stack should probably try to figure out that it is talking to itself without relying on netif then?

[yomimono]

That was my first thought, but I ran into a wall thinking of how to do anything with that information (although we can certainly discover it on the ip/ether layer), since we have no knowledge of the functions that may have been called from listen in the context of the write functions. (In other words, it's easy to know that we're talking to ourselves but more difficult to know how to answer.)

[mor1]

Possibly naive question -- how is it a problem if the TCP/IP stack connects to its own listening ports?

[yomimono]

@mor1 , we can do that easily with the stack-level abstractions (STACKV1) because we have a hashtable of the listeners in the record, but not so in modules like TCP or UDP directly.

[yomimono]

I've been thinking about this, and I'm not sure when you'd want to have a stack connect to itself anyway. In UNIX-land you do this because you have processes that want to communicate and it's the most convenient abstraction, but we're not in that model.

[yomimono]

Unless anyone objects, I'd like to close this and the related pull request #132 as WONTFIX - I haven't heard any compelling reason to put the required work in to make this case work. Please let me know if you have one!

[mor1]

Go for it. If someone comes up with one, it can always be reopened.

[MagnusS]

I'm fine with closing as WONTFIX :-) IIRC my original usecase was to test if a port was open/responding correctly from within the same unikernel.

[talex5]

Yeah, I agree on closing this.

For unit-tests, we can easily create a virtual network with multiple interfaces.

@yomimono: if I understand the correctly, the problem with handling this at the IP layer is:

1. The callbacks (tcp, udp, etc) are passed to the IPv4.input function and could (in theory) be different for every frame.
2. So in IPv4.write there is no way to know how to handle the packet, because that decision is made by the application code when it receives an ethernet frame, and depends on the frame (which doesn't exist in the loopback case).

A possible (API-breaking) solution would be to move the handlers to the constructor. e.g. instead of

IP.connect eth >>= fun ip ->
Net.listen (... IP.input ip ~tcp ~udp frame)

we'd have

IP.connect ~tcp ~udp eth >>= fun ip ->
Net.listen (... IP.input ip frame)

However, this is less flexible. For example, mirage-firewall passes the ethernet frame up to its IP handling code, which it couldn't do if ETHIF used this style.

[yomimono]

Yes, it also means that if the application wants to change anything about the listening logic during execution, it needs to disconnect and reconnect the device. I think that's not undesirable.

[rixed]

My first two unikernels are a monitoring service that tries to monitor itself and hangs, and a registration service that tries to register itself and hangs... both because of this.
Please reconsider :-)