sockets, main event loop in xs x-cli-lin platform

[dckc]

The xs socket module for Linux desktop is designed to run in the context of the simulator. It uses GTK. Getting it to work in the x-cli-lin platform might be a little tricky.

done except:

• moddable to merge only pick ipv4 addresses from g_resolver Moddable-OpenSource/moddable#304
• linux CLI platform (for agoric-sdk builds) Moddable-OpenSource/moddable#305 including exposing moddable internal mxPromiseStatus() macro as a fxPromiseIsPending()
  • using a submodule instead
• escalate ipv6 to its own issue for tracking?

see also Moddable-OpenSource/moddable#298

cc @dtribble @warner

[dckc]

I got something sorta working in https://github.com/dckc/moddable/tree/ag-linux-cli :

• 423c2c1: lin_xs_cli: connect glib event loop with fxQueuePromiseJobsCallback

I think this works as long as we're happy to run the event loop indefinitely or until we explicitly exit() or some such.

But if we want the nice node.js-like behavior where the process exits when there are no more listeners, timers, etc., I don't see how to do that.

[dckc]

gSockets is global mutable state. That makes me uneasy.

And the code doest seem to use glib's poll support, though it uses glib for the promise queue. Odd mixture.

[dckc]

The most convenient APIs are in gio (e.g. socket-client.c), but...

GIO has extension points whose implementations get loaded from modules (executable code in shared objects), which could allow an attacker to sneak his own code into your application by tricking it into loading the code as a module. -- https://developer.gnome.org/gio/stable/ch02.html

Hmm... modResolve.c uses g_resolver_get_default(), which is in gio.

xs/platforms/lin_xs.h includes <gio/gio.h> and uses GSocket* socket for the debugger interface.

handy docs etc:

https://developer.gnome.org/glib/stable/glib-The-Main-Event-Loop.html
https://developer.gnome.org/glib/stable/glib-IO-Channels.html
https://pubs.opengroup.org/onlinepubs/009695399/basedefs/netinet/in.h.html
http://man7.org/linux/man-pages/man7/socket.7.html

https://www.gnu.org/software/libc/manual/html_node/Server-Example.html#Server-Example

dns https://gist.github.com/fffaraz/9d9170b57791c28ccda9255b48315168

to avoid ambient default resolver:
https://github.com/dckc/agoric-sdk/blob/xs-net-rt/packages/cosmic-swingset/rt/nametoip.js

perhaps TODO:
https://developers.cloudflare.com/1.1.1.1/dns-over-https/json-format/
https://tools.ietf.org/html/rfc8427

[dckc]

gSockets is global mutable state. That makes me uneasy.

I came up with something that gets rid of gSockets altogether, using glib mainloop stuff instead.

@bfriedkin I wonder what you think of it:
https://github.com/dckc/moddable/blob/67d99e74330afd67620b46d89119557c98956029/modules/network/socket/lin/modSocket.c

The remaining ambient authority seems to be:

• g_main_context_default() (and hence poll and the like) via g_io_add_watch_full(), g_source_remove()
• socket(), setsockopt()
• send(), sendto(), recv(), recvfrom()
• bind(), listen(), accept(), getpeername()
• g_resolver_get_default()

I'm tempted to remove all that ambient authority by adding those to mxMachinePlatform. But when I started doing it, I realized it was going to be a bunch of work and the benefit wasn't completely clear.

[dckc]

I talked with @warner about g_resolver_get_default() vs. nametoip.js and he raised the issue of where do we get the address of the DNS service? I was comfortable to have the user provide it, but he points out that adds a system administration burden to using ag-solo.

In discussion of some options, he suggested that DNS service could be in a compartment that's only available to (some) devices and not to the kernel. I think I can make my peace with that.

[bfriedkin]

@dckc -

Thank you for the rework to eliminate the global socket list. I have follow-up with you offline to discuss the implementation.

Regards,
Brian

[dckc]

xs modSocket was ipv4 everywhere but DNS

Most of mockSocket.c uses AF_INET, which is IPv4-only. So when the first address from the resolver was ipv6, things didn't work so well. Stepping back from the global socket list and such, I found an 11 line fix (05f5d62) to skip over ipv6 addresses. Provided (a) we're OK making ipv6 a separate issue and (b) the moddable team merges Moddable-OpenSource/moddable#304 , I think this issue is addressed.

Moreover, I think I'll expand the scope of this issue to include mainloop stuff, since I think I figured that out too...

[dckc]

exit when promise from main() resolves

My solution to the "exit where there is no more work to do" is to (a) have the main() function return a promise, and (b) run the main loop until that promise is resolved.
https://github.com/dckc/moddable/blob/ag-linux-cli/xs/platforms/lin_xs_cli.c#L54-L69

I don't yet have this part packaged as a PR. It involves exposing an internal mxPromiseStatus() macro as a fxPromiseIsPending() function in the API.
dckc/moddable@8d81d24 (and 2 previous)

build linux cli SDK using github actions workflow

A big motivation to tackle the main loop issue was automated testing. So I packaged up this version of the moddable SDK for use in agoric-sdk actions:

https://github.com/dckc/moddable/releases/download/ag08/moddable-linux-sdk.tgz
https://github.com/dckc/moddable/releases/tag/ag08 ff41561

The moddable packaging is done with a github action. I wonder if the moddable folks would be interested in this workflow as a PR. It's specific to the linux CLI:
https://github.com/dckc/moddable/blob/ag-linux-cli/.github/workflows/build-release.yaml
https://github.com/dckc/moddable/commit/ff4156183ca37dbdcdb68d9f8f208be6f09ea584/checks?check_suite_id=375323280

Agoric/moddable repo?

Maybe it's time for this stuff to go under the Agoric github org? I suppose at least one other person should take a close look. Any suggestions, @dtribble ? Maybe @jfparadis ? @warner ? Or should we skip Agoric and aim to push all my work upstream to Moddable?