support connection helpers (e.g. docker -H ssh://me@server)

[AkihiroSuda]

Signed-off-by: Akihiro Suda suda.akihiro@lab.ntt.co.jp

- What I did

Added support for "connection helpers", which supports custom connection protocols. e.g. docker -H ssh://me@server.

This was originally suggested by @cpuguy83 moby/moby#31871 , as an alternative for my previous hard-coded ssh:// proposal: moby/moby#33566

- How I did it

cli/config/connhelper.GetConnectionHelper() returns net.Conn dialer for registered protocols like ssh://, dind://...

- How to verify it

Installation

1. Install contrib/connhelper/docker-connection-{dind,ssh} to $PATH.

2. Put the following config to ~/.docker/config.json:

{
...
 "connHelpers": {"ssh": "ssh", "dind": "dind"}
}

Keys are protocol names, values for helper binary names. (docker-connection- prefix is omitted, as in cred helpers)

use ssh://

1. Create a Docker host with SSH configured for public key authentication
2. Install socat to the host
3. docker -H ssh://me@server run -it --rm busybox

use dind://

1. docker run -d --name dind --privileged docker:edge-dind
2. docker -H dind://dind run -it --rm busybox

- Description for the changelog

support connection helpers (e.g. docker -H ssh://me@server)

- A picture of a cute animal (not mandatory but encouraged)

https://commons.wikimedia.org/wiki/File:Manchot_Ad%C3%A9lie_juv%C3%A9nile.jpg

---

Needs vendoring: moby/moby#36630

[cpuguy83]

Looks pretty cool!

[cpuguy83]

Pinging some other people to take a look at this as well, particularly the docker cloud folks who'd worked on the "docker cloud proxy" thing.

[tonistiigi]

This looks backwards to me. We should need helper binaries(or a flag that invokes custom functionality) on the daemon side, not client so that we don't have a dependency on socat and "dind" doesn't need any dockerd specific functionality in client.

Having a possibility to override the commands with configuration seems ok if there is some use-case but things like ssh:// should just work (and if there are no extra resources needed to support dind then I don't see a reason not to include it as well).

"dind://" seems more like "docker://" to me. It shouldn't be dockerd specific but just means that remote HTTP API should be used for making the connection to the helper binary. For example buildkit could work with docker run --name=buildkitd -d tonistiigi/buildkit && BUILDKIT_HOST=docker://buildkitd instead of opening a port that is needed currently.

[cpuguy83]

@tonistiigi Are you saying on the daemon to provide a way to setup custom listeners that we then serve the HTTP API over?

[tonistiigi]

@cpuguy83 No, daemon should provide a custom dialing command, eg. like rsync --server or systemd-stdio-bridge. Then docker client will just invoke that command using any transport they prefer, for example, through ssh or docker exec.

[AkihiroSuda]

@tonistiigi

Sorry I'm not sure I got what you meant.
Do you expect the local daemon (including D4M, D4W) to be always available, and the local daemon to proxy remote daemons?

If we want to reuse helper binaries for other programs, I think we can remove the default /var/run/docker.sock path. e.g. DOCKER_HOST=ssh://me@server/var/run/docker.sock, BUILDKIT_HOST=ssh://me@server/run/builtkitd.sock...

[tonistiigi]

Do you expect the local daemon (including D4M, D4W) to be always available, and the local daemon to proxy remote daemons?

No. I think the confusion comes because we use "helper binary" with very different meanings. I don't see big value in having custom dialer binaries in the client, we have dialers for unix, tcp, tls etc. so why not ssh and docker? This is a super helpful feature, why make it complicated to use? Otoh, I do find it valuable if we had a helper binary on daemon side so that we don't have a hard dependency on socat for this or leak a container to docker ps for every connection. Especially because it is very simple logic to implement. I expect that a helper binary/flag in the same host as dockerd can dial into local dockerd to access the API. I don't even mind if socat is the fallback for older daemons.

If we want to reuse helper binaries for other programs, I think we can remove the default /var/run/docker.sock path. e.g. DOCKER_HOST=ssh://me@server/var/run/docker.sock, BUILDKIT_HOST=ssh://me@server/run/builtkitd.sock

A bigger problem is all the docker specific stuff currently in dind. If there was daemon side helper binary it would already know how to connect by default.

[AkihiroSuda]

OK, makes sense.

we have dialers for unix, tcp, tls etc. so why not ssh and docker? This is a super helpful feature, why make it complicated to use?

I think we should not enable ssh:// and docker:// by default, until we can get usability and security assessment from users.
Maybe we should put these functionalities (I mean, scheme:// resolver, not socat-like stuff) to the docker binary but disable unless CLI is running as experimental mode?

@cpuguy83 WDYT?

[tonistiigi]

I think we should not enable ssh:// and docker:// by default, until we can get usability and security assessment from users.

I disagree on ssh. It is old and very widely used, both by users and other tools needing similar functionality. Nothing experimental about that.

[cpuguy83]

Didn't we have a bunch of issues implementing ssh because of either:

1. Using pure go implementation we lose a lot of client features
2. Using distro client, it may or may not be available (particularly on Windows)

[tonistiigi]

@cpuguy83 And I think we decided to go with the conservative option(that everyone else uses). If this was reimplementing ssh I would definitely want it to be experimental or opt-in.

[codecov-io]

Codecov Report

Merging #889 into master will decrease coverage by 0.03%.
The diff coverage is 48.14%.

@@            Coverage Diff             @@
##           master     #889      +/-   ##
==========================================
- Coverage   53.92%   53.88%   -0.04%     
==========================================
  Files         262      262              
  Lines       16604    16621      +17     
==========================================
+ Hits         8954     8957       +3     
- Misses       7049     7061      +12     
- Partials      601      603       +2

[AkihiroSuda]

I do find it valuable if we had a helper binary on daemon side so that we don't have a hard dependency on socat for this

On second thought I think we should embed this socat-like functionality into docker client binary rather than dockerd daemon binary, because

• Regular user who is allowed to execute docker is not guaranteed to be allowed to execute dockerd.
• dockerd binary name may change in future Moby releases.

WDYT?

[tonistiigi]

We discussed this in the maintainers meeting with @thaJeztah @cpuguy83

We would like to implement SSH support similarly to the other tools that use this pattern, eg. systemd, and have it enabled by default.

The cli should accept ssh://me@server for DOCKER_HOST and -H. Using that would execute ssh with the passed config. The ssh command would call a hidden command on the docker CLI binary on the remote side. For example, docker dial-stdio. This command will make a connection to the local DOCKER_HOST variable (almost always the default local socket) and forward that connection on the commands stdio. Even though this command is supposed to run locally to the dockerd binary, we think that it is an invalid configuration for this feature to remove the local docker binary so we can rely on it always being present.

If calling the remote command fails there can be a fallback using socat and hardcoded default socket path.

The other possible transports seem less critical atm. We can discuss supporting them after SSH is merged. Including the discussion, if some of them should be behind experimental flags or special config.

[thaJeztah]

ping @AkihiroSuda wdyt?

[AkihiroSuda]

+1, will update soon

[AkihiroSuda]

opened #1014