Docker Remote Engine

[nicholasjackson]

This PR enables the ability for the Docker plugin to target a remote docker engine when doing a deployment.

While it is possible to target the Remote Docker Engine using the environment variables DOCKER_HOST this would take effect for the build and deploy phases. It is not possible to build locally and then deploy to a remote instance. The individual options can be specified independently.

project = "wpmini"

app "wpmini" {
  labels = {
    "service" = "wpmini",
    "env"     = "dev"
  }

  build {
    use "pack" {}
  }

  deploy {
    use "docker" {
      client_config {
        host = "tcp://remote.ip:2375"
       cert_path = "./remote_certs"
       api_version = "19.03"
    }
  }
}

In addition, this PR checks the presence of a Docker Image in the Engine cache before attempting to start a container, an option has been created to force pull the image regardless of it being cached locally.

  deploy {
    use "docker" {
      force_pull  = true
    }
 }

[mitchellh]

Looks great! Thanks 😄

[elimisteve]

@nicholasjackson Congratulations on Waypoint! Seems compelling.

Can I use the above feature to build a container image locally and push directly to a remote Docker server, or must I first publish to a registry and deploy by telling the remote Docker server to pull from a registry? (I'm looking to not have to deal with registries at all and either push directly or do what rkt awesomely enabled: cryptographically signing container images + pulling them from anywhere, including behind Nginx, S3, etc without needing to mess with a registry nor trust such a registry.)

Thanks!

[julioselva]

@nicholasjackson please, how i am supposed to pull the container image from my remote docker host if it's not logged in my private registry?

project = "example-nodejs"

app "example-nodejs" {
  labels = {
    "service" = "example-nodejs",
    "env" = "dev"
  }

  build {
    use "pack" {}

    registry {
      use "aws-ecr" {
        region     = "us-east-1"
        repository = "test"
        tag        = "test"
      }
    }
  }

  deploy { 
    use "docker" {
      force_pull = true

      client_config {
        host = "tcp://my.remote.docker.host:2375"
        api_version = "1.41"
      }
    }
  }
}

[elimisteve]

@juliomsilva Ever get an answer? Looks like we're looking for similar functionality: deployments without involving a public registry.

[briancain]

@elimisteve @juliomsilva Is this what you're looking for? https://www.waypointproject.io/docs/lifecycle/build#private-registries

If that isn't what you're looking for, or if it doesn't work, please feel free to open an issue (enhancement or bug depending 😄 ) and we'll take a look!

[julioselva]

@elimisteve are these deployments using remote docker machines? if so, please let me know, because I was not able to figure it out. thank you!

[josephlim75]

Just curious, there are 2 types of docker deployment orchestration, one is using docker run, whereas another is docker stack deploy OR docker service (Both are using Docker Swarm mode). Is there a way for Waypoint to differentiate how deployment should be used if docker is being used as the deployment target? Would it be better to have some kind of differentiation on how we should initiate a deployment in docker?

# Docker swarm mode deployment. DOCKER_HOST must point to a Swarm Manager, allow Swarm manager to decide where the container will be deployed
deploy {
   use "docker" {
      orchestration = "swarm"
   }
}

# Regular docker run deploy.  DOCKER_HOST can be any Docker host, and the container will be deployed into the specific DOCKER_HOST
deploy {
   use "docker" {
      orchestration = "direct"
   }
}

For the time being, I've suggested the following in another discussion, but haven't tried it yet if it will work

deploy {
  use "exec" {
    command = ["docker", "stack", "deploy", "-c", "<TPL>", "stackname"]

  template {
    path = "./template/docker-compose.yaml"
  }
 }
}

By the way, does waypoint hcl allowed to pass in environment variables?

Assuming I have an exported variable call DOCKER_STACK, can I do the following, will it expand the environment variable?

deploy {
  use "exec" {
    command = ["docker", "stack", "deploy", "-c", "<TPL>", "${DOCKER_STACK}"]

  template {
    path = "./template/docker-compose.yaml"
  }
 }
}