docker provider tries to recreate running containers with swarm

[maguec]

setup a swarm cluster ( I used https://github.com/shokunin/puppet-docker-swarm )

# Configure the Docker provider
provider "docker" {
    host = "tcp://${var.swarm_host}:${var.swarm_port}/"
}

# Create a container
resource "docker_container" "redis" {
    image = "${docker_image.redis.latest}"
    name = "redis1"
}

resource "docker_image" "redis" {
    name = "redis:latest"
}

run terraform apply and the container gets created properly and is running

$ docker -H tcp://localhost:2378 ps
CONTAINER ID        IMAGE                                                                     COMMAND                CREATED             STATUS              PORTS               NAMES
65c285bf8a65        95af5842ddb9b03f7c6ec7601e65924cec516fcedd7e590ae31660057085cf67:latest   "/entrypoint.sh redi   28 minutes ago      Up 25 minutes       6379/tcp            node2/redis1 

run terraform apply again and it says the resource has already been created

* Unable to create container: API error (409): Conflict, The name redis1 is already assigned to 65c285bf8a657eaff12c35f7d7df58a01d588d0bf4d8e6c7fa18889b52a1e83e. You have to delete (or rename) that container to be able to assign redis1 to a container again.

Looking at a tcpdump of the second call to terraform apply I see the following traffic where the images are listed, but nothing ever checks to see if any container with that name is running before attempting to create it again

GET /_ping HTTP/1.1
Host: localhost:2378
User-Agent: go-dockerclient
Accept-Encoding: gzip

HTTP/1.1 200 OK
Date: Fri, 15 May 2015 04:26:48 GMT
Content-Length: 2
Content-Type: text/plain; charset=utf-8

OKGET /_ping HTTP/1.1
Host: localhost:2378
User-Agent: go-dockerclient
Accept-Encoding: gzip

HTTP/1.1 200 OK
Date: Fri, 15 May 2015 04:26:48 GMT
Content-Length: 2
Content-Type: text/plain; charset=utf-8

OKGET /images/json HTTP/1.1
Host: localhost:2378
User-Agent: go-dockerclient
Accept-Encoding: gzip

HTTP/1.1 200 OK
Content-Type: application/json
Date: Fri, 15 May 2015 04:26:48 GMT
Content-Length: 1355

[{"Created":1431128117,"Id":"95af5842ddb9b03f7c6ec7601e65924cec516fcedd7e590ae31660057085cf67","ParentId":"eb3ba1568decb2efcd22b311d15fd59218fe58df24c453ce36af9151fef542a2","RepoTags":["redis:latest"],"Size":0,"VirtualSize":111053414,"Node":{"Cpus":1,"Memory":513798144}},{"Created":1430430613,"Id":"07f8e8c5e66084bef8f848877857537ffe1c47edd01a93af27e7161672ad0e95","ParentId":"37bea4ee0c816e3a3fa025f36127ef8ef0817b3f8fcd7b49eb7b26064f647bb0","RepoTags":["ubuntu:latest"],"Size":0,"VirtualSize":188304295,"Node":{"Cpus":1,"Memory":513798144}},{"Created":1429687130,"Id":"73504b2882a32542f56bffefc4f7b7a0606b285bc819bc161356b91f0e32b4b0","ParentId":"57ec2f5f3e06dd77bcfdfaa91781528f201edf2933dec96b87c650d3017011d7","RepoTags":["swarm:latest"],"Size":0,"VirtualSize":7247824,"Node":{"Cpus":1,"Memory":513798144}},{"Created":1430430613,"Id":"07f8e8c5e66084bef8f848877857537ffe1c47edd01a93af27e7161672ad0e95","ParentId":"37bea4ee0c816e3a3fa025f36127ef8ef0817b3f8fcd7b49eb7b26064f647bb0","RepoTags":["ubuntu:latest"],"Size":0,"VirtualSize":188304295,"Node":{"Cpus":1,"Memory":513798144}},{"Created":1429687130,"Id":"73504b2882a32542f56bffefc4f7b7a0606b285bc819bc161356b91f0e32b4b0","ParentId":"57ec2f5f3e06dd77bcfdfaa91781528f201edf2933dec96b87c650d3017011d7","RepoTags":["swarm:latest"],"Size":0,"VirtualSize":7247824,"Node":{"Cpus":1,"Memory":513798144}}]
GET /_ping HTTP/1.1
Host: localhost:2378
User-Agent: go-dockerclient
Accept-Encoding: gzip

HTTP/1.1 200 OK
Date: Fri, 15 May 2015 04:26:48 GMT
Content-Length: 2
Content-Type: text/plain; charset=utf-8

OKGET /_ping HTTP/1.1
Host: localhost:2378
User-Agent: go-dockerclient
Accept-Encoding: gzip

HTTP/1.1 200 OK
Date: Fri, 15 May 2015 04:26:48 GMT
Content-Length: 2
Content-Type: text/plain; charset=utf-8

OKGET /images/json HTTP/1.1
Host: localhost:2378
User-Agent: go-dockerclient
Accept-Encoding: gzip

HTTP/1.1 200 OK
Content-Type: application/json
Date: Fri, 15 May 2015 04:26:48 GMT
Content-Length: 1355

[{"Created":1431128117,"Id":"95af5842ddb9b03f7c6ec7601e65924cec516fcedd7e590ae31660057085cf67","ParentId":"eb3ba1568decb2efcd22b311d15fd59218fe58df24c453ce36af9151fef542a2","RepoTags":["redis:latest"],"Size":0,"VirtualSize":111053414,"Node":{"Cpus":1,"Memory":513798144}},{"Created":1430430613,"Id":"07f8e8c5e66084bef8f848877857537ffe1c47edd01a93af27e7161672ad0e95","ParentId":"37bea4ee0c816e3a3fa025f36127ef8ef0817b3f8fcd7b49eb7b26064f647bb0","RepoTags":["ubuntu:latest"],"Size":0,"VirtualSize":188304295,"Node":{"Cpus":1,"Memory":513798144}},{"Created":1429687130,"Id":"73504b2882a32542f56bffefc4f7b7a0606b285bc819bc161356b91f0e32b4b0","ParentId":"57ec2f5f3e06dd77bcfdfaa91781528f201edf2933dec96b87c650d3017011d7","RepoTags":["swarm:latest"],"Size":0,"VirtualSize":7247824,"Node":{"Cpus":1,"Memory":513798144}},{"Created":1430430613,"Id":"07f8e8c5e66084bef8f848877857537ffe1c47edd01a93af27e7161672ad0e95","ParentId":"37bea4ee0c816e3a3fa025f36127ef8ef0817b3f8fcd7b49eb7b26064f647bb0","RepoTags":["ubuntu:latest"],"Size":0,"VirtualSize":188304295,"Node":{"Cpus":1,"Memory":513798144}},{"Created":1429687130,"Id":"73504b2882a32542f56bffefc4f7b7a0606b285bc819bc161356b91f0e32b4b0","ParentId":"57ec2f5f3e06dd77bcfdfaa91781528f201edf2933dec96b87c650d3017011d7","RepoTags":["swarm:latest"],"Size":0,"VirtualSize":7247824,"Node":{"Cpus":1,"Memory":513798144}}]
POST /containers/create?name=redis1 HTTP/1.1
Host: localhost:2378
User-Agent: go-dockerclient
Content-Length: 76
Content-Type: application/json
Accept-Encoding: gzip

{"Image":"95af5842ddb9b03f7c6ec7601e65924cec516fcedd7e590ae31660057085cf67"}HTTP/1.1 409 Conflict
Content-Type: text/plain; charset=utf-8
Date: Fri, 15 May 2015 04:26:48 GMT
Content-Length: 211

Conflict, The name redis1 is already assigned to 65c285bf8a657eaff12c35f7d7df58a01d588d0bf4d8e6c7fa18889b52a1e83e. You have to delete (or rename) that container to be able to assign redis1 to a container again.

[grahamgreen]

👍

[jwthomp]

It appears that when a container is getting created through the swarm api, the .tfstate file is not getting filled out with all of the data.

{
    "version": 1,
    "serial": 1,
    "modules": [
        {
            "path": [
                "root"
            ],
            "outputs": {},
            "resources": {
                "docker_image.genepool": {
                    "type": "docker_image",
                    "primary": {
                        "id": "a66b67d1dbd23a6fc0866a160f58d1d80253b2a076a9ea48ba4648d347d5809bregistry:0.9.1",
                        "attributes": {
                            "id": "a66b67d1dbd23a6fc0866a160f58d1d80253b2a076a9ea48ba4648d347d5809bregistry:0.9.1",
                            "latest": "a66b67d1dbd23a6fc0866a160f58d1d80253b2a076a9ea48ba4648d347d5809b",
                            "name": "registry:0.9.1"
                        }
                    }
                }
            }
        }
    ]
}

This comes from a run where I create a docker_image and a docker_container. The container does run on the remote docker system, but it is not populated in the tfstate file. The image appears to work however.

[jwthomp]

I've narrowed the problem down to resourceDockerContainerRead in resource_docker_container_funcs.go. The issue appears to be that swarm appends the hostname the container is run on in front of the specified name.

So in the example I pasted above, 'genepool' is referenced by Swarm as 'hostname/genepool' When the code below executes, it tries to find 'genepool' through the Swarm API and fails. SetId is set to "" and the function returns.

func resourceDockerContainerRead(d *schema.ResourceData, meta interface{}) error {
    client := meta.(*dc.Client)

    apiContainer, err := fetchDockerContainer(d.Get("name").(string), client)
    if err != nil {
        return err
    }

    if apiContainer == nil {
        debug(fmt.Sprintf("Container does not exist anymore: %s\n", d.Get("name").(string)))
        // This container doesn't exist anymore
        d.SetId("")

        return nil
    }
....

Looking into a fix in 'fetchDockerContainer'

[jwthomp]

I have created a pull request that fixes this issue. #1990

[catsby]

This was fixed in #4148

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.