Can't reattach volumes with new contents while service is running

[CWSpear]

I am pretty sure this is related to #622, and you were asking for a use-case for it, and I think I've got one.

The issue here is that new volumes aren't being properly attached to containers if the containers were already running.

Reduced test case

Gists don't allow subdirectories, but the --SLASH-- represents... well, a / (a subdirectory division).

https://gist.github.com/CWSpear/3bdacca128641164a836

Anyway, have the above files in this structure:

├── data
│   ├── Dockerfile
│   └── index.html
├── docker-compose.yml
└── server
    └── Dockerfile

Steps to reproduce

Step 1: Run:

docker-compose up -d

Step 2: Visit http://localhost:8080/ (don't forget to replace localhost with your Docker Machine's IP if you need to) and you should see Hello World!

Step 3: Change the contents of data/index.html to Hello Compose! then run:

Step 4: Rebuild data:

docker-compose build data

Step 5: Start up data?

docker-compose up data

Step 6: Refresh http://localhost:8080/.

Results

Expected: Hello Compose!
Actual: Hello World!

Addendum

You can replace Step 5 with any of these and you get the same result:

Step 5 alt (a)

docker-compose up -d

Step 5 alt (b)

docker-compose restart

Step 5 alt (c)

docker-compose rm -f data
docker-compose up data

Step 5 alt (d)

docker-compose stop
docker-compose start

Workaround

The following does work, but it will restart server, and the goal here would be to not have any down time:

Step 5 alt (e)

docker-compose rm -f data
docker-compose up -d

[dnephin]

This is by design: https://docs.docker.com/compose/#preserve-volume-data-when-containers-are-created

If we were to ignore the old volume and instead create a new one from the image then anyone storing database data in a volume would lose their data when they make any change to their database image.

The only way to get the new data is to rm -f data. The server container needs to be recreated because its dependency (data) was recreated. There's no way to change a volume on an existing container.

[CWSpear]

Ah... that's unfortunate, but I understand. My idea was I had an image build front-end dependencies and attach the volume of the compiled files to the other containers that needed them.

That way, I didn't need gulp build before docker-compose build (nor, alternatively, would I need install node on the other images just to build static files), and I could update static files linked to multiple containers without restarting the underlying services.

[dnephin]

You should able to do that using named volumes instead of volumes_from (in many cases named volumes should be replacing volumes_from).

You can have one container populate the named volume and the other services mount the named volume by name (name:/path/in/container). That way when you need to rebuild things, you can just run the "builder" container to re-build all the static assets, and other containers will be able to use them without restarting.

Compose doesn't really support the "builder" container pattern, but I've been prototyping a tool (https://github.com/dnephin/dobi) to handle some of these scenarios. Currently it only supports host volumes (which would work as well), but I should add support for named volumes as well.

[CWSpear]

I am seeing some changes in docker-compose:1.6, and I can see how I'd do this named volumes thing there, but in 1.5.2, are named volumes supported?

Related, named volumes made it into docker:1.9? Or not until the impending 1.10?

[dnephin]

Named volumes are in docker 1.9. They are also supported in compose 1.5.2.

The difference is that in 1.5.2 the name you use in volumes: is exactly that name. It's not namespaced using the project name.

In 1.6.2 it needs to be declared in a top level volumes: section, and the name is namespaced using the project name (so that you can multiple isolated copies using a different project name).

It should work in both versions, but will require a small change when you upgrade to 1.6 (to define either an external named volume, or changed it to use a namespaced one).