Switch to boot2docker as recommended way to run Docker on OS X

[bfirsh]

Probably blocked by boot2docker/boot2docker#63, boot2docker/boot2docker#84 and boot2docker/boot2docker#85.

See julien-duponchelle/docker-osx#17 and julien-duponchelle/docker-osx#29 for more background.

[bfirsh]

https://github.com/fnichol/dvm could also be an alternative, but suffers from the same shortcomings as boot2docker.

[dduportal]

Here are my first tries bringin up fig into bootdocker :

• Standalone method (fetching the binary, chmoding and calling)

                        ##        .
                  ## ## ##       ==
               ## ## ## ##      ===
           /""""""""""""""""\___/ ===
      ~~~ {~~ ~~~~ ~~~ ~~~~ ~~ ~ /  ===- ~~~
           \______ o          __/
             \    \        __/
              \____\______/
 _                 _   ____     _            _
| |__   ___   ___ | |_|___ \ __| | ___   ___| | _____ _ __
| '_ \ / _ \ / _ \| __| __) / _` |/ _ \ / __| |/ / _ \ '__|
| |_) | (_) | (_) | |_ / __/ (_| | (_) | (__|   <  __/ |
|_.__/ \___/ \___/ \__|_____\__,_|\___/ \___|_|\_\___|_|
boot2docker: 0.6.0
docker@boot2docker:~$ curl -L https://github.com/orchardup/fig/releases/download/0.3.1/linux > fig
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   332  100   332    0     0    529      0 --:--:-- --:--:-- --:--:--   534
100 4979k  100 4979k    0     0  1068k      0  0:00:04  0:00:04 --:--:-- 1325k
docker@boot2docker:~$ ls -ltr
total 4980
-rw-r--r--    1 docker   staff      5099163 Mar  5 14:06 fig
docker@boot2docker:~$ chmod +x fig
docker@boot2docker:~$ echo $PATH
/home/docker/.local/bin:/usr/local/sbin:/usr/local/bin:/apps/bin:/usr/sbin:/usr/bin:/sbin:/bin
docker@boot2docker:~$ mv fig /home/docker/.local/bin/
docker@boot2docker:~$ which fig
/home/docker/.local/bin/fig
docker@boot2docker:~$ /home/docker/.local/bin/fig --version
-sh: /home/docker/.local/bin/fig: not found
docker@boot2docker:~$ fig --version
docker@boot2docker:~$

• Second method with pip package :

docker@boot2docker:~$ rm -rf /home/docker/.local/bin/fig
# Installing python into boot2docker with TCZ package
docker@boot2docker:~$ wget http://www.tinycorelinux.net/5.x/x86/tcz/python.tcz && tce-load -i python.tcz && rm -f python.tcz
python.tcz           100% |****************************************************************************|  8892k  0:00:00 ETA
python.tcz: OK
docker@boot2docker:~$ python --version
Python 2.7.6
# Installing setup tools
docker@boot2docker:~$ curl -LO https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py && sudo /usr/local/bin/python2.7 ez_setup.py && rm ez_setup.py
<bunch of lines>
# Installing pip
docker@boot2docker:~$ sudo /usr/local/bin/easy_install-2.7 pip
# Installing fig
docker@boot2docker:~$ sudo /usr/local/bin/pip2.7 install -U fig
<bunc of lines>
# Go Go !
docker@boot2docker:~$ fig --version
docker@boot2docker:~$

In both cases, no output are given by the new fig :-(
I successfully tried the 2nd method with a < 3.x version of fig, it was working very well.

[bfirsh]

It's probably not worth trying to get Fig running inside the boot2docker VM itself – you can run it outside on OS X and point it at the remote API.

[dduportal]

Yeah, the point is that i'm not on OS X :-)
My goal is to bring docker dev env into Win 7/8 x64, with Vagrant + VirtualBox + boot2docker + fig, using sharing folders + git/hg + NAT resolving.

Maybe Docker guys will push a Windows CLI someday :-)

[bfirsh]

Ahah – got you, sorry. VirtualBox + Ubuntu might work better. boot2docker is meant to be nothing more than a hypervisor.

[dduportal]

Yeah, i was expecting a very very light docker dev VM : i'm going to only run fig + docker into this Linux box, and a <30 Mb box was an awesome path for me (instead of a big ubuntu or centos box :-) ).

Any plan to support fig on Windows ?

[bfirsh]

No plans currently! Open to pull requests, and it might be something we look into ourselves if there's enough demand.

[bfirsh]

More details on current state of this:

boot2docker/boot2docker#284
https://github.com/boot2docker/boot2docker/blob/master/doc/WORKAROUNDS.md

[SvenDowideit]

If anyone's looking into boot2docker and fig, The idea of boot2docker is that the host's filesystem is essentially read only. Everything should be in a container - so rather than installing fig into the boot2docker host, create a fig container which you run with a bindmounted docker exe, and access to a docker socket.

[bfirsh]

This works great with Fig: https://medium.com/boot2docker-lightweight-linux-for-docker/boot2docker-together-with-virtualbox-guest-additions-da1e3ab2465c

[bfirsh]

In summary, install the boot2docker installer, then run:

$ mkdir ~/.boot2docker
$ curl http://static.dockerfiles.io/boot2docker-v1.0.0-virtualbox-guest-additions-v4.3.12.iso > ~/.boot2docker/boot2docker.iso
$ boot2docker init
$ VBoxManage sharedfolder add boot2docker-vm -name home -hostpath /Users
$ boot2docker up

You can add a boot2docker host to /etc/hosts:

$ sudo sh -c "echo `boot2docker -m 123 ssh ip addr show eth1 |sed -ne 's/^[ \t]*inet[ \t]*\([0-9.]*\)\/.*$/\1/p'` boot2docker >> /etc/hosts"

To set up your shell to automatically connect Fig and Docker to boot2docker, set DOCKER_HOST in .bashrc:

$ echo export DOCKER_HOST=tcp://`boot2docker -m 123 ssh ip addr show eth1 |sed -ne 's/^[ \t]*inet[ \t]*\([0-9.]*\)\/.*$/\1/p'`:2375 >> .bashrc

[b00gizm]

@bfirsh

Doesn't work for me. Volumes still won't show up.

/edit: And besides that: Pulling images with boot2docker is really slow compared to docker-osx. Any idea what's wrong?

[kirel]

@bfirsh You probably mean

/Users/`whoami`

[bfirsh]

@kirel Nope. It mounts the entire /Users directory at /Users inside the VM.

[bfirsh]

Oops. There's also a boot2docker ip command, so those last two examples can be:

$ sudo sh -c "echo `boot2docker ip` boot2docker >> /etc/hosts"
$ echo export DOCKER_HOST=tcp://`boot2docker ip`:2375 >> .bashrc

[aanand]

Works great.

I did this in my profile to suppress the extra output from boot2docker ip ("The VM's Host only interface IP address is"):

export DOCKER_HOST=tcp://$(boot2docker ip 2>/dev/null):2375

[djmaze]

We experienced severe delays on OSX (> 30 secs) when using volumes through the VBox shared folder approach. This is a general problem as described at moby/moby#4023 (comment). Makes it unusable for developing web apps.

Waiting for the docker team to implement a clean solution (using a FUSE-based filesystem, it seems).

[freewil]

@djmaze see julien-duponchelle/docker-osx#61

[saulshanabrook]

What needs to happen for boot2docker to be able to mount volumes through fig as easily as docker-osx can? Is that on the roadmap somewhere?

[bfirsh]

It's being worked on. If you want to use boot2docker, this is the best solution currently: #26 (comment)

[saulshanabrook]

Yeah, I saw that, but it would be ideal to be able to mount specific host directories to specific container destinations, as you can now with docker-osx. Is it being worked on in boot2docker? Is there an issue for it?

[larrycai]

@SvenDowideit in this case, it seems fig docker container doesn't work, it looks quite tricky for volume setup. see Dockerfile using your solution. https://github.com/larrycai/docker-images/blob/master/fig/Dockerfile

inside I put the docker and fig binary files and pass the docker.sock

docker run -t -p 5000 -v /var/run/docker.sock:/docker.sock -v /home/docker/figapp:/app larrycai/fig

The volume in fig.yml

volumes:
  - .:/figapp

or (since docker is running host, so I specified the host absolute directory instead)

volumes:
  - /home/docker/figapp:/figapp

I use the fig.yml sample code from dockerbook.com

It doesn't work, really hope it can work in boot2docker (windows), it will ease the work

[SvenDowideit]

yup. we are working towards resolving this - but its not been easy

[larrycai]

I am fans of boot2docker and hope soon it can be solved, thank for your effort.

[larrycai]

@aer0s though vbox share solution is not recommended from b2d point of view officially ;-), for experiment, your solution works perfect for me. (windows 7) .

Well done.

[chrisconley]

As another option, here's how I'm running Fig with boot2docker and rsync (instead of guest additions)

boot2docker init
boot2docker up
export DOCKER_HOST=tcp://$(boot2docker ip 2>/dev/null):2375

# Setup working directory on b2d host and install rsync
boot2docker ssh "sudo mkdir -p `pwd` && sudo chown docker `pwd`"
boot2docker ssh "tce-load -wi rsync"

# rsync the working directory to b2d host
ssh-add ~/.ssh/id_boot2docker
rsync -av ./ docker@$(boot2docker ip 2>/dev/null):$(pwd)

fig up

[SvenDowideit]

nice!

[patrickheeney]

@chrisconley How easy would it be to make it a two way sync back to the host?

[AkeemMcLennon]

I was able to get @chrisconley 's method to work in Windows with a few extra steps (through Cygwin). This might be useful for @dduportal as well.

# Needed for ssh-add to work
eval `ssh-agent -s`
# Fix permissions issue in Windows for private key
chgrp Users ~/.ssh/id_boot2docker
chmod 600 ~/.ssh/id_boot2docker

From there, you'll need to be aware of the differences in file permissions and line ending on Windows. If you have any shell scripts, you might want to run dos2unix on the first. You may also want to change the rsync command to

rsync -av --chmod=ugo=rwX ./ docker@$(boot2docker ip 2>/dev/null):$(pwd)

The --chmod=ugo=rwX uses a folder's default unix file permissions instead of the potentially problematic guess that may come from WIndows.

[SvenDowideit]

@patrickheeney bi-directional is considered something you shouldn't use rsync for: http://unix.stackexchange.com/questions/12197/syncing-directories-in-both-directions-with-rsync

I've just added your suggestion as boot2docker/boot2docker-cli#247

this is not likely to see the light of day in this form, but its interesting none-the-less. (yes, I've not tested it on Windows yet)

[SvenDowideit]

boot2docker/boot2docker-cli#247 now has an sshfs based version of boot2docker share - the b2d vm mounts the desktop's current dir into its FS. Its hardcoded in places, and lots of the setup would move into the boot2docker iso....

[amaltson]

test-kitchen does something similar by scping the data over, and this works well with kitchen-docker and boot2docker out of the box. There's a project that makes the copying faster, kitchen-sync by using rsync (as suggested above). It'd be nice if it'd be solved in boot2docker (go @SvenDowideit!), but this could theoretically be solved in fig as well?

[bfirsh]

It's getting close. boot2docker/boot2docker#534

[bfirsh]

Also boot2docker/boot2docker-cli#258

Just needs docs now.

[larrycai]

@SvenDowideit @dduportal , just let you know my solution for run fig in docker is working in b2d 1.3 release, my Dockerfile https://github.com/larrycai/docker-images/blob/master/fig/Dockerfile

One confused for beginner will be the run fig in docker, but the volume shall be host directory still.

volumes:

• .:/figapp

since docker is running host, so I specified the host absolute directory instead)

volumes:

• /c/Users/larry/figapp:/figapp

[SvenDowideit]

@bfirsh @aanand ^^^^^

[ybrs]

i can't mount local folders on macosx via fig using boot2docker, though docker works fine. its a brand new installation (though i tried, pulling latest of Fig from master too)

(env)$ fig --version
fig 1.0.1

(env)$ boot2docker version
Boot2Docker-cli version: v1.3.2
Git commit: e41a9ae

installed VMBox extensions and mounted /Users directory in boot2docker

(env)$ boot2docker ssh ls /Users
Guest
Shared
aybarsbadur

my fig.yml is simple as this

web:
  build: .
  command: ls -lha /foo/
  volumes:
    - /Users:/foo
  ports:
    - "8000:8000"

when i run it - verbosely with logging.debug enabled,

(env)$ fig up
Starting new HTTPS connection (1): 192.168.59.103
"GET /v1.14/containers/json?all=1&limit=-1&trunc_cmd=1&size=0 HTTP/1.1" 200 None
Recreating rallyweb_web_1...
"POST /v1.14/containers/a6a6be9de9a961f96650b3e29512f123deb57dc5c3eadf55fcd2d9f142e8583c/stop?t=10 HTTP/1.1" 304 0
======== post =======
{"Tty": false, "NetworkDisabled": false, "Image": "rallyweb_web:latest", "Cmd": [], "StdinOnce": false, "Entrypoint": ["/bin/echo"], "MemorySwap": 0, "Memory": 0, "AttachStdin": false, "AttachStderr": true, "AttachStdout": true, "OpenStdin": false}
// ====== post //////
"POST /v1.14/containers/create HTTP/1.1" 201 90
"GET /v1.14/containers/9e0e13cb3ebdb5ed25c3cc13c520c0aa0909be5e018e0e80d82fa0a1c636d3ce/json HTTP/1.1" 200 1628
======== post =======
{"PublishAllPorts": false, "VolumesFrom": ["a6a6be9de9a961f96650b3e29512f123deb57dc5c3eadf55fcd2d9f142e8583c"], "Privileged": false}
// ====== post //////
"POST /v1.14/containers/9e0e13cb3ebdb5ed25c3cc13c520c0aa0909be5e018e0e80d82fa0a1c636d3ce/start HTTP/1.1" 204 0
"POST /v1.14/containers/9e0e13cb3ebdb5ed25c3cc13c520c0aa0909be5e018e0e80d82fa0a1c636d3ce/wait HTTP/1.1" 200 17
"DELETE /v1.14/containers/a6a6be9de9a961f96650b3e29512f123deb57dc5c3eadf55fcd2d9f142e8583c?link=False&force=False&v=False HTTP/1.1" 204 0
"GET /v1.14/containers/json?all=1&limit=-1&trunc_cmd=1&size=0 HTTP/1.1" 200 None
volumes >>> {u'/foo': {}}
"GET /v1.14/images/json?filter=rallyweb_web&only_ids=0&all=0 HTTP/1.1" 200 250
======== post =======
{"Volumes": {"/foo": {}}, "Tty": false, "NetworkDisabled": false, "Image": "rallyweb_web", "Cmd": ["ls", "-lha", "/foo/"], "StdinOnce": false, "AttachStdin": false, "Env": [], "Memory": 0, "MemorySwap": 0, "ExposedPorts": {"8000/tcp": {}}, "AttachStderr": true, "AttachStdout": true, "OpenStdin": false}
// ====== post //////
"POST /v1.14/containers/create?name=rallyweb_web_1 HTTP/1.1" 201 90
"GET /v1.14/containers/5fa7f02192295525cf81b70937a8233e85609ca82e47a29849cdd9a8af99e4b7/json HTTP/1.1" 200 1772
========volumes =========
{u'/Users': {u'bind': u'/foo', u'ro': False}}
=========volumes ========
======== post =======
{"NetworkMode": "bridge", "PortBindings": {"8000/tcp": [{"HostPort": "8000", "HostIp": ""}]}, "Binds": ["/Users:/foo:rw"], "PublishAllPorts": false, "Privileged": false, "VolumesFrom": ["9e0e13cb3ebdb5ed25c3cc13c520c0aa0909be5e018e0e80d82fa0a1c636d3ce"]}
// ====== post //////
"POST /v1.14/containers/5fa7f02192295525cf81b70937a8233e85609ca82e47a29849cdd9a8af99e4b7/start HTTP/1.1" 204 0
"DELETE /v1.14/containers/9e0e13cb3ebdb5ed25c3cc13c520c0aa0909be5e018e0e80d82fa0a1c636d3ce?link=False&force=False&v=False HTTP/1.1" 204 0
"GET /v1.14/containers/json?all=0&limit=-1&trunc_cmd=1&size=0 HTTP/1.1" 200 291
Attaching to rallyweb_web_1
"POST /v1.14/containers/5fa7f02192295525cf81b70937a8233e85609ca82e47a29849cdd9a8af99e4b7/attach?stderr=1&logs=1&stream=1&stdout=1 HTTP/1.1" 200 None
web_1 | total 8.0K
web_1 | drwxr-xr-x  2 root root 4.0K Dec 16 13:35 .
web_1 | drwxr-xr-x 86 root root 4.0K Dec 16 13:35 ..
Starting new HTTPS connection (2): 192.168.59.103
"POST /v1.14/containers/5fa7f02192295525cf81b70937a8233e85609ca82e47a29849cdd9a8af99e4b7/wait HTTP/1.1" 200 17
rallyweb_web_1 exited with code 0
Gracefully stopping... (press Ctrl+C again to force)
"GET /v1.14/containers/json?all=0&limit=-1&trunc_cmd=1&size=0 HTTP/1.1" 200 2
(env)$

though if i run the same with docker

(env)$ docker run -v /Users:/foo rallyweb_web:latest ls /foo
Guest
Shared
aybarsbadur

rebuilding fig with ADD in Dockerfile works fine, but its a pain to rebuild everytime when your code changes, so its a show stopper to use fig in development environment.

I am not sure what else to do - or is it just me -, it will be great if you can advice on what to do.

Thank you,

[ybrs]

for anyone landing here by googling and not #723 , docker 1.4.0 has a bug moby/moby#9628 which is fixed.

simply downloading latest docker build into boot2docker fixes the problem.

root@boot2docker:/home/docker# sudo /etc/init.d/docker stop
root@boot2docker:/home/docker# curl 'https://master.dockerproject.com/linux/amd64/docker' > /usr/local/bin/docker
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 13.9M  100 13.9M    0     0   726k      0  0:00:19  0:00:19 --:--:--  771k
root@boot2docker:/home/docker# sudo /etc/init.d/docker start
Need TLS certs for boot2docker,127.0.0.1,10.0.2.15,192.168.59.103

[Gasol]

@ybrs Thank you, You have saved my life.

[NathanHazout]

I know this is a very old issue.
Are there any plans for Docker to work "out-of-the-box" on OSX?
Without boot2docker or similar workaround?

[thaJeztah]

@nasht00 unfortunately at this point, I don't see that happening. OS X is not a Linux based distribution and simply doesn't have the kernel features that Docker relies on.

I know some maintainers looked into using the "Sandbox API" in OS X, but that didn't look sufficient

So, unless we see some major effort from Apple's side (like Microsoft is currently doing for porting to Windows), don't hold your breath.