Add enhancements for working with files not added as projects

[wernight]

Having a, Docker by default, sharable runtime is a cool concept. However could you explain a bit how practically it's expected to work. I've many projects using Python/PHP/... and none of them would run out of the box on one of the preset runtime.

However most websites (I'm focusing here mostly on websites here) builds generate a few Docker images. Those images can get deployed or run locally with local folder mounted to develop. So how is it supposed to work?

1. Using existing preset seems not to include what I need (that's fine IMO).
2. Sure I could add all packages required by the various images into a single Dockerfile that would allow to run all components. This would require keeping them in sync and increases the risk also of having something behave differently in production.
3. Use a very basic Dockerfile for Che that can do Docker-in-Docker and basically continue like I'm coding. This isn't a bad solution.

Both are fine, I was mostly wondering if you'd mind explaining a bit some practical usage examples and especially detail how to build a Dockerfile for Che. When I tried I had an error but no details (using Codenvy), so I can only guess I'm missing some app that is expected on it.

[TylerJewell]

We have an improving set of documentation on how to build a Dockerfile that can be used as a recipe when creating a new workspace. We document that in the Recipes page. We discuss the various trade offs of extending one of Che's base images or starting with your own.

The basic model is that the projects (source folders) are separated from the runtimes, and then married together. The sequence is that Che creates a workspace, and then launches the workspace runtimes. While the runtimes can have files pre-configured inside of them, the files from a runtime are usually the files needed to support the stack. So you might have npm libraries pre-loaded into the workspace runtime, but not necessarily your project file stack.

The project files reside outside the workspace and then mounted into the worksapce after its runtimes are started. This can be from a subversion clone, git clone, or a zip mount. These projects are then located in /projects, which makes them available when you directly access the workspace runtime or from the IDE.

Now, you can have packages added for your workspace either from within the image that powers the runtime, or after the runtime is started. The basic difference is that if there are many shared packages, it's silly to have each user download those packages repeatedly, so include them in the runtime image. If there are packages that are user-specific or frequently changing, then maybe better to have the user download and install them when they get into the workspace.

Essentially, Che is setting up the scenario you describe - where you have a set of images and then the folders are mounted in to work. We just eliminate the need for the user to figure out how to do the mount and setup of files.

On Monday we will be at Red Hat's conference previewing multi-container support in a workspace using Docker Compose syntax. This will help with modularity.

And also, as the year goes on, we will be working to remove Che's requirements from Docker images, so that any Docker image can be pulled and then we inject Che's requirements into the Docker image. This way you will not have to maintain custom Dockerfiles.

[ghost]

@wernight can you perhaps elaborate on a particular project that won't run?

Also, I wonder what is building a Dockerfile for Che means in your case. Eager to get additional info :)

[wernight]

Commenting on what you suggest:

• Multi-container using docker-compose
• Docker images not requiring Che-specific stuff

Can you detail how that might work?

I want to explore more about Docker-in-Docker / Docker-socket I think should be supported and would support the points above without much trouble. It'd be a great base image. At the moment I'm more stuck on #1560 .

@elvantsov For example I may need ElasticSearch, or MySQL or whatever weird thing. There are many projects. Even if they worked out of the box, it'd be in a very different environment from the deployed one, so I'd really rather have Docker-in-Docker / Docker-socket access than environments that work out of the box. So I'd prefer to provide the Dockerfiles and be able to run them in a private cloud for development.

Having simple way to start them for me and allow me to invite others to either directly use it or do the same on their "Eclipse Che" with minimal pain sounds really cool! Mostly for open source, but not only.

[james10174]

@wernight Docker-in-docker is possible with privileged outer docker container and volume mounting /var/run/docker.sock. The volume mounting is the tricky part right now as che conf file only allows for one volume to be mounted. If all you need is /var/run/docker.sock in your workspace/container you are set. When you create another container inside your workspace you can choose nor to include the docker.sock but maybe do the following docker run -v /projects:/projects <my_image> to include you compile target folders. You could compile the program in workspace then run docker run -v /projects:/projects <my_image> java -jar \project\<projectname>\target\<filename>.jar. I would as a starting point get the following to run in workspace before trying anything else docker run ubuntu /bin/echo 'Hello world'. Just remember when you modify a container to include programs like mysql etc you can upload the image to dockerhub. Then use that image name in place of <my_image> given above.

The following is typical command to do docker in docker
docker run --privileged -v /var/run/docker.socker:/var/run/docker.sock <my_image>

You can get docker in docker to work without privileged too but you need two mounts with something like the following. http://stackoverflow.com/questions/29612463/can-i-run-docker-in-docker-without-using-the-privileged-flag
docker run -it -v /var/run/docker.sock:/var/run/docker.sock -v $(which docker):/bin/docker alpine docker ps -a

Look at issue #1338 for additional information on mounting volumes additional volumes.
Also I have a helpful sshfs mount script you can use. Take a look at #1533 for more info on it.
git clone git@bitbucket.org:james10174/usefulscripts.git

[wernight]

Thanks. I know how to do docker-in-docker, but how do you or affect the docker flags the Che uses when it creates a Che workspace/runtime?

[james10174]

@wernight che.properties file is the only way to do this currently that I know of. You do not have access to all flags through che.properties. Are you requesting/asking for access to create custom docker flags?

[wernight]

Yes it'd be extremely useful. As I said, if from the Che runtime I could build and run Docker images, it'd allow real development.

A "safer" way may be by providing a docker-compose.yml to Che with mount points relative to the project directory, and Che would start multiple containers and mount the files, but this has issues. A simpler and more powerful approach, but requiring #1560 to be fixed, is having a runtime which gives access to docker, and then it's basically running in a Che workspace terminal the commands:

$ docker-compose up -d

The advantage of have access to docker from within the runtime is that some projects have a build phase which is more than compiling Dockerfiles. It may compile some, run them to generate some files and include those to build another Docker image. This would not be possible without access to docker from within Che terminal.

The risk is that the Che user must be trusted with root access on the Che server (this isn't too much an issue and Docker-in-docker should actually not require that level of confidence). The second is the user must trust the code their's running on their Che as it has root access. It's similar to currently running Docker but it's more risky because a browser click may at the moment trigger running a lot of commands which is a lot easier to exploit and equivalent in power to asking the user to run an unknown shell script on their machine. That means that a warning should be shown with an easy way to see the code that will be executed.

[james10174]

@wernight The only thing I have had trouble with not being able to use without custom docker flags is volumes cannot be used in dockerfiles. One work around right now for che single volume access is to mount /var/run/docker.sock into another directory such as ~/shareDocker/. Then put in che.properties machine.server.extra.volume=~/shareDocker/:/shareDocker/. You can put other things you want access to in ~/shareDocker/ too.

Example below can be modified to work with che but shows the concept.

mkdir ~/shareDocker/
touch ~/shareDocker/docker.sock
homeDir=~
sudo mount --bind /var/run/docker.sock  $homeDir/shareDocker/docker.sock
docker run -v ~/shareDocker/:/shareDocker/ -ti docker
docker -H unix:///shareDocker/docker.sock run -v /<directory>/:/<directory>/ docker /bin/echo 'Hello world'

NTS:

/ # docker -H unix:///shareDocker/docker.sock run -v /shareDocker/:/shareDocker -ti docker
/ # ls -l /shareDocker/
total 4
drwxr-xr-x    2 root     root          4096 Jun 27 22:42 docker.sock

docker in docker in docker volume mount results in directory of docker.sock. So docker-in-docker-in-docker does not work with docker.sock...

[james10174]

@wernight Sorry had to make a couple of edits to above. A really easy way to do docker in docker with che is just machine.server.extra.volume=/var/run/docker.sock:/var/run/docker.sock then sudo apt-get install docker.io in workspace. Additional volumes is currently labelled as enhancement in issue #1338. Security is lacking sharing docker.sock though. I am researching security for docker myself right now so forgive me for some of my ignorance. I am reading through some security features. I will let you know if I can come up with a secure way to use docker without access to docker flags.

[ghost]

@wernight @james10174 another way to use Docker from within a container is to run host Docker in tcp mode and export DOCKER_HOST in the container so that this variable has the following value tcp://$hostIP:2375. This way when Docker is invoked in a container a host Docker server is triggered.

If you run Che in a cloud, you may use an internal IP rather than a public one.

[wernight]

@elvantsov Exposing the TCP socket should be the same (but often more complex) as exposing Linux socket /var/run/docker.sock. May be simpler to use right now with Che though.

Yes machine.server.extra.volume should be perfect. If I had to rate the currently blocking issues it'd be:

1. Exposing less ports #1560 All HTTP through port 80 and secure SSH port
2. Docker Volumes. #1338 machine.server.extra.volume
3. Search and replace in multiple files #1564 Search & replace in multiple files
4. Add enhancements for working with files not added as projects #1580 Show a warning and allow users to quickly see potentially risky code (mounted volumes. Dockerfile, commands...)
5. Add enhancements for working with files not added as projects #1580 mutl-container support using docker-compose and Docker images not requiring Che-specific stuff

[wernight]

Another useful thing that can be done more easily by supporting having Docker from Che: One can setup a Che runtime for developing which includes for example your own .profile with common aliases you're used to run, and that would not impact how the code is compiled or run.

[TylerJewell]

@wernight - FYI, we have a couple things that are getting worked on this quarter that adds to some of the items that you are thinking about. First, basic docker compose support will ship soon - it has just a couple sprints left of development. Second, we also now have a design and vision on how to support docker images that are not based upon Codenvy templates. We will break out our various required capabilities into "agents" that users can add / remove from the dashboard. We will then install, register and start any necessary agents when the workspace is started. The available agents will be "dev", "ssh", and "terminal".

[wernight]

@TylerJewell Having a runtime to build and run stuff is good and it's the most important. I was wondering also if you could think of ways to customize the Terminal. Like there are settings for Che/Codenvy there one could have a custom home directory that would persist so as the customize the aliases, shell... I do see issues here, which is why I found the most flexible to be allowing a custom runtime per user, with all runtimes allowing to build and run Dockerfiles. I agree it's a hack and it's not necessarily the most important feature right now.

I'll be glad to try it out once you're ready.

[TylerJewell]

I think that right now the best we can offer around terminal customizations is that you can create different base images for users into different stacks, and then with each stack you can define terminal properly within the .bashrc file of the image.

[ghost]

@wernight what should I do with this issue? The conversation ended up in a number of suggestions on how to improve Che - all of them captured as GitHub issues.

[wernight]

I feel it's blocked at the moment. I cannot do development using Che right now. May be the docker-compose would work, I'm not sure. Once the #1560 and #1338 issues are fixed, I'll try also machine.server.extra.volume as that would really work and I'd suggest documenting it more formally.

[ghost]

@wernight I am not sure what I need to do with this particular issue - it started as Document the practical usage for real existing (non-Java) web projects. How id docker and docker compose stuff related?

[wernight]

At the moment I cannot use Che for my projects (except may be one or two, the rest can only use Che as a text editor), it's blocked on the issues I cited. Once that is fixed, the documentation could be updated to explain how to use Che for many more projects with practical examples.

For context, I've almost no real maintained Java project, so I'd rather skip that. Point is, it's not really usable for me. I'd have to create a new fat Docker image just for Che, for each project, and that image may well not reflect what's happening in production, and would require more maintenance time. Ideally I'd run locally Kubernetes via Minicube or such and mount volumes to edit files in "real time". Practically for now, during development, I run the same container images locally via docker-compose with the same network as on Kubernetes. It's almost always like a LAMP: Nginx + Python/PHP/... + Redis/MySQL/...

[TylerJewell]

@wernight - I am going to close this issue. There are tracking issues for the other items, and the ones that are being taken into the roadmap have been linked to the roadmap for tracking.

[wernight]

Ok, I'll probably write an article about Che which would kind of solve this issue.

[TylerJewell]

That would be appreciated. Our roadmap is robust, so we are working through the backlog as best we can.

[ghost]

I am going to close this issue. Feel free to reopen if you have particular problems/suggestions for Che.