Improvements to platforms building and prebuilt images offering

[fsouza]

This is a proposal for improving the platform management and the structure of our repositories. Currently, we have one repository with all the platforms, and users need to always build the platform

We've discussed internally about supporting prebuilt images in platform-add, and it has been requested by the community as well (see tsuru/tsuru#1252). So here's a plan:

First, have three repositories for tsuru platforms:

1. github.com/tsuru/base-platform: would have a Dockerfile describing how to build the base image for all platforms.
2. github.com/tsuru/platforms: would have Dockerfiles for building all platforms.
3. github.com/tsuru/basebuilder: would exist just for compatibility. We'd replace the Dockerfile with the workaround described by @keymon in Add platform from Docker image (instead of Dockerfile) tsuru#1252 while tsuru doesn't support adding prebuilt images (i.e.: the python Dockerfile would contain just "FROM tsuru/python".

github.com/tsuru/base-platform

It would contain a Dockerfile that generates an image that runs /var/lib/tsuru/base/install. As well as the first four lines in the Dockerfile that are common to almost all platforms:

FROM   ubuntu-debootstrap:14.04
RUN    locale-gen en_US.UTF-8
ENV    LANG=en_US.UTF-8 LANGUAGE=en_US:en LC_ALL=en_US.UTF-8 DEBIAN_FRONTEND=noninteractive
RUN    apt-get update && apt-get install curl sudo -y --force-yes

This repository will be used to automatically build the image tsuru/base-platform in Docker Hub.

github.com/tsuru/platforms

It would contain the following structure:

.
├── cordova
│   ├── Dockerfile
├── elixir
│   ├── Dockerfile
├── go
│   ├── Dockerfile
├── java
│   ├── Dockerfile
├── java-7-openjdk
│   ├── Dockerfile
├── lisp
│   ├── Dockerfile
├── lua
│   ├── Dockerfile
├── nodejs
│   ├── Dockerfile
├── php
│   ├── Dockerfile
├── play
│   ├── Dockerfile
├── python
│   ├── Dockerfile
├── python3
│   ├── Dockerfile
├── ruby
│   ├── Dockerfile
├── static
│   ├── Dockerfile

For each platform, there will be a Dockerfile, that will probably start with "FROM tsuru/base-platform" and then describe the platform customization (this is not mandatory, users may build their Dockerfile from scratch). All platforms in this repository will be used to automatically build the matching image in Docker Hub (Docker Hub allows us to specify the Dockerfile path when configuring automatic builds, so it's fine to have).

The platform images would be linked to the base platform image, so new pushes to the base image would also rebuild all platform images (using the "Repository Links" feature in Docker Hub).

Having all platforms in a single repository make it easier for contributors to add new platforms (GitHub doesn't allow to send a whole repository pull request yet :P), but harder for users to fork a platform (not really, but it's a "uglier" fork). I'd rather ease contributing than forking :-)

github.com/tsuru/basebuilder

In the future, when tsuru supports adding or updating platforms from prebuilt image, we could do:

% tsuru-admin platform-update ruby -i tsuru/ruby

While we're not in the future, all Dockerfile's in this repository would be replaced with one-liners, containing just FROM <prebuilt-image>. For example, the Ruby Dockerfile looks like this (stripping comments):

FROM ubuntu-debootstrap:14.04
RUN  locale-gen en_US.UTF-8
ENV  LANG=en_US.UTF-8 LANGUAGE=en_US:en LC_ALL=en_US.UTF-8 DEBIAN_FRONTEND=noninteractive
RUN  apt-get update && apt-get install curl sudo -y --force-yes
RUN  curl -Lo basebuilder.tar.gz https://github.com/tsuru/basebuilder/tarball/master
RUN  mkdir /var/lib/tsuru
RUN  tar -xvf basebuilder.tar.gz -C /var/lib/tsuru --strip 1
RUN  rm basebuilder.tar.gz
RUN  cp /var/lib/tsuru/ruby/deploy /var/lib/tsuru
RUN  /var/lib/tsuru/ruby/install

It will then look like this:

FROM tsuru/ruby

This allows users to keep their current workflow and doesn't break anything in our current testing infrastructure.