ibmcloud-image-builder

https://hub.docker.com/r/syibm/ibmcloud-image-builder

v0.1.0 Release Readiness: Planned to release on 07/06/2020

The first official release: v0.1.0

• templates for base and docker per each OS : Ubuntu 18.04, CentOS 7
• CI build
• github releases is ready
• github packages is ready
• DockerHub image repository: https://hub.docker.com/r/syibm/ibmcloud-image-builder

Motivation

Declarative Image Build

What required to build a cloud image, declaratively? The minimum requirements will be the name and version of the Operation System. That's the start, when someone wants to customize an image with the list of one's desirable packages in terms of cloud image, then the list of packages will be cloud-init user data. That should be it to build a cloud image, and an image builder should take care of the rest of chores. That's the motivation of this project: Building a cloud image declaratively.

Custom Encrypted Image

The same token applies to encrypt an image. Anyone should be able to build one by just providing an uncrypted image and encryption key.

Introduction

This repo is for the project that is going to build various virtual machine images in qcow2 format. The images can be imported into IBMCLOUD Cloud Object Storage (COS) and be served as custom images.

The required tools are from open source projects such as:

• QEMU
• Hashicorp Packer

The base images that will be built upon:

• Ubuntu
• CentOS
• TBD

This repo will also provide how to encrypt the images with Linux Unified Key Setup luks based encryption so that those encrypted images can be imported and used to spin up Virtual Server Instances (VSI) from IBM Virtual Private Cloud Generation 2.

The docker image that has all the required tools to build this VM images can be pulled from docker hub: https://hub.docker.com/r/syibm/ibmcloud-image-builder

The CI environment will be provided as a Dockerfile based on Alpine latest, and the CI environment will include:

• qemu
• packer
• cloud-utils
• ansible

The building time of this Docker image is < 1 min, I guess it can be used as an alternative while preparing docker pull from docker hub.

The development environment will be provided as a Dockerfile.ubuntu based on Ubuntu 20.04, and the development environment will include:

• qemu
• packer
• ibmcloud cli client & plugins
• terraform & terraform-provider-ibm
• cloud-utils
• go 1.13
• python3 3.6.9 (pyenv, pipenv)
• ansible 2.9.9

The building time of this Docker image is about 7 mins from my machines, it takes long. While preparing a repository in docker hub, maybe worthy to try once. It won't be changed very often any more.

Available images

So far we have Ubuntu 18.04 and CentOS 7 images as below:

• Ubuntu 18.04 Base
• Ubuntu 18.04 Base + Docker Installed
• CentOS 7 Base
• CentOS 7 Base + Docker Installed

$ tree -L 3
.
├── centos
│   └── 7
│       ├── base
│       └── docker
└── ubuntu
    ├── bionic
    │   ├── base
    │   └── docker
    ├── focal
    └── xenial

10 directories, 0 files

How To

Without building the dev docker, just pull the docker image:

$ git clone git@github.com:IBM-Cloud/ibmcloud-image-builder.git
$ cd ibmcloud-image-builder
$ docker pull syibm/ibmcloud-image-builder
$ docker tag  syibm/ibmcloud-image-builder ibmcloud-image-builder
$ make build-images

With building the dev docker:

$ git clone git@github.com:IBM-Cloud/ibmcloud-image-builder.git
$ cd ibmcloud-image-builder
$ make build
$ make build-images

Note: If a new packer template needs to be created, then please repeat yourself. The extra docker templates in addtion to base templates are for the information purpose on how to add new templates.

1. copy the existing folder and rename the directory
2. change either shell/user-data.sh or ansible/playbook.yml
3. change the image name in packer-builder.sh ... hmm, this needs to be refactored later.

How to build an encrypted image with you DEK (Data Encryption Key)

By default all the images will be encrypted with a given encryption key: JustMySimpleSecret. If the image required to be encrypted with a different encryption key, then attach to the build docker, and

cd "proper directory"
./packer-build.sh "Your DEK here"

$ tree -L 5
.
├── centos
│   └── 7
│       ├── base
│       │   ├── ansible
│       │   │   └── playbook.yml
│       │   ├── centos.json
│       │   ├── http
│       │   ├── packer-build.sh
│       │   ├── packer-delete.sh
│       │   └── shell
│       │       └── user-data.sh
│       └── docker
│           ├── ansible
│           │   └── playbook.yml
│           ├── centos.json
│           ├── http
│           ├── packer-build.sh
│           ├── packer-delete.sh
│           └── shell
│               └── user-data.sh
└── ubuntu
    ├── bionic
    │   ├── base
    │   │   ├── ansible
    │   │   │   └── playbook.yml
    │   │   ├── http
    │   │   ├── packer-build.sh
    │   │   ├── packer-delete.sh
    │   │   ├── shell
    │   │   │   └── user-data.sh
    │   │   └── ubuntu.json
    │   └── docker
    │       ├── ansible
    │       │   └── playbook.yml
    │       ├── http
    │       ├── packer-build.sh
    │       ├── packer-delete.sh
    │       ├── shell
    │       │   └── user-data.sh
    │       └── ubuntu.json
    ├── focal
    └── xenial

22 directories, 20 files

Example Virtual Server Instance created with the custom image generated with one of the templates

Acknowledgement

Thanks to the colleagues and IBM for sponsoring this project.

• Albert Camacho
• Chad Huesgen
• Dan Wiggins
• Zack Grossbart
• Irene Yip
• IBM CloudLab
• IBM