The cloud engineer's toolbox.
CloudControl is a Docker based configuration environment containing all the tools required and configured to manage modern cloud infrastructures.
The toolbox comes in different "flavours" depending on what cloud you are working in. Currently supported cloud flavours are:
AWS (based on amazon/aws-cli)
Azure (based on mcr.microsoft.com/azure-cli)
Simple (based on alpine)
Tanzu (based on alpine)
Following features and tools are supported:
- 🐟 Fish Shell
- 📷 AzCopy
- ⚙️ Direnv
- ⛵️ Helm
- 🛠 JQ
- ⌨️ kc Quick Kubernetes Context switch
- 🐚 Kubectlnodeshell
- 🐳 Kubernetes
- 📦 Packages
- 📦 Packer
- 👟 Run
- 📜 Stern
- 🌏 Terraform
- 🐗 Terragrunt
- 🕰 Timezone configuration
- 🌊 Velero
- 𝑉 Vim
- 🛠 YQ
CloudControl can be used best with docker-compose. Check out the sample directory in a flavour for a sample
compose file and to convenience scripts. It includes a small web server written in Go and Vuejs-client dubbed
"CloudControlCenter", which is used as a status screen. It listens to port 8080 inside the container.
Copy the compose file and configure it to your needs. Check below for configuration options per flavour and feature.
Run init.sh. This script basically just runs docker-compose up -d and tells you the URL for CloudControlCenter.
Open it and wait for CloudControl to finish initializing.
The initialization process will download and configure the additional tools and completes with a message when its done. It will run each time when the stack is recreated.
After the initialization process you can simply run docker-compose exec cli /usr/local/bin/cloudcontrol run to jump
into the running container and work with the installed features.
If you need to change any of the configuration environment variables, rerun the init script afterwards to apply the changes. Remember, that CloudControl needs to reininitialize for this.
There are two ways to configure a feature and the version it should use. The first way is to use the given
USE_<feature name>=yes environment variable and specifying the version with <FEATURE NAME>_VERSION=<version>.
If there are multiple features configured, this can get a bit messy. Another approach is to use the FEATURES
environment variable and list the features and optionally the version like this:
FEATURES=kubernetes helm:3.5.1 terraform:1.1.9
This would install the version 3.5.1 of Helm and version 1.1.9 of terraform. (Kubernetes uses the flavour's provided version of kubectl, e.g. using az aks install-cli)
Note: Please see the feature documentation below if a feature supports specifying a version string.
CloudControl is targeted to run on a local machine. It requires the following features to work:
- host path volumes
- host based networking
Some Kubernetes distributions such as Rancher desktop support this and can be used to run CloudControl.
The sample directories of each flavour provide an example Kubernetes configuration based on a deployment and a
service. They were preliminary tested on Rancher desktop.
Modify them to your local requirements and then run
kubectl apply -f k8s.yaml
to apply them.
This will create a new namespace for your project and a deployment and a service in that. Check
kubectl get -n <project> pod to watch the progress until a cli pod has been created.
Use kubectl get -n <project> svc cli to see the bound ports for the cli service and use your browser to connect
to the CloudControlCenter instance.
After the initialization is done, use kubectl -n <project> exec -it deployment/cli -- /usr/local/bin/cloudcontrol run
to enter CloudControl.
Warning: This implementation is currently a preview feature and hasn't been tested thoroughly. It highly depends on the proper support for host based volumes and networking of the Kubernetes distribution. Please refer to the documentation and support of your Kubernetes distribution if something isn't working.
If you want to display a custom login message when users enter the container, set environment variable MOTD
to that message. If you want to display the default login message as well, also
set the environment variable MOTD_DISPLAY_DEFAULT to yes.
If you'd like to forward traffic into a cluster using kubectl port-forward you can do the following:
-
Add a ports key to the cli-service in your docker-compose file to forward a free port on your host to a defined port in your container. The docker-compose-files in the sample directories already use port 8081 for this.
-
Inside CloudControl, check the IP of the container:
bash-5.0$ ifconfig eth0
eth0 Link encap:Ethernet HWaddr 02:42:AC:15:00:02
inet addr:172.21.0.2 Bcast:172.21.255.255 Mask:255.255.0.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:53813 errors:0 dropped:0 overruns:0 frame:0
TX packets:20900 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:75260363 (71.7 MiB) TX bytes:2691219 (2.5 MiB)
- Use the IP address used by the container as the bind address for port-forward to forward traffic to the previously defined container port to a service on its port (e.g. port 8081 to the service my-service listening on port 8080):
kubectl port-forward --address 172.21.0.2 svc/my-service 8081:8080
- Check out, which host port docker bound to the private port you set up (e.g. 8081)
docker-compose port cli 8081
- Connect to localhost: on your host
If you'd like to set up aliases to save some typing, you can use the run feature. Run your container with these environment variables:
USE_run=yes: Set up the run featureRUN_COMMANDS=alias firstalias=command;alias secondalias=command: Set up some aliases
First, mount your .ssh directory into the container at /home/cloudcontrol/.ssh.
Also, to not enter your passphrase every time you use the key, you should mount the ssh agent socket into the container and set the environment variable SSH_AUTH_SOCK to that path. CloudControl will automatically fix the permissions of that file so the CloudControl user can use it.
Here are snippets for your docker-compose file for convenience:
(...)
volumes:
- "<Path to .ssh directory>:/home/cloudcontrol/.ssh"
# for Linux:
- "${SSH_AUTH_SOCK}:/ssh-agent"
# for macOS:
- "/run/host-services/ssh-auth.sock:/ssh-agent"
environment:
- "SSH_AUTH_SOCK=/ssh-agent"
Because of how CloudControl is designed it uses a defined user named "cloudcontrol", so Terraform state lock messages look like this:
Error: Error locking state: Error acquiring the state lock: storage: service returned error: StatusCode=409, ErrorCode=LeaseAlreadyPresent, ErrorMessage=There is already a lease present. RequestId:56c21b95-501e-0096-7082-41fa0d000000 Time:2021-05-05T07:41:25.9164547Z, RequestInitiated=Wed, 05 May 2021 07:41:25 GMT, RequestId=56c21b95-501e-0096-7082-41fa0d000000, API Version=2018-03-28, QueryParameterName=, QueryParameterValue= Lock Info: ID: a1cef2cc-fec4-1765-4da8-d068a729ba7e Path: path/terraform.tfstate Operation: OperationTypeApply Who: cloudcontrol@5c47a37f920b Version: 0.12.17 Created: 2021-05-05 07:38:01.188897776 +0000 UTC Info:
It's hard to identify from that who the other CloudControl user is, that may have opened a lock. The system user can't be changed, but it's possible to set a better hostname than the one Docker autogenerated.
See this docker-compose snippet on how to set a better hostname:
version: "3"
services:
cli:
image: "dodevops/cloudcontrol-azure:latest"
hostname: "<TODO yourname>"
volumes:
(...)
If you set hostname in that snippet to "alice", the state lock will look like this now:
Error: Error locking state: Error acquiring the state lock: storage: service returned error: StatusCode=409, ErrorCode=LeaseAlreadyPresent, ErrorMessage=There is already a lease present. RequestId:56c21b95-501e-0096-7082-41fa0d000000 Time:2021-05-05T07:41:25.9164547Z, RequestInitiated=Wed, 05 May 2021 07:41:25 GMT, RequestId=56c21b95-501e-0096-7082-41fa0d000000, API Version=2018-03-28, QueryParameterName=, QueryParameterValue= Lock Info: ID: a1cef2cc-fec4-1765-4da8-d068a729ba7e Path: path/terraform.tfstate Operation: OperationTypeApply Who: cloudcontrol@alice Version: 0.12.17 Created: 2021-05-05 07:38:01.188897776 +0000 UTC Info:
I get an error "repomd.xml signature could not be verified for kubernetes" when using Kubernetes in the AWS flavour
CloudControl uses the
official guide to install kubectl on an RPM-based system.
However, Google seems to
regularly have problems with its key-signing in the used repository,
so we added a workaround to this problem. If you add the environment variable AWS_SKIP_GPG=1 to your
docker-compose.yaml, it will ignore an invalid GPG key during the installation.
Please note though, that this affects the security of the system and should not be used constantly.
Can be used to connect to infrastructure in the AWS cloud. Also see the AWS CLI documentation for more configuration options. If you have activated MFA, set AWS_MFA_ARN to the ARN of your MFA device so CloudControl will ask you for your code. To start a new session in the CloudControl context, run `createSession <token>` afterwards
- Environment AWS_ACCESS_KEY_ID: Specifies an AWS access key associated with an IAM user or role
- Environment AWS_SECRET_ACCESS_KEY: Specifies the secret key associated with the access key. This is essentially the password for the access key
- Environment AWS_DEFAULT_REGION: Specifies the AWS Region to send the request to
- Environment AWS_MFA_ARN: ARN of the MFA device to use to log in
Can be used to connect to infrastructure in the Azure cloud. Because we're using a container, a device login will happen, requiring the user to go to a website, enter a code and login. This only happens once during initialization phase.
- Environment AZ_SUBSCRIPTION: The Azure subscription to use in this container
- Environment AZ_TENANTID: The Azure tenant id to log into (optional)
Can be used to connect to infrastructure outside of a specific cloud provider.
Includes workflows and tools to connect to a Tanzu cluster.
The kubernetes login tokens usually expire after a few hours already. You can run the `relogin` script (located in ~/bin, thus available without path) to re-execute the same login commands as the initialization process does.
Installs and configures the Fish Shell with configured Spacefish theme
- USE_fish: Enable this feature
- DEBUG_fish: Debug this feature
Installs AzCopy
- USE_azcopy: Enable this feature
- DEBUG_azcopy: Debug this feature
Installs Direnv
- USE_direnv: Enable this feature
- DEBUG_direnv: Debug this feature
Installs Helm
- USE_helm: Enable this feature
- DEBUG_helm: Debug this feature
- Environment HELM_VERSION: Valid Helm version to install (e.g. 1.5.4)
Installs the JSON parser and processor jq
- USE_jq: Enable this feature
- DEBUG_jq: Debug this feature
Installs kc, a quick context switcher for kubernetes.
- USE_kc: Enable this feature
- DEBUG_kc: Debug this feature
Installs kubectl node-shell
- USE_kubectlnodeshell: Enable this feature
- DEBUG_kubectlnodeshell: Debug this feature
Installs and configures kubernetes with kubectl to connect to the flavour's kubernetes clusters
-
USE_kubernetes: Enable this feature
-
DEBUG_kubernetes: Debug this feature
-
(azure flavour) Environment AZ_K8S_CLUSTERS: A comma separated list of AKS clusters to manage inside CloudControl (only for azure flavour). Each cluster is a pair of resource group and cluster name, separated by a colon. Optionally, you can specify the target subscription. For example: myresourcegroup:myk8s,myotherresourcegroup@othersubscription:mysecondk8s will install myk8s from the myresourcegroup resource group and mysecondk8s from the resource group myotherresourcegroup in the subscription othersubscription. Prefix a cluster name with an ! to load the admin-credentials for that cluster instead of the user credentials. This generates the script `k8s-relogin` which allows you to recreate the Kubernetes credentials.
-
(azure flavour) Environment AZ_K8S_INSTALL_OPTIONS: Additional options for the az aks install-cli programm. (Defaults to empty)
-
(azure flavour) Environment AZ_KUBELOGIN_VERSION: Version of kubelogin to install (e.g. v0.0.13). Kubelogin won't be installed, if not set (which is the default)
-
(aws flavour) Environment AWS_K8S_CLUSTERS: A comma separated list of EKS clusters to manage inside CloudControl (only for aws flavour). For each cluster give the cluster name. If you need to assume an ARN role, add that to the clustername with an additional | added. For example: myekscluster|arn:aws:iam::32487234892:sample/sample
If you additionally need to assume a role before fetching the EKS credentials, add the role, prefixed with an @: myekscluster|arn:aws:iam::4327849324:sample/sample@arn:aws:iam::specialrole
-
(aws flavour) Environment AWS_SKIP_GPG: If set, skips the gpg checks for the yum repo of kubectl, as this sometimes seems to fail.
-
(simple and aws flavour) Environment KUBECTL_VERSION: The version of kubectl to install
-
(tanzu flavour)
-
Environment TANZU_HOST: The tanzu host to download the kubectl vsphere plugin from and authenticate against
-
Environment TANZU_USERNAME: The username to authenticate with
-
Environment KUBECTL_VSPHERE_PASSWORD: The password to authenticate with
-
Environment TANZU_CLUSTERS: A comma separated list of namespace:cluster name pairs
-
Environment TANZU_ADD_CONTROL_CLUSTER: Whether to also authenticate against the control cluster [false]
-
Environment TANZU_SKIP_TLS_VERIFY: Skip TLS verification [false]
-
Environment TANZU_VSPHERE_PLUGIN_PATH: The path where to find the kubectl vsphere plugin [/wcp/plugin/linux-amd64/vsphere-plugin.zip]
This generates the script `k8s-relogin` which allows you to recreate the Kubernetes credentials.
Installs additional packages into the container
- USE_packages: Enable this feature
- DEBUG_packages: Debug this feature
- Environment PACKAGES: A whitespace separated list of packages to install. The packages will be installed with the flavour's default package manager.
Installs Packer
- USE_packer: Enable this feature
- DEBUG_packer: Debug this feature
- Environment PACKER_VERSION: Valid Packer version to install (e.g. 1.5.4)
Runs commands inside the shell when entering the cloud control container
- USE_run: Enable this feature
- DEBUG_run: Debug this feature
- Environment RUN_COMMANDS: Valid shell commands to run
Installs stern, a multi pod and container log tailing for Kubernetes
- USE_stern: Enable this feature
- DEBUG_stern: Debug this feature
- Environment STERN_VERSION: Valid Stern version (e.g. 1.11.0)
Installs and configures Terraform
- USE_terraform: Enable this feature
- DEBUG_terraform: Debug this feature
- Volume-target /terraform: Your local terraform base directory
- Volume-target /credentials.terraform: A Terraform variable file holding sensitive information when working with terraform (e.g. Terraform app secrets, etc.)
- Environment TERRAFORM_VERSION: A valid terraform version to install (e.g. 0.12.17)
Installs Terragrunt
- USE_terragrunt: Enable this feature
- DEBUG_terragrunt: Debug this feature
- Environment TERRAGRUNT_VERSION: Valid version of terragrunt to install
Configures the container's timezone
- USE_timezone: Enable this feature
- DEBUG_timezone: Debug this feature
- Environment TZ: The timezone to use
Installs the Velero kubernetes backup CLI
- USE_velero: Enable this feature
- DEBUG_velero: Debug this feature
- Environment VELERO_VERSION: Valid velero version to install (e.g. v1.4.2)
Installs Vim
- USE_vim: Enable this feature
- DEBUG_vim: Debug this feature
Installs the YAML parser and processor yq
- USE_yq: Enable this feature
- DEBUG_yq: Debug this feature
- Environment YQ_VERSION: Valid YQ version to install (e.g. 4.5.0)
CloudControl supports a decoupled development of features and flavours. If you're missing something, just fork this repository, create a subfolder for your new feature under "features" and add these files:
- feature.yaml: A descriptor for your feature with a title, a description and configuration notes
- install.sh: A shell script that is run by CloudControlCenter and should install everything you need for your new feature
- motd.sh: (optional) If you want to show some information to the users upon login, put them here.
If you need another flavour (aka cloud provider), add a new subdirectory under "flavour" and add a flavour.yaml describing your flavour the same way as a feature. For the rest of the files, please check out existing flavours for details. Please, include a sample configuration for your flavour to make it easier for other people to work with it.
In your install script, you can source the utils library
. /feature-installer-utils.sh
The environment variable <FEATURE NAME>_VERSION is the default variable to specify a feature version, if you support
it.
Installation scripts usually echo out some kind of progress, execute something and have to check for errors. The command
execHandle does all this in a one liner:
execHandle "Progress message" command
This will print out "Progress message...", run the command and if it exits with a non-zero status code, it will print the output of the command and exit with status code 1.
Using this makes installer script way shorter and easier to maintain.
Build a flavor container image with the base of the repository as the build context like this:
build.sh <tag> <flavour>
To build all flavours with the same tag, use
build.sh <tag>