Skip to content

Latest commit

 

History

History
83 lines (58 loc) · 3.87 KB

README.md

File metadata and controls

83 lines (58 loc) · 3.87 KB
Raw

Bootstrap a Kubernetes cluster with custom binaries. No Kubeadm!

Devkube

Currently, a lot of Kubernetes bootstrappers are black boxes which don't provide a lot of flexibility on how the cluster is set up. Devkube changes that. Based off of Kelsey's KTHW, it wires together a cluster with custom binaries, complete with Flannel and CoreDNS.

devkube-demo

Features

  • Use custom binaries for each component (kube-scheduler, kube-api-server, kubelet etc).
    • The variables.yml file has links for the components.
    • This is useful if you are developing a new Kubernetes feature and wish to test the change out on a real cluster
  • Change the flags on any component
    • Update the config file for the component and run the relevant playbook to deploy the changes
  • Add new nodes
    • Create the VM, run the relevant playbooks and the node should automatically join the cluster
  • Use cluster for testing manifests etc
    • Since the cluster can be created in a few minutes on bare VMs, devkube can be used to quickly get clusters up and down for testing purposes

Getting started

  • Provision the VMs on your favorite cloud provider

    • The VMs should be running Ubuntu 18.04 and be able to communicate with each other via private IPs (on all ports)
    • The master node should have the 6443 port open for the api-server.

  • Download the required dependencies on your machine

  • Edit the hosts.ini file

    • Enter the public ip, the ssh-able username and the private ip of the server
    • Make sure the server is ssh-able with the command ssh <user>@<public-ip> and the user has root access
    • The first server (k8s-node-1) will become the master and all the other nodes will join in as workers

  • Run the play.yml playbook

    • ansible-playbook play.yml -i hosts.ini

How does it work

Some of the main files are:

  • variables.yml
    • This file has various top level configs, including release
    • release has 2 possible values:
      • latest - this will pull in the latest artifact from kubernetes CI
      • v1.16.0-beta.1 etc - specific version, can be anything
  • root_certs.yml
    • This sets up the root CA and generates the public and signing key for it
  • certs.yml
    • This sets up all the certificates for the master and worker node components and transfers them to the servers
  • control_plane.yml
    • This downloads the control plane binaries and bootstraps the control plane
  • workers.yml
    • This downloads the worker node binaries and bootstraps the worker nodes.
    • It also downloads installs flannel as the CNI network plugin and adds CoreDNS for DNS.

How do I...

  • Add a new node to the existing cluster

    • Run certs.yml to generate the certs for the new node and transfer them to the server
    • Run workers.yml to install the binaries and get the node up.

  • Change the flag on the api-server

    • edit the kube-apiserver.service.j2 file and run the control_plane.yml playbook.

  • Change the kubelet binary

    • update the variables.yml file with the new binary endpoint
    • run workers.yml playbook

  • Avoid downloading worker node binaries

    • this might be required if for example you change a flag on kubelet and don't need a fresh download of binaries
    • run ansible-playbook workers.yml -i hosts.ini --skip-tags "downloads"

Contributing

Please file an issue if you face any problems. Better still, help fix it and make a PR!