Note
This is a modified version of the README.md file found at github.com/intel/ccloudvm
Configurable Cloud VM (ccloudvm) is a command line tool for creating and managing Virtual Machine (VM) based development environments. ccloudvm creates these development environments from the cloud images of Linux distributions. These images have a number of interesting properties. They are small, quick to download, quick to start and are configurable using a tool, cloud-init, widely adopted in the cloud. ccloudvm builds VM based development environments from either pre-shipped or user supplied annotated cloud-init files.
All you need to have installed on your machine is:
- Go 1.16 or greater.
The installation instructions for the latest version of Go can be found here. Once installed, ensure that your PATH environment variable contains the location of executables built with the Go tool chain. This can be done by executing the following command.
$ export PATH=$PATH:$(go env GOPATH)/bin
Then, to create a new Ubuntu 16.04 VM, simply type
$ go install github.com/markdryan/ccloudvm/...@latest
$ ccloudvm setup
$ ccloudvm create xenial
The go install command downloads, builds and installs ccloudvm. The ccloudvm setup command installs some needed dependencies on your local PC such as qemu and xorriso, and initialises a systemd user service. The ccloudvm create command downloads an Ubuntu Cloud Image and creates a VM based on this image. It then boots the VM, creates an account for you, with the same name as your account on your host, and optionally updates the default packages.
ccloudvm will cache the Ubuntu image locally so the next time you create another xenial based VM you won't have to wait very long. It also knows about HTTP proxies and will mirror your host computer's proxy settings inside the VMs it creates.
There's currently a bug which may cause the ccloudvm create command to fail if the user on the host computer is not a member of the kvm group. The failure message will look something like this.
Booting VM with 6 GB RAM and 2 cpus
Failed to launch qemu : exit status 1, Could not access KVM kernel module: Permission denied
failed to initialize KVM: Permission denied
You can resolve this problem by adding yourself to the group of users permitted to access the kvm device. Assuming that this group is called kvm you would execute
$ sudo gpasswd -a $USER kvm
and then close your existing terminal and open a new one. Typing groups in the new terminal should confirm that you are now a member of the kvm group.
Once the ccloudvm create command is finished you'll be able to connect to the the VM via SSH using the following command.
$ ccloudvm connect
The command above assumes that either $GOPATH/bin or ~/go/bin is in your PATH.
You'll notice that the instance was assigned a name when it was created. This name is used refer to the instance when issuing future ccloudvm commands. You have the option to specify a name when creating an instance, using the --name option. If you do not provide a name, ccloudvm will create one for you. Assuming the name of our new instance was 'vague-nimue' the above connect command could have been written as.
$ ccloudvm connect vague-nimue
If there is only one ccloudvm instance in existence the name is optional and need not be specified when issuing most commands. However, if you have created two or more instances you will need to specify an instance name when issuing ccloudvm commands so ccloudvm knows which instance to operate on.
You can delete the VM you've just created by running,
$ ccloudvm delete
Each VM is created from a workload. A workload is a text file which contains a set of instructions for initialising the VM. A workload, among other things, can specify:
- The resources to be consumed by the VM, e.g., VCPUs, memory, disk space
- The base image from which the VM is to be created
- The folders that should be shared between the host and the VM
- Any file backed storage that should appear as a device in the VM
- An annotated cloud-init file that contains the set of instructions to run on the first boot of the VM. This file is used to create user accounts, install packages and configure the VM.
ccloudvm ships with a number of workloads for creating VMs based on standard images, such as Ubuntu 16.04 and Fedora 25. Users are also free to create their own workloads. Standard workloads defined at $GOPATH/src/github.com/intel/ccloudvm/workloads are stored in the ccvm binary as data files. User created workloads are stored in ~/.ccloudvm/workloads. ccloudvm always checks the ~/.ccloudvm/workloads directory first so if a workload exists in both directories with the same name, ccloudvm will use the workload in ~/.ccloudvm/workloads.
When creating a new instance via the create command the user must specify a workload. This can be done by providing the name of a workload, present in one of the two directories mentioned above, a URI pointing to either a local or a remote file, or an absolute path.
When specifying a workload by name the .yaml extension should be omitted. For example, the command,
$ ccloudvm create xenial
creates a new instance from the workload $GOPATH/src/github.com/intel/ccloudvm/workloads/xenial.yaml.
Three schemes are supported when specifying a workload via a URI; http, https and file. Note that workloads fetched from URIs are not stored locally in the two workload directories mentioned above. Each time this option is used, ccloudvm will try to retrieve the workload from the remote location.
An absolute path can also be specified. This is equivalent to using the file scheme. For example, to create a workload using the file /home/x/workload.yaml we have two options.
Using the file scheme:
$ ccloudvm create file:///home/ccloudvm/workload.yaml
or the absolute path:
$ ccloudvm create /home/ccloudvm/workload.yaml
ccloudvm workloads are multi-doc YAML files containing two documents. The first document describes how to create and boot instances. The second is a cloud-init file containing a list of instructions for configuring the guest OS running inside the instances.
An example workload is shown below:
---
base_image_url: https://cloud-images.ubuntu.com/xenial/current/xenial-server-cloudimg-amd64-disk1.img
base_image_name: Ubuntu 16.04
vm:
mem_mib: 2048
disk_gib: 16
cpus: 2
...
---
#cloud-config
package_upgrade: false
users:
- name: {{.User}}
gecos: CIAO Demo User
lock-passwd: true
shell: /bin/bash
sudo: ALL=(ALL) NOPASSWD:ALL
ssh-authorized-keys:
- {{.PublicKey}}
...
This workload can be used to create new instances from the Ubuntu 16.04 cloud images. These instances will be assigned 2 GiB of RAM, 2 VCPUs and 16 GiB for their root file systems. A new user will be created inside the guest OS using the user name of the user on the host computer that invoked ccloudvm.
Note that ccloudvm only works with images that are designed to run cloud-init on their first boot. Typically, these are the cloud images, e.g.,. the Ubuntu Cloud Images.
Go templates can be used in each document in a workload yaml file. These templates have access to an instance of a structure called Workspace that contains some useful information about the host's environment and the instance that is being created. The following pieces of information are available via the workload structure.
- GoPath : The GoPath on the host or ~/go if GOPATH is not defined
- Home : The home directory of the user who ran ccloudvm
- HTTPProxy : The hosts HTTP proxy if set
- HTTPSProxy : The hosts HTTPS proxy if set
- NoProxy : The value of the host's no_proxy variable
- User : The user who invoked ccloudvm
- UID : The UID of the user who invoked ccloudvm
- GID : The GID of the user who invoked ccloudvm
- PublicKey : The public key that will be used to access the instance over SSH
- GitUserName : The git user name of the user who ran ccloudvm
- GitEmail : The git email address of the user who ran ccloudvm
- Mounts : A slice of mounts, each describing a path to be shared between guest and host
- Hostname : The instance hostname
- UUID : A UUID for the new instance
- PackageUpgrade : Indicates whether package upgrade should be performed during the first boot.
As an example consider the second document in the workload definition above. The User and PublicKey fields are accessed via the {{.User}} and {{.PublicKey}} Go templates. Abstracting all of the user and instance specific information via templates in this way allows us to keep our workload definitions generic.
The first yaml document is called the instance specification document. It defines the fixed characteristics of instances created from the workload which cannot be altered. Three fields are currently defined:
- base_image_url : The URL of the qcow2 image upon which instances of the workload should be based.
- base_image_name : Friendly name for the base image. This is optional.
- vm : Contains information about creating and booting the instance.
- bios : A URI (file, http, or https) pointing to the BIOS file, e.g, OVMF.fd, with which to boot the image. Should be omitted for legacy boots.
- kernel : A URI (file, http, or https) pointing to a kernel file to use instead of the kernel image in the rootfs.
- kernel_args : arguments to pass to the kernel.
- initrd : A URI (file, http, or https) pointing to an initrd to use
- qemu : Executable name or full path to the QEMU binary to use
- machine : can be used to specify a value for QEMU's --machine option
- cpu : can be used to specify a value for QEMU's --cpu option. On X86 "host" will be used if not specified.
Normally, there is no need to specify kernel, kernel_args or initrd. These settings are only required when you want to use a custom kernel with a cloud image. Note ccloudvm uses the cdrom drive to store the cloud-config data, so the guest kernel you supply must support CDROMs. The following kernel config options are required.
CONFIG_CDROM=y
CONFIG_ISO9660_FS=y
The qemu field must be provided if the machine field is specified.
The vm field supports a number of child fields.
- mem_mib : Number of mebibytes to assign to the VM. Defaults to 1024 MiBs.
- disk_gib : Number of gibibytes to assign to the rootfs of the VM. Defaults to 60 GiB.
- cpus : Number of CPUs to assign to the VM. Defaults to 1 VCPU.
- ports : Sequence of port objects which map host ports to guest ports
- mounts : Sequence of mount objects which describe the folders shared between the host and the guest
- drives : Sequence of drive objects which identify resources accessible on the host to be made available as block devices on the guest.
Each instance is associated with one of the host's IP addresses. Only one instance can be associated with one host IP address at any one time. The user can specify which host IP address to use for an instance when the instance is created using the --hostip option. If no host IP address is specified, ccloudvm will select an unused IP address from the subnet 127.0.0.0/8. This is usually what you want unless you need to expose a guest service to devices other than your host.
Each port object has two members, host and guest. They are both integers and they specify the mapping of port numbers from host IP address of the instance to the guest. A default mapping of 10022 to 22 is always added so that users can connect to their VMs via ccloudvm connect. An example port mapping is shown below:
ports:
- host: 10022
guest: 22
Mount objects can be used to share folders between the host and guest. Folders are shared using the 9p protocol. Each mount object has three pieces of information.
- tag : An id for the mount. This information is needed when mounting the shared folder in the guest
- security_model : The 9p security model to use
- path : The path of the host folder to share
An example of a mount is given below.
mounts:
- tag: hostgo
security_model: passthrough
path: /home/user/go
Note that specifying a mount in the instance data document only creates a 9p device which is visible inside the guest. To actually access the shared folder from inside the guest you need to mount the folder. This can be done in the cloud-init file discussed below.
Drive objects allow the user to make a resource accessible from the host available as a block device in the guest VM. Currently, these resources are restricted to being file backed storage located on the host. Each drive object has three pieces of information.
- path : The path of the resource as accessed from the host
- format : The format of the resource, e.g., qcow2
- options : A comma separate list of options. This field is optional.
An example of a drive is given below.
drives:
- path: /tmp/img.qcow2
format: qcow2
options: aio=native
The second document contains a cloud-init user data file that can be used to create users, mount folders, install and update packages and perform general configuration tasks on the newly created VM. The user is expected to be familiar with the cloudinit file format.
The cloudinit document is processed by the Go template engine before being passed to cloudinit running inside the guest VM. The template engine makes a number of functions available to the cloudinit document. These functions are used to perform common tasks such as configuring proxies and communicating with ccloudvm running on the host. Each function must be passed the workspace object as the first parameter. The following functions are available:
Generates a string containing variable definitions for all the proxy variables. It is useful for prepending to commands inside the runcmd: section of the cloudinit document that require proxies to be set, e.g,
- {{proxyvars .}} wget https://storage.googleapis.com/golang/go1.8.linux-amd64.tar.gz" "/tmp/go1.8.linux-amd64.tar.gz
may expand to something like after being processed by the template engine
- https_proxy=http://myproxy.mydomain.com:911 wget https://storage.googleapis.com/golang/go1.8.linux-amd64.tar.gz" "/tmp/go1.8.linux-amd64.tar.gz
Generates proxy definitions in a format that is suitable for the /etc/environment file. It should be used as follows:
write_files:
{{with proxyEnv . 5}}
- content: |
{{.}}
path: /etc/environment
{{end}}
Download should be used to download files from inside the guest. Download is preferable to directly downloading files inside the cloudinit document using curl or wget for example, as the downloaded files are cached by ccloudvm. This means that the next time an instance of this workload is created the file will not have to be retrieved from the Internet, resulting in quicker boot times for the new VM.
download takes three parameters
- The workspace object
- The URL of the file to download
- The location on the guest where the downloaded file should be stored
An example of its usage is
- {{download . "https://storage.googleapis.com/golang/go1.8.linux-amd64.tar.gz" "/tmp/go1.8.linux-amd64.tar.gz"}}
There are four functions that can be used to provide visual information back to the user who invoked ccloudvm on the host. These functions should always be issued in pairs. The first function invoked should always be beginTask.
beginTask takes two parameters. The first is the workspace object. The second is a string to present to the user. beginTask is typically followed by a command and then a call to either endTaskOk, endTaskFail or endTaskCheck.
endTaskOk prints "[OK]" to the screen, endTaskFail prints "[FAIL]" to the screen and endTaskCheck prints "[OK]" or "[FAIL]" depending on the result of the previous command.
For example,
- {{beginTask . "Unpacking Go" }}
- tar -C /usr/local -xzf /tmp/go1.8.linux-amd64.tar.gz
- {{endTaskCheck .}}
will print
Unpacking Go : [OK]
if the tar command succeeds
and
Unpacking Go : [FAIL]
if it fails.
Reporting a failure back to ccloudvm does not cause the create command to exit. The failure is presented to the user and the setup of the VM continues.
While the task functions are used to provide feedback on tasks run on the guest, this function is used to just provide visual information to the user who invoked ccloudvm on the host.
For example,
- {{message . "All tasks complete. Have fun using the machine" }}
will print
All tasks complete. Have fun using the machine
As previously mentioned, mounts specified in the instance data document will only create 9p devices in the guest. In order to access the files in the shared folder you need to arrange to have these devices mounted. The way you do this might differ from distro to distro. On Ubuntu it is done by adding a mount: section to the cloudinit document, e.g.,
mounts:
{{range .Mounts}} - [{{.Tag}}, {{.Path}}, 9p, "x-systemd.automount,x-systemd.device-timeout=10,nofail,trans=virtio,version=9p2000.L", "0", "0"]
{{end -}}
The above command will arrange for all mounts specified in the instance data document or on the create command line to be mounted to the same location that they are mounted on the host.
Mounts added later via the start command will need to be mounted manually.
ccloudvm ships with some basic workloads for common Linux distributions such as Ubuntu 16.04 and Fedora 25. These workloads contain a number of generic instructions that create a user, configure proxies and arrange for any shared host folders to be mounted in the guest. When creating a new workload it's convenient to be able to re-use all of this functionality. This could be done by simply copying your base workload of choice into a new workload file and modifying it. However, there's a much nicer way of doing this. ccloudvm allows you to create a new workload by inheriting from an existing workload. This is done by specifying the inherits field in the first document of the workload. When workload B inherits from workload A, the contents of workload A are merged into the contents of workload B. If there is a conflict, i.e., workload A and workload B specify different values for the same field, workload B, the inheriting workload, takes priority.
---
inherits: xenial
vm:
disk_gib: 20
ports:
- host: 8000
guest: 80
...
---
package_update: true
packages:
- apache2
...
In this example we are creating a new workload that inherits from the existing xenial workload. It request a 20 GiB rootfs disk, overriding the 16 GiB allocation requested by the parent. It adds a port mapping from 8000 on the instance's host IP address to 80 on the guest's, updates the package cache and installs a new package, apache2. As the workload inherits from the xenial workload, instances created from this workload will have a user account created for them, will have their proxies, if any, set up correctly and any mounts specified at creation time will be automatically mounted in the guest.
There are a few last things to note when using workload inheritance. You can only inherit from a workload stored in one of the two standard workload directories, i.e., it's not possible to specify a URI or an absolute path when using the inherits field. Workload hierarchies are supported, i.e., arbitrary levels of inheritance are supported. There is however no multiple inheritance. A workload can only directly inherit from one other workload.
ccloudvm create creates and configures a new ccloudvm VM. All the files associated with the VM are stored under the ~/.ccloudvm/instances folder.
An example of ccloudvm create is given below:
$ ccloudvm create xenial
Downloading Ubuntu 16.04
Downloaded 289 MB of 289
Booting VM with 1024 MiB RAM and 1 cpus
Booting VM : [OK]
Adding amused-lancelot to /etc/hosts : [OK]
VM successfully created!
Instance amused-lancelot created
Type ccloudvm connect amused-lancelot to start using it.
By default, ccloudvm will assign 1 GiB of memory, 1 VCPU and 60 GiB of disk space to each new instance it creates. These resource allocations can be overridden both in the workload specification and on the command line using the --mem, --cpu, and --disk options. For example,
$ ccloudvm create --cpus 2 --mem 2048 --disk 10 xenial
Creates and boots a VM with 2 VCPUs, 2 GiB of RAM and a rootfs of max 10 GiB.
The --package-upgrade option can be used to provide a hint to workloads indicating whether packages contained within the base image should be updated or not during the first boot. Updating packages can be quite time consuming and may not be necessary if the user just wants to create a throw away VM to test something under a particular distro. This hint only works if the workloads define the following in their cloudinit documents.
package_upgrade: {{with .PackageUpgrade}}{{.}}{{else}}false{{end}}
The --qemuport option can be used to follow qemu boot logs by connecting to user defined port or an entry in the workload yaml file.
For example,
--qemuport=9999
will let you track qemu logs by running the below command
nc localhost 9999
You can also allow a login via this port in case ssh fails to work by modifying the workload file and changing lock_passwd to false and providing a passwd: "....." entry following that.
The --cpu option can be specified to request that QEMU use a specific cpu model. If not specified "host" will be used unless the machine type is "virt".
The --qemupath option can be used to specify a full path to the qemu binary you wish to use for running the new instance.
Finally, create also accepts -kernel and -kernalargs parameters that allow the user to provide a custom kernel on the command line.
Each new instance created by ccloudvm is assigned a host IP address on which guest services can be exposed to the host and in some cases the outside world. When creating a new instance you can specify the host IP address on which to expose guest services using the --hostip option. However, unless you need those services to be accessible externally, it's usually best to go with the default behaviour and let ccloudvm create choose the host IP address for you. If you do not specify the --hostip option an IP address from the range 127.0.0.0/8 will be allocated to your instance.
To expose a guest service you need to map a port on the host IP address assigned to your instance to the port on which the service is listening on the guest. By default, ccloudvm creates one port mapping for new VMs, 10022-22 for SSH access. You can specify additional port mappings, mounts or drives or even override the default settings on the command line.
For example,
$ ccloudvm create --mount docs,passthrough,$HOME/Documents --port 10000-80 xenial
shares the host directory, $HOME/Documents, with the ccloudvm VM using the 9p passthrough security model. The directory can be mounted inside the VM using the docs tag. The command also adds a new port mapping. HOSTIP:10000 on the host now maps to port 80 on the guest, where HOSTIP is the host IP address assigned to the instance.
New file backed storage devices can be added to the guest using the drive option. --drive requires at least two parameters. The first is the location of the file backed storage, e.g., the location on the host of a qcow2 file. The second indicates the format of that storage. A user can specify additional options which are passed straight through to the underlying hypervisor. For example, let's suppose we have a qcow2 file called $HOME/img.qcow2. We could make this file accessible as a block device in our VM as follows.
$ ccloudvm create --drive $HOME/img.qcow2,qcow2,aio=threads
The drive will appear as a device, e.g., /dev/vdc in the VM.
Multiple --mount, --drive and --port options can be provided and it's also possible to override the existing values. Existing mounts, ones specified in the instance data document, can be overridden by specifying an existing tag with new options, existing ports can be overridden by mapping a new host port to an existing guest port and existing drives can be overridden by providing a new set of options for an existing drive path. For example,
$ ccloudvm create --mount hostgo,none,$HOME/go -port 10023-22 xenial
changes the security model of the mount with the hostgo tag and makes the instance available via ssh on HOSTIP:10023.
The copy command is used to copy files between the host and the guest. Files can be transferred in both directions and it's also possible to recursively copy directories. The instance name is optional if there is only one instance. Let's look at some examples.
$ ccloudvm copy local.conf local.conf
Copies the local.conf file in the user's current directory on the host to the user's home directory in the guest.
$ ccloudvm copy sad-nimue -r inc code/inc
Recursively copies the contents of the directory inc from the host's current directory to the ~/code/inc directory on the sad-nimue guest.
Finally,
$ ccloudvm copy -t /var/log/cloud-init-output.log cloud-init-output.log
copies the log file from /var/log/cloud-init-output.log on the guest to the current host directory.
ccloudvm delete, shuts down and deletes all the files associated with the VM.
ccloudvm instances, displays information about the existing instances, e.g.,
$ ccloudvm instances
Name HostIP Workload VCPUs Mem Disk
alarmed-agravain 127.3.232.2 xenial 1 1024 MiB 16 Gib
tense-peles 127.3.232.1 xenial 2 2048 MiB 10 Gib
The run command can be used to execute a command on a running guest instance without having to manually SSH into the guest and type the command. The first argument to the run command is the instance name. This argument is always required even if there is only one instance. All subsequent arguments are passed to the guest to be executed. For example,
$ ccloudvm run gloomy-arthur "lsb_release --release"
Release: 16.04
Note that it's best to quote the command that is to be executed on the guest, if that command contains more than one word.
ccloudvm status provides information about the current ccloudvm VM, e.g., whether it is running, and how to connect to it. For example,
$ ccloudvm status tense-peles
Name : tense-peles
HostIP : 127.3.232.1
Workload: xenial
Status : VM up
SSH : ssh -q -F /dev/null -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o IdentitiesOnly=yes -i /home/markus/.ccloudvm/id_rsa 127.3.232.1 -p 10022
VCPUs : 2
Mem : 2048 MiB
Disk : 10 GiB
ccloudvm stop is used to power down a ccloudvm VM cleanly.
ccloudvm start boots a previously created but not running ccloudvm VM. The start command also supports the --mem and --cpu options. So it's possible to change the resources assigned to the guest VM by stopping it and restarting it, specifying --mem and --cpu. It's also possible to specify additional port mappings or mounts via the start command.
Any parameters you pass to the start command override the parameters you originally passed to create. These settings are also persisted. For example, if you were to run
$ ccloudvm create --mem=2 xenial
$ ccloudvm stop
$ ccloudvm start --mem=1
$ ccloudvm stop
$ ccloudvm start
The final ccloudvm instance would boot with 1GB of RAM even though no mem parameter was provided.
The -qemuport option can be used to follow qemu boot logs by connecting to user defined port or an entry in the workload yaml file.
For example,
$ ccloudvm start -qemuport 9999
will let you track qemu logs by running the below command
$ nc localhost 9999
ccloudvm quit terminates the VM immediately. It does not shut down the OS running in the VM cleanly.
The setup command installs any needed dependencies and enables a systemd user service. ccloudvm is actually a very simple command line tool. It delegates most of the work to a systemd user service. This service is launched by socket activation and only runs when needed. If it has no work to do it quits.
setup takes one optional parameter, -d or --datadir. By default, ccloudvm will store all of the files it downloads and all of the instance data that it creates under $HOME/.ccloudvm. This directory can get quite large if you have multiple instances as it will contain the rootfs files for those instances. By specifying the --datadir you can opt to have your instance data stored elsewhere.
The ccloudvm teardown command serves two purposes:
- it deletes all existing instances
- it stops the systemd user service, disables it and deletes all the unit files associated with it
An example of its use is as follows:
$ ccloudvm teardown
Deleting alarmed-agravain
Deleting tense-peles
Removing ccloudvm service
Once you run ccloudvm teardown, ccloudvm will be unusable until you run ccloudvm setup.
The script that might help with auto completion is located
here. This can be either placed at
/etc/bash_completion.d/ of your machine or as described below.
Note: you will need to start a new shell to be able to make use of auto completion.
$ mkdir ~/.bash_completion.d
$ cp scripts/ccloudvm_bashcomplete ~/.bash_completion.d/ccloudvm
$ cat << \EOF > ~/.bash_completion
for file in ~/.bash_completion.d/*; do
[ -f "${file}" ] && . ${file}
done
EOF