dcos-launch configuration YAML

To see sample config files, see the sample_configs directory.

This is a WIP to more thoroughly document all the parameters. The code this is based off of is in dcos_launch/config.py. Please report any errors or unclear items.

TODO: blank template for each of the deployment types that contained all the possible params with blank values.

Here optional means you don't have to manually provide it in the config.yaml file, and that it will be set to the default value if not provided.

Design Intention

The intention of this configuration file is to provide an interface by which all deployments of DC/OS, regardless of provider, have a similar format, thus complementing the goal of dcos-launch to provide a single tool for launching across a variety of provider APIs.

Supported Deployments and Examples

See sample_configs for a full list.

Simple AWS Cloudformation
Zen AWS Cloudformation
Onprem Install on AWS Bare Cluster
Azure Template Deployment
Onprem Installation on Google Cloud Platform
GCP Onprem with fault-domain helper

onprem can only be provisioned via aws and gcp platforms
DC/OS deployed from aws or azure provider do not technically need ssh_user or ssh_private_key_filename. However, without this additional data, the integration tests will not be trigger-able from dcos-launch. Thus, it is not recommended, but allowable, to omit these fields when not using the onprem provider

Sections

Universal params
Template-based deploy params
Onprem deploy params
AWS onprem params
ACS (Azure) deploy params
GCP (Google Cloud Platform) deploy params
Terraform params

Universal params

These options are for any config

`key_helper`

boolean, optional

Generate private SSH keys for the underlying hosts if true. In platform: aws, this means the user does not have to supply KeyName in the template parameters and dcos-launch will fill it in. Similarly, in platform: azure, sshRSAPublicKey is populated automatically. In the aws case, this key will be deleted from EC2 when the deployment is deleted with dcos-launch

Default: false

`launch_config_version`

integer, required

This is still a tool under active development and as such a strict version specifier must be included. (Right now there is only v1.)

Allowed: 1

`provider`

string, required

Which provider you will deploy a cluster to.

aws: Uses Amazon Web Services (AWS) CloudFormation console. Supports both zen and simple templates. (Can only be used with platform: aws. Requires: template_url, template_parameters
azure: Uses Azure Resource Manager deployment templates. Supports both ACS (Azure Container Service) and DC/OS templates. (Can only be used with platform: azure. Requires template_url, and template_parameters
onprem: Uses the DC/OS bash installer to orchestrate a deployment on arbitrary hosts of a bare cluster. Requires num_masters, num_private_agents, num_public_agents, installer_url, instance_type, os_name, and dcos_config

Allowed: aws, azure, dcos-engine, onprem, gcp, terraform.

`ssh_port`

integer, optional

What port to use on SSH.

Default: 22

`ssh_private_key`

string, optional

If this is not set and you are not using key_helper, you must provide ssh_private_key_filename.

`ssh_private_key_filename`

string, optional (required if you are doing an onprem deploy)

This is required if you don't use key_helper or ssh_private_key.

If key_helper: true then this field cannot be supplied.

`ssh_user`

string, optional

Username to log into cluster with. If provider: onprem then the host VM configuraiton is known to dcos-launch and this value will be calculated based on os_name.

Default: core

`tags`

dict, optional

Arbitrary tags to set on your deployment. This can be anything (subject to tag length restrictions according to which provider you're deploying on) as dcos-launch does not do anything with these other than pass them through.

One example on how to use this would be setting KubernetesCluster tags when deploying via AWS CloudFormation templates (these are required if the cluster is running k8s).

Note that the expiration tags in some of the example configurations are something meant to run with internal Mesosphere services that garbage collect clusters (e.g. expiration: 4h creates a cluster that will be deleted after 4 hours, whereas leaving this blank will default to the cluster being deleted after 2 hours). An option to update tags after a cluster has been deployed is in the backlog.

`zen_helper`

boolean, optional

Only to be used with provider: aws and zen templates. If true, then the network prerequisites for launching a zen cluster will be provided if missing. The resources potentially covered are: Vpc, InternetGateway, PrivateSubnet, and PublicSubnet. As with key_helper, these resources will be deleted if dcos-launch is used for destroying the deployment.

Default: false

Template-based deploy params

These params should be present if you are deploying with a template.

`template_parameters`

dict, required

Parameters that should be injected into the template you are using. Example params for an AWS template:

template_parameters:
    AdminLocation: 0.0.0.0/0
    PublicSlaveInstanceCount: 1
    SlaveInstanceCount: 2
    DefaultInstanceType: m4.large

`template_url`

string, required

URL of template to use. E.g. should resolve to a page with a CloudFormation template if you are deploying on AWS.

Onprem deploy params

`dcos_config`

dict, required

Config options for DC/OS itself (not options about resources for the provider).

Params that can be nested include:

ip_detect_filename
ip_detect_public_filename
fault_domain_detect_filename
license_key_filename

but it will also accept other options (TODO: Where are these defined?)

dcos_config:
    cluster_name: My Awesome DC/OS
    resolvers:
        - 8.8.4.4
        - 8.8.8.8
    dns_search: mesos
    master_discovery: static

`deployment_name`

string, required

The name of the cloud resource that will be provided by dcos-launch. E.g. if you are deploying with an AWS CloudFormation template, then this will be the name of your stack.

`fault_domain_helper`

dict, optional

Items in the dict should have dicts containing:

num_zones int, required, default true
num_private_agents, int, required, default 0
num_public_agents, int, required, default 0
local, boolean, required, default false

Only to be used with provider: onprem. This option allows defining an arbitrary number of named regions by creating a spoofed fault-domain-detect script. Each region can configure the number of private agents, public agents, and sub-zones. One region must declared with local: true to designate it as the region which will host the masters. Agents are assigned distributed evenly amongst the zones within a region per a given role (master/private/public). Do not set the num_private_agents and num_public_agents in the top-level config. These values will computed automatically from the numbers you provide in the fault_domain_helper. For example consider this fault domain helper:

num_masters: 3
fault_domain_helper:
    USA:
        num_zones: 2
        num_private_agents: 3
        local: true
    Germany:
        num_zones: 3
        num_public_agents: 2
        num_private_agents: 4
    Europe:
        num_private_agents: 1

will produce the following region/zones:

USA-1:
    masters: 2
    private_agents: 1
USA-2:
    masters: 1
    private_agents: 2
Germany-1:
    public_agents: 1
    private_agents: 1
Germany-2:
    public_agents: 1
    private_agents: 1
Germany-3:
    public_agents: 0
    private_agents: 2
Europe-1:
    private_agents: 1

string, required

Allowed: aws, gcp, gce

`prereqs_script_filename`

string, required if install_rereqs is true

If the image you are going to be installing DC/OS on needs requirements (e.g. Docker) installed before running the installer script, provide a file to run to install those here.

Default: dcos_launch/scripts/install_prereqs.sh

AWS onprem params

`admin_location`

string, required

Default: 0.0.0.0/0

`aws_block_device_mappings`

list of dicts, optional

`aws_key_name`

string, required if key_helper:false and provider:aws and platform:aws

Pre-existing EC2 SSH KeyPair to be supplied for launching the VPC

`bootstrap_ssh_user`

string, required (if not set, this will be set based on bootstrap_os_name)

Example: centos

`iam_role_permissions`

list of dict with keys Resource, Action, Effect, optional

`instance_device_name`

string, required

Default: /dev/xvda if coreos else /dev/sda1

`instance_type`

string, required

E.g. m4.large

`os_name`

string, optional (you can set the machine image directly with instance_ami)

Default: cent-os-7-dcos-prereqs

Allowed: see OS_AMIS in dcos_launch/aws.py

`ssh_user`

string, required (if not set, will be set based on os_name)

ACS (Azure) deploy params

⚠️ TODO: descriptions

`dcos_engine_tarball_url`

string, optional

Default: 'https://github.com/Azure/dcos-engine/releases/download/v0.1.0/dcos-engine-v0.1.0-<sys.platform>-amd64.tar.gz'

`acs_template_filename`

string, optional

`dcos_linux_bootstrap_url`

string, optional

string, optional

Default: Azure

`provider_package_id`

string, optional

`ssh_public_key`

string, optional

`template_parameters`

dict, optional?

`windows_admin_password`

string, optional

Default: Replacepassword123

`windows_admin_user`

string, optional

Default azureuser

⚠️ TODO: descriptions

`disable_updates`

boolean, optional

Default: false

`disk_size`

integer, optional

Default: 42

`disk_type`

string, optional

`ssh_public_key`

string, optional

`use_preemptible_vms`

boolean, optional

Default: False

Terraform params

`dcos-enterprise`

boolean

Are you installing Enterprise DC/OS (not open source DC/OS)?

Default: false

`init_dir`

string, optional

Default: terraform-init-<uuid4>

string, optional

Default: latest Terraform

Files

CONFIG_OPTIONS.md

Latest commit

History

CONFIG_OPTIONS.md

File metadata and controls

dcos-launch configuration YAML

Design Intention

Supported Deployments and Examples

Universal params

key_helper

launch_config_version

provider

ssh_port

ssh_private_key

ssh_private_key_filename

ssh_user

tags

zen_helper

Template-based deploy params

template_parameters

template_url

Onprem deploy params

dcos_config

deployment_name

fault_domain_helper

genconf_dir

install_prereqs

installer_port

installer_url

num_masters

num_private_agents

num_public_agents

onprem_install_parallelism

platform

prereqs_script_filename

AWS onprem params

admin_location

aws_block_device_mappings

aws_key_name

bootstrap_ssh_user

iam_role_permissions

instance_device_name

instance_type

os_name

ssh_user

ACS (Azure) deploy params

dcos_engine_tarball_url

acs_template_filename

dcos_linux_bootstrap_url

deployment_name

linux_admin_user

linux_private_vm_size

linux_public_vm_size

master_vm_size

num_linux_private_agents

num_linux_public_agents

num_masters

num_windows_private_agents

num_windows_public_agents

platform

provider_package_id

ssh_public_key

template_parameters

windows_admin_password

windows_admin_user

windows_image_source_url

windows_offer

windows_private_vm_size

windows_public_vm_size

windows_publisher

windows_sku

GCP (Google Cloud Platform) deploy params

disable_updates

disk_size

disk_type

image_project

machine_type

os_name

source_image

`key_helper`

`launch_config_version`

`provider`

`ssh_port`

`ssh_private_key`

`ssh_private_key_filename`

`ssh_user`

`tags`

`zen_helper`

`template_parameters`

`template_url`

`dcos_config`

`deployment_name`

`fault_domain_helper`

`genconf_dir`

`install_prereqs`

`installer_port`

`installer_url`

`num_masters`

`num_private_agents`

`num_public_agents`

`onprem_install_parallelism`

`platform`

`prereqs_script_filename`

`admin_location`

`aws_block_device_mappings`

`aws_key_name`

`bootstrap_ssh_user`

`iam_role_permissions`

`instance_device_name`

`instance_type`

`os_name`

`ssh_user`

`dcos_engine_tarball_url`

`acs_template_filename`

`dcos_linux_bootstrap_url`

`deployment_name`

`linux_admin_user`

`linux_private_vm_size`

`linux_public_vm_size`

`master_vm_size`

`num_linux_private_agents`

`num_linux_public_agents`

`num_masters`

`num_windows_private_agents`

`num_windows_public_agents`

`platform`

`provider_package_id`

`ssh_public_key`

`template_parameters`

`windows_admin_password`

`windows_admin_user`

`windows_image_source_url`

`windows_offer`

`windows_private_vm_size`

`windows_public_vm_size`

`windows_publisher`

`windows_sku`

`disable_updates`

`disk_size`

`disk_type`

`image_project`

`machine_type`

`os_name`

`source_image`

`ssh_public_key`

`use_preemptible_vms`

`dcos-enterprise`

`init_dir`

`key_helper`

`platform`

`terraform_config`

`terraform_dcos_enterprise_version`

`terraform_dcos_version`

`terraform_tarball_url`

`terraform_version`