Skip to content

Commit 72137a4

Browse files
gregharveygregharvey
authored
Enhanced quick start pr 2.x (#2207)
* Added more generic firewall rules that might be useful. * Added an update step to the installer. * Providing some default playbooks people can use in ce-provision. * Slight docs tweak for showtime!
1 parent 8ddb254 commit 72137a4

File tree

25 files changed

+489
-1
lines changed

25 files changed

+489
-1
lines changed

‎install.sh

+3-1
Original file line numberDiff line numberDiff line change
@@ -161,7 +161,9 @@ if [ ! -d "/home/$CONTROLLER_USER/ce-provision" ]; then
161161
/usr/bin/su - "$CONTROLLER_USER" -c "git clone --branch $CONFIG_REPO_BRANCH $CONFIG_REPO /home/$CONTROLLER_USER/ce-provision/config"
162162
/usr/bin/su - "$CONTROLLER_USER" -c "/usr/bin/ln -s /home/$CONTROLLER_USER/ce-provision/config/ansible.cfg /home/$CONTROLLER_USER/ce-provision/ansible.cfg"
163163
else
164-
/usr/bin/echo "ce-provision directory at /home/$CONTROLLER_USER/ce-provision already exists. Skipping."
164+
/usr/bin/echo "ce-provision directory at /home/$CONTROLLER_USER/ce-provision already exists. Updating."
165+
/usr/bin/su - "$CONTROLLER_USER" -c "cd /home/$CONTROLLER_USER/ce-provision && git pull origin $VERSION"
166+
/usr/bin/su - "$CONTROLLER_USER" -c "cd /home/$CONTROLLER_USER/ce-provision/config && git pull origin $CONFIG_REPO_BRANCH"
165167
/usr/bin/echo "-------------------------------------------------"
166168
fi
167169
/usr/bin/mkdir -p "/home/$CONTROLLER_USER/ce-provision/galaxy/roles"

‎plays/aws_account/README.md

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# Base playbook for configuring an AWS account.
2+
3+
@TODO provide example infra repo for use with the AWS EC2 inventory plugin.

‎plays/aws_account/aws_account.yml

+16
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
---
2+
# Global infra setup.
3+
- hosts: localhost
4+
connection: local
5+
become: false
6+
vars:
7+
_init:
8+
vars_dirs:
9+
- "{{ _ce_provision_build_dir }}/vars/_global"
10+
# used for tagging
11+
_profile: core
12+
_env_type: core
13+
roles:
14+
- _init
15+
- _meta/aws_account
16+
- _exit

‎plays/aws_asg/README.md

+14
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# Base playbooks for creating a new AWS ASG.
2+
For a standard ASG build just add `cluster.yml` to your environment play, like this:
3+
4+
```yaml
5+
- import_playbook: "{{ _ce_provision_base_dir }}/plays/aws_asg/cluster.yml"
6+
vars:
7+
_aws_region: eu-west-1
8+
_env_type: dev
9+
_aws_resource_name: cluster-acme-com
10+
```
11+
12+
If you have specific requirements for your AMIs you can copy these plays to your infra repository and alter them accordingly. Don't forget to copy/include `launch.yml` from the `_ec2_standalone` plays or orchestration of brand new clusters will fail.
13+
14+
@TODO provide example infra repo for use with the AWS EC2 inventory plugin.

‎plays/aws_asg/ami.yml

+35
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
---
2+
# This is the provisioning for the AMI and will run inside a temporary instance using Packer.
3+
- hosts: default
4+
become: true
5+
6+
vars:
7+
_init:
8+
vars_dirs:
9+
- "{{ _ce_provision_build_dir }}/vars/_global"
10+
- "{{ _ce_provision_build_dir }}/vars/_regions/{{ _aws_region }}/_common"
11+
- "{{ _ce_provision_build_dir }}/vars/_regions/{{ _aws_region }}/{{ _env_type }}"
12+
- "{{ _ce_provision_build_dir }}/vars/{{ _aws_resource_name }}"
13+
_profile: asg
14+
15+
tasks:
16+
- name: Upgrade the system and update cache
17+
ansible.builtin.apt:
18+
upgrade: dist
19+
update_cache: true
20+
- ansible.builtin.import_role:
21+
name: _init
22+
- ansible.builtin.import_role:
23+
name: _meta/aws_client_instance
24+
- ansible.builtin.import_role:
25+
name: _meta/webserver
26+
- ansible.builtin.import_role:
27+
name: debian/aws_efs_client
28+
- ansible.builtin.import_role:
29+
name: debian/squashfs
30+
- ansible.builtin.import_role:
31+
name: debian/mount_sync
32+
- ansible.builtin.import_role:
33+
name: debian/swap
34+
- ansible.builtin.import_role:
35+
name: _exit

‎plays/aws_asg/asg.yml

+25
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
---
2+
# Common ASG infra.
3+
- hosts: "_{{ _aws_resource_name | regex_replace('-', '_') }}"
4+
connection: local
5+
become: false
6+
7+
vars:
8+
_init:
9+
vars_dirs:
10+
- "{{ _ce_provision_build_dir }}/vars/_global"
11+
- "{{ _ce_provision_build_dir }}/vars/_regions/{{ _aws_region }}/_common"
12+
- "{{ _ce_provision_build_dir }}/vars/_regions/{{ _aws_region }}/{{ _env_type }}"
13+
- "{{ _ce_provision_build_dir }}/vars/{{ _aws_resource_name }}"
14+
_profile: asg
15+
16+
tasks:
17+
- ansible.builtin.import_role:
18+
name: _init
19+
run_once: true
20+
- ansible.builtin.import_role:
21+
name: aws/aws_ec2_autoscale_cluster
22+
run_once: true
23+
- ansible.builtin.import_role:
24+
name: _exit
25+
run_once: true

‎plays/aws_asg/cluster.yml

+5
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
# Creates hosts entry so play isn't skipped.
3+
- ansible.builtin.import_playbook: ../aws_ec2_standalone/launch.yml
4+
# Spins up the cluster.
5+
- ansible.builtin.import_playbook: asg.yml

‎plays/aws_ec2_standalone/README.md

+8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
# Base playbook for setting up a standalone EC2 instance.
2+
IMPORTANT: these plays deliberately exclude the `_init._profile` variable because it usually needs to be set at runtime or in a separate infrastructure config repo.
3+
4+
The `server.yml` file is the 'main' play, to customise we suggest this is copied to an infra repo and renamed as `hostname.yml` e.g. `acme-dev1.yml`. You also need to copy `provision.yml` so you can control what is provisioned. The `provision.yml` file is intended only as a model.
5+
6+
If you want a separate RDS instance to pair with your EC2 instance then uncomment the last two play import lines in `server.yml`, however note you do need to sort out outbound firewall ports in iptables and a Security Group for inbound traffic to the RDS instance - usually port `3306` outbound from the EC2 instance in `firewall_config` and an SG that allows `3306` inbound to RDS.
7+
8+
@TODO provide example infra repo for use with the AWS EC2 inventory plugin.

‎plays/aws_ec2_standalone/ami.yml

+20
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
---
2+
# This is the bare provisioning for the AMI.
3+
- hosts: default
4+
become: true
5+
6+
vars:
7+
_init:
8+
vars_dirs:
9+
- "{{ _ce_provision_build_dir }}/vars/_global"
10+
- "{{ _ce_provision_build_dir }}/vars/_regions/{{ _aws_region }}/_common"
11+
- "{{ _ce_provision_build_dir }}/vars/_regions/{{ _aws_region }}/{{ _env_type }}"
12+
- "{{ _ce_provision_build_dir }}/vars/{{ _aws_resource_name }}"
13+
14+
tasks:
15+
- ansible.builtin.import_role:
16+
name: _init
17+
- ansible.builtin.import_role:
18+
name: debian/user_provision
19+
- ansible.builtin.import_role:
20+
name: _exit

‎plays/aws_ec2_standalone/ec2.yml

+24
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
---
2+
# First step. Spin up a "blank" instance from a fresh AMI.
3+
- hosts: "_{{ _aws_resource_name | regex_replace('-', '_') }}"
4+
connection: local
5+
become: false
6+
7+
vars:
8+
_init:
9+
vars_dirs:
10+
- "{{ _ce_provision_build_dir }}/vars/_global"
11+
- "{{ _ce_provision_build_dir }}/vars/_regions/{{ _aws_region }}/_common"
12+
- "{{ _ce_provision_build_dir }}/vars/_regions/{{ _aws_region }}/{{ _env_type }}"
13+
- "{{ _ce_provision_build_dir }}/vars/{{ _aws_resource_name }}"
14+
15+
tasks:
16+
- ansible.builtin.import_role:
17+
name: _init
18+
- ansible.builtin.import_role:
19+
name: aws/aws_ami
20+
- ansible.builtin.import_role:
21+
name: aws/aws_ec2_with_eip
22+
- ansible.builtin.import_role:
23+
name: _exit
24+
- ansible.builtin.meta: refresh_inventory

‎plays/aws_ec2_standalone/launch.yml

+38
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
---
2+
# Prepare the ground for a new EC2 machine
3+
- hosts: localhost
4+
connection: local
5+
become: false
6+
7+
vars:
8+
_init:
9+
vars_dirs:
10+
- "{{ _ce_provision_build_dir }}/vars/_global"
11+
- "{{ _ce_provision_build_dir }}/vars/_regions/{{ _aws_region }}/_common"
12+
- "{{ _ce_provision_build_dir }}/vars/_regions/{{ _aws_region }}/{{ _env_type }}"
13+
- "{{ _ce_provision_build_dir }}/vars/{{ _aws_resource_name }}"
14+
# copied from aws_ami.yml in group_vars/all because we do not want to load aws_ami vars yet
15+
ami_groups:
16+
- "all"
17+
- "_{{ _aws_resource_name | regex_replace('-', '_') }}"
18+
- "_{{ _infra_name | regex_replace('-', '_') }}"
19+
- "_{{ _env_type | regex_replace('-', '_') }}"
20+
21+
tasks:
22+
- ansible.builtin.import_role:
23+
name: _init
24+
- name: Blank the _aws_hostname variable.
25+
ansible.builtin.set_fact:
26+
_aws_hostname: ""
27+
- name: Check to see if an Ansible host exists.
28+
ansible.builtin.set_fact:
29+
_aws_hostname: "{{ item }}"
30+
with_inventory_hostnames:
31+
- "_{{ _aws_resource_name | regex_replace('-', '_') }}"
32+
- name: If an Ansible host is not found, create it so we can execute EC2 orchestration.
33+
ansible.builtin.add_host:
34+
name: "_{{ _aws_resource_name | regex_replace('-', '_') }}"
35+
groups: "{{ ami_groups }}"
36+
when: _aws_hostname | length == 0
37+
- ansible.builtin.import_role:
38+
name: _exit

‎plays/aws_ec2_standalone/mysql_client.yml

+32
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
---
2+
- hosts: "_{{ _aws_resource_name | regex_replace('-', '_') }}"
3+
become: true
4+
5+
vars:
6+
_init:
7+
vars_dirs:
8+
- "{{ _ce_provision_build_dir }}/vars/_global"
9+
- "{{ _ce_provision_build_dir }}/vars/_regions/{{ _aws_region }}/_common"
10+
- "{{ _ce_provision_build_dir }}/vars/_regions/{{ _aws_region }}/{{ _env_type }}"
11+
- "{{ _ce_provision_build_dir }}/vars/{{ _aws_resource_name }}"
12+
13+
tasks:
14+
- ansible.builtin.import_role:
15+
name: _init
16+
- ansible.builtin.import_role:
17+
name: debian/user_deploy
18+
# Look up RDS hostname
19+
- name: Get information about an instance
20+
community.aws.rds_instance_info:
21+
region: "{{ _aws_region }}"
22+
profile: "{{ _aws_profile }}"
23+
db_instance_identifier: "{{ _aws_resource_name }}"
24+
become: true
25+
become_user: "{{ user_provision.username }}"
26+
delegate_to: localhost # needs to run on controller
27+
register: _database_info
28+
# Install MySQL client
29+
- ansible.builtin.import_role:
30+
name: debian/mysql_client
31+
- ansible.builtin.import_role:
32+
name: _exit

‎plays/aws_ec2_standalone/provision.yml

+29
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
---
2+
- hosts: "_{{ _aws_resource_name | regex_replace('-', '_') }}"
3+
become: true
4+
5+
vars:
6+
_init:
7+
vars_dirs:
8+
- "{{ _ce_provision_build_dir }}/vars/_global"
9+
- "{{ _ce_provision_build_dir }}/vars/_regions/{{ _aws_region }}/_common"
10+
- "{{ _ce_provision_build_dir }}/vars/_regions/{{ _aws_region }}/{{ _env_type }}"
11+
- "{{ _ce_provision_build_dir }}/vars/{{ _aws_resource_name }}"
12+
13+
tasks:
14+
- ansible.builtin.import_role:
15+
name: _init
16+
- ansible.builtin.import_role:
17+
name: ce_ldap_safelist
18+
- ansible.builtin.import_role:
19+
name: _meta/common_base
20+
- ansible.builtin.import_role:
21+
name: _meta/aws_client_instance
22+
- ansible.builtin.import_role:
23+
name: debian/ssh_server
24+
- ansible.builtin.import_role:
25+
name: debian/firewall_config
26+
- ansible.builtin.import_role:
27+
name: debian/swap
28+
- ansible.builtin.import_role:
29+
name: _exit

‎plays/aws_ec2_standalone/rds.yml

+56
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
---
2+
# Create an RDS instance.
3+
- hosts: "_{{ _aws_resource_name | regex_replace('-', '_') }}"
4+
connection: local
5+
become: false
6+
7+
vars:
8+
_init:
9+
vars_dirs:
10+
- "{{ _ce_provision_build_dir }}/vars/_global"
11+
- "{{ _ce_provision_build_dir }}/vars/_regions/{{ _aws_region }}/_common"
12+
- "{{ _ce_provision_build_dir }}/vars/_regions/{{ _aws_region }}/{{ _env_type }}"
13+
- "{{ _ce_provision_build_dir }}/vars/{{ _aws_resource_name }}"
14+
15+
tasks:
16+
- ansible.builtin.import_role:
17+
name: _init
18+
19+
# Automate subnet fetching
20+
- name: Create empty var to hold subnet IDs.
21+
ansible.builtin.set_fact:
22+
_aws_rds_vpc_subnet_ids: []
23+
24+
- name: Gather VPC information.
25+
amazon.aws.ec2_vpc_net_info:
26+
profile: "{{ aws_rds.aws_profile }}"
27+
region: "{{ aws_rds.region }}"
28+
filters:
29+
"tag:Name": "{{ _infra_name }}"
30+
register: _aws_rds_vpc
31+
32+
- name: Set the VPC id from name.
33+
ansible.builtin.set_fact:
34+
_aws_rds_vpc_id: "{{ _aws_rds_vpc.vpcs[0].vpc_id }}"
35+
36+
- name: Gather public subnet information.
37+
amazon.aws.ec2_vpc_subnet_info:
38+
profile: "{{ aws_rds.aws_profile }}"
39+
region: "{{ aws_rds.region }}"
40+
filters:
41+
vpc-id: "{{ _aws_rds_vpc_id }}"
42+
tag:Env: "{{ _env_type }}"
43+
tag:Profile: "core"
44+
register: _aws_rds_vpc_subnets
45+
46+
- name: Place subnet IDs in a list.
47+
ansible.builtin.set_fact:
48+
_aws_rds_vpc_subnet_ids: "{{ _aws_rds_vpc_subnet_ids + [item.subnet_id] }}"
49+
loop: "{{ _aws_rds_vpc_subnets.subnets }}"
50+
51+
# Build the RDS instance.
52+
- ansible.builtin.import_role:
53+
name: aws/aws_rds
54+
55+
- ansible.builtin.import_role:
56+
name: _exit

‎plays/aws_ec2_standalone/server.yml

+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
# Prepares a host entry so the ec2.yml play succeeds.
2+
- ansible.builtin.import_playbook: "{{ _ce_provision_base_dir }}/plays/aws_ec2_standalone/launch.yml"
3+
# Spins up the instance.
4+
# We use the central _deploy role to provision the EC2 instance to avoid duplication.
5+
- ansible.builtin.import_playbook: "{{ _ce_provision_base_dir }}/plays/aws_ec2_standalone/ec2.yml"
6+
# Actual provisioning
7+
- ansible.builtin.import_playbook: provision.yml
8+
# RDS instance
9+
#- ansible.builtin.import_playbook: "{{ _ce_provision_base_dir }}/plays/aws_ec2_standalone/rds.yml"
10+
# MySQL client - needs to happen after RDS instance is created
11+
#- ansible.builtin.import_playbook: "{{ _ce_provision_base_dir }}/plays/aws_ec2_standalone/mysql_client.yml"

‎plays/aws_region/README.md

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# Base playbook for configuring an AWS region.
2+
3+
@TODO provide example infra repo for use with the AWS EC2 inventory plugin.

‎plays/aws_region/aws_region.yml

+17
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
---
2+
# Global infra setup.
3+
- hosts: localhost
4+
connection: local
5+
become: false
6+
vars:
7+
_init:
8+
vars_dirs:
9+
- "{{ _ce_provision_build_dir }}/vars/_global"
10+
- "{{ _ce_provision_build_dir }}/vars/_regions/{{ _aws_region }}/_common"
11+
- "{{ _ce_provision_build_dir }}/vars/_regions/{{ _aws_region }}/{{ _env_type }}"
12+
# used for tagging
13+
_profile: core
14+
roles:
15+
- _init
16+
- _meta/aws_region
17+
- _exit

‎plays/controller/README.md

+31
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
# Base playbook for setting up an infra controller.
2+
This playbook provides a model for managing an Ansible infra controller with ce-provision based at AWS.
3+
4+
If your server is not in AWS or you are not using the AWS EC2 inventory plugin, you must ensure your server's hostname is in your Ansible hosts file (`config/hosts/hosts` or `hosts.yml`) and provide the same hostname in the `_provision_host` variable. Then call `provision.yml` directly, for example:
5+
6+
```yaml
7+
---
8+
- name: Configure my controller server.
9+
ansible.builtin.import_playbook: "{{ _ce_provision_base_dir }}/plays/controller/provision.yml"
10+
vars:
11+
_env_type: util
12+
_provision_host: controller.acme.com
13+
_profile: controller
14+
```
15+
16+
If you are using the AWS EC2 inventory plugin and the Code Enigme recommended set-up, you must provide the `_aws_resource_name` variable - note, this is hyphenated, no dots - and call `aws_controller.yml`, for example:
17+
18+
```yaml
19+
---
20+
- name: Configure my controller server at AWS.
21+
ansible.builtin.import_playbook: "{{ _ce_provision_base_dir }}/plays/controller/aws_controller.yml"
22+
vars:
23+
_env_type: util
24+
_aws_region: eu-west-1
25+
_aws_resource_name: controller-acme-com
26+
_profile: controller
27+
```
28+
29+
This will create or find an EC2 instance with the AWS tag of `Name: controller-acme-com` which will be in an inventory group called `_controller_acme_com`.
30+
31+
@TODO provide example infra repo for use with the AWS EC2 inventory plugin.

0 commit comments

Comments
 (0)