Consistent default region pr devel

.github/workflows/ce-provision-build-docs.yml

@@ -33,6 +33,7 @@ jobs:
           git checkout documentation
           contribute/toc.sh
           git add docs
+          git add roles
           git diff --quiet && git diff --staged --quiet || git commit -am 'GitHub Actions - Rebuilt documentation.' && git push origin documentation
         shell: bash
 

docs/_Sidebar.md

@@ -3,26 +3,22 @@
   - [Install](/install)
   - [Usage](/scripts)
   - [Roles](roles)
-     - [Init role](/roles/_init)
-     - ["Meta" roles that group individual roles together.](/roles/_meta)
-       - [AWS account](/roles/_meta/aws_account)
-       - [AWS client](/roles/_meta/aws_client_instance)
-       - [AWS region](/roles/_meta/aws_region)
-     - [\_overrides.](/roles/_overrides)
      - [Ansible](/roles/ansible)
      - [Extra packages](/roles/apt_extra_packages)
+     - [AWS Cloudwatch agent](/roles/aws_cloudwatch_agent)
      - [AWS Infrastructure](/roles/aws)
-       - [AMI Debian Buster](/roles/aws/ami_debian_buster)
+       - [AWS Certificate Manager](/roles/aws/aws_acm)
+       - [AWS AMI](/roles/aws/aws_ami)
        - [AWS Backup](/roles/aws/aws_backup)
        - [AWS CLI](/roles/aws/aws_cli)
        - [AWS CloudFront distribution](/roles/aws/aws_cloudfront_distribution)
        - [Cloudwatch log group](/roles/aws/aws_cloudwatch_log_group)
        - [Amazon credentials](/roles/aws/aws_credentials)
        - [Autoscale cluster](/roles/aws/aws_ec2_autoscale_cluster)
        - [EC2 CloudWatch Metric Alarm](/roles/aws/aws_ec2_metric_alarm)
-       - [AMI Debian Buster](/roles/aws/aws_ec2_with_eip)
-       - [AWS EFS](/roles/aws/aws_efs)
+       - [EC2 instance with EIP](/roles/aws/aws_ec2_with_eip)
        - [EFS client](/roles/aws/aws_efs_client)
+       - [AWS EFS](/roles/aws/aws_efs)
        - [AWS IAM EC2](/roles/aws/aws_iam_role)
        - [AWS IAM SAML](/roles/aws/aws_iam_saml)
        - [AWS key pair.](/roles/aws/aws_provision_ec2_keypair)
@@ -32,32 +28,37 @@
        - [VPC](/roles/aws/aws_vpc)
        - [Update main route for a given VPC](/roles/aws/aws_vpc_route)
        - [VPC](/roles/aws/aws_vpc_subnet)
-     - [AWS Cloudwatch agent](/roles/aws_cloudwatch_agent)
      - [AWS SSM agent](/roles/aws_ssm_agent)
      - [ce-deploy](/roles/ce_deploy)
      - [Extra packages](/roles/ce_dev)
+     - [Automated patching](/roles/ce_patcher)
      - [ce-provision](/roles/ce_provision)
-     - [ClamAV Clamscan](/roles/clamav_clamscan)
-     - [ClamAV Daemon](/roles/clamav_daemon)
-     - [UFW Firewall](/roles/firewall)
      - [Firewall Config](/roles/firewall_config)
      - [Frontail](/roles/frontail)
+     - [Ansible Role: Apache Solr](/roles/geerlingguy.solr)
      - [Gitlab](/roles/gitlab)
      - [Gitlab Runner](/roles/gitlab_runner)
      - [GPG Key](/roles/gpg_key)
      - [HA Proxy](/roles/haproxy)
      - [Managed /etc/hosts](/roles/hosts)
+     - [Init role](/roles/_init)
      - [Jenkins](/roles/jenkins)
      - [Jitsi](/roles/jitsi)
      - [LDAP Server](/roles/ldap_server)
      - [LHCI](/roles/lhci)
+     - ["Meta" roles that group individual roles together.](/roles/_meta)
+       - [AWS account](/roles/_meta/aws_account)
+       - [AWS client](/roles/_meta/aws_client_instance)
+       - [AWS region](/roles/_meta/aws_region)
      - [Mount sync](/roles/mount_sync)
      - [MariaDB Client](/roles/mysql_client)
      - [NGINX](/roles/nginx)
      - [NodeJS](/roles/nodejs)
      - [opcache](/roles/opcache)
-     - [[openvpn](#openvpn)](/roles/openvpn)
+     - [OpenVPN Config](/roles/openvpn_config)
      - [OSSEC](/roles/ossec)
+     - [\_overrides.](/roles/_overrides)
+     - [PHP Composer](/roles/php_composer)
      - [PHP XDebug](/roles/php_xdebug)
      - [Postfix](/roles/postfix)
      - [Process Manager](/roles/process_manager)

docs/roles.md

@@ -2,3 +2,9 @@
 Ansible roles and group of roles that constitute the deploy stack.
 <!--TOC-->
 <!--ENDTOC-->
+
+# Required variables
+You must pass the following variables into an Ansible play before running it with ce-provision:
+
+* `_aws_profile` - the Boto3 profile to use
+* `_aws_region` - the AWS region to act in

docs/roles/_init.md

@@ -13,10 +13,12 @@ _init:
   # A list of var directories to include. We only support .yml extensions.
   # This is used to detect if the playbook must re-run or not.
   vars_dirs: []
+  force_play: false
 
-# ce_provision vars are sometimes needed even when the role isn't used so we define them here
+# ce_provision vars are sometimes needed even when the role isn't used so we define them here.
+# If you are using ce_provision and *not* using _init you can copy these vars to your playbook.
 _ce_provision:
-    username: "{% if is_local is defined and is_local %}ce-dev{% else %}controller{% endif %}"
+  username: "{% if is_local is defined and is_local %}ce-dev{% else %}controller{% endif %}"
 
 ce_provision:
   username: "{{ _ce_provision.username }}"

docs/roles/aws/aws_acm.md

@@ -0,0 +1,32 @@
+# AWS Certificate Manager
+
+Fork from https://github.com/FairwindsOps/ansible-acm
+
+Creates AWS certificate requests. Allows for passing a validation domain. From the AWS [documentation](http://docs.aws.amazon.com/acm/latest/userguide/gs-acm-validate.html):
+
+> To ensure that email is sent to the administrative addresses for an apex domain, such as example.com, rather than to the administrative addresses for a subdomain, such as test.example.com, specify the ValidationDomain option in the RequestCertificate API or the request-certificate AWS CLI command. This feature is not currently supported in the console.
+
+Additionally, this role attempts to be idempotent by running `aws acm list-certificates` and ensuring that the domain of the cert being requested is not included in the current list of certificates.
+
+<!--TOC-->
+<!--ENDTOC-->
+
+<!--ROLEVARS-->
+## Default variables
+```yaml
+---
+aws_acm:
+  region: "{{ _aws_region }}"
+  aws_profile: "{{ _aws_profile }}"
+  tags: "{{ _aws_tags }}"
+  domain_name: subdomain.example.com
+  validate: true # you need to set this to false if the validation zone is not in Route 53 or you do not have CLI access
+  export: true
+  route_53:
+    aws_profile: "{{ _aws_profile }}" # the zone might not be in the same account as the certificate
+    state: present
+    zone: example.com
+
+```
+
+<!--ENDROLEVARS-->

docs/roles/aws/ami_debian_buster.md → docs/roles/aws/aws_ami.md

@@ -1,5 +1,5 @@
-# AMI Debian Buster
-Creates an image from Debian Buster base with Packer, provisioned with an Ansible Playbook.
+# AWS AMI
+Creates an image from a selected base with Packer, provisioned with an Ansible Playbook.
 
 ## Dependencies
 This requires boto and Packer on the "provisioning" server.
@@ -10,11 +10,15 @@ This requires boto and Packer on the "provisioning" server.
 ## Default variables
 ```yaml
 ---
-ami_debian_buster:
+aws_ami:
   aws_profile: "{{ _aws_profile }}"
-  region: us-east-2
+  region: "{{ _aws_region }}"
   instance_type: t2.micro
+  virtualization_type: hvm
+  root_device_type: ebs
+  name_filter: "debian-10-amd64-*"
   ami_name: "example"
+  owner: "136693071363" # Global AWS account ID of owner, defaults to Debian official
   encrypt_boot: false
   playbook_file: "{{ playbook_dir }}/base-playbook.yml" # Path to a playbook used to provision the image.
   # Operation can be one of:

docs/roles/aws/aws_backup.md

@@ -17,7 +17,7 @@ aws_backup:
   # copy_vault:
   #   name: "Default"
   #   encryption_key: "Default" # An Amazon Resource Name (ARN) that identifies the encryption key to use in the copy region. If 'default', the default AWS encryption key will be used. If NOT 'default', the key must already exist so the ARN can be passed in.
-  #   region: "eu-central-1"
+  #   region: "{{ _aws_region }}"
   plans: [] # A list of backup plans. See below for usage example.
   # plans:
   #   - name: "ExampleDev"
@@ -49,7 +49,7 @@ aws_backup:
   #             delete_after_days: 0 # Specifies the number of days after creation that a recovery point is deleted.
   #         continuous_backup: false # Specifies whether Backup creates continuous backups.
   backup:
-    iam_role_arn: "Default" # Set to the ARN of an existing IAM role or leave as 'Default' to use the AWSBackupDefaultServiceRole role.
+    iam_role_arn: "Default" # Set to the ARN of an existing IAM role or leave as 'Default' to create a role with the name '_infra_name-backup'.
     backup_plan_name: "" # Name of the backup plan to use. Must match one in the plans list.
     selection_name: "" # Name of the resource assignation; this is set in the roles which create the resources such as aws/aws_ec2_with_eip and aws/aws_efs.
     resource_id: "" # The unique ID of the resource. For EC2, this is the instance ID. For EFS, the filesystem ID. For RDS, the DB identifier.
@@ -69,6 +69,7 @@ aws_backup:
     subscriptions:
       - endpoint: "admin@example.com"
         protocol: "email"
+
 ```
 
 <!--ENDROLEVARS-->

docs/roles/aws/aws_cloudwatch_log_group.md

@@ -11,7 +11,7 @@ Manage log groups states and retention policies.
 ---
 aws_cloudwatch_log_group:
   aws_profile: "{{ _aws_profile }}"
-  region: "eu-west-3"
+  region: "{{ _aws_region }}"
   tags: {}
   state: present
   # Number of days to keep logs, in days.

docs/roles/aws/aws_ec2_autoscale_cluster.md

@@ -8,7 +8,7 @@
 ```yaml
 aws_ec2_autoscale_cluster:
   aws_profile: "{{ _aws_profile }}"
-  region: eu-west-3
+  region: "{{ _aws_region }}"
   name: "example"
   vpc_id: vpc-XXXX # One of vpc_id or vpc_name is mandatory.
   # vpc_name: example-vpc
@@ -95,12 +95,14 @@ aws_ec2_autoscale_cluster:
   # Hosts to peer with. This will gather vpc info from the Name tag and create a peering connection and route tables.
   peering:
     - name: utility-server.example.com
-      region: eu-west-3
+      region: "{{ _aws_region }}"
   # Associated RDS instance.
   rds:
     rds: false # wether to create an instance.
     db_instance_class: db.m5.large
+    #db_cluster_identifier: example-aurora-cluster
     engine: mariadb
+    aurora_reader: false
     #engine_version: 5.7.9
     allocated_storage: 100 # Initial size in GB. Minimum is 100.
     max_allocated_storage: 1000 # Max size in GB for autoscaling.

docs/roles/aws/aws_ec2_with_eip.md

@@ -1,6 +1,6 @@
-# AMI Debian Buster
+# EC2 instance with EIP
 
-Creates an image from Debian Buster base with Packer, provisioned with an Ansible Playbook.
+Creates a new EC2 instance at AWS with a static IP address.
 
 <!--TOC-->
 <!--ENDTOC-->
@@ -11,15 +11,18 @@ Creates an image from Debian Buster base with Packer, provisioned with an Ansibl
 ---
 aws_ec2_with_eip:
   aws_profile: "{{ _aws_profile }}"
-  region: eu-west-3
+  region: "{{ _aws_region }}"
   instance_type: t2.micro
   key_name: "{{ ce_provision.username }}@{{ ansible_hostname }}" # This needs to match your "provision" user SSH key.
   ami_name: "{{ _domain_name }}" # The name of an AMI image to use. Image must exists in the same region.
   ami_owner: self # Default to self-created image.
-  vpc_subnet_id: subnet-xxx
+  # vpc_subnet_id: subnet-xxx # One of vpc_subnet_id or vpc_name + vpc_subnet_profile is mandatory.
+  vpc_name: "{{ _infra_name }}"
+  vpc_subnet_profile: core # if you are looking up subnets we need a Profile tag to search against
   # An IAM Role name to associate with the instance.
   iam_role_name: "example"
-  state: present
+  state: started
+  termination_protection: false # set to true to disable termination and avoid accidents
   instance_name: "{{ _domain_name }}"
   root_volume_size: 80
   ebs_optimized: true
@@ -29,6 +32,7 @@ aws_ec2_with_eip:
   # Add an A record tied to the EIP.
   # Set the zone to empty to skip.
   route_53:
+    state: present
     zone: "example.com"
     record: "{{ _domain_name }}"
     aws_profile: another # Not necessarily the same as the "target" one.

docs/roles/aws/aws_efs.md

@@ -10,7 +10,7 @@ Creates or update an EFS volume.
 ```yaml
 aws_efs:
   aws_profile: "{{ _aws_profile }}"
-  region: eu-west-3
+  region: "{{ _aws_region }}"
   name: example
   # If false, we omit tags enterly and leave them as is.
   purge_tags: false

docs/roles/aws/aws_efs_client.md

@@ -10,7 +10,7 @@ It uses the "Name" tag for a given volume to retrieve the volume path.
 ---
 aws_efs_client:
   aws_profile: "{{ _aws_profile }}"
-  region: eu-west-3
+  region: "{{ _aws_region }}"
   version: 1.26.2 # Version of AWS EFS utils to use.
   # See https://docs.ansible.com/ansible/latest/modules/mount_module.html
   mounts:

docs/roles/aws/aws_iam_role.md

@@ -14,7 +14,7 @@ aws_iam_role:
   # Pass either names or ARNs for the role.
   managed_policies: []
   # Which document policy to apply.
-  # Current options are 'ec2' or 'ecs'
+  # Current options are 'ec2', 'ecs' or 'backup'
   policy_document: ec2
   purge_policies: true # set to false if you want to add policies to an existing role
   tags:

docs/roles/aws/aws_iam_saml.md

@@ -69,7 +69,7 @@ This requires boto and AWS-CLI on the provisioning server.
 ```yaml
 aws_iam_saml:
   aws_profile: "{{ _aws_profile }}" # Boto profile to use for AWS connections
-  region: "eu-west-2" # AWS region to use
+  region: "{{ _aws_region }}" # AWS region to use
   tags: {} # Dict of AWS tags to apply
     #Tagname: "TagValue"
   aws_account_alias: "" # IAM account alias - human readable name to order SSO page

docs/roles/aws/aws_provision_ec2_keypair.md

@@ -9,7 +9,7 @@ Creates a key pair for the current "provision user"
 ---
 aws_provision_ec2_keypair:
   aws_profile: "{{ _aws_profile }}"
-  region: eu-west-3
+  region: "{{ _aws_region }}"
   key_name: "{{ ce_provision.username }}@{{ ansible_hostname }}"
 
 ```

docs/roles/aws/aws_rds.md

@@ -1,5 +1,14 @@
 # AWS RDS
 Creates an RDS instance and associated ressources.
+
+If the `engine` variable is set to **aurora-mysql**, you'll need to manually create the Aurora cluster first. Typically, a controller will already exist, so something like this can be run from the controller:
+
+```
+AWS_PROFILE=example aws rds create-db-cluster --db-cluster-identifier example-aurora-cluster --engine aurora-mysql --engine-version 5.7.mysql_aurora.2.10.2 --db-subnet-group-name example-aurora --vpc-security-group-ids sg-abcdefghijklmnop --storage-encrypted --master-username "auroradev" --master-user-password "aurora12345"
+```
+
+You'll need to have created the subnet group first as well as the security groups.
+
 <!--TOC-->
 <!--ENDTOC-->
 
@@ -8,7 +17,7 @@ Creates an RDS instance and associated ressources.
 ```yaml
 aws_rds:
   aws_profile: "{{ _aws_profile }}"
-  region: eu-west-3
+  region: "{{ _aws_region }}"
   multi_az: true
   subnets:
     - subnet-aaaaaaaa
@@ -19,7 +28,13 @@ aws_rds:
   state: present
   description: example
   engine: mariadb
+  aurora_reader: false # If true, an Aurora reader instance will be created.
   # engine_version: '5.7.2' # Omit to use latest.
+  # See parameter group docs: https://docs.ansible.com/ansible/latest/collections/community/aws/rds_param_group_module.html
+  # db_parameter_group_name: "example" # Omit to use default
+  # db_parameter_group_description: "Custom parameter group" # Description of parameter group
+  # db_parameter_group_engine: "mariadb10.5" # accepts different values to RDS instance 'engine'
+  # db_parameters: {} # dictionary of available parameters
   allocated_storage: 100 # Initial size in GB. Minimum is 100.
   max_allocated_storage: 1000 # Max size in GB for autoscaling.
   storage_encrypted: false # Whether to encrypt the RDS instance or not.

docs/roles/aws/aws_s3_bucket.md

@@ -10,7 +10,7 @@ Creates an S3 bucket and a matching policy.
 ```yaml
 aws_s3_bucket:
   aws_profile: "{{ _aws_profile }}"
-  region: eu-west-3
+  region: "{{ _aws_region }}"
   name: "example"
   tags:
     Name: "example"

docs/roles/aws/aws_sns.md

@@ -9,7 +9,7 @@ Creates an SNS topic and subscription.
 ---
 aws_sns:
   name: "alarms" # Name of the topic.
-  region: "eu-west-2" # The region to create the SNS topic.
+  region: "{{ _aws_region }}" # The region to create the SNS topic.
   display_name: "" # Display name for the topic, for when the topic is owned by this AWS account.
   purge_subscriptions: true # Purge subscriptions not specified in the subscriptions list.
   policy: "" # A dictionary containing the policy to use when creating the topic

docs/roles/aws/aws_vpc.md

@@ -8,7 +8,7 @@ Creates a VPC and associated subnets.
 ```yaml
 aws_vpc:
   aws_profile: "{{ _aws_profile }}"
-  region: eu-west-3
+  region: "{{ _aws_region }}"
   name: example-vpc-2
   cidr_block: "10.0.0.0/16"
   tags: {}

docs/roles/aws/aws_vpc_route.md

@@ -9,7 +9,7 @@ This will add/update routes on the "main" route table for a given VPC, leaving e
 # @todo Support associating route with several subnets.
 aws_vpc_route:
   aws_profile: "{{ _aws_profile }}"
-  region: eu-west-3
+  region: "{{ _aws_region }}"
   # Beware when calling this on "peering" routes, not to override tags with the target peer's one.
   # tags:
   #   Name: "example"

docs/roles/aws/aws_vpc_subnet.md

@@ -10,7 +10,7 @@ aws_vpc_subnet:
   vpc_id: vpc-XXXX # One of vpc_id or vpc_name is mandatory.
   # vpc_name: example-vpc
   aws_profile: "{{ _aws_profile }}"
-  region: eu-west-3
+  region: "{{ _aws_region }}"
   subnets:
     - cidr_block: "10.0.0.0/24"
       az: b