Support ebs encryption pr devel (#619)

gregharvey · ce-jenkins · EmlynK · web-flow · commit 4f93da99aecf · 2022-02-25T14:20:41.000+01:00
* GitHub Actions - Rebuilt documentation. * Need to check if is_local is defined in webserver meta dependencies. (#522) * Ce dev refactor pr 1.x (#518) * Making it easier to test with provision-target and ce-dev. * Moving the provision forcing var back to plays so _init has it. * Adding defaults vars and test script extra options. * Adding a web server test to CI. * examples string needs to be in quotes. * Making sure is_local and _ce_provision_force_play are available to the _init role. * Adding SSH keys to the provision user. * Adding a --force to the test script. * Explicitly adding vars to role. * Fixing _init behaviour and adding SSH key for web role. * Setting default PHP version to 7.4. * Looking up the generated ce-dev SSH key instead of hard-coding one. * We cannot run the ssh_server role locally, so excluding for tests of webserver role. * Trying to remove user_root.yml in case it's breaking CI. * Adding a verbose mode to the test script. * Exposing the command in the test script. * Trying hard-coded keys again. * Changing location of data dir for test containers. * Putting vars back and restricting CI to the 'web' example. * Adding backup handling to ldap_server. (#525) * Adding backup handling to ldap_server. * Improving SSL docs and handling perms for openldap and letsencrypt. * Cron user must be specified with file. * Running as root, do not need a 'sudo' in this cron. * Allowing 'gitLab' to disable Prometheus. (#530) * Allowing 'gitLab' to disable Prometheus. * Booleans to use in jinja2 as strings must be cast as strings. * GitHub Actions - Rebuilt documentation. (#526) Co-authored-by: Code Enigma CI <sysadm@codeenigma.com> * Prometheus pr 1.x (#533) * Allowing 'gitLab' to disable Prometheus. * Booleans to use in jinja2 as strings must be cast as strings. * Tidying up CI and adding a GitLab test. * Fixing CI job description. * Add private files support for Drupal in Nginx. (#535) * Prometheus pr 1.x (#539) * Allowing 'gitLab' to disable Prometheus. * Booleans to use in jinja2 as strings must be cast as strings. * Tidying up CI and adding a GitLab test. * Fixing CI job description. * Adding a firewall config preset to open port 80 for LetsEncrypt. * Removing our unused ClamAV roles and adding a Galaxy role to common base. (#541) * Revert "Moving OSSEC pkill to use process_manager role instead. (#258)" (#544) This reverts commit 73c7bd0. * Backing out of Packer logging. * Moving key servers to a variable so we can set them. (#555) * Moving key servers to a variable so we can set them. * Allowing us to disable sending keys completely. * Oops, doubled up on existing functionality. * Fixing var name. * Adding a reboot option to the patching role. (#557) * Add minimal support for Aurora RDS instances (#567) * Attempt to create an RDS read replica. * Use new task to create Aurora RDS instances. * Try and fix linting issues. * Don't pass max_storage variable for Aurora instances. * Remove more storage related vars from Aurora RDS instance creation task. * Add profile and region to read replica creation. * Try creating the Aurora read replica another way. * Add some debug info. * Work around the silly registering of variables in Ansible. * Rename an RDS CloudWatch task for Aurora DBs and remove RDS debug info. * Add some Aurora info to aws_rds README file. * Use reader instead of replica for Aurora readers. * Remove db_cluster_identifier variable from non-Aurora RDS task. * Gpg servers fix pr 1.x (#571) * Moving key servers to a variable so we can set them. * Allowing us to disable sending keys completely. * Oops, doubled up on existing functionality. * Fixing var name. * Using a pipe to grep with 'command' cannot work, refactoring. * Making CI use the meta deploy role to test gitlab. * We mustn't assume AWS servers for deploy and controller. * Support termination protection in EC2. (#573) * Support termination protection in EC2. * Fixing CI vars. * Fixing CI vars. * Fix managed SSL key perms and the variable used for the private key. (#575) * Ec2 subnet lookup pr 1.x (#583) * First pass at EC2 subnet detection. * Touching subnet file to ensure it exists. * Trying a different approach, file module didn't work. * Switching back to file module. * We need to create the directory for new servers too. * Bad variable name. * Ec2 subnet lookup pr 1.x (#589) * First pass at EC2 subnet detection. * Touching subnet file to ensure it exists. * Trying a different approach, file module didn't work. * Switching back to file module. * We need to create the directory for new servers too. * Bad variable name. * Changing subnet lookup order to check for defined subnet first. * Fixing gitlab-runner overriders so upgrades do not break the runner. (#586) * Fixing gitlab-runner overriders so upgrades do not break the runner. * Fixing override file template. * Hopefully fixing CI. * Making sure the service directory exists. * We cannot use the deploy meta role in CI because of LDAP. * Changing dir perms and adding a force. * Gitlab runner service override pr 1.x (#591) * Fixing gitlab-runner overriders so upgrades do not break the runner. * Fixing override file template. * Hopefully fixing CI. * Making sure the service directory exists. * We cannot use the deploy meta role in CI because of LDAP. * Changing dir perms and adding a force. * Debugging gitlab-runner directory creation issues in CI. * Fixing linting error. * Removing verbosity again but leaving 'stat' command in. * Pass db_cluster_identifier for RDS instance during ASG build (#600) * Pass RDS db_cluster_identifier, if present, during an ASG build. * Use correct variable name for RDS db_cluster_identifier. * Add a commented variable to ASG role for db_cluster_identifier so it's documented. * Also pass in the aurora_reader var from the ASG role when including the aws_rds role. (#605) * Removing obsolete MySQL config option log_syslog from template. (#607) * Adding volume encryption and type options plus a bit more flexibility on EBS control for EC2. * Setting more sane default instance sizes. * Adding more EBS options for ASGs. * GitHub Actions - Rebuilt documentation. (#536) Co-authored-by: Code Enigma CI <sysadm@codeenigma.com> * Setting encryption to match AMI settings. * Setting encryption to match AMI settings. * We also need to dynamically set the ASGs own encrypt_boot var. * Consistent default region pr 1.x (#611) * Moving all region settings to _aws_region var and adding README update. * Documentation update. * We need to merge the new branch changes before we can rebuild the docs. * No need for region, IAM SAML setup is global, (#617) * Fixing merge command in CI. * Not sure toc.sh is actually executing. * Refactoring encrypt EBS flags to avoid detected loop condition in vars. Co-authored-by: Code Enigma CI <sysadm@codeenigma.com> Co-authored-by: EmlynK <emlyn.kinzett@codeenigma.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
diff --git a/.github/workflows/ce-provision-build-docs.yml b/.github/workflows/ce-provision-build-docs.yml
@@ -31,8 +31,8 @@ jobs:
           git remote set-url origin https://x-access-token:${{ secrets.GITHUB_TOKEN }}@github.com/${{ github.repository }}
           git fetch
           git checkout documentation
-          git merge ${{ github.event.pull_request.head.ref }}
-          contribute/toc.sh
+          git merge origin/${{ github.event.pull_request.head.ref }}
+          /bin/sh ./contribute/toc.sh
           git add docs
           git add roles
           git diff --quiet && git diff --staged --quiet || git commit -am 'GitHub Actions - Rebuilt documentation.' && git push origin documentation
diff --git a/docs/_Sidebar.md b/docs/_Sidebar.md
@@ -3,9 +3,14 @@
   - [Install](/install)
   - [Usage](/scripts)
   - [Roles](roles)
+     - [Init role](/roles/_init)
+     - ["Meta" roles that group individual roles together.](/roles/_meta)
+       - [AWS account](/roles/_meta/aws_account)
+       - [AWS client](/roles/_meta/aws_client_instance)
+       - [AWS region](/roles/_meta/aws_region)
+     - [\_overrides.](/roles/_overrides)
      - [Ansible](/roles/ansible)
      - [Extra packages](/roles/apt_extra_packages)
-     - [AWS Cloudwatch agent](/roles/aws_cloudwatch_agent)
      - [AWS Infrastructure](/roles/aws)
        - [AWS Certificate Manager](/roles/aws/aws_acm)
        - [AWS AMI](/roles/aws/aws_ami)
@@ -19,6 +24,7 @@
        - [EC2 instance with EIP](/roles/aws/aws_ec2_with_eip)
        - [EFS client](/roles/aws/aws_efs_client)
        - [AWS EFS](/roles/aws/aws_efs)
+       - [EFS client](/roles/aws/aws_efs_client)
        - [AWS IAM EC2](/roles/aws/aws_iam_role)
        - [AWS IAM SAML](/roles/aws/aws_iam_saml)
        - [AWS key pair.](/roles/aws/aws_provision_ec2_keypair)
@@ -28,36 +34,33 @@
        - [VPC](/roles/aws/aws_vpc)
        - [Update main route for a given VPC](/roles/aws/aws_vpc_route)
        - [VPC](/roles/aws/aws_vpc_subnet)
+     - [AWS Cloudwatch agent](/roles/aws_cloudwatch_agent)
      - [AWS SSM agent](/roles/aws_ssm_agent)
      - [ce-deploy](/roles/ce_deploy)
      - [Extra packages](/roles/ce_dev)
      - [Automated patching](/roles/ce_patcher)
      - [ce-provision](/roles/ce_provision)
+     - [ClamAV Clamscan](/roles/clamav_clamscan)
+     - [ClamAV Daemon](/roles/clamav_daemon)
+     - [UFW Firewall](/roles/firewall)
      - [Firewall Config](/roles/firewall_config)
      - [Frontail](/roles/frontail)
-     - [Ansible Role: Apache Solr](/roles/geerlingguy.solr)
      - [Gitlab](/roles/gitlab)
      - [Gitlab Runner](/roles/gitlab_runner)
      - [GPG Key](/roles/gpg_key)
      - [HA Proxy](/roles/haproxy)
      - [Managed /etc/hosts](/roles/hosts)
-     - [Init role](/roles/_init)
      - [Jenkins](/roles/jenkins)
      - [Jitsi](/roles/jitsi)
      - [LDAP Server](/roles/ldap_server)
      - [LHCI](/roles/lhci)
-     - ["Meta" roles that group individual roles together.](/roles/_meta)
-       - [AWS account](/roles/_meta/aws_account)
-       - [AWS client](/roles/_meta/aws_client_instance)
-       - [AWS region](/roles/_meta/aws_region)
      - [Mount sync](/roles/mount_sync)
      - [MariaDB Client](/roles/mysql_client)
      - [NGINX](/roles/nginx)
      - [NodeJS](/roles/nodejs)
      - [opcache](/roles/opcache)
      - [OpenVPN Config](/roles/openvpn_config)
      - [OSSEC](/roles/ossec)
-     - [\_overrides.](/roles/_overrides)
      - [PHP Composer](/roles/php_composer)
      - [PHP XDebug](/roles/php_xdebug)
      - [Postfix](/roles/postfix)
diff --git a/roles/aws/aws_ec2_autoscale_cluster/defaults/main.yml b/roles/aws/aws_ec2_autoscale_cluster/defaults/main.yml
@@ -19,10 +19,9 @@ aws_ec2_autoscale_cluster:
   ami_owner: self # Default to self-created image.
   root_volume_size: 30
   root_volume_type: gp2 # available options - https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-volume-types.html
-  root_volume_encrypted: "{{ aws_ami.encrypt_boot }}" # in most cases this should match encrypt_boot in the aws_ami role
   root_volume_delete_on_termination: true
   ebs_optimized: true
-  encrypt_boot: "{{ aws_ami.encrypt_boot }}" # Whether to encrypt the EBS volumes or not, taken from the aws_ami role
+  encrypt_boot: false # Whether to encrypt the EBS volumes or not, passed to the aws_ami role and to EBS volumes when instances are built
   ami_playbook_file: "{{ playbook_dir }}/ami.yml"
   ami_refresh: true # Whether to build a new AMI or not.
   asg_refresh: true # Whether to build a new ASG or not.
diff --git a/roles/aws/aws_ec2_autoscale_cluster/tasks/main.yml b/roles/aws/aws_ec2_autoscale_cluster/tasks/main.yml
@@ -169,7 +169,7 @@
       - device_name: /dev/xvda
         volume_size: "{{ aws_ec2_autoscale_cluster.root_volume_size }}"
         volume_type: "{{ aws_ec2_autoscale_cluster.root_volume_type }}"
-        encrypted: "{{ aws_ec2_autoscale_cluster.root_volume_encrypted }}"
+        encrypted: "{{ aws_ec2_autoscale_cluster.encrypt_boot }}"
         delete_on_termination: "{{ aws_ec2_autoscale_cluster.root_volume_delete_on_termination }}"
   register: _aws_ec2_lc_created
   when:
