Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhanced quick start pr 2.x #2289

Merged
merged 49 commits into from
Feb 5, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
536254d
Added more generic firewall rules that might be useful.
gregharvey Jan 9, 2025
d0cf00e
Added an update step to the installer.
gregharvey Jan 9, 2025
edb18c2
Providing some default playbooks people can use in ce-provision.
gregharvey Jan 9, 2025
a2e0283
Slight docs tweak for showtime!
gregharvey Jan 9, 2025
c4516b8
Modernising hostname handling to use systemd.
gregharvey Jan 9, 2025
615870d
Switching default key type to ED25519 because it is supported by both…
gregharvey Jan 9, 2025
ae50f72
Switching to ED25519 SSH keys and adding hostname and hosts handling …
gregharvey Jan 9, 2025
e82c4d3
Merge branch '2.x' of github.com:codeenigma/ce-provision into enhance…
gregharvey Jan 9, 2025
8f8b532
Merge branch 'enhanced_quick_start' into enhanced_quick_start-PR-2.x
gregharvey Jan 9, 2025
20c7c9a
Merge branch '2.x' into enhanced_quick_start
gregharvey Jan 9, 2025
9ec571e
Adding iproute2 package so hosts role works.
gregharvey Jan 9, 2025
0b54bd8
Merge branch 'enhanced_quick_start' into enhanced_quick_start-PR-2.x
gregharvey Jan 9, 2025
d9268ed
Also need an apt-get update in CI.
gregharvey Jan 9, 2025
364469d
Merge branch 'enhanced_quick_start' into enhanced_quick_start-PR-2.x
gregharvey Jan 9, 2025
64969f3
Change of plan, stop hosts running in containers.
gregharvey Jan 9, 2025
acee3e0
Merge branch 'enhanced_quick_start' into enhanced_quick_start-PR-2.x
gregharvey Jan 9, 2025
d497ebd
Small inline docs change.
gregharvey Jan 9, 2025
ffa5773
Merge branch 'enhanced_quick_start' into enhanced_quick_start-PR-2.x
gregharvey Jan 9, 2025
e4113b1
Providing sane defaults for VPC security groups.
gregharvey Jan 10, 2025
72da198
Merge branch 'enhanced_quick_start' into enhanced_quick_start-PR-2.x
gregharvey Jan 10, 2025
e12d9e4
Making key name dynamic in the installer.
gregharvey Jan 13, 2025
7d542cb
Merge branch '2.x' of github.com:codeenigma/ce-provision into enhance…
gregharvey Jan 13, 2025
f9532f9
Merge branch 'enhanced_quick_start' into enhanced_quick_start-PR-2.x
gregharvey Jan 13, 2025
89caa26
Merge branch '2.x' into enhanced_quick_start
gregharvey Jan 13, 2025
25dc545
Error in variable namespace.
gregharvey Jan 13, 2025
23167f7
Merge branch 'enhanced_quick_start' into enhanced_quick_start-PR-2.x
gregharvey Jan 13, 2025
338115c
Merging 2.x
gregharvey Jan 30, 2025
0e3bd42
Adding GPG and SOPS to installer.
gregharvey Jan 30, 2025
05b6ad1
Merging 2.x
gregharvey Jan 30, 2025
1b3742b
Merge branch 'enhanced_quick_start' into enhanced_quick_start-PR-2.x
gregharvey Jan 30, 2025
73099eb
The GPG role should make sure GPG is installed!
gregharvey Jan 30, 2025
3ae48bb
Enforcing correct working directory when installing Galaxy roles and …
gregharvey Jan 30, 2025
8584619
Merge branch '2.x' of github.com:codeenigma/ce-provision into enhance…
gregharvey Jan 30, 2025
d3390f8
Merge branch 'enhanced_quick_start' into enhanced_quick_start-PR-2.x
gregharvey Jan 30, 2025
d146195
Merge branch '2.x' into enhanced_quick_start
gregharvey Jan 30, 2025
93af9d6
Indentation fix.
gregharvey Jan 30, 2025
8557860
Merge branch 'enhanced_quick_start' into enhanced_quick_start-PR-2.x
gregharvey Jan 30, 2025
6ebe965
Ensuring controller user is used when delegating back to localhost.
gregharvey Feb 5, 2025
503e057
Merge branch '2.x' of github.com:codeenigma/ce-provision into enhance…
gregharvey Feb 5, 2025
dad85ea
Merge branch 'enhanced_quick_start' into enhanced_quick_start-PR-2.x
gregharvey Feb 5, 2025
0f0eca7
Merge branch '2.x' into enhanced_quick_start
gregharvey Feb 5, 2025
508437f
Adding missing become parameters.
gregharvey Feb 5, 2025
4ab5334
Merge branch 'enhanced_quick_start' into enhanced_quick_start-PR-2.x
gregharvey Feb 5, 2025
54fea9d
Adding the acl Debian package to the installer.
gregharvey Feb 5, 2025
7587766
Merge branch 'enhanced_quick_start' into enhanced_quick_start-PR-2.x
gregharvey Feb 5, 2025
bbc5315
Removing unnecessary become_user parameters where it breaks CI.
gregharvey Feb 5, 2025
5352f33
Merge branch 'enhanced_quick_start' into enhanced_quick_start-PR-2.x
gregharvey Feb 5, 2025
43973e8
Providing a README file for the launcher playbook.
gregharvey Feb 5, 2025
78f27e4
Merge branch 'enhanced_quick_start' into enhanced_quick_start-PR-2.x
gregharvey Feb 5, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,7 @@ fi
git ca-certificates git-lfs \
openssh-client nfs-common stunnel4 \
python3-venv python3-debian \
zip unzip gzip tar dnsutils net-tools
acl zip unzip gzip tar dnsutils net-tools
/usr/bin/echo "-------------------------------------------------"

# Install Ansible in a Python virtual environment.
Expand Down
2 changes: 1 addition & 1 deletion plays/aws_ec2_standalone/mysql_client.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
profile: "{{ _aws_profile }}"
db_instance_identifier: "{{ _aws_resource_name }}"
become: true
become_user: "{{ user_provision.username }}"
become_user: "{{ _ce_provision_username }}"
delegate_to: localhost # needs to run on controller
register: _database_info
# Install MySQL client
Expand Down
24 changes: 24 additions & 0 deletions plays/launcher/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Base playbook to configure a new target server
This playbook configures a new target server, and should just work if you have followed the installation and configuration instructions [in our GitHub wiki](https://github.com/codeenigma/ce-provision/wiki/Installation). It will add the server to `hosts` in your config repository and install Ansible dependencies and the controller user on the target host. Once this is done you can orchestrate its future configuration with ce-provision.

## Prerequisites
* You have followed the official installation instructions for your controller
* Your new target server has the controller's SSH public key assigned to a Linux user
* Your new target server has port 22 open to the controller
* Your Ansible `hosts` inventory file has `ansible_connection=local` set for your controller at `localhost`

## Usage
On your controller server:

```shell
sudo su -l controller
cd /home/controller/ce-provision && \
./scripts/provision.sh \
--workspace /home/controller/ce-provision \
--repo none --branch none \
--playbook plays/launcher/configure.yml \
--ansible-extra-vars "_provision_host=1.2.3.4 _target_username=admin" \
--force
```

Change the value of `1.2.3.4` on the last line to match your IP address or hostname. Change the value of `admin` to whatever the Linux username is installed by default with your SSH public key attached. This may vary, see your provider's documentation for details.
32 changes: 21 additions & 11 deletions plays/launcher/configure.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,34 +6,44 @@
# ./scripts/provision.sh --workspace /home/controller/ce-provision \
# --repo none --branch none \
# --playbook /home/controller/ce-provision/plays/launcher/configure.yml \
# --ansible-extra-vars "_provision_host=X.X.X.X"
# --ansible-extra-vars "_provision_host=X.X.X.X _target_username=admin" \
# --force
#
# Replace X.X.X.X with your hostname or IP address.
# Replace admin with the username that has your SSH public key, see provider docs for details.
- hosts: localhost
vars:
_ce_provision_username: controller
_add_host: true # set to false if you do not want to add the new server to hosts.yml
_add_host: true # set to false if you do not want to add the new server to hosts file
_ce_provision_config_branch: 2.x # change this if you didn't use our example config and have a different main branch name
tasks:
- name: Add the server to Ansible hosts in memory.
ansible.builtin.add_host:
hostname: "{{ _provision_host }}"

- name: Add the server to hosts.yml.
when: _add_host
- name: Check if the hosts INI file exists.
ansible.builtin.stat:
path: "/home/{{ _ce_provision_username }}/ce-provision/config/hosts/hosts"
register: _hosts_ini_file

- name: Add the server to hosts if INI file exists.
when:
- _add_host
- _hosts_ini_file.stat.exists
block:
- name: Edit hosts.yml file.
- name: Edit hosts file.
ansible.builtin.lineinfile:
path: "/home/{{ _ce_provision_username }}/ce-provision/config/hosts/hosts.yml"
line: "{{ _provision_host }}:"
create: true
path: "/home/{{ _ce_provision_username }}/ce-provision/config/hosts/hosts"
line: "{{ _provision_host }}"
insertafter: "### ce-provision launcher hosts - do not remove"
state: present

- name: Add hosts.yml changes.
- name: Add hosts changes.
ansible.builtin.command: git add .
args:
chdir: "/home/{{ _ce_provision_username }}/ce-provision/config"

- name: Commit hosts.yml changes.
- name: Commit hosts changes.
ansible.builtin.shell: "git diff --staged --quiet || git commit -m 'Ansible autogenerated - host {{ _provision_host }} added.'"
args:
chdir: "/home/{{ _ce_provision_username }}/ce-provision/config"
Expand All @@ -45,7 +55,7 @@

- hosts: "{{ _provision_host }}"
vars:
ansible_user: admin
ansible_user: "{{ _target_username }}"
_ce_provision_build_id: 0
become: true
tasks:
Expand Down
8 changes: 5 additions & 3 deletions roles/_exit/tasks/allowed_vars.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@
ansible.builtin.command: "git status -s"
args:
chdir: "{{ _ce_provision_build_tmp_dir }}/extra"
become: true
become_user: "{{ _ce_provision_username }}"
delegate_to: localhost
register: _ce_provision_extra_vars_git_changed

Expand All @@ -30,7 +32,7 @@
args:
chdir: "{{ _ce_provision_build_tmp_dir }}/extra"
become: true
become_user: "{{ ce_provision.username }}"
become_user: "{{ _ce_provision_username }}"
delegate_to: localhost
when:
- ce_provision_extra_vars_computed is defined
Expand All @@ -41,7 +43,7 @@
args:
chdir: "{{ _ce_provision_build_tmp_dir }}/extra"
become: true
become_user: "{{ ce_provision.username }}"
become_user: "{{ _ce_provision_username }}"
delegate_to: localhost
when:
- ce_provision_extra_vars_computed is defined
Expand All @@ -52,7 +54,7 @@
args:
chdir: "{{ _ce_provision_build_tmp_dir }}/extra"
become: true
become_user: "{{ ce_provision.username }}"
become_user: "{{ _ce_provision_username }}"
delegate_to: localhost
when:
- ce_provision_extra_vars_computed is defined
Expand Down
6 changes: 3 additions & 3 deletions roles/_init/tasks/allowed_vars.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,14 @@
version: "{{ _init.ce_provision_extra_repository_branch }}"
delegate_to: localhost
become: true
become_user: "{{ ce_provision.username }}"
become_user: "{{ _ce_provision_username }}"

- name: Check if file exists.
ansible.builtin.stat:
path: "{{ _ce_provision_build_tmp_dir }}/extra/{{ _init.ce_provision_extra_repository_vars_file }}"
delegate_to: localhost
become: true
become_user: "{{ ce_provision.username }}"
become_user: "{{ _ce_provision_username }}"
register: _extra_repository_vars_file

- name: Include custom variables.
Expand All @@ -21,7 +21,7 @@
name: _ce_provision_extra_vars
delegate_to: localhost
become: true
become_user: "{{ ce_provision.username }}"
become_user: "{{ _ce_provision_username }}"
when:
- _extra_repository_vars_file.stat.exists

Expand Down
4 changes: 2 additions & 2 deletions roles/debian/aws_efs_client/tasks/fstab.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
region: "{{ aws_efs_client.region }}"
name: "{{ mount.src }}"
become: true
become_user: "{{ user_provision.username }}"
delegate_to: "localhost"
become_user: "{{ _ce_provision_username }}"
delegate_to: localhost
register: aws_efs_client_efs_volume

- name: Make sure mount point exists.
Expand Down
70 changes: 34 additions & 36 deletions roles/debian/gitlab/tasks/main.yml
Original file line number Diff line number Diff line change
@@ -1,33 +1,32 @@
---
- name: Add a DNS record for GitLab
amazon.aws.route53:
state: "{{ gitlab.gitlab_route_53.state }}"
profile: "{{ gitlab.gitlab_route_53.aws_profile }}"
zone: "{{ gitlab.gitlab_route_53.zone }}"
record: "{{ gitlab.server_name }}"
type: "{{ gitlab.gitlab_route_53.type }}"
value: "{{ gitlab.gitlab_route_53.value }}"
overwrite: true
delegate_to: localhost
become: false
- name: Add DNS records.
when:
- gitlab.gitlab_route_53.zone is defined
- gitlab.gitlab_route_53.zone | length > 0

- name: Add a DNS record for Mattermost
amazon.aws.route53:
state: "{{ gitlab.mattermost_route_53.state }}"
profile: "{{ gitlab.mattermost_route_53.aws_profile }}"
zone: "{{ gitlab.mattermost_route_53.zone }}"
record: "{{ gitlab.mattermost_url }}"
type: CNAME
value: "{{ gitlab.server_name }}"
overwrite: true
delegate_to: localhost
become: false
when:
- gitlab.mattermost_route_53.zone is defined
- gitlab.mattermost_route_53.zone | length > 0
block:
- name: Add a DNS record for GitLab.
amazon.aws.route53:
state: "{{ gitlab.gitlab_route_53.state }}"
profile: "{{ gitlab.gitlab_route_53.aws_profile }}"
zone: "{{ gitlab.gitlab_route_53.zone }}"
record: "{{ gitlab.server_name }}"
type: "{{ gitlab.gitlab_route_53.type }}"
value: "{{ gitlab.gitlab_route_53.value }}"
overwrite: true
delegate_to: localhost
become: false

- name: Add a DNS record for Mattermost.
amazon.aws.route53:
state: "{{ gitlab.mattermost_route_53.state }}"
profile: "{{ gitlab.mattermost_route_53.aws_profile }}"
zone: "{{ gitlab.mattermost_route_53.zone }}"
record: "{{ gitlab.mattermost_url }}"
type: CNAME
value: "{{ gitlab.server_name }}"
overwrite: true
delegate_to: localhost
become: false

- name: Add GitLab repository and key.
ansible.builtin.include_role:
Expand All @@ -50,19 +49,18 @@
enabled: true
key_refresh_timer_OnCalendar: "Mon *-*-* 00:20:00"

- name: Add GitLab repository to unattended-upgrades origins list.
ansible.builtin.set_fact:
_apt_unattended_upgrades_default_origins: "{{ _apt_unattended_upgrades_default_origins + [gitlab.apt_origin] }}"
when:
- apt_unattended_upgrades.enable is defined
- apt_unattended_upgrades.enable

- name: Install the unattended-upgrades config.
ansible.builtin.include_role:
name: debian/apt_unattended_upgrades
- name: Set up unattended-upgrades.
when:
- apt_unattended_upgrades.enable is defined
- apt_unattended_upgrades.enable
block:
- name: Add GitLab repository to unattended-upgrades origins list.
ansible.builtin.set_fact:
_apt_unattended_upgrades_default_origins: "{{ _apt_unattended_upgrades_default_origins + [gitlab.apt_origin] }}"

- name: Install the unattended-upgrades config.
ansible.builtin.include_role:
name: debian/apt_unattended_upgrades

- name: Ensure Gitlab is installed.
ansible.builtin.apt:
Expand Down
17 changes: 8 additions & 9 deletions roles/debian/gitlab_runner/tasks/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,19 +20,18 @@
enabled: true
key_refresh_timer_OnCalendar: "Mon *-*-* 00:30:00"

- name: Add GitLab Runner repository to unattended-upgrades origins list.
ansible.builtin.set_fact:
_apt_unattended_upgrades_default_origins: "{{ _apt_unattended_upgrades_default_origins + [gitlab_runner.apt_origin] }}"
- name: Set up unattended-upgrades.
when:
- apt_unattended_upgrades.enable is defined
- apt_unattended_upgrades.enable
block:
- name: Add GitLab Runner repository to unattended-upgrades origins list.
ansible.builtin.set_fact:
_apt_unattended_upgrades_default_origins: "{{ _apt_unattended_upgrades_default_origins + [gitlab_runner.apt_origin] }}"

- name: Install the unattended-upgrades config.
ansible.builtin.include_role:
name: debian/apt_unattended_upgrades
when:
- apt_unattended_upgrades.enable is defined
- apt_unattended_upgrades.enable
- name: Install the unattended-upgrades config.
ansible.builtin.include_role:
name: debian/apt_unattended_upgrades

- name: Ensure Gitlab Runner is installed.
ansible.builtin.apt:
Expand Down
2 changes: 1 addition & 1 deletion roles/debian/gpg_key/tasks/gpg.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@
- gpg_user.publish
- not user_ansible_gpg_key_exists.stdout

- name: Publish GPG key to openpgp.org.
- name: Publish GPG key to key servers.
ansible.builtin.command:
cmd: "gpg --keyserver {{ server }} --send-keys {{ user_ansible_gpg_key.stdout_lines[1] }}"
become: true
Expand Down
2 changes: 2 additions & 0 deletions roles/debian/ldap_server/tasks/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,8 @@
ansible.builtin.stat:
path: "{{ ldap_server.config.path }}"
register: _ldap_server_ldifs
become_user: "{{ _ce_provision_username }}"
become: true
delegate_to: localhost

- name: Copy LDAP schemas from config repo.
Expand Down
2 changes: 2 additions & 0 deletions roles/debian/wazuh/tasks/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@
- name: Generate certificates.
ansible.builtin.import_role:
name: "contrib/{{ wazuh.path }}/roles/wazuh/wazuh-indexer"
become_user: "{{ _ce_provision_username }}"
become: true
delegate_to: localhost
vars:
indexer_node_master: "{{ wazuh.indexer.indexer_node_master }}"
Expand Down
Loading