Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .config/ansible-lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ warn_list:
- name[casing]
- 'risky-shell-pipe'
- no-handler # backup of old certificates
- var-naming[no-role-prefix] # remove when https://github.com/ansible/ansible-lint/discussions/3451 is done
skip_list:
- '106'
- 'command-instead-of-module'
Expand Down
34 changes: 18 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@

This collection installs and manages the Elastic Stack. It provides roles every component which is part of the Stack. Furthermore it is possible to differentiate between Enterprise or OSS releases. Every role is documented with all variables, please refer to the documentation found in **[Getting-Started](./docs/getting-started.md)**

**Please note**: If you are already using this collection before version `1.0.0`, please note that we had to rename a significant amount of variables due to naming schema changes made by Ansible. Please review the variables you have set in your playbooks and variable files.

## Roles Documentation

* [Beats](docs/role-beats.md)
Expand Down Expand Up @@ -57,9 +59,9 @@ We have known issues with the following Distributions.

## Usage

* *elastic_version*: Version number of tools to install Only set if you don't want the latest. (default: none). For OSS version see `elastic_variant` below. **IMPORTANT** Do not change the version once you have set up the stack. There are unpredictable effects to be expected when using this for upgrades. And upgrade mechanism is already on it's way. (default: none. Example: `7.17.2`
*elastic_release*: Major release version of Elastic stack to configure. (default: `7`)
*elastic_variant*: Variant of the stack to install. Valid values: `elastic` or `oss`. (default: `elastic`)
* *elasticstack_version*: Version number of tools to install Only set if you don't want the latest. (default: none). For OSS version see `elasticstack_variant` below. **IMPORTANT** Do not change the version once you have set up the stack. There are unpredictable effects to be expected when using this for upgrades. And upgrade mechanism is already on it's way. (default: none. Example: `7.17.2`
*elasticstack_release*: Major release version of Elastic stack to configure. (default: `7`)
*elasticstack_variant*: Variant of the stack to install. Valid values: `elastic` or `oss`. (default: `elastic`)

Make sure all hosts that should be configured are part of your playbook. (See below for details on groups etc.). The collection is built to first collect all facts from all hosts (including those only running beats) and then use facts like hostnames or ip addresses to connect the tools to each other.

Expand All @@ -70,7 +72,7 @@ You will want to have reliable DNS resolution or enter all hosts of the stack in
Default Passwords can be seen during generation, or found later in `/usr/share/elasticsearch/initial_passwords`

To turn off security currently:
`elastic_override_beats_tls: true`
`elasticstack_override_beats_tls: true`
### Redis

0) You need to install the redis role which is maintained by geerlingguy.
Expand Down Expand Up @@ -99,8 +101,8 @@ The execution order of the roles is important! (see below)
collections:
- netways.elasticstack
vars:
elastic_variant: elastic #oss
# elastic_release: 8 #7
elasticstack_variant: elastic #oss
# elasticstack_release: 8 #7
roles:
- repos

Expand All @@ -110,9 +112,9 @@ The execution order of the roles is important! (see below)
collections:
- netways.elasticstack
vars:
elastic_variant: elastic #oss
elasticstack_variant: elastic #oss
elasticsearch_jna_workaround: true
# elastic_release: 8 #7
# elasticstack_release: 8 #7
roles:
- elasticsearch

Expand All @@ -122,9 +124,9 @@ The execution order of the roles is important! (see below)
collections:
- netways.elasticstack
vars:
elastic_variant: elastic #oss
elastic_override_beats_tls: true
# elastic_release: 8 #7
elasticstack_variant: elastic #oss
elasticstack_override_beats_tls: true
# elasticstack_release: 8 #7
roles:
- geerlingguy.redis
- logstash
Expand All @@ -135,8 +137,8 @@ The execution order of the roles is important! (see below)
collections:
- netways.elasticstack
vars:
elastic_variant: elastic #oss
# elastic_release: 8 #7
elasticstack_variant: elastic #oss
# elasticstack_release: 8 #7
roles:
- kibana

Expand All @@ -146,9 +148,9 @@ The execution order of the roles is important! (see below)
collections:
- netways.elasticstack
vars:
elastic_variant: elastic #oss
elastic_override_beats_tls: true
# elastic_release: 8 #7
elasticstack_variant: elastic #oss
elasticstack_override_beats_tls: true
# elasticstack_release: 8 #7
pre_tasks:
- name: Install Rsyslog
package:
Expand Down
4 changes: 2 additions & 2 deletions docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,5 +59,5 @@ Variables

Every role got its own set of variables, in addition a few variables are useable on any role. Below are all general collection vars.

* *elastic_release*: Major release version of Elastic stack to configure. (default: `7`)
* *elastic_variant*: Variant of the stack to install. Valid values: `elastic` or `oss`. (default: `elastic`)
* *elasticstack_release*: Major release version of Elastic stack to configure. (default: `7`)
* *elasticstack_variant*: Variant of the stack to install. Valid values: `elastic` or `oss`. (default: `elastic`)
74 changes: 37 additions & 37 deletions docs/role-beats.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,17 +14,17 @@ Role Variables
--------------

* *beats_filebeat*: Install and manage filebeat (Default: `true`)
* *filebeat_enable*: Automatically start Filebeat (Default: `true`)
* *filebeat_output*: Set to `logstash` or `elasticsearch`. (default: `logstash`)
* *filebeat_syslog_udp*: Use UDP Syslog input (Default: `false`)
* *filebeat_syslog_udp_port*: Port of UDP Syslog input (Default: `514`)
* *filebeat_syslog_tcp*: Use TCP Syslog input (Default: `false`)
* *filebeat_syslog_tcp_port*: Port of TCP Syslog input (Default: `514`)
* *filebeat_log_input*: Enable Logfile reading (Default: `true`)
* *filebeat_mysql_slowlog_input*: Enable MySQL/MariaDB slow query log collection incl. multiline (Default: `false`)
* *filebeat_log_inputs*: Logfiles to read (Default: see below)

Default of `filebeat_log_inputs`
* *beats_filebeat_enable*: Automatically start Filebeat (Default: `true`)
* *beats_filebeat_output*: Set to `logstash` or `elasticsearch`. (default: `logstash`)
* *beats_filebeat_syslog_udp*: Use UDP Syslog input (Default: `false`)
* *beats_filebeat_syslog_udp_port*: Port of UDP Syslog input (Default: `514`)
* *beats_filebeat_syslog_tcp*: Use TCP Syslog input (Default: `false`)
* *beats_filebeat_syslog_tcp_port*: Port of TCP Syslog input (Default: `514`)
* *beats_filebeat_log_input*: Enable Logfile reading (Default: `true`)
* *beats_filebeat_mysql_slowlog_input*: Enable MySQL/MariaDB slow query log collection incl. multiline (Default: `false`)
* *beats_filebeat_log_inputs*: Logfiles to read (Default: see below)

Default of `beats_filebeat_log_inputs`

```
messages:
Expand All @@ -37,7 +37,7 @@ You can optionally add `fields` to every input as well. You can also add a `mult

Here's a longer example for an input:
```
filebeat_log_inputs:
beats_filebeat_log_inputs:
messages:
name: messages
paths:
Expand All @@ -50,36 +50,36 @@ filebeat_log_inputs:
negate: false
match: after
```
* *filebeat_journald*: Enable collection of JournalD logs (default: `false`) - available since Filebeat 7.16
* *filebeat_journald_inputs*: List of journald inputs. Use for different filters on events. You can add a list of `include_matches` entries for filtering.
Default of `filebeat_journald_inputs`:
* *beats_filebeat_journald*: Enable collection of JournalD logs (default: `false`) - available since Filebeat 7.16
* *beats_filebeat_journald_inputs*: List of journald inputs. Use for different filters on events. You can add a list of `include_matches` entries for filtering.
Default of `beats_filebeat_journald_inputs`:
```
filebeat_journald_inputs:
beats_filebeat_journald_inputs:
everything:
id: everything
```
* *filebeat_docker*: Enable collection of Docker logs (default: `false`) **ONLY WORKS ON RELEASE 7 SO FAR**
* *filebeat_docker_ids*: IDs of containers to collect. (default: `*`)
* *beats_filebeat_docker*: Enable collection of Docker logs (default: `false`) **ONLY WORKS ON RELEASE 7 SO FAR**
* *beats_filebeat_docker_ids*: IDs of containers to collect. (default: `*`)

* *filebeat_loadbalance*: Enable loadbalancing for Filebeats Logstash output (default: `true`)
* *filebeat_modules*: **EXPERIMENTAL**: Give a list of modules to enable. (default: none)
* *beats_filebeat_loadbalance*: Enable loadbalancing for Filebeats Logstash output (default: `true`)
* *beats_filebeat_modules*: **EXPERIMENTAL**: Give a list of modules to enable. (default: none)

* *beats_auditbeat*: Install and manage filebeat (Default: `false`)
* *auditbeat_output*: Output for Auditbeat Set to `logstash` or `elasticsearch`. (default: `elasticsearch`)
* *auditbeat_enable*: Automatically start Auditbeat (Default: `true`)
* *auditbeat_setup*: Run Auditbeat Setup (Default: `true`) (Only works with Elasticsearch output)
* *auditbeat_loadbalance*: Enable loadbalancing for Auditbeats Logstash output (default: `true`)
* *beats_auditbeat_output*: Output for Auditbeat Set to `logstash` or `elasticsearch`. (default: `elasticsearch`)
* *beats_auditbeat_enable*: Automatically start Auditbeat (Default: `true`)
* *beats_auditbeat_setup*: Run Auditbeat Setup (Default: `true`) (Only works with Elasticsearch output)
* *beats_auditbeat_loadbalance*: Enable loadbalancing for Auditbeats Logstash output (default: `true`)

* *beats_metricbeat*: Enable installation and management of Metricbeat (Default: `false`)
* *metricbeat_enable*: Start Metricbeat automatically (Default: `true`)
* *metricbeat_output*: Set to `logstash` or `elasticsearch`. (default: `elasticsearch`)
* *metricbeat_modules*: List of modules to enable. (Default: `- system`)
* *metricbeat_loadbalance*: Enable loadbalancing for Metricbeats Logstash output (default: `true`)
* *beats_metricbeat_enable*: Start Metricbeat automatically (Default: `true`)
* *beats_metricbeat_output*: Set to `logstash` or `elasticsearch`. (default: `elasticsearch`)
* *beats_metricbeat_modules*: List of modules to enable. (Default: `- system`)
* *beats_metricbeat_loadbalance*: Enable loadbalancing for Metricbeats Logstash output (default: `true`)

* *beats_security*: Activate TLS for connections to targets. Can either be use with our other roles and `elastic_stack_full_stack` to automatically create certificates or `beats_tls*` variables for custom certificates. (default: `false`)
* *beats_security*: Activate TLS for connections to targets. Can either be use with our other roles and `elasticstack_full_stack` to automatically create certificates or `beats_tls*` variables for custom certificates. (default: `false`)
* *beats_target_hosts*: Only use when this role is used standalone. When used in combination with our other roles, the target hosts will be determined automatically. Use a YAML list. (default: `- localhost`)
* *elastic_elasticsearch_http_port*: Port of Elasticsearch to send events to (Default: `9200`)
* *elastic_beats_port*: Port of Logstash to send events to (Default: `5044`)
* *elasticstack_elasticsearch_http_port*: Port of Elasticsearch to send events to (Default: `9200`)
* *elasticstack_beats_port*: Port of Logstash to send events to (Default: `5044`)
* *beats_logging*: Where to log (Default: `file`)
* *beats_loglevel*: Level of logging (for all beats) (Default: `info`)
* *beats_logpath*: If logging to file, where to put logfiles (Default: `/var/log/beats`)
Expand All @@ -88,12 +88,12 @@ filebeat_journald_inputs:

The following variables only apply if you use this role together with our other Elastic Stack roles.

* *elastic_stack_full_stack*: Use `elasticsearch` as well (default: `false`)
* *elastic_variant*: Define which variant of elastic stack to use. (default: `elastic`)
* *elastic_ca_dir*: Directory where on the Elasticsearch CA host certificates are stored. This is only useful in connection with out other Elastic Stack related roles. (default: `/opt/es-ca`)
* *elastic_ca_pass*: Password for Elasticsearch CA (default: `PleaseChangeMe`)
* *elastic_initial_passwords*: Path to file with initical elasticsearch passwords (default: `/usr/share/elasticsearch/initial_passwords`)
* *elastic_version*: Install specific version (Default: none. Possible values: e.g. `7.10.1` or `latest`)
* *elasticstack_full_stack*: Use `elasticsearch` as well (default: `false`)
* *elasticstack_variant*: Define which variant of elastic stack to use. (default: `elastic`)
* *elasticstack_ca_dir*: Directory where on the Elasticsearch CA host certificates are stored. This is only useful in connection with out other Elastic Stack related roles. (default: `/opt/es-ca`)
* *elasticstack_ca_pass*: Password for Elasticsearch CA (default: `PleaseChangeMe`)
* *elasticstack_initial_passwords*: Path to file with initical elasticsearch passwords (default: `/usr/share/elasticsearch/initial_passwords`)
* *elasticstack_version*: Install specific version (Default: none. Possible values: e.g. `7.10.1` or `latest`)

If you want to use this role with your own TLS certificates, use these variables.

Expand Down
16 changes: 8 additions & 8 deletions docs/role-elasticsearch.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,6 @@ Role Variables

* *elasticsearch_enable*: Start and enable Elasticsearch (default: `true`)
* *elasticsearch_heap*: Heapsize for Elasticsearch. (Half of free memory on host. Maximum 30GB. (default: Half of hosts memory. Min 1GB, Max 30GB)
* *elasticsearch_ca*: Set to the inventory hostname of the host that should house the CA for certificates for inter-node communication. (default: First node in the `elasticsearch` host group)
* *elastic_ca_pass*: Password for Elasticsearch CA (default: `PleaseChangeMe`)
* *elastic_ca_expiration_buffer*: Ansible will renew the CA if its validity is shorter than this value, which should be number of days. (default: 30)
* *elastic_ca_will_expire_soon*: Set it to true to renew the CA and the certificate of all Elastic Stack components (default: `false`), Or run the playbook with `--tags renew_ca` to do that.
* *elasticsearch_tls_key_passphrase*: Passphrase for elasticsearch certificates (default: `PleaseChangeMeIndividually`)
* *elasticsearch_cert_expiration_buffer*: Ansible will renew the elasticsearch certificate if its validity is shorter than this value, which should be number of days. (default: 30)
* *elasticsearch_cert_will_expire_soon*: Set it to true to renew elasticsearch certificate (default: `false`), Or run the playbook with `--tags renew_elasticsearch_cert` to do that.
Expand All @@ -36,17 +32,21 @@ This variable activates a workaround to start on systems that have certain harde

These variables are identical over all our elastic related roles, hence the different naming schemes.

* *elastic_release*: Major release version of Elastic stack to configure. (default: `7`)
* *elastic_variant*: Variant of the stack to install. Valid values: `elastic` or `oss`. (default: `elastic`)
* *elastic_elasticsearch_http_port*: Port of Elasticsearch http (Default: `9200`)
* *elasticstack_ca*: Set to the inventory hostname of the host that should house the CA for certificates for inter-node communication. (default: First node in the `elasticsearch` host group)
* *elasticstack_ca_pass*: Password for Elasticsearch CA (default: `PleaseChangeMe`)
* *elasticstack_ca_expiration_buffer*: Ansible will renew the CA if its validity is shorter than this value, which should be number of days. (default: 30)
* *elasticstack_ca_will_expire_soon*: Set it to true to renew the CA and the certificate of all Elastic Stack components (default: `false`), Or run the playbook with `--tags renew_ca` to do that.
* *elasticstack_release*: Major release version of Elastic stack to configure. (default: `7`)
* *elasticstack_variant*: Variant of the stack to install. Valid values: `elastic` or `oss`. (default: `elastic`)
* *elasticstack_elasticsearch_http_port*: Port of Elasticsearch http (Default: `9200`)

```
- name: Install Elasticsearch
collections:
- netways.elasticstack
hosts: elasticsearch-hosts
vars:
elastic_variant: oss
elasticstack_variant: oss
elasticsearch_jna_workaround: true
elasticsearch_disable_systemcallfilterchecks: true
roles:
Expand Down
26 changes: 13 additions & 13 deletions docs/role-kibana.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,23 +16,23 @@ Role Variables
* *kibana_tls_cert*: Path to the certificate Kibana should show to its clients (default: `/etc/kibana/certs/cert.pem`)
* *kibana_tls_key*: Path to the key Kibana should use when communicating with clients (default: `/etc/kibana/certs/key.pem`)

* *kibana_security*: Activate TLS and authentication when connecting to Elasticsearch. **Note**: Only works when `elastic_stack_full_stack` is enabled. (default: `true`)
* *kibana_security*: Activate TLS and authentication when connecting to Elasticsearch. **Note**: Only works when `elasticstack_full_stack` is enabled. (default: `true`)

These variables are identical over all our elastic related roles, hence the different naming scheme.

* *elastic_stack_full_stack*: Use `ansible-role-elasticsearch` as well (default: `false`)
* *elastic_elasticsearch_http_port*: Port of Elasticsearch http (Default: `9200`)
* *elasticstack_full_stack*: Use `ansible-role-elasticsearch` as well (default: `false`)
* *elasticstack_elasticsearch_http_port*: Port of Elasticsearch http (Default: `9200`)
* *kibana_tls_key_passphrase*: Passphrase for kibana certificates (default: `PleaseChangeMe`)
* *kibana_cert_expiration_buffer*: Ansible will renew the kibana certificate if its validity is shorter than this value, which should be number of days. (default: 30)
* *kibana_cert_will_expire_soon*: Set it to true to renew kibana certificate (default: `false`), Or run the playbook with `--tags renew_kibana_cert` to do that.
* *elastic_kibana_host*: Hostname users use to connect to Kibana (default: FQDN of the host the role is executed on)
* *elastic_kibana_port*: Port Kibana webinterface is listening on (default: `5601`)
* *elasticsearch_ca*: Set to the inventory hostname of the host that should house the CA for certificates for inter-node communication. (default: First node in the `elasticsearch` host group)
* *elastic_ca_dir*: Directory where on the Elasticsearch CA host certificates are stored. This is only useful in connection with out other Elastic Stack related roles. (default: `/opt/es-ca`)
* *elastic_ca_pass*: Password for Elasticsearch CA (default: `PleaseChangeMe`)
* *elastic_initial_passwords*: Path to file with initical elasticsearch passwords (default: `/usr/share/elasticsearch/initial_passwords`)
* *elastic_release*: Major release version of Elastic stack to configure. (default: `7`)
* *elastic_variant*: Variant of the stack to install. Valid values: `elastic` or `oss`. (default: `elastic`)
* *elasticstack_kibana_host*: Hostname users use to connect to Kibana (default: FQDN of the host the role is executed on)
* *elasticstack_kibana_port*: Port Kibana webinterface is listening on (default: `5601`)
* *elasticstack_ca*: Set to the inventory hostname of the host that should house the CA for certificates for inter-node communication. (default: First node in the `elasticsearch` host group)
* *elasticstack_ca_dir*: Directory where on the Elasticsearch CA host certificates are stored. This is only useful in connection with out other Elastic Stack related roles. (default: `/opt/es-ca`)
* *elasticstack_ca_pass*: Password for Elasticsearch CA (default: `PleaseChangeMe`)
* *elasticstack_initial_passwords*: Path to file with initical elasticsearch passwords (default: `/usr/share/elasticsearch/initial_passwords`)
* *elasticstack_release*: Major release version of Elastic stack to configure. (default: `7`)
* *elasticstack_variant*: Variant of the stack to install. Valid values: `elastic` or `oss`. (default: `elastic`)

If you use `localhost` in `kibana_elasticsearch_hosts` , certificate verification will skip hostname checks

Expand All @@ -44,8 +44,8 @@ If you use `localhost` in `kibana_elasticsearch_hosts` , certificate verificatio
- netways.elasticstack
hosts: kibana-host
vars:
elastic_stack_full_stack: true
elastic_variant: oss
elasticstack_full_stack: true
elasticstack_variant: oss
roles:
- repos
- kibana
Expand Down
Loading