-
Notifications
You must be signed in to change notification settings - Fork 69
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
adding docs for default container in 8.9 #386
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -81,7 +81,7 @@ and avoid placing passwords in plain text. | |
|
||
The stream to use for logs collection, for example, stdout/stderr. | ||
|
||
If the specified package has no logs support, a generic container's logs input will be used as a fallback. | ||
If the specified package has no logs support, a generic container's logs input will be used as a fallback. See the `Hints autodiscovery for kubernetes log collection` example below. | ||
|
||
[discrete] | ||
== Available packages that support hints autodiscovery | ||
|
@@ -92,7 +92,7 @@ https://github.com/elastic/elastic-agent/tree/main/deploy/kubernetes/elastic-age | |
[discrete] | ||
== Configure hints autodiscovery | ||
|
||
To enable hints, you must add `hints.enabled: true` to the provider's configuration: | ||
To enable hints autodiscovery, you must add `hints.enabled: true` to the provider's configuration: | ||
|
||
[source,yaml] | ||
---- | ||
|
@@ -134,8 +134,15 @@ initContainers: | |
mountPath: /etc/elastic-agent/inputs.d | ||
---- | ||
|
||
|
||
NOTE: The {agent} can load multiple configuration files from `{path.config}/inputs.d` and finally produce a unified one (refer to <<elastic-agent-configuration>>). Users have the ability to manually mount their own templates under `/etc/elastic-agent/inputs.d` *if they want to skip enabling initContainers section*. | ||
|
||
|
||
[discrete] | ||
== Examples: | ||
|
||
[discrete] | ||
== Example: Hints autodiscovery | ||
=== Hints autodiscovery for redis | ||
|
||
Enabling hints allows users deploying Pods on the cluster to automatically turn on Elastic | ||
monitoring at Pod deployment time. | ||
|
@@ -164,6 +171,115 @@ After deploying this Pod, the data will start flowing in automatically. You can | |
NOTE: All assets (dashboards, ingest pipelines, and so on) related to the Redis integration are not installed. You need to explicitly <<install-uninstall-integration-assets,install them through {kib}>>. | ||
|
||
|
||
[discrete] | ||
=== Hints autodiscovery for kubernetes log collection | ||
|
||
The log collection for Kubernetes autodiscovered pods can be supported by using https://github.com/elastic/elastic-agent/tree/main/deploy/kubernetes/elastic-agent-standalone/templates.d/container_logs.yml[container_logs.yml template]. Elastic Agent needs to emit a container_logs mapping so as to start collecting logs for all the discovered containers *even if no annotations are present in the containers*. | ||
|
||
1. Follow steps described above to enable Hints Autodiscover | ||
2. Make sure that relevant `container_logs.yml` template will be mounted under /etc/elastic-agent/inputs.d/ folder of Elastic Agent | ||
3. Deploy Elastic Agent Manifest | ||
4. Elastic Agent should be able to discover all containers inside kuernetes cluster and to collect available logs. | ||
|
||
The previous default behaviour can be disabled with `hints.default_container_logs: false`. | ||
So this will disable the automatic logs collection from all discovered pods. Users need specifically to annotate their pod with following annotations: | ||
|
||
[source,yaml] | ||
---- | ||
annotations: | ||
co.elastic.hints/package: "container_logs" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Just a note that |
||
---- | ||
|
||
|
||
[source,yaml] | ||
---- | ||
providers.kubernetes: | ||
node: ${NODE_NAME} | ||
scope: node | ||
hints: | ||
enabled: true | ||
default_container_logs: false | ||
... | ||
---- | ||
|
||
In the following sample nginx manifest, we will additionally provide specific stream annotation, in order to configure the filestream input to read only stderr stream: | ||
|
||
[source,yaml] | ||
---- | ||
apiVersion: apps/v1 | ||
kind: Deployment | ||
metadata: | ||
labels: | ||
app: nginx | ||
name: nginx | ||
namespace: default | ||
spec: | ||
selector: | ||
matchLabels: | ||
app: nginx | ||
template: | ||
metadata: | ||
labels: | ||
app: nginx | ||
annotations: | ||
co.elastic.hints/package: "container_logs" | ||
co.elastic.hints/stream: "stderr" | ||
spec: | ||
containers: | ||
- image: nginx | ||
name: nginx | ||
... | ||
---- | ||
|
||
Users can monitor the final rendered Elastic Agent configuration: | ||
|
||
[source,bash] | ||
---- | ||
kubectl exec -ti -n kube-system elastic-agent-7fkzm -- bash | ||
|
||
|
||
/usr/share/elastic-agent# /elastic-agent inspect -v --variables --variables-wait 2s | ||
|
||
inputs: | ||
- data_stream.namespace: default | ||
id: hints-container-logs-3f69573a1af05c475857c1d0f98fc55aa01b5650f146d61e9653a966cd50bd9c-kubernetes-1780aca0-3741-4c8c-aced-b9776ba3fa81.nginx | ||
name: filestream-generic | ||
original_id: hints-container-logs-3f69573a1af05c475857c1d0f98fc55aa01b5650f146d61e9653a966cd50bd9c | ||
[output truncated ....] | ||
streams: | ||
- data_stream: | ||
dataset: kubernetes.container_logs | ||
type: logs | ||
exclude_files: [] | ||
exclude_lines: [] | ||
parsers: | ||
- container: | ||
format: auto | ||
stream: stderr | ||
paths: | ||
- /var/log/containers/*3f69573a1af05c475857c1d0f98fc55aa01b5650f146d61e9653a966cd50bd9c.log | ||
prospector: | ||
scanner: | ||
symlinks: true | ||
tags: [] | ||
type: filestream | ||
use_output: default | ||
outputs: | ||
default: | ||
hosts: | ||
- https://elasticsearch:9200 | ||
password: changeme | ||
type: elasticsearch | ||
username: elastic | ||
providers: | ||
kubernetes: | ||
hints: | ||
default_container_logs: false | ||
enabled: true | ||
node: control-plane | ||
scope: node | ||
---- | ||
|
||
|
||
[discrete] | ||
== Troubleshooting | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That section does not exist in https://github.com/elastic/ingest-docs/pull/304/files. Is this coming from another PR? If so we should mention it as well in the description.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated the desc, it was the #380