Skip to content

Commit d053a9f

Browse files
AlexanderWertChrsMark
AlexanderWert
and
ChrsMark
authored
Initial version of the OpenTelemetry System content pack (#14400)
* Initial version of the OpenTelemetry System content pack Signed-off-by: Alexander Wert <alexander.wert@elastic.co> * added codeowner Signed-off-by: Alexander Wert <alexander.wert@elastic.co> * fix dataview problem Signed-off-by: Alexander Wert <alexander.wert@elastic.co> * fix hosts filtering in logs view Signed-off-by: Alexander Wert <alexander.wert@elastic.co> * Update packages/system_otel/changelog.yml * Update packages/system_otel/docs/README.md Co-authored-by: Christos Markou <chrismarkou92@gmail.com> * Update packages/system_otel/docs/README.md Co-authored-by: Christos Markou <chrismarkou92@gmail.com> * fixed some review comments Signed-off-by: Alexander Wert <alexander.wert@elastic.co> * updated readme Signed-off-by: Alexander Wert <alexander.wert@elastic.co> * fix that pods are shown in hosts overview Signed-off-by: Alexander Wert <alexander.wert@elastic.co> * Update packages/system_otel/changelog.yml Co-authored-by: Christos Markou <chrismarkou92@gmail.com> * fix CPU utilization label Signed-off-by: Alexander Wert <alexander.wert@elastic.co> * fixed bug with filter Signed-off-by: Alexander Wert <alexander.wert@elastic.co> * Update packages/system_otel/docs/README.md Co-authored-by: Christos Markou <chrismarkou92@gmail.com> * Update packages/system_otel/docs/README.md Co-authored-by: Christos Markou <chrismarkou92@gmail.com> * Update packages/system_otel/docs/README.md Co-authored-by: Christos Markou <chrismarkou92@gmail.com> * Update packages/system_otel/docs/README.md Co-authored-by: Christos Markou <chrismarkou92@gmail.com> * Update packages/system_otel/docs/README.md Co-authored-by: Christos Markou <chrismarkou92@gmail.com> * Update packages/system_otel/docs/README.md Co-authored-by: Christos Markou <chrismarkou92@gmail.com> * Update packages/system_otel/docs/README.md Co-authored-by: Christos Markou <chrismarkou92@gmail.com> * Update packages/system_otel/docs/README.md Co-authored-by: Christos Markou <chrismarkou92@gmail.com> * Update packages/system_otel/docs/README.md Co-authored-by: Christos Markou <chrismarkou92@gmail.com> * Update packages/system_otel/docs/README.md Co-authored-by: Christos Markou <chrismarkou92@gmail.com> * Update packages/system_otel/docs/README.md Co-authored-by: Christos Markou <chrismarkou92@gmail.com> * removed cloud.availability_zone attributes from metadata Signed-off-by: Alexander Wert <alexander.wert@elastic.co> --------- Signed-off-by: Alexander Wert <alexander.wert@elastic.co> Co-authored-by: Christos Markou <chrismarkou92@gmail.com>
1 parent 4154e72 commit d053a9f

14 files changed

+8038
-0
lines changed

‎.github/CODEOWNERS‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -419,6 +419,7 @@
419419
/packages/system/data_stream/uptime @elastic/obs-infraobs-integrations
420420
/packages/system/manifest.yml @elastic/obs-infraobs-integrations @elastic/sec-linux-platform @elastic/sec-windows-platform
421421
/packages/system_audit @elastic/sec-linux-platform
422+
/packages/system_otel @elastic/ingest-otel-data
422423
/packages/tanium @elastic/security-service-integrations
423424
/packages/tcp @elastic/sec-deployment-and-devices
424425
/packages/teleport @elastic/security-service-integrations

‎packages/system_otel/changelog.yml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
# newer versions go on top
2+
- version: "0.1.0"
3+
changes:
4+
- description: Initial version of the System OpenTelemetry Assets content pack
5+
type: enhancement
6+
link: https://github.com/elastic/integrations/pull/14400 # FIXME Replace with the real PR link

‎packages/system_otel/docs/README.md‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
# System OpenTelemetry Assets
2+
3+
System OpenTelemetry Assets provides dashboards for vizualising OpenTelemetry hosts' metrics and logs.
4+
5+
## Requirements
6+
7+
Collect and ingest OpenTelemetry data from the Collector's [`hostmetrics` receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/v0.127.0/receiver/hostmetricsreceiver) through:
8+
9+
- the [Elastic Distributions of OpenTelemetry](https://www.elastic.co/docs/reference/opentelemetry/quickstart/)
10+
- or using the vanilla / upstream OpenTelemetry Collector
11+
12+
Compatible `hostmetrics` receiver versions are all versions `>= v0.102.0`.
13+
14+
For full functionality of the dashboards included in this content pack, you will need to ensure the following metrics are enabled in the [`hostmetrics` receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/v0.127.0/receiver/hostmetricsreceiver):
15+
16+
| Metric | Enabled by default in EDOT Collector | Enabled by default in upstream Contrib Collector |
17+
|---|---|---|
18+
|**[CPU](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/v0.127.0/receiver/hostmetricsreceiver/internal/scraper/cpuscraper/documentation.md)**|||
19+
| `system.cpu.time` | ✅ | ✅ |
20+
| `system.cpu.utilization` | ✅ | ❌ |
21+
| `system.cpu.logical.count` | ✅ | ❌ |
22+
|**[Load](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/v0.127.0/receiver/hostmetricsreceiver/internal/scraper/loadscraper/documentation.md)**|||
23+
| `system.cpu.load_average.1m` | ✅ | ✅ |
24+
| `system.cpu.load_average.5m` | ✅ | ✅ |
25+
| `system.cpu.load_average.15m` | ✅ | ✅ |
26+
|**[Memory](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/v0.127.0/receiver/hostmetricsreceiver/internal/scraper/memoryscraper/documentation.md)**|||
27+
| `system.memory.usage` | ✅ | ✅ |
28+
| `system.memory.utilization` | ✅ | ❌ |
29+
|**[Network](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/v0.127.0/receiver/hostmetricsreceiver/internal/scraper/networkscraper/documentation.md)**|||
30+
| `system.network.connections` | ✅ | ✅ |
31+
| `system.network.dropped` | ✅ | ✅ |
32+
| `system.network.errors` | ✅ | ✅ |
33+
| `system.network.io` | ✅ | ✅ |
34+
| `system.network.packets` | ✅ | ✅ |
35+
|**[Disk](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/v0.127.0/receiver/hostmetricsreceiver/internal/scraper/diskscraper/documentation.md)**|||
36+
| `system.disk.io` | ✅ | ✅ |
37+
| `system.disk.io_time` | ✅ | ✅ |
38+
| `system.disk.operations` | ✅ | ✅ |
39+
|**[File System](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/v0.127.0/receiver/hostmetricsreceiver/internal/scraper/filesystemscraper/documentation.md)**|||
40+
| `system.filesystem.usage` | ✅ | ✅ |
41+
|**[Processes](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/v0.127.0/receiver/hostmetricsreceiver/internal/scraper/processesscraper/documentation.md)**|||
42+
| `system.processes.count` | ✅ | ✅ |
43+
| `system.processes.created` | ✅ | ✅ |
44+
45+
For step-by-step instructions on how to ingest OpenTelemetry data using Elastic's distribution of the OpenTelemetry Collector, see the
46+
[quickstart guide](https://www.elastic.co/docs/reference/opentelemetry/quickstart/).
47+
48+
Also, it's recommended to enable the [`resourcedetection` processor](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/v0.127.0/processor/resourcedetectionprocessor/README.md) with detection of the following resource attributes explicitly enabled:
49+
50+
- `host.name`
51+
- `host.id`
52+
- `host.arch`
53+
- `host.ip`
54+
- `host.mac`
55+
- `host.cpu.vendor.id`
56+
- `host.cpu.family`
57+
- `host.cpu.model.id`
58+
- `host.cpu.model.name`
59+
- `host.cpu.stepping`
60+
- `host.cpu.cache.l2.size`
61+
- `os.description`
62+
- `os.type`
63+
64+
## Troubleshooting
65+
66+
If individual widgets in the dashboard show errors that certain fields are not evailable, that might be an indicator of one of the following:
67+
68+
- For your use case the missing data is not relevant (e.g. you are only running plain, local VMs, no `Cloud` metadata will be available)
69+
- You are using the OpenTelemetry upstream Contrib Collector (or any other, non-EDOT Collector) and some of the above-mentioned requirements are not met
70+
71+
See also:
72+
73+
- [Scraper limitations](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/v0.127.0/receiver/hostmetricsreceiver#host-metrics-receiver) with the `hostmetrics` receiver for certain systems
74+
- [Related EDOT Collector limitations](https://www.elastic.co/docs/reference/opentelemetry/compatibility/limitations#infrastructure-and-host-metrics) for host metrics

‎packages/system_otel/img/host-details-metadata.png‎

314 KB
Loading

‎packages/system_otel/img/host-details-overview.png‎

777 KB
Loading

‎packages/system_otel/img/logs-view.png‎

473 KB
Loading

‎packages/system_otel/img/metrics.png‎

424 KB
Loading

‎packages/system_otel/img/system-otel.svg‎

Lines changed: 7 additions & 0 deletions
Loading

0 commit comments

Comments
 (0)