[Request]: Create new inventory docs and remove docs for new service inventory

[roshan-elastic]

Description

We will be building a new inventory that will eventually supersede the existing inventory that will show all entities in one place (see PRD).

The first iteration will only show Hosts, Containers and Services but it will quickly start showing other entity types (next up is Kubernetes) - eventually all entities (e.g. S3, RDS, Kafka, user-provided entities).

Additionally, we will be removing the 'new service inventory' that we started and promoting the new inventory within the existing services inventory - the new inventory will also include services so this effectively envelopes the new services inventory.

This issue is to:

(a) Create documentation explaining what this feature is
(b) Remove the new services inventory documentation

Note : The changes in the product will be made in serverless on 7th Oct

Resources

• New Services Inventory docs
• Epic for new Inventory
• New Inventory Designs
• PRD for new Inventory

The main pitch for the new inventory is that you can "See everything you have in one place". It will help users do two things during an incident:

1. Find their stuff : Find whatever it is they're investigating in one place
2. See what needs attention : See what isn't working as expected so they can begin root cause analysis

Which documentation set does this change impact?

Stateful and Serverless

Feature differences

Identical in stateful and serverless (although the new service inventory wasn't live in stateful yet)

What release is this request related to?

8.16

Collaboration model

The documentation team

Point of contact.

Main contact: @roshan-elastic

Stakeholders: @kpatticha @kkurstak

[roshan-elastic]

Key things

What is it?

Call out - It relies on Elastic Entity Model (make sure they can easily find the docs on this)

• 1. Find all of your stuff : A place to find everything you are observing in one place...even with only logs
• 2. See what needs attention : so you can investigate problems (not available yet...not sure if you need to say this yet)

Limitations

• Only hosts, containers and services
• No metrics
• No alerts

Why would you use it?

• Find any* entity in a single place...so you can troubleshoot their problems (even with just logs)

*Limited to Hosts, Containers and Services

How do you use it?

• Search* for your entity

*Let them know they are querying entities (i.e. entities-*-latest - this will be an ad hoc data view). They can look up the data per entity in Discover.

• View an entity (click on an entity)
• Observe more entities (see below)

How do you add entities?

Key Onboarding Journeys

• Auto-detect logs and metrics (https://YOURCLUSTERURL/app/observabilityOnboarding/?category=logs):
  • Detects Hosts (with metrics & logs)
• Kubernetes (https://YOURCLUSTERURL/app/observabilityOnboarding/?category=infra)
  • Detects Hosts, Containers and Services* (https://YOURCLUSTERURL/app/observabilityOnboarding/?category=apm)
• Elastic APM / OpenTelemetry / Synthetic Monitor
  • Detects Services

*The Kubernetes integration will set service.name for users automatically based on this logic - this isn't documented anywhere...I would probably steer clear of documenting this

Important Information

Users will need to understand the definitions for how each entity is detected (this could probably be in the EEM doc)

Hosts

• Anything where host.name is set in a metrics-* or `logs-* index

Tip! Elastic Agent, FileBeat and MetricBeat typically set host.name in every log line so users using these will find a lot of hosts

Containers

• Anything where container.id is set in a metrics-* or `logs-* index

Services

• Anything where service.name is set in a apm-* or `logs-* index

Tip! Setting service.name in logs is a new thing that we're promoting directly from the UI - it's a key benefit (doc)

Note : We will eventually surface the entity definitions in the UI so we hopefully don't need to maintain these definitions in the docs. For now, the docs are the only places where this is defined (and they'll likely change). It might be worth having a specific page where entity definitions are stored in the docs (with a section per entity type)?

[roshan-elastic]

@alaudazzi - updated!

[alaudazzi]

Thank you @roshan-elastic
Additional notes from our chat this morning:

• @alaudazzi will share a gdoc with an initial draft based on the structure discussed with @roshan-elastic (see above)
• Deadline remains Oct 7, unless otherwise stated
• @kpatticha might help with access to the latest UI
• the new doc page on Inventory will be located after Log monitoring
  

[alaudazzi]

@roshan-elastic
Here is an initial draft of the new Inventory doc page.
It's very much WIP, but it gives us a baseline to work on.
I've added a few questions/comments where your feedback is appreciated.

CC @kpatticha

[alaudazzi]

@roshan-elastic

PRs are ready for your review:

• Create new Inventory documentation => [Inventory] Create new Inventory docs #4341
• Remove the new services inventory documentation => [Inventory] Remove the new services inventory docs #4342

[alaudazzi]

Close as completed.