[Infra UI] Show non-metric details on node detail page

[skh]

We have been asked to show more information than metrics on the node detail page:

• AWS tags (if applicable) -- see discussion in [Infra UI] Display cloud metrics on the node detail page #39282 (comment)
• Cloud Instance Id -- as alternative solution to [Infra UI] Make the displayed node title configurable #41449

We currently have this type of information available (from the metadata endpoint), this could possibly be extended if more information is desired:

"info": {
        "cloud": {
            "account": {
                "id": "015351775590"
            },
            "availability_zone": "us-east-2c",
            "image": {
                "id": "ami-0d8f6eb4f641ef691"
            },
            "instance": {
                "id": "i-011454f72559c510b"
            },
            "machine": {
                "type": "t2.micro"
            },
            "provider": "aws",
            "region": "us-east-2"
        },
        "host": {
            "architecture": "x86_64",
            "containerized": false,
            "hostname": "ip-172-31-47-9.us-east-2.compute.internal",
            "id": "ded64cbff86f478990a3dfbb63a8d238",
            "name": "ip-172-31-47-9.us-east-2.compute.internal",
            "os": {
                "codename": "Karoo",
                "family": "redhat",
                "kernel": "4.14.123-111.109.amzn2.x86_64",
                "name": "Amazon Linux",
                "platform": "amzn",
                "version": "2"
            }
        }
    }

Currently, the node detail page only shows metrics:

A possible source of inspiration could be the top part of the Host information page in the SIEM app:

[elasticmachine]

Pinging @elastic/infra-logs-ui

[skh]

@hbharding @sorantis Your input here would be most welcome!

[sorantis]

@skh, I think the SIEM's Hosts page for hosts looks good for our purpose and hopefully will be easy to reuse. As a bonus we'll gain some consistency across the apps and apply to other cloud providers.
One thing I'd try is to have this host info part of the page expandable/collapsible in case users won't find this information useful all the time. E.g.

Collapsed:

Expanded:

Thoughts?

Would be great to also have a similar overview page for other entities - Kubernetes, Docker, Host, but that's a different story :)

[sorantis]

[ALT2]
Collapsed view:

[skh]

@sorantis I like ALT2 better as it shows the most important additional information even in the collapsed state, and it's clearer that there is something to uncollapse.

Would be great to also have a similar overview page for other entities - Kubernetes, Docker, Host, but that's a different story :)

This element would be on every node detail page, so we already have meta information for every node type. The easiest way to find out which data is already there is to open the network tab in the dev console in the browser, find the call to metadata, and look in the response what is returned. This is how that looks in Chrome, other browsers should have something similar:

Depending on which type of node you're looking at, you'll see slightly different information there. If something is missing, please say so, it might be trivial to add.

[sorantis]

I like ALT2 better as it shows the most important additional information even in the collapsed state, and it's clearer that there is something to uncollapse.

+1

The metadata looks good. Will host tags become part of this metadata? Also, would be great to distinguish between system tags and user tags in UI.

[simianhacker]

@skh @sorantis This would probably be a good time to review all the metadata values we are returning. We should make sure we can support everything , what we have now is based on what we see in the observability cluster.

[sorantis]

@skh looking at my setup with only one host - my machine, can't find metadata in the list:

Do we have metadata for physical hosts? Or you're adding it right now :)

@simianhacker agreed. Where and how is metadata object assembled? @skh mentioned that we collect it from the metadata endpoint. Which endpoint is that?

Generally, isn't it so that we'd need to revisit metadata once in a while, since we're gradually adding new metricsets to the aws module? Right now the metadata information for ec2 hosts looks good. What happens when the user creates, for example a database, which is a separate service? In the end it's still a host, but will the metadata object be different? Is there a way to distinguish between instantiated services?

cc @kaiyan-sheng.

[skh]

@sorantis You may need to refresh the page with the dev tools open to see the request to the metadata endpoint. It is called for every node type when the node details page is opened (== when you click on "view metrics" in the inventory view).

[skh]

Generally, isn't it so that we'd need to revisit metadata once in a while, since we're gradually adding new metricsets to the aws module?

Yes, whenever we want to show more data in the UI, we need to adapt our own API endpoints. This may happen when metricbeat gains more functionality, or just when we decide to show more data than before. When new metricsets are added they show up in the response from metadata automatically, but other types of metadata (like AWS tags) won't.

(Please note that these endpoints are part of the Infrastructure app in Kibana. They are executed on the Kibana server (the nodejs part of Kibana) and are responsible to put together the correct elasticsearch queries, which then grab the data from Elasticsearch that has been put there by metricbeat.)

[sorantis]

You may need to refresh the page with the dev tools open to see the request to the metadata endpoint.

Of course. I did that too. I just filtered the results on my screenshot to make sure I'm not missing any metadata.

UPDATE: Looks like a version mismatch. metadata became a standalone field after 7.3.

#notetoself, always specify the Kibana version.

[skh]

I forgot to mention, metadata has been added recently and will only be visible if you run master or 7.x. Apologies!

In earlier versions, similar information is returned by one of the calls to the graphql endpoint, but it will not include the info part in the response.

[hbharding]

Hi there. I think the mockups @sorantis provided are fine. I've cleaned them up a little and used the proper size for the EuiButtonIcon.

The only thing I'm unable to wrap my head around is what data do we show by default in the contracted state, and in what order? I imagine each node type could have its own unique metadata that doesn't apply to other node types. If possible, it'd be nice if in the contracted state we could always show the same key/values for consistency.

Figma link: https://www.figma.com/file/NIsuFRMURBKr9S292etQ4J/42689-Host-Metadata?node-id=0%3A1

[skh]

The only thing I'm unable to wrap my head around is what data do we show by default in the contracted state, and in what order? I imagine each node type could have its own unique metadata that doesn't apply to other node types. If possible, it'd be nice if in the contracted state we could always show the same key/values for consistency.

In some situations we use different fields for a node "name" and a node "id". The id fields are configurable, their defaults are in https://github.com/elastic/kibana/blob/master/x-pack/legacy/plugins/infra/server/lib/sources/defaults.ts, the name fields come from https://github.com/elastic/kibana/blob/master/x-pack/legacy/plugins/infra/server/lib/constants.ts .

By default, the configured id fields are: host.name, container.id and kubernetes.pod.uid.
The hardcoded name fields are: host.name, kubernetes.pod.name, container.name.

If they are different, we use whatever is in the "name" field as the page title, and in that case I think it would make sense to also show whatever is in the "id" field in either the contracted or the extended state. This is admittedly a bit messy because it only applies to containers and pods as long as the configuration hasn't been touched. Also, the metadata endpoint doesn't seem to return this data right now, but only what is in the "name" field.

---

My proposal for a first implementation would therefore be the following. All field names refer to the info object from the response from the metadata endpoint (the useMetadata hook will have to be enhanced to pick them out):

Candidates for the contracted state:

• cloud instance id. So many things run in the cloud today, and we've been explicitly asked to show this (see [Infra UI] Make the displayed node title configurable #41449) : cloud.instance.id
• cloud provider: cloud.provider
• operating system: host.os.name
• kernel version: host.os.kernel

In the expanded state:

• host.name
• host.hostname (for the difference between host.name and host.hostname see https://www.elastic.co/guide/en/ecs/current/ecs-host.html)
• host.architecture
• host.containerized
• cloud.project.id
• cloud.availability_zone
• cloud.machine.type
• cloud.instance.name (not sure about this one but we have it)

Candidates to be added to the metadata endpoint and displayed in a followup PR:

• IP addresses. These can be an array, so maybe there's not enough room for them in the contracted state, but I think it's important information. Once we have them I think they could go into the contracted state.
• Anything that comes out of solution to the "id field" vs. "name field" issue explained above

@sorantis @hbharding what do you think?

[sorantis]

@skh I'd consider swapping host.os.kernel with cloud.availability_zone in the contracted state. host.os.* can entirely go into the expanded state.

+1 on adding IP addresses. Are we able to pull AWS tags into the metadata object and make it visible in the expanded state?