[APM] Deployment annotations: Show service.version annotations and filter

[formgeist]

Related:

• Proposal
• Product brief
• APM UI issue

Summary

Observability UIs exist in large part to assist users in investigating specific issues with their applications. Overlaying important events in graphs, dashboards, log views, etc furthers the usefulness of these UIs in those investigations. (from the proposal doc)

Summary of the problem
I want to know if a bad deploy is causing production downtime or outages. I’d like to quickly identify if a deployment results in performance degradation that might potentially impact our end users.”

I want to tag my deployments, so I can easily discern if my deploy has increased the response times for an end-user or if the memory/CPU footprint of the application has gone up. A visibility into my deployments will help me identify bad deploys and will allow me to rollback the deployment if need be and fix the issues without causing outages.

Ideal solution
For example: the ideal solution would provide an availability summary and detailed status information of my site.

List known (technical) restrictions and requirements

• APM chart annotations with deployment information so it is easy to identify when a new deploy happened.

Design solution

Figma prototype

---

Annotations for service.version

For all time-series charts in APM, we will automatically display annotations for service.version field. More importantly, we'd need to timestamp when we find a version bump in order to annotate.

Show service.version filter in UI Filters

We want to enable users to filter by service.version in the UI, so we're adding the service.version filter to the Transactions overview (for supported agents).

[formgeist]

@nehaduggal will you help me fill out the design solution section in the Summary as part of the design issue template?

[nehaduggal]

Done.

[formgeist]

@nehaduggal @elastic/apm-ui Based on our sync on Monday, I've updated the description with an updated scope for the upcoming release wherein we'll show the service.version annotations in the time-series charts and add a service.version filter option in the UI filters on the Transactions view. Where did we land on the agent implementation of the service.version field?

I've also separated out the next iteration design for a Versions view, which can be seen here #171 - happy to take feedback on that initial concept.

[dgieselaar]

AFAIU from the discussion in #apm-agents, there's a lot of agents that don't report this out of the box. Some of them try to get a version from the VCS (like a git hash), and others have a configuration option. If I look at the opbeans data, Java, Ruby, Python and Go don't report it.

VCS proposal: #147

Looking at the discussion there it might make sense for us to focus on a deploy API?

[sorenlouv]

I think the intention is for all agents to standardize on sending something. If we can expect agents to eventually do this it still make sense to derive deployments from transactions.
With this approach we can add value immediately without requiring the user to do anything.
We can then add API integration later for more rich deployment tracking for those that want it.

[nehaduggal]

The designs for the annotation with the filter looks great!

As per our discussion, we should add deployment annotations to all time series charts for APM including the charts on the metrics page.

[dgieselaar]

To query the metrics, error and transaction indices to get all the versions that were first seen in the given time frame (replace __START__ and __END__ with start end end of the date range):

{
  "query": {
    "bool": {
      "filter": [
        {
          "terms": {
            "processor.event": [ "metric", "error", "transaction" ]
          }
        },
        {
          "term": {
            "service.name": "opbeans-node"
          }
        },
        {
          "exists": {
            "field": "service.version"
          }
        }
      ]
    }
  },
  "size": 0, 
  "aggs": {
    "by_version": {
      "terms": {
        "field": "service.version"
      },
      "aggs": {
        "first_seen": {
          "min": {
            "field": "@timestamp"
          }
        },
        "select": {
          "bucket_selector": {
            "buckets_path": {
              "first_seen": "first_seen.value"
            },
            "script": "params.first_seen >= __START__ && params.first_seen <= __END__"
          }
        }
      }
    }
  }
}

[formgeist]

Closing in favor of implementation issues elastic/kibana#51426 and elastic/kibana#51427