[Elastic Agent] Enable HTTP endpoint configuration for Elastic Agent

[nimarezainia]

Describe the enhancement:

Enable the HTTP Endpoint for Elastic Agent to bring us to parity. This should be done on a "per policy basis"

Describe a specific use case for the enhancement or feature:

as per what is already available for filebeat

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[elasticmachine]

Pinging @elastic/fleet (Team:Fleet)

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[jlind23]

@ph moving it to control plane as the team should own this.

[ph]

@jlind23 Agree.

Looking at the linked "enhancement request", exposing "only" the HTTP endpoint of Agent would not solve the real problem or the underlying need. Which is Expose the state of Syslog input or at least Filebeat state through HTTP endpoint (/stats available through HTTP endpoint do not reveal anything about underlying Beats).

If I wanted to solve that case I would like to have something like this

Elastic Agent exposes the health status of input (TCP with id 3433 is healthy)
Elastic Agent exposes available port.

The two options above allow you to have "healthy" check + know if you need to expose it to a load balancer.

[jlind23]

@ph shouldn't we close this one and rather rely on the liveness probe one?

[jlind23]

@nimarezainia is this one still required? Haven't seen traction on this for a while.

[cmacknz]

For standalone this has been implemented for a while, and #4586 made it reloadable from Fleet. Closing.

[zez3]

@cmacknz or @nimarezainia what would then be the recommend way to LB healthcheck managed Agents?

[cmacknz]

In 8.15.0 there will be a /liveness endpoint that is enabled whenever agent.monitoring.http.enabled: true is set in the policy, available where that same server is configured to be exposed. By default it returns a 200 OK status as long agent's internal main loop is responsive and can process configuration changes. It can be configured to also monitor the component states to return an error if anything is degraded or failed.

There's no documentation to point at yet because it isn't released.

[nimarezainia]

fyi: @kilfoyle

[nimarezainia]

@zez3 you can preview the Fleet changes here: elastic/kibana#180922