[Agent] First release of the Elastic Agent.

[elasticmachine]

Original comment by @ph:*

MVP

Tasks : Standalone

• Support multiples outputs using "streams" [Agent] Add support for multiple outputs for the Beat agent. #14445
• Verify any configurations in the agent is necessary and secure.
• Better error handling between config / reporter -> transpiler -> operator. [Agent] Better handle retryable errors in the emitter. #14447
• Refactor architecture [closed on old Repo]
• Support for Filebeat, Metricbeat, Audibeat.
  • Secure communication between agents and beats [closed on old Repo]
  • Define program spec or how agent run a beats [closed on old Repo]
  • Define communication contracts between agent and beats ([Agent] Define contract between Agent and a Beat #14433 [Agent] Onboard beats to fleet #14442)
    • Update filebeat to run under agent. [Agent] Update Filebeat to run under the Agent using Central Management's manager interface #14449 (code done was waiting on move to beats)
    • Update Metricbeat to run under agent. [Agent] Update Metricbeat to run under the Agent using Central Management's manager interface #107 #14450 (code done was waiting on move to beats)
    • Update Auditbeat to run under agent. [Agent] Update Metricbeat to run under the Agent using Central Management's manager interface #107 #14451 (need to be done)
  • Enable Agent to configure a beats (closed on old repo)
  • Agent running beats (processes)
  • Enable logging introspection of a specific beats ([Agent] Enable logging introspection of a specific beat #14437)
  • Enable agent to download a beats (closed on old repo)
• Local Config deployment options.
  • normalize keys in the configuration (like output.elasticsearch) closed on old repo
  • read local configuration (closed on old repository)
  • Reload a configuration closed on old repository
  • Refactor ucfg (closed on old repository)
  • Create a transpilers from Agent to specific beats. closed on old repository
  • Define execution condition. closed on old repository
• Online program install via elastic.co closed on old repository
• Build foundation with single Config, Config translation, etc. closed on old repository
• Create packaging closed on old repository
• Integration tests (first our integration tests, after kibana integration tests)
• Monitoring sends metrics. (which metrics to send, sync with monitoring team) [Agent]Allow runnings beats to be monitored by Metricbeat. #14452
• Enable seccomp from libbeats. https://github.com/elastic/fleet/issues/113 (We need different policies)
• Refactor logging + error reporting. closed on old repository
• Add keystore support.[Agent]Allow runnings beats to be monitored by Metricbeat. #14452 (only needed for Standalone)
• packaging
  • only agent close on old repository
  • Release a lightweight version of the beats agents. (no beats are included in the distribution)
  • with other beats
• Respect permission on configuration files [Agent] Autoreload of configuration of files and permission check. #14455
• Make sure beats under the control of the agent doesn't install templates or ILM policies, or create alias. [Agent] Index, template, ILM setup strategy for the Agent. #14456
• Add pre-checks for the agent concerning the presence of template see https://github.com/elastic/fleet/issues/116
• Ensure that the Agent daemon correctly handle all the OS Signal.
• Move logging back to the libbeat framework.
• Docs
• Investigate privileges modes under windows.

Task in Beats / Libbeats

• Allow beats to send to a specific index (Allow inputs / module to define a target index. #13255)
• Move Agent to beats repository
  • Improve CI in beats repository @mikemadden42 Selective testing in beats and test with latest version of the branch. #9496
• Refactor the elasticsearch output in libbeat and fix the memory leak (Split the Elasticsearch output and the Elasticsearch client #11068)

Tasks for integrating with Fleet

• Support enrollment
  • Allow to save on disk the access token
  • Refactor the Encryption used by the keystore to be an io.Writer and io.Reader so we can save any received information.
• Report events to fleet (Code done in agent need testings)
• Fetch config from the fleet
• Cache configuration security to work if fleet is not reachable?
• Retrieves errors from the beats and send then to fleet.
• Allow Agent to download artifact from Fleet.

Tasks after MVP

• Add support for autodiscover [Agent] Discuss: Autodiscover configuration feature #14466
• Self update of the agent and the required beats.
• Allow the keystore to bundle certificates
• Implement Endpoint security
• Standalone: Investigate if we have autoreload on by default or not with security team. LINK REDACTED (its currently off by default)

Project members

The following list is for the owners of the different items on the list above across teams.

• Integrations Registry: @ruflin
• Indexing Strategy for integration: @ruflin
• Fleet: @mattapperson
• Beats Agent: @ph
• docs @dedemorton

cc @acchen97 @monicasarbu @michalpristas

[elasticmachine]

Original comment by @ph:

issue updated with current work.

[elasticmachine]

Original comment by @ph:

cc @clintongormley

[ph]

meta issues links updated.

[ph]

closing this, we are tracking the work elsewhere.