Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fleet] fix duplicate ingest pipeline refs #82078

Merged
merged 2 commits into from
Oct 30, 2020

Conversation

neptunian
Copy link
Contributor

@neptunian neptunian commented Oct 29, 2020

Resolves #76818

An epm-packges saved object was saving duplicates of pipelines in the installed_es property when installing the same version, each time it was installed.

Pipelines are handled differently than other elasticsearch assets because they cannot be removed until other assets are installed, so during an upgrade they aren't removed until the end and the old versionized pipelines are searched for, eg: logs-nginx.ingress_controller-0.3.5. In the case where the user is installing the same version (reinstall), there is no previous version to search for, and the new assets are the same as the old, so we should do a check during the installation of pipelines to make sure this version's pipelines don't already exist in the saved object, and if they do, remove them, before saving the new ones of the same version. This prevents duplication of the same pipelines.

Tests are updated to check that installing and reinstalling have the same installed assets and saved objects to cover this case. I separated the tests out into one function that both cases can call.

@neptunian neptunian requested a review from a team October 29, 2020 20:55
@neptunian neptunian self-assigned this Oct 29, 2020
@botelastic botelastic bot added the Team:Fleet Team label for Observability Data Collection Fleet team label Oct 29, 2020
@elasticmachine
Copy link
Contributor

Pinging @elastic/ingest-management (Team:Ingest Management)

@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

✅ unchanged

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@neptunian neptunian merged commit 81c0e12 into elastic:master Oct 30, 2020
@neptunian neptunian deleted the 76818-bug-reinstall-duplicates branch October 30, 2020 12:28
neptunian added a commit to neptunian/kibana that referenced this pull request Oct 30, 2020
* check if pipeline refs for version already exists

* use the right version
neptunian added a commit that referenced this pull request Oct 30, 2020
* check if pipeline refs for version already exists

* use the right version
gmmorris added a commit to gmmorris/kibana that referenced this pull request Oct 30, 2020
* master: (71 commits)
  [Chrome] Extension to append an element to the last breadcrumb (elastic#82015)
  [Monitoring] Thread pool rejections alert (elastic#79433)
  [Actions] Fix actionType type on registerType function (elastic#82125)
  [Security Solution] Modal for saving timeline (elastic#81802)
  add tests for index pattern switching (elastic#81987)
  TS project references for share plugin (elastic#82051)
  [Graph] Fix problem with duplicate ids (elastic#82109)
  skip 'returns a single bucket if array has 1'.  related elastic#81460
  Add a link to documentation in the alerts and actions management UI (elastic#81909)
  [Fleet] fix duplicate ingest pipeline refs (elastic#82078)
  Context menu trigger for URL Drilldown (elastic#81158)
  SO management: fix legacy import index pattern selection being reset when switching page (elastic#81621)
  Fixed dead links (elastic#78696)
  [Search] Add "restore" to session service (elastic#81924)
  fix Lens heading structure (elastic#81752)
  [ML] Data Frame Analytics: Fix feature importance cell value and decision path chart (elastic#82011)
  Remove legacy app arch items from codeowners. (elastic#82084)
  [TSVB] Renamed 'positive rate' to 'counter rate' (elastic#80939)
  Expressions/migrations2 (elastic#81281)
  [Telemetry] [Schema] remove number type and support all es number types (elastic#81774)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release_note:fix Team:Fleet Team label for Observability Data Collection Fleet team v7.11.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Ingest Manager] When reinstalling a package, pipeline refs are not removed from saved object
4 participants