[dot-kibana-split] Split analyst experience SOs into .kibana_analytics
#155140
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
gsoldevila
added
Team:Core
|
Pinging @elastic/kibana-core (Team:Core) |
|
Pinging @elastic/kibana-presentation (Team:Presentation) |
|
Pinging @elastic/kibana-data-discovery (Team:DataDiscovery) |
gsoldevila
changed the title
.kibana_analyst_experience
|
@thomasneirynck we also have the query saved object which stores the queries saved by the unified search. I think they should possible be part of the .kibana-analyst experience index or do you have a different opinion? |
|
Update: I saw that we are keeping this in the .kibana index. I am fine with it 👍 but there are some discussions that we might combine this with the saved-search SO in the future #153809 (not decided yet). Will this distinction be a problem if we decide in favor of it? Why this SO is on the .kibana index and not on the .kibana_analyst_experience? |
|
Agreed with @stratoula. Is there any harm in moving the query object to the .kibana_analytics index? Regardless of outcome, ideally we can have saved-search and saved-queries in the same index. If the decision is later made to merge them, it will be a more natural migration. |
12 tasks
Good catch, in fact it was not properly categorised in the spreadsheet, so I totally missed it. I will move it to the |
Is the above a typo in the description? Should ".kibana_security_solution" be something else? |
|
@gsoldevila this is a total nit 😄 but imho it reads kind of funny cc @kertal |
Initially it was .kibana_analytics already, but I renamed it cause I thought the term 'analytics' could be confusing. |
You can add visualizations team for now, although we co-own some things with the data-discovery team 😓 |
…ana-split-analyst-experience
pgayvallet
changed the title
.kibana_analyst_experience.kibana_analytics
…ana-split-analyst-experience
💔 Build FailedFailed CI Steps
Test Failures
Metrics [docs]
History
To update your PR or re-run it, just comment with: |
gsoldevila
added a commit
that referenced
this pull request
Apr 25, 2023
## Description Fix #104081 This PR move some of the SO types from the `.kibana` index into the following ones: - `.kibana_alerting_cases` - `.kibana_analytics` - `.kibana_security_solution` - `.kibana_ingest` This split/reallocation will occur during the `8.8.0` Kibana upgrade (*meaning: from any version older than `8.8.0` to any version greater or equal to `8.8.0`*) **This PR main changes are:** - implement the changes required in the SO migration algorithm to support this reallocation - update the FTR tools (looking at you esArchiver) to support these new indices - update hardcoded references to `.kibana` and usage of the `core.savedObjects.getKibanaIndex()` to use new APIs to target the correct index/indices - update FTR datasets, tests and utility accordingly ## To reviewers **Overall estimated risk of regressions: low** But, still, please take the time to review changes in your code. The parts of the production code that were the most impacted are the telemetry collectors, as most of them were performing direct requests against the `.kibana` index, so we had to adapt them. Most other contributor-owned changes are in FTR tests and datasets. If you think a type is misplaced (either we missed some types that should be moved to a specific index, or some types were moved and shouldn't have been) please tell us, and we'll fix the reallocation either in this PR or in a follow-up. ## .Kibana split The following new indices are introduced by this PR, with the following SO types being moved to it. (any SO type not listed here will be staying in its current index) Note: The complete **_type => index_** breakdown is available in [this spreadsheet](https://docs.google.com/spreadsheets/d/1b_MG_E_aBksZ4Vkd9cVayij1oBpdhvH4XC8NVlChiio/edit#gid=145920788). #### `.kibana_alerting_cases` - action - action_task_params - alert - api_key_pending_invalidation - cases - cases-comments - cases-configure - cases-connector-mappings - cases-telemetry - cases-user-actions - connector_token - rules-settings - maintenance-window #### `.kibana_security_solution` - csp-rule-template - endpoint:user-artifact - endpoint:user-artifact-manifest - exception-list - exception-list-agnostic - osquery-manager-usage-metric - osquery-pack - osquery-pack-asset - osquery-saved-query - security-rule - security-solution-signals-migration - siem-detection-engine-rule-actions - siem-ui-timeline - siem-ui-timeline-note - siem-ui-timeline-pinned-event #### `.kibana_analytics` - canvas-element - canvas-workpad-template - canvas-workpad - dashboard - graph-workspace - index-pattern - kql-telemetry - lens - lens-ui-telemetry - map - search - search-session - search-telemetry - visualization #### `.kibana_ingest` - epm-packages - epm-packages-assets - fleet-fleet-server-host - fleet-message-signing-keys - fleet-preconfiguration-deletion-record - fleet-proxy - ingest_manager_settings - ingest-agent-policies - ingest-download-sources - ingest-outputs - ingest-package-policies ## Tasks / PRs ### Sub-PRs **Implementation** - 🟣 #154846 - 🟣 #154892 - 🟣 #154882 - 🟣 #154884 - 🟣 #155155 **Individual index split** - 🟣 #154897 - 🟣 #155129 - 🟣 #155140 - 🟣 #155130 ### Improvements / follow-ups - 👷🏼 Extract logic into [runV2Migration](#154151 (comment)) @gsoldevila - Make `getCurrentIndexTypesMap` resillient to intermittent failures #154151 (comment) - 🚧 Build a more structured [MigratorSynchronizer](#154151 (comment)) - 🟣 #155035 - 🟣 #155116 - 🟣 #155366 ## Reallocation tweaks Tweaks to the reallocation can be done after the initial merge, as long as it's done before the public release of 8.8 - `url` should get back to `.kibana` (see [comment](#154888 (comment))) ## Release Note For performance purposes, Kibana is now using more system indices to store its internal data. The following system indices will be created when upgrading to `8.8.0`: - `.kibana_alerting_cases` - `.kibana_analytics` - `.kibana_security_solution` - `.kibana_ingest` --------- Co-authored-by: pgayvallet <pierre.gayvallet@elastic.co> Co-authored-by: Christos Nasikas <christos.nasikas@elastic.co> Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>
rudolf
added
the
Epic:ScaleMigrations
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This is targeting a feature branch
Part of #154888
This PR aims at moving the following saved object types to a new
.kibana_analyticssystem index.