Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Saved Objects should reduce the migrationVersion field count #70815

Closed
rudolf opened this issue Jul 6, 2020 · 1 comment · Fixed by #150075
Closed

Saved Objects should reduce the migrationVersion field count #70815

rudolf opened this issue Jul 6, 2020 · 1 comment · Fixed by #150075
Labels
Feature:Saved Objects Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc v8.0.0

Comments

@rudolf
Copy link
Contributor

rudolf commented Jul 6, 2020

Related to #43673

The Saved Objects migrationVersion object field used for Saved Object migrations adds a dynamic mapping to detect outdated documents:

migrationVersion: {
// Saved Objects can't redefine dynamic, but we cheat here to support migrations
// @ts-expect-error
dynamic: 'true',
type: 'object',
},

This mapping definition can add as many as 2*Saved Object Type Count or ~114 fields. This can be reduced in two ways:

  1. Change the dynamic mapping template https://www.elastic.co/guide/en/elasticsearch/reference/current/dynamic-templates.html so that dynamic string fields aren't mapped to a string/keyword multi type (2 fields) but only to a keyword field. This will reduce the mappings to 1*Saved Object Type Count
  2. Remove the dynamic mapping altogether. The underlying design for Saved Object migrations allows for any saved object type to define migrations on any other saved object type. This required a map of migrationVersions
    const response = await callCluster('count', {
    body: {
    query: {
    bool: {
    should: Object.entries(migrationVersion).map(([type, latestVersion]) => ({
    bool: {
    must: [
    { exists: { field: type } },
    { bool: { must_not: { term: { [`migrationVersion.${type}`]: latestVersion } } } },
    ],
    },
    })),
    },
    },
    },
    index,
    });
    However, this functionality has never been leveraged and the NP API's prevent defining a migration for a different type of saved object. So migrationVersion could be changed to a single field and the query could limit results to the saved object type and a migrationVersion field which doesn't match the current version (maybe we can leverage SEMVER fields if available WIP First draft for version field elasticsearch#58256)
@rudolf rudolf added Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc Feature:Saved Objects v8.0.0 labels Jul 6, 2020
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-platform (Team:Platform)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Saved Objects Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc v8.0.0
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants