Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Saved Object Migrations should clean up unused mappings #67086

Closed
rudolf opened this issue May 20, 2020 · 1 comment · Fixed by #70951
Closed

Saved Object Migrations should clean up unused mappings #67086

rudolf opened this issue May 20, 2020 · 1 comment · Fixed by #70951
Labels
Feature:Saved Objects Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc

Comments

@rudolf
Copy link
Contributor

rudolf commented May 20, 2020

Each time new Saved Object types are registered or existing type's mappings change Kibana will run a migration on startup to create a new index with updated mappings. However, the new index's mappings consists of the mappings for saved object types from both the old index's mappings and the updated mappings roughly something like: const newMappings = {...outdatedMappings, ...newMappings}.

mappings: {
...activeMappings,
properties: {
...source.mappings.properties,
...activeMappings.properties,
},
},

This ensure that data created by plugins that have since been disabled can be migrated to the new index without any mapping conflicts. But it means that mappings from unused Saved Object types or disabled plugins are never cleaned up so the field count will keep on slowly increasing with every migration. This also happens when a Saved Object type is "renamed", upon migration Kibana will add the mappings for the new type name, but won't remove the old mappings.

To fix this we should set the mappings of any saved object type found in the source index but not present in the upgrading node's mappings to type: 'object', enabled: false. This will ensure we can still write migrate any old documents, but the old types they belong to will no longer contribute to the field count.

Related: #43673

@rudolf rudolf added Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc Feature:Saved Objects labels May 20, 2020
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-platform (Team:Platform)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Saved Objects Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants