Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support migrating regular data streams to tsdb data streams #83520

Closed
martijnvg opened this issue Feb 4, 2022 · 1 comment · Fixed by #83843
Closed

Support migrating regular data streams to tsdb data streams #83520

martijnvg opened this issue Feb 4, 2022 · 1 comment · Fixed by #83843
Assignees
@martijnvg
Labels
:Data Management/Data streams Data streams and their lifecycles Team:Data Management Meta label for data/management team

Comments

@martijnvg
Copy link
Member

Migrating a regular data stream to be a tsdb data stream should be achieved by
setting the index_mode parameter to time_series in the data stream snippet
of the composable index template that created the data stream. This should only
be possible when no index_mode has been specified in the composable index template.
When this change has been made to the composable index template, then the next
time the data stream rolls over the index_mode property of a data stream will
be set to time_series and a new backing index with index.mode index setting
set time_series will be created. All new backing indices created by rollover
for this migrated data stream will be tsdb indices.
@martijnvg martijnvg added the :Data Management/Data streams Data streams and their lifecycles label Feb 4, 2022
@martijnvg martijnvg self-assigned this Feb 4, 2022
@elasticmachine elasticmachine added the Team:Data Management Meta label for data/management team label Feb 4, 2022
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-data-management (Team:Data Management)

@martijnvg martijnvg mentioned this issue Feb 4, 2022
Closed
martijnvg added a commit to martijnvg/elasticsearch that referenced this issue Feb 11, 2022
@martijnvg
Allow regular data streams to be migrated to tsdb data streams.
449609f 
A regular data stream can be migrated to a tsdb data stream if the `index_mode` field is set to `time_series`.
The `index_mode` field can only be update from null to `time_series`.
Then on the next rollover the data stream's index mode's is changed
from null to time_series and the new backing indices will have `index.mode` index setting set to `time_series` as well.

Closes elastic#83520
@martijnvg martijnvg mentioned this issue Feb 11, 2022
Merged
martijnvg added a commit that referenced this issue Feb 17, 2022
@martijnvg
Allow regular data streams to be migrated to tsdb data streams. (#83843)
ae7defa 
A regular data stream can be migrated to a tsdb data stream if in template that created the data stream, the `index_mode` field is set to `time_series` and the data stream's `index_mode` property is either not specified or set to `standard`. Then on the next rollover the data stream is migrated to be a tsdb data stream.

When that happens the data stream's `index_mode` property is set to `time_series` and the new backing index's `index.mode` index setting is also set to `time_series`.

Closes #83520
probakowski pushed a commit to probakowski/elasticsearch that referenced this issue Feb 23, 2022
@martijnvg @probakowski
Allow regular data streams to be migrated to tsdb data streams. (elas…
90e837c 
…tic#83843)

A regular data stream can be migrated to a tsdb data stream if in template that created the data stream, the `index_mode` field is set to `time_series` and the data stream's `index_mode` property is either not specified or set to `standard`. Then on the next rollover the data stream is migrated to be a tsdb data stream.

When that happens the data stream's `index_mode` property is set to `time_series` and the new backing index's `index.mode` index setting is also set to `time_series`.

Closes elastic#83520
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@martijnvg martijnvg

Labels
:Data Management/Data streams Data streams and their lifecycles Team:Data Management Meta label for data/management team
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Allow regular data streams to be migrated to tsdb data streams. martijnvg/elasticsearch
2 participants
@martijnvg @elasticmachine