Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Proposal] Enable searchable snapshots to read legacy versions #5451

Closed
kartg opened this issue Dec 5, 2022 · 4 comments · Fixed by #5812
Closed

[Proposal] Enable searchable snapshots to read legacy versions #5451

kartg opened this issue Dec 5, 2022 · 4 comments · Fixed by #5812
Labels
backwards-compatibility discuss Issues intended to help drive brainstorming and decision making distributed framework enhancement Enhancement or improvement to existing feature or request experimental Marks issues as experimental feature New feature or request idea Things we're kicking around. Indexing & Search Search Search query, autocomplete ...etc

Comments

@kartg
Copy link
Member

kartg commented Dec 5, 2022

Is your feature request related to a problem? Please describe.
Expand searchable snapshots to support read-only access to snapshots written by legacy versions. This would be useful to customers storing data for long periods of time (and thereby across major version upgrades) for log analysis or audit use-cases.

Describe the solution you'd like
Leverage Lucene 9's ability to read older versions to expand backwards compatibility for searchable snapshots to allow reading older snapshots.

Describe alternatives you've considered
N/A

Additional context
N/A

@kartg kartg added enhancement Enhancement or improvement to existing feature or request untriaged discuss Issues intended to help drive brainstorming and decision making feature New feature or request backwards-compatibility idea Things we're kicking around. Indexing & Search distributed framework experimental Marks issues as experimental Search Search query, autocomplete ...etc and removed untriaged labels Dec 5, 2022
@kartg kartg changed the title [Feature] Enable searchable snapshots to read legacy snapshots [Proposal] Enable searchable snapshots to read legacy snapshots Dec 5, 2022
@dblock dblock changed the title [Proposal] Enable searchable snapshots to read legacy snapshots [Proposal] Enable searchable snapshots to read legacy versions Dec 5, 2022
@dblock
Copy link
Member

dblock commented Dec 5, 2022

What is considered "legacy" @kartg in 1.x, 2.x, and 3.x?

@kartg
Copy link
Member Author

kartg commented Dec 6, 2022

What is considered "legacy" @kartg in 1.x, 2.x, and 3.x?

@dblock IMO a "legacy" version is any version older than OpenSearch's standard backwards compatibility guarantee (one prior major version). So:

  • For 3.x, OpenSearch 1.x (or ElasticSearch 7.10.2) and lower versions are considered legacy
  • For 2.x -> ElasticSearch 6.x and lower
  • For 1.x -> ElasticSearch 5.x and lower

I think these should also have a lower bound, but I don't have a value in mind at this time. For example, the implementation in my fork does not currently work for ElasticSearch 5.x snapshots, but I believe I can get it to function.

@kartg
Copy link
Member Author

kartg commented Jan 4, 2023

PR to 2.x - #5429

@kartg
Copy link
Member Author

kartg commented Jan 11, 2023

PR to main - #5812

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backwards-compatibility discuss Issues intended to help drive brainstorming and decision making distributed framework enhancement Enhancement or improvement to existing feature or request experimental Marks issues as experimental feature New feature or request idea Things we're kicking around. Indexing & Search Search Search query, autocomplete ...etc
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants