You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe
During the development of shallow v1 snapshots, the clone operation code was modified to handle both shallow and existing full cluster incremental snapshots with the PR - #7496. However, in this PR, we have inadvertently introduced degraded experience for doc-rep cluster's clone snapshot operation where for each shard, there is dependency on repository data (index-N file for repository). If the index-N file size is above 512KB, then data needs to be downloaded from snapshot repository each time. This makes the clone snapshot operation slow. This can be reproduced by creating a lot of snapshots in a repo so that the index-N file size is above 512 KB.
Describe the solution you'd like
For doc-rep clusters, we don't need to depend on the repository data to decide which type of snapshot was taken. For docrep clusters, we only support full cluster incremental snapshots. In this case, we will check whether the cluster is remote store enabled or not, and if not, then directly jump to the part of cloning without seeking the repository data for any shard.
Related component
Storage:Snapshots
Describe alternatives you've considered
No response
Additional context
No response
The text was updated successfully, but these errors were encountered:
ashking94
changed the title
[Feature Request] Optimised snapshot clone operation for doc-rep clusters
[Feature Request] Optimise snapshot clone operation for doc-rep clusters
Oct 12, 2024
ashking94
changed the title
[Feature Request] Optimise snapshot clone operation for doc-rep clusters
[Feature Request] Optimise clone operation for incremental full cluster snapshots
Oct 12, 2024
Is your feature request related to a problem? Please describe
During the development of shallow v1 snapshots, the clone operation code was modified to handle both shallow and existing full cluster incremental snapshots with the PR - #7496. However, in this PR, we have inadvertently introduced degraded experience for doc-rep cluster's clone snapshot operation where for each shard, there is dependency on repository data (index-N file for repository). If the index-N file size is above 512KB, then data needs to be downloaded from snapshot repository each time. This makes the clone snapshot operation slow. This can be reproduced by creating a lot of snapshots in a repo so that the index-N file size is above 512 KB.
Describe the solution you'd like
For doc-rep clusters, we don't need to depend on the repository data to decide which type of snapshot was taken. For docrep clusters, we only support full cluster incremental snapshots. In this case, we will check whether the cluster is remote store enabled or not, and if not, then directly jump to the part of cloning without seeking the repository data for any shard.
Related component
Storage:Snapshots
Describe alternatives you've considered
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: