Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Extend the backup manifest #4130

Open
karol-kokoszka opened this issue Nov 27, 2024 · 2 comments · May be fixed by #4177
Open

Extend the backup manifest #4130

karol-kokoszka opened this issue Nov 27, 2024 · 2 comments · May be fixed by #4177
Assignees
@VAveryanov8
Milestone
1-1 Restore

Comments

@karol-kokoszka
Copy link
Collaborator

Include intro backup manifest rack and instance details.
@karol-kokoszka karol-kokoszka added this to the 1-1 Restore milestone Nov 27, 2024
@VAveryanov8 VAveryanov8 self-assigned this Dec 13, 2024
@Michal-Leszczynski Michal-Leszczynski mentioned this issue Dec 13, 2024
Closed
VAveryanov8 added a commit that referenced this issue Dec 16, 2024
@VAveryanov8
feat(swagger): extends sm agent /node_info' with data_directory`.
2a6d053 
This extends `/node_info` response of scylla manager agent with `data_directory` field which
should contain first entry from the `data_file_directories` list of
scylladb configuration.

Refs: #4130
VAveryanov8 added a commit that referenced this issue Dec 16, 2024
@VAveryanov8
feat(swagger): extends sm agent /node_info' with data_directory`.
8ced873 
This extends `/node_info` response of scylla manager agent with `data_directory` field which
should contain first entry from the `data_file_directories` list of
scylladb configuration.

Refs: #4130
@VAveryanov8 VAveryanov8 mentioned this issue Dec 16, 2024
Merged
@Michal-Leszczynski
Copy link
Collaborator

@VAveryanov8 @karol-kokoszka let's update the list of things that should be added to the manifest (e.g. shards, storage size, instance type, etc.).
Let's also take #3916 into consideration.

VAveryanov8 added a commit that referenced this issue Dec 16, 2024
@VAveryanov8
feat(swagger): extends sm agent /node_info' with data_directory`. (#…
2a0edaf 
…4168)

This extends `/node_info` response of scylla manager agent with `data_directory` field which
should contain first entry from the `data_file_directories` list of
scylladb configuration.

Refs: #4130
VAveryanov8 added a commit that referenced this issue Dec 16, 2024
@VAveryanov8
feat(swagger): extends sm agent /node_info' with storage_size`.
b5f6af3 
This extends `/node_info` agent response with `storage_size` - total disk size
in bytes. It's needed for the backup manifest and should be calculated
on the agent side.

Refs: #4130
@VAveryanov8 VAveryanov8 mentioned this issue Dec 16, 2024
Merged
VAveryanov8 added a commit that referenced this issue Dec 17, 2024
@VAveryanov8
feat(swagger): extends sm agent /node_info with storage_size. (#4170
cafa851 
)

This extends `/node_info` agent response with `storage_size` - total disk size
in bytes. It's needed for the backup manifest and should be calculated
on the agent side.

Refs: #4130
VAveryanov8 added a commit that referenced this issue Dec 17, 2024
@VAveryanov8
feat(backup): extends manifest with info needed for 1-to-1 restore.
f851f3d 
This adds following data to the backup manifest:
General:
  rack: rack from the scylla configuration
  host_id: host id of the scylla node
Instance Details:
  shard_count: number of shard in scylla node
  storage_size: total size of the disk in bytes
  cloud_provider: aws|gcp|azure or empty in case of on-premise
  instance_type: instance type, e.g. t2.nano or empty when on-premise

Fixes: #4130
@VAveryanov8 VAveryanov8 linked a pull request Dec 17, 2024 that will close this issue
Open
@karol-kokoszka
Copy link
Collaborator Author

Let's also take #3916 into consideration.

We must add to the manifest file things like cluster-id, node-id (which is the host-id) and the dc-name. Basically everything which is used to identify the particular manifest files with the path.

Path is defined as follow

/backup/sst/cluster/<cluster ID>/dc/<data center name>/node/<node ID>

VAveryanov8 added a commit that referenced this issue Dec 18, 2024
@VAveryanov8
feat(backup): extends manifest with info needed for 1-to-1 restore.
f03311a 
This adds following data to the backup manifest:
General:
  cluster_id: uuid of the cluster
  dc: data center name
  rack: rack from the scylla configuration
  node_id: id of the scylla node (equals to host id)
Instance Details:
  shard_count: number of shard in scylla node
  storage_size: total size of the disk in bytes
  cloud_provider: aws|gcp|azure or empty in case of on-premise
  instance_type: instance type, e.g. t2.nano or empty when on-premise

Fixes: #4130
VAveryanov8 added a commit that referenced this issue Dec 20, 2024
@VAveryanov8
feat(swagger): adds /cloud/metadata to agent api definition.
6e90ce8 
This adds /cloud/metadata api call to agent which should return cloud
instance metadata, such as instance_type and cloud_provider.

Refs: #4130
@VAveryanov8 VAveryanov8 mentioned this issue Dec 20, 2024
Open
VAveryanov8 added a commit that referenced this issue Dec 23, 2024
@VAveryanov8
feat(swagger): adds /cloud/metadata to agent api definition.
79ffa4e 
This adds /cloud/metadata api call to agent which should return cloud
instance metadata, such as instance_type and cloud_provider.

Refs: #4130
VAveryanov8 added a commit that referenced this issue Dec 23, 2024
@VAveryanov8
feat(backup): extends manifest with info needed for 1-to-1 restore.
418aa9a 
This adds following data to the backup manifest:
General:
  cluster_id: uuid of the cluster
  dc: data center name
  rack: rack from the scylla configuration
  node_id: id of the scylla node (equals to host id)
  task_id: uuid of the backup task
  snapshot_tag: snapshot tag
Instance Details:
  shard_count: number of shard in scylla node
  storage_size: total size of the disk in bytes
  cloud_provider: aws|gcp|azure or empty in case of on-premise
  instance_type: instance type, e.g. t2.nano or empty when on-premise

Fixes: #4130
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@VAveryanov8 VAveryanov8

Labels
None yet
Projects
None yet
Milestone
1-1 Restore
Development

Successfully merging a pull request may close this issue.

feat(backup): extends backup manifest with info needed for 1-to-1 restore. scylladb/scylla-manager
3 participants
@VAveryanov8 @karol-kokoszka @Michal-Leszczynski