Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Backups: option to reuse the current VM (in-place restore) #6053

Closed
1 of 3 tasks
atodorov-storpool opened this issue Dec 15, 2022 · 3 comments
Closed
1 of 3 tasks

Backups: option to reuse the current VM (in-place restore) #6053

atodorov-storpool opened this issue Dec 15, 2022 · 3 comments
Assignees
@rsmontero @paczerny @vichansson @1gramos
Labels
Category: Drivers - Storage Priority: High Priority: Normal Sponsored Status: Accepted Type: Feature
Milestone
Release 6.10.0

Comments

@atodorov-storpool
Copy link
Contributor

atodorov-storpool commented Dec 15, 2022 •
edited by vichansson
Loading

Description
The current workflow for restoring from a backup is as follows (as far as I can understand it, please correct me if I am wrong):

  • The user click on the Restore button in the Backup tab to restore a VM from a backup:
    • The VM disk backups are extracted from the backup storage. and imported into an Image datastore as new images
    • The VM Metadata, stored with the backups, is used to define a new VM Template that uses the restored new images from the image datastore.
  • The user/administrator must manually review the new VM Template and updated/fixed whenever the Restore script can't handle.
  • Then the user/admin must terminate the old VM to free resources like IP reservations and so on.
  • Finally, the user/admin could instantiate a new VM instance fromt the backup VM template in place of the terminated VM.

So, as you see there are a lot of steps and a lot of internal knowledge needed by the user to complete a single restore task. This is experience that is not needed with any other Cloud Management platform. So I don't belive it is usable on practice...

Use case
The common use case the backup needed for recovery is very close to the current state of the VM. So this proposal to use the existing metadata in OpenNebula and only replace the disks. All of the other metadata could be altered on the current VM too - if needed. That way. I believe, much less data need to be processed/replaced/altered that when defining a new VM.
For the end user perspective - it copuld have the VM up and running from the backup after a single click. Well, definitely with a confirmation dialog. Also, it will ease the stress on the billing department too ;)

Interface Changes
Option to restore a VM using the runnig VM metadata instead of defining a new instance. Also option to select what to be the default behaviour.
Driver changes - To terminate the VM, replace the disks on the KVM host, optionally (if preferred by the user) alter the contextualization and the networking, then resume the VM.

Sunstone

  • New VM state (LCM, 71, RESTORE)
  • New VM action (one.vm.restore)

Additional Context
Having a VM in complete previous state maybe have its use case, but IMHO it is not to broad one. Because usually the VM is linked with some kind of billing, in that billing there various aspects of the service could be accounted/billed, etc.

Progress Status

  • Code committed
  • Testing - QA
  • Documentation (Release notes - resolved issues, compatibility, known issues)
@atodorov-storpool atodorov-storpool mentioned this issue Dec 15, 2022
Open
3 tasks
@rsmontero
Copy link
Member

We have been disusing this internally and, what we call "in-place" restore is something we want to tackle, We'll keep this issue to address this feature.

However, I don't agree that the current restore procedure is cumbersome and requires knowledge. First you are assuming that the VM still lives and even that you want to restore the VM in the same host/cluster. In general this is not true, if you want to restore a previous point for a running VM a disk or system snapshost is much better option than a backup.

Backup and snapshotting are different things but both have their use cases, you should not be using one for the other.

The restore procedure needs to select (and we are in the process of giving more options) what you want to preserve (this is a common practice in other backup solutions like Veeam etc...) So, to some extent the user needs to decide what needs to be restored. We can give more options like (--no-ip or --no-nic``) but having the ability to fine tune what is restored, We think is a nice feature. We aim to a trade-off because having tens of toggles is even a worse solution, so we will provide some more high level options, and leave the user to fine tune the rest (if fine tune is needed)

@rsmontero rsmontero changed the title new Backups: option to reuse the current VM Backups: option to reuse the current VM (in-place restore) Dec 15, 2022
@rsmontero rsmontero added this to the Release 7.0 milestone Dec 15, 2022
@rsmontero
Copy link
Member

Even more important as incremental backups impose important limitations for snapshots.

@rsmontero rsmontero modified the milestones: Release 7.0, Release 6.8.1 Sep 27, 2023
@tinova tinova modified the milestones: Release 6.8.1, Release 6.8.2 Dec 12, 2023
@tinova tinova modified the milestones: Release 6.8.2, Release 6.8.3 Feb 5, 2024
@rsmontero rsmontero assigned rsmontero and unassigned xorel Apr 16, 2024
rsmontero added a commit that referenced this issue Apr 30, 2024
@rsmontero @paczerny
F #6053: In-place restore of VM backups (shared and ssh drivers)
3dfd53d 
This commit implements the in-place restore of VM backups. Selected VM disks will
be replaced with the specified backup:

* A new API call has been added to the XML-RPC API (`one.vm.restore`) with
  the following arguments:
    - VM ID to be restored, needs to be in **poweroff** state
    - IMAGE ID of the backup to restore
    - INCREMENT ID, only for incremental backups, the increment to use
      (defults to -1 to use the last increment available)
    - DISK ID of the disk to restore (defaults to -1 to restore all VM
      disks)

* Datastore drivers needs to implemente a new operation `ls`. This new
  operation takes the VM, image information of the backup and datastore
  information and returns the restore URL for the disks in the backup.

* This commit includes the implementation for qcow2 and ssh drivers,
  ceph will be implemented in a separated PR. The new driver action is
  `restore host:vm_dir vm_id img_id inc_id disk_id`

* The restore operation is performed in a new state `PROLOG_RESTORE`
  rendered as `RESTORE` and `rest` in short form. State in in RSuntone.

TODO:
  - Remove any existing VM snapshot (system/disk) in the VM. Note that
    snapshots are not included in a backup.

  - Ceph drivers

  - JAVA, GO Lang API bindings

  - Sunstone interface, new state and new operation. Review new state in
    RSuntone.

co-authored-by: Pavel Czerny <pczerny@opennebula.io>
rsmontero added a commit to OpenNebula/docs that referenced this issue May 6, 2024
@rsmontero
F OpenNebula/one#6053: documentation for in-place restore.
17d47c2 
Includes documentation for:

- XML-RPC api call one.vm.restore.
- in-place backup operation
- ls operation for datastore driver
- restore operation for tm driver
- what's new entry
rsmontero added a commit that referenced this issue May 14, 2024
@rsmontero
F #6053: Renamed oned internal state name PROLOG_RESTORE to RESTORE
c509eaa 
Now it is consistent with the API naming
rsmontero pushed a commit that referenced this issue May 14, 2024
@paczerny
F #6053: Rename all prolog_restore to restore (#3056)
e68305c
rsmontero pushed a commit that referenced this issue May 16, 2024
@1gramos
F #6053: In-place restore of VM backups (Ceph) (#3051)
4db0e74
rsmontero added a commit that referenced this issue May 28, 2024
@rsmontero
F #6053: Consolidate datastore operation in Transfarmaner::Datastore
ca461e4 
- Access methods to BRIDGE_LIST
- Generic execution of datastore driver operations
- Adapt restore operations to new interface
- Adapt rsync action
rsmontero pushed a commit that referenced this issue May 28, 2024
@1gramos
F #6053: Fix Ceph restore (#3065)
cff2bb6 
Fix wrong handling of the two datastores involved (backup+ceph)

Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
@tinova tinova closed this as completed Jun 17, 2024
@vichansson vichansson reopened this Jun 17, 2024
@vichansson
Copy link
Contributor

Needs tests + docs

@tinova tinova closed this as completed Jun 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rsmontero rsmontero

@paczerny paczerny

@vichansson vichansson

@1gramos 1gramos

Labels
Category: Drivers - Storage Priority: High Priority: Normal Sponsored Status: Accepted Type: Feature
Projects
None yet
Milestone
Release 6.10.0
Development

No branches or pull requests
7 participants
@tinova @rsmontero @xorel @atodorov-storpool @paczerny @vichansson @1gramos