Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create new Instance from VM backup #10140

Draft
wants to merge 37 commits into
base: main
Choose a base branch
from

Conversation

abh1sar
Copy link
Collaborator

@abh1sar abh1sar commented Dec 25, 2024

Description

This PR adds the ability to create a new Instance from a VM backup for Dummy, NAS and Veeam backup Providers.
This will still work if the original Instance used to create the backup was expunged.

New API

  • createVmFromBackup extends deployVIrtualMachine
  • new argument : backupId
  • User can just specify the backupId and a new instance will be created by getting the VM and volumes' configuration stored in backup_details table.
  • User can also choose to configure the details themselves using the parameters present in deployVIrtualMachine cmd

UI

  • New action button in backups list view - 'Create Instance from Backup'
  • User can choose to directly create the VM or configure VM parameters before creating it.

DB changes

  • New backup_details table
  • to store vm config data such as templateUuid, serviceOfferingUuid, hypervisor, networkUuids
  • to store volumes config data such as diskOfferingUuids, size, miniops and maxiops for custom disk offerings.

Other Changes

  • Added new parameter Map datadisksdetails to deployVirtualMachine api to create multiple data volumes at the time of instance creation.

Plugins related changes

  • NAS and Veeam plugin changes to restore backup to a new VM.

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)
  • build/CI
  • test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

Screenshots (if appropriate):

Screenshot 2025-01-14 at 2 17 16 PM Screenshot 2025-01-14 at 2 17 35 PM

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Copy link

codecov bot commented Dec 25, 2024

Codecov Report

Attention: Patch coverage is 30.78035% with 479 lines in your changes missing coverage. Please review.

Project coverage is 16.20%. Comparing base (0ed6895) to head (0cebdae).

Files with missing lines Patch % Lines
.../src/main/java/com/cloud/vm/UserVmManagerImpl.java 33.48% 114 Missing and 31 partials ⚠️
...rg/apache/cloudstack/backup/BackupManagerImpl.java 65.06% 36 Missing and 22 partials ⚠️
...ack/api/command/user/vm/CreateVMFromBackupCmd.java 3.57% 54 Missing ⚠️
...rg/apache/cloudstack/backup/dao/BackupDaoImpl.java 0.00% 42 Missing ⚠️
...a/org/apache/cloudstack/backup/BackupDetailVO.java 0.00% 37 Missing ⚠️
...he/cloudstack/api/command/user/vm/DeployVMCmd.java 50.00% 10 Missing and 10 partials ⚠️
...che/cloudstack/backup/NetworkerBackupProvider.java 0.00% 17 Missing ⚠️
...in/java/org/apache/cloudstack/backup/BackupVO.java 0.00% 16 Missing ⚠️
.../apache/cloudstack/backup/VeeamBackupProvider.java 0.00% 16 Missing ⚠️
...rg/apache/cloudstack/backup/NASBackupProvider.java 0.00% 12 Missing ⚠️
... and 10 more
Additional details and impacted files
@@             Coverage Diff              @@
##               main   #10140      +/-   ##
============================================
+ Coverage     16.17%   16.20%   +0.02%     
- Complexity    13039    13075      +36     
============================================
  Files          5645     5651       +6     
  Lines        494749   496247    +1498     
  Branches      59946    60247     +301     
============================================
+ Hits          80042    80414     +372     
- Misses       405877   406909    +1032     
- Partials       8830     8924      +94     
Flag Coverage Δ
uitests 3.90% <ø> (-0.11%) ⬇️
unittests 17.08% <30.78%> (+0.05%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@leo79901
Copy link

Yes, we need this !

@abh1sar
Copy link
Collaborator Author

abh1sar commented Jan 6, 2025

@blueorangutan package

@blueorangutan
Copy link

@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 11996

Copy link

github-actions bot commented Jan 8, 2025

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

@abh1sar abh1sar changed the title [WIP] Create new Instance from VM backup Create new Instance from VM backup Jan 14, 2025
Copy link

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

@abh1sar
Copy link
Collaborator Author

abh1sar commented Jan 16, 2025

@blueorangutan package

@blueorangutan
Copy link

@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12095

Copy link
Contributor

@shwstppr shwstppr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good work @abh1sar . Left some comments

@@ -147,6 +148,13 @@ public class DeployVMCmd extends BaseAsyncCreateCustomIdCmd implements SecurityG
since = "4.4")
private Long rootdisksize;

@Parameter(name = ApiConstants.DATADISKS_DETAILS,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@abh1sar is it possible to use the existing details param here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shwstppr I am not sure how we can input a list of datadiskdetails using the details Map.
Anyway, It seems better to me to keep the parameter separate like ipToNetworkList for easier usage and understanding (this could be used for deployvm in general to create instances with multiple volumes)

@@ -102,6 +103,10 @@ public class BackupResponse extends BaseResponse {
@Param(description = "zone name")
private String zone;

@SerializedName(ApiConstants.VM_DETAILS)
@Param(description = "Lists the vm specific details for the backup")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since attribute here

@@ -1009,32 +1011,40 @@ public UserVm resetVMSSHKey(ResetVMSSHKeyCmd cmd) throws ResourceUnavailableExce
throw new InvalidParameterValueException("Vm " + userVm + " should be stopped to do SSH Key reset");
}

if (cmd.getNames() == null || cmd.getNames().isEmpty()) {
List<String> names = cmd.getNames();
if (names == null || names.isEmpty()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use StringUtils?

@@ -6139,6 +6186,11 @@ public UserVm createVirtualMachine(DeployVMCmd cmd) throws InsufficientCapacityE
}
}

List<DiskOfferingInfo> dataDiskOfferingsInfo = cmd.getDataDiskOfferingsInfo();
if (dataDiskOfferingsInfo != null && diskOfferingId != null) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If diskOfferingId is the ID of just another data disk, is it better to generate a diskOfferingInfo using it and use a single variable dataDiskOfferingsInfo in the subsequent method calls?

Copy link
Collaborator Author

@abh1sar abh1sar Jan 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

diskOfferingId also corresponds to root disk if vm is created from iso.
It is definitely better if we have a single argument for all data disk creation, but we can't avoid passing diskOfferingId to other methods due to the iso limitation.
I can check the ISO condition and put the diskoffering details for single data disk in diskOfferingInfoList if you think it is still worth doing.

@@ -2626,11 +2628,13 @@
"label.bucket.policy": "Bucket Policy",
"label.usersecretkey": "Secret Key",
"label.create.bucket": "Create Bucket",
"label.create.instance.from.backup": "Create new instance from backup",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why two different keys? label.create.new.instance.from.backup and label.create.instance.from.backup

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not needed. Will remove one.


List<Backup.VolumeInfo> backupVolumes = backup.getBackedUpVolumes();
if (backupVolumes == null) {
throw new CloudRuntimeException("Backed-up volumes not found");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better message can be used


BackupOffering offering = backupOfferingDao.findByIdIncludingRemoved(backup.getBackupOfferingId());
if (offering == null) {
String errorMessage = "Failed to find backup offering of the VM backup.";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not used

String errorMessage = "Failed to find backup offering of the VM backup.";
throw new CloudRuntimeException("Failed to find backup offering of the VM backup.");
}
if ("networker".equals(offering.getProvider())) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe better to define in the provider itself - offering.getProvider().supportsInstanceFromBackup()

// The restore process is executed by a backup provider outside of ACS, I am using the catch-all (Exception) to
// ensure that no provider-side exception is missed. Therefore, we have a proper handling of exceptions, and rollbacks if needed.
} catch (Exception e) {
logger.error(String.format("Failed to restore backup [%s] due to: [%s].", backupDetailsInMessage, e.getMessage()), e);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ActionEventUtils.onFailedActionEvent?

}
// The restore process is executed by a backup provider outside of ACS, I am using the catch-all (Exception) to
// ensure that no provider-side exception is missed. Therefore, we have a proper handling of exceptions, and rollbacks if needed.
} catch (Exception e) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

possible to avoid catch all?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shwstppr I can catch specific exceptions, but will it be better to catch all exceptions - handle them and then rethrow the same exception ? In case any unknown exceptions are thrown by the providers (doesn't seem possible now, but maybe in the future)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@abh1sar up to you. Catching all will also mask any NPEs.

Copy link

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

Copy link
Contributor

@shwstppr shwstppr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code LGTM

@abh1sar only one diskoffering - zone link edge case seems to be remaining for testing/changes
Please see if we need to add some documentation for the feature

@blueorangutan package

@abh1sar abh1sar mentioned this pull request Jan 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants