Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restore backup from specific location #5491

Merged
merged 1 commit into from
Dec 17, 2024
Merged

Restore backup from specific location #5491

merged 1 commit into from
Dec 17, 2024

Conversation

mdegat01
Copy link
Contributor

@mdegat01 mdegat01 commented Dec 17, 2024

Proposed change

Allow users to restore a backup from a specific location in case they prefer a particular mount over another (perhaps one is flaky). Request comes from the cloud backup design.

Type of change

  • Dependency upgrade
  • Bugfix (non-breaking change which fixes an issue)
  • New feature (which adds functionality to the supervisor)
  • Breaking change (fix/feature causing existing functionality to break)
  • Code quality improvements to existing code or addition of tests

Additional information

Checklist

  • The code change is tested and works locally.
  • Local tests pass. Your PR cannot be merged unless tests pass
  • There is no commented out code in this PR.
  • I have followed the development checklist
  • The code has been formatted using Ruff (ruff format supervisor tests)
  • Tests have been added to verify that the new code works.

If API endpoints or add-on configuration are added/changed:

Summary by CodeRabbit

  • New Features

    • Enhanced backup and restore functionalities with flexible location handling.
    • Asynchronous context management support for backup operations.
  • Bug Fixes

    • Improved error handling during backup creation and restoration processes.
  • Tests

    • Added tests for restoring backups from specific locations, including scenarios for missing files.
    • Updated tests to reflect changes in context management for backup operations.

@mdegat01 mdegat01 added the new-feature A new feature label Dec 17, 2024
Copy link

coderabbitai bot commented Dec 17, 2024

📝 Walkthrough

Walkthrough

The pull request introduces enhancements to the backup and restore functionalities across multiple files in the supervisor's backup system. The changes primarily focus on improving location handling during backup and restore operations, adding more flexibility for specifying backup locations. The modifications include updates to the API, backup management, and context handling, with new methods for creating and opening backups asynchronously. The changes also include corresponding test updates to validate the new functionality.

Changes

File Change Summary
supervisor/api/backups.py - Added optional ATTR_LOCATION to restore and remove schemas
- Updated location validation methods
- Modified backup and restore methods to handle location attributes more robustly
supervisor/backups/backup.py - Introduced asynchronous context management
- Added create() and open() methods
- Removed previous __aenter__ and __aexit__ methods
- Updated backup file creation and restoration logic
supervisor/backups/manager.py - Updated restore methods to support optional location parameter
- Modified context management for backup operations
- Enhanced error handling and logging for backup processes
tests/api/test_backups.py - Added new test function test_restore_backup_from_location
- Tested backup restoration from specific locations
tests/backups/test_backup.py - Updated backup test to use new backup.create() method

Sequence Diagram

sequenceDiagram
    participant API as Backup API
    participant Manager as BackupManager
    participant Backup as Backup Class
    
    API->>Manager: Request backup/restore
    Manager->>Backup: Create or Open backup
    alt Backup Creation
        Backup-->>Manager: Prepare backup file
    else Backup Restoration
        Backup->>Manager: Validate location
        alt Location Specified
            Manager->>Backup: Restore from specified location
        else Default Location
            Manager->>Backup: Restore from default location
        end
    end
    Manager-->>API: Confirm backup/restore operation
Loading

Tip

CodeRabbit's docstrings feature is now available as part of our Early Access Program! Simply use the command @coderabbitai generate docstrings to have CodeRabbit automatically generate docstrings for your pull request. We would love to hear your feedback on Discord.


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR. (Beta)
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai or @coderabbitai title anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (4)
supervisor/backups/backup.py (1)

6-7: Imports Update: Ensure Unused Imports are Necessary

The addition of Awaitable in the import statement may not be necessary if it is not used elsewhere in the code. Please verify and remove any unused imports to keep the code clean.

tests/api/test_backups.py (3)

823-823: Consider making test file creation more robust

The test file creation could be more robust by adding content to verify after restoration and ensuring the parent directory exists.

-(test_file := coresys.config.path_share / "test.txt").touch()
+(test_file := coresys.config.path_share / "test.txt").parent.mkdir(parents=True, exist_ok=True)
+test_file.write_text("test content")

849-852: Improve error message assertion precision

The error message assertion could be more precise by using a constant or helper method to avoid hardcoding the error message string.

-assert (
-    body["message"]
-    == f"Cannot open backup at {backup.all_locations[None].as_posix()}, file does not exist!"
-)
+# Define error message constant at module level
+BACKUP_NOT_FOUND_ERROR = "Cannot open backup at {}, file does not exist!"
+assert body["message"] == BACKUP_NOT_FOUND_ERROR.format(
+    backup.all_locations[None].as_posix()
+)

858-859: Add more comprehensive restoration verification

The test could verify more aspects of the restored file to ensure the restoration was completely successful.

 assert resp.status == 200
 assert test_file.is_file()
+# Verify file content and permissions
+assert test_file.read_text() == "test content"
+assert test_file.stat().st_mode & 0o777 == 0o644  # Verify default permissions
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 90590ae and e3cc263.

📒 Files selected for processing (5)
  • supervisor/api/backups.py (3 hunks)
  • supervisor/backups/backup.py (5 hunks)
  • supervisor/backups/manager.py (7 hunks)
  • tests/api/test_backups.py (1 hunks)
  • tests/backups/test_backup.py (1 hunks)
🔇 Additional comments (13)
supervisor/backups/backup.py (4)

380-401: Enhancement: Introduce Asynchronous Context Managers

The introduction of create and open methods with @asynccontextmanager improves the clarity and management of asynchronous operations in the Backup class. This change enhances the code's readability and follows best practices for asynchronous context management.


402-417: Consistent Error Handling in open Method

The open method properly raises a BackupError if the specified backup location does not exist or the file is missing. This ensures that errors are caught early and handled appropriately, maintaining the robustness of the backup restoration process.


Line range hint 436-459: Validation and Backup Metadata Addition

The _create_cleanup method effectively validates the backup data and adds the backup.json to the tar file. This ensures that backups are correctly formatted and contain the necessary metadata.


Line range hint 578-590: Proper Usage of the location Parameter in _do_restore

The addition of the location parameter in the _do_restore method allows for restoring backups from specified locations. The parameter is correctly passed to backup.open(location), ensuring that the backup restoration process utilizes the correct backup file.

supervisor/backups/manager.py (4)

408-408: Update Context Manager Usage with New create Method

The change from async with backup: to async with backup.create(): reflects the updated context manager implementation in the Backup class. This ensures that backups are created using the new asynchronous context manager, improving code clarity and consistency.


Line range hint 578-590: Enhanced Restore Functionality with location Parameter

The addition of the location parameter in the _do_restore method and its usage in async with backup.open(location): correctly integrates the ability to restore backups from specific locations. This change aligns with the updated backup handling and enhances the flexibility of the restore process.


Line range hint 675-714: Update do_restore_full Method with location Parameter

Including the location parameter in do_restore_full allows users to specify the backup location when performing a full restore. The method correctly passes the parameter to _do_restore, ensuring seamless integration with the updated restore logic.


Line range hint 743-784: Update do_restore_partial Method with location Parameter

The do_restore_partial method now includes the location parameter, providing consistency with the full restore method. This addition enhances the partial restore functionality by allowing location specification, and correctly passes the parameter to _do_restore.

tests/backups/test_backup.py (1)

17-20: Adjust Test to Use Updated create Context Manager

The test test_new_backup_stays_in_folder is updated to use async with backup.create():, aligning with the changes made to the Backup class. This ensures that the test accurately reflects the new backup creation process and remains valid.

supervisor/api/backups.py (3)

86-88: Addition of ATTR_LOCATION to SCHEMA_RESTORE_FULL

Including ATTR_LOCATION in SCHEMA_RESTORE_FULL allows users to specify a backup location when performing a full restore. This enhancement increases the flexibility of the API and aligns with the extended restore capabilities.


384-386: Validate Backup Location in restore_full Method

The _validate_cloud_backup_location method is appropriately called with the ATTR_LOCATION from the request body or the backup's default location. This ensures that the restore operation is authorized and the location is accessible.


404-406: Validate Backup Location in restore_partial Method

Similarly, the restore_partial method now validates the backup location, enhancing security and reliability when restoring backups. This consistent validation prevents unauthorized access to backups in restricted locations.

tests/api/test_backups.py (1)

815-859: LGTM! Well-structured test for backup restoration from specific locations

The test comprehensively covers:

  • Backup creation with multiple locations
  • Error handling for missing backups
  • Successful restoration from alternate location

supervisor/backups/backup.py Show resolved Hide resolved
@agners agners merged commit de68868 into main Dec 17, 2024
21 checks passed
@agners agners deleted the restore-from-location branch December 17, 2024 10:09
@github-actions github-actions bot locked and limited conversation to collaborators Dec 19, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants