Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add python script and requirements to automatically create daily backups of gcp filestores #4417

Merged
merged 9 commits into from
Jul 12, 2024

Conversation

This is a positional argument, so if it's a hyphen or an underscore is irrelevant
Copy link
Member

@yuvipanda yuvipanda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The gist of this looks right to me for a reconciliation loop! Thanks for working on this, @sgibson91.

I'd suggest to make it clearer, the main function be split into separate functions that do the various parts of the loop (get_existing_backups, create_backup_if_necessary, delete_old_backups).

# return immediately, without waiting for the operation in progress
# to complete. Given that we only expect to be creating a backup
# once a day, this feels safe enough to try for now.
"--async",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think calling it --async is fine, as long as our logic to detect 'do we need to create another backup right now?' takes into account incomplete backups. We want to avoid running into the rate limit and also avoid endlessly creating a ton of backups (as that can cost money).

Copy link
Member Author

@sgibson91 sgibson91 Jul 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked and it looks like this is handled on GCP's side. I ran the script twice in a row (and remember the script is programmed to run every 10mins), and the returned JSON was already populated with the backup just created, even though it's state was "FINALIZING". So I didn't need to make any tweaks to the script here.

$ python gcp-filestore-backups.py pilot-hubs-homedirs two-eye-two-see us-central1-b
There have been no recent backups of the filestore for project two-eye-two-see. Creating a backup now...
Create request issued
Check operation [projects/two-eye-two-see/locations/us-central1/operations/operation-1720791941383-61d0d18660679-d749fa00-3bdf9958] for status. 
No outdated backups found.
...

$ python gcp-filestore-backups.py pilot-hubs-homedirs two-eye-two-see us-central1-b
[
  {
    "capacityGb": "5120",
    "createTime": "2024-07-12T13:45:41.576982250Z",
    "name": "projects/two-eye-two-see/locations/us-central1/backups/pilot-hubs-homedirs-homes-backup-2024-07-12",
    "sourceFileShare": "homes",
    "sourceInstance": "projects/two-eye-two-see/locations/us-central1-b/instances/pilot-hubs-homedirs",
    "sourceInstanceTier": "BASIC_HDD",
    "state": "FINALIZING"
  }
]
Recent backup found.
No outdated backups found.
...

@sgibson91 sgibson91 marked this pull request as ready for review July 12, 2024 13:56
@sgibson91 sgibson91 merged commit 72cb723 into 2i2c-org:main Jul 12, 2024
7 of 8 checks passed
@sgibson91 sgibson91 deleted the script-gcp-filestore-backups branch July 12, 2024 13:56
Copy link

🎉🎉🎉🎉

Monitor the deployment of the hubs here 👉 https://github.com/2i2c-org/infrastructure/actions/runs/9909050922

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Create a python script (gcp-filestore-backups) to perform backups and retention of filestore instances
2 participants