Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question: How to associate a bucket for a future FSx for Lustre file system to quickly repopulate the reference data? #27

Open
LanceWesterhoff opened this issue May 3, 2024 · 0 comments

Comments

@LanceWesterhoff
Copy link

LanceWesterhoff commented May 3, 2024

Great job with this environment! It is [almost] exactly what we need. This may be a basic question, but we only need occasional folding predictions. If I understand the FSx for Lustre file system pricing, we're paying for the Lustre filesystem regardless of whether we're actually running any folding predictions. In your instructions, you note the following:

To remove the stack and stop further charges, first slect the root stack from the CloudFormation console and then the Delete button. This will remove all resources EXCEPT for the S3 bucket containing job data and the FSx for Lustre backup. You can associate this bucket as a data repository for a future FSx for Lustre file system to quickly repopulate the reference data.

What is the best practice to accomplish this? It would seem we need to:

  • Run Launch the Stack once to create run the default download for the database and populate the FSx for Lustre file system.
  • Then we need to some how get the data in the Lustre file system to S3. How do we do that?
  • Then we need to remove the stack (which as per the note above, should remove everything except for the S3 bucket containing the above noted data).
  • Finally, when we want to perform another prediction, we should run something to create the Stack again (perhaps the batch-protein-folding-cfn-packaged.yaml again?) but instead of performing the 5hr download, it should pull from the S3 bucket, right? How do we communicate this to the YAML script? Is there already an option to perform this step (which I'm missing)?

Thanks for your help!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant