Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Adds push and pull project steps for S3 #229

Merged
merged 7 commits into from
Apr 4, 2023
Merged

Adds push and pull project steps for S3 #229

merged 7 commits into from
Apr 4, 2023

Conversation

desertaxle
Copy link
Member

@desertaxle desertaxle commented Apr 3, 2023

Adds pull_project_from_s3 and push_project_to_s3 project steps for storing and retrieving projects in S3 buckets.

Closes #226
Closes #227

Example

Push a project to an S3 bucket:

build:
    - prefect_aws.projects.steps.push_project_to_s3:
        requires: prefect-aws
        bucket: my-bucket
        folder: my-project

Push a project to an S3 bucket using credentials stored in a block:

build:
    - prefect_aws.projects.steps.push_project_to_s3:
        requires: prefect-aws
        bucket: my-bucket
        folder: my-project
        credentials: "{{ prefect.blocks.aws-credentials.dev-credentials }}"

Pull a project from S3 using the default credentials and client parameters:

build:
    - prefect_aws.projects.steps.pull_project_from_s3:
        requires: prefect-aws
        bucket: my-bucket
        folder: my-project

Pull a project from S3 using credentials stored in a block:

build:
    - prefect_aws.projects.steps.pull_project_from_s3:
        requires: prefect-aws
        bucket: my-bucket
        folder: my-project
        credentials: "{{ prefect.blocks.aws-credentials.dev-credentials }}"

Screenshots

Screenshot 2023-04-03 at 4 01 51 PM

Checklist

  • References any related issue by including "Closes #" or "Closes ".
    • If no issue exists and your change is not a small fix, please create an issue first.
  • Includes tests or only affects documentation.
  • Passes pre-commit checks.
    • Run pre-commit install && pre-commit run --all locally for formatting and linting.
  • Includes screenshots of documentation updates.
    • Run mkdocs serve view documentation locally.
  • Summarizes PR's changes in CHANGELOG.md

@desertaxle desertaxle requested a review from cicdw April 3, 2023 16:57
@desertaxle desertaxle changed the title Adds push and pull steps for S3 Adds push and pull project steps for S3 Apr 3, 2023
@desertaxle desertaxle marked this pull request as ready for review April 3, 2023 21:06
@desertaxle desertaxle requested a review from a team April 3, 2023 21:06
Copy link
Member

@cicdw cicdw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

@desertaxle desertaxle merged commit 7b64b70 into main Apr 4, 2023
@desertaxle desertaxle deleted the project-steps branch April 4, 2023 15:40
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Create S3 Project Pull Step Create S3 Project Push Step
2 participants