upload-and-deploy

Decouple S3 to JSON workflow from JSON to Parquet workflow #409

Sign in to view logs

Triggered via push October 13, 2023 15:57

philerooski

etl-568

Status Success

Total duration 20m 25s

Artifacts –

upload-and-deploy.yaml

on: push

Run pre-commit hooks against all files

Matrix: Build and push testing docker images to the pytest ECR repository.

Upload files to S3 bucket in development

Runs unit tests that are not dependent on aws-glue package resources

Matrix: Run Pytest unit tests for AWS glue

Deploys branch using sceptre

Triggers ETL workflow with S3 test files

Deploys to staging of prod using sceptre

Triggers staging workflow with production data

Annotations

5 warnings

Run pre-commit hooks against all files

The `python-version` input is not set. The version of Python currently in `PATH` will be used.

Build and push testing docker images to the pytest ECR repository. (aws_glue_3, tests/Dockerfile....

Your docker password is not masked. See https://github.com/aws-actions/amazon-ecr-login#docker-credentials for more information.

Build and push testing docker images to the pytest ECR repository. (aws_glue_4, tests/Dockerfile....

Your docker password is not masked. See https://github.com/aws-actions/amazon-ecr-login#docker-credentials for more information.

Deploys branch using sceptre

The following actions uses node12 which is deprecated and will be forced to run on node16: gagoar/invoke-aws-lambda@v3. For more info: https://github.blog/changelog/2023-06-13-github-actions-all-actions-will-run-on-node16-instead-of-node12-by-default/

Deploys branch using sceptre

The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/