Skip to content
This repository has been archived by the owner on Aug 2, 2022. It is now read-only.

Change CD workflow to use new staging bucket for artifacts #248

Closed
wants to merge 1 commit into from
Closed

Change CD workflow to use new staging bucket for artifacts #248

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 23 additions & 8 deletions .github/workflows/CD.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,11 +32,26 @@ jobs:
- name: Run build
run: |
./gradlew buildPackages --console=plain -Dbuild.snapshot=false
artifact=`ls build/distributions/*.zip`
rpm_artifact=`ls build/distributions/*.rpm`
deb_artifact=`ls build/distributions/*.deb`

aws s3 cp $artifact s3://artifacts.opendistroforelasticsearch.amazon.com/downloads/elasticsearch-plugins/opendistro-anomaly-detection/
aws s3 cp $rpm_artifact s3://artifacts.opendistroforelasticsearch.amazon.com/downloads/rpms/opendistro-anomaly-detection/
aws s3 cp $deb_artifact s3://artifacts.opendistroforelasticsearch.amazon.com/downloads/debs/opendistro-anomaly-detection/
aws cloudfront create-invalidation --distribution-id E1VG5HMIWI4SA2 --paths "/downloads/*"

- name: Upload to S3
shell: bash
run: |
zip=`ls build/distributions/*.zip`
rpm=`ls build/distributions/*.rpm`
deb=`ls build/distributions/*.deb`

# Inject the build number before the suffix
zip_outfile=`basename ${zip%.zip}-build-${GITHUB_RUN_NUMBER}.zip`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GITHUB_RUN_NUMBER is unique number for each run, so the artifact name will be unique for each run, while the original artifact name keep the same for same version. Is this change compatible with current CD workflow?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are making corresponding changes to how we generate ODFE distributions to handle the change to the naming convention. If you need to do an out-of-cycle release then there may be manual work required; the infrastructure team can help with that.

rpm_outfile=`basename ${rpm%.rpm}-build-${GITHUB_RUN_NUMBER}.rpm`
deb_outfile=`basename ${deb%.deb}-build-${GITHUB_RUN_NUMBER}.deb`

s3_prefix="s3://staging.artifacts.opendistroforelasticsearch.amazon.com/snapshot/elasticsearch-plugins/anomaly-detection/"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So all plugins are uploading artifact to staging only in CD workflow ? Don't see production location in this file. Where will the production location be used ?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The infrastructure team will promote artifacts to production as part of the ODFE release process. For out-of-cycle releases, you can promote the artifacts to production using your own workflows, or the infrastructure team can help.


echo "Copying ${zip} to ${s3_prefix}${zip_outfile}"
aws s3 cp --quiet $zip ${s3_prefix}${zip_outfile}

echo "Copying ${rpm} to ${s3_prefix}${rpm_outfile}"
aws s3 cp --quiet $rpm ${s3_prefix}${rpm_outfile}

echo "Copying ${deb} to ${s3_prefix}${deb_outfile}"
aws s3 cp --quiet $deb ${s3_prefix}${deb_outfile}