Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(docker/awscli): upgrade to python3 and install python3-dev #1140

Merged
merged 2 commits into from
Apr 19, 2021

Conversation

zwliew
Copy link
Contributor

@zwliew zwliew commented Apr 15, 2021

Problem

A recent change (not sure what) started causing pip install awscli to
fail without the python2-dev package. This causes backend and worker
builds on Travis to fail.

Travis build log: https://travis-ci.com/github/opengovsg/postmangovsg/jobs/498665286

Solution

Improvements:

  • Upgrade to python3 and remove py-pip
  • Install python3-dev on the backend and worker Docker instances

Before & After Screenshots

There are no visual changes.

Tests

  1. Deploy the change.
  2. Ensure that the backend and worker builds and deployments on Travis, AWS EBS, and AWS ECS succeed.

Deploy Notes

There are no deploy notes.

Todo

  • Look into upgrading Python from 2.7 to 3.x
  • Look into upgrading AWS CLI from v1 to v2 (which eliminates the dependency on a system installation of Python)

Notes regarding AWS CLI v2

There is a prebuilt binary, but it depends on glibc. Our Docker image (Alpine) only includes musl libc, so we can't use the binary as is.

Options:

  1. Install glibc (([v2] Distribute binaries for alpine / musl libc aws/aws-cli#4685 (comment)))
  2. Build the binary from scratch using musl ([v2] Distribute binaries for alpine / musl libc aws/aws-cli#4685 (comment)))

Both of these options seem more trouble than they're worth, so I've decided not to upgrade to v2 for now.

References

zwliew added 2 commits April 15, 2021 14:44
A recent change (not sure what) started causing `pip install awscli` to
fail without the python2-dev package.

Install it to fix the build errors.

Travis build log: https://travis-ci.com/github/opengovsg/postmangovsg/jobs/498665286
Note:
1. Removed `py-pip` since it seems unnecessary
2. Use `python3 -m pip` instead of `pip` to avoid using Python2's pip
   (unlikely since Python2 should not be installed, but just in case)
@zwliew zwliew changed the title fix(dockerfile): install python2-dev where awscli is needed fix(dockerfile): upgrade to python3 and install python3-dev where awscli is needed Apr 15, 2021
@zwliew zwliew changed the title fix(dockerfile): upgrade to python3 and install python3-dev where awscli is needed fix(docker/awscli): upgrade to python3 and install python3-dev Apr 15, 2021
@zwliew zwliew marked this pull request as ready for review April 15, 2021 08:25
Copy link
Contributor

@lamkeewei lamkeewei left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm! Thanks for fixing this.

@lamkeewei lamkeewei merged commit 20c62a0 into develop Apr 19, 2021
@lamkeewei lamkeewei deleted the fix-missing-docker-dep branch April 19, 2021 04:15
lamkeewei added a commit that referenced this pull request Apr 19, 2021
* develop:
  fix(docker/awscli): upgrade to python3 and install python3-dev (#1140)
  feat(frontend): set up test fixtures and deployment processes (#1086)
lamkeewei added a commit that referenced this pull request Apr 19, 2021
* develop:
  fix(docker/awscli): upgrade to python3 and install python3-dev (#1140)
  feat(frontend): set up test fixtures and deployment processes (#1086)
  feat: allow activation of SNS as fallback for all SMS campaigns (#1019)
  1.22.4
  fix: re-fetch campaign details after campaign is sent (#1116)
  chore: upgrade dependencies (#1120)
  chore: fix build warnings (#1103)
  fix(moment): disable periodic updates for react-moment components (#1127)
  fix: avoid calling setState when components are unmounted (#1109)
  chore: add callback server (#1112)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants