Refactor travis.yml to deduplicate Pyenv and AWS pex code #7397
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Eric-Arellano
changed the title
Originally I had this from the setup repo, but realized we can't use it and must use Mustache because we need the OpenSSL env vars to be included in the anchor for things like &py36_osx_env, and we couldn't mix this anchor with that new anchor definition.
Two pants.pex found for Py36 Linux :/
stuhood
approved these changes
Mar 19, 2019
Eric-Arellano
added a commit
to Eric-Arellano/pants
that referenced
this pull request
Mar 19, 2019
stuhood
pushed a commit
that referenced
this pull request
Mar 20, 2019
### Problem Our `.travis.yml` is quite complex. While Mustache has helped with addressing this, we currently duplicate several intricate values like the command to publish `pants.pex` to AWS. While working on pantsbuild/setup#39, a better design came up to de-duplicate individual script entries through anchors. ### Solution * Introduce new `PYENV_BIN` env var to avoid having to type `${PYENV_ROOT}/bin/pyenv` multiple times. * Define Pyenv env vars globally. They do not hurt anything if we don't use Pyenv on the shard, and allow us to deduplicate code such as which version of Py3 to install. * Key detail: we first check if $PYENV_ROOT is defined and only redefine if not. Travis's Ubuntu images use Pyenv to get multiple Python versions installed, so we must do this check to avoid overwriting what comes on the system. * Introduce new globally defined commands `pyenv_setup`, `pyenv_install_py36` and `pyenv_global_py36`. * For bootstrapping `pants.pex`, introduce new globally defined commands `aws_deploy_pants_pex` and `aws_get_pants_pex`. * Only build `fs_util` on the Py27 shard. * OSX was building and deploying for every shard * Linux was building on Py36 even though we never deployed it.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
Our
.travis.ymlis quite complex. While Mustache has helped with addressing this, we currently duplicate several intricate values like the command to publishpants.pexto AWS.While working on pantsbuild/setup#39, a better design came up to de-duplicate individual script entries through anchors.
Solution
PYENV_BINenv var to avoid having to type${PYENV_ROOT}/bin/pyenvmultiple times.pyenv_setup,pyenv_install_py36andpyenv_global_py36.pants.pex, introduce new globally defined commandsaws_deploy_pants_pexandaws_get_pants_pex.fs_utilon the Py27 shard.