Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-25079][python] update python3 executable to 3.6.x #24266

Closed
wants to merge 2 commits into from
Closed

[SPARK-25079][python] update python3 executable to 3.6.x #24266

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1d14800
update python3 executable to 3.6.x
shaneknapp Apr 1, 2019
ba77b41
update comment for future shane and new jenkins workers
shaneknapp Apr 17, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion dev/run-tests.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -546,7 +546,8 @@ def main():
hadoop_version = os.environ.get("AMPLAB_JENKINS_BUILD_PROFILE", "hadoop2.7")
test_env = "amplab_jenkins"
# add path for Python3 in Jenkins if we're calling from a Jenkins machine
os.environ["PATH"] = "/home/anaconda/envs/py3k/bin:" + os.environ.get("PATH")
# TODO(sknapp): s/py36/py3k before merging!
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK by me; does this need to change before you merge?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah... this is so i can test against the py36 conda environment on the jenkins workers. right before i merge, i'll pause jenkins, update the conda env name, push my revert and then let jenkins build again.

folks will need to update their PRs post-merge as well.

i'm waiting on @BryanCutler and the arrow 0.12.1 release before doing any of this, however. probably early to mid next week.

there will be a detailed email going out to dev@ about this on monday.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this also reminds me that one day, i would love to go through everything and remove all mention of amplab. that project has been over since 2016! :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

quick update: py36 will actually remain the path for this python env on master...

os.environ["PATH"] = "/home/anaconda/envs/py36/bin:" + os.environ.get("PATH")
else:
# else we're running locally and can use local settings
build_tool = "sbt"
Expand Down
2 changes: 1 addition & 1 deletion python/run-tests.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,7 @@ def run_individual_python_test(target_dir, test_name, pyspark_python):


def get_default_python_executables():
python_execs = [x for x in ["python2.7", "python3.4", "pypy"] if which(x)]
python_execs = [x for x in ["python2.7", "python3.6", "pypy"] if which(x)]
if "python2.7" not in python_execs:
LOGGER.warning("Not testing against `python2.7` because it could not be found; falling"
" back to `python` instead")
Expand Down