build: allow build with system python 3 #16058

forivall · 2017-10-07T00:21:48Z

When the system python is python 3, configure now creates a directory
with a symlink called 'python' to python2, uses it when it calls
run_gyp, and puts it in config.mk so that it propagates to everything
that make launches

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
commit message follows commit guidelines

Affected core subsystem(s)

build

forivall · 2017-10-07T00:22:09Z

Waiting for CI to determine if changes are needed for windows.

gibfahn · 2017-10-07T00:28:27Z

CI: https://ci.nodejs.org/job/node-test-commit/12832/

EDIT: CI was green.

gibfahn · 2017-10-07T03:58:08Z

cc/ @nodejs/build @nodejs/python

bnoordhuis · 2017-10-07T09:48:22Z

configure

+      raise e
+  os.symlink(sys.executable, python_link)
+
+  os.environ['PATH'] = bin_override + ':' + os.environ['PATH']


This is a no-op as far as I can tell, it doesn't affect the parent shell.

gyp does a "python -c" call, and this is required for it to work. Will add comment

bnoordhuis · 2017-10-07T09:54:03Z

configure

+bin_override = None
+
+def setup_bin_overrides():
+  global bin_override


What is the reason you use a global instead of returning a value?

It mirrored what other stages were doing, with setting values in "o". Returning makes more sense, will change

bnoordhuis · 2017-10-07T09:56:38Z

configure

+  bin_override = os.path.abspath('out/tools/bin')
+  try:
+    os.makedirs(bin_override)
+  except os.error as e:


For consistency with the rest of the file, can you use OSError in this function?

bnoordhuis · 2017-10-07T09:56:50Z

configure

@@ -1428,6 +1458,10 @@ if options.prefix:

 config = '\n'.join(map('='.join, config.iteritems())) + '\n'

+setup_bin_overrides()
+if bin_override:
+  config = 'export PATH:=' + bin_override + ':${PATH}\n' + config


Out of curiosity, why ${PATH} and not $(PATH)?

It's what was on the stack overflow answer. 😝 Will change if I can

When the system python is python 3, configure now creates a directory with a symlink called 'python' to python2, uses it when it calls run_gyp, and puts it in config.mk so that it propagates to everything that make launches

forivall · 2017-10-07T18:47:07Z

Updated for reviews. Diff @ forivall@528a99d

bnoordhuis

LGTM. CI: https://ci.nodejs.org/job/node-test-pull-request/10497/

gibfahn · 2017-10-08T14:45:17Z

Landed in 2b5b423.

Thanks for fixing this @forivall , and congrats on becoming a Contributor 🎉 .

When the system python is python 3, configure now creates a directory with a symlink called 'python' to python2, uses it when it calls run_gyp, and puts it in config.mk so that it propagates to everything that make launches PR-URL: #16058 Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Bryan English <bryan@bryanenglish.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>

When the system python is python 3, configure now creates a directory with a symlink called 'python' to python2, uses it when it calls run_gyp, and puts it in config.mk so that it propagates to everything that make launches PR-URL: nodejs/node#16058 Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Bryan English <bryan@bryanenglish.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>

hefangshi · 2017-10-12T08:55:57Z

This change made Node.js build need install distutils packages which is a dependency change, python3 won't break old build environment but distutils dependency will, maybe we can implement a less dependent method to find default python binary?

bnoordhuis · 2017-10-12T09:39:22Z

@hefangshi distutils is part of python's stdlib. Are you thinking of setuptools?

gibfahn · 2017-10-12T19:00:53Z

Looks like distutils is included by default from Python 2.6 onwards, but maybe not in some distributions (looking at https://stackoverflow.com/questions/3810521/how-to-install-python-distutils).

@hefangshi what platform and version of Python are you on?

Also this commit doesn't add the distutils dependency, it was there before (from the diff 2b5b423):

gibfahn · 2017-10-16T21:20:11Z

+1 on landing on a future v6.x release, and it should land with #16241 (assuming that lands).

forivall · 2017-10-16T22:30:29Z

Yeah, i had assumed the distutils would be fine since it would be loaded if it was python 3 or <= 2.5. there's probably an alternative version of 'which' that doesn't depend on distutils, since that's the only thing it's used for.

MylesBorins · 2017-10-17T20:19:52Z

waiting for #16241 before landing this on v6.x. Please LMK if we don't need to wait

When the system python is python 3, configure now creates a directory with a symlink called 'python' to python2, uses it when it calls run_gyp, and puts it in config.mk so that it propagates to everything that make launches PR-URL: #16058 Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Bryan English <bryan@bryanenglish.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>

Notable Changes: * build: - fix npm install with --shared (Ben Noordhuis) #16438 * build: - building with python 3 is now supported (Emily Marigold Klassen) #16058 * src: - v8 options can be specified with either '\_' or '-' in NODE_OPTIONS (Sam Roberts) #14093 PR-URL: #17180

nodejs-github-bot added the build Issues and PRs related to build files or the CI. label Oct 7, 2017

gibfahn approved these changes Oct 7, 2017

View reviewed changes

Trott added the code-and-learn Issues related to the Code-and-Learn events and PRs submitted during the events. label Oct 7, 2017

benjamingr approved these changes Oct 7, 2017

View reviewed changes

bnoordhuis reviewed Oct 7, 2017

View reviewed changes

addaleax approved these changes Oct 7, 2017

View reviewed changes

build: allow build with system python 3

741a772

When the system python is python 3, configure now creates a directory with a symlink called 'python' to python2, uses it when it calls run_gyp, and puts it in config.mk so that it propagates to everything that make launches

forivall force-pushed the python2-bin-stub branch from 73bcd94 to 741a772 Compare October 7, 2017 18:43

addaleax approved these changes Oct 7, 2017

View reviewed changes

bengl approved these changes Oct 7, 2017

View reviewed changes

forivall mentioned this pull request Oct 7, 2017

build: allow build with system python 3 ayojs/ayo#97

Closed

2 tasks

bnoordhuis approved these changes Oct 8, 2017

View reviewed changes

gibfahn closed this Oct 8, 2017

MylesBorins mentioned this pull request Oct 11, 2017

v8.7.0 proposal #15762

Merged

gibfahn added the lts-watch-v6.x label Oct 16, 2017

gireeshpunathil mentioned this pull request Oct 17, 2017

build: use bin path override if no python is found in PATH #16241

Closed

2 tasks

gibfahn mentioned this pull request Oct 23, 2017

configure fails on linux for distributions using python 3 as default #9512

Closed

MylesBorins added land-on-v6.x and removed lts-watch-v6.x labels Nov 14, 2017

MylesBorins mentioned this pull request Nov 21, 2017

v6.12.1 proposal #17180

Merged

corwin-of-amber mentioned this pull request Dec 4, 2018

passing python2.7 path doesnot have effect nwjs/nw-gyp#122

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

build: allow build with system python 3 #16058

build: allow build with system python 3 #16058

forivall commented Oct 7, 2017

forivall commented Oct 7, 2017

gibfahn commented Oct 7, 2017 •

edited

Loading

gibfahn commented Oct 7, 2017

bnoordhuis Oct 7, 2017

forivall Oct 7, 2017

bnoordhuis Oct 7, 2017

forivall Oct 7, 2017

bnoordhuis Oct 7, 2017

bnoordhuis Oct 7, 2017

forivall Oct 7, 2017

forivall commented Oct 7, 2017

bnoordhuis left a comment

gibfahn commented Oct 8, 2017

hefangshi commented Oct 12, 2017

bnoordhuis commented Oct 12, 2017

gibfahn commented Oct 12, 2017

gibfahn commented Oct 16, 2017

forivall commented Oct 16, 2017 •

edited

Loading

MylesBorins commented Oct 17, 2017

build: allow build with system python 3 #16058

build: allow build with system python 3 #16058

Conversation

forivall commented Oct 7, 2017

Checklist

Affected core subsystem(s)

forivall commented Oct 7, 2017

gibfahn commented Oct 7, 2017 • edited Loading

gibfahn commented Oct 7, 2017

bnoordhuis Oct 7, 2017

Choose a reason for hiding this comment

forivall Oct 7, 2017

Choose a reason for hiding this comment

bnoordhuis Oct 7, 2017

Choose a reason for hiding this comment

forivall Oct 7, 2017

Choose a reason for hiding this comment

bnoordhuis Oct 7, 2017

Choose a reason for hiding this comment

bnoordhuis Oct 7, 2017

Choose a reason for hiding this comment

forivall Oct 7, 2017

Choose a reason for hiding this comment

forivall commented Oct 7, 2017

bnoordhuis left a comment

Choose a reason for hiding this comment

gibfahn commented Oct 8, 2017

hefangshi commented Oct 12, 2017

bnoordhuis commented Oct 12, 2017

gibfahn commented Oct 12, 2017

gibfahn commented Oct 16, 2017

forivall commented Oct 16, 2017 • edited Loading

MylesBorins commented Oct 17, 2017

gibfahn commented Oct 7, 2017 •

edited

Loading

forivall commented Oct 16, 2017 •

edited

Loading