Fix Travis runs with Python 3 #1843

joaocgreis · 2019-07-23T15:54:18Z

Checklist

npm install && npm test passes
commit message follows commit guidelines

Description of change

This fixes the remaining issues with test-find-python.js and running on Windows when using Python 3. This makes node-gyp compatible with Python 3 for everything that we run in CI (which is not that much).

@cclauss the fixes here seemed logical to me, but I don't have much experience with Python, so a careful review would be very welcome.

This PR is semver-patch, using Python 3 still depends on EXPERIMENTAL_NODE_GYP_PYTHON3. I am preparing a different PR to address that.

joaocgreis · 2019-07-23T15:54:51Z

CI: https://ci.nodejs.org/view/All/job/nodegyp-test-pull-request/151/

cclauss

Very, very impressive! Thanks massively!

MattIPv4

lgtm.

cclauss · 2019-07-24T15:54:39Z

It would be of interest to know if the Travis tests would pass on Python 3 on macOS. Please consider adding the changes from #1846 into this PR.

richardlau · 2019-07-24T16:53:09Z

test/test-find-python.js

-      t.strictEqual(stdout, '')
-      t.ok(/Python 2/.test(stderr))
+      t.ok(stdout === '' || stderr === '')
+      t.ok(/Python 2/.test(stderr) || /Python 3/.test(stdout))


Non-blocking but I think these two tests are going to be hard to debug if they fail (in the previous code if stdout wasn't equal to '' the contents would be included in the test failure).

Would it be possible to refactor this so that if we're on Python 3, we run checks for that, which are strict and so output contents if they fail, otherwise we run stricts for Python 2?

Updated, PTAL.

Refs: nodejs#1846 PR-URL: nodejs#1843

PR-URL: nodejs#1843

joaocgreis · 2019-07-25T00:01:11Z

Updated to include the macOS Travis run as discussed in #1846.

@rvagg @MattIPv4 @richardlau let me know if you have any objection.

cclauss · 2019-07-25T01:47:45Z

.travis.yml

@@ -44,6 +45,12 @@ matrix:
      python: 3.7
      env: NODE_GYP_FORCE_PYTHON=python3 EXPERIMENTAL_NODE_GYP_PYTHON3=1
      before_install: nvm install 12
+    - name: "Python 3.7 on macOS"
+      os: osx
+      osx_image: xcode11


I know that I am trying your patience but... can you please do me a favor and comment out line 50. The error messages (and investigations elsewhere) suggest to me that using the current xcode11 might be part of our problem.

https://docs.travis-ci.com/user/reference/osx/#xcode-version

+No receipt for 'com.apple.pkg.DeveloperToolsCLILeo' found at '/'. +No receipt for 'com.apple.pkg.DeveloperToolsCLI' found at '/'. +gyp: No Xcode or CLT version detected!

No problem. Pushed a commit just to get a Travis run, I will discard it if xcode11 is not the problem.

The Python 2 version also uses xcode11, so I don't think this will fix it. If it does, we have to understand why it's not a problem with Python 2.

cclauss · 2019-07-25T02:24:49Z

It moved us from 6 failing down to 3 failing. Thanks much!

This PR is a sold step forward.

@gibfahn Can you provide us any guidance on these three failures which appear when Travis CI uses the current osx_image: xcode11. This PR gets Linux and Windows passing our Travis CI tests on Python 3.7 so it would be great to add macOS to that list.

Refs: #1846 PR-URL: #1843 Reviewed-By: João Reis <reis@janeasystems.com>

Fixes: #1826 PR-URL: #1843 Reviewed-By: Christian Clauss <cclauss@me.com> Reviewed-By: Rod Vagg <r@va.gg> Reviewed-By: Matt Cowley <me@mattcowley.co.uk> Reviewed-By: Richard Lau <riclau@uk.ibm.com>

PR-URL: #1843 Reviewed-By: Christian Clauss <cclauss@me.com> Reviewed-By: Rod Vagg <r@va.gg> Reviewed-By: Matt Cowley <me@mattcowley.co.uk> Reviewed-By: Richard Lau <riclau@uk.ibm.com>

joaocgreis · 2019-07-26T15:04:44Z

CI: https://ci.nodejs.org/view/All/job/nodegyp-test-pull-request/153/ ✔️

Landed in 5459eca...2592036.

Thanks for the reviews!

@cclauss left the xcode line commented. Notice the number of passing tests is the same, so probably when a test fails we can't really consider the remaining tests in that file. xcode11 works with Python 2, so it may make sense to duplicate the Python 3 entry, with and without xcode.

For reference:

Run with xcode11: https://travis-ci.com/nodejs/node-gyp/jobs/219308516
Run without xcode: https://travis-ci.com/nodejs/node-gyp/jobs/219321367

cclauss · 2019-07-26T15:23:51Z

Prefect. Thanks for this work. Legacy Python continues to pass no matter what.

On Python 3 on macOS xcode 9.4.1: 3 failing
On Python 3 on macOS xcode 11: 6 failing

Will add tests for both over the weekend.

Refs: #1846 PR-URL: #1843 Reviewed-By: João Reis <reis@janeasystems.com>

Fixes: #1826 PR-URL: #1843 Reviewed-By: Christian Clauss <cclauss@me.com> Reviewed-By: Rod Vagg <r@va.gg> Reviewed-By: Matt Cowley <me@mattcowley.co.uk> Reviewed-By: Richard Lau <riclau@uk.ibm.com>

PR-URL: #1843 Reviewed-By: Christian Clauss <cclauss@me.com> Reviewed-By: Rod Vagg <r@va.gg> Reviewed-By: Matt Cowley <me@mattcowley.co.uk> Reviewed-By: Richard Lau <riclau@uk.ibm.com>

Refs: nodejs/node-gyp#1820 Refs: nodejs/node-gyp#1843

Refs: nodejs/node-gyp#1820 Refs: nodejs/node-gyp#1843 PR-URL: #29897 Reviewed-By: Christian Clauss <cclauss@me.com> Reviewed-By: Sam Roberts <vieuxtech@gmail.com>

Refs: nodejs/node-gyp#1820 Refs: nodejs/node-gyp#1843 PR-URL: nodejs#29897 Reviewed-By: Christian Clauss <cclauss@me.com> Reviewed-By: Sam Roberts <vieuxtech@gmail.com> (cherry picked from commit 66b9532)

joaocgreis mentioned this pull request Jul 23, 2019

Accept Python 3 by default #1844

Closed

2 tasks

cclauss approved these changes Jul 23, 2019

View reviewed changes

cclauss mentioned this pull request Jul 24, 2019

WIP: Python 3 on Linux passes Travis CI tests #1839

Closed

4 tasks

rvagg approved these changes Jul 24, 2019

View reviewed changes

MattIPv4 approved these changes Jul 24, 2019

View reviewed changes

cclauss requested a review from bnoordhuis July 24, 2019 15:59

richardlau reviewed Jul 24, 2019

View reviewed changes

cclauss mentioned this pull request Jul 24, 2019

build: add test run Node.js 12 & Python 3.7 on macOS #1846

Closed

4 tasks

richardlau approved these changes Jul 24, 2019

View reviewed changes

cclauss and others added 3 commits July 24, 2019 22:43

build: add test run Node.js 12 & Python 3.7 on macOS

81ebc1a

Refs: nodejs#1846 PR-URL: nodejs#1843

test: accept Python 3 in test-find-python.js

0a4ee0f

PR-URL: nodejs#1843

gyp: Python 3 Windows fixes

5c89edb

PR-URL: nodejs#1843

joaocgreis force-pushed the joaocgreis-J7N-python3-2 branch from a7b1d46 to 5c89edb Compare July 24, 2019 21:47

fixup: macOS actually uses Node 10

6ff85b5

cclauss reviewed Jul 25, 2019

View reviewed changes

try without xcode11

3428750

joaocgreis pushed a commit that referenced this pull request Jul 26, 2019

build: add test run Python 3.7 on macOS

a2bca07

Refs: #1846 PR-URL: #1843 Reviewed-By: João Reis <reis@janeasystems.com>

joaocgreis added a commit that referenced this pull request Jul 26, 2019

gyp: Python 3 Windows fixes

2592036

PR-URL: #1843 Reviewed-By: Christian Clauss <cclauss@me.com> Reviewed-By: Rod Vagg <r@va.gg> Reviewed-By: Matt Cowley <me@mattcowley.co.uk> Reviewed-By: Richard Lau <riclau@uk.ibm.com>

joaocgreis closed this Jul 26, 2019

rvagg mentioned this pull request Sep 26, 2019

Release proposal: v6.0.0 #1892

Closed

rvagg pushed a commit that referenced this pull request Sep 26, 2019

build: add test run Python 3.7 on macOS

1267b4d

Refs: #1846 PR-URL: #1843 Reviewed-By: João Reis <reis@janeasystems.com>

rvagg pushed a commit that referenced this pull request Sep 26, 2019

gyp: Python 3 Windows fixes

8bcb1fb

PR-URL: #1843 Reviewed-By: Christian Clauss <cclauss@me.com> Reviewed-By: Rod Vagg <r@va.gg> Reviewed-By: Matt Cowley <me@mattcowley.co.uk> Reviewed-By: Richard Lau <riclau@uk.ibm.com>

rvagg mentioned this pull request Sep 26, 2019

Release proposal: v5.0.4 #1893

Closed

targos added a commit to targos/node that referenced this pull request Oct 9, 2019

tools: port Python 3 compat patches from node-gyp to gyp

ad02117

Refs: nodejs/node-gyp#1820 Refs: nodejs/node-gyp#1843

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix Travis runs with Python 3 #1843

Fix Travis runs with Python 3 #1843

joaocgreis commented Jul 23, 2019

joaocgreis commented Jul 23, 2019

cclauss left a comment

MattIPv4 left a comment

cclauss commented Jul 24, 2019

richardlau Jul 24, 2019

MattIPv4 Jul 24, 2019

joaocgreis Jul 24, 2019

joaocgreis commented Jul 25, 2019

cclauss Jul 25, 2019 •

edited

Loading

joaocgreis Jul 25, 2019

cclauss commented Jul 25, 2019 •

edited

Loading

joaocgreis commented Jul 26, 2019

cclauss commented Jul 26, 2019 •

edited

Loading

Fix Travis runs with Python 3 #1843

Fix Travis runs with Python 3 #1843

Conversation

joaocgreis commented Jul 23, 2019

Checklist

Description of change

joaocgreis commented Jul 23, 2019

cclauss left a comment

Choose a reason for hiding this comment

MattIPv4 left a comment

Choose a reason for hiding this comment

cclauss commented Jul 24, 2019

richardlau Jul 24, 2019

Choose a reason for hiding this comment

MattIPv4 Jul 24, 2019

Choose a reason for hiding this comment

joaocgreis Jul 24, 2019

Choose a reason for hiding this comment

joaocgreis commented Jul 25, 2019

cclauss Jul 25, 2019 • edited Loading

Choose a reason for hiding this comment

joaocgreis Jul 25, 2019

Choose a reason for hiding this comment

cclauss commented Jul 25, 2019 • edited Loading

joaocgreis commented Jul 26, 2019

cclauss commented Jul 26, 2019 • edited Loading

cclauss Jul 25, 2019 •

edited

Loading

cclauss commented Jul 25, 2019 •

edited

Loading

cclauss commented Jul 26, 2019 •

edited

Loading