Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: python3 support for configure #30047

Closed
wants to merge 1 commit into from

Conversation

rvagg
Copy link
Member

@rvagg rvagg commented Oct 21, 2019

Discovered while messing with some infra switching things around to python3. On Linux, if you call configure directly with a python3 (not letting it find python for you) and have LC_ALL=C set, as we do in some of our infra, apparently open() defaults to ascii. This is a problem for icu version detection because there's a © at the top of deps/icu-small/source/common/unicode/uvernum.h which it reads to find major version number.

$ LC_ALL=C python3 ./configure.py
Traceback (most recent call last):
  File "./configure.py", line 1660, in <module>
    configure_intl(output)
  File "./configure.py", line 1498, in configure_intl
    for line in open(uvernum_h).readlines():
  File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 3: ordinal not in range(128)

This change fixes it and makes it work for Python 2 & 3. Although I don't really know what I'm doing with the conversion back to ascii for later compatibility so someone who knows better should check.

@rvagg rvagg requested review from srl295 and cclauss October 21, 2019 11:38
@nodejs-github-bot nodejs-github-bot added the build Issues and PRs related to build files or the CI. label Oct 21, 2019
@nodejs-github-bot
Copy link
Collaborator

@targos
Copy link
Member

targos commented Oct 21, 2019

This is similar to #30023

@rvagg
Copy link
Member Author

rvagg commented Oct 21, 2019

indeed it is! but in this case the return value needs to mix with non-utf8 strings and python2 borks at that, hence the str(x).

@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

Copy link
Contributor

@cclauss cclauss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Minor nit: for line in open(uvernum_h).readlines(): leaves the filehandle open so the preferred approach is:

with open(uvernum_h) as in_file:
    for line in in_file:

because the with open() as context manager will automate the filehandle close().

@rvagg
Copy link
Member Author

rvagg commented Oct 21, 2019

ok, have tried that approach, ptal @cclauss

@nodejs-github-bot
Copy link
Collaborator

@cclauss
Copy link
Contributor

cclauss commented Oct 21, 2019

I have already approved. For those who want to dig in...
https://www.python.org/dev/peps/pep-0538/ and https://www.python.org/dev/peps/pep-0540/

@rvagg rvagg force-pushed the rvagg/python3-icu-utf8 branch from 40c91e1 to 88ecfee Compare October 23, 2019 00:23
@nodejs-github-bot
Copy link
Collaborator

rvagg added a commit that referenced this pull request Oct 23, 2019
PR-URL: #30047
Reviewed-By: Christian Clauss <cclauss@me.com>
Reviewed-By: Sam Roberts <vieuxtech@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
@rvagg
Copy link
Member Author

rvagg commented Oct 23, 2019

Landed in 779d7ef

@rvagg rvagg closed this Oct 23, 2019
@rvagg rvagg deleted the rvagg/python3-icu-utf8 branch October 23, 2019 23:29
targos pushed a commit that referenced this pull request Oct 24, 2019
PR-URL: #30047
Reviewed-By: Christian Clauss <cclauss@me.com>
Reviewed-By: Sam Roberts <vieuxtech@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
@targos targos mentioned this pull request Nov 5, 2019
targos pushed a commit that referenced this pull request Nov 8, 2019
PR-URL: #30047
Reviewed-By: Christian Clauss <cclauss@me.com>
Reviewed-By: Sam Roberts <vieuxtech@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
targos pushed a commit that referenced this pull request Nov 10, 2019
PR-URL: #30047
Reviewed-By: Christian Clauss <cclauss@me.com>
Reviewed-By: Sam Roberts <vieuxtech@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
targos pushed a commit that referenced this pull request Nov 11, 2019
PR-URL: #30047
Reviewed-By: Christian Clauss <cclauss@me.com>
Reviewed-By: Sam Roberts <vieuxtech@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: David Carlier <devnexen@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build Issues and PRs related to build files or the CI.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants