-
-
Notifications
You must be signed in to change notification settings - Fork 610
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix unicode encoding errors inside pip #567
Conversation
On Python 2, if a distribution package contains non-ASCII file names, the Resolver failed with the following UnicodeDecodeError, because RequirementSet was initialized with unicode paths rather than str. Fix this by making sure that the path components used in PyPIRepository are all filesystem encoded str. Drop the unicode_literals import from `_compat/tempfile.py`, because the strings used in there should also be str, not unicode. Also make sure that the InstallRequirement is always constructed from str, not unicode. Traceback of the error looked like this: Traceback (most recent call last): File ".../piptools/scripts/compile.py", line 184, in cli results = resolver.resolve(max_rounds=max_rounds) File ".../piptools/resolver.py", line 107, in resolve has_changed, best_matches = self._resolve_one_round() File ".../piptools/resolver.py", line 195, in _resolve_one_round for dep in self._iter_dependencies(best_match): File ".../piptools/resolver.py", line 274, in _iter_dependencies dependencies = self.repository.get_dependencies(ireq) File ".../piptools/repositories/pypi.py", line 145, in get_dependencies self._dependencies_cache[ireq] = reqset._prepare_file(self.finder, ireq) File ".../pip/req/req_set.py", line 620, in _prepare_file session=self.session, hashes=hashes) File ".../pip/download.py", line 821, in unpack_url hashes=hashes File ".../pip/download.py", line 663, in unpack_http_url unpack_file(from_path, location, content_type, link) File ".../pip/utils/__init__.py", line 605, in unpack_file untar_file(filename, location) File ".../pip/utils/__init__.py", line 551, in untar_file path = os.path.join(location, fn) File "/usr/local/lib/python2.7/posixpath.py", line 73, in join path += '/' + b UnicodeDecodeError: 'ascii' codec can't decode byte 0xe3 in position 66: ordinal not in range(128) Fixes jazzband#564
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you add the bugfix in the changelog? That fix deserves a mention.
For the change, LGTM
Checked out your branch, tested, it works. |
I noticed your branch isn't based on master. I'll rebase your branch, add the changelog entry and submit a PR, that'll save you the trouble. Thanks for the nice work! |
Fix unicode encoding errors inside pip (Copy of #567)
"Merged" by #568 |
On Python 2, if a distribution package contains non-ASCII file names,
the Resolver failed with the following UnicodeDecodeError, because
RequirementSet was initialized with unicode paths rather than str.
Fix this by making sure that the path components used in PyPIRepository
are all filesystem encoded str.
Drop the unicode_literals import from
_compat/tempfile.py
, because thestrings used in there should also be str, not unicode.
Also make sure that the InstallRequirement is always constructed from
str, not unicode.
Traceback of the error looked like this:
Fixes #564