Skip to content

Fix encoding issues with StringIO #5

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

mdornseif
Copy link

The current code took the (unicode) view and stuffed
it into a StringIO byte stream. The resulting encoding being
seen by PythonTidy was dependent on the configuration of the
system the code was running on and on implementation
details in StringIO.

It than read the result (a bytestream) and wrote it into
the (unicode) view. Again with indeterminable results
based on the default encodings of the host system.

For me opening the file http://filez.foxel.org/211a0I1L3t3n2I3P1L25
and pressing CTRL-ALT-CMD-T worked once but pressing it twice
resulted in an UnicodeException.

#1 shows
the same issue and https://github.com/do3cc/SublimePythonTidy/commit/efc88be
seems to be an attempt to fix it.

This changeset ensures that

a) views are treated as unicode
b) StringIOs are treated as byte stream

This fixes #1

PythonTidy also has a bunch of (not fixed so far) unicode issues,
e.g. see mdornseif/PythonTidy#1

Maximillian Dornseif added 2 commits May 27, 2012 22:05
The current code took the (unicode) view and stuffed
it into a `StringIO`. The resulting encoding beeing seen
by PythonTidy was dependant on the configuration of the
system the code was running on and on implementation
details in StringIO.

It than read the result (a bytestream) and wrote it into
the (unicode) view. Again with indeterminable results
based on the default encodings of the host system.

For me opening the file http://filez.foxel.org/211a0I1L3t3n2I3P1L25
and pressing CTRL-ALT-CMD-T worked once but pressing it twice
resulted in an `UnicodeException`.

witsch#1 shows
the same issue and https://github.com/do3cc/SublimePythonTidy/commit/efc88be
seems to be an attempt to fix it.

This changeset ensures that

a) views are treated as unicode
b) StringIOs are treated as bytestream
@softside
Copy link

cool,it works

@witsch
Copy link
Owner

witsch commented Mar 12, 2013

i've just merged #7, which seems to address the same problem. it's slightly newer, so i should have looked at your pull request first. sorry about that!

the patches looks pretty much equal. is there anything missing from your POV, or can this be closed? i'm also happy to merge if you update your pull request again. that way the !# would go in as well...

ps: and sorry for taking so long!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Problem with utf-8 content in python file
3 participants