encoding issue writing

[beville]

I am not sure if this is a bug or just my not understanding. This is under Python 2.7, using v5.0.4 from pip. I can't seem to write out a CfgObj that has unicode strings. Here's a simple scenario:

File: 'in.ini'

folder="Arbeitsfläche"

code:

from  configobj import ConfigObj
cfg = ConfigObj("in.ini", encoding="UTF8")

print u"folder string[{0}] and repr [{1}]".format(cfg['folder'], repr(cfg['folder']))

cfg.filename = "out.ini"
cfg.write()

produces this:

folder string[Arbeitsfläche] and repr [u'Arbeitsfl\xe4che']
Traceback (most recent call last):
  File "stackoverflow.py", line 7, in <module>
    cfg.write()
  File "/usr/local/lib/python2.7/site-packages/configobj.py", line 2124, in write
    'ascii'))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 18: ordinal not in range(128)

There doesn't seem to be any problem reading the file in. Also, if I change the input file to have the value "Desktop" instead of "Arbeitsfläche", no problem.

What I can't understand is why it wants to use the ascii codec, when I've explicitly set the UTF8 encoder.

Any help would be appreciated!

[beville]

Just wanted to confirm that this example works with ConfigObj 4.7.2 on Python 2.7, but not with any version 5.0 and up.

[robdennis]

Thanks, and much appreciated. Clearly there's been a lot of work on ironing
out encoding bugs, so I'll take a look tonight.

just as a sanity check. Does setting utf8 or utf-8 work
differently? Does setting default_encoding as well as encoding do
anything?

On Thu, Apr 24, 2014 at 1:15 PM, beville notifications@github.com wrote:

Just wanted to confirm that this example works with ConfigObj 4.7.2 on
Python 2.7, but not with any versions in the 5.0 and up.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/55#issuecomment-41306934
.

[beville]

With 5.0.4, trying "utf8", "utf-8", and "UTF8" didn't seem to help. I also tried setting default_encoding in the constructor, also to no avail.

[robdennis]

Ah, that saved me a step, thanks. By look at it tonight, I think I mean tomorrow night since I'm just now (midnight est) finishing some remodeling.

On Thu, Apr 24, 2014 at 9:10 PM, beville notifications@github.com wrote:

With 5.0.4, trying "utf8", "utf-8", and "UTF8" didn't seem to help. I also tried setting default_encoding in the constructor, also to no avail.

Reply to this email directly or view it on GitHub:
#55 (comment)

[robdennis]

alright, I can reproduce this, and apparently all of the tests for similar issues (e.g. #18 and #44) didn't cover the case of writing out to a file. Those test only used ascii characters. Working it now, and this will be captured in 5.0.5

[robdennis]

pinging @EliAndrewC for a sanity check of the comments when you get a chance. Otherwise I'll merge in a release tomorrow.

[beville]

thanks for the quick fix on this!

[robdennis]

not a problem, thanks for the report. :)
https://pypi.python.org/pypi/configobj is updated with version 5.0.5