improve encoding handling for setup.cfg

[benoit-pierre]

Support the same mechanism as for Python sources for declaring the encoding to be used when reading setup.cfg (see PEP 263), and return the results of reading it as Unicode.

Fix #1062 and #1136.

[jaraco]

Does it make sense to defer this work until setuptools adopts distutils (#417)?

[jaraco]

I know interpolation causes problems when a percent appears in the setup.cfg (such as for a password). I think there are tests to protect this behavior (perhaps only in CPython).

[benoit-pierre]

It's the other way round, interpolation as always been enabled, see: #889 (comment)

This compatibility code was actually not used, that's why I added a test first to make sure this behavior (interpolation enabled) did not change.

[jaraco]

Oh, right. Good point. I wonder then if that means we (I) need to update the code in distutils for Python 3.7 to restore interpolation before it goes live.

[jaraco]

Okay, reviewing the history on the CPython issue, I see that there's not yet a patch in CPython that's affected by this change.

[benoit-pierre]

Reworked and slightly expanded tests.

[benoit-pierre]

I agree that this could wait for distutils code to be included in setuptools.

[jaraco]

We can either wait for the distutils code to be adopted by Setuptools or we can simply override the parse_config_files in Distribution (and own it, rather than working to rely on the upstream version). I'm slightly inclined toward the latter, not knowing when/if distutils will land. Would you like to work on those changes or would you rather I do it (I feel some responsibility having put the compatibility layer in place)?

[RonnyPfannschmidt]

ouch, this is still an issue :/

[c00kiemon5ter]

Hi and thanks for working on this. I also got bitten by #1136 and followed the thread here. Is this planned to be merged?

[jaraco]

I think we do basically want to adopt this change, but as part of a refactor to have setuptools take ownership of the relevant functionality from distutils (rather than trying to contribute it upstream first, which was the intention here).

[jaraco]

I'm sorry to have left this to languish for so long. I want to accept these changes with some small tweaks (mainly proper adoption of the config file parsing functionality). I'll resolve the conflicts and press forward.

[jaraco]

I believe this work is ready to go, but tests are failing due to #1644.

[jaraco]

There's still a failing test on Python 2.7. I'm not yet clear on why.

[benoit-pierre]

That's an issue somewhere in the bowels of distutils, when handling boolean options:

 setuptools/dist.py | 45 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 45 insertions(+)

diff --git i/setuptools/dist.py w/setuptools/dist.py
index 4cc3bdfe..6cabfb60 100644
--- i/setuptools/dist.py
+++ w/setuptools/dist.py
@@ -12,6 +12,7 @@
 import distutils.cmd
 import distutils.dist
 from distutils.errors import DistutilsOptionError
+from distutils.fancy_getopt import translate_longopt
 from distutils.util import strtobool
 from distutils.debug import DEBUG
 import itertools
@@ -560,6 +561,50 @@ def _clean_req(self, req):
         req.marker = None
         return req
 
+    def _set_command_options(self, command_obj, option_dict=None):
+        """Set the options for 'command_obj' from 'option_dict'.  Basically
+        this means copying elements of a dictionary ('option_dict') to
+        attributes of an instance ('command').
+
+        'command_obj' must be a Command instance.  If 'option_dict' is not
+        supplied, uses the standard option dictionary for this command
+        (from 'self.command_options').
+        """
+        command_name = command_obj.get_command_name()
+        if option_dict is None:
+            option_dict = self.get_option_dict(command_name)
+
+        if DEBUG:
+            self.announce("  setting options for '%s' command:" % command_name)
+        for (option, (source, value)) in option_dict.items():
+            if DEBUG:
+                self.announce("    %s = %s (from %s)" % (option, value,
+                                                         source))
+            try:
+                bool_opts = [translate_longopt(o)
+                             for o in command_obj.boolean_options]
+            except AttributeError:
+                bool_opts = []
+            try:
+                neg_opt = command_obj.negative_opt
+            except AttributeError:
+                neg_opt = {}
+
+            try:
+                is_string = isinstance(value, six.string_types)
+                if option in neg_opt and is_string:
+                    setattr(command_obj, neg_opt[option], not strtobool(value))
+                elif option in bool_opts and is_string:
+                    setattr(command_obj, option, strtobool(value))
+                elif hasattr(command_obj, option):
+                    setattr(command_obj, option, value)
+                else:
+                    raise DistutilsOptionError(
+                        "error in %s: command '%s' has no such option '%s'"
+                        % (source, command_name, option))
+            except ValueError as msg:
+                raise DistutilsOptionError(msg)
+
     def _parse_config_files(self, filenames=None):
         """
         Adapted from distutils.dist.Distribution.parse_config_files,

[jaraco]

The remaining test failure (just one) is due to an option "tag-date=0" being parsed as u'0' but on Python 2 not being interpreted as isinstance(tag_date, str) (ref).

[jaraco]

@benoit-pierre Assuming tests pass (they do for me locally), would you please review the latest changes?

[benoit-pierre]

Well, I can't approve my own PR, but looks good to me!