Fix config file parsing of disallow_any/disallow_untyped_defs combination

[JelleZijlstra]

Fixes #4075.

Due to the way disallow_any is parsed in config files, any disallow_any directive basically overrode disallow_untyped_defs. This confused me in #4075, and it also meant that disallow_untyped_defs was implicitly disabled for mypy itself, because of the way mypy_self_check.ini was written.

This PR solves the problem by making disallow_any override disallow_untyped_defs only if the config file either has disallow_any = (disabling all disallow_any checks) or explicitly lists unannotated (enabling disallow_untyped_defs). This passes all tests (including a new one), but it's awkward because it makes unannotated behave differently from other disallow_any flags: if a more general config has disallow_any = A B and a more specific one has disallow_any = A, then B should be disabled, but this is no longer true for unannotated.

Here are two other options for solving the problem:

• We could parse disallow_untyped_defs and disallow_any completely independently, and just AND them at runtime when we check whether to give an error for an untyped def. This leads to an awkward situation because now if you have both disallow_any = unannotated in a general config and disallow_untyped_defs = False in the specific one, untyped defs still produce errors.
• The root cause of the problem is that we have two options that do the same thing, so we should get rid of one of them so we don't have to worry about how they interact. I think that's what @ilinum originally did, but I argued against it because disallow_untyped_defs is more intuitively named. What if we just get rid of disallow_any = unannotated instead?

In this PR I also fix the untyped defs that crept into mypy itself due to the bug.

[JelleZijlstra]

I'm not sure this is true, but the file is syntactically valid in Python 2 and it tests a piece of code (mypy_extensions.py) that is supposed to work in Python 2. @ilevkivskyi can you confirm?

[ilevkivskyi]

Yes, this file supposed to run on both Python 3 and Python 2. There are still some problem with type checking it, but there is a separate issue for this #2755

[gvanrossum]

Thanks for finding this. I have to agree that I would rather get rid of one of the two spellings (your second bullet). I also dislike the double negative of "disallow_any = unannotated" so I would prefer to get rid of that.

FWIW I also would prefer to have a bunch of separate Boolean flags instead of the single complex flag that is disallow_any. I argued against the current set-based design during its inception but lost; IIRC @ilinum eventually found a problem that would have been easier with the separate-flags design, but he was too far into the implementation to switch.

(That problem, FWIW, is that if you want to have disallow_any = X, Y, Z in your main section and then disable Y in a module-specific section, you have to write disallow_any = X, Z in that subsection. And then later if you change your main section to disallow_any = X, Y, Z, U you have to remember to add U to all the module-specific sections that override it (assuming they just meant to remove Y).)

[JelleZijlstra]

Let's do this for now:

• Remove disallow_any = unannotated in this PR, leaving only disallow_untyped_defs = True
• File an issue to replace the other disallow_any flags with boolean flags. (I might be able to work on that, but not immediately.)

Does that sound good?

[ilinum]

That sounds good to me!
Removing disallow_any=unannotated seems like a good call. Do we need to deprecate it first? Or can we just remove it? What's the right approach here?

I am also okay with replacing disallow_any with boolean flags for the reasons that Guido pointed out.

[gvanrossum]

I propose to just drop it. People who use it will get failures after they update (possibly at 0.540 already, which will be frozen this Friday and released Wednesday next week) but IMO that's fine, people always test new releases somewhat when they upgrade.

The only thing we might add would be to have a special case in the error message that checks if unannotated is among the given strings and then prints use --disallow-untyped-defs instead.

[JukkaL]

The PR has merge conflicts.

[JelleZijlstra]

Fixed.