Externalize flake8 linting, introduce additional checks

[ambv]

This change is a step towards removing runtests.py (see #1673).

The exclusion list in the flake8 configuration in setup.cfg has been updated
to enable running the linter from the root of the project by simply invoking
flake8. This enables it to leverage its own file discovery and its own
multiprocessing queue without excessive subprocessing for linting every file.
This gives a minor speed up in local test runs. Before (runtests.py -j4):

  total time in lint: 130.914682

After:

  total time in lint: 20.379915

There's an additional speedup on Travis because linting is now only performed
on Python 3.6.

More importantly, this means flake8 is now running over all files unless
explicitly excluded in setup.cfg. This will help avoiding unintentional
omissions in the future (see comments on #2637).

Note: running flake8 as a single lazy subprocess in runtests.py doesn't
sacrifice any parallelism because the linter has its own process pool.

Minimal whitespace changes were required to mypy_extensions.py but in return
flake8 will check it now exactly like it checks the rest of the mypy/*
codebase. Those are also done on #2637 but that hasn't landed yet.

Finally, flake8-bugbear and flake8-pyi were added to test requirements to make
the linter configuration consistent with typeshed. I hope the additional
checks will speed up future pull requests by automating bigger parts of the
code review. The pyi plugin enables forward reference support when linting
.pyi files. That means it's now possible to run flake8 inside the typeshed
submodule or on arbitrary .pyi files during development (which your editor
could do for you), for example on fixtures. See discussion on #2629 on checks
that are disabled and why.

[gvanrossum]

I have to wring my hands some more before merging this. I have two issues:

• in my own mypy directory I have tons of throw-away scripts -- the flake8 run now checks those, reporting lots of silly issues.
• I'm not completely happy with invoking things from runtest that themselves manage process pools; this defeats the point of the process pool in runtests. In the past there were complaints from people about this, where runtests.py -j1 would still use multiple CPUs -- a problem when you are trying to keep some other CPU usable for interactive work you're doing on the same machine.

[gvanrossum]

What's this?

[ambv]

This doesn't belong here, I'll remove it.

[gvanrossum]

I wonder if we should change this and line 8 to 3.6 and add 3.7-dev?

[ambv]

Last time I checked 3.6 still wasn't a thing which is why I kept 3.6-dev. Will change and see what happens.

[ambv]

I have tons of throw-away scripts

How do you keep them from being accidentally committed? If there's a .gitignore rule that applies to them, I could reuse it in our .flake8 config. I personally have a global .gitignore rule to ignore folders called ".local" and keep my throw-away stuff there.

runtests.py -j1 would still use multiple CPUs

Right, that's an easy fix, I could pass -j to flake8, too. Will change.

[ambv]

Alright, all fixed except for the open question about your one-off scripts.