Merge branch 'master' into normalize-type-union

Author: pkch

.travis.yml

@@ -15,6 +15,7 @@ python:
 
 install:
   - pip install -r test-requirements.txt
+  - python2 -m pip install --user typing
   - python setup.py install
 
 script:

LICENSE

@@ -27,8 +27,7 @@ DEALINGS IN THE SOFTWARE.
 = = = = =
 
 Portions of mypy are licensed under different licenses.  The files
-under stdlib-samples and lib-typing are licensed under the PSF 2
-License, reproduced below.
+under stdlib-samples are licensed under the PSF 2 License, reproduced below.
 
 = = = = =
 

README.md

@@ -97,6 +97,23 @@ programs, even if they have type errors:
 [statically typed parts]: http://mypy.readthedocs.io/en/latest/basics.html#function-signatures
 
 
+IDE & Linter Integrations
+-------------------------
+
+Mypy can be integrated into popular IDEs:
+
+* Vim: [vim-mypy](https://github.com/Integralist/vim-mypy)
+* Emacs: using [Flycheck](https://github.com/flycheck/) and [Flycheck-mypy](https://github.com/lbolla/emacs-flycheck-mypy/issues)
+* Sublime Text: [SublimeLinter-contrib-mypy]
+* Atom: [linter-mypy](https://atom.io/packages/linter-mypy)
+* PyCharm: PyCharm integrates [its own implementation of PEP 484](https://www.jetbrains.com/help/pycharm/2017.1/type-hinting-in-pycharm.html).
+
+Mypy can also be integrated into [Flake8] using [flake8-mypy].
+
+[Flake8]: http://flake8.pycqa.org/
+[flake8-mypy]: https://github.com/ambv/flake8-mypy
+
+
 Web site and documentation
 --------------------------
 
@@ -157,6 +174,15 @@ If you want to contribute, first clone the mypy git repository:
 
     $ git clone --recurse-submodules https://github.com/python/mypy.git
 
+If you've already cloned the repo without `--recurse-submodules`,
+you need to pull in the typeshed repo as follows:
+
+    $ git submodule init
+    $ git submodule update
+
+Either way you should now have a subdirectory `typeshed` containing a
+clone of the typeshed repo (`https://github.com/python/typeshed`).
+
 From the mypy directory, use pip to install mypy:
 
     $ cd mypy
@@ -170,10 +196,6 @@ the above as root. For example, in Ubuntu:
 Now you can use the `mypy` program just as above.  In case of trouble
 see "Troubleshooting" above.
 
-The mypy wiki contains some useful information for contributors:
-
-  https://github.com/python/mypy/wiki/Developer-Guides
-
 
 Working with the git version of mypy
 ------------------------------------

ROADMAP.md

@@ -0,0 +1,96 @@
+# Mypy Roadmap
+
+The goal of the roadmap is to document areas the mypy core team is
+planning to work on in the future or is currently working on. PRs
+targeting these areas are very welcome, but please check first with a
+core team member that nobody else is working on the same thing.
+
+**Note:** This doesn’t include everything that the core team will work
+on, and everything is subject to change. Near-term plans are likely
+more accurate.
+
+## April-June 2017
+
+- Add more comprehensive testing for `--incremental` and `--quick`
+  modes to improve reliability. At least write more unit tests with
+  focus on areas that have previously had bugs.
+  ([issue](https://github.com/python/mypy/issues/3455))
+
+- Speed up `--quick` mode to better support million+ line codebases
+  through some of these:
+
+  - Make it possible to use remote caching for incremental cache
+    files. This would speed up a cold run with no local cache data.
+    We need to update incremental cache to use hashes to determine
+    whether files have changes to allow
+    [sharing cache data](https://github.com/python/mypy/issues/3403).
+
+  - See if we can speed up deserialization of incremental cache
+    files. Initial experiments aren’t very promising though so there
+    might not be any easy wins left.
+    ([issue](https://github.com/python/mypy/issues/3456))
+
+- Improve support for complex signatures such as `open(fn, 'rb')` and
+  specific complex decorators such as `contextlib.contextmanager`
+  through type checker plugins/hooks.
+  ([issue](https://github.com/python/mypy/issues/1240))
+
+- Document basic properties of all type operations used within mypy,
+  including compatibility, proper subtyping, joins and meets.
+  ([issue](https://github.com/python/mypy/issues/3454))
+
+- Make TypedDict an officially supported mypy feature. This makes it
+  possible to give precise types for dictionaries that represent JSON
+  objects, such as `{"path": "/dir/fnam.ext", "size": 1234}`.
+  ([issue](https://github.com/python/mypy/issues/3453))
+
+- Make error messages more useful and informative.
+  ([issue](https://github.com/python/mypy/labels/topic-usability))
+
+- Resolve [#2008](https://github.com/python/mypy/issues/2008) (we are
+  converging on approach 4).
+
+## July-December 2017
+
+- Invest some effort into systematically filling in missing
+  annotations and stubs in typeshed, with focus on features heavily
+  used at Dropbox. Better support for ORMs will be a separate
+  project.
+
+- Improve opt-in warnings about `Any` types to make it easier to keep
+  code free from unwanted `Any` types. For example, warn about using
+  `list` (instead of `List[x]`) and calling `open` if we can’t infer a
+  precise return type, or using types imported from ignored modules
+  (they are implicitly `Any`).
+
+- Add support for protocols and structural subtyping (PEP 544).
+
+- Switch completely to pytest and remove the custom testing framework.
+  ([issue](https://github.com/python/mypy/issues/1673))
+
+- Make it possible to run mypy as a daemon to avoid reprocessing the
+  entire program on each run. This will improve performance
+  significantly. Even when using the incremental mode, processing a
+  large number of files is not cheap.
+
+- Refactor and simplify specific tricky parts of mypy internals, such
+  as the [conditional type binder](https://github.com/python/mypy/issues/3457),
+  [symbol tables](https://github.com/python/mypy/issues/3458) or
+  the various [semantic analysis passes](https://github.com/python/mypy/issues/3459).
+
+- Implement a general type system plugin architecture. It should be
+  able to support some typical ORM features at least, such as
+  metaclasses that add methods with automatically inferred signatures
+  and complex descriptors such as those used by Django models.
+  ([issue](https://github.com/python/mypy/issues/1240))
+
+- Add support for statically typed
+  [protobufs](https://developers.google.com/protocol-buffers/).
+
+- Provide much faster, reliable interactive feedback through
+  fine-grained incremental type checking, built on top the daemon
+  mode.
+
+- Start work on editor plugins and support for selected IDE features.
+
+- Turn on `--strict-optional` by default.

appveyor.yml

@@ -1,26 +1,16 @@
 environment:
   matrix:
 
-    - PYTHON: "C:\\Python35"
-      PYTHON_VERSION: "3.5.1"
-      PYTHON_ARCH: "32"
-
-    - PYTHON: "C:\\Python35-x64"
-      PYTHON_VERSION: "3.5.1"
-      PYTHON_ARCH: "64"
-
-    - PYTHON: "C:\\Python36"
-      PYTHON_VERSION: "3.6.x"
-      PYTHON_ARCH: "32"
-
     - PYTHON: "C:\\Python36-x64"
       PYTHON_VERSION: "3.6.x"
       PYTHON_ARCH: "64"
 
-
 install:
+    - "git config core.symlinks true"
+    - "git reset --hard"
     - "%PYTHON%\\python.exe -m pip install -r test-requirements.txt"
     - "git submodule update --init typeshed"
+    - "cd typeshed && git config core.symlinks true && git reset --hard && cd .."
     - "%PYTHON%\\python.exe setup.py -q install"
 
 build: off
@@ -47,3 +37,5 @@ skip_commits:
     - .travis.yml
     - CREDITS
     - LICENSE
+
+skip_branch_with_pr: true

conftest.py

@@ -1,3 +1,13 @@
+import os.path
+
+import pytest
+
 pytest_plugins = [
     'mypy.test.data',
 ]
+
+
+def pytest_configure(config):
+    mypy_source_root = os.path.dirname(os.path.abspath(__file__))
+    if os.getcwd() != mypy_source_root:
+        os.chdir(mypy_source_root)

docs/source/cheat_sheet.rst

@@ -134,7 +134,6 @@ When you're puzzled or when things are complicated
             # type: (*str, **str) -> str
             request = make_request(*args, **kwargs)
             return self.do_api_query(request)
-
    
    # Use `ignore` to suppress type-checking on a given line, when your
    # code confuses mypy or runs into an outright bug in mypy.
@@ -150,6 +149,18 @@ When you're puzzled or when things are complicated
    reveal_type(c)  # -> error: Revealed type is 'builtins.list[builtins.str]'
    print(c)  # -> [4] the object is not cast
 
+   # if you want dynamic attributes on your class, have it override __setattr__ or __getattr__
+   # in a stub or in your source code.
+   # __setattr__ allows for dynamic assignment to names
+   # __getattr__ allows for dynamic access to names
+   class A:
+       # this will allow assignment to any A.x, if x is the same type as `value`
+       def __setattr__(self, name, value):
+           # type: (str, int) -> None
+           ...
+   a.foo = 42  # works
+   a.bar = 'Ex-parrot'  # fails type checking
+
    # TODO: explain "Need type annotation for variable" when
    # initializing with None or an empty container
 

docs/source/cheat_sheet_py3.rst

@@ -122,6 +122,12 @@ When you're puzzled or when things are complicated
    # dynamic to write a type for.
    x = mystery_function()  # type: Any
 
+   # This is how to deal with varargs.
+   # This makes each positional arg and each keyword arg a 'str'.
+   def call(self, *args: str, **kwargs: str) -> str:
+            request = make_request(*args, **kwargs)
+            return self.do_api_query(request)
+
    # Use `ignore` to suppress type-checking on a given line, when your
    # code confuses mypy or runs into an outright bug in mypy.
    # Good practice is to comment every `ignore` with a bug link
@@ -136,6 +142,19 @@ When you're puzzled or when things are complicated
    reveal_type(c)  # -> error: Revealed type is 'builtins.list[builtins.str]'
    print(c)  # -> [4] the object is not cast
 
+   # if you want dynamic attributes on your class, have it override __setattr__ or __getattr__
+   # in a stub or in your source code.
+   # __setattr__ allows for dynamic assignment to names
+   # __getattr__ allows for dynamic access to names
+   class A:
+       # this will allow assignment to any A.x, if x is the same type as `value`
+       def __setattr__(self, name: str, value: int) -> None: ...
+       # this will allow access to any A.x, if x is compatible with the return type
+       def __getattr__(self, name: str) -> int: ...
+   a.foo = 42  # works
+   a.bar = 'Ex-parrot'  # fails type checking
+
+
    # TODO: explain "Need type annotation for variable" when
    # initializing with None or an empty container
 

docs/source/command_line.rst

@@ -14,14 +14,15 @@ flag (or its long form ``--help``)::
               [--disallow-untyped-calls] [--disallow-untyped-defs]
               [--check-untyped-defs] [--disallow-subclassing-any]
               [--warn-incomplete-stub] [--warn-redundant-casts]
-              [--warn-no-return] [--warn-unused-ignores] [--show-error-context]
-              [-i] [--cache-dir DIR] [--strict-optional]
-              [--strict-optional-whitelist [GLOB [GLOB ...]]] [--strict]
+              [--no-warn-no-return] [--warn-return-any] [--warn-unused-ignores]
+              [--show-error-context] [-i] [--quick-and-dirty] [--cache-dir DIR]
+              [--strict-optional]
+              [--strict-optional-whitelist [GLOB [GLOB ...]]]
               [--junit-xml JUNIT_XML] [--pdb] [--show-traceback] [--stats]
               [--inferstats] [--custom-typing MODULE]
               [--custom-typeshed-dir DIR] [--scripts-are-modules]
               [--config-file CONFIG_FILE] [--show-column-numbers]
-              [--find-occurrences CLASS.MEMBER] [--strict-boolean]
+              [--find-occurrences CLASS.MEMBER] [--strict] [--strict-boolean]
               [--cobertura-xml-report DIR] [--html-report DIR]
               [--linecount-report DIR] [--linecoverage-report DIR]
               [--memory-xml-report DIR] [--old-html-report DIR]
@@ -298,10 +299,31 @@ Here are some more useful flags:
   the base class even though that may not actually be the case.  This
   flag makes mypy raise an error instead.
 
-- ``--incremental`` is an experimental option that enables incremental
-  type checking. When enabled, mypy caches results from previous runs
+.. _incremental:
+
+- ``--incremental`` is an experimental option that enables a module
+  cache. When enabled, mypy caches results from previous runs
   to speed up type checking. Incremental mode can help when most parts
-  of your program haven't changed since the previous mypy run.
+  of your program haven't changed since the previous mypy run.  A
+  companion flag is ``--cache-dir DIR``, which specifies where the
+  cache files are written.  By default this is ``.mypy_cache`` in the
+  current directory.  While the cache is only read in incremental
+  mode, it is written even in non-incremental mode, in order to "warm"
+  the cache.  To disable writing the cache, use
+  ``--cache-dir=/dev/null`` (UNIX) or ``--cache-dir=nul`` (Windows).
+  Cache files belonging to a different mypy version are ignored.
+
+.. _quick-mode:
+
+- ``--quick-and-dirty`` is an experimental, unsafe variant of
+  :ref:`incremental mode <incremental>`.  Quick mode is faster than
+  regular incremental mode, because it only re-checks modules that
+  were modified since their cache file was last written (regular
+  incremental mode also re-checks all modules that depend on one or
+  more modules that were re-checked).  Quick mode is unsafe because it
+  may miss problems caused by a change in a dependency.  Quick mode
+  updates the cache, but regular incremental mode ignores cache files
+  written by quick mode.
 
 - ``--python-version X.Y`` will make mypy typecheck your code as if it were
   run under Python version X.Y. Without this option, mypy will default to using
@@ -366,11 +388,6 @@ Here are some more useful flags:
 - ``--warn-return-any`` causes mypy to generate a warning when returning a value
   with type ``Any`` from a function declared with a non- ``Any`` return type.
 
-- ``--strict-boolean`` will make using non-boolean expressions in conditions
-  an error. This means ``if x`` and ``while x`` are disallowed when ``x`` has any
-  type other than ``bool``. Instead use explicit checks like ``if x > 0`` or
-  ``while x is not None``.
-
 - ``--strict`` mode enables all optional error checking flags.  You can see the
   list of flags enabled by strict mode in the full ``mypy -h`` output.
 

docs/source/common_issues.rst

@@ -180,7 +180,7 @@ not support ``sort()``) as a list and sort it in-place:
         # Type of x is List[int] here.
         x.sort()  # Okay!
 
-.. _invariance-vs-covariance:
+.. _variance:
 
 Invariance vs covariance
 ------------------------