Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Stub generation #176

Open
wants to merge 49 commits into
base: master
Choose a base branch
from
Open

[WIP] Stub generation #176

wants to merge 49 commits into from

Commits on Dec 21, 2018

  1. Add flag member details to stubs

    kaiw committed Dec 21, 2018
    Configuration menu
    Copy the full SHA
    d91cbf3 View commit details
    Browse the repository at this point in the history
  2. Add constant typing to stubs

    This also retains the original constant value on the Constant class so
    that the stubbing can introspect its type. We could instead do the
    typing-annotation style formatting on the constant class, but this same
    logic is going to be needed for e.g., formatting function parameter
    types.
    kaiw committed Dec 21, 2018
    Configuration menu
    Copy the full SHA
    182b6f1 View commit details
    Browse the repository at this point in the history
  3. Add stubs of module-level functions

    This also retains the full function signature on functions so that we
    can get the Python typing annotations for args and return values
    correct.
    kaiw committed Dec 21, 2018
    Configuration menu
    Copy the full SHA
    800869b View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    d8d3e7d View commit details
    Browse the repository at this point in the history

Commits on Dec 22, 2018

  1. Configuration menu
    Copy the full SHA
    b5c8f07 View commit details
    Browse the repository at this point in the history
  2. Add typing details for classes

    kaiw committed Dec 22, 2018
    Configuration menu
    Copy the full SHA
    1b14ae1 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    fe62d77 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    49a99bd View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    afdedb2 View commit details
    Browse the repository at this point in the history

Commits on Dec 23, 2018

  1. Configuration menu
    Copy the full SHA
    f2c4599 View commit details
    Browse the repository at this point in the history
  2. Update the shebang to use env

    This makes the virtualenv-installed setup work.
    kaiw committed Dec 23, 2018
    Configuration menu
    Copy the full SHA
    948e8ee View commit details
    Browse the repository at this point in the history
  3. Import order cleanup

    kaiw committed Dec 23, 2018
    Configuration menu
    Copy the full SHA
    30e9ca7 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    ad9e737 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    b6ff08f View commit details
    Browse the repository at this point in the history
  6. Module prefix all typing annotation strings for import ease

    This makes it a lot easier because we can just add a `typing` import to
    the top of the `.pyi` file and forget about everything else.
    kaiw committed Dec 23, 2018
    Configuration menu
    Copy the full SHA
    82c4cc9 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    d967c62 View commit details
    Browse the repository at this point in the history
  8. Handling current-module-prefix stripping for stub generation

    The problem here is that we need class/struct/etc. references within the
    module being annotated to not have the module prefix, because otherwise
    we just get unresolvable typing references. We also need all of these
    to be forward references (i.e., strings) unless we want to do some kind
    of dependency ordering, which I don't.
    
    I'm not a fan of doing this with a global, but the other solutions are
    much more complicated. Maybe we can make a stubber state class instead
    of this, but it's not obvious how much better that would be.
    kaiw committed Dec 23, 2018
    Configuration menu
    Copy the full SHA
    0cedbb1 View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    c0d06f4 View commit details
    Browse the repository at this point in the history

Commits on Dec 25, 2018

  1. Handle indirect GI module dependencies for imports

    While there's dependency information in the namespace class, it's not
    reliable when we need to have all relevant modules in the current
    namespace for name resolution. The solution here is to constantly update
    a list of modules that have been referenced by current-module
    annotations, since that's what mypy, etc. are going to need.
    
    This change also means that we can't write import order until the end of
    the stubbing, so we're now keeping the stubs in a StringIO.
    kaiw committed Dec 25, 2018
    Configuration menu
    Copy the full SHA
    b5e6591 View commit details
    Browse the repository at this point in the history
  2. Generate stubs for pyclasses

    This doesn't cover everything, but now at least we get GType, which was
    causing a lot of issues.
    kaiw committed Dec 25, 2018
    Configuration menu
    Copy the full SHA
    61b98c6 View commit details
    Browse the repository at this point in the history
  3. Fix incorrect class name quoting

    kaiw committed Dec 25, 2018
    Configuration menu
    Copy the full SHA
    1c5ea23 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    f365caf View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    c1896b1 View commit details
    Browse the repository at this point in the history
  6. Ignore typing errors for constructors

    Firstly, it's worth noting that this only ignores the typing error.
    Things that call these APIs will still be checked! The problem is that
    the checking may be incorrect because of the API itself.
    
    There's a long discussion about this and related Liskov issues at:
        python/mypy#1237
    
    The short version is that this API is wrong-ish, but there's no
    mechanism within mypy to correctly annotate what's going on here. There
    *is* some of this around, because mypy treats __init__ and __new__
    differently itself, but we can't apply that treatment to our
    constructors.
    
    This would be better if we actually knew which methods were
    constructors, instead of the name-based guessing here... but that's way
    too much complexity for me right now.
    kaiw committed Dec 25, 2018
    Configuration menu
    Copy the full SHA
    656afda View commit details
    Browse the repository at this point in the history

Commits on Dec 26, 2018

  1. Configuration menu
    Copy the full SHA
    140dcf5 View commit details
    Browse the repository at this point in the history
  2. Reuse enum stub generation code for flags

    These are functionally identical anyway, and are *almost* the same as
    the generic class stubber.
    kaiw committed Dec 26, 2018
    Configuration menu
    Copy the full SHA
    d915b89 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    eb086d9 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    51e5bd3 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    05fb137 View commit details
    Browse the repository at this point in the history

Commits on Dec 30, 2018

  1. Handle explicit NoneType types differently

    These are special cases for some base types lacking introspection
    information. Since nothing else seems to have this issue other than
    low-level GObject types, we'll treat this as a weird case.
    
    This gets rid of mypy errors like:
    
        GObject.pyi:1416: error: Name 'NoneType' is not defined
    
    in module-level definitions for e.g., GBoxed, GPointer, etc.
    kaiw committed Dec 30, 2018
    Configuration menu
    Copy the full SHA
    afa7c5d View commit details
    Browse the repository at this point in the history

Commits on Dec 31, 2018

  1. Use the builtins module to explicitly namespace built-in types

    This isn't a generic solution for shadowing... that would require actual
    parsing and tricks. However, in practice the risky shadowing here
    happens when e.g., a class implements
    
        def int(self) -> int:
            ...
    
    and everything breaks. This works around that in a somewhat-inelegant
    but basically-correct kind of way.
    kaiw committed Dec 31, 2018
    Configuration menu
    Copy the full SHA
    0cdb17e View commit details
    Browse the repository at this point in the history

Commits on Jan 11, 2019

  1. Remove old-style variable annotations and string-ified class names

    According to mypy, these aren't necessary in stub files.
    kaiw committed Jan 11, 2019
    Configuration menu
    Copy the full SHA
    5342f75 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    462a2a8 View commit details
    Browse the repository at this point in the history

Commits on Jan 13, 2019

  1. Configuration menu
    Copy the full SHA
    cd18e22 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    8246f86 View commit details
    Browse the repository at this point in the history
  3. Annotate GObject.Enum and GObject.Flags as also being integers

    They support common integer methods, so this is definitely reasonable.
    kaiw committed Jan 13, 2019
    Configuration menu
    Copy the full SHA
    8ae025e View commit details
    Browse the repository at this point in the history
  4. noqa a flake8 error

    flake8 is trying to do the right thing here, but in this instance we're
    dealing with getting an actual type, so the "use isinstance instead"
    error really isn't helpful or correct.
    kaiw committed Jan 13, 2019
    Configuration menu
    Copy the full SHA
    17042a5 View commit details
    Browse the repository at this point in the history

Commits on Aug 10, 2019

  1. Configuration menu
    Copy the full SHA
    fde7826 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    3641107 View commit details
    Browse the repository at this point in the history
  3. main: Don't hard-require apt libraries

    These aren't in the module requirements, and aren't installed by default
    on non-Debian-based systems.
    kaiw committed Aug 10, 2019
    Configuration menu
    Copy the full SHA
    7db61ad View commit details
    Browse the repository at this point in the history
  4. Ignore spurious nodes in private member check

    The check for whether a record is private assumes a fixed child node
    structure that isn't the case in some environments. This patch tries
    to keep the spirit of the check by filtering out all known-unimportant
    child nodes and simply checking whether there's anything left.
    
    The expected unimportant nodes are:
    
     * empty text nodes containing formatting white-space
     * source-position elements that map the member to the original source
       file
    
    neither of which indicate that the member actually needs to be
    annotated.
    kaiw committed Aug 10, 2019
    Configuration menu
    Copy the full SHA
    14855eb View commit details
    Browse the repository at this point in the history

Commits on Aug 13, 2019

  1. Add support for ignoring type errors in class inheritance

    This is needed for e.g., classes that inherit from both Atk.Object and
    Atk.Action, which both define methods like `get_description` with
    very different signatures.
    kaiw committed Aug 13, 2019
    Configuration menu
    Copy the full SHA
    dde6e34 View commit details
    Browse the repository at this point in the history
  2. Allow kwargs for GObject constructors

    GObject-derived widgets always accept keyword args for setting GObject
    properties. Ideally we'd check that the kwargs matched the object's
    properties, but for now we'll just accept all kwargs to avoid the type
    errors.
    kaiw committed Aug 13, 2019
    Configuration menu
    Copy the full SHA
    9ef2d3f View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    cd92bba View commit details
    Browse the repository at this point in the history
  4. stubs: Add a way to provide module-level aliases

    This is currently only for GObject, but there's several cases where
    backwards-compatible aliases have been provided.
    kaiw committed Aug 13, 2019
    Configuration menu
    Copy the full SHA
    a6dffe8 View commit details
    Browse the repository at this point in the history

Commits on Aug 14, 2019

  1. Configuration menu
    Copy the full SHA
    ccf067a View commit details
    Browse the repository at this point in the history
  2. stubs: Add a range of constructor-type names

    Ideally this would come from the GI constructor information, but we
    don't maintain that through to the point of use here yet. This list
    should be able to be removed with some work in the future though.
    kaiw committed Aug 14, 2019
    Configuration menu
    Copy the full SHA
    f3ed013 View commit details
    Browse the repository at this point in the history

Commits on Aug 16, 2019

  1. stubs: Add new module for manual stub additions and handle in generation

    Object.Property and Object.Signal are the main two missing parts for
    general use, and both require hand-written stub information.
    
    The Property stub here is probably wrong in various ways and could
    definitely be a lot better, but it's better than nothing.
    kaiw committed Aug 16, 2019
    Configuration menu
    Copy the full SHA
    eca1c0e View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    992976e View commit details
    Browse the repository at this point in the history