Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] stub generation (Reboot) #211

Draft
wants to merge 50 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
d91cbf3
Add flag member details to stubs
kaiw Dec 18, 2018
182b6f1
Add constant typing to stubs
kaiw Dec 18, 2018
800869b
Add stubs of module-level functions
kaiw Dec 20, 2018
d8d3e7d
Add typing details for enums and support functions on class stubs
kaiw Dec 20, 2018
b5c8f07
Rework constant formatting so that we can reuse it for class fields
kaiw Dec 22, 2018
1b14ae1
Add typing details for classes
kaiw Dec 22, 2018
fe62d77
Fix function annotations to insert self for non-static methods
kaiw Dec 22, 2018
49a99bd
Adapt class stubbing to work for structs as well
kaiw Dec 22, 2018
afdedb2
Add stubbing of unions (as though they're classes)
kaiw Dec 22, 2018
f2c4599
Avoid re-stubbing dependency modules multiple times
kaiw Dec 23, 2018
948e8ee
Update the shebang to use env
kaiw Dec 23, 2018
30e9ca7
Import order cleanup
kaiw Dec 23, 2018
ad9e737
Handle empty class bodies in stub generation
kaiw Dec 23, 2018
b6ff08f
Handle function arguments with no typing information at all
kaiw Dec 23, 2018
82c4cc9
Module prefix all typing annotation strings for import ease
kaiw Dec 23, 2018
d967c62
Handle typing and dependency imports for stub generation
kaiw Dec 23, 2018
0cedbb1
Handling current-module-prefix stripping for stub generation
kaiw Dec 23, 2018
c0d06f4
Check whether fields are valid Python identifiers before annotating
kaiw Dec 23, 2018
b5e6591
Handle indirect GI module dependencies for imports
kaiw Dec 25, 2018
61b98c6
Generate stubs for pyclasses
kaiw Dec 25, 2018
1c5ea23
Fix incorrect class name quoting
kaiw Dec 25, 2018
f365caf
Break out function return value handling for reuse
kaiw Dec 25, 2018
c1896b1
Generate stubs for callback signatures
kaiw Dec 25, 2018
656afda
Ignore typing errors for constructors
kaiw Dec 25, 2018
140dcf5
Consolidate all the class-like stubbing
kaiw Dec 26, 2018
d915b89
Reuse enum stub generation code for flags
kaiw Dec 26, 2018
eb086d9
Unify GEnum and GFlag code under the generic class stubber
kaiw Dec 26, 2018
51e5bd3
Refactor function stub formatters to be more consistent
kaiw Dec 26, 2018
05fb137
Simplify the class-stub-container behaviour
kaiw Dec 26, 2018
afa7c5d
Handle explicit NoneType types differently
kaiw Dec 30, 2018
0cdb17e
Use the builtins module to explicitly namespace built-in types
kaiw Dec 31, 2018
5342f75
Remove old-style variable annotations and string-ified class names
kaiw Jan 11, 2019
462a2a8
Topologically sort GObject classes by MRO to avoid mypy errors
kaiw Jan 11, 2019
cd18e22
Specifically handle an odd `_Value__data__union` annotation in GObject
kaiw Jan 13, 2019
8246f86
Break out class-like access for easier testing
kaiw Jan 13, 2019
8ae025e
Annotate GObject.Enum and GObject.Flags as also being integers
kaiw Jan 13, 2019
17042a5
noqa a flake8 error
kaiw Jan 13, 2019
fde7826
Merge branch 'master' into stub-generation
kaiw Aug 10, 2019
3641107
Fix bad function call in debug path
kaiw Aug 10, 2019
7db61ad
main: Don't hard-require apt libraries
kaiw Aug 10, 2019
14855eb
Ignore spurious nodes in private member check
kaiw Aug 10, 2019
dde6e34
Add support for ignoring type errors in class inheritance
kaiw Aug 13, 2019
9ef2d3f
Allow kwargs for GObject constructors
kaiw Aug 13, 2019
cd92bba
stubs: Add a method-specific ignore list for bad/weird/unused methods
kaiw Aug 13, 2019
a6dffe8
stubs: Add a way to provide module-level aliases
kaiw Aug 13, 2019
ccf067a
stubs: Add the start of a list of Liskov issues that aren't fixable
kaiw Aug 14, 2019
f3ed013
stubs: Add a range of constructor-type names
kaiw Aug 14, 2019
eca1c0e
stubs: Add new module for manual stub additions and handle in generation
kaiw Aug 16, 2019
992976e
stubs: Make GLib.Flags boolean operators retain flag type
kaiw Aug 16, 2019
9e5b3dd
Merge branch 'kaiw/stub-generation' into fco/stub-generation
fcole90 Jul 12, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions pgidocgen/docobj.py
Original file line number Diff line number Diff line change
Expand Up @@ -338,6 +338,7 @@ def from_object(cls, repo, parent_fullname, sig):
print("FIXME: signal: %s " % inst.fullname)
signature_desc = "(FIXME pgi-docgen: arguments are missing here)"

inst.full_signature = fsig
inst.signature_desc = signature_desc
inst.info = DocInfo.from_object(repo, "signals", inst,
current_type=parent_fullname)
Expand Down Expand Up @@ -653,6 +654,7 @@ def from_object(cls, repo, parent_fullname, field_info):
name = field_info.name
field = cls(parent_fullname, name)

field.py_type = field_info.py_type
field.type_desc = py_type_to_class_ref(field_info.py_type)
field.readable = field_info.readable
field.writable = field_info.writeable
Expand Down Expand Up @@ -793,6 +795,7 @@ def render(docs):
signature = get_signature_string(obj)

assert signature
instance.full_signature = func_sig
instance.signature_desc = signature_desc
instance.signature = signature
instance.info.desc = desc
Expand Down Expand Up @@ -887,12 +890,13 @@ def from_object(cls, repo, obj):

class Constant(BaseDocObject):

def __init__(self, parent_fullname, name, value):
def __init__(self, parent_fullname, name, value, py_type):
self.fullname = parent_fullname + "." + name
self.name = name
self.info = None

self.value = value
self.py_type = py_type

@classmethod
def from_object(cls, repo, parent_fullname, name, obj):
Expand All @@ -904,7 +908,7 @@ def from_object(cls, repo, parent_fullname, name, obj):
else:
value = repr(obj)

instance = Constant(parent_fullname, name, value)
instance = Constant(parent_fullname, name, value, type(obj))
instance.info = DocInfo.from_object(repo, "all", instance)
return instance

Expand Down
10 changes: 8 additions & 2 deletions pgidocgen/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,12 @@

import argparse

from . import create, build, stubs, create_debian, update
from . import create, build, stubs, update

try:
from . import create_debian
except ImportError:
create_debian = None


def main(argv):
Expand All @@ -19,7 +24,8 @@ def main(argv):
create.add_parser(subparser)
build.add_parser(subparser)
stubs.add_parser(subparser)
create_debian.add_parser(subparser)
if create_debian:
create_debian.add_parser(subparser)
update.add_parser(subparser)

args = parser.parse_args(argv[1:])
Expand Down
25 changes: 23 additions & 2 deletions pgidocgen/namespace.py
Original file line number Diff line number Diff line change
Expand Up @@ -545,6 +545,20 @@ def is_available(mod, name):
return types, type_structs, shadow_map, instance_params


def _is_empty_text_node(child) -> bool:
return (
child.nodeType == child.TEXT_NODE and
not child.data.strip()
)


def _is_source_position_node(child) -> bool:
return (
child.nodeType == child.ELEMENT_NODE and
child.tagName == 'source-position'
)


def _parse_private(dom, namespace):
private = set()

Expand All @@ -563,8 +577,15 @@ def is_empty(node):
is_gtype_struct = bool(record.getAttribute("glib:is-gtype-struct-for"))
is_private = record.getAttribute("name").endswith("Private")
if is_private and not is_gtype_struct and is_empty(record):
name = namespace + "." + record.getAttribute("name")
private.add(name)

meaningful_children = [
c for c in record.childNodes
if not _is_empty_text_node(c) and not _is_source_position_node(c)
]

if not meaningful_children:
name = namespace + "." + record.getAttribute("name")
private.add(name)

return private

Expand Down
80 changes: 80 additions & 0 deletions pgidocgen/stuboverrides.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@

PROPERTY_STUB = """# TODO: Constrain T if possible. Builtins + GTypes might be sufficient?
T = typing.TypeVar('T')

PropertyGetterFn = typing.Callable[[typing.Any], T]
PropertySetterFn = typing.Callable[[typing.Any, T], None]


class Property(typing.Generic[T]):

name: typing.Optional[str]
type: typing.Type[T]
default: typing.Optional[T]
nick: str
blurb: str
flags: ParamFlags
minimum: typing.Optional[T]
maximum: typing.Optional[T]

def __init__(
self,
getter: typing.Optional[PropertyGetterFn[T]] = None,
setter: typing.Optional[PropertySetterFn[T]] = None,
type: typing.Optional[typing.Type[T]] = None,
default: typing.Optional[T] = None,
nick: str = '',
blurb: str = '',
flags: ParamFlags = ParamFlags.READWRITE,
minimum: typing.Optional[T] = None,
maximum: typing.Optional[T] = None,
) -> None:
...

def __get__(self, instance: typing.Any, klass: typing.Type) -> T:
...

def __set__(self, instance: typing.Any, value: T) -> None:
...

def __call__(self, PropertyGetterFn) -> Property[T]:
...

def getter(self: Property[T], fget: PropertyGetterFn) -> Property[T]:
...

def setter(self: Property[T], fset: PropertySetterFn) -> Property[T]:
...

# TODO: There's three Tuple variant structures that could be
# returned here, and they're all unpleasantly complicated.
def get_pspec_args(self) -> typing.Sequence[typing.Any]:
...
"""

FLAGS_TYPEVAR = """FlagsT = typing.TypeVar('FlagsT')
"""


#: Map of namespace to additional manually-written stub classes that
#: should be added to the top of the generated stub.
NAMESPACE_OVERRIDES = {
'GLib': [
FLAGS_TYPEVAR,
],
'GObject': [
PROPERTY_STUB,
],
}

#: Map of class full name to attributes to be added to the class.
OBJECT_OVERRIDES = {
'GLib.Flags': [
'def __or__(self: FlagsT, other: typing.Union[int, FlagsT]) -> FlagsT: ...',
'def __and__(self: FlagsT, other: typing.Union[int, FlagsT]) -> FlagsT: ...',
'def __xor__(self: FlagsT, other: typing.Union[int, FlagsT]) -> FlagsT: ...',
],
'GObject.Object': [
'Property = Property',
],
}
Loading