Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Added generation for Safe-DS stubs files #33

Merged
merged 122 commits into from
Dec 7, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
122 commits
Select commit Hold shift + click to select a range
bb1e449
Added stub generator and functionality for generating a module + impo…
Masara Oct 13, 2023
a7a067c
Stubs generator now generates global functions and enums; some small …
Masara Oct 13, 2023
d756015
Added superclasses & parameter default values to stub generator
Masara Oct 14, 2023
59d2732
Refactoring & some small fixes
Masara Oct 14, 2023
a2baa05
Refactoring
Masara Oct 14, 2023
50e18c0
Merge branch 'main' into stubs_generation
Masara Oct 17, 2023
4a3b8bd
fixes & refactoring
Masara Oct 17, 2023
975293b
Fixed a bug in the get_api call | Stub-Generator: some fixes and adde…
Masara Oct 20, 2023
519aaae
Added nested classes for the stubs generator
Masara Oct 20, 2023
78143ea
Refactoring stubs generator: Access to data of the API via API class,…
Masara Oct 20, 2023
aa6e543
some fixes
Masara Oct 20, 2023
5e51e2e
Fixed a bug in _get_api.py which prevented analysing packages with di…
Masara Oct 23, 2023
cfa90eb
Merge branch 'fixes-path_lengths-module_id' into stubs_generation
Masara Oct 23, 2023
3a61470
style: apply automated linter fixes
megalinter-bot Oct 23, 2023
87d8886
Created new package for stubs generation functions, created test file…
Masara Oct 23, 2023
3d8aee4
Added test cases for files, directories, classes, functions and attri…
Masara Oct 23, 2023
7201cda
Merge remote-tracking branch 'origin/fixes-path_lengths-module_id' in…
Masara Oct 23, 2023
7f4dd64
linter fix
Masara Oct 23, 2023
e5b023f
Merge branch 'fixes-path_lengths-module_id' into stubs_generation
Masara Oct 23, 2023
cbf2efa
Removed OPT_POS_ONLY from ParameterAssignment enum and added todos
Masara Nov 5, 2023
5ad4b69
Added tests for the stubs generation
Masara Nov 5, 2023
cf745d8
Added more test data and cases, added todos, refactoring
Masara Nov 6, 2023
e433458
Refactoring
Masara Nov 6, 2023
082b1b9
Merge branch 'main' into stubs_generation
Masara Nov 6, 2023
291e13c
Linter fix
Masara Nov 6, 2023
3872ee0
Added a TODO for stubs if a class has multiple inheritances, empty en…
Masara Nov 8, 2023
7c902f5
Added handling for methods with @classmethod decorator and adjusted t…
Masara Nov 8, 2023
37efa81
Refactoring
Masara Nov 8, 2023
3f008d4
(WIP) Refactoring, tiny bug fixes and adding todos
Masara Nov 10, 2023
3321666
Merge branch 'main' into stubs_generation
Masara Nov 13, 2023
852767f
Attributes, Parameter and Enum instances will be converted to camelCa…
Masara Nov 13, 2023
ef587b6
Refactoring
Masara Nov 13, 2023
d04c4f7
Test snapshot update
Masara Nov 13, 2023
541fd4d
Removed test packages and updated test snapshots
Masara Nov 13, 2023
79fd8a2
Adjusted test snapshots and removed enum imports for stub generator
Masara Nov 13, 2023
1cda792
Adjusted test snapshots and removed enum imports for stub generator
Masara Nov 13, 2023
9ec9516
Adjusted test snapshots
Masara Nov 13, 2023
507e3f0
Adjusted test snapshots and added escaping for keywords in imports
Masara Nov 13, 2023
912b6ca
api_analyzer: Fixed a bug where lists with multiple types would not b…
Masara Nov 13, 2023
35f0dd1
Added information for class methods to api analyzer and stubs generator
Masara Nov 14, 2023
164a416
Fixed create type function for stubs generator for None types in Unions
Masara Nov 14, 2023
c2f34fc
Removed automatic trailing whitespace trimming for .ambr (test snapsh…
Masara Nov 17, 2023
bc188d4
Issue #37 - Added new type "CallableType" for api_analyzer and stubs …
Masara Nov 17, 2023
804f86a
Updated test snapshots
Masara Nov 17, 2023
6cd1892
Updated test snapshots
Masara Nov 17, 2023
d29d3d4
Issue #34: Added inference for return, parameter and attribute types.
Masara Nov 18, 2023
b6a7051
Issue #34: Test fixes
Masara Nov 18, 2023
6b0e74e
Issue #34: Test and bug fixes
Masara Nov 18, 2023
46e75fe
Issue #34: Test and bug fixes
Masara Nov 18, 2023
fa6c5ab
Issue #34: Test and bug fixes
Masara Nov 18, 2023
9eb19f5
Issue #34: Added TODOs for the stub generator if parameters, attribut…
Masara Nov 20, 2023
69d421f
Issue #35 & #36 added variance parsing api_analyzer; added variance a…
Masara Nov 20, 2023
477a386
Fixed a bug for the stubs generator
Masara Nov 21, 2023
f1bf09f
Big overhaul of the tests: Removed redundant test data files and merg…
Masara Nov 21, 2023
fb2256b
[WIP] Added handling for @property and @abstractmethod decorators
Masara Nov 21, 2023
d0dac2b
[WIP] Stubs generator: ABC classes won't have a constructor
Masara Nov 22, 2023
7085ee0
Fixed bugs and adjusted test snapshots
Masara Nov 24, 2023
b7a7730
Stubs generator: Added @PythonModule annotation for package informati…
Masara Nov 24, 2023
6466dfb
AST Visitor: Parameter can have callables as type, fixed representati…
Masara Nov 27, 2023
4aea41e
Added handling for final type attributes and UpperCamelCase stub name…
Masara Nov 27, 2023
4bff2cb
Fixed bugs and tests, added qnames for some NamedTypes
Masara Nov 27, 2023
8b44fe6
Fixed a bug where attributes with call expressions as type hint would…
Masara Nov 27, 2023
efaab1f
fixed a test snapshot bug where return types would be unsorted and mi…
Masara Nov 27, 2023
f448d1e
Added more cases where NamedType get qname information and adjusted t…
Masara Nov 29, 2023
75df41e
Added more cases where NamedType get qname information and added more…
Masara Nov 29, 2023
684d40a
Adjusted test snapshots
Masara Nov 29, 2023
d6b8bf7
API-Analyzer: Better handling for Any types and illegal list and set …
Masara Nov 29, 2023
e171213
Added more test cases
Masara Nov 29, 2023
006f98b
Refactoring tests
Masara Nov 29, 2023
5bd771d
Added test case for snake_case to camelCase converter
Masara Nov 29, 2023
852d48f
Issue #39: The snake_case to camelCase converter can be toggled with …
Masara Nov 29, 2023
f0ffa79
Merge branch 'main' into stubs_generation
lars-reimann Nov 30, 2023
6b29c16
Mostly refactoring, fixing some bugs and adding test cases
Masara Nov 30, 2023
757b667
Merge remote-tracking branch 'origin/stubs_generation' into stubs_gen…
Masara Nov 30, 2023
b654b68
Removed OptionalType class, since it is not in use.
Masara Nov 30, 2023
0a2b3cf
Fixed a bug where "None" could not be set as the default value for pa…
Masara Nov 30, 2023
509fb30
Updated snapshot tests
Masara Nov 30, 2023
7e7eb94
Updated snapshot tests
Masara Nov 30, 2023
e5b5b1a
Linter and bug fixes
Masara Nov 30, 2023
794020d
Linter and bug fixes
Masara Nov 30, 2023
5f8bfc7
Changed test data directory and file names, since the GitHub test run…
Masara Nov 30, 2023
71090e4
Changed test data directory and file names, since the GitHub test run…
Masara Nov 30, 2023
c680399
Added .pytest.ini for pytest to ignore tests/data directory while sea…
Masara Nov 30, 2023
6a57205
Added a temporary raise for the github test runner
Masara Nov 30, 2023
d7bd3b9
Changed test for github test runner
Masara Nov 30, 2023
8a66557
Changed test for github test runner
Masara Nov 30, 2023
876cb95
Changed test for github test runner
Masara Nov 30, 2023
7053942
Changed test for github test runner
Masara Nov 30, 2023
3435a54
Linter fixes
Masara Nov 30, 2023
b9d8fa2
style: apply automated linter fixes
megalinter-bot Nov 30, 2023
944c863
style: apply automated linter fixes
megalinter-bot Nov 30, 2023
e78a3a5
codecov & refactoring
Masara Nov 30, 2023
f3604f1
Merge remote-tracking branch 'origin/stubs_generation' into stubs_gen…
Masara Nov 30, 2023
bd4a5ff
codecov & refactoring
Masara Nov 30, 2023
df1a986
codecov & refactoring
Masara Nov 30, 2023
64e26a2
codecov & refactoring
Masara Nov 30, 2023
93bc3cc
codecov & refactoring
Masara Nov 30, 2023
6de83ce
chore: swap two config sections
lars-reimann Dec 1, 2023
2be7ef4
chore: move pytest config into `pyproject.toml`
lars-reimann Dec 1, 2023
599dbad
Merge branch 'main' into stubs_generation
lars-reimann Dec 1, 2023
cb9d9d9
[Refactoring] Renamed Variance class to TypeParameter and removed fro…
Masara Dec 4, 2023
3609a29
Merge remote-tracking branch 'origin/stubs_generation' into stubs_gen…
Masara Dec 4, 2023
09c2586
[Refactoring] Removed the is_abstract constructor field from the Clas…
Masara Dec 4, 2023
077f192
[Refactoring] Removed the default_is_none constructor field from the …
Masara Dec 4, 2023
d058b83
[Refactoring] Fixed typos
Masara Dec 4, 2023
1afef9f
[Refactoring] Changed call argument from -ci to -nc
Masara Dec 4, 2023
01864b8
[Refactoring]
Masara Dec 4, 2023
566e929
Refactoring and bug fixing
Masara Dec 4, 2023
d7ff691
bug fix
Masara Dec 4, 2023
032027c
Removed unused is_type_inferred information
Masara Dec 5, 2023
77c4be8
LiteralTypes are joined for stubs
Masara Dec 5, 2023
2fe4bb8
Added test data for codecov
Masara Dec 5, 2023
8dcb2e2
refactoring
Masara Dec 5, 2023
779e87c
style: apply automated linter fixes
megalinter-bot Dec 5, 2023
09fb9cd
Fixing tests
Masara Dec 5, 2023
936a557
Merge remote-tracking branch 'origin/stubs_generation' into stubs_gen…
Masara Dec 5, 2023
2df5093
fixing a bug where None default values would not correctly be parsed,…
Masara Dec 6, 2023
63c7d03
codecov and linter fixes
Masara Dec 6, 2023
64ad07f
style: apply automated linter fixes
megalinter-bot Dec 6, 2023
a08450d
style: apply automated linter fixes
megalinter-bot Dec 6, 2023
0c26c80
codecov
Masara Dec 6, 2023
20282b0
Merge remote-tracking branch 'origin/stubs_generation' into stubs_gen…
Masara Dec 6, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/safeds_stubgen/api_analyzer/_mypy_helpers.py
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ def mypy_type_to_abstract_type(
elif isinstance(mypy_type, mp_types.NoneType):
return sds_types.NamedType(name="None", qname="builtins.None")
elif isinstance(mypy_type, mp_types.LiteralType):
return sds_types.LiteralType(literal=mypy_type.value)
return sds_types.LiteralType(literals=[mypy_type.value])
elif isinstance(mypy_type, mp_types.UnboundType):
if mypy_type.name in {"list", "set"}:
return {
Expand Down
8 changes: 4 additions & 4 deletions src/safeds_stubgen/api_analyzer/_types.py
Original file line number Diff line number Diff line change
Expand Up @@ -337,17 +337,17 @@ def __hash__(self) -> int:

@dataclass(frozen=True)
class LiteralType(AbstractType):
literal: str | int | float | bool
literals: list[str | int | float | bool]

@classmethod
def from_dict(cls, d: dict[str, Any]) -> LiteralType:
return LiteralType(d["literal"])
return LiteralType(d["literals"])

def to_dict(self) -> dict[str, Any]:
return {"kind": self.__class__.__name__, "literal": self.literal}
return {"kind": self.__class__.__name__, "literals": [literal for literal in self.literals]}
Masara marked this conversation as resolved.
Show resolved Hide resolved

def __hash__(self) -> int:
return hash(frozenset([self.literal]))
return hash(frozenset(self.literals))


@dataclass(frozen=True)
Expand Down
39 changes: 33 additions & 6 deletions src/safeds_stubgen/stubs_generator/_generate_stubs.py
Original file line number Diff line number Diff line change
Expand Up @@ -404,7 +404,7 @@
default_value = f"{param_default_value}"
param_value = f" = {default_value}"
elif parameter.is_optional and param_default_value is None:
param_value = " = null"

Check warning on line 407 in src/safeds_stubgen/stubs_generator/_generate_stubs.py

View check run for this annotation

Codecov / codecov/patch

src/safeds_stubgen/stubs_generator/_generate_stubs.py#L407

Added line #L407 was not covered by tests

# Mypy assignes *args parameters the tuple type, which is not supported in Safe-DS. Therefor we
# overwrite it and set the type to a list.
Expand Down Expand Up @@ -569,8 +569,32 @@
return f"{name}<{', '.join(types)}>"
return f"{name}<Any>"
elif kind == "UnionType":
# Union items have to be unique. 'types' has to be a sorted list, since otherwise the snapshot tests would
# fail b/c element order in sets is non-deterministic.
# In Mypy LiteralTypes are getting seperated into unions of LiteralTypes,
# and we have to join them for the stubs.
literal_data = []
other_type_data = []
for type_information in type_data["types"]:
if type_information["kind"] == "LiteralType":
literal_data.append(type_information)
else:
other_type_data.append(type_information)

if len(literal_data) >= 2:
all_literals = [
literal_type
for literal in literal_data
for literal_type in literal["literals"]
]

# We overwrite the old types of the union with the joined literal types
type_data["types"] = other_type_data
type_data["types"].append({
"kind": "LiteralType",
"literals": all_literals,
})

# Union items have to be unique, therefore we use sets. But the types set has to be a sorted list, since
# otherwise the snapshot tests would fail b/c element order in sets is non-deterministic.
types = list({self._create_type_string(type_) for type_ in type_data["types"]})
types.sort()

Expand All @@ -596,10 +620,13 @@
value_data = self._create_type_string(type_data["value_type"])
return f"Map<{key_data}, {value_data}>"
elif kind == "LiteralType":
literal_type = type_data["literal"]
if isinstance(literal_type, str):
literal_type = f'"{literal_type}"'
return f"literal<{literal_type}>"
types = []
for literal_type in type_data["literals"]:
if isinstance(literal_type, str):
types.append(f'"{literal_type}"')
else:
types.append(f'{literal_type}')
return f"literal<{', '.join(types)}>"

raise ValueError(f"Unexpected type: {kind}") # pragma: no cover

Expand Down
2 changes: 2 additions & 0 deletions tests/data/various_modules_package/attribute_module.py
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,8 @@ def some_func() -> bool:
finals: Final[str, int] = "Value"
final_union: Final[str | int] = "Value"
literal: Literal["Some String"]
multiple_literals: Literal["Literal_1", "Literal_2", 3, True]
mixed_literal_union: Literal["L1", 2] | int | Literal[4, False] | str

multi_attr_1, _multi_attr_2_private = (123456, "I am a String")
multi_attr_3 = _multi_attr_4_private = ["I am some", "kind of list"]
Expand Down
118 changes: 111 additions & 7 deletions tests/safeds_stubgen/api_analyzer/__snapshots__/test__get_api.ambr
Original file line number Diff line number Diff line change
Expand Up @@ -490,7 +490,59 @@
'name': 'literal',
'type': dict({
'kind': 'LiteralType',
'literal': 'Some String',
'literals': list([
'Some String',
]),
}),
}),
dict({
'docstring': dict({
'default_value': '',
'description': '',
'type': '',
}),
'id': 'various_modules_package/attribute_module/AttributesClassB/mixed_literal_union',
'is_public': True,
'is_static': True,
'name': 'mixed_literal_union',
'type': dict({
'kind': 'UnionType',
'types': list([
dict({
'kind': 'LiteralType',
'literals': list([
'L1',
]),
}),
dict({
'kind': 'LiteralType',
'literals': list([
2,
]),
}),
dict({
'kind': 'NamedType',
'name': 'int',
'qname': 'builtins.int',
}),
dict({
'kind': 'LiteralType',
'literals': list([
4,
]),
}),
dict({
'kind': 'LiteralType',
'literals': list([
False,
]),
}),
dict({
'kind': 'NamedType',
'name': 'str',
'qname': 'builtins.str',
}),
]),
}),
}),
dict({
Expand Down Expand Up @@ -594,6 +646,46 @@
'qname': 'builtins.str',
}),
}),
dict({
'docstring': dict({
'default_value': '',
'description': '',
'type': '',
}),
'id': 'various_modules_package/attribute_module/AttributesClassB/multiple_literals',
'is_public': True,
'is_static': True,
'name': 'multiple_literals',
'type': dict({
'kind': 'UnionType',
'types': list([
dict({
'kind': 'LiteralType',
'literals': list([
'Literal_1',
]),
}),
dict({
'kind': 'LiteralType',
'literals': list([
'Literal_2',
]),
}),
dict({
'kind': 'LiteralType',
'literals': list([
3,
]),
}),
dict({
'kind': 'LiteralType',
'literals': list([
True,
]),
}),
]),
}),
}),
dict({
'docstring': dict({
'default_value': '',
Expand Down Expand Up @@ -2239,11 +2331,15 @@
'types': list([
dict({
'kind': 'LiteralType',
'literal': 1,
'literals': list([
1,
]),
}),
dict({
'kind': 'LiteralType',
'literal': 2,
'literals': list([
2,
]),
}),
]),
}),
Expand Down Expand Up @@ -2290,11 +2386,15 @@
'types': list([
dict({
'kind': 'LiteralType',
'literal': 1,
'literals': list([
1,
]),
}),
dict({
'kind': 'LiteralType',
'literal': 2,
'literals': list([
2,
]),
}),
]),
}),
Expand Down Expand Up @@ -3513,7 +3613,9 @@
'name': 'literal',
'type': dict({
'kind': 'LiteralType',
'literal': 'Some String',
'literals': list([
'Some String',
]),
}),
}),
dict({
Expand Down Expand Up @@ -4626,7 +4728,9 @@
'name': 'result_1',
'type': dict({
'kind': 'LiteralType',
'literal': 'Some String',
'literals': list([
'Some String',
]),
}),
}),
])
Expand Down
12 changes: 6 additions & 6 deletions tests/safeds_stubgen/api_analyzer/test_types.py
Original file line number Diff line number Diff line change
Expand Up @@ -221,20 +221,20 @@ def test_set_type() -> None:


def test_literal_type() -> None:
type_ = LiteralType("Literal_1")
type_ = LiteralType(["Literal_1", 2])
type_dict = {
"kind": "LiteralType",
"literal": "Literal_1",
"literals": ["Literal_1", 2],
}

assert AbstractType.from_dict(type_dict) == type_
assert LiteralType.from_dict(type_dict) == type_
assert type_.to_dict() == type_dict

assert LiteralType("a") == LiteralType("a")
assert hash(LiteralType("a")) == hash(LiteralType("a"))
assert LiteralType("a") != LiteralType("b")
assert hash(LiteralType("a")) != hash(LiteralType("b"))
assert LiteralType(["a"]) == LiteralType(["a"])
assert hash(LiteralType(["a"])) == hash(LiteralType(["a"]))
assert LiteralType(["a"]) != LiteralType(["b"])
assert hash(LiteralType(["a"])) != hash(LiteralType(["b"]))


def test_final_type() -> None:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,10 @@
@PythonName("final_union")
static attr finalUnion: union<Int, String>
static attr `literal`: literal<"Some String">
@PythonName("multiple_literals")
static attr multipleLiterals: literal<"Literal_1", "Literal_2", 3, True>
@PythonName("mixed_literal_union")
static attr mixedLiteralUnion: union<Int, String, literal<"L1", 2, 4, False>>
@PythonName("multi_attr_1")
static attr multiAttr1: Int
@PythonName("multi_attr_3")
Expand Down
Loading