Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[mypy] Improve types in _grammar.py #1002

Merged
merged 5 commits into from
Aug 30, 2024

Conversation

riedgar-ms
Copy link
Collaborator

@riedgar-ms riedgar-ms commented Aug 27, 2024

Working to improve the type annotations in _grammar.py:

  • Try to sort out class constructors in GrammarFunction.... at least make them less bad
  • Introduce a ComposableGrammar type (which would ideally be crammed back to GrammarFunction)
  • Various other minor fixes
  • Remove commented out functions
  • Re-blacken

guidance/_grammar.py Outdated Show resolved Hide resolved
@codecov-commenter
Copy link

codecov-commenter commented Aug 27, 2024

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

Attention: Patch coverage is 85.88235% with 12 lines in your changes missing coverage. Please review.

Project coverage is 61.24%. Comparing base (56a419a) to head (5aa42f8).

Files with missing lines Patch % Lines
guidance/_grammar.py 86.25% 11 Missing ⚠️
guidance/library/_substring.py 75.00% 1 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1002      +/-   ##
==========================================
- Coverage   62.73%   61.24%   -1.50%     
==========================================
  Files          62       62              
  Lines        4396     4412      +16     
==========================================
- Hits         2758     2702      -56     
- Misses       1638     1710      +72     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@@ -494,37 +447,31 @@ def commit_point(value, hidden=False):
raise NotImplementedError("commit_point is not implemented (may remove in the future)")


def as_regular_grammar(value):
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had to move this lower when I added type annotations.

self.values = [v for v in values if not isinstance(v, Null)]
super().__init__(capture_name=None)
# wrap raw strings
converted_values = [string(v) if isinstance(v, (str, bytes)) else v for v in values]
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TBH, I'd prefer to have Select and Join be strict in their accepted types, and have all the type conversions done by select() and join().

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Totally agreed. Bit of a (related, I think) side note...

How would you feel about making Join a generic type with one argument, bounded above by Function?

Then we can make a String class that subclasses Join[Bytes], where we can have some nice semantics in join, e.g. join(*args: String) -> String. Might be able to do something similar with Select. With the right type annotations, we might be able to get the type-checker to recognize errors like selecting over RawFunctions. Am I making any sense? (I can throw together a little POC if not...)

if name is not None:
name = "__LIST_APPEND:" + name
else:
raise ValueError("list_append requires a name")
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New little bit of logic here

@riedgar-ms riedgar-ms merged commit 46e5144 into guidance-ai:main Aug 30, 2024
100 checks passed
@riedgar-ms riedgar-ms deleted the riedgar-ms/grammar-types-01 branch August 30, 2024 10:58
@@ -322,9 +329,8 @@ def __add__(self, other):
return other
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this is already merged -- just looking over it though. Thanks for doing this..!

Do we want to check if other is a str here and upcast the return type via string(other)?

stack = [
(grammar, None, None)
] # Stack stores tuples of (node, parent_node, child_index)
stack = [(grammar, None, None)] # Stack stores tuples of (node, parent_node, child_index)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a type annotation that matches the comment?

@@ -919,14 +869,23 @@ def str_to_grammar(value: str):
return partial_grammar


def _is_string_literal(node: GrammarFunction):
def _is_string_literal(node: ComposableGrammar) -> bool:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would essentially be the TypeGuard for the proposed String = Join[Bytes] class

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants