Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python-native typing support #136

Merged
merged 124 commits into from
Jan 26, 2021
Merged

Python-native typing support #136

merged 124 commits into from
Jan 26, 2021

Conversation

wild-endeavor
Copy link
Contributor

@wild-endeavor wild-endeavor commented Jul 13, 2020

TL;DR

Please see the proposal doc at https://docs.google.com/document/d/17rNKg6Uvow8CrECaPff96Tarr87P2fn4ilf_Tv2lYd4
https://flytecookbook.readthedocs.io/en/latest/

Type

  • Bug Fix
  • Feature
  • Plugin

Are all requirements met?

  • Code completed
  • Smoke tested
  • Unit tests added
  • Code documentation added
  • Any pending items have an associated Issue

retries=0,
interruptible=None,
deprecated='',
storage_request=None,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i want to club the requests and limits as structs. This simplifies adding new ones

cpu_limit=None,
gpu_limit=None,
memory_limit=None,
cache=False,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

samething for cache?

@workflow(outputs=["real_b"])
def my_workflow() -> WorkflowOutputs:
# a = x(s=3)
b = x(s=x(s=3))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we get a little more complex example, where we have 3 nodes, multiple outputs and inputs?

@codecov-commenter
Copy link

codecov-commenter commented Aug 6, 2020

Codecov Report

Merging #136 into master will decrease coverage by 0.46%.
The diff coverage is 71.87%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #136      +/-   ##
==========================================
- Coverage   80.88%   80.41%   -0.47%     
==========================================
  Files         225      235      +10     
  Lines       14669    15441     +772     
  Branches     1204     1323     +119     
==========================================
+ Hits        11865    12417     +552     
- Misses       2521     2706     +185     
- Partials      283      318      +35     
Impacted Files Coverage Δ
flytekit/common/types/helpers.py 69.64% <ø> (ø)
flytekit/contrib/notebook/supported_types.py 100.00% <ø> (ø)
flytekit/engines/flyte/engine.py 62.50% <ø> (ø)
flytekit/engines/unit/engine.py 80.45% <ø> (ø)
flytekit/models/core/workflow.py 100.00% <ø> (ø)
flytekit/models/workflow_closure.py 100.00% <ø> (ø)
tests/flytekit/unit/sdk/tasks/test_hive_tasks.py 94.28% <ø> (ø)
flytekit/bin/entrypoint.py 69.11% <14.28%> (-13.34%) ⬇️
flytekit/engine.py 25.80% <25.80%> (ø)
flytekit/typing.py 57.89% <57.89%> (ø)
... and 28 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bf3c40f...6f333cd. Read the comment docs.

@codecov-io
Copy link

codecov-io commented Oct 7, 2020

Codecov Report

Merging #136 (4de24ea) into master (8e96772) will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master     #136   +/-   ##
=======================================
  Coverage   96.00%   96.00%           
=======================================
  Files           2        2           
  Lines          75       75           
  Branches        8        8           
=======================================
  Hits           72       72           
  Misses          1        1           
  Partials        2        2           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8e96772...4de24ea. Read the comment docs.

kumare3 and others added 8 commits October 7, 2020 14:15
* comparison working with operator overloading

* Branching work in progress

* Complete evaluation

* Branches working

* All conditional cases

* Update flytekit/common/workflow.py

Co-authored-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>

* fixed one comment

* Fixes to execution state

Co-authored-by: Ketan Umare <kumare@lyft.com>
Co-authored-by: Ketan Umare <16888709+kumare3@users.noreply.github.com>
@wild-endeavor wild-endeavor changed the title [wip] do not review [pending] Python-native typing support Jan 12, 2021
@wild-endeavor wild-endeavor changed the title [pending] Python-native typing support Python-native typing support Jan 25, 2021
@kumare3 kumare3 self-requested a review January 25, 2021 23:59
Copy link
Contributor

@kumare3 kumare3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh My God! Hurrah!

@wild-endeavor wild-endeavor merged commit 701d4e8 into master Jan 26, 2021
max-hoffman pushed a commit to dolthub/flytekit that referenced this pull request May 11, 2021
ringohoffman added a commit to ringohoffman/flytekit that referenced this pull request Oct 12, 2023
Fixes #3680; this pin was added in flyteorg#136: https://github.com/flyteorg/flytekit/pull/136/files#diff-60f61ab7a8d1910d86d9fda2261620314edcae5894d5aaa236b821c7256badd7R18 ~3 years ago when the latest pandas version was 1.2.1; flytekit is not using any features that have broken any of its tests in upgrading to use pandas >=2.0.0
@ringohoffman ringohoffman mentioned this pull request Oct 12, 2023
8 tasks
ringohoffman added a commit to ringohoffman/flytekit that referenced this pull request Oct 12, 2023
Fixes #3680; this pin was added in flyteorg#136: https://github.com/flyteorg/flytekit/pull/136/files#diff-60f61ab7a8d1910d86d9fda2261620314edcae5894d5aaa236b821c7256badd7R18 ~3 years ago when the latest pandas version was 1.2.1; flytekit is not using any features that have broken any of its tests in upgrading to use pandas >=2.0.0

Signed-off-by: Matthew Hoffman <matthew@protopia.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants