-
Notifications
You must be signed in to change notification settings - Fork 277
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[release v1.2] Backport v1.6 #1645
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Ketan Umare <ketan.umare@gmail.com> Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
* Remove retry from flytekit's setup.py and regenerate requirements Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> * Add to sagemaker Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> * Remove retry from sagemaker plugin requirements file Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> * Restore doc-requirements.txt Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com> * Fix bad merge Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com> --------- Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com> Co-authored-by: Eduardo Apolinario <eapolinario@users.noreply.github.com>
Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
Signed-off-by: Ketan Umare <ketan.umare@gmail.com> Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com> Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
* fix: Make FlyteFile compatible with Annotated[..., HashMethod] See issue #3424 Signed-off-by: Adrian Rumpold <a.rumpold@gmail.com> * tests: Add test case for FlyteFile with HashMethod annotation Issue: #3424 Signed-off-by: Adrian Rumpold <a.rumpold@gmail.com> * fix: Use typing_extensions.Annotated for py3.8 compatibility Issue: #3424 Signed-off-by: Adrian Rumpold <a.rumpold@gmail.com> * fix: Use `get_args` and `get_origin` from typing_extensions for py3.8 compatibility Issue: #3424 Signed-off-by: Adrian Rumpold <a.rumpold@gmail.com> * fix(tests): Use fixture for local dummy file Signed-off-by: Adrian Rumpold <a.rumpold@gmail.com> --------- Signed-off-by: Adrian Rumpold <a.rumpold@gmail.com>
* move FlyteSchema deprecation warning to initialization method Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com> * update Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com> --------- Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>
…1515) * add pod_template and pod_template_name arguments for ContainerTask Signed-off-by: Felix Ruess <felix.ruess@roboception.de> * factor out _serialize_pod_spec into separate util function Signed-off-by: Felix Ruess <felix.ruess@roboception.de> * model file changes, couple other changes Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * minor cleanup Signed-off-by: Felix Ruess <felix.ruess@roboception.de> * add unit test for container_task pod_template Signed-off-by: Felix Ruess <felix.ruess@roboception.de> * bump min version of flyteidl to 1.3.12 for pod template data config support Signed-off-by: Felix Ruess <felix.ruess@roboception.de> * require flyteidl==1.3.12 in doc-requirements.txt Signed-off-by: Felix Ruess <felix.ruess@roboception.de> --------- Signed-off-by: Felix Ruess <felix.ruess@roboception.de> Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> Co-authored-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
Signed-off-by: franco-bocci <franco.bocci@wolt.com>
* TypeTransformer for TensorFlow model Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * clean up Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * clean up Signed-off-by: Samhita Alla <aallasamhita@gmail.com> * fix lint Signed-off-by: Samhita Alla <aallasamhita@gmail.com> --------- Signed-off-by: Samhita Alla <aallasamhita@gmail.com>
Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
* Device auth flow Signed-off-by: Ketan Umare <ketan.umare@gmail.com> * Device AuthFlow is now available in flytekit Signed-off-by: Ketan Umare <ketan.umare@gmail.com> * unit tests Signed-off-by: Ketan Umare <ketan.umare@gmail.com> * test added Signed-off-by: Ketan Umare <ketan.umare@gmail.com> * updated Signed-off-by: Ketan Umare <ketan.umare@gmail.com> * Fixed test Signed-off-by: Ketan Umare <ketan.umare@gmail.com> * Fixed unit test Signed-off-by: Ketan Umare <ketan.umare@gmail.com> * Fixed lint errors Signed-off-by: Ketan Umare <ketan.umare@gmail.com> --------- Signed-off-by: Ketan Umare <ketan.umare@gmail.com>
* url encode secret first Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * nit Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> --------- Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
* wip Signed-off-by: Kevin Su <pingsutw@apache.org> * wip Signed-off-by: Kevin Su <pingsutw@apache.org> * test Signed-off-by: Kevin Su <pingsutw@apache.org> * nit Signed-off-by: Kevin Su <pingsutw@apache.org> * lint Signed-off-by: Kevin Su <pingsutw@apache.org> * update comment Signed-off-by: Kevin Su <pingsutw@apache.org> * nit Signed-off-by: Kevin Su <pingsutw@apache.org> --------- Signed-off-by: Kevin Su <pingsutw@apache.org>
#1556) Signed-off-by: Ketan Umare <ketan.umare@gmail.com>
Co-authored-by: ggydush-fn <greg.gydush@freenome.com>
Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
* Automatically remove unused import Signed-off-by: Kevin Su <pingsutw@apache.org> * nit Signed-off-by: Kevin Su <pingsutw@apache.org> --------- Signed-off-by: Kevin Su <pingsutw@apache.org>
* Disallow partial lists in map tasks Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com> * Lint Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com> --------- Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com> Co-authored-by: eduardo apolinario <eapolinario@users.noreply.github.com>
Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
* Convert List[Any] to a single pickle file * remove redundant code * keep batchSize only if type contain flytePickle * fix error * add batch support to translate_inputs_to_literals * add ci test Signed-off-by: Yicheng-Lu-llll <luyc58576@gmail.com> * improve comment Signed-off-by: Yicheng-Lu-llll <luyc58576@gmail.com> * add more ci test Signed-off-by: Yicheng-Lu-llll <luyc58576@gmail.com> * improve Signed-off-by: Yicheng-Lu-llll <luyc58576@gmail.com> * handle HashMethod case Signed-off-by: Yicheng-Lu-llll <luyc58576@gmail.com> * improve format Signed-off-by: Yicheng-Lu-llll <luyc58576@gmail.com> * improve Signed-off-by: Yicheng-Lu-llll <luyc58576@gmail.com> * improve Signed-off-by: Yicheng-Lu-llll <luyc58576@gmail.com> * improve Signed-off-by: Yicheng-Lu-llll <luyc58576@gmail.com> * add test_is_batchable Signed-off-by: Yicheng-Lu-llll <luyc58576@gmail.com> * Add BatchSize Signed-off-by: Kevin Su <pingsutw@apache.org> * test_batch_size Signed-off-by: Kevin Su <pingsutw@apache.org> --------- Signed-off-by: Yicheng-Lu-llll <luyc58576@gmail.com> Signed-off-by: Kevin Su <pingsutw@apache.org> Co-authored-by: root <root@ip-172-31-10-58.ec2.internal> Co-authored-by: Kevin Su <pingsutw@apache.org>
Signed-off-by: Samhita Alla <aallasamhita@gmail.com>
* Unify sqlalchemy Dockerfiles Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com> * Simplifyte sqlalchemy Dockerfile Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com> * Set pythonpath to /root Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com> --------- Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com> Co-authored-by: eduardo apolinario <eapolinario@users.noreply.github.com>
* wip Signed-off-by: Kevin Su <pingsutw@apache.org> * wip Signed-off-by: Kevin Su <pingsutw@apache.org> * wip Signed-off-by: Kevin Su <pingsutw@apache.org> * wip Signed-off-by: Kevin Su <pingsutw@apache.org> * wip Signed-off-by: Kevin Su <pingsutw@apache.org> * wip Signed-off-by: Kevin Su <pingsutw@apache.org> * wip Signed-off-by: Kevin Su <pingsutw@apache.org> * wip Signed-off-by: Kevin Su <pingsutw@apache.org> * wip Signed-off-by: Kevin Su <pingsutw@apache.org> * wip Signed-off-by: Kevin Su <pingsutw@apache.org> * wip Signed-off-by: Kevin Su <pingsutw@apache.org> * wip Signed-off-by: Kevin Su <pingsutw@apache.org> * pyflyte build Signed-off-by: Kevin Su <pingsutw@apache.org> * nit Signed-off-by: Kevin Su <pingsutw@apache.org> * nit Signed-off-by: Kevin Su <pingsutw@apache.org> * wip Signed-off-by: Kevin Su <pingsutw@apache.org> * wip Signed-off-by: Kevin Su <pingsutw@apache.org> * wip Signed-off-by: Kevin Su <pingsutw@apache.org> * Support serialize and package Signed-off-by: Kevin Su <pingsutw@apache.org> * more tests Signed-off-by: Kevin Su <pingsutw@apache.org> * wip Signed-off-by: Kevin Su <pingsutw@apache.org> * wip Signed-off-by: Kevin Su <pingsutw@apache.org> * wip Signed-off-by: Kevin Su <pingsutw@apache.org> * move to plugin Signed-off-by: Kevin Su <pingsutw@apache.org> * test Signed-off-by: Kevin Su <pingsutw@apache.org> * nit Signed-off-by: Kevin Su <pingsutw@apache.org> * test Signed-off-by: Kevin Su <pingsutw@apache.org> * test Signed-off-by: Kevin Su <pingsutw@apache.org> * test Signed-off-by: Kevin Su <pingsutw@apache.org> * test Signed-off-by: Kevin Su <pingsutw@apache.org> * wip Signed-off-by: Kevin Su <pingsutw@apache.org> * wip Signed-off-by: Kevin Su <pingsutw@apache.org> * wip Signed-off-by: Kevin Su <pingsutw@apache.org> * wip Signed-off-by: Kevin Su <pingsutw@apache.org> * nit Signed-off-by: Kevin Su <pingsutw@apache.org> * fixed tested Signed-off-by: Kevin Su <pingsutw@apache.org> * fixed tested Signed-off-by: Kevin Su <pingsutw@apache.org> * more tests Signed-off-by: Kevin Su <pingsutw@apache.org> * Add support passing yaml in pyflyte run Signed-off-by: Kevin Su <pingsutw@apache.org> * lint Signed-off-by: Kevin Su <pingsutw@apache.org> * lint Signed-off-by: Kevin Su <pingsutw@apache.org> * nit Signed-off-by: Kevin Su <pingsutw@apache.org> * nit Signed-off-by: Kevin Su <pingsutw@apache.org> --------- Signed-off-by: Kevin Su <pingsutw@apache.org>
…ror messages (#1582) * improve input type conversion error Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com> * fix lint Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com> * fix Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com> * add tests Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com> * add tests Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com> * add rich Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com> * fix lint Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com> * remove prototyping script, update loggers Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com> * update __init__.py Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com> * update logger Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com> * update logger Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com> * fix GE and pandera tests Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com> * fix lint Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com> --------- Signed-off-by: Niels Bantilan <niels.bantilan@gmail.com>
* Add support nested FlyteFile in pyflyte run Signed-off-by: Kevin Su <pingsutw@apache.org> * nit Signed-off-by: Kevin Su <pingsutw@apache.org> * nit Signed-off-by: Kevin Su <pingsutw@apache.org> --------- Signed-off-by: Kevin Su <pingsutw@apache.org>
* Set cache_regions=True in the case of s3fs * Linting Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com> * Fix tests Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com> * More linting Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com> --------- Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com> Co-authored-by: eduardo apolinario <eapolinario@users.noreply.github.com>
* set default ComSpec when running on Windows Signed-off-by: Kevin Su <pingsutw@apache.org> * lint Signed-off-by: Kevin Su <pingsutw@apache.org> * nit Signed-off-by: Kevin Su <pingsutw@apache.org> --------- Signed-off-by: Kevin Su <pingsutw@apache.org>
* add activate-launchplan command to pyflyte Signed-off-by: Artem Petrov <58334441+wckdman@users.noreply.github.com> * fix linting Signed-off-by: Artem Petrov <58334441+wckdman@users.noreply.github.com> * rework launchplan command Signed-off-by: Artem Petrov <58334441+wckdman@users.noreply.github.com> * fix linting Signed-off-by: Artem Petrov <58334441+wckdman@users.noreply.github.com> --------- Signed-off-by: Artem Petrov <58334441+wckdman@users.noreply.github.com>
* checkpoint Signed-off-by: Mike Zhong <mzhong@embarkvet.com> * Experimental implementation works well. Instead of messing with the class, we utilize the interpolation and context to dynamically generate a wrapper around the desired script. In the wrapper, we cd to the ctx.working_directory, export the env variables (handled by an additional method to convert dict to str), and then pass the arguments to the script Signed-off-by: Mike Zhong <mzhong@embarkvet.com> * fix spacing Signed-off-by: Mike Zhong <mzhong@embarkvet.com> * remove breakpoint Signed-off-by: Mike Zhong <mzhong@embarkvet.com> * Output ctx.working_directory as single output Signed-off-by: Mike Zhong <mzhong@embarkvet.com> * more doc strings Signed-off-by: Mike Zhong <mzhong@embarkvet.com> * more comments Signed-off-by: Mike Zhong <mzhong@embarkvet.com> * Added comments Signed-off-by: Mike Zhong <mzhong@embarkvet.com> * Add tests and test files from other branch Signed-off-by: Mike Zhong <mzhong@embarkvet.com> * minor fix, have function return the instance rather than create. It seems to get registered when flyte packages your project Signed-off-by: Mike Zhong <mzhong@embarkvet.com> * fix tests Signed-off-by: Mike Zhong <mzhong@embarkvet.com> * remove set flags not supported by sh Signed-off-by: Mike Zhong <mzhong@embarkvet.com> * fix spellcheck lint errors Signed-off-by: Mike Zhong <mzhong@embarkvet.com> * don't have a windows equivalent test script so bypassing those tests for now Signed-off-by: Mike Zhong <mzhong@embarkvet.com> * Add typing to make_export_string_from_env_dict params Signed-off-by: Mike Zhong <mzhong@embarkvet.com> * fixup doc string Signed-off-by: Mike Zhong <mzhong@embarkvet.com> * Refactored the new behavior out into a new class. Did not change implementation details at all Signed-off-by: Mike Zhong <mzhong@embarkvet.com> * Address linter issues and up test cov Signed-off-by: Mike Zhong <mzhong@embarkvet.com> * Skip test on windows, no equivalent script Signed-off-by: Mike Zhong <mzhong@embarkvet.com> * Run black and isort, address SC2236 Signed-off-by: Mike Zhong <mzhong@embarkvet.com> * Refactored class name to remove _, make utility function require name so multiple uses in a workflow don't break Signed-off-by: Mike Zhong <mzhong@embarkvet.com> * fix utility function Signed-off-by: Mike Zhong <mzhong@embarkvet.com> * fix utility function call in tests Signed-off-by: Mike Zhong <mzhong@embarkvet.com> * Fix isort on test_shell.py Signed-off-by: Mike Zhong <mzhong@embarkvet.com> * adding logging settings to papermill plugin Signed-off-by: Calvin Leather <cleather@embarkvet.com> * set level to info logging * improved comments/docs --------- Signed-off-by: Mike Zhong <mzhong@embarkvet.com> Signed-off-by: Calvin Leather <cleather@embarkvet.com> Co-authored-by: Mike Zhong <mzhong@embarkvet.com> Co-authored-by: Mike Zhong <mike.zhong89@gmail.com>
* wip Signed-off-by: Kevin Su <pingsutw@apache.org> * Fix mypy errors Signed-off-by: Kevin Su <pingsutw@apache.org> * Fix mypy errors Signed-off-by: Kevin Su <pingsutw@apache.org> * Fix tests Signed-off-by: Kevin Su <pingsutw@apache.org> * Fix tests Signed-off-by: Kevin Su <pingsutw@apache.org> * Fix tests Signed-off-by: Kevin Su <pingsutw@apache.org> * wip Signed-off-by: Kevin Su <pingsutw@apache.org> * wip Signed-off-by: Kevin Su <pingsutw@apache.org> * fix tests Signed-off-by: Kevin Su <pingsutw@apache.org> * fix tests Signed-off-by: Kevin Su <pingsutw@apache.org> * fix test Signed-off-by: Kevin Su <pingsutw@apache.org> * nit Signed-off-by: Kevin Su <pingsutw@apache.org> * Update type Signed-off-by: Kevin Su <pingsutw@apache.org> * Fix tests Signed-off-by: Kevin Su <pingsutw@apache.org> * Fix tests Signed-off-by: Kevin Su <pingsutw@apache.org> * Fix tests Signed-off-by: Kevin Su <pingsutw@apache.org> * nit Signed-off-by: Kevin Su <pingsutw@apache.org> * update dev-requirements.txt Signed-off-by: Kevin Su <pingsutw@apache.org> * Address comment Signed-off-by: Kevin Su <pingsutw@apache.org> * upgrade torch Signed-off-by: Kevin Su <pingsutw@apache.org> * nit Signed-off-by: Kevin Su <pingsutw@apache.org> * lint Signed-off-by: Kevin Su <pingsutw@apache.org> --------- Signed-off-by: Kevin Su <pingsutw@apache.org> Signed-off-by: Kevin Su <pingsutw@gmail.com> Co-authored-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>
Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com>
Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com>
eapolinario
requested review from
wild-endeavor,
kumare3,
pingsutw and
cosmicBboy
as code owners
May 17, 2023 00:57
8 tasks
Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com>
Codecov Report
@@ Coverage Diff @@
## release-v1.2 #1645 +/- ##
===============================================
Coverage ? 71.13%
===============================================
Files ? 331
Lines ? 30348
Branches ? 2907
===============================================
Hits ? 21588
Misses ? 8208
Partials ? 552 |
Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com>
Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com>
Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com>
This reverts commit af49155. Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com>
Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com>
Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com>
Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com>
Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com>
Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com>
Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com>
Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com>
Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com>
Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com>
Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com>
wild-endeavor
approved these changes
May 17, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
TL;DR
Please replace this text with a description of what this PR accomplishes.
Type
Are all requirements met?
Complete description
How did you fix the bug, make the feature etc. Link to any design docs etc
Tracking Issue
https://github.com/flyteorg/flyte/issues/
Follow-up issue
NA
OR
https://github.com/flyteorg/flyte/issues/