Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prepopulate ArrayNode output literals with TaskNode interface output variables #5080

Merged
merged 3 commits into from
Mar 20, 2024

Conversation

hamersaw
Copy link
Contributor

Tracking issue

NA

Why are the changes needed?

When ArrayNode is executed with an empty input collection, it writes a 0 length output file. This results in an error when resolving downstream literals, for example:

failed at Node[end-node]. BindingResolutionError: Error binding Var [wf].[o0], caused by: failed at Node[n0]. OutputsNotFoundError: Outputs not found at [s3://my-s3-bucket/metadata/propeller/flytesnacks-development-f91c3120d59d74d4783b/n0/data/0/outputs.pb]

What changes were proposed in this pull request?

This PR prepopulates the outputLiteral of an ArrayNode with empty collections using the subNode output variable names. Therefore, when provided an input collection of 0 length we correctly write an output collection of 0 length.

How was this patch tested?

locally.

Setup process

Screenshots

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Related PRs

NA

Docs link

NA

Signed-off-by: Daniel Rammer <daniel@union.ai>
Copy link

codecov bot commented Mar 20, 2024

Codecov Report

Attention: Patch coverage is 65.38462% with 9 lines in your changes are missing coverage. Please review.

Project coverage is 59.00%. Comparing base (eb24107) to head (a829d4b).
Report is 1 commits behind head on master.

Files Patch % Lines
...ytepropeller/pkg/controller/nodes/array/handler.go 65.38% 8 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #5080      +/-   ##
==========================================
+ Coverage   58.98%   59.00%   +0.01%     
==========================================
  Files         645      645              
  Lines       55590    55616      +26     
==========================================
+ Hits        32791    32814      +23     
- Misses      20204    20207       +3     
  Partials     2595     2595              
Flag Coverage Δ
unittests 59.00% <65.38%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Daniel Rammer <daniel@union.ai>
@hamersaw hamersaw requested review from EngHabu and kumare3 March 20, 2024 16:00
Signed-off-by: Daniel Rammer <daniel@union.ai>
@hamersaw hamersaw marked this pull request as ready for review March 20, 2024 16:02
@dosubot dosubot bot added the size:M This PR changes 30-99 lines, ignoring generated files. label Mar 20, 2024
@hamersaw hamersaw requested a review from pvditt March 20, 2024 16:02
@dosubot dosubot bot added documentation Improvements or additions to documentation enhancement New feature or request labels Mar 20, 2024
@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Mar 20, 2024
@hamersaw hamersaw merged commit 1caa86d into master Mar 20, 2024
48 checks passed
@hamersaw hamersaw deleted the bug/arraynode-empty-inputs branch March 20, 2024 22:01
yubofredwang pushed a commit to yubofredwang/flyte that referenced this pull request Mar 26, 2024
…variables (flyteorg#5080)

* prepopulate output literals with TaskNode interface output variables

Signed-off-by: Daniel Rammer <daniel@union.ai>

* added unit test

Signed-off-by: Daniel Rammer <daniel@union.ai>

* docs

Signed-off-by: Daniel Rammer <daniel@union.ai>

---------

Signed-off-by: Daniel Rammer <daniel@union.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation enhancement New feature or request lgtm This PR has been approved by a maintainer size:M This PR changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants