-
Notifications
You must be signed in to change notification settings - Fork 154
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Operation output check is too restrictive #931
Comments
The fix requires changing in a Producer model to distinguish between an empty schema and an undefined schema. |
wajda
added a commit
that referenced
this issue
Aug 5, 2021
wajda
added a commit
that referenced
this issue
Aug 11, 2021
wajda
added a commit
that referenced
this issue
Aug 11, 2021
wajda
added a commit
that referenced
this issue
Aug 11, 2021
wajda
added a commit
that referenced
this issue
Aug 12, 2021
wajda
changed the title
Operation output check is too tight
Operation output check is too restrictive
Aug 19, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Causes AbsaOSS/spline-spark-agent#264
The following check constraint in
ExecutionPlanPersistentModelBuilder
doesn't assume existence of outputs schema with no attributes ([]
).But there are some specific cases when this can be valid. For instance, when a data frame is created by selecting literals from an empty data frame.
Technically the execution plan could start from a
SELECT
operation, but someone would argue that it would not be 100% correct representation of an execution plan. ASELECT
operation projects something on something else, so it has to be preceded by another (synthetic) operation withNil
output, and we want to preserve that relation.The text was updated successfully, but these errors were encountered: