You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
With the addition of AST expression support in #3165, we need the ability to generate documentation describing what is supported by an AST expression. We will also have to consider how best to present this in a way that doesn't confuse users with respect to normal GPU expression support.
The text was updated successfully, but these errors were encountered:
We already have a section for each expression saying if it works as a part of lambda or not. Right now nothing works with lambda. But AST is not a one to one mapping to lambda. For some lambda operations, like ArrayTransform, we can do it without any AST support. But we could also do it with AST support. Where as others, like ArraySort, I don't see any good way to do it fully without AST support.
To me it feels like we almost want to replace the Lambda section with an AST section, and then have a way in the docs to explain what operations, expressions or variable require AST support. This could then be expanded in the future to possibly include some hybrid AST/Non-AST support.
So for example with Add we could update it to look like
Expression
SQL Functions(s)
Description
Notes
Context
Param/Output
BOOLEAN
BYTE
SHORT
INT
LONG
FLOAT
DOUBLE
DATE
TIMESTAMP
STRING
DECIMAL
NULL
BINARY
CALENDAR
ARRAY
MAP
STRUCT
UDT
Add
`+`
Addition
None
project
lhs
S
S
S
S
S
S
PS max DECIMAL precision of 18
NS
rhs
S
S
S
S
S
S
PS max DECIMAL precision of 18
NS
result
S
S
S
S
S
S
PS max DECIMAL precision of 18
NS
AST
lhs
NS
NS
S
S
S
S
NS
NS
rhs
NS
NS
S
S
S
S
NS
NS
result
NS
NS
S
S
S
S
NS
NS
And for join when we do support parameters for SparkPlan execs we could have something more like
With the addition of AST expression support in #3165, we need the ability to generate documentation describing what is supported by an AST expression. We will also have to consider how best to present this in a way that doesn't confuse users with respect to normal GPU expression support.
The text was updated successfully, but these errors were encountered: