Add min, max Spark aggregate functions #9868
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facebook-github-bot
added
the
CLA Signed
✅ Deploy Preview for meta-velox canceled.
|
zhli1142015
changed the title
zhli1142015
force-pushed
the
min_max_timestamp
branch
from
82b5b7f
to
a4cda31
Compare
rui-mo
approved these changes
May 21, 2024
zhli1142015
force-pushed
the
min_max_timestamp
branch
from
8ecc602
to
7488e38
Compare
|
@mbasmanova and @Yuhta , could you help to review? Thanks. |
zhli1142015
force-pushed
the
min_max_timestamp
branch
3 times, most recently
from
d2676f3
to
dfa9e24
Compare
Kindly ping, could you help to review? Thanks. |
zhli1142015
force-pushed
the
min_max_timestamp
branch
from
f62e8fd
to
238c4f6
Compare
|
@jinchengchenghh , do you have more comments? Thanks. |
jinchengchenghh
approved these changes
Jun 3, 2024
zhli1142015
force-pushed
the
min_max_timestamp
branch
2 times, most recently
from
3e0d02d
to
954f780
Compare
|
@mbasmanova , could you please help review this PR? Thanks. |
zhli1142015
force-pushed
the
min_max_timestamp
branch
3 times, most recently
from
d276b70
to
8f239a3
Compare
zhli1142015
force-pushed
the
min_max_timestamp
branch
from
8f239a3
to
42ccc17
Compare
|
@rui-mo , is it ok to merge this PR? Thanks. |
zhli1142015
force-pushed
the
min_max_timestamp
branch
from
8d00f3a
to
de9d83f
Compare
mbasmanova
reviewed
Jul 22, 2024
| @@ -290,6 +290,20 @@ struct Timestamp { | |||
| /// successful. | |||
| static int64_t calendarUtcToEpoch(const std::tm& tm); | |||
|
|
|||
| /// Truncates a Timestamp value to the specified precision. | |||
| static Timestamp truncate(Timestamp ts, TimestampPrecision precision) { | |||
mbasmanova
added
the
ready-to-merge
|
@kevinwilfong has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@zhli1142015 Would you resolve the conflicts? Thanks. |
zhli1142015
force-pushed
the
min_max_timestamp
branch
from
a794277
to
ae6a473
Compare
Done, thanks. |
|
@kevinwilfong has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
Sorry but could you rebase again, I'm still seeing conflicts in velox/functions/lib/aggregates/CMakeLists.txt |
zhli1142015
force-pushed
the
min_max_timestamp
branch
from
ae6a473
to
c9ff748
Compare
|
@kevinwilfong has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@kevinwilfong merged this pull request in e5671c0. |
weiting-chen
pushed a commit
to weiting-chen/velox
that referenced
this pull request
Nov 12, 2024
…ox/functions/prestosql/aggregates/MinMaxAggregates.cpp Add min, max Spark aggregate functions (facebookincubator#9868) Summary: There are two semantic differences between Presto and Spark. 1. Nested NULLs are compared as values in Spark and as "unknown value" in Presto. 2. The timestamp type represents a time instant in microsecond precision in Spark, but millisecond precision in Presto. Therefore, we need to implement min and max functions for Spark. In this PR, 1. Move Presto `min` and `max` aggregation function implements to lib folder. 2. Add `getMinFunctionFactory` and `getMaxFunctionFactory` which allow callers to register max & min functions with different behaviors. Pull Request resolved: facebookincubator#9868 Reviewed By: mbasmanova Differential Revision: D60051468 Pulled By: kevinwilfong fbshipit-source-id: 1f056420d6909174a35d336e4e1b413a87ef7665
weiting-chen
added a commit
to oap-project/velox
that referenced
this pull request
Nov 14, 2024
…ox/functions/prestosql/aggregates/MinMaxAggregates.cpp (#503) Add min, max Spark aggregate functions (facebookincubator#9868) Summary: There are two semantic differences between Presto and Spark. 1. Nested NULLs are compared as values in Spark and as "unknown value" in Presto. 2. The timestamp type represents a time instant in microsecond precision in Spark, but millisecond precision in Presto. Therefore, we need to implement min and max functions for Spark. In this PR, 1. Move Presto `min` and `max` aggregation function implements to lib folder. 2. Add `getMinFunctionFactory` and `getMaxFunctionFactory` which allow callers to register max & min functions with different behaviors. Pull Request resolved: facebookincubator#9868 Reviewed By: mbasmanova Differential Revision: D60051468 Pulled By: kevinwilfong fbshipit-source-id: 1f056420d6909174a35d336e4e1b413a87ef7665 Co-authored-by: zhli1142015 <zhli@microsoft.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There are two semantic differences between Presto and Spark.
Presto.
Spark, but millisecond precision in Presto.
Therefore, we need to implement min and max functions for Spark. In this PR,
minandmaxaggregation function implements to lib folder.getMinFunctionFactoryandgetMaxFunctionFactorywhich allow callersto register max & min functions with different behaviors.