Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add datetime functions FROM_UNIXTIME and UNIX_TIMESTAMP #835

Merged

Conversation

Yury-Fridlyand
Copy link
Collaborator

FROM_UNIXTIME

  • (DOUBLE) -> DATETIME
  • (DOUBLE, STRING) -> STRING

Implementation details

  • For string formatting DATE_FORMAT function is used
  • Function returns 0 for values outside of range 1970-01-01 00:00:00 - 3001-01-18 23:59:59.999999

NOTES

Example
opensearchsql> select FROM_UNIXTIME(1220249547.12);
fetched rows / total rows = 1/1
+--------------------------------+
| FROM_UNIXTIME(1220249547.12)   |
|--------------------------------|
| 2008-09-01 06:12:27.119999885  |
+--------------------------------+

Test samples

SQL
select FROM_UNIXTIME(1337.42), FROM_UNIXTIME(123235435, '%T %M %D, %Y');
+--------------------------+--------------------------------------------+
| FROM_UNIXTIME(1337.42)   | FROM_UNIXTIME(123235435, '%T %M %D, %Y')   |
|--------------------------+--------------------------------------------|
| 1970-01-01 00:22:17.42   | 08:03:55 November 27th, 1973               |
+--------------------------+--------------------------------------------+
PPL
source=date | top 1 date_keyword | eval `FROM_UNIXTIME(1337.42)` = FROM_UNIXTIME(1337.42), `FROM_UNIXTIME(123235435, '%T %M %D, %Y')` = FROM_UNIXTIME(123235435, '%T %M %D, %Y') | fields `FROM_UNIXTIME(1337.42)`, `FROM_UNIXTIME(123235435, '%T %M %D, %Y')`;
+--------------------------+--------------------------------------------+
| FROM_UNIXTIME(1337.42)   | FROM_UNIXTIME(123235435, '%T %M %D, %Y')   |
|--------------------------+--------------------------------------------|
| 1970-01-01 00:22:17.42   | 08:03:55 November 27th, 1973               |
+--------------------------+--------------------------------------------+

UNIX_TIMESTAMP

  • () -> LONG
  • DATE -> DOUBLE
  • DATETIME -> DOUBLE
  • TIMESTAMP -> DOUBLE
  • DOUBLE -> DOUBLE

Implementation details

  • First function signature calls for LocalDateTime.now() without caching the value. Sequential calls in one query might return different values.
  • Function returns 0 for values outside of range 1970-01-01 00:00:00 - 3001-01-18 23:59:59.999999.

NOTES

Example
opensearchsql> select UNIX_TIMESTAMP(20771122143845.2345);
fetched rows / total rows = 1/1
+---------------------------------------+
| UNIX_TIMESTAMP(20771122143845.2345)   |
|---------------------------------------|
| 3404817524.234375                     |
+---------------------------------------+

Test samples

SQL
select UNIX_TIMESTAMP(), UNIX_TIMESTAMP(20771122143845), UNIX_TIMESTAMP(881226), UNIX_TIMESTAMP(881226231058.123), UNIX_TIMESTAMP(DATE('1995-05-06')), UNIX_TIMESTAMP(TIMESTAMP('2022-08-09 12:16:12')), UNIX_TIMESTAMP(ADDDATE(DATE('2020-08-26'), INTERVAL 1 HOUR));
UNIX_TIMESTAMP()                                             | 1662664900
UNIX_TIMESTAMP(20771122143845)                               | 3404817525.0
UNIX_TIMESTAMP(881226)                                       | 599097600.0
UNIX_TIMESTAMP(881226231058.123)                             | 599181057.1230469
UNIX_TIMESTAMP(DATE('1995-05-06'))                           | 799718400.0
UNIX_TIMESTAMP(TIMESTAMP('2022-08-09 12:16:12'))             | 1660047372.0
UNIX_TIMESTAMP(ADDDATE(DATE('2020-08-26'), INTERVAL 1 HOUR)) | 1598403600.0
PPL
source=date | eval `UNIX_TIMESTAMP()` = UNIX_TIMESTAMP(), `UNIX_TIMESTAMP(20771122143845)` = UNIX_TIMESTAMP(20771122143845), `UNIX_TIMESTAMP(881226)` = UNIX_TIMESTAMP(881226), `UNIX_TIMESTAMP(881226231058.123)` = UNIX_TIMESTAMP(881226231058.123), `UNIX_TIMESTAMP(DATE('1995-05-06'))` = UNIX_TIMESTAMP(DATE('1995-05-06')), `UNIX_TIMESTAMP(TIMESTAMP('2022-08-09 12:16:12'))` = UNIX_TIMESTAMP(TIMESTAMP('2022-08-09 12:16:12')), `UNIX_TIMESTAMP(ADDDATE(DATE('2020-08-26'), INTERVAL 1 HOUR))` = UNIX_TIMESTAMP(ADDDATE(DATE('2020-08-26'), INTERVAL 1 HOUR)) | fields `UNIX_TIMESTAMP()`, `UNIX_TIMESTAMP(20771122143845)`, `UNIX_TIMESTAMP(881226)`, `UNIX_TIMESTAMP(881226231058.123)`, `UNIX_TIMESTAMP(DATE('1995-05-06'))`, `UNIX_TIMESTAMP(TIMESTAMP('2022-08-09 12:16:12'))`, `UNIX_TIMESTAMP(ADDDATE(DATE('2020-08-26'), INTERVAL 1 HOUR))`;
UNIX_TIMESTAMP()                                             | 1662664900
UNIX_TIMESTAMP(20771122143845)                               | 3404817525.0
UNIX_TIMESTAMP(881226)                                       | 599097600.0
UNIX_TIMESTAMP(881226231058.123)                             | 599181057.1230469
UNIX_TIMESTAMP(DATE('1995-05-06'))                           | 799718400.0
UNIX_TIMESTAMP(TIMESTAMP('2022-08-09 12:16:12'))             | 1660047372.0
UNIX_TIMESTAMP(ADDDATE(DATE('2020-08-26'), INTERVAL 1 HOUR)) | 1598403600.0

Check List

  • New functionality includes testing.
    • All tests pass, including unit test, integration test and doctest
  • New functionality has been documented.
    • New functionality has javadoc added
    • New functionality has user manual doc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@Yury-Fridlyand Yury-Fridlyand requested a review from a team as a code owner September 15, 2022 23:32
@codecov-commenter
Copy link

codecov-commenter commented Sep 15, 2022

Codecov Report

Merging #835 (181b263) into 2.x (e28ac78) will increase coverage by 32.16%.
The diff coverage is 100.00%.

@@              Coverage Diff              @@
##                2.x     #835       +/-   ##
=============================================
+ Coverage     62.76%   94.93%   +32.16%     
- Complexity        0     2973     +2973     
=============================================
  Files            10      291      +281     
  Lines           658     7949     +7291     
  Branches        118      578      +460     
=============================================
+ Hits            413     7546     +7133     
- Misses          192      349      +157     
- Partials         53       54        +1     
Flag Coverage Δ
query-workbench 62.76% <ø> (ø)
sql-engine 97.83% <100.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...nsearch/data/value/OpenSearchExprValueFactory.java 100.00% <ø> (ø)
...a/org/opensearch/sql/data/model/ExprTimeValue.java 100.00% <100.00%> (ø)
.../opensearch/sql/data/model/ExprTimestampValue.java 100.00% <100.00%> (ø)
...arch/sql/expression/datetime/DateTimeFunction.java 100.00% <100.00%> (ø)
...h/sql/expression/function/BuiltinFunctionName.java 100.00% <100.00%> (ø)
...a/org/opensearch/sql/utils/DateTimeFormatters.java 100.00% <100.00%> (ø)
...opensearch/sql/ppl/utils/UnresolvedPlanHelper.java 100.00% <0.00%> (ø)
...h/sql/opensearch/data/type/OpenSearchDataType.java 100.00% <0.00%> (ø)
...ensearch/sql/exception/SemanticCheckException.java 100.00% <0.00%> (ø)
...opensearch/sql/expression/ReferenceExpression.java 100.00% <0.00%> (ø)
... and 277 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

dai-chen
dai-chen previously approved these changes Sep 20, 2022
Copy link
Collaborator

@dai-chen dai-chen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for the changes!

@Yury-Fridlyand
Copy link
Collaborator Author

Maybe I need to rename DateFormatters.java to DateTimeFormatters.java?

@dai-chen
Copy link
Collaborator

Maybe I need to rename DateFormatters.java to DateTimeFormatters.java?

Sure, we can do it now or later. Thanks!

dai-chen
dai-chen previously approved these changes Sep 21, 2022
* Add implementation for `FROM_UNIXTIME` and `UNIX_TIMESTAMP` functions, UT and IT.

Signed-off-by: Yury-Fridlyand <yuryf@bitquilltech.com>
Signed-off-by: Yury-Fridlyand <yuryf@bitquilltech.com>
Signed-off-by: Yury-Fridlyand <yuryf@bitquilltech.com>
@Yury-Fridlyand
Copy link
Collaborator Author

Please, re-approve re-review. I rebased the branch to fix merge conflicts.

@dai-chen dai-chen merged commit e04d6f8 into opensearch-project:2.x Sep 27, 2022
MitchellGale pushed a commit to Bit-Quill/opensearch-project-sql that referenced this pull request Oct 3, 2022
…ch-project#835)

* Add datetime functions `FROM_UNIXTIME` and `UNIX_TIMESTAMP` (#114)

* Add implementation for `FROM_UNIXTIME` and `UNIX_TIMESTAMP` functions, UT and IT.

Signed-off-by: Yury-Fridlyand <yuryf@bitquilltech.com>

* Collent all DateTime formatters into one place.

Signed-off-by: Yury-Fridlyand <yuryf@bitquilltech.com>

* Rename `DateFormatters` -> `DateTimeFormatters`.

Signed-off-by: Yury-Fridlyand <yuryf@bitquilltech.com>

Signed-off-by: Yury-Fridlyand <yuryf@bitquilltech.com>
@Yury-Fridlyand Yury-Fridlyand deleted the integ-datetime-unix-convert branch January 12, 2023 19:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request PPL Piped processing language SQL
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants