-
Notifications
You must be signed in to change notification settings - Fork 24.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SQL: Introduce SQL DATE data type #37693
Changes from 8 commits
95a0990
b410903
30f4fde
7409ce4
3be8945
7b9656f
5a2c05c
3ea4bdf
ffdfc3f
6bcc0af
89ed0ba
e7e880c
471187d
dc339f1
8dc2c4d
2341144
e4e6507
14d4986
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -5,9 +5,6 @@ | |
|
||
beta[] | ||
|
||
Most of {es} <<mapping-types, data types>> are available in {es-sql}, as indicated below. | ||
As one can see, all of {es} <<mapping-types, data types>> are mapped to the data type with the same | ||
name in {es-sql}, with the exception of **date** data type which is mapped to **datetime** in {es-sql}: | ||
|
||
[cols="^,^m,^,^"] | ||
|
||
|
@@ -46,13 +43,22 @@ s|SQL precision | |
|
||
|=== | ||
|
||
[NOTE] | ||
Most of {es} <<mapping-types, data types>> are available in {es-sql}, as indicated above. | ||
As one can see, all of {es} <<mapping-types, data types>> are mapped to the data type with the same | ||
name in {es-sql}, with the exception of **date** data type which is mapped to **datetime** in {es-sql}. | ||
This is to avoid confusion with the ANSI SQL **DATE** (date only) type, which is also supported by {es-sql} | ||
in queries (with the use of <<sql-functions-type-conversion-cast>>/<<sql-functions-type-conversion-convert>>), | ||
but doesn't correspond to an actual mapping in {es} (see the <<es-sql-extra-types, `table`>> below). | ||
|
||
Obviously, not all types in {es} have an equivalent in SQL and vice-versa hence why, {es-sql} | ||
uses the data type _particularities_ of the former over the latter as ultimately {es} is the backing store. | ||
|
||
In addition to the types above, {es-sql} also supports at _runtime_ SQL-specific types that do not have an equivalent in {es}. | ||
Such types cannot be loaded from {es} (as it does not know about them) however can be used inside {es-sql} in queries or their results. | ||
|
||
[[es-sql-extra-types]] | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Please rename this to |
||
|
||
The table below indicates these types: | ||
|
||
[cols="^m,^"] | ||
|
@@ -62,6 +68,7 @@ s|SQL type | |
s|SQL precision | ||
|
||
|
||
| date | 24 | ||
| interval_year | 7 | ||
| interval_month | 7 | ||
| interval_day | 23 | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,90 @@ | ||
// | ||
// Date | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 👍 |
||
// | ||
|
||
dateExtractDateParts | ||
SELECT | ||
DAY(CAST(birth_date AS DATE)) d, | ||
DAY_OF_MONTH(CAST(birth_date AS DATE)) dm, | ||
DAY_OF_WEEK(CAST(birth_date AS DATE)) dw, | ||
DAY_OF_YEAR(CAST(birth_date AS DATE)) dy, | ||
ISO_DAY_OF_WEEK(CAST(birth_date AS DATE)) iso_dw, | ||
WEEK(CAST(birth_date AS DATE)) w, | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You could, also, add the ISO_WEEK_OF_YEAR function, since you added the iso_dow one. |
||
IW(CAST(birth_date AS DATE)) iso_w, | ||
QUARTER(CAST(birth_date AS DATE)) q, | ||
YEAR(CAST(birth_date AS DATE)) y, | ||
birth_date, last_name l FROM "test_emp" WHERE emp_no < 10010 ORDER BY emp_no; | ||
|
||
d:i | dm:i | dw:i | dy:i | iso_dw:i | w:i |iso_w:i | q:i | y:i | birth_date:ts | l:s | ||
2 |2 |4 |245 |3 |36 |35 |3 |1953 |1953-09-02T00:00:00Z |Facello | ||
2 |2 |3 |154 |2 |23 |22 |2 |1964 |1964-06-02T00:00:00Z |Simmel | ||
3 |3 |5 |337 |4 |49 |49 |4 |1959 |1959-12-03T00:00:00Z |Bamford | ||
1 |1 |7 |121 |6 |18 |18 |2 |1954 |1954-05-01T00:00:00Z |Koblick | ||
21 |21 |6 |21 |5 |4 |3 |1 |1955 |1955-01-21T00:00:00Z |Maliniak | ||
20 |20 |2 |110 |1 |17 |16 |2 |1953 |1953-04-20T00:00:00Z |Preusig | ||
23 |23 |5 |143 |4 |21 |21 |2 |1957 |1957-05-23T00:00:00Z |Zielinski | ||
19 |19 |4 |50 |3 |8 |8 |1 |1958 |1958-02-19T00:00:00Z |Kalloufi | ||
19 |19 |7 |110 |6 |16 |16 |2 |1952 |1952-04-19T00:00:00Z |Peac | ||
; | ||
|
||
|
||
dateExtractTimePartsTimeSecond | ||
SELECT | ||
SECOND(CAST(birth_date AS DATE)) d, | ||
MINUTE(CAST(birth_date AS DATE)) m, | ||
HOUR(CAST(birth_date AS DATE)) h | ||
FROM "test_emp" WHERE emp_no < 10010 ORDER BY emp_no; | ||
|
||
d:i | m:i | h:i | ||
0 |0 |0 | ||
0 |0 |0 | ||
0 |0 |0 | ||
0 |0 |0 | ||
0 |0 |0 | ||
0 |0 |0 | ||
0 |0 |0 | ||
0 |0 |0 | ||
0 |0 |0 | ||
; | ||
|
||
dateAsFilter | ||
SELECT birth_date, last_name FROM "test_emp" WHERE birth_date <= CAST('1955-01-21' AS DATE) ORDER BY emp_no LIMIT 5; | ||
|
||
birth_date:ts | last_name:s | ||
1953-09-02T00:00:00Z |Facello | ||
1954-05-01T00:00:00Z |Koblick | ||
1955-01-21T00:00:00Z |Maliniak | ||
1953-04-20T00:00:00Z |Preusig | ||
1952-04-19T00:00:00Z |Peac | ||
; | ||
|
||
|
||
// The date return can differ due to the randomized timezones (it's transformed to the JDBC client's timezone) | ||
dateAsGroupingKey-Ignore | ||
SELECT CAST(birth_date AS DATE) AS d, CAST(SUM(emp_no) AS INT) s FROM test_emp GROUP BY d ORDER BY d LIMIT 5; | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. why not use a function than to extract just the month or the year? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is here already: https://github.com/elastic/elasticsearch/pull/37693/files/b4109031e770ac52153184b32baf921c112e8acd#diff-51f788d55d8ade70340fdf2ca8af1f20R74 I used this query to discover this issue: #37693 (comment) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Then maybe use the year. I don't understand why we would add a test which is disabled since the start. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. No, there is not point, I will remove. |
||
|
||
d:date | s:i | ||
null |100445 | ||
1952-02-26 |10097 | ||
1952-04-18 |10009 | ||
1952-05-14 |10072 | ||
1952-06-12 |10076 | ||
; | ||
|
||
dateAndFunctionAsGroupingKey | ||
SELECT MONTH(CAST(birth_date AS DATE)) AS m, CAST(SUM(emp_no) AS INT) s FROM test_emp GROUP BY m ORDER BY m LIMIT 5; | ||
|
||
m:i | s:i | ||
null |100445 | ||
1 |60288 | ||
2 |80388 | ||
3 |20164 | ||
4 |80401 | ||
; | ||
|
||
dateAndInterval | ||
SELECT YEAR(CAST('2019-01-21' AS DATE) + INTERVAL '1-2' YEAR TO MONTH) AS y, MONTH(INTERVAL '1-2' YEAR TO MONTH + CAST('2019-01-21' AS DATE)) AS m; | ||
|
||
y:i | m:i | ||
2020 | 3 | ||
; |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -6,8 +6,9 @@ | |
// Time NOT IMPLEMENTED in H2 on TIMESTAMP WITH TIME ZONE - hence why these are moved to CSV | ||
// | ||
|
||
// WEEK_OF_YEAR moved to CSV tests, because H2 builds its Calendar with the local Locale, we consider ROOT as the default Locale | ||
// This has implications on the results, which could change given specific locales where the rules for determining the start of a year are different. | ||
// WEEK_OF_YEAR moved to CSV tests, because H2 builds its Calendar with the local Locale, | ||
// we consider ROOT as the default Locale. This has implications on the results, which could | ||
// change given specific locales where the rules for determining the start of a year are different. | ||
|
||
// | ||
// DateTime | ||
|
@@ -31,10 +32,10 @@ SELECT MONTHNAME(CAST('2018-09-03' AS TIMESTAMP)) month FROM "test_emp" limit 1; | |
dayNameFromStringDateTime | ||
SELECT DAYNAME(CAST('2018-09-03' AS TIMESTAMP)) day FROM "test_emp" limit 1; | ||
|
||
quarterSelect | ||
dateTimeQuarter | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. the file name is already used as a prefix and adding dateTime in the test name becomes redundant: There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yep, but currently I just tried to make ti consistent with all the others: https://github.com/elastic/elasticsearch/pull/37693/files/95a0990cfcc48c1aee6fdb0ab1ff9cb569aafd3a#diff-6079120a3b84ce0d0410436f58ec8a5bR17 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I see - the others are inconsistent though :) |
||
SELECT QUARTER(hire_date) q, hire_date FROM test_emp ORDER BY hire_date LIMIT 15; | ||
|
||
dayOfWeek | ||
dateTimeDayOfWeek | ||
SELECT DAY_OF_WEEK(birth_date) day, birth_date FROM test_emp ORDER BY DAY_OF_WEEK(birth_date); | ||
|
||
// | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -95,7 +95,9 @@ public Object extract(Bucket bucket) { | |
if (object == null) { | ||
return object; | ||
} else if (object instanceof Long) { | ||
object = DateUtils.of(((Long) object).longValue(), zoneId); | ||
object = DateUtils.asDateTime(((Long) object).longValue(), zoneId); | ||
} else if (object instanceof String) { // CAST(<value> AS DATE) is used | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This looks suspicious - why would the key be a string and that would represent only a date and not a datetime? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Maybe add a more detailed comment for the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The painless script is unnecessary and complicated namely because a histogram with a granularity of 1 day is going to ignore the time component anyway. We can and should identify this pattern I'm fine with addressing this in a different PR. |
||
object = DateUtils.asDateOnly(object.toString()); | ||
} else { | ||
throw new SqlIllegalArgumentException("Invalid date key returned: {}", object); | ||
} | ||
|
@@ -129,4 +131,4 @@ public boolean equals(Object obj) { | |
public String toString() { | ||
return "|" + key + "|"; | ||
} | ||
} | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -74,4 +74,4 @@ public boolean equals(Object obj) { | |
public int hashCode() { | ||
return Objects.hash(field(), zoneId()); | ||
} | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would make this
IMPORTANT
.Also drop the "then" : `when... DATETIME, the interval".