-
Notifications
You must be signed in to change notification settings - Fork 24.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SQL: Rename SQL type DATE to DATETIME #37395
Conversation
Pinging @elastic/es-search |
SQL data type DATE has only the date part (e.g.: 2019-01-14) without any time information. Previously the SQL type DATE was referring to the ES DATE which contains also the time part along with TZ information. To conform with SQL data types the data type `DATE` is renamed to `DATETIME`, since it includes also the time, as a new runtime SQL `DATE` data type will be introduced down the road, which only contains the date part and meets the SQL standard. Closes: elastic#36440
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've requested some small changes however they should be fairly obvious; once their fixed I'm fine with merging the PR.
} catch (IllegalArgumentException ex) { | ||
if (uppercase.equals("DATE")) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please move the if before the try - that is first check if it's a date, otherwise just do the typical DataType conversion.
| <<keyword, `keyword`>> | varchar | based on <<ignore-above>> | ||
| <<text, `text`>> | varchar | 2,147,483,647 | ||
| <<binary, `binary`>> | varbinary | 2,147,483,647 | ||
| <<date, `date`>> | timestamp/datetime | 24 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe it makes sense to add another column so the table contains ES/ES-SQL/SQL types. It might be redundant but it might make the datetime
case clearer - can be done in a separate PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Opened issue to track it: #37519
error("SELECT AVG(date) FROM test")); | ||
} | ||
|
||
public void testNotSupportedAggregateOnString() { | ||
assertEquals("1:8: [MAX(keyword)] argument must be [numeric or date], found value [keyword] type [keyword]", | ||
assertEquals("1:8: [MAX(keyword)] argument must be [NUMERIC or DATETIME], found value [keyword] type [KEYWORD]", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's with the uppercasing of the types? It is used as lowercase since that's the convention used by ES itself.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hm, My bad. I was trying to make it consistent with those:
https://github.com/elastic/elasticsearch/blob/master/x-pack/plugin/sql/src/test/java/org/elasticsearch/xpack/sql/analysis/analyzer/VerifierErrorMessagesTests.java#L336
but I guess we should change those to lowercase instead. Will revert.
@@ -22,9 +22,9 @@ The table below shows the mapping between {es} and {es-sql}: | |||
|========================== | |||
s|{es} | |||
s|{es-sql} | |||
2+h| Index/Table date math | |||
2+h| Index/Table date/time math |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would use datetime
in one word - /
suggest one can pick either date
or time
and the latter it's not actually available.
@@ -190,6 +190,6 @@ public static TypeResolution typeMustBe(Expression e, | |||
paramOrd == null || paramOrd == ParamOrdinal.DEFAULT ? "" : " " + paramOrd.name().toLowerCase(Locale.ROOT), | |||
Strings.arrayToDelimitedString(acceptedTypes, " or "), | |||
Expressions.name(e), | |||
e.dataType().esType)); | |||
e.dataType().esType.toUpperCase(Locale.ROOT))); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should keep the lower casing in place since that's how the types are defined in ES as well (keyword
vs KEYWORD
).
If anything, we should change for example the column descriptors to always return the ES types in lowercase to make the context (SQL type vs ES type) even clearer.
@@ -28,13 +28,13 @@ DESCRIBE test_alias; | |||
|
|||
column | type | mapping | |||
--------------------+---------------+--------------- | |||
birth_date |TIMESTAMP |DATE | |||
birth_date |TIMESTAMP |DATETIME |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ideally we should change mapping
to be lower-case to follow the ES convention - this can be done in a follow-up PR.
birth_date | TIMESTAMP | datetime
dep | STRUCT | nested
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tracked here: #37521
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
run the gradle build tests 1 |
run the gradle build tests 2 |
1 similar comment
run the gradle build tests 2 |
run the gradle build tests 1 |
run the gradle build tests 2 |
run the gradle build tests 2 |
run the gradle build tests 1 |
run the gradle build tests 1 |
1 similar comment
run the gradle build tests 1 |
run the gradle build tests 2 |
run the gradle build tests 2 |
run the gradle build tests 1 |
2 similar comments
run the gradle build tests 1 |
run the gradle build tests 1 |
run the gradle build tests 2 |
run the gradle build tests 1 |
run the gradle build tests 2 |
1 similar comment
run the gradle build tests 2 |
* SQL: Rename SQL data type DATE to DATETIME SQL data type DATE has only the date part (e.g.: 2019-01-14) without any time information. Previously the SQL type DATE was referring to the ES DATE which contains also the time part along with TZ information. To conform with SQL data types the data type `DATE` is renamed to `DATETIME`, since it includes also the time, as a new runtime SQL `DATE` data type will be introduced down the road, which only contains the date part and meets the SQL standard. Closes: #36440 * Address comments
Backported to |
…-response-header-performance * elastic/master: Remove Redundant RestoreRequest Class (elastic#37535) Create specific exception for when snapshots are in progress (elastic#37550) Mute UnicastZenPingTests#testSimplePings [DOCS] Adds size limitation to the get datafeeds APIs (elastic#37578) Fix assertion at end of forceRefreshes (elastic#37559) Propagate Errors in executors to uncaught exception handler (elastic#36137) [DOCS] Adds limitation to the get jobs API (elastic#37549) Add set_priority action to ILM (elastic#37397) Make recovery source send operations non-blocking (elastic#37503) Allow field types to optimize phrase prefix queries (elastic#37436) Added fatal_exception field for ccr stats in monitoring mapping. (elastic#37563) Fix testRelocateWhileContinuouslyIndexingAndWaitingForRefresh (elastic#37560) Moved ccr integration to the package with other ccr integration tests. Mute TransportClientNodesServiceTests#testListenerFailures Decreased time out in test Fix erroneous docstrings for abstract bulk by scroll request (elastic#37517) SQL: Rename SQL type DATE to DATETIME (elastic#37395) Remove the AbstracLifecycleComponent constructor with Settings (elastic#37523)
SQL data type DATE has only the date part (e.g.: 2019-01-14)
without any time information. Previously the SQL type DATE was
referring to the ES DATE which contains also the time part along
with TZ information. To conform with SQL data types the data type
DATE
is renamed toDATETIME
, since it includes also the time,as a new runtime SQL
DATE
data type will be introduced down the road,which only contains the date part and meets the SQL standard.
Closes: #36440