Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SQL: Rename SQL type DATE to DATETIME #37395

Merged
merged 6 commits into from
Jan 17, 2019
Merged

Conversation

matriv
Copy link
Contributor

@matriv matriv commented Jan 13, 2019

SQL data type DATE has only the date part (e.g.: 2019-01-14)
without any time information. Previously the SQL type DATE was
referring to the ES DATE which contains also the time part along
with TZ information. To conform with SQL data types the data type
DATE is renamed to DATETIME, since it includes also the time,
as a new runtime SQL DATE data type will be introduced down the road,
which only contains the date part and meets the SQL standard.

Closes: #36440

@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search

@matriv matriv changed the title WIP SQL: Rename SQL type DATE to DATETIME Jan 14, 2019
SQL data type DATE has only the date part (e.g.: 2019-01-14)
without any time information. Previously the SQL type DATE was
referring to the ES DATE which contains also the time part along
with TZ information. To conform with SQL data types the data type
`DATE` is renamed to `DATETIME`, since it includes also the time,
as a new runtime SQL `DATE` data type will be introduced down the road,
which only contains the date part and meets the SQL standard.

Closes: elastic#36440
@matriv matriv requested review from costin and astefan January 14, 2019 18:36
@matriv matriv removed the WIP label Jan 14, 2019
Copy link
Member

@costin costin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've requested some small changes however they should be fairly obvious; once their fixed I'm fine with merging the PR.

} catch (IllegalArgumentException ex) {
if (uppercase.equals("DATE")) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please move the if before the try - that is first check if it's a date, otherwise just do the typical DataType conversion.

| <<keyword, `keyword`>> | varchar | based on <<ignore-above>>
| <<text, `text`>> | varchar | 2,147,483,647
| <<binary, `binary`>> | varbinary | 2,147,483,647
| <<date, `date`>> | timestamp/datetime | 24
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it makes sense to add another column so the table contains ES/ES-SQL/SQL types. It might be redundant but it might make the datetime case clearer - can be done in a separate PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Opened issue to track it: #37519

error("SELECT AVG(date) FROM test"));
}

public void testNotSupportedAggregateOnString() {
assertEquals("1:8: [MAX(keyword)] argument must be [numeric or date], found value [keyword] type [keyword]",
assertEquals("1:8: [MAX(keyword)] argument must be [NUMERIC or DATETIME], found value [keyword] type [KEYWORD]",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's with the uppercasing of the types? It is used as lowercase since that's the convention used by ES itself.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, My bad. I was trying to make it consistent with those:
https://github.com/elastic/elasticsearch/blob/master/x-pack/plugin/sql/src/test/java/org/elasticsearch/xpack/sql/analysis/analyzer/VerifierErrorMessagesTests.java#L336
but I guess we should change those to lowercase instead. Will revert.

@@ -22,9 +22,9 @@ The table below shows the mapping between {es} and {es-sql}:
|==========================
s|{es}
s|{es-sql}
2+h| Index/Table date math
2+h| Index/Table date/time math
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would use datetime in one word - / suggest one can pick either date or time and the latter it's not actually available.

@@ -190,6 +190,6 @@ public static TypeResolution typeMustBe(Expression e,
paramOrd == null || paramOrd == ParamOrdinal.DEFAULT ? "" : " " + paramOrd.name().toLowerCase(Locale.ROOT),
Strings.arrayToDelimitedString(acceptedTypes, " or "),
Expressions.name(e),
e.dataType().esType));
e.dataType().esType.toUpperCase(Locale.ROOT)));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should keep the lower casing in place since that's how the types are defined in ES as well (keyword vs KEYWORD).
If anything, we should change for example the column descriptors to always return the ES types in lowercase to make the context (SQL type vs ES type) even clearer.

@@ -28,13 +28,13 @@ DESCRIBE test_alias;

column | type | mapping
--------------------+---------------+---------------
birth_date |TIMESTAMP |DATE
birth_date |TIMESTAMP |DATETIME
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally we should change mapping to be lower-case to follow the ES convention - this can be done in a follow-up PR.

birth_date | TIMESTAMP | datetime
dep        | STRUCT    | nested

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tracked here: #37521

Copy link
Contributor

@astefan astefan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@matriv
Copy link
Contributor Author

matriv commented Jan 16, 2019

run the gradle build tests 1

@matriv
Copy link
Contributor Author

matriv commented Jan 16, 2019

run the gradle build tests 2

1 similar comment
@matriv
Copy link
Contributor Author

matriv commented Jan 16, 2019

run the gradle build tests 2

@matriv
Copy link
Contributor Author

matriv commented Jan 16, 2019

run the gradle build tests 1

@costin
Copy link
Member

costin commented Jan 16, 2019

run the gradle build tests 2

@matriv
Copy link
Contributor Author

matriv commented Jan 16, 2019

run the gradle build tests 2

@matriv
Copy link
Contributor Author

matriv commented Jan 16, 2019

run the gradle build tests 1

@matriv
Copy link
Contributor Author

matriv commented Jan 16, 2019

run the gradle build tests 1

1 similar comment
@matriv
Copy link
Contributor Author

matriv commented Jan 16, 2019

run the gradle build tests 1

@matriv
Copy link
Contributor Author

matriv commented Jan 16, 2019

run the gradle build tests 2

@matriv
Copy link
Contributor Author

matriv commented Jan 16, 2019

run the gradle build tests 2

@matriv
Copy link
Contributor Author

matriv commented Jan 16, 2019

run the gradle build tests 1

2 similar comments
@matriv
Copy link
Contributor Author

matriv commented Jan 16, 2019

run the gradle build tests 1

@matriv
Copy link
Contributor Author

matriv commented Jan 16, 2019

run the gradle build tests 1

@matriv
Copy link
Contributor Author

matriv commented Jan 16, 2019

run the gradle build tests 2

@costin
Copy link
Member

costin commented Jan 16, 2019

run the gradle build tests 1

@matriv
Copy link
Contributor Author

matriv commented Jan 16, 2019

run the gradle build tests 2

1 similar comment
@matriv
Copy link
Contributor Author

matriv commented Jan 16, 2019

run the gradle build tests 2

@matriv matriv merged commit 1686c32 into elastic:master Jan 17, 2019
@matriv matriv deleted the mt/fix-36440 branch January 17, 2019 08:18
matriv added a commit that referenced this pull request Jan 17, 2019
* SQL: Rename SQL data type DATE to DATETIME

SQL data type DATE has only the date part (e.g.: 2019-01-14)
without any time information. Previously the SQL type DATE was
referring to the ES DATE which contains also the time part along
with TZ information. To conform with SQL data types the data type
`DATE` is renamed to `DATETIME`, since it includes also the time,
as a new runtime SQL `DATE` data type will be introduced down the road,
which only contains the date part and meets the SQL standard.

Closes: #36440

* Address comments
@matriv
Copy link
Contributor Author

matriv commented Jan 17, 2019

Backported to 6.x with 2a13652

jasontedor added a commit to jasontedor/elasticsearch that referenced this pull request Jan 17, 2019
…-response-header-performance

* elastic/master:
  Remove Redundant RestoreRequest Class (elastic#37535)
  Create specific exception for when snapshots are in progress (elastic#37550)
  Mute UnicastZenPingTests#testSimplePings
  [DOCS] Adds size limitation to the get datafeeds APIs (elastic#37578)
  Fix assertion at end of forceRefreshes (elastic#37559)
  Propagate Errors in executors to uncaught exception handler (elastic#36137)
  [DOCS] Adds limitation to the get jobs API (elastic#37549)
  Add set_priority action to ILM (elastic#37397)
  Make recovery source send operations non-blocking (elastic#37503)
  Allow field types to optimize phrase prefix queries (elastic#37436)
  Added fatal_exception field for ccr stats in monitoring mapping. (elastic#37563)
  Fix testRelocateWhileContinuouslyIndexingAndWaitingForRefresh (elastic#37560)
  Moved ccr integration to the package with other ccr integration tests.
  Mute TransportClientNodesServiceTests#testListenerFailures
  Decreased time out in test
  Fix erroneous docstrings for abstract bulk by scroll request (elastic#37517)
  SQL: Rename SQL type DATE to DATETIME (elastic#37395)
  Remove the AbstracLifecycleComponent constructor with Settings (elastic#37523)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants