Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"VERSION / TIMESTAMP AS OF" unsupported in SQL? #772

Closed
AFFogarty opened this issue Sep 1, 2021 · 2 comments
Closed

"VERSION / TIMESTAMP AS OF" unsupported in SQL? #772

AFFogarty opened this issue Sep 1, 2021 · 2 comments

Comments

@AFFogarty
Copy link
Contributor

AFFogarty commented Sep 1, 2021

Hello, similar to #702 , I think there might be an issue with VERSION AS OF and TIMESTAMP AS OF in SQL.

The docs show examples of VERSION AS OF and TIMESTAMP AS OF here: https://docs.delta.io/1.0.0/delta-batch.html#syntax

However, I'm getting SQL parsing errors when trying to run these examples:

scala> spark.sql("SELECT * FROM events TIMESTAMP AS OF '2018-10-18T22:15:12.013Z'").show

org.apache.spark.sql.catalyst.parser.ParseException:
mismatched input 'AS' expecting {<EOF>, ';'}(line 1, pos 31)

== SQL ==
SELECT * FROM events TIMESTAMP AS OF '2018-10-18T22:15:12.013Z'
-------------------------------^^^

  at org.apache.spark.sql.catalyst.parser.ParseException.withCommand(ParseDriver.scala:255)
  at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parse(ParseDriver.scala:124)
  at org.apache.spark.sql.execution.SparkSqlParser.parse(SparkSqlParser.scala:49)
  at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parsePlan(ParseDriver.scala:75)
  at io.delta.sql.parser.DeltaSqlParser.$anonfun$parsePlan$1(DeltaSqlParser.scala:73)
  at io.delta.sql.parser.DeltaSqlParser.parse(DeltaSqlParser.scala:100)
  at io.delta.sql.parser.DeltaSqlParser.parsePlan(DeltaSqlParser.scala:70)
  at org.apache.spark.sql.SparkSession.$anonfun$sql$2(SparkSession.scala:616)
  at org.apache.spark.sql.catalyst.QueryPlanningTracker.measurePhase(QueryPlanningTracker.scala:111)
  at org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:616)
  at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:775)
  at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:613)
  ... 47 elided
scala> spark.sql("SELECT * FROM events VERSION AS OF 123").show

org.apache.spark.sql.catalyst.parser.ParseException:
mismatched input 'AS' expecting {<EOF>, ';'}(line 1, pos 29)

== SQL ==
SELECT * FROM events VERSION AS OF 123
-----------------------------^^^

  at org.apache.spark.sql.catalyst.parser.ParseException.withCommand(ParseDriver.scala:255)
  at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parse(ParseDriver.scala:124)
  at org.apache.spark.sql.execution.SparkSqlParser.parse(SparkSqlParser.scala:49)
  at org.apache.spark.sql.catalyst.parser.AbstractSqlParser.parsePlan(ParseDriver.scala:75)
  at io.delta.sql.parser.DeltaSqlParser.$anonfun$parsePlan$1(DeltaSqlParser.scala:73)
  at io.delta.sql.parser.DeltaSqlParser.parse(DeltaSqlParser.scala:100)
  at io.delta.sql.parser.DeltaSqlParser.parsePlan(DeltaSqlParser.scala:70)
  at org.apache.spark.sql.SparkSession.$anonfun$sql$2(SparkSession.scala:616)
  at org.apache.spark.sql.catalyst.QueryPlanningTracker.measurePhase(QueryPlanningTracker.scala:111)
  at org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:616)
  at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:775)
  at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:613)
  ... 47 elided

I searched through the unit tests and it seems that the tests don't cover the SQL scenario for these commands.

@AFFogarty AFFogarty changed the title "VERSION / TIMESTAMP AS OF" unsupported "VERSION / TIMESTAMP AS OF" unsupported? Sep 1, 2021
@AFFogarty AFFogarty changed the title "VERSION / TIMESTAMP AS OF" unsupported? "VERSION / TIMESTAMP AS OF" unsupported in SQL? Sep 1, 2021
@zsxwing
Copy link
Member

zsxwing commented Sep 1, 2021

@AFFogarty this is the same as #128. Closing the duplicate issue.

@zsxwing zsxwing closed this as completed Sep 1, 2021
@AFFogarty
Copy link
Contributor Author

AFFogarty commented Sep 1, 2021

Sorry for the duplicate, @zsxwing. I searched earlier but didn't find that issue.

Since this SQL syntax currently doesn't work in 1.0.0, can we remove the examples from the public docs for now? https://docs.delta.io/1.0.0/delta-batch.html#syntax

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants