Skip to content

Conversation

@gatorsmile
Copy link
Member

What changes were proposed in this pull request?

Just document the impact of spark.sql.debug:

When enabling the debug, Spark SQL internal table properties are not filtered out; however, some related DDL commands (e.g., Analyze Table and CREATE TABLE LIKE) might not work properly.

How was this patch tested?

N/A

@rxin
Copy link
Contributor

rxin commented Oct 14, 2016

Why do they not work properly?

@SparkQA
Copy link

SparkQA commented Oct 15, 2016

Test build #66987 has finished for PR 15494 at commit a020f66.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@gatorsmile
Copy link
Member Author

gatorsmile commented Oct 15, 2016

@rxin. The failed test cases are accessible in the PR: #15478

ANALYZE TABLE will fail due to the checking when we try to alter the table properties.

SHOW CREATE TABLE also outputs the table properties that should not be part of the output CREAT TABLE statement.

CREATE TABLE LIKE always excludes all the table properties of the source table. However, we might make a change in this part. This is still waiting for your input in another PR. See the discussion

@cloud-fan
Copy link
Contributor

@rxin because we are assuming that all the hive hacks are isolated inside HiveExternalCatalog, but the debug mode will break it and spread the hive metastore specific table properties outside of HiveExternalCatalog. Then some commands like SHOW CREATE TABLE, ALTER TABLE will be broken, but should be ok as it's a "debug" mode anyway.

@cloud-fan
Copy link
Contributor

thanks, merging to master!

@asfgit asfgit closed this in e18d02c Oct 17, 2016
robert3005 pushed a commit to palantir/spark that referenced this pull request Nov 1, 2016
### What changes were proposed in this pull request?
Just document the impact of `spark.sql.debug`:

When enabling the debug, Spark SQL internal table properties are not filtered out; however, some related DDL commands (e.g., Analyze Table and CREATE TABLE LIKE) might not work properly.

### How was this patch tested?
N/A

Author: gatorsmile <gatorsmile@gmail.com>

Closes apache#15494 from gatorsmile/addDocForSQLDebug.
uzadude pushed a commit to uzadude/spark that referenced this pull request Jan 27, 2017
### What changes were proposed in this pull request?
Just document the impact of `spark.sql.debug`:

When enabling the debug, Spark SQL internal table properties are not filtered out; however, some related DDL commands (e.g., Analyze Table and CREATE TABLE LIKE) might not work properly.

### How was this patch tested?
N/A

Author: gatorsmile <gatorsmile@gmail.com>

Closes apache#15494 from gatorsmile/addDocForSQLDebug.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants