Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for create table as via MemTable #1243

Merged
merged 8 commits into from
Nov 6, 2021

Conversation

Dandandan
Copy link
Contributor

@Dandandan Dandandan commented Nov 4, 2021

Which issue does this PR close?

Closes #1213

Rationale for this change

Being able to use syntax: CREATE TABLE AS SELECT ...

What changes are included in this PR?

Are there any user-facing changes?

@Dandandan Dandandan changed the title WIP Add support for create table as via MemTable WIP Add support for create table as via MemTable Nov 4, 2021
@github-actions github-actions bot added ballista datafusion Changes in the datafusion crate sql SQL Planner labels Nov 4, 2021
@houqp houqp added the enhancement New feature or request label Nov 5, 2021
@Dandandan Dandandan requested review from houqp and alamb November 5, 2021 07:52
@Dandandan Dandandan changed the title WIP Add support for create table as via MemTable Add support for create table as via MemTable Nov 5, 2021
@Dandandan Dandandan marked this pull request as ready for review November 5, 2021 07:52
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it looks great -- thank you @Dandandan !

> create table foo as select * from (values (1,2), (3,4)) as sq;
0 rows in set. Query took 0.007 seconds.
> select * from foo;
+---------+---------+
| column1 | column2 |
+---------+---------+
| 1       | 2       |
| 3       | 4       |
+---------+---------+

@@ -237,6 +240,21 @@ impl ExecutionContext {
Ok(Arc::new(DataFrameImpl::new(self.state.clone(), &plan)))
}

LogicalPlan::CreateMemoryTable { input, name } => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is pretty cool

@@ -791,6 +791,13 @@ impl DefaultPhysicalPlanner {
"Unsupported logical plan: CreateExternalTable".to_string(),
))
}
| LogicalPlan::CreateMemoryTable {..} => {
// Create a dummy exec.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it might make more sense here to throw an error about "unsupported logical plan" -- if we return an EmptyExec that will result in a plan that does nothing (aka doesn't actually create any tables / table providers) but doesn't error which might be surprising

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried that before, and it failed I think somehow at the planning stage?

let mut ctx = ExecutionContext::new();
register_aggregate_simple_csv(&mut ctx).await?;

let sql = "CREATE TABLE my_table AS SELECT * FROM aggregate_simple";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So cool ❤️

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one thing you can try is to create table as select from values lists

@alamb
Copy link
Contributor

alamb commented Nov 5, 2021

FYI I pushed 68b6cef to fix a clippy error on your branch

@Dandandan
Copy link
Contributor Author

FYI I pushed 68b6cef to fix a clippy error on your branch

Thanks!

@jimexist
Copy link
Member

jimexist commented Nov 6, 2021

thanks @Dandandan this is very cool!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
datafusion Changes in the datafusion crate enhancement New feature or request sql SQL Planner
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support CREATE TABLE AS SELECT ... in datafusion-cli
4 participants