feat(api): make create_table uniform #5736
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cpcloud
added
feature
cpcloud
force-pushed
the
create-table-uniformity
branch
6 times, most recently
from
a07a119
to
969e26b
Compare
|
Thoughts on forcing keyword arguments for a bunch of these DDL functions? |
|
SGTM |
gforsyth
force-pushed
the
create-table-uniformity
branch
2 times, most recently
from
00f2e31
to
d4e9e9b
Compare
cpcloud
force-pushed
the
create-table-uniformity
branch
from
21ce5af
to
cc54ea2
Compare
cpcloud
force-pushed
the
create-table-uniformity
branch
from
cc54ea2
to
0280bef
Compare
gforsyth
approved these changes
Mar 14, 2023
|
There are some snowflake things that are failing locally for me. #5741 should help with that |
jcrist
approved these changes
Mar 14, 2023
gforsyth
force-pushed
the
create-table-uniformity
branch
3 times, most recently
from
a20b677
to
455c1a5
Compare
cpcloud
force-pushed
the
create-table-uniformity
branch
from
455c1a5
to
43099cb
Compare
gforsyth
force-pushed
the
create-table-uniformity
branch
from
43099cb
to
46c1330
Compare
cpcloud
force-pushed
the
create-table-uniformity
branch
2 times, most recently
from
bc60aee
to
b6f5df7
Compare
|
Cloud backends: |
cpcloud
force-pushed
the
create-table-uniformity
branch
from
571b466
to
ada0f20
Compare
|
I am authoring a longer commit message, hold off on merging please! |
cpcloud
force-pushed
the
create-table-uniformity
branch
from
ada0f20
to
c433691
Compare
This commit addresses most outstanding DDL API discrepancy issues including: - `create_table`/`create_view` for pandas, dask and polars - making the various DDL APIs as uniform as possible (see clickhouse for an example of divergence) - deprecation of `load_data` (except on impala, since it's significantly different from the others) - add clickhouse implementations of `create_table`/`create_view`/`create_database` - standardization of APIs for creating tables During the process of getting all of this to work, I uncovered multiple issues with `snowflake-sqlalchemy`'s quoting behavior and had to monkey patch in `normalize_name` to avoid the broken heuristic they are using. Additionally, to avoid having to solve the "which case should I use?" problem in multiple places, I decided to remove our backend-scoped use of `sqlalchemy.MetaData`. Without removing it, we'd have to deal with identifiers' case not matching. It's possible there's a performance hit, but removing this maintenance burden until someone comes along saying it's slow is worth it IMO. BREAKING CHANGE: Snowflake identifiers are now kept **as is** from the database. Many table names and column names may now be in SHOUTING CASE. Adjust code accordingly.
cpcloud
force-pushed
the
create-table-uniformity
branch
from
c433691
to
dbfe647
Compare
|
Ok, merging on green! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR address recent issues created around DDL.
Notes
ImpalaTable.load_data. I'm not entirely what should be done with it.create_table, since clickhouse itself doesn't have a defaulttable engine, I preserved that: you have to pass
engine="Something"to itscreate_table. It breaks the uniformity, but I'm not sure how else to handle
it.
TODO
create_viewfor pandas/dask/polarscreate_view/drop_table/truncate_tablefor clickhousetruncate_table(I don't think these makes sense for the in-memory backends, so those'll raise an exception)Closes #5394
Closes #5392
Closes #5390
Closes #4316
Closes #2339