Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Doc: Add a page explaining migration from other table formats to iceberg #6600

Merged
merged 22 commits into from
Apr 22, 2023

Conversation

JonasJ-ap
Copy link
Contributor

@JonasJ-ap JonasJ-ap commented Jan 16, 2023

Add documentation for the iceberg-delta-lake module created in #6449 and pointers to Hive/Spark table migration procedures specified in Spark Doc.

Closes #6770

@jackye1995
Copy link
Contributor

jackye1995 commented Mar 22, 2023

I think this can fix #6770 , can we update this page to a dedicated page for all migration use cases and also include Hive to Iceberg?

@JonasJ-ap JonasJ-ap changed the title Doc: Add descriptions and examples for Snapshot Delta Lake Table to Iceberg Table Doc: Add a page explaining migration from other table formats to iceberg Mar 23, 2023
docs/table-migration.md Outdated Show resolved Hide resolved
docs/table-migration.md Outdated Show resolved Hide resolved
docs/table-migration.md Outdated Show resolved Hide resolved
docs/table-migration.md Outdated Show resolved Hide resolved
docs/table-migration.md Outdated Show resolved Hide resolved
docs/table-migration.md Outdated Show resolved Hide resolved
docs/table-migration.md Outdated Show resolved Hide resolved
docs/table-migration.md Outdated Show resolved Hide resolved
docs/table-migration.md Outdated Show resolved Hide resolved
docs/table-migration.md Outdated Show resolved Hide resolved
docs/table-migration.md Outdated Show resolved Hide resolved
docs/table-migration.md Outdated Show resolved Hide resolved
Copy link
Contributor Author

@JonasJ-ap JonasJ-ap left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jackye1995 Thanks for the review. I updated the pages and made up two html preview here:

https://drive.google.com/drive/folders/1XNZ1s2rXNSUxpP_ccUBVockV5KZ3gMOG?usp=share_link
(Since github does not allow me to upload html files directly)

docs/table-migration.md Outdated Show resolved Hide resolved
docs/hive-migration.md Outdated Show resolved Hide resolved
docs/table-migration.md Outdated Show resolved Hide resolved
Signed-off-by: Rushan Jiang <rushanj@andrew.cmu.edu>
Signed-off-by: Rushan Jiang <rushanj@andrew.cmu.edu>
Signed-off-by: Rushan Jiang <rushanj@andrew.cmu.edu>
@jackye1995
Copy link
Contributor

Thanks for keeping the work going on! I think we are very close, just 3 nit comments and we should be good to go! Let me know when you have another look

Signed-off-by: Rushan Jiang <rushanj@andrew.cmu.edu>
Signed-off-by: Rushan Jiang <rushanj@andrew.cmu.edu>
Signed-off-by: Rushan Jiang <rushanj@andrew.cmu.edu>
Signed-off-by: Rushan Jiang <rushanj@andrew.cmu.edu>
Copy link
Contributor

@jackye1995 jackye1995 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me!

@jackye1995
Copy link
Contributor

Thanks for all the work Jonas, merging

@jackye1995 jackye1995 merged commit 1531e59 into apache:master Apr 22, 2023
Copy link
Member

@RussellSpitzer RussellSpitzer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo

```
See [Spark Procedure: migrate](../spark-procedures/#migrate) for more details.

## Add Files From Delta Lake Table to Iceberg
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should say hive

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops. Thank you for catching this. Sorry for the typo.

Have opened a new PR here: #7407, to address that.

@RussellSpitzer
Copy link
Member

Thanks!

manisin pushed a commit to Snowflake-Labs/iceberg that referenced this pull request May 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add dedicated documentation page for table migrations
3 participants