Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Quickstart now uses DuckDB WASM instead of CLI #6092

Merged
merged 9 commits into from
Jun 26, 2023
Merged

Conversation

rmoff
Copy link
Contributor

@rmoff rmoff commented Jun 15, 2023

  • Update Quickstart to use DuckDB WASM
  • Updated samplerepo README to use DuckDB WASM

Linked Issue

Closes #6076


Change Description

Background

See #6076 and #6001

New Feature

The Quickstart (both docs, and samplerepo) now uses the DuckDB that is integrated in the lakeFS UI for all data manipulation including writing back to lakeFS. Previously this had to be done through the DuckDB CLI which we shipped as a Docker image. #6044 added the necessary functionality to DuckDB-WASM so now we will use it in the quickstart.

Testing Details

Manually

@rmoff rmoff added docs Improvements or additions to documentation include-changelog PR description should be included in next release changelog minor-change Used for PRs that don't require issue attached labels Jun 15, 2023
@rmoff rmoff marked this pull request as draft June 15, 2023 14:27
@github-actions
Copy link

github-actions bot commented Jun 15, 2023

🎊 PR Preview a8b026a has been successfully built and deployed to https://treeverse-lakeFS-preview-pr-6092.surge.sh

🕐 Build time: 0.015s

🤖 By surge-preview

@ozkatz
Copy link
Collaborator

ozkatz commented Jun 15, 2023

Still in draft, I know but:

  1. this looks GREAT and definitely simplifies things
  2. the screenshot at the bottom of commit-and-merge still uses LAKEFS_OBJECT() so should be replaced.
  3. One suggestion: except for reverting which is a relatively advanced function, the rest (branching merging and committing) could be done directly in the UI too. Perhaps at some point we want to show tabs (UI, CLI) so that users could try out a simpler way if they are not terminal savvy? (not for this PR obviously)

@rmoff
Copy link
Contributor Author

rmoff commented Jun 16, 2023

  • this looks GREAT and definitely simplifies things

Thanks 😄

  • the screenshot at the bottom of commit-and-merge still uses LAKEFS_OBJECT() so should be replaced.

Fixed (and another one like this too)

  • One suggestion: except for reverting which is a relatively advanced function, the rest (branching merging and committing) could be done directly in the UI too. Perhaps at some point we want to show tabs (UI, CLI) so that users could try out a simpler way if they are not terminal savvy? (not for this PR obviously)

Good idea. Logged #6103

@rmoff rmoff requested a review from ozkatz June 16, 2023 10:13
@rmoff rmoff marked this pull request as ready for review June 16, 2023 10:13
@rmoff
Copy link
Contributor Author

rmoff commented Jun 26, 2023

Tested manually against both 0.104.0 (Docker) and build from current master. Previous DuckDB gremlins have been vanquished and this PR is good to go.

Copy link
Contributor

@arielshaqed arielshaqed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neat!

Is there any way to keep an explanation of access keys around? Probably towards the end of the quickstart -- so readers will be able to get started with their own tools.

In general I think we might make it clear that DuckDB is only one tool that works with lakeFS. I really like lakeFS for being a tool rather than a framework, and prefer that our messaging (also) explain that. I realize that this is a lot to ask, especially of a quickstart.

@rmoff rmoff merged commit ad1bdda into master Jun 26, 2023
@rmoff rmoff deleted the docs/wasm-quickstart branch June 26, 2023 12:04
@rmoff
Copy link
Contributor Author

rmoff commented Jun 26, 2023

thanks @arielshaqed -- logged your suggestion and will triage & track :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs Improvements or additions to documentation include-changelog PR description should be included in next release changelog minor-change Used for PRs that don't require issue attached
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Update Quickstart to use DuckDB WASM so we can remove the DuckDB Docker image
3 participants