Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[611] RFC: Module Restructuring #612

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

the-other-tim-brown
Copy link
Contributor

What is the purpose of the pull request

Issue: #611
This is a proposal for how the project can be restructured to allow the user to only include the dependencies that they desire. It also contains a proposal for allowing the project to support multiple versions of a spec, like Iceberg v2 and v3.

Brief change log

  • Adds RFC

Verify this pull request

NA

This will allow the user to easily add jars and the required dependencies when executing the standalone RunSync command as well as avoid issues with version differences when adding these jars to an existing class-path that may contain table format related dependencies already.
In addition to this, the jars will still be published to allow users to compose their own shaded jar if they have specific needs or customizations that they want.

We can also create these modules with a suffix indicating which version of the format is supported to allow for different implementations or versions of dependencies for the same format. For example, have an xtable-iceberg-v2 and xtable-iceberg-v3 module.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ashvina this is based off of our conversation, can you include some of the more practical items for why you would want this split?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant