Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for materializing reference tables #16792

Closed
rohit-nayak-ps opened this issue Sep 16, 2024 · 0 comments · Fixed by #16787
Closed

Add support for materializing reference tables #16792

rohit-nayak-ps opened this issue Sep 16, 2024 · 0 comments · Fixed by #16787

Comments

@rohit-nayak-ps
Copy link
Contributor

rohit-nayak-ps commented Sep 16, 2024

Feature Description

Sharded databases often need the ability to join their tables with smaller “reference” tables. For example, the product table could be seen as a reference table. Other use cases are tables that map static information like zip code to city, etc.

Joining against these tables across keyspaces results in cross-shard joins that may not be very efficient or fast.

Vitess allows you to create a table in a sharded keyspace as a reference table. This means that it will treat the table as having an identical set of rows across all shards. A query that joins a sharded table against such reference tables is then performed locally within each shard.

However maintaining reference tables is not supported intrinsically by Vitess. The user is expected to keep copies of the tables in all the shards of the keyspaces where they are needed and sync them with the source tables as they get modified. They would setup a Materialize workflow with a select * from <table> filter for all the reference tables.

It will be great if Vitess can support reference table materialization by extending the current Materialize command where we can provide a list of tables to copy to all shards in the target keyspace.

See: https://vitess.io/docs/20.0/user-guides/vschema-guide/advanced-vschema/#reference-tables

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant