Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

on import prefilter database tables based on table S3 location #745

Closed
zsaltys opened this issue Sep 12, 2023 · 4 comments
Closed

on import prefilter database tables based on table S3 location #745

zsaltys opened this issue Sep 12, 2023 · 4 comments
Labels
effort: medium priority: medium status: not-picked-yet At the moment we have not picked this item. Anyone can pick it up type: newfeature New feature request

Comments

@zsaltys
Copy link
Contributor

zsaltys commented Sep 12, 2023

Is your idea related to a problem? Please describe.
A user may have 1 Glue database which has tables which point to many S3 buckets. Importing such a database into a single dataset does not make sense as potentially many of the tables will not point to the same bucket as the dataset.

Describe the solution you'd like
When a user imports a database which has tables which point to a buckets which are not part of a dataset then those tables are filtered out and will never show up as part of the dataset.

@zsaltys
Copy link
Contributor Author

zsaltys commented Sep 12, 2023

related to:
#720
#284

@zsaltys
Copy link
Contributor Author

zsaltys commented Sep 12, 2023

We will contribute this fix as part of #284

@dlpzx
Copy link
Contributor

dlpzx commented Sep 13, 2023

Hi @zsaltys thanks for opening the issue. I like the separation between the multiple-buckets datasets and the filtering imported tables. We'll keep the discussion on both #720 and here

@dlpzx dlpzx added type: newfeature New feature request priority: medium status: not-picked-yet At the moment we have not picked this item. Anyone can pick it up effort: medium labels Sep 13, 2023
@dlpzx
Copy link
Contributor

dlpzx commented Dec 5, 2023

Closing as completed

@dlpzx dlpzx closed this as completed Dec 5, 2023
@dlpzx dlpzx moved this to Done in Data.all Backlog Aug 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
effort: medium priority: medium status: not-picked-yet At the moment we have not picked this item. Anyone can pick it up type: newfeature New feature request
Projects
Status: Done
Development

No branches or pull requests

2 participants