Optimize list_relations_without_caching() macro to avoid calling get_relationship() on every object. #181
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Describe your changes
Currently inside list_relations_without_caching() macro, we are calling get_relation_type for each and every relation. Impala doesn't have direct way to fetch all the relationships along with the table type in one query. This was causing issues where the schemas had 1000s of tables in them.
As part of this change, fetching the table type dynamically as part of the getter method. Although it isn't a standard interface provided by DBT, solving it this way for impala will reduce the number of get_relation_type() metadata calls dramatically.
Internal Jira ticket number or external issue link:
https://jira.cloudera.com/browse/DBT-744
Testing procedure/screenshots(if appropriate):
Checklist before requesting a review