Skip to content

Commit

Permalink
Raise exception on unexpected error of list relations (#270)
Browse files Browse the repository at this point in the history
Because of a AWS Glue issue, list releations was set to handle any exception and returning empty list of tables. The problem is that further logic decides to create or replace table, which leads to table being overwritten and all previous partitions are lost.

I believe that the problem the previous fix solves is no where near as important to Databricks, and the problem it causes is very bad.

resolves #266

Signed-off-by: Andre Furlan <andre.furlan@databricks.com>
  • Loading branch information
andrefurlan-db authored Feb 17, 2023
1 parent 3cd53c9 commit 54cf1b0
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions dbt/adapters/databricks/impl.py
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,7 @@ def list_relations_without_caching( # type: ignore[override]
else:
description = "Error while retrieving information about"
logger.debug(f"{description} {schema_relation}: {e.msg}")
return []
raise e

return [
self.Relation.create(
Expand Down Expand Up @@ -165,7 +165,7 @@ def _list_relations_with_information(
else:
description = "Error while retrieving information about"
logger.debug(f"{description} {schema_relation.without_identifier()}: {e.msg}")
results = []
raise e

relations: List[Tuple[DatabricksRelation, str]] = []
for row in results:
Expand Down

0 comments on commit 54cf1b0

Please sign in to comment.