Skip to content

Commit

Permalink
Add max operator to get_relation_last_modified macro (#234)
Browse files Browse the repository at this point in the history
### Summary

Metadata freshness checks were not working properly if `dbt source
freshness` was ran without using the `loaded_at_field` keyword. This is
because we were not reducing the amount of results to the max
commited_at value in our metadata select statement.

### Description

Added the max operator as suggested by @KingLommel. Also fixed the
relevant test to include more than one snapshot.

### Changelog

-   [x] Added a summary of what this PR accomplishes to CHANGELOG.md

### Related Issue

#229
  • Loading branch information
ravjotbrar authored Jun 26, 2024
1 parent 8a295da commit 9a9391e
Show file tree
Hide file tree
Showing 3 changed files with 7 additions and 3 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# dbt-dremio MAIN

- [#223](https://github.com/dremio/dbt-dremio/issues/224) Implement merge strategy for incremental materializations
- [#229](https://github.com/dremio/dbt-dremio/issues/229) Add max operator to get_relation_last_modified macro

# dbt-dremio v1.7.0

Expand Down
2 changes: 1 addition & 1 deletion dbt/include/dremio/macros/adapters/metadata.sql
Original file line number Diff line number Diff line change
Expand Up @@ -312,7 +312,7 @@ limitations under the License.*/
{%- if relation.type != 'view' -%}

{%- call statement('last_modified', fetch_result=True) -%}
select committed_at as last_modified,
select max(committed_at) as last_modified,
{{ current_timestamp() }} as snapshotted_at
from TABLE( table_snapshot('{{relation}}') )
{%- endcall -%}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import pytest
from dbt.tests.util import run_dbt
from tests.utils.util import BUCKET
from tests.utils.util import BUCKET, relation_from_name


freshness_via_metadata_schema_yml = """version: 2
Expand Down Expand Up @@ -83,7 +83,10 @@ def test_get_last_relation_modified(self, project):

# run command
result = run_dbt(["seed"])

relation = relation_from_name(project.adapter, "test_source")
result = project.run_sql(
f"INSERT INTO {relation} VALUES (10, 'name')", fetch="one"
)
results = run_dbt(["source", "freshness"])
assert len(results) == 1
result = results[0]
Expand Down

0 comments on commit 9a9391e

Please sign in to comment.