Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: upgrade to dbt 1.8 #614

Merged
merged 5 commits into from
Apr 15, 2024
Merged

chore: upgrade to dbt 1.8 #614

merged 5 commits into from
Apr 15, 2024

Conversation

Jrmyy
Copy link
Contributor

@Jrmyy Jrmyy commented Mar 29, 2024

Description

Fixes #610

fix: remove table_owner retrieved from manifest
In DBT documentation, 2 owners are displayed: on the top line, we have the dbt owner as defined in the metadata, on the second line it is the owner of the sql representation of the model in Athena. But Athena does not support table owner except if used with LakeFormation. But in any case it is not an owner, in a traditional way as we can see it in Postgres. Since dbt 1.8 remove the ability to use the manifest object when retrieving catalog or schemas, we were forced to remove it. But it feels like we can remove it now because we have an owner displayed and the documentation and it reflects more what happened in reality.

Checklist

  • You followed contributing section
  • You kept your Pull Request small and focused on a single feature or bug fix.
  • You added unit testing when necessary
  • You added functional testing when necessary

@Jrmyy Jrmyy force-pushed the chore-upgrade-to-dbt-1-8 branch 2 times, most recently from f2ccbac to 3c7e441 Compare March 29, 2024 17:54
@Jrmyy Jrmyy force-pushed the chore-upgrade-to-dbt-1-8 branch 2 times, most recently from df9bd79 to 96d2e61 Compare April 2, 2024 08:48
Jrmyy added 2 commits April 2, 2024 11:46
In DBT documentation, 2 owners are displayed: on the top line, we have the dbt owner as defined in the metadata, on the second line it is the owner of the sql representation of the model in Athena. But Athena does not support table owner except if used with LakeFormation. But in any case it is not an owner, in a traditional way as we can see it in Postgres. Since dbt 1.8 remove the ability to use the manifest object when retrieving catalog or schemas, we were forced to remove it. But it feels like we can remove it now because we have an owner displayed and the documentation and it reflects more what happened in reality.
@Jrmyy Jrmyy force-pushed the chore-upgrade-to-dbt-1-8 branch from 98b1a9c to 85f7178 Compare April 2, 2024 09:49
@nicor88
Copy link
Contributor

nicor88 commented Apr 3, 2024

Overall looks good.

dbt is still releasing new beta versions, see this https://github.com/dbt-labs/dbt-core/releases/tag/v1.8.0b2.

My main question is: shall we consider to merge and release this and keep updating based on beta/final release? or we keep this open till the final release is reached?

@Jrmyy
Copy link
Contributor Author

Jrmyy commented Apr 4, 2024

If we follow what other adapters did, they all merged and release beta versions, so we might want to also do that.

It makes things trickier to deploy hotfixes (targeting commits or a v1.7 branch) but it will also let more time to stress test the beta version of production-grade projects

@nicor88
Copy link
Contributor

nicor88 commented Apr 4, 2024

Others adapters have people and resources to allocate to dbt. If we find time to keep up with the dbt 1.8 pre-, releases, I'm fine to merge it already.
@svdimchenko wdyt?

@svdimchenko
Copy link
Contributor

svdimchenko commented Apr 9, 2024

Others adapters have people and resources to allocate to dbt. If we find time to keep up with the dbt 1.8 pre-, releases, I'm fine to merge it already. @svdimchenko wdyt?

I believe we should create v1.7 branch as @Jrmyy mentioned to be able do hotfixes on previous dbt version. I'm not sure if other dbt-dependent projects will upgrade to 1.8 smoothly as well (like elementary)

I've created v1.7 branch. Now I believe we are ready to go

@nicor88
Copy link
Contributor

nicor88 commented Apr 9, 2024

@svdimchenko I don't see much sense of creating another branch tbh, as it will just create more extra work.
Let's use KISS, we target the main branch and we keep releasing periodically based on the beta release, given community contributions that target main will be way easier to maintain.

The good part of 1.8,it is that we don't have to follow anymore dbt semantic library versioning. so we are starting to become free on the semantic library version of the adapter.

Said so, I'm in favor of merging this, and do a release. cc @Jrmyy

@Jrmyy Jrmyy marked this pull request as ready for review April 15, 2024 14:16
@Jrmyy Jrmyy merged commit e458b59 into main Apr 15, 2024
10 checks passed
@Jrmyy Jrmyy deleted the chore-upgrade-to-dbt-1-8 branch April 15, 2024 15:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

upgrade to support dbt-core v1.8.0
3 participants