Adapter Maintainers: Upgrading to dbt-core 1.3.0

[dataders]

This discussion is for communicating to adapter maintainers the scope of work needed to make use of the changes in 1.3.0. If you have questions and concerns, please ask them here for posterity.

For the guide to upgrading an adapter to be compatible with dbt-core=1.2.0, please see #5468.

The latest release cut for 1.3.0, dbt-core==1.3.0rc2 was published on October 3, 2022 (PyPI | Github). We are targeting releasing the official cut of 1.3.0 in time for the week of October 16 (in time for Coalesce conference).

This minor release contains no breaking changes to interfaces but does change the default behavior in ways that may impact your adapter. In addition there are new features for end-users and some developer quality-of-life improvements that will benefit adapter maintainers. This means that end users can install run dbt-core==1.2.0 alongside an adapter of a previous minor version (e.g. 1.1.0 or 1.0.0). However, they won't necessarily have access to the new features and most importantly: the SQL being generated is not guaranteed to be consistent.

Support for python models

The statement() and create_table_as() macros accept a new argument, language, with a default value of sql.

You might not even need to add this argument if your adapter doesn't currently support Python. Beyond this potential change, there should be no other breaking changes to your adapter.

If you'd like to bring Python support to dbt, reach out! we don't have great docs on this yet, but if you're interested, we're happy to help you get started and support where we are able!

Incremental mats: more standard and more error-proof

user-facing changes

1. new in 1.3:
   1. breaking change. the default strategy of incremental models used to be delete+insert. it is now append. See point below to override this.
   2. a new macro, adapter_get_incremental_default_sql(), in which adapter maintainers can specify a 'default' strategy for incremental models.
   3. a new Adapter method, valid_incremental_strategies() in which adapter maintainers can define all supported strategies. Previously, if i user defined an unsupported incremental strategy, they would not learn of their error until compile time or later. Now dbt will throw an error before attempting to compile.

non-user facing changes

1. there was significant work ([CT-646] [Proposal] Streamline Incremental Strategies #5245 Initial refactoring of incremental materialization #5359) put into refactoring the default incremental materialization. If your adapter overrides the default, we recommend that you either:
   2. refactor your materialization to match that of the default
   1. even better, determine exactly why you currently need to override the default, and perhaps we can find a way to bring that flexibility into the core adapter so that you can stay up to date with the default mat

New functional adapter tests:

The directory of tests just for folks like us, tests/functional/adapter/ has some new goodies for us!

• test_basic.py
  • BaseIncrementalNotSchemaChange
• test_data_types.py
  • BaseTypeBoolean
• GRANTs test_grants.py
  • BaseInvalidGrants
• new utils tests (test_utils.py) for the macros recently migrated from dbt-utils
  • BaseCurrentTimestampNaive
  • BaseArrayAppend
  • BaseArrayConcat
  • BaseArrayConstruct
• test_concurrency.py
  • BaseConcurrency (example: dbt-snowflake#252)
• tests/functional/adapter/test_ephemeral.py
  • BaseEphemeralMulti(example: dbt-snowflake#244)

[dbeatty10]

Example implementations of get_incremental_default_sql + valid_incremental_strategies:

• https://github.com/databricks/dbt-databricks/pull/148/files
• https://github.com/dbt-labs/dbt-snowflake/pull/196/files

The crucial piece is implementing strategy_sql_macro_func within the incremental materialization of the adapter.