Skip to content

Latest commit

 

History

History
76 lines (63 loc) · 7.03 KB

CHANGELOG.md

File metadata and controls

76 lines (63 loc) · 7.03 KB
Raw

dbt-spark Changelog

  • This file provides a full account of all changes to dbt-spark.
  • Changes are listed under the (pre)release in which they first appear. Subsequent releases include changes from previous releases.
  • "Breaking changes" listed under a version may require action from end users or external maintainers when upgrading to that version.
  • Do not edit this file directly. This file is auto-generated using changie. For details on how to document a change, see the contributing guide

dbt-spark 1.3.3 - August 04, 2023

dbt-spark 1.3.2 - May 26, 2023

Fixes

  • Fixed issue where table materialization was not always properly refreshing for non-admin users on Databricks (#725)

Contributors

dbt-spark 1.3.1 - January 20, 2023

Fixes

  • Support new error messages in the future Spark. (#515, #520)

Under the Hood

  • updating python version in tox (#536, #534)
  • fix post new release tox issues around passenv and allowlist_externals (#544, #544)

Contributors

dbt-spark 1.3.0 - October 12, 2022

Features

  • Support python model through notebook, currently supported materializations are table and incremental (#417, #377)
  • Add changie to dbt-spark (#416, #418)
  • merge exclude columns for spark models (#5260, #390)
  • Array macros (#453, #454)
  • implement testing for type_boolean in spark (#470, #471)
  • Support job cluster in notebook submission method, remove requirement for user for python model submission (#444, #467)
  • Migrate dbt-utils current_timestamp macros into core + adapters (#483, #480)

Fixes

  • Pin pyodbc to version 4.0.32 to prevent overwriting libodbc.so and libltdl.so on Linux (#397, #398)
  • python incremental model tmp table using correct schema (#441, #445)
  • change to get_columns_in_relation to fix cache inconsistencies to fix cache issues in incremental models causing failure on on_schema_change (#447, #451)

Under the Hood

  • Support core incremental refactor (#4402, #394)
  • specify supported_languages for materialization that support python models (#437, #440)
  • Submit python model with Command API by default. Adjusted run name (#424, #442)
  • Better interface for python submission (#452, #452)
  • ignore mypy typing issues (#461, #462)
  • Enable Pandas and Pandas-on-Spark DataFrames for dbt python models (#468, #469)
  • Convert df to pyspark DataFrame if it is koalas before writing (#473, #474)

Dependency

  • Bump pyodbc from 4.0.32 to 4.0.34 (#417, #459)
  • Bump black from 22.3.0 to 22.8.0 (#417, #458)
  • Update click requirement from ~=8.0.4 to ~=8.1.3 (#417, #457)
  • Bump mypy from 0.950 to 0.971 (#417, #456)
  • Bump thrift-sasl from 0.4.1 to 0.4.3 (#417, #455)

Contributors

Previous Releases

For information on prior major and minor releases, see their changelogs: