Skip to content

Commit

Permalink
Merge branch 'main' into patch-1
Browse files Browse the repository at this point in the history
  • Loading branch information
RNHTTR authored Jan 3, 2024
2 parents 63379e6 + d787616 commit d45036b
Show file tree
Hide file tree
Showing 2 changed files with 22 additions and 5 deletions.
22 changes: 19 additions & 3 deletions docs/getting_started/execution-modes-local-conflicts.rst
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
.. _execution-modes-local-conflicts:

Airflow and DBT dependencies conflicts
Airflow and dbt dependencies conflicts
======================================

When using the `Local Execution Mode <execution-modes.html#local>`__, users may face dependency conflicts between
Apache Airflow and DBT. The conflicts may increase depending on the Airflow providers and DBT plugins being used.
Apache Airflow and dbt. The conflicts may increase depending on the Airflow providers and dbt adapters being used.

If you find errors, we recommend users look into using `alternative execution modes <execution-modes.html>`__.

Expand All @@ -25,10 +25,26 @@ In the following table, ``x`` represents combinations that lead to conflicts (va
+---------------+-----+-----+-----+-----+-----+-----+-----+-----+
| 2.7 | x | x | x | x | x | | | |
+---------------+-----+-----+-----+-----+-----+-----+-----+-----+
| 2.8 | x | x | x | x | x | | x | x |
+---------------+-----+-----+-----+-----+-----+-----+-----+-----+


Examples of errors
-----------------------------------

.. code-block:: bash
The conflict is caused by:
apache-airflow 2.8.0 depends on pydantic>=2.3.0
dbt-semantic-interfaces 0.4.2 depends on pydantic~=1.10
apache-airflow 2.8.0 depends on pydantic>=2.3.0
dbt-semantic-interfaces 0.4.2.dev0 depends on pydantic~=1.10
apache-airflow 2.8.0 depends on pydantic>=2.3.0
dbt-semantic-interfaces 0.4.1 depends on pydantic~=1.10
apache-airflow 2.8.0 depends on pydantic>=2.3.0
dbt-semantic-interfaces 0.4.0 depends on pydantic~=1.10
.. code-block:: bash
ERROR: Cannot install apache-airflow==2.2.4 and dbt-core==1.5.0 because these package versions have conflicting dependencies.
Expand Down Expand Up @@ -78,7 +94,7 @@ The table was created by running `nox <https://nox.thea.codes/en/stable/>`__ wi
@nox.parametrize(
"dbt_version", ["1.0", "1.1", "1.2", "1.3", "1.4", "1.5", "1.6", "1.7"]
)
@nox.parametrize("airflow_version", ["2.2.4", "2.3", "2.4", "2.5", "2.6", "2.7"])
@nox.parametrize("airflow_version", ["2.2.4", "2.3", "2.4", "2.5", "2.6", "2.7", "2.8"])
def compatibility(session: nox.Session, airflow_version, dbt_version) -> None:
"""Run both unit and integration tests."""
session.run(
Expand Down
5 changes: 3 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ kubernetes = [
"apache-airflow-providers-cncf-kubernetes>=5.1.1",
]
pydantic = [
"pydantic>=1.10.0,<2.0.0",
"pydantic>=1.10.0",
]

[project.entry-points.cosmos]
Expand Down Expand Up @@ -159,7 +159,7 @@ dependencies = [

[[tool.hatch.envs.tests.matrix]]
python = ["3.8", "3.9", "3.10"]
airflow = ["2.3", "2.4", "2.5", "2.6", "2.7"]
airflow = ["2.3", "2.4", "2.5", "2.6", "2.7", "2.8"]

[tool.hatch.envs.tests.overrides]
matrix.airflow.dependencies = [
Expand All @@ -169,6 +169,7 @@ matrix.airflow.dependencies = [
{ value = "apache-airflow==2.6", if = ["2.6"] },
{ value = "pydantic>=1.10.0,<2.0.0", if = ["2.6"]},
{ value = "apache-airflow==2.7", if = ["2.7"] },
{ value = "apache-airflow==2.8", if = ["2.8"] },
]

[tool.hatch.envs.tests.scripts]
Expand Down

0 comments on commit d45036b

Please sign in to comment.