Skip to content

Commit 7362bbd

Browse files
authored
test: pyiceberg-core, pin datafusion and unpin pyarrow (#1648)
## Which issue does this PR close? - Closes #. ## What changes are included in this PR? Follow up to #1635 We dont need to pin pyarrow specifically, just grab it from pyiceberg. Revert datafusion version threshold to 45. Version 45 is the lowest version which will still work. The underlying issue for #1635 is with the **newer** versions of datafusion, mainly datafusion 49 due to a breaking change described in #1647 So pyiceberg-core works with datafusion (python) library 45, 46, 47, and 48. just not 49 I verified locally with all the versions above: ``` hatch env prune && hatch run dev:test ``` i double checked. `insert_into` is available in datafusion 45. apache/datafusion@7f9a8c0 ## Are these changes tested?
1 parent bd228fe commit 7362bbd

File tree

2 files changed

+3
-3
lines changed

2 files changed

+3
-3
lines changed

bindings/python/pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@ python-source = "python"
4444
ignore = ["F403", "F405"]
4545

4646
[tool.hatch.envs.dev]
47-
dependencies = ["maturin>=1.0,<2.0", "pytest>=8.3.2", "pyarrow==16.*", "datafusion==47.*", "pyiceberg[sql-sqlite]>=0.9.1"]
47+
dependencies = ["maturin>=1.0,<2.0", "pytest>=8.3.2", "datafusion==45.*", "pyiceberg[sql-sqlite,pyarrow]>=0.9.1"]
4848

4949
[tool.hatch.envs.dev.scripts]
5050
build = "maturin build --out dist --sdist"

bindings/python/tests/test_datafusion_table_provider.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -27,8 +27,8 @@
2727
import datafusion
2828

2929
assert (
30-
datafusion.__version__ >= "47"
31-
) # iceberg table provider only works for datafusion >= 47
30+
datafusion.__version__ >= "45"
31+
) # iceberg table provider only works for datafusion >= 45
3232

3333

3434
@pytest.fixture(scope="session")

0 commit comments

Comments
 (0)