Skip to content

Merge remote-tracking branch 'upstream/main' into gb/merge-upstream-a… #30

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
gbrgr merged 25 commits into from
Jan 9, 2026
Merged

Merge remote-tracking branch 'upstream/main' into gb/merge-upstream-a… #30

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
decab01
Merge remote-tracking branch 'upstream/main' into gb/merge-upstream-a…
gbrgr Jan 8, 2026
63dca52
Fix merge mistakes
gbrgr Jan 8, 2026
6bba319
.
gbrgr Jan 8, 2026
d325940
.
gbrgr Jan 8, 2026
13008d7
.
gbrgr Jan 8, 2026
044ecac
.
gbrgr Jan 8, 2026
b2674f7
.
gbrgr Jan 8, 2026
665f4a6
.
gbrgr Jan 8, 2026
b5b979b
.
gbrgr Jan 8, 2026
9b3a12c
.
gbrgr Jan 8, 2026
fdbfb22
.
gbrgr Jan 8, 2026
c9c3bdd
.
gbrgr Jan 8, 2026
59516b0
format
gbrgr Jan 8, 2026
4050a2c
Disable python bindings again
gbrgr Jan 8, 2026
fa63bcd
Fix clippy errors
gbrgr Jan 8, 2026
6fc3fe1
Enable tests
gbrgr Jan 8, 2026
847ad50
Fix merge mistake
gbrgr Jan 8, 2026
bcf5a8d
.
gbrgr Jan 8, 2026
3e919ec
.
gbrgr Jan 8, 2026
3e6313d
.
gbrgr Jan 8, 2026
826aa98
Clippy fix
gbrgr Jan 8, 2026
6f1667c
fix: Reserved sort order ID cannot contain any fields (#1978)
aditya-subrahmanyan Jan 9, 2026
3d5e160
feat(datafusion): Add LIMIT pushdown support (#2006)
viirya Jan 9, 2026
b75d17a
Redo comment
gbrgr Jan 9, 2026
3ae9f8b
Add case-sensitive attribute to incremental scan for consistency
gbrgr Jan 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .asf.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,9 @@ github:
dismiss_stale_reviews: false

required_linear_history: true
del_branch_on_merge: true
pull_requests:
# auto-delete head branches after being merged
del_branch_on_merge: true
features:
wiki: false
issues: true
Expand Down
11 changes: 11 additions & 0 deletions .cargo/audit.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,4 +25,15 @@ ignore = [
#
# Introduced by hive_metastore, tracked at https://github.com/cloudwego/pilota/issues/293
"RUSTSEC-2024-0388",
# `paste` is unmaintained; consider using an alternative
#
# Introduced by hive_metastore, tracked at https://github.com/cloudwego/pilota/issues/293
"RUSTSEC-2024-0436",
# `rustls-pemfile` is unmaintained
#
# Introduced by object_store, see https://github.com/apache/arrow-rs-object-store/issues/564
"RUSTSEC-2025-0134",

# Tracked here: https://github.com/paupino/rust-decimal/issues/766
"RUSTSEC-2026-0001",
]
3 changes: 1 addition & 2 deletions .github/workflows/audit.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,12 +40,11 @@ jobs:
security_audit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v6
- name: Setup Rust toolchain
uses: ./.github/actions/setup-builder
with:
rust-version: stable
- uses: rustsec/audit-check@v2.0.0
with:
token: ${{ secrets.GITHUB_TOKEN }}
ignore: RUSTSEC-2024-0436
51 changes: 50 additions & 1 deletion .github/workflows/bindings_python_ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,10 +41,21 @@ concurrency:
cancel-in-progress: true

jobs:
# check-rust:
# runs-on: ubuntu-latest
# steps:
# - uses: actions/checkout@v6
# - name: Check format
# working-directory: "bindings/python"
# run: cargo fmt --all -- --check
# - name: Check clippy
# working-directory: "bindings/python"
# run: cargo clippy --all-targets --all-features -- -D warnings

check-python:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v6
- uses: astral-sh/setup-uv@v7
with:
version: "0.9.3"
Expand All @@ -60,3 +71,41 @@ jobs:
working-directory: "bindings/python"
run: |
uvx ruff check .

# test:
# runs-on: ${{ matrix.os }}
# strategy:
# matrix:
# os:
# - ubuntu-latest
# - macos-latest
# - windows-latest
# steps:
# - uses: actions/checkout@v6
# - uses: actions/setup-python@v6
# with:
# python-version: 3.12
# - uses: PyO3/maturin-action@v1
# with:
# working-directory: "bindings/python"
# command: build
# args: --out dist --sdist
# - uses: astral-sh/setup-uv@v7
# with:
# version: "0.9.3"
# enable-cache: true
# - name: Sync dependencies
# working-directory: "bindings/python"
# shell: bash
# run: |
# make install
# - name: Install built wheel
# working-directory: "bindings/python"
# shell: bash
# run: |
# uv pip install --reinstall dist/pyiceberg_core-*.whl
# - name: Run tests
# working-directory: "bindings/python"
# shell: bash
# run: |
# make test
11 changes: 5 additions & 6 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ jobs:
- ubuntu-latest
- macos-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v6

- name: Setup Rust toolchain
uses: ./.github/actions/setup-builder
Expand Down Expand Up @@ -101,7 +101,7 @@ jobs:
root-reserve-mb: 10240
temp-reserve-mb: 10240

- uses: actions/checkout@v5
- uses: actions/checkout@v6

- name: Setup Rust toolchain
uses: ./.github/actions/setup-builder
Expand All @@ -126,7 +126,7 @@ jobs:
- macos-latest
- windows-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v6

- name: Setup Rust toolchain
uses: ./.github/actions/setup-builder
Expand All @@ -143,7 +143,6 @@ jobs:
matrix:
test-suite:
- { name: "default", args: "--all-targets --all-features --workspace" }
- { name: "smol", args: "--all-targets --no-default-features --features smol --features storage-all --workspace" }
- { name: "doc", args: "--doc --all-features --workspace" }
name: Unit Tests (${{ matrix.test-suite.name }})
steps:
Expand All @@ -158,7 +157,7 @@ jobs:
root-reserve-mb: 10240
temp-reserve-mb: 10240

- uses: actions/checkout@v5
- uses: actions/checkout@v6

- name: Setup Rust toolchain
uses: ./.github/actions/setup-builder
Expand All @@ -180,7 +179,7 @@ jobs:
name: Verify MSRV
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v6
- name: Install protoc
uses: arduino/setup-protoc@v3
with:
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/ci_typos.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,6 @@ jobs:
env:
FORCE_COLOR: 1
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v6
- name: Check typos
uses: crate-ci/typos@v1.39.2
uses: crate-ci/typos@v1.41.0
3 changes: 2 additions & 1 deletion .github/workflows/publish.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,10 +42,11 @@ jobs:
- "crates/catalog/glue"
- "crates/catalog/hms"
- "crates/catalog/rest"
- "crates/catalog/s3tables"
- "crates/catalog/sql"
- "crates/integrations/datafusion"
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v6

- name: Setup Rust toolchain
uses: ./.github/actions/setup-builder
Expand Down
10 changes: 5 additions & 5 deletions .github/workflows/release_python.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ jobs:
runs-on: ubuntu-latest
needs: [validate-release-tag]
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v6

- name: Install toml-cli
if: ${{ needs.validate-release-tag.outputs.is-rc == 'true' }}
Expand All @@ -107,7 +107,7 @@ jobs:
command: sdist
args: -o dist
- name: Upload sdist
uses: actions/upload-artifact@v5
uses: actions/upload-artifact@v6
with:
name: wheels-sdist
path: bindings/python/dist
Expand All @@ -128,7 +128,7 @@ jobs:
}
- { os: ubuntu-latest, target: "armv7l" }
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v6

- name: Install toml-cli
if: ${{ needs.validate-release-tag.outputs.is-rc == 'true' }}
Expand Down Expand Up @@ -159,7 +159,7 @@ jobs:
command: build
args: --release -o dist
- name: Upload wheels
uses: actions/upload-artifact@v5
uses: actions/upload-artifact@v6
with:
name: wheels-${{ matrix.os }}-${{ matrix.target }}
path: bindings/python/dist
Expand All @@ -178,7 +178,7 @@ jobs:

steps:
- name: Download all the dists
uses: actions/download-artifact@v6
uses: actions/download-artifact@v7
with:
pattern: wheels-*
merge-multiple: true
Expand Down
37 changes: 32 additions & 5 deletions .github/workflows/release_python_nightly.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ jobs:
if: github.repository == 'apache/iceberg-rust' # Only run for apache repo
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v6

- uses: ./.github/actions/overwrite-package-version # Overwrite package version with timestamp
with:
Expand All @@ -56,7 +56,7 @@ jobs:
args: -o dist

- name: Upload sdist
uses: actions/upload-artifact@v5
uses: actions/upload-artifact@v6
with:
name: wheels-sdist
path: bindings/python/dist
Expand All @@ -78,7 +78,7 @@ jobs:
}
- { os: ubuntu-latest, target: "armv7l" }
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v6

- uses: ./.github/actions/overwrite-package-version # Overwrite package version with timestamp
with:
Expand All @@ -102,7 +102,7 @@ jobs:
args: --release -o dist

- name: Upload wheels
uses: actions/upload-artifact@v5
uses: actions/upload-artifact@v6
with:
name: wheels-${{ matrix.os }}-${{ matrix.target }}
path: bindings/python/dist
Expand All @@ -120,17 +120,44 @@ jobs:

steps:
- name: Download all the dists
uses: actions/download-artifact@v6
uses: actions/download-artifact@v7
with:
pattern: wheels-*
merge-multiple: true
path: bindings/python/dist
- name: List downloaded artifacts
run: ls -R bindings/python/dist
- name: Publish to TestPyPI
id: publish-testpypi
continue-on-error: true
uses: pypa/gh-action-pypi-publish@release/v1
with:
repository-url: https://test.pypi.org/legacy/
skip-existing: true
packages-dir: bindings/python/dist
verbose: true
- name: Display error message on publish failure
if: steps.publish-testpypi.outcome == 'failure'
run: |
echo "::error::Failed to publish to TestPyPI"
echo ""
echo "⚠️ TestPyPI Publish Failed"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo ""
echo "This may be due to TestPyPI storage limits."
echo "See: https://docs.pypi.org/project-management/storage-limits"
echo ""
echo "To resolve this issue, use the pypi-cleanup utility to clean up old TestPyPI artifacts:"
echo "https://pypi.org/project/pypi-cleanup/"
echo ""
echo " uvx pypi-cleanup --package pyiceberg-core --host https://test.pypi.org/ \\"
echo " --verbose -d 10 --do-it --username <username>"
echo ""
echo "Requirements:"
echo " • Must be a maintainer for pyiceberg-core on TestPyPI"
echo " (https://test.pypi.org/project/pyiceberg-core)"
echo " • Requires TestPyPI password and 2FA"
echo " • ⚠️ ONLY do this for TestPyPI, NOT for production PyPI!"
echo ""
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
exit 1
2 changes: 1 addition & 1 deletion .github/workflows/stale.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ jobs:
if: github.repository_owner == 'apache'
runs-on: ubuntu-22.04
steps:
- uses: actions/stale@v10.1.0
- uses: actions/stale@v10.1.1
with:
stale-issue-label: 'stale'
exempt-issue-labels: 'not-stale'
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/website.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ jobs:
permissions:
contents: write
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v6

- name: Setup mdBook
uses: peaceiris/actions-mdbook@v2
Expand Down
1 change: 1 addition & 0 deletions .licenserc.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ header:
- "**/DEPENDENCIES.*.tsv"
# Release distributions
- "dist/*"
- "target"
- "Cargo.lock"
- "bindings/python/uv.lock"
- ".github/PULL_REQUEST_TEMPLATE.md"
Expand Down
3 changes: 3 additions & 0 deletions .typos.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,8 @@
# specific language governing permissions and limitations
# under the License.

[type.rust]
extend-ignore-identifiers-re = ["^bimap$"]

[files]
extend-exclude = ["**/testdata", "CHANGELOG.md"]
Loading
Loading