Skip to content

Conversation

@alamb
Copy link
Owner

@alamb alamb commented Mar 24, 2025

This is a PR to test running extended tests from a PR

irenjj and others added 30 commits March 9, 2025 08:19
* Implement tree explain for PartialSortExec

* simplify

* fix

* remove square brackets
* feat: implement tree explain for ProjectionExec

Signed-off-by: Alan Tang <jmtangcs@gmail.com>

* feat(test): support more tests

Signed-off-by: Alan Tang <jmtangcs@gmail.com>

* chore(explain): Reduce redundant output

Signed-off-by: Alan Tang <jmtangcs@gmail.com>

* Propose a different projection formatting

* feat: add project exec tree rendering for hash join

---------

Signed-off-by: Alan Tang <jmtangcs@gmail.com>
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
* Implement `tree` explain for `SortExec`

* fix issues

---------

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
…e#15073)

* [branch-46] Update changelog for backports to 46.0.0 (apache#14977)

* Add note about upgrade guide into the release notes (apache#14979)

* Add new crates

---------

Co-authored-by: xudong.w <wxd963996380@gmail.com>
apache#15081)

* Implement tree explain for `NestedLoopJoinExec`, `CrossJoinExec`, `SortMergeJoinExec` and `SymmetricHashJoinExec`

* fix issues

* fix issues

* fix

* fix proj
…c` (apache#15084)

* Implement `tree explain for `BoundedWindowAggExec` and `WindowAggExec`

* fix clippy

* add bounded win agg test

* add test

* remove

* merge

---------

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
* feat: support tree rendering for StreamingTableExec

Signed-off-by: Alan Tang <jmtangcs@gmail.com>

* feat: simpler expr for streamingExec

Signed-off-by: Alan Tang <jmtangcs@gmail.com>

chore: Describe more precisely

Signed-off-by: Alan Tang <jmtangcs@gmail.com>

---------

Signed-off-by: Alan Tang <jmtangcs@gmail.com>
Bumps [semver](https://github.com/dtolnay/semver) from 1.0.25 to 1.0.26.
- [Release notes](https://github.com/dtolnay/semver/releases)
- [Commits](dtolnay/semver@1.0.25...1.0.26)

---
updated-dependencies:
- dependency-name: semver
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [clap](https://github.com/clap-rs/clap) from 4.5.30 to 4.5.31.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](clap-rs/clap@clap_complete-v4.5.30...v4.5.31)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* implement tree explain for global limit exec

* optimize output
…cross optimizer runs. (apache#15074)

* refactor(15003): permit any combination of runs desired

* refactor(15003): convert macro to a function call on the TestConfig, and highlight when the same testing setup, but different ordering of optimizer runs, effect the outcome.

* chore: remove unneeded comments

* test: update test harness to use passed ref
* Improve parsing `extra_info` in tree explain

* Improve documentation and CrossJoinDisplay

---------

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
* Minor: Fix invalid query in test

* other query
* fix: unparse for subqueryalias

* update

* move test
…Rel.filter (apache#14194)

* Propagate filter info from TableScan to ReadRel

Propagate information in datafusion::logical_expr::TableScan.filters
to substrait::proto::ReadRel.best_effort_filter.

* Add test

* cargo fmt

* Fix clippy error

* Use conjunction

* cargo fmt

* Use ReadRel.filter instead of best_effort_filter

* Check filter types in TableScan.filters

Use TableScan.source.supports_filters_pushdown() to determine if each
filter in TableScan.filters should be included in ReadRel.filter or
ReadRel.best_effort_filter

* Propagate Substrait ReadRel filter to consumer

* Address PR comments

* Propagate TableScan filters to ReadRel filter

---------

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
* fix build

* try to install clang in ci
* Fix broked `serde` feature

* Test `serde` feature

* consolidate serde test into core_integration, update run

---------

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
* Do not normalize values

* Fix tests & update docs

* Prettier

* Lowercase config params

* Add snap to CLI & set up AWS mock

* Refactor tests

* Unify transform and parse

* Fix tests

* Setup CLI

* Show minio output

* Format Cargo.toml

* Do not hardcode AWS params

* Test options parsing

* Add allow http

* Fix aws build

* Fix ip

* Remove slash ☠️

* Format cargo toml

* Remove integration_setup.bash

* Update docs

* Do not hardcode test names

* Relock cargo

* Remove aws sdk and set up minio in-place

* Nit: Add missing ready local to the docs

* Fix backslash test

* Add missing backslash

* put integration scripts in a separate folder

* Move s3 tests from extended to rust flow

* Reorganise the docs

* Prettier

* Do not use rust container to get docker

* Add missing protobuf

* revert change to extended.yml

---------

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Bumps [tempfile](https://github.com/Stebalien/tempfile) from 3.17.1 to 3.18.0.
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/CHANGELOG.md)
- [Commits](Stebalien/tempfile@v3.17.1...v3.18.0)

---
updated-dependencies:
- dependency-name: tempfile
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [syn](https://github.com/dtolnay/syn) from 2.0.98 to 2.0.100.
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](dtolnay/syn@2.0.98...2.0.100)

---
updated-dependencies:
- dependency-name: syn
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Implement tree explain for AggregateExec

* Extract expr formatting logic for readability

* fix empty group_by display
…che#15137)

* feat: implement tree rendering for RepartitionExec

Signed-off-by: Alan Tang <jmtangcs@gmail.com>

* feat: implement tree rendering for WorkTableExec

Signed-off-by: Alan Tang <jmtangcs@gmail.com>

* bug: fix clippy error

Signed-off-by: Alan Tang <jmtangcs@gmail.com>

---------

Signed-off-by: Alan Tang <jmtangcs@gmail.com>
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
…he#15090)

* first draft

* fix tests

* cleanup

* assert

* clippy

* fix test

* fix test
* Config: Add support default sql varchar to view types

* Fix test

* fix test

* Address comments

* Address comments

* Fix slt test
* fixed PushDownFilter bug [15047] by adding a new branch to match to prevent this specific situation

* improved syntax as request by CICL process

* moved check empty node logic into LogicalPlan::Extension(extensioon_plan)

* removed unecessary clone

* removed unecessary test
dependabot bot and others added 22 commits March 21, 2025 09:07
Bumps [tempfile](https://github.com/Stebalien/tempfile) from 3.18.0 to 3.19.1.
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/CHANGELOG.md)
- [Commits](Stebalien/tempfile@v3.18.0...v3.19.1)

---
updated-dependencies:
- dependency-name: tempfile
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…5329)

* added explaination for Schema and DFSchema to documentation

* change code block to quote block since CICL request a full code but we are only introducing syntax

* improved language

* Update docs/source/library-user-guide/working-with-exprs.md

---------

Co-authored-by: Oleks V <comphead@users.noreply.github.com>
…s out of core (apache#15316)

* First Iteration

* fix: CI tests

* stable waypoint, documentation pending, memory moce pending

* stable waypoint 2: add document pending, get a heads up pending

* pushing for test and review

* fix:mock in test

* fix:cliipy
* feat: simplify regex wildcard pattern

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update slt result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* make a constant

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* handle null value

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add rowsort to union

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
* Fix empty aggregation function count() in Substrait

* Fix window function function count() with no arguments in Substrait

* Add explanatory comments
* Improve groupby error message for new version of expand wildcard logic, also modified related CICL content

* Improved error message for new expand wildcard logic, also editted related CICL test

* improved syntax requested by CICL process

* unified  and , updated test error message
* Basic setup done for physical plan insta tests

* Migrated tests to insta in aggregate/mod.rs

* Migrated tests to insta in aggregate/topk/priority_map.rs

* Migrated tests to insta in joins

* Fallback to previous implementation in a test due to an issue (apache#15312)

* Fix formatting

* Update Cargo.toml

* Revert any Cargo.lock changes

* Clean workspace and attempt fixing failing build test

* format Cargo.toml

* Add insta as a dev-dependency instead

* Update datafusion/physical-plan/src/aggregates/mod.rs

Co-authored-by: Dmitrii Blaginin <dmitrii@blaginin.me>

* Preserved comments from aggregates/mod.rs

* Resolved errors regarding dependencies and formatting

---------

Co-authored-by: Dmitrii Blaginin <dmitrii@blaginin.me>
* fixEmptyArrayHasAll

* Fix the array

* Change to more efficient BooleanBuffer

* Separate test case

* small fix

* Update datafusion/functions-nested/src/array_has.rs

Co-authored-by: Alex Huang <huangweijun1001@gmail.com>

* Add the string failure test cases

* fix

---------

Co-authored-by: Alex Huang <huangweijun1001@gmail.com>
* chore(deps): Update sqlparser to 0.55.0

* merge upstream

* style: cargo fmt

* fix: import

* style: cargo fmt

* fix: struct.slt, update.slt and lateral join

* chore: cargo fmt & remove unwrap

* fix: remove meaningless comment for rustfmt

* refactor: plan_datafusion_err, remove clone

* remove println

---------

Co-authored-by: silezhou <zhousile2002@gmail.com>
Co-authored-by: jonahgao <jonahgao@msn.com>
* Only unnest source for `EmptyRelation`

* Add a note on new condition

* Remove new test

* Put all unnest assumptions into one function
Bumps [rust_decimal](https://github.com/paupino/rust-decimal) from 1.37.0 to 1.37.1.
- [Release notes](https://github.com/paupino/rust-decimal/releases)
- [Changelog](https://github.com/paupino/rust-decimal/blob/master/CHANGELOG.md)
- [Commits](paupino/rust-decimal@1.37.0...1.37.1)

---
updated-dependencies:
- dependency-name: rust_decimal
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [chrono-tz](https://github.com/chronotope/chrono-tz) from 0.10.1 to 0.10.2.
- [Release notes](https://github.com/chronotope/chrono-tz/releases)
- [Commits](https://github.com/chronotope/chrono-tz/commits)

---
updated-dependencies:
- dependency-name: chrono-tz
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* include some BinaryOperator from sqlparser

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* slt for new operators

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add document about availability

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
@alamb alamb marked this pull request as ready for review March 24, 2025 18:42
@alamb
Copy link
Owner Author

alamb commented Mar 24, 2025

Step 1: Wait for all normal PR checks to pass

(Screen shot)

Step 2: Try to trigger extended tests

Step 3: Observe the reported checks

@alamb
Copy link
Owner Author

alamb commented Mar 24, 2025

🤔 it seem like the extended tests just ran for my PR by default 🤔

Screenshot 2025-03-24 at 5 08 10 PM

@alamb
Copy link
Owner Author

alamb commented Mar 25, 2025

@alamb alamb closed this Mar 25, 2025
alamb added a commit that referenced this pull request Jul 15, 2025
…apache#16732)

* refactor filter pushdown to remove assumptions blocking hashjoinexec pushdown

* fix assertion

* fix

* fix any/all

* remove hashjoinexec impl

* lint

* resolve merge

* fix imports

* fix merge

* fix typo

* lint

* Improve some documentation in filter pushdown (#32)

* refactor to a struct with a field instead of enum

* better docs, rename methods

* fix cargo doc

---------

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.