Skip to content

Conversation

@itholic
Copy link
Contributor

@itholic itholic commented Dec 6, 2023

What changes were proposed in this pull request?

This PR proposes to re-organize GroupByTests.

Why are the changes needed?

Break the big test file into small groups by topic such as Pandas does.

Does this PR introduce any user-facing change?

No, it's test-only.

How was this patch tested?

The existing CI should pass with new structure of test files.

Was this patch authored or co-authored using generative AI tooling?

No.

@itholic itholic marked this pull request as ready for review December 6, 2023 03:26
@itholic
Copy link
Contributor Author

itholic commented Dec 6, 2023

cc @zhengruifeng

@itholic itholic changed the title [WIP][SPARK-46278][CONNECT][PS][TESTS] Re-organize GroupByTests [SPARK-46278][CONNECT][PS][TESTS] Re-organize GroupByTests Dec 6, 2023
@zhengruifeng
Copy link
Contributor

merged to master

dbatomic pushed a commit to dbatomic/spark that referenced this pull request Dec 11, 2023
### What changes were proposed in this pull request?

This PR proposes to re-organize `GroupByTests`.

### Why are the changes needed?

Break the big test file into small groups by topic such as Pandas does.

### Does this PR introduce _any_ user-facing change?

No, it's test-only.

### How was this patch tested?

The existing CI should pass with new structure of test files.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes apache#44196 from itholic/reorg_groupby_test.

Authored-by: Haejoon Lee <haejoon.lee@databricks.com>
Signed-off-by: Ruifeng Zheng <ruifengz@apache.org>
@itholic itholic deleted the reorg_groupby_test branch January 1, 2024 23:41
HyukjinKwon pushed a commit that referenced this pull request Jan 7, 2024
### What changes were proposed in this pull request?
1, check the testing mode:

- `test_assert_vanilla_mode` in `ReusedPySparkTestCase`
- `test_assert_remote_mode` in `ReusedConnectTestCase`

use different function names in case a test suite inherit them both

2, fix the incorrect testing mode introduced in #44196

### Why are the changes needed?
incorrect usage of `PandasOnSparkTestCase` (subclass of `ReusedPySparkTestCase`) in parity tests cause test with vanilla Spark Session:
#44196
#44592

### Does this PR introduce _any_ user-facing change?
no

### How was this patch tested?
ci, added UT

### Was this patch authored or co-authored using generative AI tooling?
no

Closes #44611 from zhengruifeng/py_testing_mode.

Authored-by: Ruifeng Zheng <ruifengz@apache.org>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants