Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Minor: Move some aggregate error tests to sqllogictests #5055

Merged
merged 3 commits into from
Jan 26, 2023

Conversation

alamb
Copy link
Contributor

@alamb alamb commented Jan 25, 2023

Which issue does this PR close?

part of #4495

Rationale for this change

We are slowly moving tests to the easier to use / faster to run sqllogictest framework

What changes are included in this PR?

port some tests

Are these changes tested?

Yes

Are there any user-facing changes?

No

@alamb alamb added the development-process Related to development process of DataFusion label Jan 25, 2023
@github-actions github-actions bot added core Core DataFusion crate sqllogictest SQL Logic Tests (.slt) and removed development-process Related to development process of DataFusion labels Jan 25, 2023
SELECT approx_percentile_cont_with_weight(c3, c2, c1) FROM aggregate_test_100


# csv_query_approx_percentile_cont_with_weight
Copy link
Contributor

@korowa korowa Jan 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like duplicated block, with minor differences like double spaces on line 51 and dot in the end of line 48

Copy link
Contributor

@korowa korowa Jan 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And it also seems that each of these 3 error messages should end with dot, though sqllogictests completely ignores actual error message - even

statement error just some random text here
SELECT approx_percentile_cont_with_weight(c1, c2, 0.95) FROM aggregate_test_100

counts as passed test 🤔

Copy link
Contributor Author

@alamb alamb Jan 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call with duplicated block. Thank you 🙏

As for the error messages, I am not sure what you mean by "though sqllogictests completely ignores actual error message - even". Can you please tell me what you did to observe this behavior?

I tried changing the expected error message like this

diff --git a/datafusion/core/tests/sqllogictests/test_files/aggregate.slt b/datafusion/core/tests/sqllogictests/test_files/aggregate.slt
index 3b527dcd8..6931018da 100644
--- a/datafusion/core/tests/sqllogictests/test_files/aggregate.slt
+++ b/datafusion/core/tests/sqllogictests/test_files/aggregate.slt
@@ -56,7 +56,7 @@ SELECT approx_percentile_cont_with_weight(c3, c2, c1) FROM aggregate_test_100
 
 
 # csv_query_approx_percentile_cont_with_weight
-statement error Error during planning: The function ApproxPercentileContWithWeight does not support inputs of type Utf8.
+statement error some random text
 SELECT approx_percentile_cont_with_weight(c1, c2, 0.95) FROM aggregate_test_100
 
 statement error Error during planning: The weight argument for ApproxPercentileContWithWeight does not support inputs of type Utf8

And then test failed, as I expected:

cargo test --test sqllogictests
....
STORED AS CSV
WITH HEADER ROW
LOCATION '../../testing/data/csv/aggregate_test_100.csv'"
[aggregate.slt] Running query: "SELECT approx_distinct(c9) count_c9, approx_distinct(cast(c9 as varchar)) count_c9_str FROM aggregate_test_100"
[aggregate.slt] Running query: "SELECT approx_percentile_cont_with_weight(c1, c2, 0.95) FROM aggregate_test_100"
[aggregate.slt] Running query: "SELECT approx_percentile_cont_with_weight(c3, c1, 0.95) FROM aggregate_test_100"
[aggregate.slt] Running query: "SELECT approx_percentile_cont_with_weight(c3, c2, c1) FROM aggregate_test_100"
[aggregate.slt] Running query: "SELECT approx_percentile_cont_with_weight(c1, c2, 0.95) FROM aggregate_test_100"
Error: statement is expected to fail with error:
	some random text
but got error:
	DataFusion error: Error during planning: The function ApproxPercentileContWithWeight does not support inputs of type Utf8.
[SQL] SELECT approx_percentile_cont_with_weight(c1, c2, 0.95) FROM aggregate_test_100
at tests/sqllogictests/test_files/aggregate.slt:59

error: test failed, to rerun pass `-p datafusion --test sqllogictests`

🤔

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I should have checked this more careful - it was unsaved version of file.

Now I see that it's ok if expected result is leftmost substring of SQL execution result, so it perfectly explains why missing dot in the end of string doesn't affect test results.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for double checking @korowa

Copy link
Member

@jackwener jackwener left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks @alamb

@jackwener jackwener merged commit be45868 into apache:master Jan 26, 2023
@ursabot
Copy link

ursabot commented Jan 26, 2023

Benchmark runs are scheduled for baseline = a4ebd39 and contender = be45868. be45868 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Skipped ⚠️ Benchmarking of arrow-datafusion-commits is not supported on ec2-t3-xlarge-us-east-2] ec2-t3-xlarge-us-east-2
[Skipped ⚠️ Benchmarking of arrow-datafusion-commits is not supported on test-mac-arm] test-mac-arm
[Skipped ⚠️ Benchmarking of arrow-datafusion-commits is not supported on ursa-i9-9960x] ursa-i9-9960x
[Skipped ⚠️ Benchmarking of arrow-datafusion-commits is not supported on ursa-thinkcentre-m75q] ursa-thinkcentre-m75q
Buildkite builds:
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

@alamb alamb deleted the alamb/port_some_tests branch July 26, 2024 10:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Core DataFusion crate sqllogictest SQL Logic Tests (.slt)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants