Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

read_sql Error: The function 'PERCENTILE_DISC' must havce an OVER clause. code=10753 #3075

Closed
athensofamerica opened this issue Oct 18, 2024 · 3 comments
Assignees
Labels
bug Something isn't working

Comments

@athensofamerica
Copy link

Describe the bug

When querying SQL server table, we encountered an error.
[ ... ERROR tiberius::tds::stream::token] The function 'PERCENTILE_DISC' must have an OVER clause. code=10753

This doesn't happen for all the table queries, but frequent enough. It seems only happen when read_sql is called with with partition column and number of partitions specified.

This happens on both Linux and Windows system. I was told we can safely ignore this, but it will be nice if it can be fixed. This might be SQL code generation bug for SQL Server (maybe for other database as well). So fix might be pretty easy.

To Reproduce

Create a SQL database table with large number of rows with an integer column. Query the table using read_sql with SELECT * FROM table. Specify the integer column as the partition column, and a number > 1 as the partition number.

Expected behavior

return a dafa dataframe with no error

Component(s)

SQL

Additional context

No response

@athensofamerica athensofamerica added bug Something isn't working needs triage labels Oct 18, 2024
@colin-ho colin-ho self-assigned this Oct 18, 2024
@colin-ho
Copy link
Contributor

On this

colin-ho added a commit that referenced this issue Oct 23, 2024
Addresses: #3075

SQL server requires an `OVER` clause to be specified in percentile
queries (because it's a window function). Read sql uses percentiles to
determine partition bounds.

Adds AzureSqlEdge as a test database. Might as well since a lot of ppl
use us to read sqlserver, and have had bugs with sql server. Kind of a
pain to get it set up since it requires odbc and drivers etc. but it
works. It's also not much of a hit on CI times, installing drivers takes
around ~15s and the extra tests take around 5s.

Additionally made some modifications to some tests and pushdowns, left
comments on the rationale.

---------

Co-authored-by: Colin Ho <colinho@Colins-MacBook-Pro.local>
Co-authored-by: Colin Ho <colinho@Colins-MBP.localdomain>
@colin-ho
Copy link
Contributor

The fix is in the latest release!

@athensofamerica
Copy link
Author

Great, thanks @colin-ho!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants