Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Misleading error when using read_xxx functions with invalid URLs #106

Closed
Y-- opened this issue Aug 8, 2024 · 1 comment · Fixed by #277
Closed

Misleading error when using read_xxx functions with invalid URLs #106

Y-- opened this issue Aug 8, 2024 · 1 comment · Fixed by #277
Labels
enhancement New feature or request
Milestone
0.2.0

Comments

@Y--
Copy link
Collaborator

Y-- commented Aug 8, 2024

I get ERROR: Function `read_csv(TEXT)` only works with Duckdb execution. when using read_csv (along with the actual error as a warning):

y=# SELECT * from read_csv('https://www.data.gouv.fr/fr/datasets/r/6d186965-f41b-41f3-9b23-88241cc6890c') as (n int);
WARNING:  (DuckDB) HTTP Error: Unable to connect to URL "https://www.data.gouv.fr/fr/datasets/r/6d186965-f41b-41f3-9b23-88241cc6890c": 404 (Not Found)
ERROR:  Function `read_csv(TEXT)` only works with Duckdb execution.
CONTEXT:  PL/pgSQL function read_csv(text) line 3 at RAISE

I would expect the WARNING to be the ERROR and the ERROR to not show.
@JelteF
Copy link
Collaborator

JelteF commented Aug 19, 2024

Once we merge #123 we can use/cache the NeedsDuckdbExecution function and fail with an ERROR instead of WARNING when execution in DuckDB fails. i.e. for queries where NeedsDuckdbExecution returns true there's no point in retrying the query in postgres.

@JelteF JelteF added the enhancement New feature or request label Aug 19, 2024
@JelteF JelteF added this to the 0.2.0 milestone Sep 30, 2024
JelteF added a commit that referenced this issue Oct 13, 2024
@JelteF
Improve error when read_xxx functions fail
8802c3e 
Previously the error would be shown as a WARNING and then the query
would be run as a fallback using the Postgres executor, which would fail
because the DuckDB `read_csv`/`read_parquet` functions cannot be
executed by the Postgres executor. This is quite confusing as many
people only look at the last ERROR, and not the warnings before.

This is also relevant for MotherDuck support, since the MotherDuck
backed tables can only be read using the DuckDB executor.

Fixes #106
@JelteF JelteF mentioned this issue Oct 13, 2024
Merged
JelteF added a commit that referenced this issue Oct 13, 2024
@JelteF
Improve error when read_xxx functions fail
f7c770d 
Previously the error would be shown as a WARNING and then the query
would be run as a fallback using the Postgres executor, which would fail
because the DuckDB `read_csv`/`read_parquet` functions cannot be
executed by the Postgres executor. This is quite confusing as many
people only look at the last ERROR, and not the warnings before.

This is also relevant for MotherDuck support, since the MotherDuck
backed tables can only be read using the DuckDB executor.

Fixes #106
JelteF added a commit that referenced this issue Oct 14, 2024
@JelteF
Improve error when read_xxx functions fail (#277)
d07a5de 
Previously the error would be shown as a WARNING and then the query
would be run as a fallback using the Postgres executor, which would fail
because the DuckDB `read_csv`/`read_parquet` functions cannot be
executed by the Postgres executor. This is quite confusing as many
people only look at the last ERROR, and not the warnings before.

This is also relevant for MotherDuck support, since the MotherDuck
backed tables can only be read using the DuckDB executor.

Fixes #106
Related to #209 (the error there looks terribly confusing)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
0.2.0
Development

Successfully merging a pull request may close this issue.

Improve error when read_xxx functions fail duckdb/pg_duckdb
2 participants
@Y-- @JelteF