Skip to content

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

query execution slow, unable to enable extensions or query parquet #177

Closed
thebuck25 opened this issue Sep 13, 2024 · 3 comments · Fixed by #212
Closed

query execution slow, unable to enable extensions or query parquet #177

thebuck25 opened this issue Sep 13, 2024 · 3 comments · Fixed by #212

Comments

@thebuck25
Copy link

No change in query performance directly against tables (Postgres: 2.1s, pg_duckdb: 2.1s, Mac M1 DuckDB: 0.025s) which leads me to believe it's falling back to the pgsql engine. Also unable to query parquet files, enable extensions, etc:

CREATE EXTENSION pg_duckdb;

WARNING:  To actually execute queries using DuckDB you need to run "SET duckdb.execution TO true;"
CREATE EXTENSION

Query returned successfully in 108 msec.

SET duckdb.execution TO true;

SET

Query returned successfully in 63 msec.

SELECT duckdb.enable_extension('iceberg');

ERROR:  function duckdb.enable_extension(unknown) does not exist
LINE 1: SELECT duckdb.enable_extension('iceberg');
               ^
HINT:  No function matches the given name and argument types. You might need to add explicit type casts. 

SQL state: 42883
Character: 8

SELECT SUM('DISTANCE') AS total
FROM read_parquet('r2://1101503dfsadfb........../test.parquet)

ERROR:  a column definition list is required for functions returning "record"
LINE 2: FROM read_parquet('r2://1101503dfsadfb......83479.r2....
             ^ 

SQL state: 42601
Character: 38

Postgres 16.3 Citus Cluster (Stackgres Operator) 1 Coordinator, 1 Shard with replicas - running in an Azure AKS x64 node
Compiled from source using custom docker image for Stackgres' Ubuntu containers running patroni
(Stackgres uses an older kernel w/ glibc 2.28)

Build image w/ make output is available here: https://hub.docker.com/repository/docker/killinterpol/quackbuster/general.

Let me know if you need more info, if I can troubleshoot further, run any available tests or otherwise contribute...

@wuputah
Copy link
Collaborator

wuputah commented Sep 14, 2024

Pardon my brevity, I'm answering from my phone. Hopefully this is enough to get you started:

  • duckdb.install_extension
  • read_parquet needs an AS statement

@mike-luabase
Copy link

No change in query performance directly against tables

@wuputah maybe your postgres table had an unsupported datatype?

The following data types are supported: numeric, character, binary, date/time, boolean, uuid, json, and arrays.

@thebuck25
Copy link
Author

thebuck25 commented Sep 16, 2024

@wuputah thanks - install_extension returns false regardless of the extension I try to install:

WARNING:  (duckdb_install_extension) HTTP Error: Failed to download extension "iceberg" at URL "http://extensions.duckdb.org/17d598fc44/linux_amd64/iceberg.duckdb_extension.gz" (HTTP 403)
Extension "iceberg" is an existing extension.

Are you using a development build? In this case, extensions might not (yet) be uploaded.

Successfully run. Total query runtime: 291 msec.
1 rows affected.

Also, do you want me to do a PR to fix the readme?
Enable the DuckDB Iceberg extension using SELECT duckdb.enable_extension('iceberg') and read Iceberg files with iceberg_scan.

Was able to query parquet after adding the second AS (missed that in README and I've never needed it using httpfs) however GROUP BY year() fails still. Not sure if date functions are supported yet.

@mike-luabase - the table in question contains only - uuid, numeric, character varying (255), timestamp w/o time zone. Lmk if you want my table def & query, or is the varchar the issue?

@duckdb duckdb locked and limited conversation to collaborators Sep 16, 2024
@wuputah wuputah converted this issue into discussion #181 Sep 16, 2024
JelteF added a commit that referenced this issue Sep 24, 2024
The readme was using an outdated/incorrect name for the
`install_extension` function. This corrects that.

Fixes #209
Partially fixes #177

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants