Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Set duckdb search_path according to postgres #1465

Closed
wants to merge 3 commits into from

Conversation

kysshsy
Copy link
Contributor

@kysshsy kysshsy commented Aug 4, 2024

Ticket(s) Closed

What

Why

How

set Duckdb search path according to postgres search path.

  1. get search path of postgres(schema name)
  2. check if search path is invalid in duckdb.
  3. set serach path in duckdb

Tests

@@ -78,9 +78,6 @@ pub trait BaseFdw {
handler,
)?;

// Ensure we are in the same DuckDB schema as the Postgres schema
connection::execute(format!("SET SCHEMA '{schema_name}'").as_str(), [])?;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If my understanding is correct, the code here is to set the duckdb schema for the exector hook. The code here can be replaced by the set_search_path.

@kysshsy
Copy link
Contributor Author

kysshsy commented Aug 4, 2024

This problem only happens in the executor hook. I couldn't find a good way to test it.

@kysshsy kysshsy changed the title fix: set duckdb search_path according to postgres fix: Set duckdb search_path according to postgres Aug 4, 2024
@joinbase0
Copy link

From big picture, this PR still has potential problem. You actually want to implicitly expand the sync scope of one kind of metas, search path, between pg and pg_lakehouse(duckdb). This may cause problem. Because two sides in pg and pg_lakehouse(duckdb) may have independent and different schemas, the sync of search path from pg to pg_lakehouse may include unwanted schemas in pg_lakehouse, and even worse results in mistakes for unwanted table including.

And further intentions to implicitly expand the sync scope of any meta may face this problem.

There are actually several ways to solve this problem. Silently setting the search path is simple, but, IMHO, it is not the good way considering its destructive consequences, especially for enterprise level apps.

A good solution needs to be analyzed and determined from a global perspective. It is great to see you issues several PRs recently in supporting prepare stmts pushdown. My suggestion is that you can put all the problems you have observed into one issue (new or original paradedb/pg_analytics#51), and we can have discussions to see what commonalities and characteristics these problems have, which ones can be easily solved first, which ones require changes to the existing foundation, and even finally where pg_lakehouse will end up.

@kysshsy
Copy link
Contributor Author

kysshsy commented Aug 6, 2024

Paradedb maintainers are busy with refactoring and migrating repositories, I guess. After the migration is complete, I'll send over the issues I've observed so far. However, this PR actually addresses an existing bug and isn't solely for 'prepare stmt' pushdown.

@philippemnoel
Copy link
Collaborator

Paradedb maintainers are busy with refactoring and migrating repositories, I guess. After the migration is complete, I'll send over the issues I've observed so far. However, this PR actually addresses an existing bug and isn't solely for 'prepare stmt' pushdown.

Appreciate you working on this PR :) You're right, we're about to push version 0.9.0 and are fixing/refactoring a few things, but we very much value your PR and will get to reviewing it in a few days

@kysshsy
Copy link
Contributor Author

kysshsy commented Aug 8, 2024

closed. move to paradedb/pg_analytics#40

@kysshsy kysshsy closed this Aug 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Queries without schema specification fail to fully push down in DuckDB
3 participants