Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support MERGE with multiple join columns and source subqueries #136

Merged
merged 47 commits into from
Sep 16, 2024

Conversation

tekumara
Copy link
Owner

@tekumara tekumara commented Sep 16, 2024

Supports MERGE with

  • multiple join columns, that may have the same name
  • when the source is a subquery

To do this, I've refactored the way merge works to align with the approach suggested here, ie:

  • use a single merge_candidates temporary table to map rows to mutation operations (delete, update, insert)
  • separate candidate table creation, mutation operations (delete, update, insert), and counts into separate functions, rather than co-mingling this functionality, for better readability
  • use join columns rather than rowids, and avoid using a transaction because duckdb doesn't support nested transactions so the current implementation will error if an existing transaction is active

resolves #24

jsibbison-square and others added 30 commits June 27, 2024 21:06
This commit adds the ability to convert snowflakes [MERGE INTO](https://docs.snowflake.com/en/sql-reference/sql/merge) functionality into a functional equivalent implementation in duckdb.
To do this we need to break apart the WHEN [NOT] MATCHED syntax into separate statements to be executed indepedently.

This commit only adds the transform, there is more refactoring required in fakes.py in order to handle a transform
that transforms a single expression into multiple expressions.
* main:
  chore(main): release 0.9.20 (#115)
  fix: $$  not considered a variable
  fix: concurrent connection write-write conflict
  refactor: extract test_connect.py
  feat: SHOW PRIMARY KEYS for table (#114)
  chore(main): release 0.9.19 (#113)
  feat: Implements basic snowflake session variables via SET/UNSET (#111)
  chore(deps-dev): bump pyright from 1.1.366 to 1.1.369 (#112)
  chore(main): release 0.9.18 (#110)
Got basic case for merge working with a merge2 function, needs to be cleaned up, deleting original merge, fix result statement.
* main:
  fix: Only set variables for SetItem expressions (#116)
* main:
  feat: alter table cluster by
  chore: cruft update
  chore: bump sqlglot 25.5.1
* main: (38 commits)
  feat(server): support time & timestamp types
  ci(server): description - cover more types
  feat(server): handle snowflake ProgrammingError
  refactor: use rowtype for sf metadata in arrow schema
  feat(server): support empty result set
  chore(main): release 0.9.24 (#128)
  fix: don't require pandas at import time
  chore(main): release 0.9.23 (#126)
  feat: support conn.is_closed()
  refactor: extract conn, cursor, pandas_tools
  refactor: describe_as_rowtype uses sf_type
  refactor: describe_as_rowtype
  refactor: extract describe_as_result_metadata
  feat(server): support cur.description
  feat(server): support bool, int, float types
  chore(main): release 0.9.22 (#124)
  wip(server): add /queries/v1/abort-request
  fix: fetchmany supports irregular sizes
  fix: column types for DESCRIBE
  feat: describe view information_schema.*
  ...
* main:
  chore(deps): update ruff requirement from ~=0.5.1 to ~=0.6.3 (#130)
  chore(deps-dev): bump pyright from 1.1.374 to 1.1.378 (#133)
@tekumara tekumara merged commit 9b5a7a0 into main Sep 16, 2024
1 check passed
@tekumara tekumara deleted the merge-more branch September 16, 2024 10:59
tekumara pushed a commit that referenced this pull request Sep 16, 2024
🤖 I have created a release *beep* *boop*
---


##
[0.9.25](v0.9.24...v0.9.25)
(2024-09-16)


### Features

* Adds MERGE INTO transform
([#109](#109))
([d5e14a7](d5e14a7))
* close duckdb connection
([223f8e2](223f8e2))
* **server:** handle snowflake ProgrammingError
([9455a43](9455a43))
* **server:** support empty result set
([b967b69](b967b69))
* **server:** support FAKESNOW_DB_PATH
([af79f77](af79f77))
* **server:** support time & timestamp types
([1606a3e](1606a3e))
* support MERGE with multiple join columns and source subqueries
([#136](#136))
([9b5a7a0](9b5a7a0)),
closes [#24](#24)


### Chores

* **deps-dev:** bump pyright from 1.1.374 to 1.1.378
([#133](#133))
([593a420](593a420))
* **deps:** update ruff requirement from ~=0.5.1 to ~=0.6.3
([#130](#130))
([6b37d8b](6b37d8b))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: potatobot-prime[bot] <132267321+potatobot-prime[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Merge upsert support?
2 participants