Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat!: transpile map retrieval to duckdb, transpile TRY_ELEMENT_AT #3277

Merged
merged 1 commit into from
Apr 5, 2024

Conversation

georgesittas
Copy link
Collaborator

@georgesittas georgesittas commented Apr 4, 2024

DuckDB says the following for retrieving values from maps:

MAPs use bracket notation for retrieving values. Selecting from a MAP returns a LIST rather than an individual value, with an empty LIST meaning that the key was not found.

This makes it tricky to transpile such retrievals to / from it, because we need to either unwrap / wrap the retrieved value, respectively.

This PR:

  • implements a best-effort approach to handle the -> DuckDB direction of the transpilation
  • facilitates the transpilation of Spark's TRY_ELEMENT_AT function

One side effect of this change is that when constructing Bracket nodes for DuckDB manually, it will also be required to set returns_list_for_maps to True, otherwise we'll generate incorrect SQL when targeting DuckDB.

References:

@georgesittas georgesittas force-pushed the jo/transpile_map_lookup_to_duckdb branch from cec4a31 to 19087a4 Compare April 4, 2024 20:43
@georgesittas georgesittas changed the title Feat!: transpile map retrieval to duckdb, Feat!: transpile map retrieval to duckdb, transpile TRY_ELEMENT_AT Apr 4, 2024
@georgesittas georgesittas merged commit 6bba030 into main Apr 5, 2024
5 checks passed
@georgesittas georgesittas deleted the jo/transpile_map_lookup_to_duckdb branch April 5, 2024 11:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants