Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unity exporter support for Azure #7551

Closed
N-o-Z opened this issue Mar 12, 2024 · 0 comments · Fixed by #7554
Closed

Unity exporter support for Azure #7551

N-o-Z opened this issue Mar 12, 2024 · 0 comments · Fixed by #7554
Assignees
Labels
bug Something isn't working export-hooks

Comments

@N-o-Z
Copy link
Member

N-o-Z commented Mar 12, 2024

Error:

Error: runtime error: lakefs/catalogexport/unity_exporter.lua:54: external table "prm_maestro_cliente_gt" creation failed: FAILED: BAD_REQUEST [UC_FILE_SCHEME_FOR_TABLE_CREATION_NOT_SUPPORTED] Creating table in Unity Catalog with file scheme https is not supported.
Instead, please create a federated data source connection using the CREATE CONNECTION command for the same table provider, then create a catalog based on the connection with a CREATE FOREIGN CATALOG command to reference the tables therein.

Trying to create an external location in databricks using https scheme results in the following error:

ErrorClass=INVALID_PARAMETER_VALUE.UNSUPPORTED_LOCATION_SCHEME] url has invalid URI scheme https. Valid URI schemes include s3, gs, s3n, r2, s3a, abfss

lakeFS is currently working with https scheme for Azure, and therefore when providing the table path we fail to register the table.

Note that according to the above error message external locations are supported only in ADLS gen2 (no wasb scheme support)

One solution is to convert the https scheme into an abfss one in the unity exporter before registering the table.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working export-hooks
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant