Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problem in ingest from MSSQL database with schemas with dots in schema names #13696

Closed
HodaGolgir opened this issue Oct 25, 2023 · 1 comment · Fixed by #14246
Closed

Problem in ingest from MSSQL database with schemas with dots in schema names #13696

HodaGolgir opened this issue Oct 25, 2023 · 1 comment · Fixed by #14246
Assignees
Labels
bug Something isn't working Ingestion P0 Highest priority

Comments

@HodaGolgir
Copy link

Affected module
Ingestion Framework

Describe the bug
Problem in ingest from MSSSQL database with schemas with dots in schema names

To Reproduce

I’ve tried to ingest from MSSSQL database and I have error when it is ingested tables from schema with name like [bof.ams] .
it can not be recognized as schema, I have this error message :

[2023-10-22, 10:56:58 UTC] {common_db_source.py:297} WARNING - Fetching tables names failed for schema bof.ams due to - (pytds.tds_base.OperationalError) Database 'bof' does not exist. Make sure that the name is entered correctly.
[SQL: use bof]
(Background on this error at: https://sqlalche.me/e/14/e3q8)

Expected behavior
Identify schemas with dots in schema names in ingest from MSSSQL database

Version:

  • OS:
  • Python version: 3.8
  • OpenMetadata version:1.1.7
  • OpenMetadata Ingestion package version:openmetadata-ingestion[docker]==1.1.7

Additional context
Reflecting the discussion on Slack:
https://app.slack.com/client/T02BVTLN3G8/later

@pmbrull pmbrull added Ingestion bug Something isn't working labels Oct 25, 2023
@pmbrull
Copy link
Collaborator

pmbrull commented Oct 25, 2023

The main problem comes from the internal logic in decorated functions within the MSSQL SQA dialect.

The function @_db_plus_owner_listing picks up the schema that we send, e.g., inspector.get_table_names(schema) and tries to identify if we are passing a database.schema to "prepare" the inspector to run the queries on the required db.

For the case where our schemas are named as hello.world, then this process wrongly assumes that hello a database, which kills the processing.

We need to find a generic way to handle this behavior

@ayush-shah ayush-shah added the P0 Highest priority label Nov 9, 2023
@ayush-shah ayush-shah changed the title Problem in ingest from MSSSQL database with schemas with dots in schema names Problem in ingest from MSSQL database with schemas with dots in schema names Nov 18, 2023
@harshach harshach moved this to Ingestion - Bugs & Minor Features in Release 1.3.0 Nov 30, 2023
@pmbrull pmbrull moved this from Ingestion - Bugs & Minor Features to Done in Release 1.3.0 Dec 12, 2023
MrVinegar pushed a commit to MrVinegar/OpenMetadata that referenced this issue Dec 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Ingestion P0 Highest priority
Projects
No open projects
Status: Done
Development

Successfully merging a pull request may close this issue.

3 participants