Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FR] Schema name appended with table name in tableName property of configuration.json file #13

Open
nihxdr opened this issue Sep 25, 2024 · 2 comments

Comments

@nihxdr
Copy link
Contributor

nihxdr commented Sep 25, 2024

Including the schema name in tableName property is restrictive and seems unnecessary, since it won't allow us to use the same metadata files generated for different databases with the same schema.

concat(tables.TABLE_SCHEMA, '.', tables.TABLE_NAME) AS TABLE_NAME,

@nihxdr
Copy link
Contributor Author

nihxdr commented Sep 25, 2024

I was able to run MySQL queries after following these steps:

  • introspect the database
  • remove all the <schema_name>. appended to tableName
  • run connector-link update command
  • add required models
  • build local and docker compose build and up

We would have the metadata file names as such: TableName, instead of SchemaNameTableName

here are the before and after of the SQL queries shown in the logs of the data connector:
before:

select json_object('rows', (
  select json_arrayagg(json_object('email', email))
  from (
    select <schema_name>.<table_name>.email as email
    from <schema_name>.<table_name>
    group by <schema_name>.<table_name>.email
  ) as <alias>
))

after:

select json_object('rows', (
  select json_arrayagg(json_object('email', email))
  from (
    select <table_name>.email as email
    from <table_name>
    group by <table_name>.email
  ) as <alias>
))

But we may have to check for and resolve the conflicting metadata files generated from multiple data sources (or instead of removing the schema name it should be taken from env? )

@GavinRay97
Copy link
Member

Hmm, I suppose there's no reason why a flag couldn't exist to omit schema name.

For some historical context, all DB connectors (MySQL/Oracle etc) used to run as one service, and you could add any type of database, or even multiple connections to the same database type.

For example, /api/v1/mysql/query, /api/v1/oracle/query

I'm not 100% sure if we still need the schema name in MySQL connector.

I have not tried using a JDBC URL with MySQL which doesn't specify a database. Since in that case you would need <db/schema_name>.<table_name> to disambiguate.

@GavinRay97 GavinRay97 changed the title Schema name appended with table name in tableName property of configuration.json file [FR] Schema name appended with table name in tableName property of configuration.json file Nov 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants