Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Superset ingestion failure when comma are in a table name #9655

Closed
rinzool opened this issue Jan 18, 2024 · 1 comment
Closed

Superset ingestion failure when comma are in a table name #9655

rinzool opened this issue Jan 18, 2024 · 1 comment
Labels
bug Bug report

Comments

@rinzool
Copy link
Contributor

rinzool commented Jan 18, 2024

Describe the bug
Hi, for superset ingestion we can have error when a dataset has a comma , in its name. In that case we have this error:

failed to write record with workunit urn:li:chart:(superset,***) with ('Unable to emit metadata to DataHub GMS: Invalid format for aspect: chart\n Cause: ERROR :: /inputs/0/string :: "Provided urn urn:li:dataset:(urn:li:dataPlatform:athena,athena.some_db.My,virtual,table,PROD)" is invalid: Failed to convert urn to entity key: urns parts and key fields do not have same length for urn:li:dataset:(urn:li:dataPlatform:athena,athena,athena.some_db.My,virtual,table,PROD)\n', {'exceptionClass': 'com.linkedin.restli.server.RestLiServiceException', 'message': 'Invalid format for aspect: chart\n Cause: ERROR :: /inputs/0/string :: "Provided urn urn:li:dataset:(urn:li:dataPlatform:athena,athena,athena.some_db.My,virtual,table,PROD)" is invalid: Failed to convert urn to entity key: urns parts and key fields do not have same length for urn:li:dataset:(urn:li:dataPlatform:athena,athena,athena.some_db.My,virtual,table,PROD)\n', 'status': 400, 'id': 'urn:li:chart:(superset,11399)'}) and info {'exceptionClass': 'com.linkedin.restli.server.RestLiServiceException', 'message': 'Invalid format for aspect: chart\n Cause: ERROR :: /inputs/0/string :: "Provided urn urn:li:dataset:(urn:li:dataPlatform:athena,athena,athena.some_db.My,virtual,table,PROD)" is invalid: Failed to convert urn to entity key: urns parts and key fields do not have same length for urn:li:dataset:(urn:li:dataPlatform:athena,athena,athena.some_db.My,virtual,table,PROD)\n', 'status': 400, 'id': 'urn:li:chart:(superset,***)'}

To Reproduce
Steps to reproduce the behavior:

  1. Go to Superset
  2. Create a virtual table with a comma in its name
  3. Run a superset ingestion

Expected behavior
We should not have an error, at least a warning or a workaround.

@rinzool rinzool added the bug Bug report label Jan 18, 2024
@rinzool
Copy link
Contributor Author

rinzool commented Jan 18, 2024

I am planning to open a PR about that with the following behavior:

  • Log a warning if a comma is in a table name here
  • Add a new input parameter like reserved_character_replacement so we can automatically replace , with a replacement characters

@rinzool rinzool changed the title A short description of the bug Superset ingestion failure when comma are in a table name Jan 18, 2024
@rinzool rinzool closed this as completed Jan 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Bug report
Projects
None yet
Development

No branches or pull requests

1 participant