Schema Pattern Option in Athena Recipe not ingesting any metadata

[vravi-li]

Describe the bug
Using the schema pattern option in Athena recipe does not ingest any metadata from the underlying tables. I am trying to ingest multiple databases from Athena. The only way to do this right now is to put multiple database names in database option in the recipe, and I have more than 100 tables. There is no option of database pattern in the ingestion recipe unfortunately.

To Reproduce
Steps to reproduce the behavior:
Try ingesting data with the attached recipes.

Recipe #1

source:
    type: athena
    config:
        aws_region: us-east-1
        work_group: primary
        database: auto_aggregation
        query_result_location: 's3://......'

Recipe #2

source:
    type: athena
    config:
        aws_region: us-east-1
        work_group: primary
        schema_pattern:
            allow:
                - '^auto_*'
        query_result_location: 's3://......'

Expected behavior
The metadata is ingested properly.

Screenshots

Desktop (please complete the following information):

• OS: MAC
• Browser: Chrome

Additional context
Add any other context about the problem here.

[github-actions]

This issue is stale because it has been open for 30 days with no activity. If you believe this is still an issue on the latest DataHub release please leave a comment with the version that you tested it with. If this is a question/discussion please head to https://slack.datahubproject.io. For feature requests please use https://feature-requests.datahubproject.io

[github-actions]

This issue was closed because it has been inactive for 30 days since being marked as stale.