dbt docs generate does not work with glue catalog with large number of tables in schema

[lguyaux]

Describe the bug

Currently when attempting to run dbt docs generate you get a failure message if you have a large number of tables in any given schema. This is because the package will attempt to run the following code show table extended in <schema_name> like '<table_1>|<table_2>|...'.

Steps To Reproduce

Use glue as your metastore
Have a large number of tables in a given schema
Attempt to run dbt docs generate

Expected behavior

dbt docs generate to work regardless of how many tables you have in a given schema.

Screenshots and log output

09:40:38 Running with dbt=1.8.3
09:40:39 Registered adapter: databricks=1.8.3
09:40:40 Found 161 models, 4 data tests, 117 sources, 1142 macros

09:41:32 Concurrency: 4 threads (target='dev')

09:41:34 Building catalog
09:41:35 Encountered an error while generating catalog: Runtime Error
Runtime Error
org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:1 validation error detected: Value 'table1|table2|table3|...' at 'expression' failed to satisfy constraint: Member must have length less than or equal to 2048 (Service: AWSGlue; Status Code: 400; Error Code: ValidationException; Request ID: 0ed28cc6-16cf-4962-9d60-ddcf2fe8ba82; Proxy: null))

System information

The output of dbt --version:

Core:
  - installed: 1.8.3
  - latest:    1.8.3 - Up to date!

Plugins:
  - databricks: 1.8.3 - Up to date!
  - spark:      1.8.0 - Up to date!

The operating system you're using:
macOS Sonoma 14.5

The output of python --version:
Python 3.9.6

Additional context

This issue is a replica of this already fixed issue that seems not to work : #325

[benc-db]

Thanks for filing. We had fixed this before, but I guess something I changed recently regressed it.

[benc-db]

Do you know when this issue reappeared? I don't have a glue setup to test against, so I could use some help tracking down the issue.

[benc-db]

Ah, have you tried setting DBT_DESCRIBE_TABLE_2048_CHAR_BYPASS=true?

[lguyaux]

Ah, have you tried setting DBT_DESCRIBE_TABLE_2048_CHAR_BYPASS=true?

Thank you very much @benc-db, it worked with this parameter !