fix(ingestion): match nested LookML files mentioned in 'include' statements

[jameslamb]

The LookML metadata ingestion logic includes a method, resolve_includes(), whose goal is to turn LookML include statements into valid Python glob patterns. This is used to identify .lkml files to be read in and used to create MCE messages.

I believe that a mistake in that logic leads to files in deeply-nested directories being ignored by datahub ingest.

the LookML

include: "/**/some_cool_view.view.lkml"

is treated as equivalent to

glob.glob(f"{config['base_folder']}/**/some_cool_view.lkml")

But I believe that's incorrect. According to https://docs.looker.com/data-modeling/getting-started/ide-folders#wildcard_examples, ** matches all directories at any level of nesting. Without recursive=True, that glob.glob() call says "match files called some_cool_view.view.lkml exactly one directory below base_folder".

This PR proposes a fix to make this project's Python code match LookML's treatment of wildcards more closely.

Checklist

• The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
• Links to related issues (if applicable)
• Tests for the changes have been added/updated (if applicable)
• Docs related to the changes have been added/updated (if applicable)

Notes for reviewers

I tested this against a DataHub environment I have access to and can confirm that before this change (as of acryl-datahub==0.8.6.1), .view.lkml files nested more than one directory below the base_folder were not found, and after this change they are.

Thanks for your time and consideration.

[hsheth2]

LGTM

Thanks for catching this @jameslamb!

[shirshanka]

LGTM!