catalog.json file does not contain sources with dbt-postgres 1.7.8

[dbeatty10]

Originally posted by @bdtoole in #9119 (comment)

@aranke I know this issue is ~3 months old, but I just encountered this problem with dbt-postgres 1.7.8. I was able to revert to 1.6 and the sources were included in the catalog.json as expected, however moving back to 1.7.8 causes this problem to come back.

If this was resolved at one point within the past few months, could something have undone the fix?

[dbeatty10]

@bdtoole #9493 is one possibility. It touched on the same code as this PR, and it fixed #9161

By any chance, are you using a selector similar to this?

dbt docs generate --select something_other_than_sources

If so, what happens if you add resource_type:source like this?

dbt docs generate --select something_other_than_sources resource_type:source

[bdtoole]

Hey @dbeatty10 - great thought! I'm not using select criteria like that, but I did override the default selector while I build out our new dbt infrastructure:

selectors:
  - name: default_prod
    description: The default production selector
    default: true # Todo for future: Change to include target-aware logic
    definition:
      union:
        - method: fqn # Fully qualified name
          value: '*' # Everything
        - exclude:
            - method: fqn # Fully qualified name
              value: marts
            - method: fqn
              value: staging

When I change the default back to false, sources end up in the catalog as expected. When I keep default as true, sources do not end up in the catalog.

[dbeatty10]

Could you try adding this to your selectors.yml?

        - method: resource_type
          value: source # Include sources

I tried out the following, and dbt docs generate included the sources for me:

selectors.yml

selectors:
  - name: default_prod
    description: The default production selector
    default: true # Todo for future: Change to include target-aware logic
    definition:
      union:
        - method: fqn # Fully qualified name
          value: '*' # Everything
        - method: resource_type
          value: source # Include sources
        - exclude:
            - method: fqn # Fully qualified name
              value: marts
            - method: fqn
              value: staging

[bdtoole]

I added that and it worked! So why is this necessary now in 1.7.x? Is there a bug that's preventing sources from getting pulled back by the fqn + * combination or is this an undocumented feature? 😆

[dbeatty10]

Glad that worked for you! 🎉

This is necessary now in 1.7.x because of #9161 which was fixed by #9493.

I'm guessing that neither sources nor analyses were ever getting selected by the fqn + * combination, so I opened these issues accordingly:

• [Feature] Include sources in dbt list -s "fqn:*" #9692
• [Feature] Include analyses in dbt list -s "fqn:*" #9693

Thanks again for reporting this. I'm going to close this issue in favor of those two.