Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: port Meltano catalog manipulation conventions to SDK config.json #1383

Open
kgpayne opened this issue Feb 2, 2023 · 3 comments
Open
Labels

Comments

@kgpayne
Copy link
Contributor

kgpayne commented Feb 2, 2023

Feature scope

Configuration (settings parsing, validation, etc.)

Description

In working on the SDK testing framework, it has become apparent that Meltanos' catalog manipulations (i.e. selection, metadata injection and schema overrides) would be very useful within the SDK config dictionary itself, so that tests can take advantage of these shorthands.

Use cases:

  • Selecting specific streams to test with (e.g. from a database containing many schema and tables).
  • Test streams in FULL_TABLE and INCREMENTAL modes, by setting metadata.
  • Override the schema to test behaviours when columns are added/removed or types change.

These things are possible by manually curating many catalog.json files, but this is labour intensive and repetitive work for Tap/Target developers.

@kgpayne kgpayne added kind/Feature New feature or request valuestream/SDK labels Feb 2, 2023
@stale
Copy link

stale bot commented Jul 18, 2023

This has been marked as stale because it is unassigned, and has not had recent activity. It will be closed after 21 days if no further activity occurs. If this should never go stale, please add the evergreen label, or request that it be added.

@stale stale bot added the stale label Jul 18, 2023
@stale stale bot closed this as completed Aug 8, 2023
@edgarrmondragon
Copy link
Collaborator

Still relevant

@stale stale bot removed the stale label Aug 8, 2023
Copy link

stale bot commented Aug 7, 2024

This has been marked as stale because it is unassigned, and has not had recent activity. It will be closed after 21 days if no further activity occurs. If this should never go stale, please add the evergreen label, or request that it be added.

@stale stale bot added the stale label Aug 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants