feat(integration/fivetran): Fivetran source ingestion integration #14
Conversation
| @@ -505,6 +506,7 @@ def get_long_description(): | |||
| "nifi", | |||
| "vertica", | |||
| "mode", | |||
| "fivetran", | |||
There was a problem hiding this comment.
Now we added one unit test case on fivetran source
|
|
||
| class SnowflakeDestinationConfig(BaseSnowflakeConfig): | ||
| database: str = Field( | ||
| default=None, description="The fivetran connector log database." |
There was a problem hiding this comment.
Is there any default for the database?
There was a problem hiding this comment.
No
Even for log_schema config there is no default hence removing default value
| default=None, | ||
| description="The destination platform where fivetran connector log tables are dumped.", | ||
| ) | ||
| snowflake_destination_config: Optional[SnowflakeDestinationConfig] = pydantic.Field( |
There was a problem hiding this comment.
let's rename it to destination_config
There was a problem hiding this comment.
But this config is specific to snowflake destination only.
There was a problem hiding this comment.
type is sufficient to understand SnowflakeDestinationConfig, later we can change it to Union[SnowflakeDestinationConfig|BigQueryDestinationConfig], so let's rename it to destination_config
metadata-ingestion/src/datahub/ingestion/source/fivetran/config.py
Outdated
Show resolved
Hide resolved
metadata-ingestion/src/datahub/ingestion/source/fivetran/config.py
Outdated
Show resolved
Hide resolved
metadata-ingestion/src/datahub/ingestion/source/fivetran/fivetran.py
Outdated
Show resolved
Hide resolved
metadata-ingestion/src/datahub/ingestion/source/fivetran/fivetran_schema.py
Outdated
Show resolved
Hide resolved
shubhamjagtap639
changed the title
| default=None, | ||
| description="The destination platform where fivetran connector log tables are dumped.", | ||
| ) | ||
| snowflake_destination_config: Optional[SnowflakeDestinationConfig] = pydantic.Field( |
There was a problem hiding this comment.
type is sufficient to understand SnowflakeDestinationConfig, later we can change it to Union[SnowflakeDestinationConfig|BigQueryDestinationConfig], so let's rename it to destination_config
…into fivetran-connector-integration
…into fivetran-connector-integration
|
@shubhamjagtap639 can we close this PR? |
Yes, I am closing this PR |
Checklist