-
Notifications
You must be signed in to change notification settings - Fork 4.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remove supportsDBT
and supportsNormalization
spec flags
#20854
Closed
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…DBT and normalizationImage fields. Added to the GenerateInputActivityImpl and TemporalClient classes code parts for read destination_definition.yaml and get suportDBT and normalizationImage fields. Added logging and comparing normalization images from NormalizationRunnerFactory and destination_definition.yaml
…korotkov/remove_factory � Conflicts: � airbyte-commons-worker/src/main/java/io/airbyte/workers/normalization/NormalizationRunnerFactory.java
…korotkov/remove_factory
…remove_factory � Conflicts: � airbyte-commons-worker/src/main/java/io/airbyte/workers/normalization/NormalizationRunnerFactory.java � airbyte-commons-worker/src/test/java/io/airbyte/workers/normalization/NormalizationRunnerFactoryTest.java � airbyte-container-orchestrator/src/main/java/io/airbyte/container_orchestrator/DbtJobOrchestrator.java � airbyte-container-orchestrator/src/main/java/io/airbyte/container_orchestrator/NormalizationJobOrchestrator.java � airbyte-integrations/bases/standard-destination-test/src/main/java/io/airbyte/integrations/standardtest/destination/DestinationAcceptanceTest.java � airbyte-workers/src/main/java/io/airbyte/workers/temporal/sync/DbtTransformationActivityImpl.java � airbyte-workers/src/main/java/io/airbyte/workers/temporal/sync/NormalizationActivityImpl.java
…ctor_definition table
…remove_spec_flags
Affected Connector ReportNOTE
|
Connector | Version | Changelog | Publish |
---|---|---|---|
source-bigquery |
0.2.3 |
✅ | ✅ |
source-kafka |
0.2.3 |
✅ | ✅ |
- See "Actionable Items" below for how to resolve warnings and errors.
❌ Destinations (47)
Connector | Version | Changelog | Publish |
---|---|---|---|
destination-aws-datalake |
0.1.1 |
✅ | ✅ |
destination-azure-blob-storage |
0.1.6 |
✅ | ✅ |
destination-bigquery |
1.2.9 |
✅ | ✅ |
destination-bigquery-denormalized |
1.2.9 |
✅ | ✅ |
destination-cassandra |
0.1.4 |
✅ | ✅ |
destination-clickhouse |
0.2.1 |
✅ | ✅ |
destination-clickhouse-strict-encrypt |
0.2.1 |
🔵 (ignored) |
🔵 (ignored) |
destination-csv |
0.2.10 |
✅ | ✅ |
destination-databricks |
0.3.1 |
✅ | ✅ |
destination-dev-null |
0.2.7 |
🔵 (ignored) |
🔵 (ignored) |
destination-doris |
0.1.0 |
✅ | ✅ |
destination-dynamodb |
0.1.7 |
✅ | ✅ |
destination-e2e-test |
0.2.4 |
✅ | ✅ |
destination-elasticsearch |
0.1.6 |
✅ | ✅ |
destination-elasticsearch-strict-encrypt |
0.1.6 |
🔵 (ignored) |
🔵 (ignored) |
destination-gcs |
0.2.12 |
✅ | ✅ |
destination-iceberg |
0.1.0 |
✅ | ✅ |
destination-jdbc |
0.3.14 |
🔵 (ignored) |
🔵 (ignored) |
destination-kafka |
0.1.10 |
✅ | ✅ |
destination-keen |
0.2.4 |
✅ | ✅ |
destination-kinesis |
0.1.5 |
✅ | ✅ |
destination-local-json |
0.2.11 |
✅ | ✅ |
destination-mariadb-columnstore |
0.1.7 |
✅ | ✅ |
destination-mongodb |
0.1.9 |
✅ | ✅ |
destination-mongodb-strict-encrypt |
0.1.9 |
🔵 (ignored) |
🔵 (ignored) |
destination-mqtt |
0.1.3 |
✅ | ✅ |
destination-mssql |
0.1.22 |
✅ | ✅ |
destination-mssql-strict-encrypt |
0.1.22 |
🔵 (ignored) |
🔵 (ignored) |
destination-mysql |
0.1.20 |
✅ | ✅ |
destination-mysql-strict-encrypt |
❌ 0.1.21 (mismatch: 0.1.20 ) |
🔵 (ignored) |
🔵 (ignored) |
destination-oracle |
0.1.19 |
✅ | ✅ |
destination-oracle-strict-encrypt |
0.1.19 |
🔵 (ignored) |
🔵 (ignored) |
destination-postgres |
0.3.26 |
✅ | ✅ |
destination-postgres-strict-encrypt |
0.3.26 |
🔵 (ignored) |
🔵 (ignored) |
destination-pubsub |
0.2.0 |
✅ | ✅ |
destination-pulsar |
0.1.3 |
✅ | ✅ |
destination-r2 |
0.1.0 |
✅ | ✅ |
destination-redis |
0.1.4 |
✅ | ✅ |
destination-redpanda |
0.1.0 |
✅ | ✅ |
destination-redshift |
0.3.51 |
✅ | ✅ |
destination-rockset |
0.1.4 |
✅ | ✅ |
destination-s3 |
0.3.18 |
✅ | ✅ |
destination-s3-glue |
0.1.1 |
✅ | ✅ |
destination-scylla |
0.1.3 |
✅ | ✅ |
destination-snowflake |
0.4.40 |
✅ | ✅ |
destination-tidb |
0.1.0 |
✅ | ✅ |
destination-yugabytedb |
0.1.0 |
✅ | ✅ |
- See "Actionable Items" below for how to resolve warnings and errors.
👀 Other Modules (1)
- base-normalization
Actionable Items
(click to expand)
Category | Status | Actionable Item |
---|---|---|
Version | ❌ mismatch |
The version of the connector is different from its normal variant. Please bump the version of the connector. |
⚠ doc not found |
The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like source-jdbc is not published or documented). Please double-check to make sure that it is not a bug. |
|
Changelog | ⚠ doc not found |
The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like source-jdbc is not published or documented). Please double-check to make sure that it is not a bug. |
❌ changelog missing |
There is no chnagelog for the current version of the connector. If you are the author of the current version, please add a changelog. | |
Publish | ⚠ not in seed |
The connector is not in the seed file (e.g. source_definitions.yaml ), so its publication status cannot be checked. This can be normal (e.g. some connectors are cloud-specific, and only listed in the cloud seed file). Please double-check to make sure that it is not a bug. |
❌ diff seed version |
The connector exists in the seed file, but the latest version is not listed there. This usually means that the latest version is not published. Please use the /publish command to publish the latest version. |
andriikorotkov
temporarily deployed
to
more-secrets
December 23, 2022 08:29 — with
GitHub Actions
Inactive
andriikorotkov
temporarily deployed
to
more-secrets
December 23, 2022 08:30 — with
GitHub Actions
Inactive
andriikorotkov
temporarily deployed
to
more-secrets
December 23, 2022 08:41 — with
GitHub Actions
Inactive
andriikorotkov
temporarily deployed
to
more-secrets
December 23, 2022 08:42 — with
GitHub Actions
Inactive
octavia-squidington-iv
added
the
area/documentation
Improvements or additions to documentation
label
Dec 23, 2022
andriikorotkov
temporarily deployed
to
more-secrets
December 23, 2022 14:15 — with
GitHub Actions
Inactive
andriikorotkov
temporarily deployed
to
more-secrets
December 23, 2022 14:15 — with
GitHub Actions
Inactive
evantahler
changed the title
Remove spec flags
Remove Jan 3, 2023
supportsDBT
and supportsNormalization
spec flags
Closing in favor of #21005 by @alafanechere |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
area/api
Related to the api
area/connectors
Connector related issues
area/documentation
Improvements or additions to documentation
area/frontend
Related to the Airbyte webapp
area/platform
issues related to the platform
area/server
connectors/destination/amazon-sqs
connectors/destination/aws-datalake
connectors/destination/azure-blob-storage
connectors/destination/bigquery
connectors/destination/bigquery-denormalized
connectors/destination/cassandra
connectors/destination/clickhouse
connectors/destination/clickhouse-strict-encrypt
connectors/destination/csv
connectors/destination/databricks
connectors/destination/dev-null
connectors/destination/doris
connectors/destination/dynamodb
connectors/destination/e2e-test
connectors/destination/elasticsearch
connectors/destination/elasticsearch-strict-encryp
connectors/destination/firebolt
connectors/destination/firestore
connectors/destination/gcs
connectors/destination/google-sheets
connectors/destination/iceberg
connectors/destination/jdbc
connectors/destination/kafka
connectors/destination/keen
connectors/destination/kinesis
connectors/destination/kvdb
connectors/destination/local-json
connectors/destination/mariadb-columnstore
connectors/destination/meilisearch
connectors/destination/mongodb
connectors/destination/mongodb-strict-encrypt
connectors/destination/mqtt
connectors/destination/mssql
connectors/destination/mssql-strict-encrypt
connectors/destination/mysql
connectors/destination/mysql-strict-encrypt
connectors/destination/oracle
connectors/destination/oracle-strict-encrypt
connectors/destination/postgres
connectors/destination/postgres-strict-encrypt
connectors/destination/pubsub
connectors/destination/pulsar
connectors/destination/r2
connectors/destination/rabbitmq
connectors/destination/redis
connectors/destination/redpanda
connectors/destination/redshift
connectors/destination/s3-glue
connectors/destination/s3
connectors/destination/scaffold-destination-python
connectors/destination/scylla
connectors/destination/sftp-json
connectors/destination/snowflake
connectors/destination/sqlite
connectors/destination/tidb
connectors/destination/typesense
connectors/destination/yugabytedb
connectors/source/bigquery
connectors/source/clickhouse
connectors/source/cockroachdb
connectors/source/db2
connectors/source/elasticsearch
connectors/source/jdbc
connectors/source/kafka
connectors/source/mongodb-v2
connectors/source/mssql
connectors/source/mysql
connectors/source/oracle
connectors/source/postgres
connectors/source/redshift
connectors/source/sftp
connectors/source/snowflake
connectors/source/tidb
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What
Removed normalization and DBT flags from spec. Closes #19855. Closes #18239.
How
Describe the solution
Recommended reading order
x.java
y.python
🚨 User Impact 🚨
Are there any breaking changes? What is the end result perceived by the user? If yes, please merge this PR with the 🚨🚨 emoji so changelog authors can further highlight this if needed.
Pre-merge Checklist
Expand the relevant checklist and delete the others.
New Connector
Community member or Airbyter
airbyte_secret
./gradlew :airbyte-integrations:connectors:<name>:integrationTest
.README.md
bootstrap.md
. See description and examplesdocs/integrations/<source or destination>/<name>.md
including changelog. See changelog exampledocs/integrations/README.md
airbyte-integrations/builds.md
Airbyter
If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.
/test connector=connectors/<name>
command is passing/publish
command described hereUpdating a connector
Community member or Airbyter
airbyte_secret
./gradlew :airbyte-integrations:connectors:<name>:integrationTest
.README.md
bootstrap.md
. See description and examplesdocs/integrations/<source or destination>/<name>.md
including changelog. See changelog exampleAirbyter
If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.
/test connector=connectors/<name>
command is passing/publish
command described hereConnector Generator
-scaffold
in their name) have been updated with the latest scaffold by running./gradlew :airbyte-integrations:connector-templates:generator:testScaffoldTemplates
then checking in your changesTests
Unit
Put your unit tests output here.
Integration
Put your integration tests output here.
Acceptance
Put your acceptance tests output here.