Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

convert EnvVariableFeatureFlag to an injected dependency #21171

Merged
merged 6 commits into from
Jan 10, 2023
Merged

convert EnvVariableFeatureFlag to an injected dependency #21171

merged 6 commits into from
Jan 10, 2023

Conversation

colesnodgrass
Copy link
Member

What

  • fix update EnvVariableFeatureFlags usages #21139
  • made it easier to change the feature-flag implementation by injecting the dependency instead of instantiating within the various classes modified in this PR
  • does not modify integrations (will do as a separate PR)

How

  • replace new EnvVariableFeatureFlags() calls with injected FeatureFlag dependency

@colesnodgrass colesnodgrass requested a review from a team as a code owner January 9, 2023 18:51
@octavia-squidington-iv octavia-squidington-iv added area/platform issues related to the platform area/worker Related to worker labels Jan 9, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Jan 9, 2023 •
edited
Loading

Affected Connector Report

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to do the following as needed:

  • Run integration tests
  • Bump connector or module version
  • Add changelog
  • Publish the new version

✅ Sources (30)
Connector Version Changelog Publish
source-alloydb 1.0.34 ✅ ✅
source-alloydb-strict-encrypt 1.0.34 🔵
(ignored)		 🔵
(ignored)
source-bigquery 0.2.3 ✅ ✅
source-clickhouse 0.1.14 ✅ ✅
source-clickhouse-strict-encrypt 0.1.14 🔵
(ignored)		 🔵
(ignored)
source-cockroachdb 0.1.18 ✅ ✅
source-cockroachdb-strict-encrypt 0.1.18 🔵
(ignored)		 🔵
(ignored)
source-db2 0.1.16 ✅ ✅
source-db2-strict-encrypt 0.1.16 🔵
(ignored)		 🔵
(ignored)
source-dynamodb 0.1.0 ✅ ✅
source-e2e-test 2.1.3 ✅ ✅
source-e2e-test-cloud 2.1.1 🔵
(ignored)		 🔵
(ignored)
source-elasticsearch 0.1.1 ✅ ✅
source-jdbc 0.3.5 🔵
(ignored)		 🔵
(ignored)
source-kafka 0.2.3 ✅ ✅
source-mongodb-strict-encrypt 0.1.19 🔵
(ignored)		 🔵
(ignored)
source-mongodb-v2 0.1.19 ✅ ✅
source-mssql 0.4.26 ✅ ✅
source-mssql-strict-encrypt 0.4.26 🔵
(ignored)		 🔵
(ignored)
source-mysql 1.0.18 ✅ ✅
source-mysql-strict-encrypt 1.0.18 🔵
(ignored)		 🔵
(ignored)
source-oracle 0.3.21 ✅ ✅
source-oracle-strict-encrypt 0.3.21 🔵
(ignored)		 🔵
(ignored)
source-postgres 1.0.35 ✅ ✅
source-postgres-strict-encrypt 1.0.35 🔵
(ignored)		 🔵
(ignored)
source-redshift 0.3.15 ✅ ✅
source-scaffold-java-jdbc 0.1.0 🔵
(ignored)		 🔵
(ignored)
source-sftp 0.1.2 ✅ ✅
source-snowflake 0.1.27 ✅ ✅
source-tidb 0.2.1 ✅ ✅
  • See "Actionable Items" below for how to resolve warnings and errors.

❌ Destinations (47)
Connector Version Changelog Publish
destination-aws-datalake 0.1.1 ✅ ✅
destination-azure-blob-storage 0.1.6 ✅ ✅
destination-bigquery 1.2.9 ✅ ✅
destination-bigquery-denormalized 1.2.10 ✅ ✅
destination-cassandra 0.1.4 ✅ ✅
destination-clickhouse 0.2.1 ✅ ✅
destination-clickhouse-strict-encrypt 0.2.1 🔵
(ignored)		 🔵
(ignored)
destination-csv 0.2.10 ✅ ✅
destination-databricks 0.3.1 ✅ ✅
destination-dev-null 0.2.7 🔵
(ignored)		 🔵
(ignored)
destination-doris 0.1.0 ✅ ✅
destination-dynamodb 0.1.7 ✅ ✅
destination-e2e-test 0.2.4 ✅ ✅
destination-elasticsearch 0.1.6 ✅ ✅
destination-elasticsearch-strict-encrypt 0.1.6 🔵
(ignored)		 🔵
(ignored)
destination-gcs 0.2.12 ✅ ✅
destination-iceberg 0.1.0 ✅ ✅
destination-jdbc 0.3.14 🔵
(ignored)		 🔵
(ignored)
destination-kafka 0.1.10 ✅ ✅
destination-keen 0.2.4 ✅ ✅
destination-kinesis 0.1.5 ✅ ✅
destination-local-json 0.2.11 ✅ ✅
destination-mariadb-columnstore 0.1.7 ✅ ✅
destination-mongodb 0.1.9 ✅ ✅
destination-mongodb-strict-encrypt 0.1.9 🔵
(ignored)		 🔵
(ignored)
destination-mqtt 0.1.3 ✅ ✅
destination-mssql 0.1.22 ✅ ✅
destination-mssql-strict-encrypt 0.1.22 🔵
(ignored)		 🔵
(ignored)
destination-mysql 0.1.20 ✅ ✅
destination-mysql-strict-encrypt ❌ 0.1.21
(mismatch: 0.1.20)		 🔵
(ignored)		 🔵
(ignored)
destination-oracle 0.1.19 ✅ ✅
destination-oracle-strict-encrypt 0.1.19 🔵
(ignored)		 🔵
(ignored)
destination-postgres 0.3.26 ✅ ✅
destination-postgres-strict-encrypt 0.3.26 🔵
(ignored)		 🔵
(ignored)
destination-pubsub 0.2.0 ✅ ✅
destination-pulsar 0.1.3 ✅ ✅
destination-r2 0.1.0 ✅ ✅
destination-redis 0.1.4 ✅ ✅
destination-redpanda 0.1.0 ✅ ✅
destination-redshift 0.3.53 ✅ ✅
destination-rockset 0.1.4 ✅ ✅
destination-s3 0.3.18 ✅ ✅
destination-s3-glue 0.1.1 ✅ ✅
destination-scylla 0.1.3 ✅ ✅
destination-snowflake 0.4.41 ✅ ✅
destination-tidb 0.1.0 ✅ ✅
destination-yugabytedb 0.1.0 ✅ ✅
  • See "Actionable Items" below for how to resolve warnings and errors.

✅ Other Modules (0)

Actionable Items

(click to expand)
Category Status Actionable Item
Version ❌
mismatch		 The version of the connector is different from its normal variant. Please bump the version of the connector.
⚠
doc not found		 The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like source-jdbc is not published or documented). Please double-check to make sure that it is not a bug.
Changelog ⚠
doc not found		 The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like source-jdbc is not published or documented). Please double-check to make sure that it is not a bug.
❌
changelog missing		 There is no chnagelog for the current version of the connector. If you are the author of the current version, please add a changelog.
Publish ⚠
not in seed		 The connector is not in the seed file (e.g. source_definitions.yaml), so its publication status cannot be checked. This can be normal (e.g. some connectors are cloud-specific, and only listed in the cloud seed file). Please double-check to make sure that it is not a bug.
❌
diff seed version		 The connector exists in the seed file, but the latest version is not listed there. This usually means that the latest version is not published. Please use the /publish command to publish the latest version.

@colesnodgrass colesnodgrass temporarily deployed to more-secrets January 9, 2023 18:54 — with GitHub Actions Inactive
@colesnodgrass colesnodgrass temporarily deployed to more-secrets January 9, 2023 18:54 — with GitHub Actions Inactive
jdpgrailsdev
jdpgrailsdev approved these changes Jan 9, 2023
View reviewed changes
Copy link
Contributor

@jdpgrailsdev jdpgrailsdev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

benmoriceau
benmoriceau approved these changes Jan 9, 2023
View reviewed changes
Copy link
Contributor

@benmoriceau benmoriceau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

on Nit comment on the test. Apart from that, I merged the server before your commit it is more FYI since this shouldn't impact you change.

...ommons-worker/src/test/java/io/airbyte/workers/general/ReplicationWorkerPerformanceTest.java
@@ -65,7 +66,7 @@ public class ReplicationWorkerPerformanceTest {
public void executeOneSync() throws InterruptedException {
final var perSource = new LimitedAirbyteSource();
final var perDestination = new EmptyAirbyteDestination();
final var messageTracker = new AirbyteMessageTracker();
final var messageTracker = new AirbyteMessageTracker(new EnvVariableFeatureFlags());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit in the test should we use mock? The test might be affected by the env of where the test are run.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tests should use a mock, and when I replace this with the new feature-flag client I'll add mocks where necessary.

The code (and the tests) were and are already referencing the real environment variables. So the changes in this PR shouldn't have any unexpected side-effects that didn't already exist.

@colesnodgrass colesnodgrass temporarily deployed to more-secrets January 9, 2023 23:21 — with GitHub Actions Inactive
@colesnodgrass colesnodgrass temporarily deployed to more-secrets January 9, 2023 23:21 — with GitHub Actions Inactive
@colesnodgrass colesnodgrass temporarily deployed to more-secrets January 10, 2023 00:35 — with GitHub Actions Inactive
@colesnodgrass colesnodgrass temporarily deployed to more-secrets January 10, 2023 00:36 — with GitHub Actions Inactive
@colesnodgrass colesnodgrass merged commit afaf2d1 into master Jan 10, 2023
@colesnodgrass colesnodgrass deleted the cole/inject-envarff-client branch January 10, 2023 16:59
@octavia-squidington-iii octavia-squidington-iii mentioned this pull request Jan 12, 2023
Merged
jbfbell pushed a commit that referenced this pull request Jan 13, 2023
@colesnodgrass @jbfbell
convert EnvVariableFeatureFlag to an injected dependency (#21171)
1a25de2 
* inject feature-flag client into integration-launcher

* inject feature-flag client into message-tracker

* inject feature-flag client into DefaultAirbyteSource

* missed reference in message-tracker
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@benmoriceau benmoriceau benmoriceau approved these changes

@gosusnp gosusnp gosusnp approved these changes

@jdpgrailsdev jdpgrailsdev jdpgrailsdev approved these changes

@alovew alovew Awaiting requested review from alovew

Assignees
No one assigned
Labels
area/platform issues related to the platform area/worker Related to worker
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

update EnvVariableFeatureFlags usages
5 participants
@colesnodgrass @jdpgrailsdev @gosusnp @benmoriceau @octavia-squidington-iv