Add Disable Failing Connections feature #10877
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
area/platform
benmoriceau
reviewed
Mar 7, 2022
...ers/src/main/java/io/airbyte/workers/temporal/scheduling/activities/DisableActivityImpl.java
Outdated
Show resolved
Hide resolved
benmoriceau
reviewed
Mar 7, 2022
| public class DisableActivityImpl implements DisableActivity { | ||
|
|
||
| @VisibleForTesting | ||
| public static final int MAX_FAILURE_JOBS_IN_A_ROW = 100; |
There was a problem hiding this comment.
This should be coming from the .env file, which means that we need to update the Configs interface to add getter for those variables
benmoriceau
reviewed
Mar 7, 2022
| public void disableConnection(final DisableActivityInput input) { | ||
| try { | ||
| // lists job in descending order by created_at | ||
| final List<Job> jobs = jobPersistence.listJobs(ConfigType.SYNC, |
There was a problem hiding this comment.
I think that we should include the reset connection type here. To be check with product.
benmoriceau
reviewed
Mar 7, 2022
...ers/src/main/java/io/airbyte/workers/temporal/scheduling/activities/DisableActivityImpl.java
Outdated
Show resolved
Hide resolved
benmoriceau
requested changes
Mar 7, 2022
...ers/src/main/java/io/airbyte/workers/temporal/scheduling/activities/DisableActivityImpl.java
Outdated
Show resolved
Hide resolved
...ers/src/test/java/io/airbyte/workers/temporal/scheduling/activities/DisableActivityTest.java
Outdated
Show resolved
Hide resolved
...ers/src/main/java/io/airbyte/workers/temporal/scheduling/activities/DisableActivityImpl.java
Outdated
Show resolved
Hide resolved
benmoriceau
reviewed
Mar 7, 2022
...ava/io/airbyte/workers/temporal/scheduling/activities/AutoDisableConnectionActivityImpl.java
Outdated
Show resolved
Hide resolved
|
Not ready for review.
|
* Remove deprecated/unused enum values from json schema, migration to update records to use corrected values * make migration-specific classes handle any string, and remove extranneous comments/annotations for readability. also test that an unrecognized enum value is left alone and doesn't cause deserialization errors * gradle format * fix test * fix jobTrackerTest with new enum values
Co-authored-by: benmoriceau <benmoriceau@users.noreply.github.com>
…bucket (#10856) * Add a test for listObjects permission to destination-s3 connector * add testIAMUserHasListObjectPermission method to S3Destination and call this method from S3Destination::check. Method throws an exception if IAM user does not have listObjects permission on the destination bucket * add a unit test to S3DestinationTest to verify that S3Destination::check fails if listObjects throws an exception * add a unit test to S3DestinationTest to verify that S3Destination::check succeeds if listObjects succeeds * Add S3DestinationConfigFactory in order to be able to mock S3 client used in S3Destination::check * Addressing review comments: - separate positive and negative unit tests - fix formatting - reuse s3 client for both positive and negative tests * Add information about PR #10856 to the changelog * Prepare for publishing new version: * Bump version to 0.2.10 in Dockerfile * Bump version to 0.2.10 in changelog * Update destination-s3 version in connector index * Update seed spec for destination-s3 connector
Co-authored-by: benmoriceau <benmoriceau@users.noreply.github.com>
Add a metric to monitor the monitoring app's rate of publishing metrics. Though this isn't perfect, it gives us some insight into whether metric publishing is okay or running into issues.
* Set up script to remove old instances. * Switch to push for testing. * Debugging. * Authenticate. * Try again. * Set region. * Add JQ. * Add iso check. * Add curr time. Add JQ query. * Correctly set env. * Format command. * Format command. * Should be inverted. * Only get running. * Filter for instance name. * Space instead of comma. * Debug. * Put back running instance filter. * Undo changes. * Test. * Move comment.
|
|
github-actions
bot
added
area/api
|
This PR has been moved to #11099. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What
Adds feature to automatically disable failing connections. This addresses issue 9715. When this feature is turned on, it will disable a connection after
MAX_FAILURE_JOBS_IN_A_ROWconsecutive failures (currently set to 100 jobs) or afterMAX_DAYS_OF_STRAIGHT_FAILUREdays of only failed jobs (currently set to 14 days).todo:
Notify user when connection has been auto-disable.Will be followed up through this issue.listJobStatusWithConnectionis not filtering by timestamp in unit testsHow
When the feature flag is turned on, after a job failure, the
DisableActivitywill check if either of the above criteria has been met. If so, it will set the connection status toINACTIVE.Recommended reading order
airbyte-scheduler/persistence/src/main/java/io/airbyte/scheduler/persistence/DefaultJobPersistence.javaairbyte-workers/src/main/java/io/airbyte/workers/temporal/scheduling/activities/AutoDisableConnectionActivityImpl.java