Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Job Tracking Issue #1071

Merged
merged 11 commits into from
Nov 26, 2020
Merged

Fix Job Tracking Issue #1071

merged 11 commits into from
Nov 26, 2020

Conversation

cgardens
Copy link
Contributor

@cgardens cgardens commented Nov 24, 2020

Issue: #1064

What

  • Job metadata was not being sent properly to segment.
  • Issue was that I had not set env variable properly in the scheduler.
  • Added tracking for submission of a job and completion of a job.

return switch (output.getStatus()) {
case SUCCEEDED -> JobStatus.COMPLETED;
case FAILED -> JobStatus.FAILED;
default -> throw new IllegalStateException("Unknown state " + output.getStatus());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should probably log instead of throw an exception

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mb -- I misread. This is correct.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thoguth this was only used in tracking code not in business logic. Wouldn't have wanted to fail over tracking issues.

.env.dev Outdated
@@ -13,7 +13,7 @@ WORKSPACE_ROOT=/tmp/workspace
WORKSPACE_DOCKER_MOUNT=/tmp/dev_root/workspace
LOCAL_ROOT=/tmp/airbyte_local
LOCAL_DOCKER_MOUNT=/tmp/airbyte_local
TRACKING_STRATEGY=logging
TRACKING_STRATEGY=segment
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

todo - revert before merge. going to do this separately.

@cgardens cgardens force-pushed the cgardens/debug_segment_issues branch from a7af49f to 8b57fef Compare November 25, 2020 23:27
@@ -142,24 +142,21 @@ public void writeStandardDestinationDefinition(final StandardDestinationDefiniti
destinationDefinition);
}

public SourceConnection getSourceConnection(final UUID sourceId)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

blank space changes.

@@ -159,12 +159,6 @@ public SourceDefinitionSpecificationRead getSourceDefinitionSpecification(Source
final String imageName = DockerUtils.getTaggedImageName(source.getDockerRepository(), source.getDockerImageTag());
final ConnectorSpecification spec = getConnectorSpecification(imageName);

TrackingClientSingleton.get().track("get_source_spec", ImmutableMap.<String, Object>builder()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is just throwing money in a bucket and lighting it on fire.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants