Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(pubsub): Restart Google Pubsub subscriber on failures #263

Merged
merged 1 commit into from
May 22, 2018

Conversation

wjoel
Copy link
Contributor

@wjoel wjoel commented May 22, 2018

Creates a new Subscriber on failures, as according to googleapis/google-cloud-java#1579 (comment) we can "restart" the Subscriber by creating a new Subscriber object.

I've also removed .setMaxAckExtensionPeriod(Duration.ofSeconds(0)) as googleapis/google-cloud-java#2465 (comment) recommends against it.

Potentially solves spinnaker/spinnaker#2762 but needs more testing. I've verified that echo resumes consuming pubsub messages after I removed its Pub/Sub Subscriber permission (took a little while before it noticed) and then added it back again.

Copy link
Member

@lwander lwander left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, @jtk54 PTAL as well

return new GooglePubsubSubscriber(subscription.getName(), subscriptionName, project, subscriber);
subscriber.addListener(new GooglePubsubFailureHandler(this, formatSubscriptionName(project, subscriptionName)), MoreExecutors.directExecutor());
subscriber.startAsync().awaitRunning();
log.info("Google Pubsub subscriber started");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add the subscription & project name to this log message?

Copy link
Contributor

@ezimanyi ezimanyi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great, and should make Echo much more robust to transient errors.

Copy link
Contributor

@jtk54 jtk54 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few comments, LGTM otherwise.

}

private void restart() {
log.info("Waiting to restart Google Pubsub subscriber");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add more context to this log statement as well?

@@ -81,7 +85,7 @@ public String subscriptionName() {

@Override
public String getName() {
return name;
return subscriptionName;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We made a distinction between the subscriptionName, which is the actual locator in GCE and the logical name which is used in the UI when suggesting pub/sub subscriptions to use. Is there a reason to change that here?

@wjoel wjoel force-pushed the restart-pubsub-on-failure branch from cebc471 to 50ab8d0 Compare May 22, 2018 20:48
@wjoel
Copy link
Contributor Author

wjoel commented May 22, 2018

@jtk54 That makes sense, I'll leave name as it was, and I've added the subscription name to the log messages.

@jtk54
Copy link
Contributor

jtk54 commented May 22, 2018

Great, thanks for the contribution.

@jtk54 jtk54 merged commit 91589ac into spinnaker:master May 22, 2018
@wjoel wjoel deleted the restart-pubsub-on-failure branch May 22, 2018 20:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants