Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ensure monitoring manager deletion and creation on endpoint update #16635

Merged
merged 4 commits into from
Jan 4, 2018
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions app/models/endpoint.rb
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,19 @@ class Endpoint < ApplicationRecord
validates :url, :uniqueness => true, :if => :url
validate :validate_certificate_authority

after_create :endpoint_created
after_destroy :endpoint_destroyed

def endpoint_created
# Make sure monitoring manager is created for the prometheus endpoint
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is a generic callback that could be used for other things I don't think you need to call out the monitoring manager or prometheus endpoint specifically here and here

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, removing.

resource.endpoint_created(role) if resource.respond_to?(:endpoint_created)
end

def endpoint_destroyed
# Make sure monitoring manager is delete for the prometheus endpoint
resource.endpoint_destroyed(role) if resource.respond_to?(:endpoint_destroyed)
end

def verify_ssl=(val)
val = resolve_verify_ssl_value(val)
super
Expand Down
31 changes: 20 additions & 11 deletions app/models/mixins/has_monitoring_manager_mixin.rb
Original file line number Diff line number Diff line change
@@ -1,21 +1,30 @@
module HasMonitoringManagerMixin
extend ActiveSupport::Concern

private
def endpoint_created(role)
if role == "prometheus_alerts" && monitoring_manager.nil?
monitoring_manager = ensure_monitoring_manager
monitoring_manager.save
end
end

def ensure_monitoring_manager
# monitoring_manager should be defined by child classes.
if try(:monitoring_manager_needed?)
build_monitoring_manager(:parent_manager => self)
monitoring_manager.name = "#{name} Monitoring Manager"
def endpoint_destroyed(role)
if role == "prometheus_alerts" && monitoring_manager.present?
# TODO: if someone deletes the alerts endpoint and then quickly readds it they can end up without a manager.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, I don't see any easy fix, and hard fixes are out of scope.
I wonder if long-term we'll want API & UI to actually create/delete managers — and deal with manager deletion being async — rather than just creating/deleting endpoints.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure this is a real risk. I think the queued destroy will be targeted to a specific manager id, and not affect the new monitoring manager.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, even less of a risk now that the logic moved to endpoint.rb.

monitoring_manager.destroy_queue
end
ensure_monitoring_manager_properties
end
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Theres a slight issue here. If a user tries to recreate the alerts endpoint while the previous manager wasnt deleted yet, he could end up with the endpoint but without a manager.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe we could fail the save if monitoring_manager.enabled = false to reduce the chance of that?


def ensure_monitoring_manager_properties
if monitoring_manager
monitoring_manager.zone_id = zone_id
monitoring_manager.provider_region = provider_region
private

def ensure_monitoring_manager
if monitoring_manager.nil?
build_monitoring_manager(:parent_manager => self,
:name => "#{name} Monitoring Manager",
:zone_id => zone_id,
:provider_region => provider_region)
end

monitoring_manager
end
end