Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ONYX-28233: Add telemetry #2854

Merged
merged 22 commits into from
Aug 4, 2023
Merged

ONYX-28233: Add telemetry #2854

merged 22 commits into from
Aug 4, 2023

Conversation

szh
Copy link
Contributor

@szh szh commented Jul 10, 2023

Desired Outcome

Merge telemetry feature branch

The Telemetry feature is complete but it was done on a feature branch. We will need to merge this back to master before we can truly consider it complete.

As part of the review, we should primarily do some sanity checking to verify that Telemetry is not on by default and is essentially invisible until enabled.

Once the cyberark/conjur changes are merged we will can also merge the quick-start guide.

Implemented Changes

Describe how the desired outcome above has been achieved with this PR. In
particular, consider:

  • What's changed? Why were these changes made?
  • How should the reviewer approach this PR, especially if manual tests are required?
  • Are there relevant screenshots you can add to the PR description?

Connected Issue/Story

CyberArk internal issue ID: ONYX-28233

Changelog

  • The CHANGELOG has been updated, or
  • This PR does not include user-facing changes and doesn't require a
    CHANGELOG update

Test coverage

  • This PR includes new unit and integration tests to go with the code
    changes, or
  • The changes in this PR do not require tests

Documentation

  • Docs (e.g. READMEs) were updated in this PR
  • A follow-up issue to update official docs has been filed here: [insert issue ID]
  • This PR does not require updating any documentation

Behavior

  • This PR changes product behavior and has been reviewed by a PO, or
  • These changes are part of a larger initiative that will be reviewed later, or
  • No behavior was changed with this PR

Security

  • Security architect has reviewed the changes in this PR,
  • These changes are part of a larger initiative with a separate security review, or
  • There are no security aspects to these changes

@szh szh self-assigned this Jul 10, 2023
@szh szh force-pushed the feature-conjur-telemetry branch 3 times, most recently from 6f40765 to dad3c0d Compare July 10, 2023 19:50
lib/conjur/conjur_config.rb Outdated Show resolved Hide resolved
lib/monitoring/pub_sub.rb Outdated Show resolved Hide resolved
lib/monitoring/prometheus.rb Show resolved Hide resolved
lib/monitoring/operations.rb Outdated Show resolved Hide resolved
lib/monitoring/operations.rb Outdated Show resolved Hide resolved
@szh szh force-pushed the feature-conjur-telemetry branch from dad3c0d to 4a0c97e Compare July 11, 2023 17:30
lib/monitoring/operations.rb Outdated Show resolved Hide resolved
lib/monitoring/operations.rb Outdated Show resolved Hide resolved
lib/monitoring/operations.rb Outdated Show resolved Hide resolved
lib/monitoring/operations.rb Outdated Show resolved Hide resolved
lib/monitoring/operations.rb Outdated Show resolved Hide resolved
@szh szh marked this pull request as ready for review July 11, 2023 18:16
@szh szh requested a review from a team as a code owner July 11, 2023 18:16
@szh szh force-pushed the feature-conjur-telemetry branch from 4a0c97e to 0d66ecc Compare July 11, 2023 18:18
@szh szh requested a review from micahlee July 11, 2023 18:20
lib/monitoring/operations.rb Outdated Show resolved Hide resolved
lib/monitoring/operations.rb Outdated Show resolved Hide resolved
lib/monitoring/operations.rb Outdated Show resolved Hide resolved
lib/monitoring/operations.rb Outdated Show resolved Hide resolved
lib/monitoring/operations.rb Outdated Show resolved Hide resolved
lib/monitoring/operations.rb Outdated Show resolved Hide resolved
lib/monitoring/operations.rb Outdated Show resolved Hide resolved
lib/monitoring/operations.rb Outdated Show resolved Hide resolved
lib/monitoring/operations.rb Outdated Show resolved Hide resolved
lib/monitoring/operations.rb Outdated Show resolved Hide resolved
@jvanderhoof
Copy link
Contributor

@szh, awesome work on this feature. My only real ask is to add some unit tests. It looks like coverage is largely handled via controller integration tests. Your code looks well written and it looks like it'll be simple to add unit tests.

@szh szh force-pushed the feature-conjur-telemetry branch from d6d0a40 to ba946a8 Compare July 18, 2023 18:19
lib/monitoring/operations.rb Outdated Show resolved Hide resolved
lib/monitoring/operations.rb Outdated Show resolved Hide resolved
lib/monitoring/operations.rb Outdated Show resolved Hide resolved
lib/monitoring/operations.rb Outdated Show resolved Hide resolved
lib/monitoring/operations.rb Outdated Show resolved Hide resolved
lib/monitoring/operations.rb Outdated Show resolved Hide resolved
lib/monitoring/operations.rb Outdated Show resolved Hide resolved
lib/monitoring/operations.rb Outdated Show resolved Hide resolved
lib/monitoring/operations.rb Outdated Show resolved Hide resolved
lib/monitoring/operations.rb Outdated Show resolved Hide resolved
lib/monitoring/operations.rb Outdated Show resolved Hide resolved
lib/monitoring/operations.rb Outdated Show resolved Hide resolved
lib/monitoring/operations.rb Outdated Show resolved Hide resolved
lib/monitoring/operations.rb Outdated Show resolved Hide resolved
lib/monitoring/operations.rb Outdated Show resolved Hide resolved
@szh szh force-pushed the feature-conjur-telemetry branch 2 times, most recently from 32335c8 to 925891f Compare July 19, 2023 14:15
@szh szh force-pushed the feature-conjur-telemetry branch from 2be30c6 to 0c9079e Compare August 2, 2023 14:12

module Monitoring
module Middleware
class PrometheusCollector
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Monitoring::Middleware::PrometheusCollector assumes too much for instance variable '@initialized'


def find_operation(method, path)
Monitoring::Metrics::OPERATIONS.each do |op|
if op[:method] == method && op[:pattern].match?(path)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Monitoring::Middleware::PrometheusCollector#find_operation is controlled by argument 'method'

setup_subscriber(metric)
end

def setup_subscriber(metric)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Monitoring::Metrics#setup_subscriber doesn't depend on instance state (maybe move it to another class?)

@@ -0,0 +1,60 @@
module Monitoring
module Metrics
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Monitoring::Metrics has no descriptive comment


def create_histogram_metric(metric)
histogram = ::Prometheus::Client::Histogram.new(
metric.metric_name,
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Monitoring::Metrics#create_histogram_metric refers to 'metric' more than self (maybe move it to another class?)

@micahlee
Copy link
Contributor

micahlee commented Aug 4, 2023

Hey @szh (@doodlesbykumbi),

If we're keeping the commit history, I suggest we remove the following commits, at a minimum, as they represent a no-op once combined:

Generally speaking, I would also expect linting fixes to be squashed back into the original commit, especially in cases where the PR introduces the files for the first time, as these commits on their own don't meaningfully contribute to the story of this PR:

With those history modifications, I think this makes sense to include with the commit history. I think we must remove the two no-op commits to avoid those existing in master history. The others are a strong preference, but if it's too onerous I'm not strictly opposed to them.

@jtuttle
Copy link
Member

jtuttle commented Aug 4, 2023

@micahlee I just made the suggested commit updates if you could re-review when you have a chance. Thanks!

Copy link
Contributor

@micahlee micahlee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! Thanks for getting the commit history cleaned up a bit.

LGTM, @szh and @jtuttle !


def create_gauge_metric(metric)
gauge = ::Prometheus::Client::Gauge.new(
metric.metric_name,
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Monitoring::Metrics#create_gauge_metric refers to 'metric' more than self (maybe move it to another class?)


def create_counter_metric(metric)
counter = ::Prometheus::Client::Counter.new(
metric.metric_name,
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Monitoring::Metrics#create_counter_metric refers to 'metric' more than self (maybe move it to another class?)

@@ -0,0 +1,29 @@
module Monitoring
module Metrics
class PolicyRoleGauge
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Monitoring::Metrics::PolicyRoleGauge has at least 6 instance variables

@@ -0,0 +1,29 @@
module Monitoring
module Metrics
class PolicyResourceGauge
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Monitoring::Metrics::PolicyResourceGauge has at least 6 instance variables

end
end

def get_authenticator_counts(authenticators)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Monitoring::Metrics::AuthenticatorGauge#get_authenticator_counts doesn't depend on instance state (maybe move it to another class?)

@codeclimate
Copy link

codeclimate bot commented Aug 4, 2023

Code Climate has analyzed commit 1fb0fad and detected 50 issues on this pull request.

Here's the issue category breakdown:

Category Count
Style 14
Complexity 36

The test coverage on the diff in this pull request is 99.6% (50% is the threshold).

This pull request will bring the total coverage in the repository to 87.6% (-0.7% change).

View more on Code Climate.

@jtuttle jtuttle merged commit 0c6c4b2 into master Aug 4, 2023
2 checks passed
@jtuttle jtuttle deleted the feature-conjur-telemetry branch August 4, 2023 20:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

7 participants