Skip to content

Comments

release-24.1: security: add ttl metrics for certificate timeout values#138659

Merged
angles-n-daemons merged 2 commits intocockroachdb:release-24.1from
angles-n-daemons:backport24.1-130110-135596
Jan 13, 2025
Merged

release-24.1: security: add ttl metrics for certificate timeout values#138659
angles-n-daemons merged 2 commits intocockroachdb:release-24.1from
angles-n-daemons:backport24.1-130110-135596

Conversation

@angles-n-daemons
Copy link
Contributor

@angles-n-daemons angles-n-daemons commented Jan 8, 2025 •
edited
Loading

The below changes address the fact that certificate expiration is exposed as a timestamp in our metrics exports, which cannot be leveraged well in metrics query languages. The two commits below add certificate expiration as a ttl in seconds, and fix a bug in the change which will update when new certificates are loaded.

Backport:

Please see individual PRs for details.

/cc @cockroachdb/release

Epic: CRDB-40209
Release justification: Adds a ttl metric for certificate expiration.

@angles-n-daemons
ops: add ttl metrics for certificate expiration
e6c4196 
Currently, cockroach only exposes point in time certificate
expiration metrics. If the certificate is to expire 1 day from now,
we expose a gauge `security.certificate.expiration.<cert-type>`
which is the unix timestamp when it will expire. This PR also
exposes a ttl metric `security.certificate.ttl.<cert-type>` so that
consumers of this information can run operations based on their
distance to certificate expiration without additional
transformations.

Additionally, this PR refactors how the expiration gauges are set,
so that reads of the gauge directly reference the value of the
certificate.

Epic: CRDB-40209
Fixes: cockroachdb#77376

Release note (ops change): new metrics which expose the ttl for various
certificates
@angles-n-daemons angles-n-daemons requested a review from a team as a code owner January 8, 2025 17:33
@angles-n-daemons angles-n-daemons requested a review from a team January 8, 2025 17:33
@angles-n-daemons angles-n-daemons requested a review from a team as a code owner January 8, 2025 17:33
@angles-n-daemons angles-n-daemons requested review from dhartunian and removed request for a team January 8, 2025 17:33
@blathers-crl
Copy link

blathers-crl bot commented Jan 8, 2025 •
edited by angles-n-daemons
Loading

Thanks for opening a backport.

Please check the backport criteria before merging:

  • Backports should only be created for serious
    issues     or test-only changes.
  • Backports should not break backwards-compatibility.
  • Backports should change as little code as possible.
  • Backports should not change on-disk formats or node communication protocols.
  • Backports should not add new functionality (except as defined
    here).
  • Backports must not add, edit, or otherwise modify cluster versions; or add version gates.
  • All backports must be reviewed by the owning areas TL. For more information as to how that review should be conducted, please consult the backport
    policy    .
If your backport adds new functionality, please ensure that the following additional criteria are satisfied:
  • There is a high priority need for the functionality that cannot wait until the next release and is difficult to address in another way.
  • The new functionality is additive-only and only runs for clusters which have specifically “opted in” to it (e.g. by a cluster setting).
  • New code is protected by a conditional check that is trivial to verify and ensures that it only runs for opt-in clusters. State changes must be further protected such that nodes running old binaries will not be negatively impacted by the new state (with a mixed version test added).
  • The PM and TL on the team that owns the changed code have signed off that the change obeys the above rules.
  • Your backport must be accompanied by a post to the appropriate Slack
    channel (#db-backports-point-releases or #db-backports-XX-X-release) for awareness and discussion.

Also, please add a brief release justification to the body of your PR to justify this
backport.

@blathers-crl blathers-crl bot added the backport Label PR's that are backports to older release branches label Jan 8, 2025
@cockroach-teamcity
Copy link
Member

cockroach-teamcity commented Jan 8, 2025

This change is Reviewable

@angles-n-daemons
security: bugfix, ensure cert expiry metrics reflect reloaded certs
83d9bcb 
The PR cockroachdb#130110 added certificate TTL metrics alongside our existing
expiration metrics. Prior to that change, the certificate metrics values
were updated on each metrics load. Afterwards, new metrics objects were
created for each load of certificates.

This created a bug in that the new expiration values would not be
found in any of the system exhaust (metrics scrape or tsdb) because the
registered metrics objects were the ones created on startup.

This new change instead allows the metrics to close the whole
CertificateManager object, so that they only need to be created once, and
therefore the initial registration of metrics reflects persistently
valid values.

Release note (bug fix): security.certificate.* metrics will now be
updated if a node loads new certificates while running.
@angles-n-daemons angles-n-daemons force-pushed the backport24.1-130110-135596 branch from 706081d to 83d9bcb Compare January 9, 2025 18:56
@angles-n-daemons angles-n-daemons merged commit c5cfea0 into cockroachdb:release-24.1 Jan 13, 2025
19 of 20 checks passed
@angles-n-daemons angles-n-daemons deleted the backport24.1-130110-135596 branch January 13, 2025 15:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dhartunian dhartunian dhartunian approved these changes

Assignees

No one assigned

Labels

backport Label PR's that are backports to older release branches v24.1.12

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@angles-n-daemons @cockroach-teamcity @dhartunian