Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[internal/exp/metrics] Add a new internal package for handling metric staleness #31089

Merged
merged 9 commits into from
Feb 27, 2024

Conversation

RichieSams
Copy link
Contributor

@RichieSams RichieSams commented Feb 6, 2024

Description:
It's a glorified wrapper over a Map type, which allows values to be expired based on a pre-supplied interval.

Link to tracking Issue:
#31016

Testing:
I added some basic tests of the PriorityQueue implementation as well as the expiry behaviour of Staleness

Documentation:

All the new structs are documented

Copy link
Member

@sh0rez sh0rez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is really nice and exactly what I had in mind. Great work!!

internal/exp/metrics/staleness/staleness.go Outdated Show resolved Hide resolved
internal/exp/metrics/staleness/staleness_test.go Outdated Show resolved Hide resolved
internal/exp/metrics/staleness/priority_queue.go Outdated Show resolved Hide resolved
Copy link
Member

@sh0rez sh0rez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tried this out, works really well!
I think this is ready for final review once rebased onto HEAD!

internal/exp/metrics/staleness/staleness.go Show resolved Hide resolved
internal/exp/metrics/staleness/staleness.go Outdated Show resolved Hide resolved
@RichieSams RichieSams force-pushed the exp/metrics/staleness branch from 18aa7c3 to e12c0ff Compare February 20, 2024 20:21
@RichieSams RichieSams requested a review from sh0rez February 20, 2024 20:43
@RichieSams RichieSams marked this pull request as ready for review February 20, 2024 20:44
@RichieSams RichieSams force-pushed the exp/metrics/staleness branch from 6bea8be to 4f9ab5c Compare February 20, 2024 23:01
@RichieSams
Copy link
Contributor Author

RichieSams commented Feb 20, 2024

@djaglowski @jpkrohling @fatsheep9146 @bryan-aguilar

This PR is ready for final review. (Can one of you add the "Skip Changelog" label, since this is an internal-only change)

@sh0rez Any final comments?

Copy link
Member

@sh0rez sh0rez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's in great shape for the initial bring-up. Everything else can be iterated upon :)

var _ Map[time.Time] = (*RawMap[identity.Stream, time.Time])(nil)

// RawMap is an implementation of the Map interface using a standard golang map
type RawMap[K comparable, V any] map[K]V
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this type doesn't creeps a bit beyond the scope of this package.
I like that this package defines the Map[T] interface as the behavior it expects to fulfill its purpose.
A general stream map would feel more fitting as a streams.HashMap or similar.

I'd be okay with this pkg not defining a map implementation at all, leaving that to consumers.
No hard opinion however, if you prefer we can leave it in here for now, as we'll likely iterate on these packages for a bit anyways.

Copy link
Contributor Author

@RichieSams RichieSams Feb 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd be okay with this pkg not defining a map implementation at all, leaving that to consumers.

That was my initial plan. I had implemented RawMap just in the tests. But then I realized that I would need to re-implement the same code in the interval processor (and perhaps you as well in deltatocumulative). So I moved it to be in the package proper as a convenience "reference" implementation.

But I'm kind of the same. I don't feel strongly either way

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you ok with leaving it in for now? And iterating as the various processors utilize it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bump

@RichieSams RichieSams force-pushed the exp/metrics/staleness branch from 4f9ab5c to a3d08a8 Compare February 21, 2024 16:15
Copy link
Member

@djaglowski djaglowski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but I will wait to approve until @sh0rez's feedback is fully addressed.

@RichieSams RichieSams force-pushed the exp/metrics/staleness branch from cde1ec7 to d0f1d14 Compare February 22, 2024 01:53
@sh0rez
Copy link
Member

sh0rez commented Feb 27, 2024

@djaglowski ready to merge from my side!

Think this still needs the "no changelog" label

@djaglowski djaglowski added the Skip Changelog PRs that do not require a CHANGELOG.md entry label Feb 27, 2024
@djaglowski djaglowski merged commit 184d094 into open-telemetry:main Feb 27, 2024
146 of 147 checks passed
@github-actions github-actions bot added this to the next release milestone Feb 27, 2024
@RichieSams RichieSams deleted the exp/metrics/staleness branch February 27, 2024 16:56
jpkrohling pushed a commit that referenced this pull request Mar 5, 2024
**Description:** Removes stale series from tracking (and thus frees
their memory) using staleness logic from
#31089

**Link to tracking Issue:**
#30705,
#31016

**Testing:** `TestExpiry`
**Documentation:** README updated
DougManton pushed a commit to DougManton/opentelemetry-collector-contrib that referenced this pull request Mar 13, 2024
)

**Description:** Removes stale series from tracking (and thus frees
their memory) using staleness logic from
open-telemetry#31089

**Link to tracking Issue:**
open-telemetry#30705,
open-telemetry#31016

**Testing:** `TestExpiry`
**Documentation:** README updated
XinRanZhAWS pushed a commit to XinRanZhAWS/opentelemetry-collector-contrib that referenced this pull request Mar 13, 2024
… staleness (open-telemetry#31089)

**Description:**
It's a glorified wrapper over a Map type, which allows values to be
expired based on a pre-supplied interval.

**Link to tracking Issue:**

open-telemetry#31016

**Testing:**
I added some basic tests of the PriorityQueue implementation as well as
the expiry behaviour of Staleness

**Documentation:**

All the new structs are documented
XinRanZhAWS pushed a commit to XinRanZhAWS/opentelemetry-collector-contrib that referenced this pull request Mar 13, 2024
)

**Description:** Removes stale series from tracking (and thus frees
their memory) using staleness logic from
open-telemetry#31089

**Link to tracking Issue:**
open-telemetry#30705,
open-telemetry#31016

**Testing:** `TestExpiry`
**Documentation:** README updated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
internal/exp/metrics Skip Changelog PRs that do not require a CHANGELOG.md entry
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants