Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pkg/otlp/metrics: fix otel.* namespaced metrics #142

Closed
wants to merge 3 commits into from
Closed
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions .chloggen/gbbr_fix-otel-namespace.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: bug_fix

# The name of the component (e.g. pkg/quantile)
component: pkg/otlp

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: System (and process.*) metrics were remapped with otel.*, overwriting the original metric and losing it. This change fixes that, keeping the original metric.

# The PR related to this change
issues: [141]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext:
7 changes: 6 additions & 1 deletion pkg/otlp/metrics/metrics_remapping.go
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,12 @@ func remapSystemMetrics(all pmetric.MetricSlice, m pmetric.Metric) {
case "system.filesystem.utilization":
copyMetric(all, m, "system.disk.in_use", 1)
}
// process.* and system.* metrics need to be prepended with the otel.* namespace
if strings.HasPrefix(name, "system.") {
// we keep the original metric for system.* metrics
Copy link
Member

@songy23 songy23 Aug 4, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I looked again at the old code, I think the behavior is still different: we prepend otel. to the original names, but we keep the remapped names with no otel. prefix. https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/23445/files#diff-59a3a2cae4395f15cad4800806f9dc7680aed0f303dd896c0ca8663e0fd0ac35L115-L129

For example, in the old code:

  • if we have "system.filesystem.utilization", first we remap it to "system.disk.in_use" and append the new one to slice
  • then we prepend otel. to "system.filesystem.utilization" but not
    "system.disk.in_use"
  • in the end we got "otel.system.filesystem.utilization" and "system.disk.in_use"

vs.

  • Before this PR, in the end we got "otel.system.filesystem.utilization" and "otel.system.disk.in_use"
  • With the current PR, in the end we got "system.filesystem.utilization", "system.disk.in_use", "otel.system.filesystem.utilization" and "otel.system.disk.in_use"

Both of ^ are wrong. In short, we prepend otel. to the original metrics, but don't prepend it to the remapped metrics.

Sorry for the confusion, I was also trying to understand the old behavior.

Copy link
Contributor Author

@gbbr gbbr Aug 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you are misunderstanding the current PR. It works exactly like you expect (in your "old code" example above). I have simplified the tests for you to make it easier for you to understand. Feel free to set up a call if it's still not clear.

Now, in the tests:

  • The in value contains the original metric
  • The out value contains any new (!) metrics added. In the case of system.* metrics you will notice there is a copy of the original.
  • The test asserts that the original metric (from in, not out) is prepended with otel.* when needed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I'm not mistaken: using the same example, now after remapMetrics we're getting 3 metrics: "otel.system.filesystem.utilization", "system.disk.in_use" and "system.filesystem.utilization"?

If so, the last one "system.filesystem.utilization" shouldn't be there

newm := all.AppendEmpty()
m.CopyTo(newm)
}
// and we prepend all of them with the otel.* namespace namespace
m.SetName("otel." + m.Name())
}

Expand Down
31 changes: 21 additions & 10 deletions pkg/otlp/metrics/metrics_remapping_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -275,16 +275,27 @@ func TestRemapMetrics(t *testing.T) {
out: []pmetric.Metric{metric("container.net.rcvd.packets", point{f: 15})},
},
} {
lena := dest.Len()
checkprefix := strings.HasPrefix(tt.in.Name(), "system.") || strings.HasPrefix(tt.in.Name(), "process.")
remapMetrics(dest, tt.in)
if checkprefix {
require.True(t, strings.HasPrefix(tt.in.Name(), "otel."), "system.* and process.* metrics need to be prepended with the otel.* namespace")
}
require.Equal(t, dest.Len()-lena, len(tt.out), "unexpected number of metrics added")
for i, out := range tt.out {
require.Equal(t, out, dest.At(dest.Len()-len(tt.out)+i))
}
t.Run("", func(t *testing.T) {
lenin := dest.Len()
issystem := strings.HasPrefix(tt.in.Name(), "system.")
out := tt.out
if issystem {
// the original system.* metrics need to be preserved
newm := pmetric.NewMetric()
tt.in.CopyTo(newm)
out = append(out, newm)
}
isprocess := strings.HasPrefix(tt.in.Name(), "process.")
remapMetrics(dest, tt.in)
if issystem || isprocess {
// system.* and process.* metrics have been prepended with the otel.* namespace
require.True(t, strings.HasPrefix(tt.in.Name(), "otel."), "system.* and process.* metrics need to be prepended with the otel.* namespace")
}
require.Equal(t, dest.Len()-lenin, len(out), "unexpected number of metrics added")
for i, o := range out {
require.Equal(t, o, dest.At(dest.Len()-len(out)+i))
}
})
}

}
Expand Down