Skip to content
This repository has been archived by the owner on Jul 31, 2023. It is now read-only.

Add runtime metrics support #1156

Merged

Conversation

brnhffmnn
Copy link
Contributor

This adds a metricproducer.Producer implementation under /plugin/ which enables the collection of runtime metrics.

Fixes #784

Copy link
Contributor

@rghetia rghetia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution. See comments inline.

//
// Supply ProducerOptions to configure the behavior of the Producer.
// An error might be returned, if creating metrics gauges fails.
func NewProducer(options ProducerOptions) (*Producer, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of creating producer and then require the user to register it, may be it can also register it.

func Enable(options RunMetricsOptions) error {
    // Based on options create and register cpu, mem  or both producers.
}
func Disable() {
    // find producer associated with the process and delete it from registry.
}

Restrict to a single producer per process.
After above changes there is no need to export Producer.

}

// ProducerOptions allows to configure Producer.
ProducerOptions struct {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be renamed to RunMetricOptions.

}

// Read reads the current runtime metrics.
func (c *Producer) Read() []*metricdata.Metric {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: rename 'c' to 'p'.

memStats := &memStats{}

// General
memStats.memAlloc, err = producer.createInt64GaugeEntry("mem_alloc", "Bytes of allocated heap objects", metricdata.UnitBytes)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

name should follow the convention used in opentelemetry-service

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please check if all names are ok with my last change.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the names look fine. I didn't see cpu_seconds similar to one in vmmetricreceiver

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CPU time is handled by prometheus/procfs in vmmetricsreceiver. I focused on what is available out-of-the-box.

@dprotaso
Copy link

dprotaso commented Sep 11, 2019

Any thoughts on adding GC metrics?

ie. https://golang.org/pkg/runtime/debug/#GCStats

The Prometheus GoCollector exports it this way
https://github.com/prometheus/client_golang/blob/master/prometheus/go_collector.go#L319-L328


Disable() // ensure no duplicate registrations
metricproducer.GlobalManager().AddProducer(producer)
return nil
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a race condition here. Without any mutex there is a possibility of adding two producers.
Maybe you can create a singleton with sync.Once and with mutex delete/add producer when it is enabled/disabled.
You can also create all entries once and simply based on boolean (memory/cpu) read those metrics.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't able to see a real data race, but it was inconsistent indeed. I adjusted it, so that there can only be one producer registered. But if you would enable/disable concurrently, the last one wins (obviously).

I didn't go with a singleton, so the producer instance can remain immutable and doesn't need to care about synchronizing on the options or anything like that.

@rghetia rghetia merged commit fa651b0 into census-instrumentation:master Oct 1, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Prometheus View of runtime.MemStats
4 participants