Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[receiver/statsd]Add support for DataDog distribution metrics type #24768

Closed
anuprout opened this issue Aug 1, 2023 · 5 comments
Closed

[receiver/statsd]Add support for DataDog distribution metrics type #24768

anuprout opened this issue Aug 1, 2023 · 5 comments
Assignees
Labels
enhancement New feature or request receiver/statsd statsd related issues

Comments

@anuprout
Copy link
Contributor

anuprout commented Aug 1, 2023

Component(s)

receiver/statsd

Is your feature request related to a problem? Please describe.

The DataDog supported statsd metrics have an additional metric type distribution which is denoted by d in the statsd protocol. The Otel statsd receivers currently recognizes metric type GAUGE(g), TIMER(ms),COUNTER(c) & HISTOGRAM(h). This causes the distribution metrics sent from a DataDog statsd client to be dropped by the receiver.

Below are some examples of the distribution metrics sent from a DataDog statsd client.
test_distribution_metric:1|d|#tag1-key:tag1-value,tag2-key:tag2-value
test_distribution_metric:2|d|#tag1-key:tag1-value,tag2-key:tag2-value
test_distribution_metric:3|d|#tag1-key:tag1-value,tag2-key:tag2-value

Describe the solution you'd like

The distribution metrics can be parsed by the receiver as histograms . The DataDog exporter converts the Otel histograms to DataDog distributions by default.

When the metric payload is parsed, the metrics with type d can be treated as histograms which would wrap all the functionalities provided for histogram metrics.

Describe alternatives you've considered

The distribution metrics can be parsed by the receiver similar to the histograms where all the configs exposed for a histogram metric can be exposed for distribution metrics as well with default observer_type as histogram.

Sample receiver config.

receivers:
  statsd:
  statsd/2:
    endpoint: "localhost:8127"
    aggregation_interval: 70s
    enable_metric_type: true
    is_monotonic_counter: false
    timer_histogram_mapping:
      - statsd_type: "distribution"
        observer_type: "histogram"
        histogram: 
          max_size: 100 

Additional context

No response

@anuprout anuprout added enhancement New feature or request needs triage New item requiring triage labels Aug 1, 2023
@anuprout anuprout changed the title Add support for DataDog distribution metrics type [receiver/statsd]Add support for DataDog distribution metrics type Aug 1, 2023
@github-actions github-actions bot added the receiver/statsd statsd related issues label Aug 1, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Aug 1, 2023

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@mx-psi
Copy link
Member

mx-psi commented Aug 2, 2023

My recommendation (speaking as a Datadog exporter codeowner here) would be to map these to an OTLP exponential histogram (or at least have the option to do so).

@anuprout
Copy link
Contributor Author

anuprout commented Aug 3, 2023

@mx-psi the statsd receiver, translates the histograms to OTLP exponential histogram with observer_type: "histogram". The same can be used for distribution type metrics as well. I would wait for the comments from the receiver code owners on this.

@pedro-stanaka
Copy link

pedro-stanaka commented Aug 4, 2023

I would like to see this implemented as well and can try taking a stab at it, if maintainers think it is okay to move forward with the solution proposed by @anuprout.

cc @jmacd

@evan-bradley evan-bradley removed the needs triage New item requiring triage label Sep 6, 2023
evan-bradley pushed a commit that referenced this issue Sep 7, 2023
…26107)

**Description:** <Describe what has changed.>
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->

Add support for statsd `distribution` type metrics . 

**Link to tracking Issue:** <Issue number if
applicable>[24768](#24768)

**Testing:** <Describe what testing was performed and which tests were
added.>
test config:
```
receivers:
  statsd:  
    endpoint: "localhost:8125"
    aggregation_interval: 10s
    enable_metric_type: true
    is_monotonic_counter: true
    timer_histogram_mapping:
      - statsd_type: "distribution"
        observer_type: "histogram"
        histogram: 
          max_size: 50        
 
exporters:
  logging:
    verbosity: detailed
  
service:
  pipelines:
    metrics:
      receivers: [statsd]
      processors: []
      exporters: [logging]
       
```
test data:
```
test.statsdreceiver.dist:1|d|#tag1:Val1
test.statsdreceiver.dist:2|d|#tag1:Val1
test.statsdreceiver.dist:3|d|#tag1:Val1
```
output:
```
ResourceMetrics #0
Resource SchemaURL:
ScopeMetrics #0
ScopeMetrics SchemaURL:
InstrumentationScope otelcol/statsdreceiver 0.83.0-dev
Metric #0
Descriptor:
     -> Name: test.statsdreceiver.dist
     -> Description:
     -> Unit:
     -> DataType: ExponentialHistogram
     -> AggregationTemporality: Delta
ExponentialHistogramDataPoints #0
Data point attributes:
     -> metric_type: Str(distribution)
     -> tag1: Str(Val1)
StartTimestamp: 2023-08-25 15:49:29.595594 +0000 UTC
Timestamp: 2023-08-25 15:49:39.596418 +0000 UTC
Count: 3
Sum: 6.000000
Min: 1.000000
Max: 3.000000
Bucket (0.957603, 1.000000], Count: 1
Bucket (1.000000, 1.044274], Count: 0
Bucket (1.044274, 1.090508], Count: 0
Bucket (1.090508, 1.138789], Count: 0
Bucket (1.138789, 1.189207], Count: 0
Bucket (1.189207, 1.241858], Count: 0
Bucket (1.241858, 1.296840], Count: 0
Bucket (1.296840, 1.354256], Count: 0
Bucket (1.354256, 1.414214], Count: 0
Bucket (1.414214, 1.476826], Count: 0
Bucket (1.476826, 1.542211], Count: 0
Bucket (1.542211, 1.610490], Count: 0
Bucket (1.610490, 1.681793], Count: 0
Bucket (1.681793, 1.756252], Count: 0
Bucket (1.756252, 1.834008], Count: 0
Bucket (1.834008, 1.915207], Count: 0
Bucket (1.915207, 2.000000], Count: 1
Bucket (2.000000, 2.088548], Count: 0
Bucket (2.088548, 2.181015], Count: 0
Bucket (2.181015, 2.277577], Count: 0
Bucket (2.277577, 2.378414], Count: 0
Bucket (2.378414, 2.483716], Count: 0
Bucket (2.483716, 2.593679], Count: 0
Bucket (2.593679, 2.708511], Count: 0
Bucket (2.708511, 2.828427], Count: 0
Bucket (2.828427, 2.953652], Count: 0
Bucket (2.953652, 3.084422], Count: 1
	{"kind": "exporter", "data_type": "metrics", "name": "logging"}
```

**Documentation:** <Describe the documentation added.>
@anuprout
Copy link
Contributor Author

implemented in #26107

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request receiver/statsd statsd related issues
Projects
None yet
Development

No branches or pull requests

4 participants