Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove DogStatsD mode #2

Merged
merged 3 commits into from
Sep 23, 2020
Merged

Remove DogStatsD mode #2

merged 3 commits into from
Sep 23, 2020

Conversation

mx-psi
Copy link
Member

@mx-psi mx-psi commented Sep 21, 2020

What does this PR do?

  • Remove DogStatsD mode
  • Refactor and simplify configuration and code to accomodate this new change

Additional notes

@mx-psi mx-psi merged commit fcc9b77 into master Sep 23, 2020
@mx-psi mx-psi deleted the mx-psi/remove-dogstatsd branch September 23, 2020 10:37
mx-psi added a commit that referenced this pull request Oct 2, 2020
* Add DataDog exporter back from old fork
commit 99129fb96e29e9c1a92da00b7e3f8efcae8a31e8
Author: Pablo Baeyens <pablo.baeyens@datadoghq.com>
Date:   Thu Sep 3 18:10:28 2020 +0200

    Handle namespace at initialization time

commit babca25927926a60c0c416294af3aadf784d41b9
Author: Pablo Baeyens <pablo.baeyens@datadoghq.com>
Date:   Thu Sep 3 17:23:53 2020 +0200

    Initialize on a separate function
    This way the variables can be checked without worrying about the env

commit 24d0cb4cc566fa5313a8650c904a27bea68bf555
Author: Pablo Baeyens <pablo.baeyens@datadoghq.com>
Date:   Thu Sep 3 14:30:35 2020 +0200

    Check environment variables for unified service tagging

commit 6695f8297ab8b1fcae71b05acb027c4a0992e3a0
Author: Pablo Baeyens <pablo.baeyens@datadoghq.com>
Date:   Wed Sep 2 14:57:37 2020 +0200

    Add support for sending metrics through the API
    - Use datadog.Metric type for simplicity
    - Get host if unset

commit c366603
Author: Pablo Baeyens <pablo.baeyens@datadoghq.com>
Date:   Wed Sep 2 09:56:24 2020 +0200

    Disable Queue and Retry settings (#72)

    These are handled by the statsd package.
    OpenTelemetry docs are confusing and the default configuration (disabled)
    is different from the one returned by "GetDefault..." functions

commit a660b56
Author: Pablo Baeyens <pablo.baeyens@datadoghq.com>
Date:   Tue Sep 1 15:26:14 2020 +0200

    Add support for summary and distribution metric types (#65)

    * Add support for summary metric type

    * Add support for distribution metrics

    * Refactor metrics construction
    - Drop name in Metrics (now they act as Metric values)
    - Refactor constructor so that errors happen at compile-time

    * Report Summary total sum and count values
    Snapshot values are not filled in by OpenTelemetry

    * Report p00 and p100 as `.min` and `.max`
    This is more similar to what we do for our own non-additive type

    * Keep hostname if it has not been overridden

commit c95adc4
Author: Pablo Baeyens <pablo.baeyens@datadoghq.com>
Date:   Thu Aug 27 13:00:02 2020 +0200

    Update dependencies and `make gofmt`
    The collector was updated to 0.9.0 upstream

commit 20afb0e
Author: Pablo Baeyens <pablo.baeyens@datadoghq.com>
Date:   Wed Aug 26 18:25:49 2020 +0200

    Refactor configuration (#45)

    * Refactor configuration
    * Implement telemetry and tags configuration handling
    * Update example configuration and README file
    Co-authored-by: Kylian Serrania <kylian.serrania@datadoghq.com>

commit fdc98b5
Author: Pablo Baeyens <pablo.baeyens@datadoghq.com>
Date:   Fri Aug 21 11:09:08 2020 +0200

    Initial DogStatsD implementation (#15)

    Initial metrics exporter through DogStatsD with support for all metric types but summary and distribution

commit e848a60
Author: Pablo Baeyens <pablo.baeyens@datadoghq.com>
Date:   Fri Aug 21 10:42:45 2020 +0200

    Bump collector version

commit 58be9a4
Author: Pablo Baeyens <pablo.baeyens@datadoghq.com>
Date:   Thu Aug 6 10:04:32 2020 +0200

    Address linter

commit 695430c
Author: Pablo Baeyens <pablo.baeyens@datadoghq.com>
Date:   Tue Aug 4 13:28:01 2020 +0200

    Fix field name error

    MetricsEndpoint was renamed to MetricsURL

commit 168b319
Author: Pablo Baeyens <pablo.baeyens@datadoghq.com>
Date:   Mon Aug 3 11:05:01 2020 +0200

    Create initial outline for Datadog exporter (#1)

    * Add support for basic configuration options
    * Documents configuration options

* go mod tidy

* Address feedback from upstream PR we did not merge (#1)

* Backport changes from upstream PR

Remove `err` from MapMetrics

* Remove usage of pdatautil

* Fix tests

* Use TCPAddr

* Review which functions should be private

* Remove DogStatsD mode (#2)

* Remove DogStatsD mode

* go mod tidy

* Remove mentions to DogStatSD

* Improve test coverage (#3)

* Improve test coverage

Added unit tests for
- API key censoring
- Hostname
- Metrics exporter

Renamed test and implementation files for consistency

* Add one additional test

* Remove client validation (#6)

The zorkian API does not validate the API key unless you also have
an application key, even though the endpoint works without it.

I am removing this validation until this gets fixed on the zorkian library

* Keep only configuration and factory methods

Following the contribution guidelines we need to make a first PR
with this

* Use latest version of collector

* Remove `report_percentiles` option
It is not being used as of now until the OTLP metrics format
stabilizes and we have a Summary type metric again

* Correct configuration

The API key is now a required setting

* Remove test not relevant for this PR

* Remove unnecessary imports after removing test

* Address review comment

* Apply suggestions from code review

Co-authored-by: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com>

* Separate documentation into two examples
One example with the minimal configuration, for sending to `datadoghq.com`
and a second one for sending to `datadoghq.eu`

Co-authored-by: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com>
mackjmr pushed a commit that referenced this pull request Nov 19, 2024
… Histo --> Histogram (open-telemetry#33824)

## Description

This PR adds a custom metric function to the transformprocessor to
convert exponential histograms to explicit histograms.

Link to tracking issue: Resolves open-telemetry#33827

**Function Name**
```
convert_exponential_histogram_to_explicit_histogram
```

**Arguments:**

- `distribution` (_upper, midpoint, uniform, random_)
- `ExplicitBoundaries: []float64`

**Usage example:**

```yaml
processors:
  transform:
    error_mode: propagate
    metric_statements:
    - context: metric
      statements:
        - convert_exponential_histogram_to_explicit_histogram("random", [10.0, 20.0, 30.0, 40.0, 50.0, 60.0, 70.0, 80.0, 90.0, 100.0]) 
```

**Converts:**

```
Resource SchemaURL: 
ScopeMetrics #0
ScopeMetrics SchemaURL: 
InstrumentationScope  
Metric #0
Descriptor:
     -> Name: response_time
     -> Description: 
     -> Unit: 
     -> DataType: ExponentialHistogram
     -> AggregationTemporality: Delta
ExponentialHistogramDataPoints #0
Data point attributes:
     -> metric_type: Str(timing)
StartTimestamp: 1970-01-01 00:00:00 +0000 UTC
Timestamp: 2024-07-31 09:35:25.212037 +0000 UTC
Count: 44
Sum: 999.000000
Min: 40.000000
Max: 245.000000
Bucket (32.000000, 64.000000], Count: 10
Bucket (64.000000, 128.000000], Count: 22
Bucket (128.000000, 256.000000], Count: 12
        {"kind": "exporter", "data_type": "metrics", "name": "debug"}
```

**To:**

```
Resource SchemaURL: 
ScopeMetrics #0
ScopeMetrics SchemaURL: 
InstrumentationScope  
Metric #0
Descriptor:
     -> Name: response_time
     -> Description: 
     -> Unit: 
     -> DataType: Histogram
     -> AggregationTemporality: Delta
HistogramDataPoints #0
Data point attributes:
     -> metric_type: Str(timing)
StartTimestamp: 1970-01-01 00:00:00 +0000 UTC
Timestamp: 2024-07-30 21:37:07.830902 +0000 UTC
Count: 44
Sum: 999.000000
Min: 40.000000
Max: 245.000000
ExplicitBounds #0: 10.000000
ExplicitBounds #1: 20.000000
ExplicitBounds #2: 30.000000
ExplicitBounds #3: 40.000000
ExplicitBounds #4: 50.000000
ExplicitBounds #5: 60.000000
ExplicitBounds #6: 70.000000
ExplicitBounds #7: 80.000000
ExplicitBounds #8: 90.000000
ExplicitBounds #9: 100.000000
Buckets #0, Count: 0
Buckets #1, Count: 0
Buckets #2, Count: 0
Buckets #3, Count: 2
Buckets #4, Count: 5
Buckets #5, Count: 0
Buckets #6, Count: 3
Buckets #7, Count: 7
Buckets #8, Count: 2
Buckets #9, Count: 4
Buckets #10, Count: 21
        {"kind": "exporter", "data_type": "metrics", "name": "debug"}
```

### Testing

- Several unit tests have been created. We have also tested by ingesting
and converting exponential histograms from the `statsdreceiver` as well
as directly via the `otlpreceiver` over grpc over several hours with a
large amount of data.

- We have clients that have been running this solution in production for
a number of weeks.

### Readme description:

### convert_exponential_hist_to_explicit_hist

`convert_exponential_hist_to_explicit_hist([ExplicitBounds])`

the `convert_exponential_hist_to_explicit_hist` function converts an
ExponentialHistogram to an Explicit (_normal_) Histogram.

`ExplicitBounds` is represents the list of bucket boundaries for the new
histogram. This argument is __required__ and __cannot be empty__.

__WARNING:__

The process of converting an ExponentialHistogram to an Explicit
Histogram is not perfect and may result in a loss of precision. It is
important to define an appropriate set of bucket boundaries to minimize
this loss. For example, selecting Boundaries that are too high or too
low may result histogram buckets that are too wide or too narrow,
respectively.

---------

Co-authored-by: Kent Quirk <kentquirk@gmail.com>
Co-authored-by: Tyler Helmuth <12352919+TylerHelmuth@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants