Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[connector/count] Add outline and documentation #18037

Merged
merged 7 commits into from
Jan 26, 2023
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,8 @@ cmd/tracegen/ @open-telemetry/collect

confmap/provider/s3provider/ @open-telemetry/collector-contrib-approvers @Aneurysm9

connector/countconnector/ @open-telemetry/collector-contrib-approvers @djaglowski @jpkrohling

examples/demo/ @open-telemetry/collector-contrib-approvers @open-telemetry/collector-approvers

exporter/alibabacloudlogserviceexporter/ @open-telemetry/collector-contrib-approvers @shabicheng @kongluoxing @qiansheng91
Expand Down
3 changes: 2 additions & 1 deletion .github/ISSUE_TEMPLATE/bug_report.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ body:
- cmd/telemetrygen
- cmd/tracegen
- confmap/provider/s3provider
- connector/count
- examples/demo
- exporter/alibabacloudlogservice
- exporter/awscloudwatchlogs
Expand Down Expand Up @@ -228,7 +229,7 @@ body:
description: Please provide as much detail as you reasonably can.
value: |
## Description

## Steps to Reproduce

## Expected Result
Expand Down
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/feature_request.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ body:
- cmd/telemetrygen
- cmd/tracegen
- confmap/provider/s3provider
- connector/count
- examples/demo
- exporter/alibabacloudlogservice
- exporter/awscloudwatchlogs
Expand Down
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/other.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ body:
- cmd/telemetrygen
- cmd/tracegen
- confmap/provider/s3provider
- connector/count
- examples/demo
- exporter/alibabacloudlogservice
- exporter/awscloudwatchlogs
Expand Down
4 changes: 4 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,10 @@ updates:
directory: "/confmap/provider/s3provider"
schedule:
interval: "weekly"
- package-ecosystem: "gomod"
directory: "/connector/count"
schedule:
interval: "weekly"
- package-ecosystem: "gomod"
directory: "/examples/demo/client"
schedule:
Expand Down
1 change: 1 addition & 0 deletions connector/countconnector/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
include ../../Makefile.Common
213 changes: 213 additions & 0 deletions connector/countconnector/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,213 @@
# Count Connector
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like how this readme is written, mostly demonstrating how connectors work. At the same time, I wonder if this wouldn't be confusing to users of this connector specifically.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I'm reworking this into a general connectors readme and a dedicated count connector readme.


| Status | |
|------------------------- |---------------------------------------------------------- |
| Stability | [in development] |
| Supported pipeline types | See [Supported Pipeline Types](#supported-pipeline-types) |
djaglowski marked this conversation as resolved.
Show resolved Hide resolved
| Distributions | [] |

The `countconnector` can be used to count spans, data points, or log records.
djaglowski marked this conversation as resolved.
Show resolved Hide resolved

## Supported Pipeline Types

Connectors are used as _both_ an exporter and as a receiver, in separate pipelines.
The pipeline in which a connector is used as an exporter is referred to below
as the "Exporter pipeline". Likewise, the pipeline in which the connector is
used as a receiver is referred to below as the "Receiver pipeline".

| Exporter pipeline | Receiver pipeline | Description | Default Metric Name |
| ----------------- | ----------------- | ---------------------------------- | ------------------------- |
| traces | metrics | Counts the number of spans. | `trace.span.count` |
| metrics | metrics | Counts the number of log records. | `metric.data_point.count` |
| logs | metrics | Counts the number of data points. | `log.record.count` |

## Configuration

Connectors are defined within a dedicated `connectors` section at the top level of the collector config.

The count connector may be used with default settings.

```yaml
receivers:
foo:
exporters:
bar:
connectors:
count:
```

Optionally, the names of the metrics emitted by the connector may be customized.

```yaml
receivers:
foo:
exporters:
bar:
connectors:
count:
traces:
metric_name: my.span.count
metrics:
metric_name: my.data_point.count
logs:
metric_name: my.log_record.count
```

### Use the connector in pipelines

A connector _is_ an exporter _and_ a receiver. It must be used as both, in separate pipelines.

```yaml
receivers:
foo:
exporters:
bar:
connectors:
count:
service:
pipelines:
traces:
receivers: [foo]
exporters: [count]
metrics:
receivers: [count]
exporters: [bar]
```

Connectors can be used alongside other exporters.

```yaml
receivers:
foo:
exporters:
bar/traces_backend:
bar/metrics_backend:
connectors:
count:
service:
pipelines:
traces:
receivers: [foo]
exporters: [bar/traces_backend, count]
metrics:
receivers: [count]
exporters: [bar/metrics_backend]
```

Connectors can be used alongside other receivers.

```yaml
receivers:
foo/traces:
foo/metrics:
exporters:
bar:
connectors:
count:
service:
pipelines:
traces:
receivers: [foo/traces]
exporters: [count]
metrics:
receivers: [foo/metrics, count]
exporters: [bar]
```

A connector can be an exporter from multiple pipelines.

```yaml
receivers:
foo/traces:
foo/metrics:
foo/logs:
exporters:
bar/traces_backend:
bar/metrics_backend:
bar/logs_backend:
connectors:
count:
service:
pipelines:
traces:
receivers: [foo/traces]
exporters: [bar/traces_backend, count]
metrics:
receivers: [foo/metrics]
exporters: [bar/metrics_backend, count]
logs:
receivers: [foo/logs]
exporters: [bar/logs_backend, count]
metrics/counts:
receivers: [count]
exporters: [bar/metrics_backend]
```

A connector can be a receiver in multiple pipelines.

```yaml
receivers:
foo/traces:
foo/metrics:
exporters:
bar/traces_backend:
bar/metrics_backend:
bar/metrics_backend/2:
connectors:
count:
service:
pipelines:
traces:
receivers: [foo/traces]
exporters: [bar/traces_backend, count]
metrics:
receivers: [count]
exporters: [bar/metrics_backend]
metrics/2:
receivers: [count]
exporters: [bar/metrics_backend/2]
```

Multiple connectors can be used in sequence.

```yaml
receivers:
foo:
exporters:
bar:
connectors:
count:
count/the_counts:
service:
pipelines:
traces:
receivers: [foo]
exporters: [count]
metrics:
receivers: [count]
exporters: [bar/metrics_backend, count/the_counts]
metrics/count_the_counts:
receivers: [count/the_counts]
exporters: [bar]
```

A connector can only be used in a pair of pipelines when it supports the combination of _exporter type_ and _receiver type_.

```yaml
receivers:
foo:
exporters:
bar:
connectors:
count:
service:
pipelines:
traces:
receivers: [foo]
exporters: [count]
logs:
receivers: [count] # Invalid. The count connector can only be used as a receiver in metrics pipelines.
exporters: [bar]
```

[in development]:https://github.com/open-telemetry/opentelemetry-collector#in-development
19 changes: 19 additions & 0 deletions connector/countconnector/config.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
// Copyright The OpenTelemetry Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package countconnector // import "github.com/open-telemetry/opentelemetry-collector-contrib/connector/countconnector"

type Config struct {
// TODO
}
15 changes: 15 additions & 0 deletions connector/countconnector/doc.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
// Copyright The OpenTelemetry Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package countconnector // import "github.com/open-telemetry/opentelemetry-collector-contrib/connector/countconnector"
75 changes: 75 additions & 0 deletions connector/countconnector/factory.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
// Copyright The OpenTelemetry Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package countconnector // import "github.com/open-telemetry/opentelemetry-collector-contrib/connector/countconnector"

import (
"context"

"go.opentelemetry.io/collector/component"
"go.opentelemetry.io/collector/connector"
"go.opentelemetry.io/collector/consumer"
)

const (
typeStr = "count"
stability = component.StabilityLevelDevelopment
scopeName = "otelcol/countconnector"
)

// NewFactory returns a ConnectorFactory.
func NewFactory() connector.Factory {
return connector.NewFactory(
typeStr,
createDefaultConfig,
connector.WithTracesToMetrics(createTracesToMetrics, component.StabilityLevelDevelopment),
connector.WithMetricsToMetrics(createMetricsToMetrics, component.StabilityLevelDevelopment),
connector.WithLogsToMetrics(createLogsToMetrics, component.StabilityLevelDevelopment),
)
}

// createDefaultConfig creates the default configuration.
func createDefaultConfig() component.Config {
return &Config{}
}

// createTracesToMetrics creates a traces to metrics connector based on provided config.
func createTracesToMetrics(
_ context.Context,
set connector.CreateSettings,
cfg component.Config,
nextConsumer consumer.Metrics,
) (connector.Traces, error) {
return nil, nil
}

// createMetricsToMetrics creates a metrics connector based on provided config.
func createMetricsToMetrics(
_ context.Context,
set connector.CreateSettings,
cfg component.Config,
nextConsumer consumer.Metrics,
) (connector.Metrics, error) {
return nil, nil
}

// createLogsToMetrics creates a logs to metrics connector based on provided config.
func createLogsToMetrics(
_ context.Context,
set connector.CreateSettings,
cfg component.Config,
nextConsumer consumer.Metrics,
) (connector.Logs, error) {
return nil, nil
}
Loading