Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add routing connector and processor #117

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

dpacheconr
Copy link

@dpacheconr dpacheconr commented Aug 12, 2024

Add routing connector and routing processor
Both allow users to send data to multiple NR accounts should they need to
Example configuration using routing connector

exporters:
  logging:
    verbosity: detailed
  otlp/masteraccount:
    endpoint: https://otlp.nr-data.net:4318
    headers:
      api-key: "xxxNRAL"
  otlp/account1:
    endpoint: https://otlp.nr-data.net:4318
    headers:
      api-key: "xxxNRAL"
  otlp/account2:
    endpoint: https://otlp.nr-data.net:4318
    headers:
      api-key: "xxxNRAL"

connectors:           
  routing/metrics:
    default_pipelines: [metrics/masteraccount]
    error_mode: propagate
    match_once: false
    table:
      - statement: route() where attributes["k8s.namespace.name"] == "account2"
        pipelines: [metrics/account2]

  routing/logs:
    default_pipelines: [logs/masteraccount]
    error_mode: propagate
    match_once: false
    table:
      - statement: route() where attributes["k8s.namespace.name"] == "account2"
        pipelines: [logs/account2]


    metrics:
      receivers:
        - hostmetrics
        - kubeletstats
        - prometheus
      processors:
        - metricstransform/hostmetrics_cpu
        - transform/truncate
        - filter/exclude_cpu_utilization
        - filter/exclude_memory_utilization
        - filter/exclude_memory_usage
        - filter/exclude_filesystem_utilization
        - filter/exclude_filesystem_usage
        - filter/exclude_filesystem_inodes_usage
        - filter/exclude_system_disk
        - filter/exclude_system_paging
        - filter/exclude_network
        - attributes/exclude_system_paging
        - resourcedetection/env
        - resourcedetection/cloudproviders
        - resource
        - k8sattributes
        - attributes/self
        - memory_limiter
        - groupbyattrs
        - batch
      exporters:
        - routing/metrics

    metrics/masteraccount:
      receivers: [routing/metrics] 
      exporters: [otlp/masteraccount]
    metrics/account1:
      receivers: [routing/metrics]
      exporters: [otlp/account1]
    metrics/account2:
      receivers: [routing/metrics]
      exporters: [otlp/account2]

    logs:
      receivers:
        - filelog
      processors:
        - transform/truncate
        - resource
        - k8sattributes
        - batch
        - groupbyattrs
      exporters:
        - routing/logs

    logs/masteraccount:
      exporters: [otlp/masteraccount]
      receivers: [routing/logs]
    logs/account1:
      receivers: [routing/logs]
      exporters: [otlp/account1]
    logs/account2:
      receivers: [routing/logs]
      exporters: [otlp/account2]

Add routing connector and processor
Both allow users to send data to multiple NR accounts should they need to
@dpacheconr dpacheconr requested a review from a team August 12, 2024 08:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant