k8sattributes Extract labels to a Map

[omegion]

Component(s)

processor/k8sattributes

Describe the issue you're reporting

Hi,

I am trying to extract Kubernetes labels to a map so that I can display them as nested JSON in the log body. However, they are all extracted as flat parameters. Any way to convert them to a Map?

My config:

      k8sattributes:
        extract:
          labels:
            - tag_name: k8s.labels.$$1
              key_regex: (.*)
              from: pod

This will extract Kubernetes pod labels to:

     -> k8s.labels.app: Str(kindnet)
    -> k8s.labels.controller-revision-hash: Str(68bf578d5f)
    -> k8s.labels.k8s-app: Str(kindnet)
    -> k8s.labels.pod-template-generation: Str(1)
    -> k8s.labels.tier: Str(node)

However, I would like to put each label in into Map({"labels":{}}) with their keys, is this possible?

[github-actions]

Pinging code owners:

• processor/k8sattributes: @dmitryax @rmfitzpatrick @fatsheep9146 @TylerHelmuth

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[omegion]

/label help-wanted

[TylerHelmuth]

This isn't possible with the k8sattributesprocessor (and I don't think we'd add it), but you can use the transformprocessor to achieve this. After the k8sattributes processor has extracted the labels:

transform:
  error_mode: ignore
  log_statements:
    - context: resource
      statements:
        - set(attributes["labels"], attributes)
        # you made need to remove unwanted keys
        - delete_key(attributes["labels"], "service.name")
        - ...
    - context: log
      statements:
        - set(body, attributes["labels") 

I've not tests that but its the general idea.

[omegion]

This isn't possible with the k8sattributesprocessor (and I don't think we'd add it), but you can use the transformprocessor to achieve this. After the k8sattributes processor has extracted the labels:

transform:
  error_mode: ignore
  log_statements:
    - context: resource
      statements:
        - set(attributes["labels"], attributes)
        # you made need to remove unwanted keys
        - delete_key(attributes["labels"], "service.name")
        - ...
    - context: log
      statements:
        - set(body, attributes["labels") 

I've not tests that but its the general idea.

Thanks for the example, however, it does not seem straight forward solution. Along with pod labels, we also extract namespace labels, this makes it hard to filter the labels.

[crobert-1]

Thanks for the example, however, it does not seem straight forward solution. Along with pod labels, we also extract namespace labels, this makes it hard to filter the labels.

Can you share the current challenge you're facing with filtering labels when extracting namespaces?

Telemetry operations can be confusing and hard to figure out, but the challenge the collector faces is doing the same kind operation in different contexts and situations. The proposed solution (using the k8sattributes and transform processor) is the simplest way to accomplish this with the k8s context as well as transforming the attributes of your data. The alternative is duplicating the same transformation logic in every processor. This would be really hard to maintain, and potentially just as hard to understand as using multiple processors together.

[github-actions]

Pinging code owners for processor/transform: @TylerHelmuth @kentquirk @bogdandrutu @evan-bradley. See Adding Labels via Comments if you do not have permissions to add labels yourself.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• processor/k8sattributes: @dmitryax @rmfitzpatrick @fatsheep9146 @TylerHelmuth
• processor/transform: @TylerHelmuth @kentquirk @bogdandrutu @evan-bradley

See Adding Labels via Comments if you do not have permissions to add labels yourself.