Provide clearer instructions on when it is appropriate to extract event information into semantic convention attributes

[TylerHelmuth]

What are you trying to achieve?

The Collector SIG maintains a Kubernetes Objects Receiver that received structured updates on any type of Kubernetes object (events, pods, etc). This receiver was made in response to the Event specification detailing the event should be in the body and that event.name and event.domain be applied.

While this works, in our use case it is difficult to correlate the data collected from this receiver to other related data. Traces, metrics, and other logs collected by the Collector in Kubernetes will include useful k8s-specific resource attributes that logs collected by the Kubernetes Objects Receiver will not. The data represented by those attributes is available in the body of the log produced by the receiver, but not in the equivalent semantic-convention resource attributes.

What did you expect to see?

We would like the specification to take a clearer stance on how events that contain semantic-convention-equivalent data in their body should treat that information. Options discussed in the SIG meetings:

1. Body for the payload + event.domain and event.name attributes
2. Body for the payload + event.domain and event.name attributes + attributes defined in the semantic convention extracted from the payload.

Option 1 is the current specification and the spec could be amended to reflect that semantic convention attributes in the payload should not be extracted. In our opinion, option 2 is preferred.

Additional context.

This issue was brought to the Specification SIG on 2023-07-05 and this issue is a continuation of that discussion.

In our exact example, we have a payload like

{
    "object": {
        "apiVersion": "events.k8s.io/v1",
        "deprecatedCount": 1,
        "deprecatedFirstTimestamp": "2023-08-31T19:51:04Z",
        "deprecatedLastTimestamp": "2023-08-31T19:51:04Z",
        "deprecatedSource": {
            "component": "kubelet",
            "host": "kind-control-plane"
        },
        "eventTime": null,
        "kind": "Event",
        "metadata": {
            "creationTimestamp": "2023-08-31T19:51:04Z",
            "managedFields": [
                {
                    "apiVersion": "v1",
                    "fieldsType": "FieldsV1",
                    "fieldsV1": {
                        "f:count": {},
                        "f:firstTimestamp": {},
                        "f:involvedObject": {},
                        "f:lastTimestamp": {},
                        "f:message": {},
                        "f:reason": {},
                        "f:source": {
                            "f:component": {},
                            "f:host": {}
                        },
                        "f:type": {}
                    },
                    "manager": "kubelet",
                    "operation": "Update",
                    "time": "2023-08-31T19:51:04Z"
                }
            ],
            "name": "testing-opentelemetry-collector-58bc7bb5c-fmkzd.17808fb4511301f2",
            "namespace": "default",
            "resourceVersion": "178081",
            "uid": "3a8f248c-b228-47a8-a561-772f0541dcff"
        },
        "note": "Stopping container opentelemetry-collector",
        "reason": "Killing",
        "regarding": {
            "apiVersion": "v1",
            "fieldPath": "spec.containers{opentelemetry-collector}",
            "kind": "Pod",
            "name": "testing-opentelemetry-collector-58bc7bb5c-fmkzd",
            "namespace": "default",
            "resourceVersion": "176163",
            "uid": "7c2d5936-9f04-4110-9fe0-a2c87d11c132"
        },
        "type": "Normal"
    },
    "type": "ADDED"
}

We would like to extract some k8s semantic convention attributes from the object.regarding field (and possibly others).

[tigrannajaryan]

1. Body for the payload + event.domain and event.name attributes
2. Body for the payload + event.domain and event.name attributes + attributes defined in the semantic convention extracted from the payload.

Option 1 is the current specification and the spec could be amended to reflect that semantic convention attributes in the payload should not be extracted. In our opinion, option 2 is preferred.

Option 2 sounds reasonable, but I am not sure we can mandate in a specification.

AFAIK option 2 is also fully compliant with the spec today. The spec does not prohibit additional attributes. In fact it explicitly allows them:

Event LogRecords have the attributes event.domain and event.name (and possibly other LogRecord attributes).

IMO, at best we can recommend Option 2 as a best practice somewhere in supplementary guidelines.

Are you looking for more in the spec? What's the goal here, why does the spec need to explicitly define this?

[TylerHelmuth]

The SIG discussion and this issue were @dmitryax and I looking for permission, according to the spec, to extract some values out of that JSON and add them as resource attributes. If you're saying the spec already supports that then we're good to go.

[tigrannajaryan]

Yes, I think the spec allows it. See the part that I quoted in bold. If it does not seem sufficient we can try to make it clearer.