Add CloudFoundry Resource Attributes

[KarstenSchnitter]

Proposal for #622:

Using the CloudFoundry documentation, provides a set of resource attribute describing CloudFoundry workloads. Both applications as well as system components are addressed. Helpful resources:

• https://github.com/cloudfoundry/loggregator-api#v2-envelope
• https://docs.cloudfoundry.org/devguide/deploy-apps/environment-variable.html#VCAP-APPLICATION
• https://bosh.io/docs/jobs/#properties-spec
• https://docs.cloudfoundry.org/devguide/deploy-apps/streaming-logs.html

Changes

Introduces a resource attributes convention for CloudFoundry. Defines resource attributes to identify CloudFoundry applications and system components, enabling and improving OpenTelemetry support. The proposed attributes are selected to provide an as small as possible payload. The changes follow the proposal in #622.

Merge requirement checklist

• CONTRIBUTING.md guidelines followed.
• CHANGELOG.md updated for non-trivial changes.
• schema-next.yaml updated with changes to existing conventions.

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: KarstenSchnitter / name: Karsten Schnitter (9ce2881, fe5afa0, 7a9b0b4, d20c70f, 542124d, d77ec4b, ed7ff7a, 99a037d, 060b67e, 887ade0, 7bd0436, d319458, f4efa59, 402c93e, d41832c, 736bee4, e7ac8f2, 742e03b, 5da2eac, 99b1cb8, 54d7997, 1183038, ad85e02, aa62acd, 2313a54, b5335d5, 2f991a8, 4808dfe, 4cb1d4e, e114dee, 643181a, 1e339c5, 2b8712f, 5f518c4, b2da4c9, c0b0a0a)
• ✅ login: joaopgrassi / name: Joao Grassi (d722e21)

[KarstenSchnitter]

This PR is meant as a proposal for discussion around #622. I am unsure about when and how to fulfil the merge requirements checklist. A changelog entry seems to be necessary. I do not know, how to verify the contribution guidelines. Locally a make check ran without issues.

[joaopgrassi]

@KarstenSchnitter

A changelog entry seems to be necessary

Yes, please add an entry to the CHANGELOG.md file. You can see an example here: #615

I do not know, how to verify the contribution guidelines. Locally a make check ran without issues.

I approved the CI run and some checks are failing. Not sure how the make check passed, but you can run them separately and they for sure should also fail locally.

[KarstenSchnitter]

@joaopgrassi I will take care of the issues and the changelog. Is there anything else?

[KarstenSchnitter]

@joaopgrassi I noticed some unmerged changes on my side. Hopefully, this resolves the checks.

[joaopgrassi]

I only looked at a high level yesterday, and I'm not experienced with CloudFoundry, so I will need to read a bit to understand the context so I can review it. I will try to do it in the upcoming days.

[FWinkler79]

I think we should review the naming of some of the attributes.
The OTEL colleagues should weigh in here, as they probably know best about the usual naming conventions.

[chombium]

Hi @FWinkler79,
thanks for the tips and the review. I'm one of the maintainers in the Cloud Foundry's logging and metrics team. We've discussed with @KarstenSchnitter and @ctlong and we also had the same doubts about the source_id, source_type and instance_id labels. We've added them as they are currently used in the Cloud Foundry's logging and metrics stack, but after checking the existing conventions, I've found the attributes-registry/source.md and general/attributes.md and I think that we should change them to source.id, source.type and instance.id.

There is one more open question to clear: In Cloud Foundry the source type label is only used for logs and not for metrics and traces. We are not sure what do do about it and if we need to define it in some specific way. @joaopgrassi Any ideas about this?

[KarstenSchnitter]

I provided a revised version with the fields renamed as suggested by @FWinkler79. Furthermore, I added some clarification on the source_type, especially the requirement level. For the three use-cases in #622 I added a comment, how to obtain the appropriate value for use-case 1, where the value needs to be set to a static value. This is the reason, why I think source.typeshould be part of this convention, but only apply to logs as now reflected in the requirement level. The branch to be merged was rebased on main in the process.

[FWinkler79]

Thanks for your reply and changes.

@chombium:

There is one more open question to clear: In Cloud Foundry the source type label is only used for logs and not for metrics and traces. We are not sure what do do about it and if we need to define it in some specific way. @joaopgrassi Any ideas about this?

I am not 100% sure, but I recently updated our internal version of the OTEL SemConv to the latest version and noticed that the attributes for different signal types are now (rather) cleanly separated into log-attributes, trace-attributes, etc.
Not sure the documentation already properly reflects that, but at least the YAML specifications do.
E.g. you can find the entire yaml-based model files here. There you find folders for logs, metrics, traces and recource attributes.
To avoid data duplication the common "base attributes" that are shared between signal types are now maintained under the "registry" folder. It's not entirely cleaned up yet, but I think that's the intention.

For example, in the registry folder you can find http.yaml, in the trace folder you also find http.yaml. The trace-related HTTP attributes simply reference HTTP attributes defined in the file under registry.

In a similar way, I think you could define log-specific attributes.

[chombium]

Thanks for the prompt change @KarstenSchnitter and for the suggestions and digging into the definitions @FWinkler79. @joaopgrassi Is there something else to be done apart from the suggestions about the source type parameter?

[github-actions]

This PR was marked stale due to lack of activity. It will be closed in 7 days.

[KarstenSchnitter]

@chombium , @ctlong, can you provide a status of your deliberations? Maybe we can agree on an initial convention and improve over time?

[joaopgrassi]

Hi @KarstenSchnitter !

We changed how the CHANGELOG.md is managed. Please take a look at https://github.com/open-telemetry/semantic-conventions/blob/main/CONTRIBUTING.md#adding-a-changelog-entry to see what needs to be done. Sorry for the disruption.

[KarstenSchnitter]

There is an issue on the main branch with the Github link to the registry, which has a 404 error:

https://github.com/open-telemetry/semantic-conventions/blob/b855a36cf4b8a3c5461ad16fa4ace0c615d11c2a/CONTRIBUTING.md?plain=1#L225C15-L225C94

This fails the tests, but is not caused by this PR.

[KarstenSchnitter]

The latest merge should fix the issue with the main branch.

[KarstenSchnitter]

@lmolkova Thanks you for the review. I am going to remove the source.type with my next commits.

[KarstenSchnitter]

I removed the source.type with the last commit.

[lmolkova]

LGTM, suggesting a few cosmetic changes

[KarstenSchnitter]

Thanks everyone for all the help, work, and dedication you provided to include this change into the repository.