Reconsidering what semantic conventions code generation should produce

[lmolkova]

Context

Today every (or almost every) language SIG generates semantic conventions from yaml files using Jinja templates.
Each SIG separates attributes based on the signal they appear on in one way or another:

• resource attributes vs all other attributes
• resource, spans, logs/events

While there are some deviations, most of jinja templates and codegen configurations are mostly consistent across SIGs.

Problem

We want attributes to be used across signals. E.g. user_agent.original is a span attribute on server and resource attribute in browser.
Code generation was already broken and is patched to keep the status quo until we find a better solution.

Questions to Otel SIGs:

How language SIGs see the 'perfect' outcome of code generation tooling?

1. How do we group attributes? Do we group them at all or generate all in one class?
2. How do we work with experimental vs stable attributes? Should we separate them into stable and preview artifacts?
3. Do we want semconv libs to have consistent approach/api across languages?
4. How do we guarantee that YAML refactoring or code generation tweaks would not change stable public API of semconv libs in the future even if we reconsider approach in the p1?
5. Do we even want to ship stable semantic convention libraries in each language?
6. anything else?

Language SIGs that generate semconv with Jinja templates:

• Java: separates resource vs all other attributes. @open-telemetry/java-maintainers
• JS: separates resource vs all other attributes @open-telemetry/javascript-maintainers
• Python: separates resource vs all other attributes @open-telemetry/python-maintainers
• .NET: separates resource vs trace @open-telemetry/dotnet-maintainers
• Go: separates resource, events, traces @open-telemetry/go-maintainers
• Collector: separates resources vs spans @open-telemetry/collector-maintainers
• Rust: separates resource vs all other @open-telemetry/rust-maintainers
• C++: separates resource vs all other attributes @open-telemetry/cpp-maintainers
• Erlang: separates resource, logs, traces @open-telemetry/erlang-maintainers
• Ruby: separates resource, traces @open-telemetry/ruby-maintainers
• PHP: separates resources vs all other @open-telemetry/php-maintainers
• Swift: separates resources vs all other @open-telemetry/swift-maintainers

[lmolkova]

/cc @lquerel in case you have some thoughts or ideas on how to change the existing basic semconv codegen

[lmolkova]

Proposal

Jinja templates still allow to customize things a lot, I'm proposing these answers as a default behavior, not as hard requirements:

1. Group attributes by root namespace.
   • server.address goes to ServerAttributes class/file.
   • http.request.method goes to HttpAttributes
   • attributes without namespaces MUST get one added (alternative is to move them to common Attributes class)
   • however the grouping is done, this decision is irreversible after first stable semconv artifact is shipped
2. Stable and/or experimental semconv artifacts: up to language SIG.
   • Jinja templates allow to filter based on stability already
   • If instrumentation libraries depend on semconv library, we need to have two versions/artifacts for stable and experimental semconv
3. Strict(ish) consistency across languages: no
   • it seems they are already mostly consistent and by providing sample templates and generation commands we can keep it this way.
   • Jinja templates are too hard to write, so having common examples would be great
4. Write validation tools preventing breaking changes. (API checks, ideally some checks in semconv repo)
   • Experiment with generating delta between vOld and vNew.
   • exclude specific attribute namespaces from code generation
5. Ship dedicated semconv libraries: up to language SIG.
   • SIGs may decide to provide source generators for user apps, or use shading/shared code in instrumentation libs

[lquerel]

@lmolkova, Here are some feedback based on my explorations with the Application Telemetry Schema project and the creation of a Client SDK Generator (similar to protobuf).

• To offer the best experience to their users, it seems important to me to allow enough freedom for language SIGs to respect the naming conventions and idioms of each language.
• Moving towards a 1 to many relationship between attributes, metrics, and telemetry signals or objects seems crucial and perfectly aligned with what I am trying to do around the Application Telemetry Schema proposal.
• Grouping attributes or metrics by root namespace or domain also seems to be a step in the right direction.
• For the project I am working on (Client SDK Generator), I also use a template engine compatible with Jinja2. However, I have implemented the following improvements with the aim of making life easier for implementors of Client SDKs, which I think could also be used in the context of this GH issue:
  • each language can define one or more templates organized in one or multiple sub-directories for generating the client SDKs.
  • two types of templates exist:
    • templates that apply to the entire Resolved Application Telemetry Schema (i.e., without reference, and where all override rules have been applied).
    • templates that apply to a specific instance of a telemetry object. For example, a template named span.jinja will be applied as many times as there are spans in the resolved application telemetry schema. A custom functionis exposed to allow template authors to define the name of the result file from applying this template to the specific instance of the span (or metric, log, event, etc.). This allows to have a different source code file for each root namespace for example.
  • a series of custom functions and filters are also exposed and accessible at the template level to make life easier for the authors of these templates. This also helps to unify certain functions.
  • a YAML configuration file can also be optionally added to parameterize the templates or to guide the generation tool. For example, an OTel data type to Language data type mapping table can be defined in this config file. This mapping will be used by a mapping function/filter exposed in the template to facilitate type conversions.

[dyladan]

JS splitting the resource attributes out from the other attributes is more of a historical accident than a considered decision. We're currently working on a 2.0 to fix a lot of our historical mistakes right now so I wouldn't worry too much about the current situation.

The primary concern for JS moving forward for something like this is code size. This means no namespaces in the generated code if we can avoid it, and ideally duplicated strings and substrings are kept to a minimum. Our ideal generated code would look something like this:

// declare all common substrings
const http = 'http';
const response = 'response';
... et cetera

export const HTTP_RESPONSE_STATUS_CODE = dotJoin(http, response, 'status_code'); // no need to declare single use strings separately

code written this way is friendly to tree shakers and code minifiers which is very important for JS, particularly in browser contexts. @MSNev may be able to expand on this or fill in details i've missed.

edit: answering some direct questions that ludmila asked in the JS sig doc:

1. How to group attributes? For us, the grouping is not really that important. Right now we don't do any grouping by file, but within the file we do have enums per-namespace. If we follow what I put above though, we would remove even that grouping. Having all names in the same file helps with intellisense discoverability.
2. do we need stable+preview semconv lib? It would be nice to have stable+preview+deprecated all available from the same library. We would need some way to mark what is stable and what is not, but there are methods for that. For example, we can use jsdoc comments to mark the deprecated/preview ones. Another possibility would be to include the semconv version number in the import path (e.g. import {} from '@otel/semconv/1.21 or similar) to guarantee backwards compatibility.

[jack-berg]

Group attributes by root namespace.

👍

Write validation tools preventing breaking changes. (API checks, ideally some checks in semconv repo)

Yeah I think we need checks in both the generation repos and in semantic-conventions. Ideally semconv would have strict checks preventing any breaking changes, but the compatibility rules might not be the same in semconv as in all languages, and so language code generation repos need to verify against their own rules.

[lmolkova]

Had a discussion at .NET SIG:

• They don't ship OTel Semconv regularly and don't depend on it from instrumentation libs but would like to start shipping it at some point.
• Diamond dependency is a concern (e.g. mySQL instr implements semconv vM and HTTP instr implements semconv vM+1). If one semconv package dependency version is resolved, they'd both pretend to be vM+1
• Grouping attributes by namespace makes sense
• Need to mark/separate experimental attributes

Great proposal from @martinjt :

• Generate semconv in version specific namespace, e.g. OTel.Semconv.v1.26.0.HttpAttributes.
• the semconv library would contain multiple namespaces for each version
• instr libraries and apps would use a class in a specific namespace that never changes until they are ready to update their instr to a new version - similar to this example https://github.com/martinjt/aks-otel-demo/blob/1efa48a57f4dc39d8d98a8b9203c155efa85badb/infra/AKSCluster.cs#L9-L11

Code generation may work this way (validation needed):

• don't touch old namespaces
• generate classes for the new version in the new namespace
• potentially separate experimental/stable attributes (maybe use new Experimental attribute in .NET, maybe call namespaces as OTel.Semconv.v1_26_0.Preview)

This removes diamond dependency problem and stability concerns.

[lmolkova]

@jack-berg curious what you think about above approach and if you'd be concerned about diamond dependency problem from the Java side (outside of the agent).

[martinjt]

A little more context with this...

• The 1_26 namespace would contain all stable conventions
• The 1_26Experimental namespace would contain all stable and experimental conventions
• The 1_26Experimental would be updated (add to and changed) until 1.27 is marked stable.
• The default namespace would contain the current conventions stable conventions only.

Example Namespaces (.NET focused)

• OpenTelemetry.SemanticConventions.Http
• OpenTelemetry.SemanticConventions.1_26_0.Http
• OpenTelemetry.SemanticConventions.1_26_0Experimental.Http

For library Releases it gets a little harder

• Release version 1.26.0nn will be released incrementally as new conventions are updated in the preview specs
• Release version 1.26.xnn will include all previous stable AND experimental classes from a point(1)

Below that, the structure would be the same, and contain namespaces that correspond to semantic conventions sections (.Http and .Messaging etc.)

The general idea being a library can choose to stick with only stable conventions, and be comfortable that regardless of which semconv dependency the client application uses, the conventions will be there (adhering to minimum version constraints).

The concern I had was having the older 1.26 yaml's available for regeneration purposes. However, I think that could be handle through having the tooling work with git tags from the semantic conventions.

(1) what this point is would be debatable, I would say everything starts at 1.26, but it's arguable that this would be "latest stable available at the point that the language started using them. However, with using git tags, I feel we can make sure that all libraries use the same starting point, and therefore libraries that are cross-language and rely on older conventions can still use them.

[lmolkova]

thanks for the context @martinjt and having a default namespace with stable things sounds great.

A small slarification:

Release version 1.26.0nn will be released incrementally as new conventions are updated in the preview specs

The new minor version of semconv is released ~every month and older versions are never changed, so essentially once 1.26.0 is generated, the corresponding namespace never needs to be updated again. Attribute may change in 1.27.0 or become stable in 1.28.0.

However, I think that could be handle through having the tooling work with git tags from the semantic conventions.

Tooling already takes a specific version (git tag).

[martinjt]

I suppose if there's a regular cadence of the minor versions being released then it might not matter. The idea there is so that when someone is adding new items to the semantic conventions, they don't have to wait for a "version" to happen for months on end, therefore causing them not to use the libraries and therefore negating the point of them being there.

The other thing I think would be really useful is to have a workflow convention so that generation, and ultimately creating maybe a PR, for the new version can be done by the semantic conventions repository to reduce the time that each SIG takes to release the new package. If those are stuck behind in the various languages, we'll get into the same state as we have been around the perception of instability and maturity across languages.

[lmolkova]

The other thing I think would be really useful is to have a workflow convention so that generatio

Great point!
Created #565 - it should be possible to do it today (as long as it's possible to detect new semconv version), but ultimately jinja templates and generation scripts can't be centralized (e.g. JS doesn't mind having all attributes in one file, but cares about code size).

[MrAlias]

From this one Go Maintainer's perspective:

• As long as the generation tooling doesn't break we are not opposed to any layout changes
• It is desirable to have the semantic conventions laid out in files that matched their groups so we can find them easier

[jack-berg]

@lmolkova joined the 11/30/23 java SIG and we talked about this issue a bit.. On the topic of the diamond dependency problem, we discussed that it would only really be a problem for experimental conventions. Since we have strong backwards compatibility guarantees for the stable generated code, if two libraries have different dependencies on the semconv artifact, they can resolve by using the most recent. This is similar to expectations around resolving dependency conflicts on the API.

However, we currently generate a constant SCHEMA_URL which references the schemaUrl of the semantic convention version the constants were generated from. This will be problematic if different libraries depend on different versions, because one library could intent to indicate that they publish telemetry conforming to 1.22.0, but because of dependency resolution, end up publishing a schema url corresponding to 1.23.0, which has implications on consumes. We'll need to either remove this constant, encode the version in the name of the constant, or something similar.

The experimental conventions ARE problematic since because we don't have compatibility guarantees, we won't be able to instruct users to simply resolve to the most recent version. I think @lmolkova suggestion here is a good solution to that problem. She also mentioned in the java SIG that we might only support generated attributes for some N recent releases of semantic attributes, where N is 5 or something. This would allow us to avoid a bloated artifact, and tell users to upgrade to the latest version, which would work most but not all of the time. This creates an incentive to move conventions from experimental to stable more quickly, which I think is a good thing.

[lmolkova]

@dyladan thanks for the input!

I believe it's possible (but not easy) to generate code in the way you mentioned even today. It's also possible to add some helpers methods into build-tools to make it much easier.

I just realized you're also suggesting the approach with including the version into import path import {} from '@otel/semconv/1.21 as we discussed later in that issue - would the artifact size be a problem here?
Based on the experiment in Java (open-telemetry/semantic-conventions-java#40) we see that every version adds ~70kb to the artifact size.

[dyladan]

I believe it's possible (but not easy) to generate code in the way you mentioned even today. It's also possible to add some helpers methods into build-tools to make it much easier.

It is indeed possible (and indeed, not easy). example draft: open-telemetry/opentelemetry-js#4298

I just realized you're also suggesting the approach with including the version into import path import {} from '@otel/semconv/1.21 as we discussed later in that issue - would the artifact size be a problem here?
Based on the experiment in Java (open-telemetry/semantic-conventions-java#40) we see that every version adds ~70kb to the artifact size.

I don't expect artifact size to be as big of a concern because that method is very friendly to tree-shaking. JS targets that are sensitive to bundle size usually use bundlers so any unused import paths are simply not included in the final bundle. It would increase the size of the package to download, which will have some impact.

Another possibility is import {} from '@otel/semconv' (stable semconv only) and import {} from '@otel/semconv/unstable' (experimental only) which would only be 2 versions (maybe also an /all path). This would potentially result in breaks if a library used an unstable semconv, but a library would typically pin the version they use anyway. The biggest concern for breaks would be if an end-user made custom instrumentation that used the library, but I think an /unstable path is a pretty strong signal of what users should expect.

[MSNev]

Before creating the above PR to "attempt" to pull out separate strings, I was envisaging either not using the jinga templates or have to post process the strings out to find the longest common strings to reduce the code size (I opted for the above semi-manual approach or eye-balling the common strings).

Some additional inputs / outputs that I would prefer to see from generation (as that is what this issue is really about, and if I can find time I may prototype) would be that the generation would.

• Include ALL of the conventions (strings) for a range of "versions" (say from v1.1 to 1.x), where duplicate strings are only listed once (so you don't have semantic convention version number included in the name or namespace.
• As part of this each string could be tagged with the version that it was introduced (or deprecated) in
• This would also ideally, have the ability to "drop" (exclude) semantic conventions that have been deprecated for what ever period.

Part of the above could also be achieved by including the additional meta data in the "current" semantic conventions, but that would also require that any of the "dropped" strings are never removed.

To summarize the JavaScript approach as mentioned by @dyladan

• JavaScript does NOT want to use namespaces (so that unused / packaged values are tree-shaking out of the packaged bundles)
• Because strings cannot be minified, we also would prefer to reduce the overall duplication of strings and have them "created" at runtime (only if required / referenced)

But for languages that don't have the unique challenges of client environments

• If they want / prefer to use namespaces, should they be able to be grouped (absolutely), as the overall code size will always be reduced by excluding unused values.
• Should there be multiple "namespaces" based on the "version" of the semantic conventions
  • This will depend on each language (IMHO), so it should be "possible" but not mandatory as stated above my preference would be to include every value in the "group" for the selected range of conventions into a single generated code, with documentation comments highlighting when it was added / removed along with the description etc.

I'm repeating myself so that's it for now.

[bryce-b]

For swift, we manually generate the semconv when necessary. We essentially have to re-learn how to do it every time, so automatic generation would be greatly appreciated. I don't have great opinion on how they are organized. It may be wise to separate out experimental into their own library, but it would be more for the convenience of downstream dependents. I envision the semconv libraries to be packaged with the sdk, since the instrumentation is dependent on them.

[marcalff]

to @lmolkova
cc @open-telemetry/cpp-maintainers

About semantic conventions in C++, from my own perspective:

• Grouping of attributes
  • Group by root namespace sounds good.
• Experimental or deprecated semconv
  • It should be possible to test for experimental semconv in the template, to decide to filter it out or not.
  • Testing for deprecated semconv is good, but this is not enough in my opinion: what I would also like, is to have a deprecated_since_version property in the yaml file, accessible in the template, so that code generation can decide for example to generate code only if deprecated_since_version >= 1.30 when generating code for semconv 1.35, in effect enforcing that deprecated symbols are maintained only for a given time, not forever. Having this field, each SIG can decide to use it or not.
• Workflow for renaming semconv
  • When "old.thing" is renamed to "new.thing", the yaml file should not just rename the entry.
  • Instead, please create a new entry "new.thing", so it gets generated, and change the "old.thing" to be deprecated.
  • This way, both old and new semconv can be generated, without the need for SIG maintainer to manually re introduce an hard coded definition for "old.thing", to avoid breaking changes.
• Jinja templates
  • The complexity of maintaining templates, without knowledge of jinja, without knowledge of the list of primitives available (what is attribute.brief, where does it come from, and how do I know it exists in the first place ?), and without knowing where to find this information, should not be underestimated. Not everyone knows the implementation of the build-tools to find this out.
  • The most important part is doc with -- working -- examples, and doc that covers the primitives available.
• Code generation
  • One observation from code generation is that it is working reasonably well and is repeatable when using the same tag on semconv.
  • What is not working well however, is when going from one semconv tag to the next: the code generates, but the output for the new version is unpredictable, generating noisy diffs from the previous release.
  • I would like to suggest to sort all the semconv by name in the code generation tool first, and then emit code for symbols always in the same order. This way, semconv "foo_a" will always appear before "foo_b" in the generated file. This helps to assess what actually changed, helps with merges, avoid polluting git history with code that just moved, etc.
• Semantic convention libraries
  • Right now, C++ generates semconv header files inside the opentelemetry-cpp repository, keeping up with the latest semconv releases. As a result, users of opentelemetry-cpp are de facto forced to use the latest semconv "library" (header files) that was available when opentelemetry-cpp released, so there is coupling.
  • We may deliver generated code into a separate repository (opentelemetry-cpp-semconv) later, this is undecided for now, and to be investigated. This would allow an instrumented application to pick opentelemetry-cpp and semconv versions independently, and allow independent release cycles.

[bryannaegele]

What @marcalff wrote up summarizes the situation and opinions for @open-telemetry/erlang-maintainers, as well. We do have an additional complication where Elixir supports deprecated annotations for users but Erlang does not. We'd like to figure that out but that's a codegen issue for us. Either way, it would helpful to to have that and maintaining backwards compatibility for some period of time before it's hard-deprecated/removed is essential imo.

[brettmc]

• noisy diffs is an annoyance for PHP sig (we have started sorting the attributes by fqn to make diffs easier to comprehend)
• being able to exclude classes of attributes that aren't relevant would be nice (I can't imagine we'd ever need jvm., device. etc for our SIG)
• we currently generate TraceAttributes and ResourceAttributes, but I'd imagine in the future we'll want to generate some attributes specific to logs and metrics

[lmolkova]

Thank you all for the information and the feedback. Let me summarize it and break it down into different categories (let me know if I missed something)

Process issues

• We should never remove attributes (even experimental ones), but mark them deprecated. Alternatively, codegen tooling should generate attributes that were removed between version X and Y. Formalize attribute deprecation, restrict removal #740
• Codegen is never validated until semconv/tooling is released Automate semconv library codegen for new versions #565

Tooling issues

• no clear documentation on otel + Jinja templates (what's available, how to use)
  • exclude certain attribute groups/namespaces (putting it here because it's possible, probably just need better docs)
  • noisy diffs, no sorting - same here, no new features needed, just docs on how to do it + better defaults
  • generating metric definitions, attributes across different signals - it's supported, need better docs
• not fully automated: need to re-learn how to do generation

Tooling feature requests

• whatever we do should be backward compatible with current jinja templates and scripts
• More flexible grouping (by namespace, or no grouping, but possibly access to namespace fragments and trie-like attribute structure)
• ability to separate stable/unstable conventions
• better defaults:
  • sorting attributes to create better diffs

Yaml feature requests

• deprecated_since property - Formalize attribute deprecation, restrict removal #740

[lmolkova]

I believe we have a prototype that addresses most of tooling feature-requests - otel/semconvgen:feature-codegen-by-namespace

It allows to

• group attributes by root namespace when --file-per-group root_namespace is added. As a result it generates ServerAttributes, ClientAttributes, HttpAttributes, etc files
• it adds a bunch of convenience methods to our jinja templates such as is_stable, is_deprecated, is_template, etc allowing to split attributes into different artifact, generate annotations, etc
• simplifies jinja templates by providing a filtered list of attribute definitions (excludes refs)
• no changes for existing usage

Usage is demonstrated in

• Generate classes by root namespace semantic-conventions-java#45 and Split out incubating artifact semantic-conventions-java#41
• New semconvgen template and update to semconv 1.25.0 opentelemetry-python#3586
• you can find more jinja examples in the tests

There are some rough edges that we need to polish (e.g. file names pattern depends on the langauge), but it's open to experiment with.

What it does not solve:

• JavaScript case when we'd want to represent all attributes in a trie-like structure and format attribute names at runtime. I can imagine an incremental change that would allow it though.
• We still need better processes, automated checks for codegen and docs

[trisch-me]

hey @lmolkova is this issue still relevant?

[trentm]

What it does not solve:

• JavaScript case when we'd want to represent all attributes in a trie-like structure and format attribute names at runtime. I can imagine an incremental change that would allow it though.

(@dyladan Please correct me if I'm wrong.) This is no longer a requirement for JavaScript's semconv generation. We are and will be generating straightforward constants, something like export const ATTR_HOST_NAME = 'host.name'; or similar.