Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modeling the body of a Semantic Convention #755

Closed
MSNev opened this issue Feb 17, 2024 · 10 comments
Closed

Modeling the body of a Semantic Convention #755

MSNev opened this issue Feb 17, 2024 · 10 comments
Assignees

Comments

@MSNev
Copy link
Contributor

MSNev commented Feb 17, 2024

What are you trying to achieve?

With the definition of Events storing their payload into the Log Body as defined in Semantic Conventions and as part of the Event Api there is a need to identify "how" the payload is described and encoded so that implementers and downstream consumers have a deterministic method of defining, decoding, processing and possible validating the content.

Therefore, we need to have a consistent way to describe the Log body.

There are 2 levels of definition that need to be considered, this issue will only focus on the Data Modeling of the body and not the transport encoding.

  • Data Model: How is the body described in semantic conventions from a data model perspective for yaml, code generation and documentation
  • Transport Encoding: How are the values encoded (with deterministic context) in the body of the Log
    • As this issue along could be quite large, I will assume the default position is as defined by Logs and the OTLP protocol as explicitly identifying the encoding does not necessarily need to be part of the data model.
    • And I would suggested that encoding should be discussed as part of a separate issue as it is a complex topic that will require a lot of discussion and agreement.

As the body is defined as an <any> we need to define how the body will be described in yaml and ultimately encoded for transportation.

Data Model

The body of a Log is defined as a value of type <any> which means that it can be a complex type that needs to be described in a consistent manner.

The following are some possible yaml representations for the body of a Log describing the expected content of the contents that will be included. These is based on the assumption that the body is a single value that can be described as a map<string, any>, array<any>, <scalar value> or a <byte array>.

Basic Modeling Rules

  • Every body definition is unique only to the encompassing Semantic Convention that is declaring it.
  • There will be no id assigned to the body definition, as it cannot be shared with other Data Models.
  • Different body definitions may define the same or similar structure, but they will be considered to be unique to the encompassing Semantic Convention.

Extensions to theses basic modeling rules will be defined for each of the possible body definitions and can be extended in future releases of the specification if required.

map< string, any >

Additional Modeling Rules

  • A body definition may have multiple field definitions.
  • Each field definition within the body definition is unique to the encompassing body definition and may include an id that is unique to the body definition only.
  • Any "fields" defined within the body definition are not global and are not available outside of their declared body definition. ie. They are not part of the global semantic conventions attributes.
  • Fields within different body definitions may re-use the same id as they are unique to the encompassing body definition only, are not global and should not be considered as the same field definition.
  • Any id defined within the body definition SHOULD not repeated withing the same fields definition.
body:
  fields:
    - id: <id>
      type: <type>
      brief: <brief>
      note: <note>
      examples: [<examples>]
    - id: <id>
      type: <type>
      brief: <brief>
      note: <note>
      examples: [<examples>]

Where fields is a list of field definitions where each field has an id that represents the key of the map and is described in the same manner as an Attribute with the biggest difference that the described fields are NOT added to the global semantic conventions attributes but are instead associated with the body of the encompassing Log.

Examples of using fields in a full semantic convention definition can be found in the Build tools draft PR to add log_event support

This is the primary proposal for the event working group to support defining the body of an event in a consistent manner, so that it can be used for code generation, documentation and validation.

Open Questions

  • Should Nested fields be allowed? ie. map<string, <any>> within the fields definition.
  • Should the initial definition be that a map<string, <any>>, should the <any> be limited to not support nesting (map<string, <any>>) or do we allow the values to be encoded based on the identified "standard".
    • If nesting is allowed, there should also be a recommended defined "depth" for this nested.

array< any >

For completeness to describe the support OTLP protocol, it is possible to have an array of values as the body of a Log. This would be a list of values that are of type <any>.

body:
  array:
    brief: <brief>
    note: <note>
    examples: [<examples>]

Where array represents that there will be an unspecified list of values of type <any>, while it would contain the detail fields of an Attribute (brief, note, examples, etc) it would NOT provide an id. As the
array would assume to be numeric index based. Specific example will be required to identify if there are any specific "fields" that are expected to be present in the array. Like min/max length, min/max values, etc.

Open Question

  • How or do we want to define the types of values that may be present in the array, along with additional attributes (Like min/max length, min/max values, etc).

scalar value

Also For completeness to describe the support OTLP protocol, it is possible to have an array of values as the body of a Log. This would be a list of values that are of type <any>.

body: 
  value:
    type: <type>
    brief: <brief>
    note: <note>
    examples: [<examples>]

Where <value> is described in the same manner as an Attribute and as with the array it would not include the id (key).

byte array

Also For completeness to describe the support OTLP protocol, it is possible to have an array of values as the body of a Log. This would be a list of values that are of type <any>.

This could also be rolled into the value definition as a type of binary but it may be better to have it as a separate definition to make it clear that the value is a binary blob.

body:
  binary:
    brief: <brief>
    note: <note>
    examples: [<examples>]

Where binary is described in the same manner as an Attribute and as with the array it would not include the id (key).

@lmolkova
Copy link
Contributor

lmolkova commented Feb 21, 2024

What drives the desire to describe full structure in yaml?
Given that you can put literally anything there, I don't see how flat structure described in fields could work to express something like

  "foo" : {"bar": [1,2,3,4,5], "baz":{ ......}}

There are big projects like json-schema that describe arbitrary schemas, can we reuse what they offer?

What would be broken if we did this:

body: 
  schema: link-to-schema goes here, e.g. https://support.google.com/analytics/answer/9216061#page_view
  brief: <brief>
  note: <note>
  examples: [<examples>]

then schema can be external to otel or page_view can be defined in otel, but not in the yaml. It can be as simple as manually written MD table.

I feel this would be the least controversial, easy to implement and use solution that can also be evolved to support stricter schema definitions.

I wonder if it'd be enough as a first step.

@lmolkova
Copy link
Contributor

or even

body: 
  brief: <brief>. link-to-schema goes here, e.g. https://support.google.com/analytics/answer/9216061#page_view
  note: <note>
  examples: [<examples>]

@patrickhousley
Copy link
Contributor

Personally, I would like the body to be represented in yaml. Having the body represented in yaml means that types will be generated and IDEs can use those to auto-complete what we would expect to be in the body of an event we define. If the body is just described in markdown, there is really nothing there "enforcing" the contents of the body for our defined events.

For the representation of body, I think we should keep it simple and stick with a collection of key/value pairs. I understand that this is not going to please everyone but it keeps our implementation simple in lieu of "body" being four possible data types and backends having to check the type for every event.

I understand that having body as map< string, any > is potentially wordy when using JSON transport. However, that should be a separate conversation as there are many possible solutions to how we transport the data.

Should Recursive fields be allowed? ie. map<string, > within the fields definition.

There are possible performance impacts if an event is excessively deep (nesting) or wide (flattened number of attributes). However, I think this needs to fall on the consumer of the API to control. If the consumer creates an event that has a degree of nesting/breadth that it does cause a performance issue, it must fall on them to fix that and the API should not try to catch/prevent this performance issue.

We can recommend a max depth/breadth but should not force it in code. With a recommendation, we can question any future PRs for defined events that exceed the recommendation.

@lmolkova
Copy link
Contributor

lmolkova commented Mar 1, 2024

To clarify my previous comment:

  • I think ability to express body structure is good
  • I don't think the ability to describe it the yaml is essential to make progress on the semconv
  • If we started simple (free-form body definition in the MD), it would allow to
    • start adding event semconv with free form definition
    • work on structured yaml, codegen in parallel witout blocking any spec efforts on it

@MSNev
Copy link
Contributor Author

MSNev commented Mar 1, 2024

I've just Updated the description of the issue to address and clarify some questions from the Event WG today, the changes are

  • Removed / replaced the wording of "recursion" to "nested" / "nesting" as that is the intent of the description
  • Changed the open question around "nesting" to be a "recommended" defined depth and not a mandated one
  • Add some links to full examples of using the body definition for some additional context to what this issue is describing for yaml and the generated markdown. I don't currently have any relevant code generation in the linked Draft PR yet, but one of the intents of the code to be generated from the yaml is to generated interfaces to provide consistency of the field names used to avoid typos etc.
  • Called out that the proposed array, scalar and binary are possible paths forward and are provided for completeness for all of the types supported by OTLP.

The intent of this issue is so that we can identify a path forward, and not necessarily to describe every possible option and enforce that as the Semantic Convention for yaml for all types.

As long as we can agree that we can define an optional body definition for some types (like an event) and we define it in a fashion to support the current requirements (defining fields specifically) without blocking possible future extensions (like array, scalar, binary, schema, etc).

Apart from the auto generation as already described, the main reason for raising this (and the (now linked) example draft build-tools PR) is that our previous attempts (years ago) to create simple free-form markdown definitions to describe the event payload (the initial use case) was previously rejected (and PR's where blocked).

While free-form markdown describing the "fields" is a possible option, it would still leave gaps for the code generation / validation, and we should really define a structure / guidance on what that "free-form" markdown should follow (IMHO).

@jsuereth
Copy link
Contributor

A few thoughts:

Every body definition is unique only to the encompassing Semantic Convention that is declaring it.

Should this be "unique to the event.id" ?

There will be no id assigned to the body definition, as it cannot be shared with other Data Models.

Don't we need event.id to distinguish event types and understand WHICH event the body is describing?

For completeness to describe the support OTLP protocol, it is possible to have an array of values as the body of a Log. This would be a list of values that are of type <any>.

This is an oddity here. We, theoretically, disallow heterogeneous lists. Personally, I'd prefer typed lists here (with an explicit list[any] if we ever allow that).

@MSNev MSNev added this to Logs SIG Apr 19, 2024
@MSNev MSNev moved this to Todo in Logs SIG Apr 19, 2024
MrAlias added a commit to open-telemetry/opentelemetry-go that referenced this issue Jun 5, 2024
Based on #5394 

This removes `event.go`, `resource.go`, and `trace.go` from generation
because they now only contain references to the attribute registry.
Thus, they will generate empty files.

This also does not include any deprecated semantic convention. Users of
deprecated semantic conventions should continue to use them from the
previous versions that have been published.

[v1.26.0 semantic conventions release
notes](https://github.com/open-telemetry/semantic-conventions/releases/tag/v1.26.0):

<div data-pjax="true" data-test-selector="body-content"
data-view-component="true" class="markdown-body my-3"><h2>v1.26.0</h2>
<h3>🛑 Breaking changes 🛑</h3>
<ul>
<li>
<p><code>db</code>: Rename <code>db.statement</code> to
<code>db.query.text</code> and introduce
<code>db.query.parameter.&lt;key&gt;</code> (<a class="issue-link
js-issue-link" data-error-text="Failed to load title"
data-id="2123681817" data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#716"
data-hovercard-type="issue"
data-hovercard-url="/open-telemetry/semantic-conventions/issues/716/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/issues/716">#716</a>)</p>
</li>
<li>
<p><code>db</code>: Renames <code>db.sql.table</code>,
<code>db.cassandra.table</code>, <code>db.mongodb.collection</code>, and
<code>db.cosmosdb.container</code> attributes to
<code>db.collection.name</code> (<a class="issue-link js-issue-link"
data-error-text="Failed to load title" data-id="2221821104"
data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#870"
data-hovercard-type="pull_request"
data-hovercard-url="/open-telemetry/semantic-conventions/pull/870/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/pull/870">#870</a>)</p>
</li>
<li>
<p><code>db</code>: Rename <code>db.operation</code> to
<code>db.operation.name</code>. (<a class="issue-link js-issue-link"
data-error-text="Failed to load title" data-id="2226761377"
data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#884"
data-hovercard-type="issue"
data-hovercard-url="/open-telemetry/semantic-conventions/issues/884/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/issues/884">#884</a>)</p>
</li>
<li>
<p><code>messaging</code>: Rename <code>messaging.operation</code> to
<code>messaging.operation.type</code>, add
<code>messaging.operation.name</code>. (<a class="issue-link
js-issue-link" data-error-text="Failed to load title"
data-id="2227637055" data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#890"
data-hovercard-type="issue"
data-hovercard-url="/open-telemetry/semantic-conventions/issues/890/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/issues/890">#890</a>)</p>
</li>
<li>
<p><code>db</code>: Deprecate the <code>db.user</code> attribute. (<a
class="issue-link js-issue-link" data-error-text="Failed to load title"
data-id="2226817211" data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#885"
data-hovercard-type="issue"
data-hovercard-url="/open-telemetry/semantic-conventions/issues/885/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/issues/885">#885</a>)</p>
</li>
<li>
<p><code>db</code>: Rename <code>db.name</code> and
<code>db.redis.database_index</code> to <code>db.namespace</code>,
deprecate <code>db.mssql.instance_name</code>. (<a class="issue-link
js-issue-link" data-error-text="Failed to load title"
data-id="2226817211" data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#885"
data-hovercard-type="issue"
data-hovercard-url="/open-telemetry/semantic-conventions/issues/885/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/issues/885">#885</a>)</p>
</li>
<li>
<p><code>db</code>: Remove <code>db.instance.id</code>. For
Elasticsearch, replace with <code>db.elasticsearch.node.name</code>. (<a
class="issue-link js-issue-link" data-error-text="Failed to load title"
data-id="2266411070" data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#972"
data-hovercard-type="pull_request"
data-hovercard-url="/open-telemetry/semantic-conventions/pull/972/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/pull/972">#972</a>)</p>
</li>
<li>
<p><code>db</code>: Clarify database span name format and fallback
values. (<a class="issue-link js-issue-link" data-error-text="Failed to
load title" data-id="2266545339" data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#974"
data-hovercard-type="pull_request"
data-hovercard-url="/open-telemetry/semantic-conventions/pull/974/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/pull/974">#974</a>,
<a class="issue-link js-issue-link" data-error-text="Failed to load
title" data-id="2123611008" data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#704"
data-hovercard-type="issue"
data-hovercard-url="/open-telemetry/semantic-conventions/issues/704/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/issues/704">#704</a>)</p>
</li>
<li>
<p><code>db</code>: Rename <code>db.client.connections.*</code> metric
namespace to <code>db.client.connection.*</code> and rename
<code>db.client.connection.usage</code> to
<code>db.client.connection.count</code>.<br>
(<a class="issue-link js-issue-link" data-error-text="Failed to load
title" data-id="1815993771" data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#201"
data-hovercard-type="issue"
data-hovercard-url="/open-telemetry/semantic-conventions/issues/201/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/issues/201">#201</a>,
<a class="issue-link js-issue-link" data-error-text="Failed to load
title" data-id="2262290114" data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#967"
data-hovercard-type="issue"
data-hovercard-url="/open-telemetry/semantic-conventions/issues/967/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/issues/967">#967</a>)</p>
</li>
<li>
<p><code>db</code>: Rename <code>pool.name</code> to
<code>db.client.connections.pool.name</code> and <code>state</code> to
<code>db.client.connections.state</code>. (<a class="issue-link
js-issue-link" data-error-text="Failed to load title"
data-id="2232095641" data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#909"
data-hovercard-type="pull_request"
data-hovercard-url="/open-telemetry/semantic-conventions/pull/909/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/pull/909">#909</a>)</p>
</li>
<li>
<p><code>system</code>: Deprecate <code>shared</code> from
<code>system.memory.state</code> values and make it a standalone metric
(<a class="issue-link js-issue-link" data-error-text="Failed to load
title" data-id="1993746916" data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#522"
data-hovercard-type="issue"
data-hovercard-url="/open-telemetry/semantic-conventions/issues/522/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/issues/522">#522</a>)</p>
</li>
<li>
<p><code>device.app.lifecycle</code>: Reformat and update the
<code>device.app.lifecycle</code> event description adds constraints for
the possible values of the <code>android.state</code> and
<code>ios.state</code>.<br>
(<a class="issue-link js-issue-link" data-error-text="Failed to load
title" data-id="2170421333" data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#794"
data-hovercard-type="pull_request"
data-hovercard-url="/open-telemetry/semantic-conventions/pull/794/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/pull/794">#794</a>)<br>
Removes the <code>ios.lifecycle.events</code> and
<code>android.lifecycle.events</code> attributes from the global
registry and adds constraints for the possible values of the
<code>android.state</code> and <code>ios.state</code> attributes.</p>
</li>
<li>
<p><code>messaging</code>: Rename <code>messaging.client_id</code> to
<code>messaging.client.id</code> (<a class="issue-link js-issue-link"
data-error-text="Failed to load title" data-id="2250463504"
data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#935"
data-hovercard-type="issue"
data-hovercard-url="/open-telemetry/semantic-conventions/issues/935/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/issues/935">#935</a>)</p>
</li>
<li>
<p><code>rpc</code>: Rename<code>message.*</code> attributes under
<code>rpc</code> to <code>rpc.message.*</code>. Deprecate old
<code>message.*</code> attributes. (<a class="issue-link js-issue-link"
data-error-text="Failed to load title" data-id="2211426432"
data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#854"
data-hovercard-type="pull_request"
data-hovercard-url="/open-telemetry/semantic-conventions/pull/854/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/pull/854">#854</a>)</p>
</li>
</ul>
<h3>🚀 New components 🚀</h3>
<ul>
<li><code>gen-ai</code>: Introducing semantic conventions for GenAI
clients. (<a class="issue-link js-issue-link" data-error-text="Failed to
load title" data-id="1897516973" data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#327"
data-hovercard-type="issue"
data-hovercard-url="/open-telemetry/semantic-conventions/issues/327/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/issues/327">#327</a>)</li>
</ul>
<h3>💡 Enhancements 💡</h3>
<ul>
<li>
<p><code>all</code>: Markdown snippets are now generated by jinja
templates in the <code>templates</code> directory. (<a class="issue-link
js-issue-link" data-error-text="Failed to load title"
data-id="2276527861" data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#1000"
data-hovercard-type="pull_request"
data-hovercard-url="/open-telemetry/semantic-conventions/pull/1000/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/pull/1000">#1000</a>)</p>
</li>
<li>
<p><code>db, messaging, gen_ai</code>: Clarify that
<code>db.system</code>, <code>messaging.system</code>,
<code>gen_ai.system</code> attributes capture the client perception and
may differ from the actual product name. (<a class="issue-link
js-issue-link" data-error-text="Failed to load title"
data-id="2184898831" data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#813"
data-hovercard-type="issue"
data-hovercard-url="/open-telemetry/semantic-conventions/issues/813/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/issues/813">#813</a>,
<a class="issue-link js-issue-link" data-error-text="Failed to load
title" data-id="2286519206" data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#1016"
data-hovercard-type="pull_request"
data-hovercard-url="/open-telemetry/semantic-conventions/pull/1016/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/pull/1016">#1016</a>)</p>
</li>
<li>
<p><code>messaging</code>: Show all applicable attributes in individual
messaging semantic conventions. (<a class="issue-link js-issue-link"
data-error-text="Failed to load title" data-id="2221751255"
data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#869"
data-hovercard-type="issue"
data-hovercard-url="/open-telemetry/semantic-conventions/issues/869/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/issues/869">#869</a>,
<a class="issue-link js-issue-link" data-error-text="Failed to load
title" data-id="2286733771" data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#1018"
data-hovercard-type="pull_request"
data-hovercard-url="/open-telemetry/semantic-conventions/pull/1018/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/pull/1018">#1018</a>)</p>
</li>
<li>
<p><code>process</code>: Add additional attributes to process attribute
registry (<a class="issue-link js-issue-link" data-error-text="Failed to
load title" data-id="2015129430" data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#564"
data-hovercard-type="pull_request"
data-hovercard-url="/open-telemetry/semantic-conventions/pull/564/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/pull/564">#564</a>)</p>
</li>
<li>
<p><code>messaging</code>: Add a GCP Pub/Sub unary pull example and the
new GCP messaging attributes: -
<code>messaging.gcp_pubsub.message.ack_deadline</code>, -
<code>messaging.gcp_pubsub.message.ack_id</code>, -
<code>messaging.gcp_pubsub.message.delivery_attempt</code> (<a
class="issue-link js-issue-link" data-error-text="Failed to load title"
data-id="1995123229" data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#527"
data-hovercard-type="issue"
data-hovercard-url="/open-telemetry/semantic-conventions/issues/527/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/issues/527">#527</a>)</p>
</li>
<li>
<p><code>db</code>: Add <code>db.client.operation.duration</code> metric
(<a class="issue-link js-issue-link" data-error-text="Failed to load
title" data-id="1991797441" data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#512"
data-hovercard-type="issue"
data-hovercard-url="/open-telemetry/semantic-conventions/issues/512/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/issues/512">#512</a>)</p>
</li>
<li>
<p><code>messaging</code>: Adds `messaging.destination.partition.id`` to
the messaging attributes (<a class="issue-link js-issue-link"
data-error-text="Failed to load title" data-id="2185047744"
data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#814"
data-hovercard-type="pull_request"
data-hovercard-url="/open-telemetry/semantic-conventions/pull/814/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/pull/814">#814</a>)</p>
</li>
<li>
<p><code>exception</code>: Replace constraints with requirement levels
on exceptions. (<a class="issue-link js-issue-link"
data-error-text="Failed to load title" data-id="2216070269"
data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#862"
data-hovercard-type="pull_request"
data-hovercard-url="/open-telemetry/semantic-conventions/pull/862/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/pull/862">#862</a>)</p>
</li>
<li>
<p><code>process</code>: Replace constraints with requirement_level in
process attributes. (<a class="issue-link js-issue-link"
data-error-text="Failed to load title" data-id="2216082891"
data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#863"
data-hovercard-type="pull_request"
data-hovercard-url="/open-telemetry/semantic-conventions/pull/863/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/pull/863">#863</a>)</p>
</li>
<li>
<p><code>db</code>: Reorganize DB conventions to be shared across span
and metric conventions. (<a class="issue-link js-issue-link"
data-error-text="Failed to load title" data-id="2232098784"
data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#910"
data-hovercard-type="pull_request"
data-hovercard-url="/open-telemetry/semantic-conventions/pull/910/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/pull/910">#910</a>)</p>
</li>
<li>
<p><code>all</code>: Migrate Attribute Registry to be completely
autogenerated. (<a class="issue-link js-issue-link"
data-error-text="Failed to load title" data-id="1811747563"
data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#197"
data-hovercard-type="issue"
data-hovercard-url="/open-telemetry/semantic-conventions/issues/197/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/issues/197">#197</a>)<br>
Migrate to using weaver for markdown generation (snippet +
registry).<br>
The entirety of the registry now is generated using weaver with
templates<br>
under the <code>templates/</code> directory. Snippets still require a
hardcoded<br>
command.</p>
</li>
<li>
<p><code>http</code>: List all HTTP client and server attributes in the
corresponding table, remove common attributes from yaml and markdown.
(<a class="issue-link js-issue-link" data-error-text="Failed to load
title" data-id="2249267191" data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#928"
data-hovercard-type="issue"
data-hovercard-url="/open-telemetry/semantic-conventions/issues/928/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/issues/928">#928</a>)</p>
</li>
<li>
<p><code>other</code>: Document patterns and suggestions for semconv
code generation. (<a class="issue-link js-issue-link"
data-error-text="Failed to load title" data-id="2005434950"
data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#551"
data-hovercard-type="issue"
data-hovercard-url="/open-telemetry/semantic-conventions/issues/551/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/issues/551">#551</a>,
<a class="issue-link js-issue-link" data-error-text="Failed to load
title" data-id="2260022392" data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#953"
data-hovercard-type="pull_request"
data-hovercard-url="/open-telemetry/semantic-conventions/pull/953/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/pull/953">#953</a>)</p>
</li>
<li>
<p><code>db</code>: Show applicable common attributes in individual
database semantic conventions. (<a class="issue-link js-issue-link"
data-error-text="Failed to load title" data-id="2266443235"
data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#973"
data-hovercard-type="pull_request"
data-hovercard-url="/open-telemetry/semantic-conventions/pull/973/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/pull/973">#973</a>)</p>
</li>
<li>
<p><code>db</code>: Add <code>error.type</code> attribute to the
database span and operation duration metric. (<a class="issue-link
js-issue-link" data-error-text="Failed to load title"
data-id="2266561839" data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#975"
data-hovercard-type="pull_request"
data-hovercard-url="/open-telemetry/semantic-conventions/pull/975/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/pull/975">#975</a>)</p>
</li>
<li>
<p><code>db</code>: Parameterized query text does not need to be
sanitized by default (<a class="issue-link js-issue-link"
data-error-text="Failed to load title" data-id="2266574593"
data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#976"
data-hovercard-type="pull_request"
data-hovercard-url="/open-telemetry/semantic-conventions/pull/976/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/pull/976">#976</a>)</p>
</li>
<li>
<p><code>http</code>: List experimental HTTP attributes applicable to
HTTP client and server spans. (<a class="issue-link js-issue-link"
data-error-text="Failed to load title" data-id="2272874112"
data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#989"
data-hovercard-type="pull_request"
data-hovercard-url="/open-telemetry/semantic-conventions/pull/989/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/pull/989">#989</a>)</p>
</li>
<li>
<p><code>db</code>: Finalizes the migration requirement for
instrumentations to follow when updating to stable database semconv. (<a
class="issue-link js-issue-link" data-error-text="Failed to load title"
data-id="2124200768" data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#719"
data-hovercard-type="issue"
data-hovercard-url="/open-telemetry/semantic-conventions/issues/719/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/issues/719">#719</a>)</p>
</li>
<li>
<p><code>http</code>: New <code>url.template</code> attribute added to
URL, HTTP client attributes are extended with optional low-cardinality
<code>url.template</code> (<a class="issue-link js-issue-link"
data-error-text="Failed to load title" data-id="2107516610"
data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#675"
data-hovercard-type="pull_request"
data-hovercard-url="/open-telemetry/semantic-conventions/pull/675/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/pull/675">#675</a>)</p>
</li>
<li>
<p><code>db</code>: Add note to <code>db.collection.name</code>,
<code>db.namespace</code>, and <code>db.operation.name</code> about
capturing those without attempting to do any case normalization.<br>
(<a class="issue-link js-issue-link" data-error-text="Failed to load
title" data-id="2226822323" data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#886"
data-hovercard-type="issue"
data-hovercard-url="/open-telemetry/semantic-conventions/issues/886/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/issues/886">#886</a>)</p>
</li>
<li>
<p><code>events</code>: Provides additional definitions of log events
and their structure. (<a class="issue-link js-issue-link"
data-error-text="Failed to load title" data-id="2139735298"
data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#755"
data-hovercard-type="issue"
data-hovercard-url="/open-telemetry/semantic-conventions/issues/755/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/issues/755">#755</a>)</p>
</li>
<li>
<p><code>k8s</code>: add container.status.last_terminated_reason
resource attribute (<a class="issue-link js-issue-link"
data-error-text="Failed to load title" data-id="2243002863"
data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#922"
data-hovercard-type="issue"
data-hovercard-url="/open-telemetry/semantic-conventions/issues/922/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/issues/922">#922</a>)</p>
</li>
</ul>
<h3>🧰 Bug fixes 🧰</h3>
<ul>
<li><code>http</code>: Add previously deprecated http attributes to
registry (<a class="issue-link js-issue-link" data-error-text="Failed to
load title" data-id="2288411532" data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#1025"
data-hovercard-type="pull_request"
data-hovercard-url="/open-telemetry/semantic-conventions/pull/1025/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/pull/1025">#1025</a>)<br>
These attributes were deprecated in 1.13</li>
<li><code>net</code>: Add previously deprecated net attributes to
registry (<a class="issue-link js-issue-link" data-error-text="Failed to
load title" data-id="2289802107" data-permission-text="Title is private"
data-url="open-telemetry/semantic-conventions#1029"
data-hovercard-type="pull_request"
data-hovercard-url="/open-telemetry/semantic-conventions/pull/1029/hovercard"
href="https://github.com/open-telemetry/semantic-conventions/pull/1029">#1029</a>)<br>
These attributes were deprecated in 1.13</li>
</ul></div>


### Follow up work

- [ ] Update all dependencies on semconv to v1.26.0

---------

Co-authored-by: Tyler Yahn <codingalias@gmail.com>
Co-authored-by: Aaron Clawson <MadVikingGod@users.noreply.github.com>
Co-authored-by: Tyler Yahn <MrAlias@users.noreply.github.com>
@jsuereth
Copy link
Contributor

I'd like to push back a little on

body: 
  value:
    type: <type>
    brief: <brief>
    note: <note>
    examples: [<examples>]
body:
  fields:

vs.

body:
  type: map
  brief:
  notes:
  fields:

That is - should we represent "Any" directly in body and be able to annotate or notes that?

Looking in the code, I understand thing will need to be disjoint, but I also don't think we have a good answer for nested maps and types.

I'd rather introduce one modelling concept for AnyValue rather than two and have to re-model structure again in body fields.

@lmolkova
Copy link
Contributor

lmolkova commented Aug 27, 2024

+1 for

body:
  type: map
  brief:
  notes:
  fields:

I assume we'd rarely have anything except map in the type

we can always do type: string or type: string[] (without fields).

Things get complicated the moment we get to complex types in arrays or maps either way (with extra value | fields | array layer or with type : something).

@MSNev
Copy link
Contributor Author

MSNev commented Sep 9, 2024

Possible combinations for supporting "Any" value, following the examples / suggestions above.

  - id: single.field.primitive.or.array.body
    body:
      brief: ... for the body / field ...
      note: ... for the body / field ...
      type: boolean | boolean[] | int | int[] | double | double[] | string | string[]

   # Extending to support "map" (fields)
  - id: nested.map.of.primitives.or.array.body
    body:
      brief: ... for the body (any value) ...
      note: ... for the body (any value) ...
      type: map | map[]
      id: map_name???
      fields:
        - id: field1
          type: boolean | boolean[] | int | int[] | double | double[] | string | string[] | any_value
          brief: ... for the field ...
          examples: ... for the field ...
        - id: nested_map
          brief: ... for the field ...
          examples: ... for the field ...
          type: map
          fields:
            - id: nested_field1
              type: boolean | boolean[] | int | int[] | double | double[] | string | string[] | any_value
              brief: ... for the field ...
              examples: ... for the field ...
            - id: nested_field2
              type: boolean | boolean[] | int | int[] | double | double[] | string | string[] | any_value
              brief: ... for the field ...
              examples: ... for the field ...
            - id: nested_enum_name1
              type:
                allow_custom_values: true
                members:
                  - id: feature1
                    value: "f1"
                  - id: feature2
                    value: "f2"
            - id: nested_map_array
              type: map[]
              fields:
                - id: nested_map_array_field1
                  type: boolean | boolean[] | int | int[] | double | double[] | string | string[] | any_value
                  brief: ... for the field ...
                  examples: ... for the field ...
                - id: nested_map_array_field2
                  type: boolean | boolean[] | int | int[] | double | double[] | string | string[] | any_value
                  brief: ... for the field ...
                  examples: ... for the field ...

@lmolkova
Copy link
Contributor

lmolkova commented Oct 7, 2024

Implemented in weaver 0.10.0

@lmolkova lmolkova closed this as completed Oct 7, 2024
@github-project-automation github-project-automation bot moved this from Todo to Done in Logs SIG Oct 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

No branches or pull requests

5 participants