Define Log Data model #97
Merged
+1,348
−0
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tigrannajaryan
requested review from
arminru,
bogdandrutu,
c24t,
carlosalberto,
iredelmeier,
jmacd,
reyang,
SergeyKanzhelev,
tedsuo and
yurishkuro
as code owners
tigrannajaryan
force-pushed
the
feature/tigran/log-data-model
branch
2 times, most recently
from
269a3da
to
e3be95e
Compare
|
@open-telemetry/specs-approvers please review. The initial wave of discussion happened in the Google Doc. I believe it is time to move to refinements and eventual approval. |
tigrannajaryan
force-pushed
the
feature/tigran/log-data-model
branch
from
e3be95e
to
a1c863d
Compare
jeffalder
reviewed
Apr 28, 2020
jeffalder
reviewed
Apr 28, 2020
jeffalder
reviewed
Apr 28, 2020
jeffalder
reviewed
Apr 28, 2020
jeffalder
reviewed
Apr 28, 2020
jeffalder
reviewed
Apr 28, 2020
tigrannajaryan
force-pushed
the
feature/tigran/log-data-model
branch
from
a07b657
to
66c2d9e
Compare
jeffalder
reviewed
Apr 29, 2020
jeffalder
reviewed
Apr 29, 2020
tigrannajaryan
force-pushed
the
feature/tigran/log-data-model
branch
from
66c2d9e
to
3720df0
Compare
|
@open-telemetry/specs-approvers please review. |
roncohen
reviewed
May 1, 2020
|
thank you for the hard work on this @tigrannajaryan and others. I understand the wish to keep up the momentum and move forward on this. One point that recently came up was whether we need to consider how the data model can be made to encompass not just logs, but also tracing events and metrics or at least how it will interoperate with these. cc'ing @tedsuo because I believe we discussed it in a recent call. |
|
one of the requirements for this model is that is can be unambiguously mapped to from common models. It doesn't look like we've been through that exercise for the Elastic Common Schema (ECS) and it is not present here. I'm happy to take a swing at it, if that sounds appropriate. |
roncohen
reviewed
May 1, 2020
@roncohen Yes, definitely, please do. Happy to add it to this OTEP or reference it from this OTEP if you post it elsewhere. |
I'll leave up to OpenTelemetry admins to decide if having this is appropriate in an OTEP. I believe examples help clarify the OTEP and should stay. |
No problem, I'll submit a PR for a logz.io example section to be added in fairness if that's how we're going to proceed. Personally, I do not believe this is the right forum for vendor names to be included. |
jkowall
approved these changes
May 22, 2020
tsloughter
approved these changes
May 22, 2020
tigrannajaryan
force-pushed
the
feature/tigran/log-data-model
branch
from
ca19785
to
75bd614
Compare
added 7 commits
May 22, 2020 15:56
This is a proposal of a data model and semantic conventions that allow to represent logs from various sources: application log files, machine generated events, system logs, etc. Existing log formats can be unambiguously mapped to this data model. Reverse mapping from this data model is also possible to the extent that the target log format has equivalent capabilities. The purpose of the data model is to have a common understanding of what a log record is, what data needs to be recorded, transferred, stored and interpreted by a logging system.
tigrannajaryan
force-pushed
the
feature/tigran/log-data-model
branch
from
75bd614
to
33bcdf0
Compare
reyang
reviewed
May 22, 2020
tigrannajaryan
force-pushed
the
feature/tigran/log-data-model
branch
from
33bcdf0
to
9b721ee
Compare
tigrannajaryan
added a commit
to open-telemetry/opentelemetry-specification
that referenced
this pull request
Apr 6, 2023
Fixes #597 ## Changes - Add a section for "generic attributes" to the log semconv - Add an attribute `log_record.id` making use of ULID as discussed in #597 Some additional notes: - I kept the PR small, so I left out some other potential attributes, e.g. something for pre-existing ID (like windows event logs) or for storing the used logging/eventing system or even something like a "signature" that might be worth discussing, etc. - I followed the structure of "generic attributes" from the spans semconv - I took some of the existing wording from #597 & open-telemetry/oteps#97 (comment) to describe the field --------- Signed-off-by: svrnm <neumanns@cisco.com> Co-authored-by: Joao Grassi <joao@joaograssi.com> Co-authored-by: jack-berg <34418638+jack-berg@users.noreply.github.com> Co-authored-by: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com>
jsuereth
pushed a commit
to jsuereth/otel-semconv-test
that referenced
this pull request
Apr 19, 2023
Fixes #597 ## Changes - Add a section for "generic attributes" to the log semconv - Add an attribute `log_record.id` making use of ULID as discussed in #597 Some additional notes: - I kept the PR small, so I left out some other potential attributes, e.g. something for pre-existing ID (like windows event logs) or for storing the used logging/eventing system or even something like a "signature" that might be worth discussing, etc. - I followed the structure of "generic attributes" from the spans semconv - I took some of the existing wording from #597 & open-telemetry/oteps#97 (comment) to describe the field --------- Signed-off-by: svrnm <neumanns@cisco.com> Co-authored-by: Joao Grassi <joao@joaograssi.com> Co-authored-by: jack-berg <34418638+jack-berg@users.noreply.github.com> Co-authored-by: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com>
jsuereth
pushed a commit
to open-telemetry/semantic-conventions
that referenced
this pull request
May 11, 2023
Fixes #597 ## Changes - Add a section for "generic attributes" to the log semconv - Add an attribute `log_record.id` making use of ULID as discussed in #597 Some additional notes: - I kept the PR small, so I left out some other potential attributes, e.g. something for pre-existing ID (like windows event logs) or for storing the used logging/eventing system or even something like a "signature" that might be worth discussing, etc. - I followed the structure of "generic attributes" from the spans semconv - I took some of the existing wording from #597 & open-telemetry/oteps#97 (comment) to describe the field --------- Signed-off-by: svrnm <neumanns@cisco.com> Co-authored-by: Joao Grassi <joao@joaograssi.com> Co-authored-by: jack-berg <34418638+jack-berg@users.noreply.github.com> Co-authored-by: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com>
carlosalberto
pushed a commit
to carlosalberto/oteps
that referenced
this pull request
Oct 23, 2024
* Define Log Data model This is a proposal of a data model and semantic conventions that allow to represent logs from various sources: application log files, machine generated events, system logs, etc. Existing log formats can be unambiguously mapped to this data model. Reverse mapping from this data model is also possible to the extent that the target log format has equivalent capabilities. The purpose of the data model is to have a common understanding of what a log record is, what data needs to be recorded, transferred, stored and interpreted by a logging system. * Move content from Google Doc to markdown here * Address PR comments * Add Google Cloud Logging mapping * Address PR comments * Resolve Open Questions * Add ECS mapping * Address PR comments
carlosalberto
pushed a commit
to carlosalberto/oteps
that referenced
this pull request
Oct 23, 2024
* Define Log Data model This is a proposal of a data model and semantic conventions that allow to represent logs from various sources: application log files, machine generated events, system logs, etc. Existing log formats can be unambiguously mapped to this data model. Reverse mapping from this data model is also possible to the extent that the target log format has equivalent capabilities. The purpose of the data model is to have a common understanding of what a log record is, what data needs to be recorded, transferred, stored and interpreted by a logging system. * Move content from Google Doc to markdown here * Address PR comments * Add Google Cloud Logging mapping * Address PR comments * Resolve Open Questions * Add ECS mapping * Address PR comments
carlosalberto
pushed a commit
to carlosalberto/oteps
that referenced
this pull request
Oct 30, 2024
* Define Log Data model This is a proposal of a data model and semantic conventions that allow to represent logs from various sources: application log files, machine generated events, system logs, etc. Existing log formats can be unambiguously mapped to this data model. Reverse mapping from this data model is also possible to the extent that the target log format has equivalent capabilities. The purpose of the data model is to have a common understanding of what a log record is, what data needs to be recorded, transferred, stored and interpreted by a logging system. * Move content from Google Doc to markdown here * Address PR comments * Add Google Cloud Logging mapping * Address PR comments * Resolve Open Questions * Add ECS mapping * Address PR comments
carlosalberto
pushed a commit
to carlosalberto/opentelemetry-specification
that referenced
this pull request
Oct 31, 2024
Fixes open-telemetry#597 ## Changes - Add a section for "generic attributes" to the log semconv - Add an attribute `log_record.id` making use of ULID as discussed in open-telemetry#597 Some additional notes: - I kept the PR small, so I left out some other potential attributes, e.g. something for pre-existing ID (like windows event logs) or for storing the used logging/eventing system or even something like a "signature" that might be worth discussing, etc. - I followed the structure of "generic attributes" from the spans semconv - I took some of the existing wording from open-telemetry#597 & open-telemetry/oteps#97 (comment) to describe the field --------- Signed-off-by: svrnm <neumanns@cisco.com> Co-authored-by: Joao Grassi <joao@joaograssi.com> Co-authored-by: jack-berg <34418638+jack-berg@users.noreply.github.com> Co-authored-by: Tigran Najaryan <4194920+tigrannajaryan@users.noreply.github.com>
carlosalberto
pushed a commit
to open-telemetry/opentelemetry-specification
that referenced
this pull request
Nov 8, 2024
* Define Log Data model This is a proposal of a data model and semantic conventions that allow to represent logs from various sources: application log files, machine generated events, system logs, etc. Existing log formats can be unambiguously mapped to this data model. Reverse mapping from this data model is also possible to the extent that the target log format has equivalent capabilities. The purpose of the data model is to have a common understanding of what a log record is, what data needs to be recorded, transferred, stored and interpreted by a logging system. * Move content from Google Doc to markdown here * Address PR comments * Add Google Cloud Logging mapping * Address PR comments * Resolve Open Questions * Add ECS mapping * Address PR comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a proposal of a data model and semantic conventions that allow to
represent logs from various sources: application log files, machine
generated events, system logs, etc.
Existing log formats can be unambiguously mapped to this data model.
Reverse mapping from this data model is also possible to the extent that
the target log format has equivalent capabilities.
The purpose of the data model is to have a common understanding of what a
log record is, what data needs to be recorded, transferred, stored and
interpreted by a logging system.