Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[exporter/loki] Make it easier to map OpenTelemetry Severity to Loki level label #14313

Closed
kago-dk opened this issue Sep 19, 2022 · 6 comments · Fixed by #14560
Closed

[exporter/loki] Make it easier to map OpenTelemetry Severity to Loki level label #14313

kago-dk opened this issue Sep 19, 2022 · 6 comments · Fixed by #14560
Assignees
Labels
enhancement New feature or request exporter/loki Loki Exporter priority:p2 Medium

Comments

@kago-dk
Copy link

kago-dk commented Sep 19, 2022

Is your feature request related to a problem? Please describe.

Currently, the log level is tracked as SeverityText (https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/logs/data-model.md) in v0.60.0, while the same information is tracked in the level label in Loki (https://grafana.com/docs/grafana/latest/explore/logs-integration/). It would be optimal if we could avoid tracking the log level as both "SeverityText" and "level" in the OTEL Collector to make Loki work and support mapping between "SeverityText" and "level" in the Loki Exporter configuration.

Example
NoLevel_Body_Purple
If the level is not set, Loki presents the log line as purple because the body text includes the word "Alert"

AttributeLevel_Body_Purple

If the level is set but not part of loki.attribute.labels, then Loki presents the logline as purple because the body text includes the word "Alert"
  attributes/logs:
    actions:
    - action: insert
      key: loki.attribute.labels
      value: net_sock_peer_addr

Level_Green

If the level is set and part of loki.attribute.labels, then Loki presents the logline as green because Loki uses the log level from the level label.
  attributes/logs:
    actions:
    - action: insert
      key: loki.attribute.labels
      value: net_sock_peer_addr,level

Describe the solution you'd like

Support mapping between "SeverityText" and "level" in the Loki Exporter configuration in the OTEL Collector, so the user is not forced into Loki-specific transformation in the OTEL Collector. Ideally, it should happen automatically since the purpose of the fields is the same on both sides.

Describe alternatives you've considered

No response

Additional context

No response

@kago-dk kago-dk added enhancement New feature or request needs triage New item requiring triage labels Sep 19, 2022
@kago-dk kago-dk changed the title [exporter/loki] Make it easier to map OpenTelemetry Severity to Loki level [exporter/loki] Make it easier to map OpenTelemetry Severity to Loki level label Sep 19, 2022
@codeboten codeboten added the exporter/loki Loki Exporter label Sep 19, 2022
@github-actions
Copy link
Contributor

Pinging code owners: @gramidt @gouthamve @jpkrohling @kovrus @mar4uk. See Adding Labels via Comments if you do not have permissions to add labels yourself.

@evan-bradley evan-bradley added priority:p2 Medium and removed needs triage New item requiring triage labels Sep 26, 2022
@jpkrohling
Copy link
Member

Sounds reasonable to me! Would you like to open a PR for that?

@jpkrohling jpkrohling self-assigned this Sep 26, 2022
@kago-dk
Copy link
Author

kago-dk commented Sep 26, 2022

@jpkrohling I do not have the skills to do it.

@jtama
Copy link
Contributor

jtama commented Sep 27, 2022

I don't know if the issue is related but the opentelemtry documentation states :

SeverityText | The severity text (also known as log level).

What actually happens is that severityText contains the original value. If I have the following config :

      - type: regex_parser
        id: extract-nginx-log #Goto end
        parse_from: body
        regex: '(?P<statuscode>\d{3}) .*'
        severity:
          parse_from: attributes.statuscode
          mapping:
            error: "5xx"
            warn: "4xx"
            info: "3xx"
            debug: "2xx"

And the following log :
201 Content created

The log entry will have "201" as severityText and "9" as severity. I would expect the severityText to be info...

@jpkrohling
Copy link
Member

I do not have the skills to do it.

Don't worry, thanks for the issue anyway. I'll assign this to me and take a look soon.

@jtama
Copy link
Contributor

jtama commented Sep 28, 2022

Nevermind my previous comment. SeverityText should be the original one.
https://github.com/open-telemetry/opentelemetry-collector/blob/1c217b366fbdb209044d8f4c3fece079ae23bd3b/pdata/internal/data/protogen/logs/v1/logs.pb.go#L371-L372).

The mapping could maybe be done through the "Severity" ? A severity of SeverityNumber_SEVERITY_NUMBER_TRACE could be mapped to TRACE ?

I would be happy to try this one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request exporter/loki Loki Exporter priority:p2 Medium
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants