Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[pkg/stanza] Operators configured with Silent Errors would still result in error logs #35008

Closed
SamerJ opened this issue Sep 4, 2024 · 1 comment
Closed

[pkg/stanza] Operators configured with Silent Errors would still result in error logs #35008

SamerJ opened this issue Sep 4, 2024 · 1 comment
Labels
bug Something isn't working needs triage New item requiring triage pkg/stanza/fileconsumer pkg/stanza receiver/filelog

Comments

@SamerJ
Copy link
Contributor

SamerJ commented Sep 4, 2024

Component(s)

pkg/stanza, pkg/stanza/fileconsumer, receiver/filelog

What happened?

Description

Operators have a config that allow silencing errors logs (log level is changed from error to debug).
With this configuration, we expect no error logs when the operator fails to process a log entry.
However, even when we use drop_quiet, we still see error logs when the Operator fails.
The error is not emitted by the Operator but by another component. Nevertheless, the error is because of the operator's log entry failure.

Steps to Reproduce

Use any operator, that might sometimes fails like regex with a pattern that might not always match.
Set the property on_error: drop_quiet.
Feed the operator log entries were some are expected to fail the operator.
Check the emitted logs.

Expected Result

No errors logs when it comes to failures by the Operator

Comparison

You can see that with quiet we hide some of the error logs but not all.

Default

2024-09-04T16:41:58.270+0200 ERROR helper/transformer.go:100 Failed to process entry {"operator_id": "regex_parser", "operator_type": "regex_parser", "error": "regex pattern does not match", "action": "send"}
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/helper.(*TransformerOperator).HandleEntryError
C:/tmp/github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.107.0/operator/helper/transformer.go:100
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/helper.(*ParserOperator).ParseWith
C:/tmp/github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.107.0/operator/helper/parser.go:138
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/helper.(*ParserOperator).ProcessWithCallback
C:/tmp/github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.107.0/operator/helper/parser.go:111
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/helper.(*ParserOperator).ProcessWith
C:/tmp/github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.107.0/operator/helper/parser.go:98
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/parser/regex.(*Parser).Process
C:/tmp/github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.107.0/operator/parser/regex/parser.go:31
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/helper.(*WriterOperator).Write
C:/tmp/github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.107.0/operator/helper/writer.go:54
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/transformer/filter.(*Transformer).Process
C:/tmp/github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.107.0/operator/transformer/filter/transformer.go:43
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/helper.(*WriterOperator).Write
C:/tmp/github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.107.0/operator/helper/writer.go:54
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/input/file.(*Input).emit
C:/tmp/github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.107.0/operator/input/file/input.go:54
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/fileconsumer/internal/reader.(*Reader).ReadToEnd
C:/tmp/github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.107.0/fileconsumer/internal/reader/reader.go:133
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/fileconsumer.(*Manager).consume.func1
C:/tmp/github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.107.0/fileconsumer/file.go:160

2024-09-04T16:45:02.929+0200 ERROR reader/reader.go:140 process: %w {"component": "fileconsumer", "path": "testdata\it\functional\logs\otel-collector.log", "error": "regex pattern does not match"}
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/fileconsumer/internal/reader.(*Reader).ReadToEnd
C:/Users/sjalaleddine/go/pkg/mod/github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.107.0/fileconsumer/internal/reader/reader.go:140
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/fileconsumer.(*Manager).consume.func1
C:/Users/sjalaleddine/go/pkg/mod/github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.107.0/fileconsumer/file.go:160

Quiet

2024-09-04T16:45:02.929+0200 ERROR reader/reader.go:140 process: %w {"component": "fileconsumer", "path": "testdata\it\functional\logs\otel-collector.log", "error": "regex pattern does not match"}
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/fileconsumer/internal/reader.(*Reader).ReadToEnd
C:/Users/sjalaleddine/go/pkg/mod/github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.107.0/fileconsumer/internal/reader/reader.go:140
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/fileconsumer.(*Manager).consume.func1
C:/Users/sjalaleddine/go/pkg/mod/github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.107.0/fileconsumer/file.go:160

Collector version

0.107.0

Environment information

Environment

OS: (e.g., "Ubuntu 20.04")
Compiler(if manually compiled): (e.g., "go 14.2")

OpenTelemetry Collector configuration

# Sample Operator config with silent errors

- type: regex_parser
  parse_from: body.message
  on_error: drop_quiet
  regex: '^Regex that can sometimes fail$'

Log output

2024-09-04T16:45:02.929+0200 ERROR reader/reader.go:140 process: %w {"component": "fileconsumer", "path": "testdata\it\functional\logs\otel-collector.log", "error": "regex pattern does not match"}
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/fileconsumer/internal/reader.(*Reader).ReadToEnd
C:/Users/sjalaleddine/go/pkg/mod/github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.107.0/fileconsumer/internal/reader/reader.go:140
github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/fileconsumer.(*Manager).consume.func1
C:/Users/sjalaleddine/go/pkg/mod/github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.107.0/fileconsumer/file.go:160

Additional context

No response
@SamerJ SamerJ added bug Something isn't working needs triage New item requiring triage labels Sep 4, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Sep 4, 2024

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@SamerJ SamerJ mentioned this issue Sep 4, 2024
Merged
djaglowski pushed a commit that referenced this issue Sep 6, 2024
@SamerJ
Operators with silent errors will log errors as debug and won't retur…
f970421 
…n an error (#35010)

An operator configured with silent errors shouldn't log errors while
processing log entries.

**Description:** 
Operators that are expected to sometimes fail, can result in error logs
that are quite verbose.
This aims to address this issue by getting rid of error logs that the
user wishes to silence (debug logs instead).
By making sure no error is returned, there won't be any need to worry
about other components logging error logs.

**Link to tracking Issue:** [Issue
#35008](#35008)

**Testing:** 
Manually tested as this is a matter of logs.
@SamerJ SamerJ closed this as completed Sep 8, 2024
@github-actions github-actions bot mentioned this issue Sep 10, 2024
Closed
f7o pushed a commit to f7o/opentelemetry-collector-contrib that referenced this issue Sep 12, 2024
@SamerJ @f7o
Operators with silent errors will log errors as debug and won't retur…
b2d6b67 
…n an error (open-telemetry#35010)

An operator configured with silent errors shouldn't log errors while
processing log entries.

**Description:** 
Operators that are expected to sometimes fail, can result in error logs
that are quite verbose.
This aims to address this issue by getting rid of error logs that the
user wishes to silence (debug logs instead).
By making sure no error is returned, there won't be any need to worry
about other components logging error logs.

**Link to tracking Issue:** [Issue
open-telemetry#35008](open-telemetry#35008)

**Testing:** 
Manually tested as this is a matter of logs.
jriguera pushed a commit to springernature/opentelemetry-collector-contrib that referenced this issue Oct 4, 2024
@SamerJ @jriguera
Operators with silent errors will log errors as debug and won't retur…
72eb1b2 
…n an error (open-telemetry#35010)

An operator configured with silent errors shouldn't log errors while
processing log entries.

**Description:** 
Operators that are expected to sometimes fail, can result in error logs
that are quite verbose.
This aims to address this issue by getting rid of error logs that the
user wishes to silence (debug logs instead).
By making sure no error is returned, there won't be any need to worry
about other components logging error logs.

**Link to tracking Issue:** [Issue
open-telemetry#35008](open-telemetry#35008)

**Testing:** 
Manually tested as this is a matter of logs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working needs triage New item requiring triage pkg/stanza/fileconsumer pkg/stanza receiver/filelog
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@SamerJ