[pkg/stanza] Add an option to resend logs instead of dropping #20864
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
pkg/stanza
processor/logstransform
Foresight Summary
View More Details✅ telemetrygen workflow has finished in 1 minute 1 second and finished at 12th Apr, 2023.
✅ check-links workflow has finished in 1 minute 33 seconds (
|
| Job | Failed Steps | Tests | |
|---|---|---|---|
| changed files | - 🔗 | N/A | See Details |
| check-links | - 🔗 | N/A | See Details |
✅ changelog workflow has finished in 2 minutes 35 seconds and finished at 12th Apr, 2023.
| Job | Failed Steps | Tests | |
|---|---|---|---|
| changelog | - 🔗 | N/A | See Details |
✅ prometheus-compliance-tests workflow has finished in 13 minutes 37 seconds (⚠️ 7 minutes 9 seconds more than main branch avg.) and finished at 12th Apr, 2023.
| Job | Failed Steps | Tests | |
|---|---|---|---|
| prometheus-compliance-tests | - 🔗 | N/A | See Details |
✅ e2e-tests workflow has finished in 17 minutes 4 seconds (⚠️ 2 minutes 57 seconds more than main branch avg.) and finished at 12th Apr, 2023.
| Job | Failed Steps | Tests | |
|---|---|---|---|
| kubernetes-test (v1.26.0) | - 🔗 | N/A | See Details |
| kubernetes-test (v1.25.3) | - 🔗 | N/A | See Details |
| kubernetes-test (v1.24.7) | - 🔗 | N/A | See Details |
| kubernetes-test (v1.23.13) | - 🔗 | N/A | See Details |
❌ build-and-test workflow has finished in 21 minutes 48 seconds (25 minutes less than main branch avg.) and finished at 12th Apr, 2023. 5 jobs failed.
| Job | Failed Steps | Tests | |
|---|---|---|---|
| setup-environment | - 🔗 | N/A | See Details |
| govulncheck | - 🔗 | N/A | See Details |
| check-collector-module-version | - 🔗 | N/A | See Details |
| check-codeowners | - 🔗 | N/A | See Details |
| lint-matrix (receiver-0) | - 🔗 | N/A | See Details |
| lint-matrix (receiver-1) | Lint 🔗 | N/A | See Details |
| lint-matrix (processor) | - 🔗 | N/A | See Details |
| lint-matrix (exporter) | - 🔗 | N/A | See Details |
| lint-matrix (extension) | - 🔗 | N/A | See Details |
| lint-matrix (connector) | - 🔗 | N/A | See Details |
| lint-matrix (internal) | - 🔗 | N/A | See Details |
| lint-matrix (other) | - 🔗 | N/A | See Details |
| checks | - 🔗 | N/A | See Details |
| build-examples | - 🔗 | N/A | See Details |
| correctness-metrics | - 🔗 | N/A | See Details |
| correctness-traces | - 🔗 | N/A | See Details |
| integration-tests | - 🔗 | N/A | See Details |
| unittest-matrix (1.20, receiver-0) | - 🔗 | N/A | See Details |
| unittest-matrix (1.20, receiver-1) | - 🔗 | N/A | See Details |
| unittest-matrix (1.20, processor) | - 🔗 | N/A | See Details |
| unittest-matrix (1.20, exporter) | - 🔗 | N/A | See Details |
| unittest-matrix (1.20, extension) | - 🔗 | N/A | See Details |
| unittest-matrix (1.20, connector) | - 🔗 | N/A | See Details |
| unittest-matrix (1.20, internal) | - 🔗 | N/A | See Details |
| unittest-matrix (1.20, other) | - 🔗 | N/A | See Details |
| unittest-matrix (1.19, receiver-0) | Run Unit Tests 🔗 | N/A | See Details |
| unittest-matrix (1.19, receiver-1) | - 🔗 | N/A | See Details |
| unittest-matrix (1.19, processor) | - 🔗 | N/A | See Details |
| unittest-matrix (1.19, exporter) | - 🔗 | N/A | See Details |
| unittest-matrix (1.19, extension) | - 🔗 | N/A | See Details |
| unittest-matrix (1.19, connector) | - 🔗 | N/A | See Details |
| unittest-matrix (1.19, internal) | - 🔗 | N/A | See Details |
| unittest-matrix (1.19, other) | - 🔗 | N/A | See Details |
| unittest (1.20) | Interpret result 🔗 | N/A | See Details |
| unittest (1.19) | Interpret result 🔗 | N/A | See Details |
| lint | Interpret result 🔗 | N/A | See Details |
| cross-compile | - 🔗 | N/A | See Details |
| build-package | - 🔗 | N/A | See Details |
| windows-msi | - 🔗 | N/A | See Details |
| publish-check | - 🔗 | N/A | See Details |
| publish-stable | - 🔗 | N/A | See Details |
| publish-dev | - 🔗 | N/A | See Details |
| rotate-milestone | - 🔗 | N/A | See Details |
❌ build-and-test-windows workflow has finished in 25 minutes 30 seconds (5 minutes 16 seconds less than main branch avg.) and finished at 12th Apr, 2023. 1 job failed.
| Job | Failed Steps | Tests | |
|---|---|---|---|
| windows-unittest-matrix (receiver-0) | - 🔗 | N/A | See Details |
| windows-unittest-matrix (receiver-1) | Run Unit tests 🔗 | N/A | See Details |
| windows-unittest-matrix (processor) | - 🔗 | N/A | See Details |
| windows-unittest-matrix (exporter) | - 🔗 | N/A | See Details |
| windows-unittest-matrix (extension) | - 🔗 | N/A | See Details |
| windows-unittest-matrix (internal) | - 🔗 | N/A | See Details |
| windows-unittest-matrix (other) | - 🔗 | N/A | See Details |
| windows-unittest | - 🔗 | N/A | See Details |
✅ load-tests workflow has finished in 20 minutes 41 seconds (⚠️ 10 minutes 10 seconds more than main branch avg.) and finished at 12th Apr, 2023.
| Job | Failed Steps | Tests | |
|---|---|---|---|
| setup-environment | - 🔗 | N/A | See Details |
| loadtest (TestIdleMode) | - 🔗 | N/A | See Details |
| loadtest (TestBallastMemory|TestLog10kDPS) | - 🔗 | N/A | See Details |
| loadtest (TestMetric10kDPS|TestMetricsFromFile) | - 🔗 | N/A | See Details |
| loadtest (TestMetricResourceProcessor|TestTrace10kSPS) | - 🔗 | N/A | See Details |
| loadtest (TestTraceNoBackend10kSPS|TestTrace1kSPSWithAttrs) | - 🔗 | N/A | See Details |
| loadtest (TestTraceBallast1kSPSWithAttrs|TestTraceBallast1kSPSAddAttrs) | - 🔗 | N/A | See Details |
| loadtest (TestTraceAttributesProcessor) | - 🔗 | N/A | See Details |
*You can configure Foresight comments in your organization settings page.
atoulme
reviewed
Apr 12, 2023
dmitryax
force-pushed
the
filelogreceiver-dont-drop-logs
branch
3 times, most recently
from
d9aa3a0
to
3492a9c
Compare
dmitryax
force-pushed
the
filelogreceiver-dont-drop-logs
branch
from
3492a9c
to
58262dd
Compare
djaglowski
reviewed
Apr 12, 2023
dloucasfx
reviewed
Apr 12, 2023
dmitryax
force-pushed
the
filelogreceiver-dont-drop-logs
branch
from
58262dd
to
27cad62
Compare
dmitryax
force-pushed
the
filelogreceiver-dont-drop-logs
branch
9 times, most recently
from
f9f872a
to
68a4760
Compare
dmitryax
force-pushed
the
filelogreceiver-dont-drop-logs
branch
from
68a4760
to
1330009
Compare
atoulme
approved these changes
Apr 18, 2023
dmitryax
changed the title
dmitryax
force-pushed
the
filelogreceiver-dont-drop-logs
branch
from
1330009
to
fa5141f
Compare
djaglowski
approved these changes
Apr 24, 2023
MovieStoreGuy
approved these changes
Apr 24, 2023
atoulme
reviewed
Apr 24, 2023
dmitryax
force-pushed
the
filelogreceiver-dont-drop-logs
branch
2 times, most recently
from
1372f3f
to
9eaab62
Compare
dmitryax
force-pushed
the
filelogreceiver-dont-drop-logs
branch
from
9eaab62
to
8c69a00
Compare
Add a `retry_on_failure` config option (disabled by default) to stanza receivers that can be used to slow down reading logs instead of dropping if downstream components return a non-permanent error. The configuration has the following options:
- `enabled`: Enable or disable the retry mechanism. Default is `false`.
- `initial_interval`: The initial interval to wait before retrying. Default is `1s`.
- `max_interval`: The maximum interval to wait before retrying. Default is `30s`.
- `max_elapsed_time`: The maximum amount of time to wait before giving up. Default is `5m`.
The configuration interface is inspired by https://github.com/open-telemetry/opentelemetry-collector/tree/main/exporter/exporterhelper#configuration
which potentially can be exposed in as another package not specific to exporter and used by any components
dmitryax
force-pushed
the
filelogreceiver-dont-drop-logs
branch
from
8c69a00
to
7d18126
Compare
|
Nice. If open-telemetry/opentelemetry-collector#7516 gets accepted let's use to test the filelog receiver. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add a
retry_on_failureconfig option (disabled by default) that can be used to slow down reading logs instead of dropping if downstream components return a non-permanent error. The configuration has the following options:-
enabled: Enable or disable the retry mechanism. Default isfalse.-
initial_interval: The initial interval to wait before retrying. Default is1s.-
max_interval: The maximum interval to wait before retrying. Default is30s.-
max_elapsed_time: The maximum amount of time to wait before giving up. Default is5m.The configuration interface is inspired by https://github.com/open-telemetry/opentelemetry-collector/tree/main/exporter/exporterhelper#configuration
which potentially can be exposed in as another package not specific to exporter and used by any components
Resolves: #20511