Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[compute_ctl] Add timeout for tracing_utils::shutdown_tracing() #3982

Merged
merged 1 commit into from
Apr 11, 2023

Conversation

ololobus
Copy link
Member

@ololobus ololobus commented Apr 7, 2023

Describe your changes

Shutting down OTEL tracing provider may hang for quite some time, see, for example:

Yet, we want computes to shut down fast enough, as we may need a new one for the same timeline ASAP. So wait no longer than 2s for the shutdown to complete, then just error out and exit the main thread.

Issue ticket number and link

Related to neondatabase/cloud#3707

Checklist before requesting a review

  • I have performed a self-review of my code.
  • If it is a core feature, I have added thorough tests.
  • Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
  • If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

  • Do not forget to reformat commit message to not include the above checklist

@ololobus ololobus requested a review from a team as a code owner April 7, 2023 11:59
@ololobus ololobus requested review from Daniel-ef and removed request for a team April 7, 2023 11:59
@github-actions
Copy link

github-actions bot commented Apr 7, 2023

Test results for 92ba921:


debug build: 212 tests run: 202 passed, 0 failed, 10 (full report)


release build: 212 tests run: 202 passed, 0 failed, 10 (full report)


@ololobus ololobus force-pushed the alexk/compute-otel-timeout branch from 4cd6bbc to a6959a4 Compare April 7, 2023 12:44
@Daniel-ef Daniel-ef removed their request for review April 7, 2023 13:10
Shutting down OTEL tracing provider may hang for quite some time,
see, for example:
- open-telemetry/opentelemetry-rust#868
- and our problems with staging neondatabase/cloud#3707 (comment)

Yet, we want computes to shut down fast enough, as we may need a new one
for the same timeline ASAP. So wait no longer than 2s for the shutdown to
complete, then just error out and exit the main thread.

Related to neondatabase/cloud#3707
@ololobus ololobus force-pushed the alexk/compute-otel-timeout branch from a6959a4 to 92ba921 Compare April 11, 2023 11:42
@ololobus ololobus merged commit 40a68e9 into main Apr 11, 2023
@ololobus ololobus deleted the alexk/compute-otel-timeout branch April 11, 2023 13:05
ThomAub added a commit to ThomAub/tracing-opentelemetry-instrumentation-sdk that referenced this pull request Jun 9, 2024
davidB pushed a commit to davidB/tracing-opentelemetry-instrumentation-sdk that referenced this pull request Jun 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants