Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deferred Log Dispatcher #10530

Merged
merged 14 commits into from
Oct 22, 2024
Merged

Deferred Log Dispatcher #10530

merged 14 commits into from
Oct 22, 2024

Conversation

RohitRanjanMS
Copy link
Member

@RohitRanjanMS RohitRanjanMS commented Oct 17, 2024

The initial telemetry from the host startup is missing in Application Insights because the ApplicationInsightsLoggerProvider is added later in the process. As a result, important startup logs are not captured in App Insights. The proposed solution addresses this by temporarily storing logs in a channel and forwarding them to Application Insights and/or OpenTelemetry once these providers are added to the logging system.

resolves #9679 and #9361

Pull request checklist

IMPORTANT: Currently, changes must be backported to the in-proc branch to be included in Core Tools and non-Flex deployments.

  • Backporting to the in-proc branch is not required
    • Otherwise: Link to backporting PR
  • My changes do not require documentation changes
    • Otherwise: Documentation issue linked to PR
  • My changes should not be added to the release notes for the next release
    • Otherwise: I've added my notes to release_notes.md
  • My changes do not need to be backported to a previous version
    • Otherwise: Backport tracked by issue/PR #issue_or_pr
  • My changes do not require diagnostic events changes
    • Otherwise: I have added/updated all related diagnostic events and their documentation (Documentation issue linked to PR)
  • I have added all required tests (Unit tests, E2E tests)

@RohitRanjanMS RohitRanjanMS marked this pull request as ready for review October 17, 2024 09:33
@RohitRanjanMS RohitRanjanMS requested a review from a team as a code owner October 17, 2024 09:33
Copy link
Contributor

@jviau jviau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments which hopefully remove the need to touch the other loggers.

Also, this approach will not capture log scopes. Is that okay? Or do you want to buffer scope data as well? Just make sure to capture scope data as a string and not their raw types (they may capture async state which we don't want)

@RohitRanjanMS RohitRanjanMS requested a review from jviau October 18, 2024 16:45
Copy link
Contributor

@jviau jviau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ILoggerProvider approach looks good. However, lets worry about capturing and re-emitting scope in a separate PR.

BUT I still recommend using the ISupportExternalScope for the ILoggerProvider and ILogger implementation in this PR.

src/WebJobs.Script.WebHost/Diagnostics/DeferredLogger.cs Outdated Show resolved Hide resolved
src/WebJobs.Script.WebHost/Diagnostics/DeferredLogger.cs Outdated Show resolved Hide resolved
src/WebJobs.Script.WebHost/Diagnostics/DeferredLogger.cs Outdated Show resolved Hide resolved
src/WebJobs.Script.WebHost/Diagnostics/DeferredLogger.cs Outdated Show resolved Hide resolved
src/WebJobs.Script.WebHost/Diagnostics/DeferredLogger.cs Outdated Show resolved Hide resolved
src/WebJobs.Script.WebHost/WebJobsScriptHostService.cs Outdated Show resolved Hide resolved
src/WebJobs.Script.WebHost/WebJobsScriptHostService.cs Outdated Show resolved Hide resolved
Copy link
Member

@fabiocav fabiocav left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, @RohitRanjanMS

The new iteration looks good, but I'm still concerned about the placeholder logs question.

Have you validated this functionality in placeholder? I want to make sure warmup site logs won't end up in customer facing Application Insights logs.

@RohitRanjanMS RohitRanjanMS dismissed jviau’s stale review October 22, 2024 21:31

Addressed all the comments.

@RohitRanjanMS RohitRanjanMS merged commit 2d36bbe into dev Oct 22, 2024
10 checks passed
@RohitRanjanMS RohitRanjanMS deleted the roranjan/BufferedLogger branch October 22, 2024 21:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Error log about "Worker failed to index functions" does not appear on Azure
5 participants