Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TrackException does not send large Aggregate Exceptions to Application Insights #2946

Open
karpikpl opened this issue Jan 16, 2025 · 5 comments
Labels

Comments

@karpikpl
Copy link

karpikpl commented Jan 16, 2025

  • List of NuGet packages and version that you are using:
    • "Microsoft.ApplicationInsights.AspNetCore": "2.22.0",
    • "Microsoft.ApplicationInsights.Kubernetes": "7.0.0",
  • Runtime version (e.g. net461, net48, netcoreapp2.1, netcoreapp3.1, etc. You can find this information from the *.csproj file):
    • <Project Sdk="Microsoft.NET.Sdk">
    • <OutputType>Exe</OutputType>
    • <TargetFramework>net8.0</TargetFramework>
  • Hosting environment (e.g. Azure Web App, App Service on Linux, Windows, Ubuntu, etc.):
    • Windows x64
    • Ubuntu 22

Describe the bug

I'm using following method to Track Exceptions:

	public void TrackException(Exception ex) {
		_telemetryClient.TrackException(new InvalidOperationException("Coordinator Crashed!"), _properties);
		// for some reason not all exceptions are sent to app insights with `TrackException`
		_telemetryClient.TrackTrace("Coordinator Crashed!: " + ex, SeverityLevel.Critical, _properties);
		_telemetryClient.TrackException(ex, _properties);
	}

I also tried with FlushAsync and Thread.Sleep/Task.Delay() to make sure telemetry is sent.

My service is throwing an exception when connection to Azure App Configuration using DefaultAzureCredentials.

Exception is:

System.TimeoutException: The provider timed out while attempting to load.
 ---> System.AggregateException: One or more errors occurred. (Service request failed.

With the reason for the error being the wrong scope for the token

The received access token does not have authorization to perform action over the requested scope or the scope is invalid. If access was recently granted, please refresh your credentials.

For some reason:

  1. _telemetryClient.TrackException(new InvalidOperationException("Coordinator Crashed!"), _properties); - WORKS
  2. _telemetryClient.TrackTrace("Coordinator Crashed!: " + ex, SeverityLevel.Critical, _properties); - WORKS
  3. _telemetryClient.TrackException(ex, _properties); - DOESN'T WORK - exception is nowhere to be found in app insights
  4. Using exception as inner exception doesn't help - it's not visible in app insights

Image

Exception is thrown when application is crashing (exiting). Adding Console.ReadLine() or flushing telemetry did not help.

Warning

Is there anything in app insights that filters out that exception? it it related to it having WWW-Authenticate data?

To Reproduce

Sample app.

For Immediate Support

For immediate support relating to the Application Insights .NET SDK we encourage you to file an Azure Support Request with Microsoft Azure instead of filing a GitHub Issue in this repository.
You can do so by going online to the Azure portal and submitting a support request. Access to subscription management and billing support is included with your Microsoft Azure subscription, and technical support is provided through one of the Azure Support Plans. For step-by-step guidance for the Azure portal, see How to create an Azure support request. Alternatively, you can create and manage your support tickets programmatically using the Azure Support ticket REST API

@karpikpl karpikpl added the bug label Jan 16, 2025
@TimothyMothra
Copy link
Member

not enough information to know why the Exception is getting dropped.
You would need to collect the internal logs to see what issue the SDK is having. The easiest way to do that would be to follow the self-diagnostics method:
https://github.com/microsoft/ApplicationInsights-dotnet/tree/main/troubleshooting/ETW#self-diagnostics

@karpikpl
Copy link
Author

does the log contains sensitive information?
I can attach it here.

I've added

		} catch (Exception ex) {
			logging.TrackException(ex);
			await logging.FlushAsync(CancellationToken.None);
			Console.WriteLine(ex.Message, Color.Red);
			Task.Delay(5000).Wait();
			Console.WriteLine("Done waiting", Color.Red);
			}

but the log contains:

Successfully opened file.
2025-01-17T05:05:25.7310290Z:Process was called on the TelemetrySink after it was disposed, the telemetry data was dropped.{xxxApp}

Is it possible that Azure Credentials is too big?
After ToString() it's 450 lines and 32803 characters long.

@karpikpl
Copy link
Author

This looks similar to #2284

@karpikpl
Copy link
Author

I've been able to reproduce here:
https://github.com/karpikpl/app-insights-large-exception

@karpikpl karpikpl changed the title TrackException does not send exception to Application Insights TrackException does not send large Aggregate Exceptions to Application Insights Jan 17, 2025
@karpikpl
Copy link
Author

My guess is that SDK is not calculating "max" values when dealing with inner exceptions.

So when I have 1 large exception this code works fine

internal static External.ExceptionDetails ConvertToExceptionDetails(
Exception exception,
External.ExceptionDetails parentExceptionDetails)
{
External.ExceptionDetails exceptionDetails = External.ExceptionDetails.CreateWithoutStackInfo(
exception,
parentExceptionDetails);
// The endpoint cannot ingest message lengths longer than a certain length
if (exceptionDetails.message != null && exceptionDetails.message.Length > MaxExceptionMessageLength)
{
exceptionDetails.message = exceptionDetails.message.Substring(0, MaxExceptionMessageLength);
}

but when there are inner exceptions, each one of them is trimmed but in aggregate, they all exceed max payload size and are getting dropped.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants