BatchFixAllProvider crashes with "An element with the same key but a different value already exists"

[jnm2]

Version Used: 17.0.0

Got this after trying to invoking "remove unnecessary usings" for a whole solution after adding global usings. VS froze for a while, then showed this (it opened at this position and didn't move at all) and remained frozen for a long time:

Then the gold bar appeared saying that BatchFixAllProvider encountered an error:

System.ArgumentException : An element with the same key but a different value already exists. Key: '(DocumentId, #bc308e8d-b194-f1c4-c02d-910f2e9fa704 - SomeTestClass.TestContainer.g.cs)'
   at System.Collections.Immutable.ImmutableSortedDictionary`2.FillFromEmpty(IEnumerable`1 items,Boolean overwriteOnCollision)
   at System.Collections.Immutable.ImmutableSortedDictionary`2.AddRange(IEnumerable`1 items,Boolean overwriteOnCollision,Boolean avoidToSortedMap)
   at System.Collections.Immutable.ImmutableSortedDictionary.ToImmutableSortedDictionary[TSource,TKey,TValue](IEnumerable`1 source,Func`2 keySelector,Func`2 elementSelector,IComparer`1 keyComparer)
   at Microsoft.CodeAnalysis.TextDocumentStates`1..ctor(IEnumerable`1 states)
   at async Microsoft.CodeAnalysis.SolutionState.CompilationTracker.FinalizeCompilationAsync(<Unknown Parameters>)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Microsoft.VisualStudio.Telemetry.WindowsErrorReporting.WatsonReport.GetClrWatsonExceptionInfo(Exception exceptionObject)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.CodeAnalysis.SolutionState.CompilationTracker.BuildCompilationInfoFromScratchAsync(<Unknown Parameters>)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.CodeAnalysis.SolutionState.CompilationTracker.BuildCompilationInfoAsync(<Unknown Parameters>)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.CodeAnalysis.SolutionState.CompilationTracker.GetOrBuildCompilationInfoAsync(<Unknown Parameters>)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.CodeAnalysis.SolutionState.CompilationTracker.GetCompilationSlowAsync(<Unknown Parameters>)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.CodeAnalysis.Diagnostics.AnalyzerHelper.CreateCompilationWithAnalyzersAsync(<Unknown Parameters>)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.CodeAnalysis.Diagnostics.EngineV2.DiagnosticIncrementalAnalyzer.IdeLatestDiagnosticGetter.AppendDiagnosticsAsync(<Unknown Parameters>)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.CodeAnalysis.Diagnostics.EngineV2.DiagnosticIncrementalAnalyzer.DiagnosticGetter.GetDiagnosticsAsync(<Unknown Parameters>)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.CodeAnalysis.CodeFixes.CodeFixService.GetProjectDiagnosticsAsync(<Unknown Parameters>)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.CodeAnalysis.CodeFixes.FixAllContext.GetFilteredDiagnosticsAsync(<Unknown Parameters>)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.CodeAnalysis.CodeFixes.FixAllContext.GetProjectDiagnosticsAsync(<Unknown Parameters>)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.CodeAnalysis.CodeFixes.FixAllContextHelper.GetDocumentDiagnosticsToFixAsync(<Unknown Parameters>)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.CodeAnalysis.CodeFixes.FixAllContext.DiagnosticProvider.<GetDocumentDiagnosticsToFixAsync>g__GetDocumentDiagnosticsToFixWorkerAsync|3_1(<Unknown Parameters>)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.CodeAnalysis.CodeFixes.FixAllContext.DiagnosticProvider.GetDocumentDiagnosticsToFixAsync(<Unknown Parameters>)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.CodeAnalysis.CodeFixes.BatchFixAllProvider.DetermineDiagnosticsAsync(<Unknown Parameters>)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.CodeAnalysis.CodeFixes.BatchFixAllProvider.FixSingleContextAsync(<Unknown Parameters>)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.CodeAnalysis.CodeFixes.BatchFixAllProvider.FixAllContextsAsync(<Unknown Parameters>)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.CodeAnalysis.CodeFixes.DefaultFixAllProviderHelpers.GetFixAsync(<Unknown Parameters>)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.CodeAnalysis.Editor.Implementation.Suggestions.FixAllGetFixesService.GetFixAllCodeActionAsync(<Unknown Parameters>)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.CodeAnalysis.Editor.Implementation.Suggestions.FixAllGetFixesService.GetFixAllOperationsAsync(<Unknown Parameters>)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at async Microsoft.CodeAnalysis.CodeActions.CodeAction.GetOperationsCoreAsync(<Unknown Parameters>)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Roslyn.Utilities.TaskExtensions.WaitAndGetResult_CanCallOnBackground[T](Task`1 task,CancellationToken cancellationToken)
   at Microsoft.CodeAnalysis.Editor.Implementation.Suggestions.SuggestedAction.InvokeWorker(Func`1 getFromDocument,IProgressTracker progressTracker,CancellationToken cancellationToken)
   at Microsoft.CodeAnalysis.Extensions.IExtensionManagerExtensions.PerformAction(IExtensionManager extensionManager,Object extension,Action action)

Restarting VS unblocked this operation.

[mavasani]

We are crashing here:

roslyn/src/Workspaces/Core/Portable/Workspace/Solution/SolutionState.CompilationTracker.cs

Line 882 in c7521c9

var generatedDocuments = new TextDocumentStates<SourceGeneratedDocumentState>(generatedDocumentsBuilder.ToImmutableAndClear());

generatedDocumentsBuilder seems to have multiple source generated document states with the same ID. I am not sure if the correct fix here is to be resilient to this in TextDocumentStates constructor or this needs to be handled in FinalizeCompilationAsync. Moving to @jasonmalinowski.

[jasonmalinowski]

Hmm, @jnm2 do you know what type of project this was? I know we had some bugs if we got the same generator added twice, but not sure if this would manifest here precisely. I'd assume that if that was happening this wouldn't have worked after restart though...

[jasonmalinowski]

So I'm not able to reproduce by a naive test, but look at telemetry it does appear there's a number of hits with this same stack, including 17.1 too so it's not something we've fixed yet. I've opened https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1463535 as an internal bug which links up with our telemetry systems so we can get more info on it.

[jasonmalinowski]

We did get a telemetry report with an attached crash dump. The GeneratorDriver's list of generators has a duplicate generator; we have two instances of MvvmGen.SourceGenerators's MvvmGen.ViewModelGenerator loaded and added to the GeneratorDriver at once. Oddly, it appears to be two separate instances of the same generator object, i.e. .GetType() on the two generators will give the same result and are loaded from the same assembly. That seems hard to have happen, as AnalyzerFileReference tries to cache the same object; my hunch is we must have removed and readded a reference and there's a bug where the first removal didn't work (due to some sort of mismatch with the object identities), and a later re-addition added a second copy.

In any case, this doesn't appear to be a duplicate of #56619 which I was hoping it might be, or at least at the point of the fault we no longer have a duplicate in the main Project.AnalyzerReferences list.

@jnm2: when you said you added global usings, do you recall if you did that by editing a source file, or editing the project file? I recognize I'm asking you to remember a subtle detail from months ago, but if you think it was a project file edit that'd make sense.

[jasonmalinowski]

The crash was in our OOP process which also makes me think it could have been fallout from #58990, but we're still seeing hits for 17.2 so that doesn't explain everything.

[jnm2]

@jasonmalinowski It for sure was by editing the project file. It's easy to remember because I haven't written a global using in a source file ever.

[jasonmalinowski]

This should be fixed by #65039.