Always run the razor generator even in balanced mode #79510

chsienki · 2025-07-22T05:04:49Z

Change DoNotCreate to CreateRequired
Add a flag to remote generation service that allows the caller to specify required generators only
Pass the flag to OOP based on the creation policy
Set the creation policy when creating a compilation tracker
Ensure we add in previous docs that weren't regenerated when running only required generators
Add a filter that only runs razor when required generators are requested

- Change DoNotCreate to CreateRequired - Add a flag to remote generation service that allows the caller to specify required generators only - Pass the flag to OOP based on the creation policy - Set the creation policy when creating a compilation tracker - Ensure we add in previous docs that weren't regenerated when running only required generators - Add a filter that only runs razor when required generators are requested

src/Compilers/Test/Core/SourceGeneration/TestGenerators.cs

CyrusNajmabadi · 2025-07-22T07:01:20Z

src/Workspaces/Core/Portable/SourceGeneration/IRemoteSourceGenerationService.cs

    /// cref="GeneratorDriver"/> should be included.</param>
+    /// <param name="requiredDocumentsOnly">Controls if the caller only wants to run required generators and use old
+    /// results for other generators, or if all generators should be run to get new documents.</param>
    ValueTask<ImmutableArray<SourceGeneratedDocumentInfo>> GetSourceGeneratedDocumentInfoAsync(


Not a complaint. But wondering if we should merge the booleans into some options/enum type.

Question (and consider doc'ing). What are the semantics of calling this multiple times, where all the values are the same, but 'requireDocumentsOnly' is true/false.

I'm fairly sure i want the following:

order doesn't matter. you get the same results regardless of the order you may have called this.

passing 'requiredDoucmentsOnly=false' always gives a superset of 'requiredDocumentsOnly=true'.

CyrusNajmabadi · 2025-07-22T07:02:18Z

src/Workspaces/Core/Portable/Workspace/Solution/SolutionCompilationState.CreationPolicy.cs

        /// references.  For both, use whatever has been generated most recently.
        /// </summary>
-        public static readonly CreationPolicy DoNotCreate = new(GeneratedDocumentCreationPolicy.DoNotCreate, SkeletonReferenceCreationPolicy.DoNotCreate);
+        public static readonly CreationPolicy DoNotCreate = new(GeneratedDocumentCreationPolicy.CreateRequired, SkeletonReferenceCreationPolicy.DoNotCreate);


nit: the parameter was requiredDocumentsOnly. perhaps .CreateRequiredOnly. I'm liking the 'only' to indicate very specifically exactly what it is controlling, and waht it isn't.

Ok. As vebose as it might be, i think CreationPolicy CreateRequiredGeneratedDocuments_DoNotCreateSkeletonReferences is likely appropriate. Having this be called 'DoNotCreate' now feels misleading.

CyrusNajmabadi · 2025-07-22T07:03:36Z

src/VisualStudio/Core/Test.Next/Services/ServiceHubServicesTests.cs

+        await ValidateSourceGeneratorDocuments(expectedCallback: 0,
+                                               expectedRazor: 0);


Suggested change

await ValidateSourceGeneratorDocuments(expectedCallback: 0,

expectedRazor: 0);

await ValidateSourceGeneratorDocuments(expectedCallback: 0,

expectedRazor: 0);

i hate this formattign :D. can you either have this be one line, or have it be:

Suggested change

await ValidateSourceGeneratorDocuments(expectedCallback: 0,

expectedRazor: 0);

await ValidateSourceGeneratorDocuments(

expectedCallback: 0,

expectedRazor: 0);

CyrusNajmabadi · 2025-07-22T07:04:11Z

src/VisualStudio/Core/Test.Next/Services/ServiceHubServicesTests.cs

+
+        // Get the documents again and ensure nothing ran
+        await ValidateSourceGeneratorDocuments(expectedCallback: executionPreference == SourceGeneratorExecutionPreference.Automatic ? 1 : 0,
+                                               expectedRazor: 1);


same with all of these.

also: thanks for validating automatic and balanced.

src/Compilers/Test/Core/SourceGeneration/TestGenerators.cs

CyrusNajmabadi · 2025-07-22T07:06:08Z

src/VisualStudio/Core/Test.Next/Services/ServiceHubServicesTests.cs

+
+        // Make another change, but this time enqueue an update too
+        Contract.ThrowIfFalse(workspace.TryApplyChanges(workspace.CurrentSolution.WithDocumentText(tempDoc.Id, SourceText.From("// more new text"))));
+        workspace.EnqueueUpdateSourceGeneratorVersion(projectId: null, forceRegeneration: false);


i wouldn't mind a test where we're in balanced, and we force-regen, and we see everything properly updated.

src/Workspaces/Core/Portable/Workspace/Solution/SolutionCompilationState.CreationPolicy.cs

CyrusNajmabadi · 2025-07-22T07:11:09Z

...paces/Core/Portable/Workspace/Solution/SolutionCompilationState.RegularCompilationTracker.cs

                {
                    if (creationPolicy.GeneratedDocumentCreationPolicy == GeneratedDocumentCreationPolicy.Create)
-                        creationPolicy = creationPolicy with { GeneratedDocumentCreationPolicy = GeneratedDocumentCreationPolicy.DoNotCreate };
+                        creationPolicy = creationPolicy with { GeneratedDocumentCreationPolicy = GeneratedDocumentCreationPolicy.CreateRequired };


I was going to complain that i didn't write docs here, and that was my bad. But turns out i did, and they're great :p

So please update the docs above this appropriately. It should be expanded to indicate what this new mode is, and how it affects 'required' and non-required sgs.

--

aside, as i talk about this more, my brain keeps referring to these generators as 'critical' vs 'required'. perhaps that name might be better. but i' mnot pushing for that.

CyrusNajmabadi · 2025-07-22T07:13:58Z

...Portable/Workspace/Solution/SolutionCompilationState.RegularCompilationTracker_Generators.cs


            // If there are no generated documents, bail out immediately.
-            if (infos.Length == 0)
+            if (creationPolicy == GeneratedDocumentCreationPolicy.Create && infos.Length == 0)


i think it woudl be worth fleshing out the comment above. specifically why the new enum check is in here. (basically, just convey taht, as per the docs, the semantics of .CreateRequired is that we reuse the gen docs from the last run, so we need to now do that below).

CyrusNajmabadi · 2025-07-22T07:17:07Z

...Portable/Workspace/Solution/SolutionCompilationState.RegularCompilationTracker_Generators.cs

+            if (creationPolicy == GeneratedDocumentCreationPolicy.CreateRequired)
+            {
+                // the documents we got back are only for the required generators, meaning any documents from other generators remain the same. 
+                var generatorsThatRan = infos.Select(di => di.DocumentIdentity.Generator).Distinct().ToImmutableHashSet();


Distinct().ToImmutableHashSet() is redundant. Can just do ToImmutableHashSet()

We avoid the heavyweight Immutable types (set/dict) unless we intend to store as a field, and we really need to ensure no one messes with it. For local processing, ToSet/ToHashSet is preferred as the types are much ligher.

i'm tempted to have you literally just ToImmutableArray this, as we should expect that there will legit only be a handful of required generators (if there are more, then the whole 'balanced' mode ends up being rather pointless).

CyrusNajmabadi · 2025-07-22T07:18:19Z

...Portable/Workspace/Solution/SolutionCompilationState.RegularCompilationTracker_Generators.cs

+            if (creationPolicy == GeneratedDocumentCreationPolicy.Create && infos.Length == 0)
                return (compilationWithoutGeneratedFiles, TextDocumentStates<SourceGeneratedDocumentState>.Empty);

            // Next, figure out what is different locally.  Specifically, what documents we don't know about, or we


note to self. man this method is big... but breaking it up may just obfuscate. feels like maybe a helper type might be good here.

Ok. the method is big enough, and complex enough that i cannot convince myself that the new code is correct.

specifically the new code has bail-out points prior to you doign the work to move the old generated docs forward. For example, it has this check below:

this is either a bug.

it's not a bug, but we need DEEP comments explaining why this logic is ok.

--

Aside: I'm not sure if this is possible, but it seems like it might be clearer if the original lgoic here stays mostly unchanged (just extracted to a helper method). Then the method that calls that new helper is the one that takes the results, and does all the manipulations related to the semantics of CreateRequired. So it goes "oh, we only ran a subset of generators, and here are hte results of that. I will now add in the old docs as necessary). That would help keep this code simpler as well, including (i think), still being able to trivially bail out in the infos.Length == 0 case, and whatnot.

The more i say this, the more i want this.

...Portable/Workspace/Solution/SolutionCompilationState.RegularCompilationTracker_Generators.cs

src/Workspaces/Core/Portable/Workspace/Solution/SolutionCompilationState.cs

CyrusNajmabadi · 2025-07-22T07:31:47Z

src/Workspaces/Core/Portable/Workspace/Solution/SolutionCompilationState.cs

-            ? GetCompilationTracker(project.Id).GetSourceGeneratedDocumentStatesAsync(this, withFrozenSourceGeneratedDocuments, cancellationToken)
+            ? GetCompilationTracker(project.Id, requiredDocumentsOnly).GetSourceGeneratedDocumentStatesAsync(this, withFrozenSourceGeneratedDocuments, cancellationToken)
            : new(TextDocumentStates<SourceGeneratedDocumentState>.Empty);
    }


note: i'm not opposed to us effectively havign two compilation trackers here depending on what clients may be asking for. What I am trying to figure out is if one compilation tracker should be built from teh other.

Specifically, it feels like we have a bit of a russian doll situation with these systems. 'CreateRequired' runs just the required generators, and 'Create' runs the rest. In other words, 'Create' should be able to pick up the results that 'CreateRequired' produced for the 'required generators'.

--

this actually made me realize something that i absolutely want audited/tested. It seemsl ike it would be bad (imo) if you had the same solution snapshot, and could observe generated required docs (the docs produced from CreateRequired) not being exactly the same if 'Create' then ran and produced docs from teh required generators.

(i don't know if that's actually something possible... since maybe that requires a different fork of the solution. )

src/Workspaces/Remote/ServiceHub/Services/SourceGeneration/RemoteSourceGenerationService.cs

Add assert

…orce the tracker if it needs it

chsienki · 2025-08-27T17:35:31Z

/backport to release/dev18.0

github-actions · 2025-08-27T17:35:40Z

Started backporting to release/dev18.0: https://github.com/dotnet/roslyn/actions/runs/17274211923

github-actions · 2025-08-27T17:53:00Z

@chsienki backporting to "release/dev18.0" failed, the patch most likely resulted in conflicts:

$ git am --3way --empty=keep --ignore-whitespace --keep-non-patch changes.patch

Applying: Always run 'required' generators: - Change DoNotCreate to CreateRequired - Add a flag to remote generation service that allows the caller to specify required generators only - Pass the flag to OOP based on the creation policy - Set the creation policy when creating a compilation tracker - Ensure we add in previous docs that weren't regenerated when running only required generators - Add a filter that only runs razor when required generators are requested
.git/rebase-apply/patch:163: trailing whitespace.
                // the documents we got back are only for the required generators, meaning any documents from other generators remain the same. 
warning: 1 line adds whitespace errors.
Using index info to reconstruct a base tree...
M	src/Workspaces/Core/Portable/SourceGeneration/IRemoteSourceGenerationService.cs
M	src/Workspaces/Core/Portable/Workspace/Solution/SolutionCompilationState.RegularCompilationTracker_Generators.cs
M	src/Workspaces/Core/Portable/Workspace/Solution/SolutionCompilationState.cs
M	src/Workspaces/Remote/ServiceHub/Services/SourceGeneration/RemoteSourceGenerationService.cs
Falling back to patching base and 3-way merge...
Auto-merging src/Workspaces/Core/Portable/SourceGeneration/IRemoteSourceGenerationService.cs
Auto-merging src/Workspaces/Core/Portable/Workspace/Solution/SolutionCompilationState.RegularCompilationTracker_Generators.cs
Auto-merging src/Workspaces/Core/Portable/Workspace/Solution/SolutionCompilationState.cs
Auto-merging src/Workspaces/Remote/ServiceHub/Services/SourceGeneration/RemoteSourceGenerationService.cs
Applying: Tests
Using index info to reconstruct a base tree...
M	src/Compilers/Test/Core/SourceGeneration/TestGenerators.cs
M	src/VisualStudio/Core/Test.Next/Services/ServiceHubServicesTests.cs
Falling back to patching base and 3-way merge...
Auto-merging src/Compilers/Test/Core/SourceGeneration/TestGenerators.cs
CONFLICT (content): Merge conflict in src/Compilers/Test/Core/SourceGeneration/TestGenerators.cs
Auto-merging src/VisualStudio/Core/Test.Next/Services/ServiceHubServicesTests.cs
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Patch failed at 0002 Tests
Error: The process '/usr/bin/git' failed with exit code 128

Please backport manually!

CyrusNajmabadi · 2025-08-27T20:33:47Z

...Portable/Workspace/Solution/SolutionCompilationState.RegularCompilationTracker_Generators.cs

-            if (creationPolicy.GeneratedDocumentCreationPolicy is GeneratedDocumentCreationPolicy.DoNotCreate)
+            var canSkipRunningGenerators = creationPolicy.GeneratedDocumentCreationPolicy is GeneratedDocumentCreationPolicy.DoNotCreate
+                                           || (creationPolicy.GeneratedDocumentCreationPolicy is GeneratedDocumentCreationPolicy.CreateOnlyRequired
+                                               && !await HasRequiredGeneratorsAsync(compilationState, cancellationToken).ConfigureAwait(false));


so. this is blocking feeddback. my brain cannot figure out stuff like this. please extract a local function. Write simple statements for each case, with simple comments on each explainign it :)

CyrusNajmabadi · 2025-08-27T20:35:33Z

src/Workspaces/Core/Portable/SourceGeneration/SourceGeneratorExtensions.cs

+        var hasRequiredGenerators = generators.Any(g => g.IsRequiredGenerator());
+        return hasRequiredGenerators
+               ? SourceGeneratorPresence.ContainsRequiredSourceGenerators
+               : SourceGeneratorPresence.OnlyOptionalSourceGenerators;


dedent. yes, i'm that anal.

CyrusNajmabadi · 2025-08-27T20:36:56Z

src/Workspaces/Core/Portable/SourceGeneration/SourceGeneratorPresence.cs

+    /// <summary>
+    /// The project contains no source generators.
+    /// </summary>
+    NoSourceGenerators,


practically, i assume we're at the point that this never happens. So you could really just have this ContainsRequiredSourceGenerators and DoesNotContainRequiredSourceGenerators (just as a named bool) right?

Note: i'm not blocking this. if you feel this is clear and helpful, then we can keep. I'm just thinking out loud and trying to determine if it really would matter in practice.

CyrusNajmabadi · 2025-08-27T20:44:03Z

src/Workspaces/Remote/ServiceHub/Services/SourceGeneration/RemoteSourceGenerationService.cs

-        return false;
+        return hasGenerators ? SourceGeneratorPresence.OnlyOptionalSourceGenerators
+                             : SourceGeneratorPresence.NoSourceGenerators;
    }


technically, i feel like this loop, and the bool is overwrought. but all the alternatives i can think of aren't much better. rimarily though, i feel like i want this to be:

references.Contains(r => r.Presense is .ContainsRequiredSourceGenerators) ? .ContainsRequiredSourceGenerators : references.Contains(r => r.Presense is .OnlyOptionalSourceGenerators) ? .OnlyOptionalSourceGenerators : .NoSourceGenerators;

it took me a while to understand the looping form. but it's ok with how you have it.

consider just adding a comment indicating that this is the intende dsemantics. s omething like "if at least one reference contains required generators, then we contain required generators. Then if at least one reference has an optional generator, then we have an optional generator. Otherwise, we do nto have generators".

CyrusNajmabadi

Signing off on latest changes again, just with rhe request to clean up that one expression. All the rest is up to you.

CyrusNajmabadi · 2025-08-27T21:36:03Z

src/Workspaces/Remote/ServiceHub/Services/SourceGeneration/RemoteSourceGenerationService.cs


-        return false;
+        // no required generators, all are optional
+        return SourceGeneratorPresence.OnlyOptionalSourceGenerators;


it's unclear to me if this is correct. what if isolatedReferences.Length > 0 but all of them return .NoGenerators for their generatorPresence.

Derp, good catch

Reverted to the previous loop but added extra comments to explain it.

- Add a CreateOnlyRequired generation state - Add a filter that only runs razor when required generators are requested Manual backport of #79510

chsienki added 2 commits July 21, 2025 22:03

Tests

51d2a8f

github-actions bot added the Area-IDE label Jul 22, 2025

dotnet-policy-service bot added the VSCode label Jul 22, 2025