Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FUSE] Fix code folding #10459

Merged
merged 5 commits into from
Jun 17, 2024
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 61 additions & 0 deletions docs/Parsing.md
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. This was incredibly helpful for reviewing this PR.

Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# Introduction

This doc aims to keep a list of decisions that have been made around how razor syntax is parsed into a syntax tree.

## Whitespace handling

Whitespace handling is currently differently parsed depending on the chosen emit strategy (runtime or design time).

When in DesignTime whitespace between a CSharp and HTML node is generally parsed as an HTML node, whereas in Runtime the whitespace is parsed as part of a CSharp node. This ensures that at runtime arbitrary whitespace isn't incorrectly emitted as part of the HTML, but in design time the editor will only identify the actual code portion as being CSharp.

An example of this can be seen here: <https://github.com/dotnet/razor/blob/9f10012f7bbee0c17be26de048aee3e5adbc6c80/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/Language/Legacy/CSharpCodeParser.cs#L743>

As part of the transition to use only runtime code generation, we had to make some subtle changes to the parsing of whitespace to ensure that the existing behavior in the editor continues to function as before.

Specifically we changed the parsing of trailing whitespace of razor code block directives (i.e. `@code`, `@function` and `@section`). Previously the whitespace was attached to a meta node that included the closing `}`

Using `^` to indicate whitespace:

```csharp
@code {
// code
}^^^

```

This would be previously be conceptually parsed as something like:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This would be previously be conceptually parsed as something like:
This would previously be conceptually parsed as something like:


```text
CSharpCode
RazorDirective
CSharpTransition
RazorDirectiveBody
RazorMetaCode
Identifier
CSharpCode
...
RazorMetaCode
Literal: }
Literal: ^^^\r\n
```

Thus when looking at the length of the RazorDirective, it includes the `^^^\r\n`. This causes issues with editor features like code folding. The user only want to fold the directive, not the directive and the following new line. (see <https://github.com/dotnet/razor/issues/10358>)

Instead, we now break the trailing whitespace into its own RazorMetaCode node, which is not a part of the directive itself. Conceptually something like:

```text
CSharpCode
RazorDirective
CSharpTransition
RazorDirectiveBody
RazorMetaCode
Identifier
CSharpCode
...
RazorMetaCode
Literal: }
RazorMetaCode
Literal: ^^^\r\n
```

In this way we keep the whitespace as belonging to the overall CSharpCode node, but don't make it part of the directive itself, ensuring the editor sees the correct length for the directive.
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
#nullable disable

using System;
using Microsoft.AspNetCore.Razor.Language.Components;
using Xunit;

namespace Microsoft.AspNetCore.Razor.Language.Legacy;
Expand Down Expand Up @@ -244,4 +245,104 @@ public void SupportsAllKindsOfImplicitMarkupInCodeBlock()
}
""");
}

[Fact]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider adding WorkItemAttributes linking to the issue

public void CodeBlocksTrailingWhitespace_01()
{
ParseDocumentTest("""
@code {
}

""", [ComponentCodeDirective.Directive]);
}

[Fact]
public void CodeBlocksTrailingWhitespace_02()
{
ParseDocumentTest("""
@code{
}

""", [ComponentCodeDirective.Directive]);
}

[Fact]
public void CodeBlocksTrailingWhitespace_03()
{
ParseDocumentTest("""
@code{
} @* comment *@

""", [ComponentCodeDirective.Directive]);
}

[Fact]
public void CodeBlocksTrailingWhitespace_04()
{
ParseDocumentTest("""
@code{
}
@* comment *@

""", [ComponentCodeDirective.Directive]);
}

[Fact]
public void CodeBlocksTrailingWhitespace_05()
{
ParseDocumentTest("""
@code {
}

@code {
}

""", [ComponentCodeDirective.Directive]);
}

[Fact]
public void CodeBlocksTrailingWhitespace_06()
{
ParseDocumentTest("""
@code {
}

<div></div>

""", [ComponentCodeDirective.Directive]);
}

[Fact]
public void CodeBlocksTrailingWhitespace_07()
{
ParseDocumentTest("""
@code {

}
<div></div>

""", [ComponentCodeDirective.Directive]);
}

[Fact]
public void CodeBlocksTrailingWhitespace_08()
{
ParseDocumentTest("""
@code {

} <div></div>

""", [ComponentCodeDirective.Directive]);
}

[Fact]
public void CodeBlocksTrailingWhitespace_09()
{
ParseDocumentTest("""
@code {

}<div></div>

""", [ComponentCodeDirective.Directive]);
}
}
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
Markup span at (0:0,0 [2] ) (Accepts:Any) - Parent: Markup block at (0:0,0 [147] )
Transition span at (2:1,0 [1] ) (Accepts:None) - Parent: Directive block at (2:1,0 [145] )
MetaCode span at (3:1,1 [9] ) (Accepts:None) - Parent: Directive block at (2:1,0 [145] )
None span at (12:1,10 [1] ) (Accepts:AllWhitespace) - Parent: Directive block at (2:1,0 [145] )
MetaCode span at (13:1,11 [1] ) (Accepts:None) - Parent: Directive block at (2:1,0 [145] )
Code span at (14:1,12 [130] ) (Accepts:Any) - Parent: Directive block at (2:1,0 [145] )
MetaCode span at (144:6,0 [3] ) (Accepts:None) - Parent: Directive block at (2:1,0 [145] )
Markup span at (147:7,0 [0] ) (Accepts:Any) - Parent: Markup block at (0:0,0 [147] )
Transition span at (2:1,0 [1] ) (Accepts:None) - Parent: Directive block at (2:1,0 [143] )
MetaCode span at (3:1,1 [9] ) (Accepts:None) - Parent: Directive block at (2:1,0 [143] )
None span at (12:1,10 [1] ) (Accepts:AllWhitespace) - Parent: Directive block at (2:1,0 [143] )
MetaCode span at (13:1,11 [1] ) (Accepts:None) - Parent: Directive block at (2:1,0 [143] )
Code span at (14:1,12 [130] ) (Accepts:Any) - Parent: Directive block at (2:1,0 [143] )
MetaCode span at (144:6,0 [1] ) (Accepts:None) - Parent: Directive block at (2:1,0 [143] )
MetaCode span at (145:6,1 [2] ) (Accepts:Any) - Parent: Statement block at (2:1,0 [145] )
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,13 @@
MarkupTextLiteral - [0..2)::2 - [LF] - Gen<Markup> - SpanEditHandler;Accepts:Any
NewLine;[LF];
CSharpCodeBlock - [2..147)::145
RazorDirective - [2..147)::145 - Directive:{functions;CodeBlock;Unrestricted}
RazorDirective - [2..145)::143 - Directive:{functions;CodeBlock;Unrestricted}
CSharpTransition - [2..3)::1 - Gen<None> - SpanEditHandler;Accepts:None
Transition;[@];
RazorDirectiveBody - [3..147)::144
RazorDirectiveBody - [3..145)::142
RazorMetaCode - [3..12)::9 - Gen<None> - SpanEditHandler;Accepts:None
Identifier;[functions];
CSharpCodeBlock - [12..147)::135
CSharpCodeBlock - [12..145)::133
UnclassifiedTextLiteral - [12..13)::1 - [ ] - Gen<None> - SpanEditHandler;Accepts:AllWhitespace
Whitespace;[ ];
RazorMetaCode - [13..14)::1 - Gen<None> - AutoCompleteEditHandler;Accepts:None,AutoComplete:[<null>];AtEnd
Expand Down Expand Up @@ -55,9 +55,8 @@
Whitespace;[ ];
RightBrace;[}];
NewLine;[LF];
RazorMetaCode - [144..147)::3 - Gen<None> - SpanEditHandler;Accepts:None
RazorMetaCode - [144..145)::1 - Gen<None> - SpanEditHandler;Accepts:None
RightBrace;[}];
NewLine;[LF];
MarkupTextLiteral - [147..147)::0 - [] - Gen<Markup> - SpanEditHandler;Accepts:Any
Marker;[];
RazorMetaCode - [145..147)::2 - Gen<None> - SpanEditHandler;Accepts:Any
NewLine;[LF];
EndOfFile;[];
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
Markup span at (0:0,0 [2] ) (Accepts:Any) - Parent: Markup block at (0:0,0 [408] )
Transition span at (2:1,0 [1] ) (Accepts:None) - Parent: Directive block at (2:1,0 [406] )
MetaCode span at (3:1,1 [9] ) (Accepts:None) - Parent: Directive block at (2:1,0 [406] )
None span at (12:1,10 [1] ) (Accepts:AllWhitespace) - Parent: Directive block at (2:1,0 [406] )
MetaCode span at (13:1,11 [1] ) (Accepts:None) - Parent: Directive block at (2:1,0 [406] )
Transition span at (2:1,0 [1] ) (Accepts:None) - Parent: Directive block at (2:1,0 [404] )
MetaCode span at (3:1,1 [9] ) (Accepts:None) - Parent: Directive block at (2:1,0 [404] )
None span at (12:1,10 [1] ) (Accepts:AllWhitespace) - Parent: Directive block at (2:1,0 [404] )
MetaCode span at (13:1,11 [1] ) (Accepts:None) - Parent: Directive block at (2:1,0 [404] )
Code span at (14:1,12 [88] ) (Accepts:Any) - Parent: Statement block at (14:1,12 [391] )
Markup span at (102:4,32 [3] ) (Accepts:None) - Parent: Tag block at (102:4,32 [3] )
Markup span at (105:4,35 [9] ) (Accepts:Any) - Parent: Markup block at (102:4,32 [26] )
Expand All @@ -25,5 +25,5 @@ Markup span at (313:11,33 [6] ) (Accepts:None) - Parent: Markup block at (292:11
Code span at (319:12,0 [31] ) (Accepts:Any) - Parent: Statement block at (14:1,12 [391] )
Code span at (350:13,29 [1] ) (Accepts:Any) - Parent: Statement block at (14:1,12 [391] )
Code span at (351:13,30 [54] ) (Accepts:Any) - Parent: Statement block at (14:1,12 [391] )
MetaCode span at (405:15,0 [3] ) (Accepts:None) - Parent: Directive block at (2:1,0 [406] )
Markup span at (408:16,0 [0] ) (Accepts:Any) - Parent: Markup block at (0:0,0 [408] )
MetaCode span at (405:15,0 [1] ) (Accepts:None) - Parent: Directive block at (2:1,0 [404] )
MetaCode span at (406:15,1 [2] ) (Accepts:Any) - Parent: Statement block at (2:1,0 [406] )
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,13 @@
MarkupTextLiteral - [0..2)::2 - [LF] - Gen<Markup> - SpanEditHandler;Accepts:Any
NewLine;[LF];
CSharpCodeBlock - [2..408)::406
RazorDirective - [2..408)::406 - Directive:{functions;CodeBlock;Unrestricted} [RZ1008(102:4,32 [1] ), RZ1008(206:8,12 [1] ), RZ1008(292:11,12 [2] ), RZ1008(350:13,29 [2] )]
RazorDirective - [2..406)::404 - Directive:{functions;CodeBlock;Unrestricted} [RZ1008(102:4,32 [1] ), RZ1008(206:8,12 [1] ), RZ1008(292:11,12 [2] ), RZ1008(350:13,29 [2] )]
CSharpTransition - [2..3)::1 - Gen<None> - SpanEditHandler;Accepts:None
Transition;[@];
RazorDirectiveBody - [3..408)::405
RazorDirectiveBody - [3..406)::403
RazorMetaCode - [3..12)::9 - Gen<None> - SpanEditHandler;Accepts:None
Identifier;[functions];
CSharpCodeBlock - [12..408)::396
CSharpCodeBlock - [12..406)::394
UnclassifiedTextLiteral - [12..13)::1 - [ ] - Gen<None> - SpanEditHandler;Accepts:AllWhitespace
Whitespace;[ ];
RazorMetaCode - [13..14)::1 - Gen<None> - AutoCompleteEditHandler;Accepts:None,AutoComplete:[<null>];AtEnd
Expand Down Expand Up @@ -186,9 +186,8 @@
Whitespace;[ ];
RightBrace;[}];
NewLine;[LF];
RazorMetaCode - [405..408)::3 - Gen<None> - SpanEditHandler;Accepts:None
RazorMetaCode - [405..406)::1 - Gen<None> - SpanEditHandler;Accepts:None
RightBrace;[}];
NewLine;[LF];
MarkupTextLiteral - [408..408)::0 - [] - Gen<Markup> - SpanEditHandler;Accepts:Any
Marker;[];
RazorMetaCode - [406..408)::2 - Gen<None> - SpanEditHandler;Accepts:Any
NewLine;[LF];
EndOfFile;[];
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
Markup span at (0:0,0 [2] ) (Accepts:Any) - Parent: Markup block at (0:0,0 [210] )
Transition span at (2:1,0 [1] ) (Accepts:None) - Parent: Directive block at (2:1,0 [208] )
MetaCode span at (3:1,1 [9] ) (Accepts:None) - Parent: Directive block at (2:1,0 [208] )
None span at (12:1,10 [1] ) (Accepts:AllWhitespace) - Parent: Directive block at (2:1,0 [208] )
MetaCode span at (13:1,11 [1] ) (Accepts:None) - Parent: Directive block at (2:1,0 [208] )
Transition span at (2:1,0 [1] ) (Accepts:None) - Parent: Directive block at (2:1,0 [206] )
MetaCode span at (3:1,1 [9] ) (Accepts:None) - Parent: Directive block at (2:1,0 [206] )
None span at (12:1,10 [1] ) (Accepts:AllWhitespace) - Parent: Directive block at (2:1,0 [206] )
MetaCode span at (13:1,11 [1] ) (Accepts:None) - Parent: Directive block at (2:1,0 [206] )
Code span at (14:1,12 [47] ) (Accepts:Any) - Parent: Statement block at (14:1,12 [193] )
Markup span at (61:4,0 [8] ) (Accepts:Any) - Parent: Markup block at (61:4,0 [139] )
Markup span at (69:4,8 [5] ) (Accepts:None) - Parent: Tag block at (69:4,8 [5] )
Expand All @@ -22,5 +22,5 @@ Markup span at (184:9,0 [8] ) (Accepts:Any) - Parent: Markup block at (61:4,0 [1
Markup span at (192:9,8 [6] ) (Accepts:None) - Parent: Tag block at (192:9,8 [6] )
Markup span at (198:9,14 [2] ) (Accepts:None) - Parent: Markup block at (61:4,0 [139] )
Code span at (200:10,0 [7] ) (Accepts:Any) - Parent: Statement block at (14:1,12 [193] )
MetaCode span at (207:11,0 [3] ) (Accepts:None) - Parent: Directive block at (2:1,0 [208] )
Markup span at (210:12,0 [0] ) (Accepts:Any) - Parent: Markup block at (0:0,0 [210] )
MetaCode span at (207:11,0 [1] ) (Accepts:None) - Parent: Directive block at (2:1,0 [206] )
MetaCode span at (208:11,1 [2] ) (Accepts:Any) - Parent: Statement block at (2:1,0 [208] )
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,13 @@
MarkupTextLiteral - [0..2)::2 - [LF] - Gen<Markup> - SpanEditHandler;Accepts:Any
NewLine;[LF];
CSharpCodeBlock - [2..210)::208
RazorDirective - [2..210)::208 - Directive:{functions;CodeBlock;Unrestricted}
RazorDirective - [2..208)::206 - Directive:{functions;CodeBlock;Unrestricted}
CSharpTransition - [2..3)::1 - Gen<None> - SpanEditHandler;Accepts:None
Transition;[@];
RazorDirectiveBody - [3..210)::207
RazorDirectiveBody - [3..208)::205
RazorMetaCode - [3..12)::9 - Gen<None> - SpanEditHandler;Accepts:None
Identifier;[functions];
CSharpCodeBlock - [12..210)::198
CSharpCodeBlock - [12..208)::196
UnclassifiedTextLiteral - [12..13)::1 - [ ] - Gen<None> - SpanEditHandler;Accepts:AllWhitespace
Whitespace;[ ];
RazorMetaCode - [13..14)::1 - Gen<None> - AutoCompleteEditHandler;Accepts:None,AutoComplete:[<null>];AtEnd
Expand Down Expand Up @@ -105,9 +105,8 @@
Whitespace;[ ];
RightBrace;[}];
NewLine;[LF];
RazorMetaCode - [207..210)::3 - Gen<None> - SpanEditHandler;Accepts:None
RazorMetaCode - [207..208)::1 - Gen<None> - SpanEditHandler;Accepts:None
RightBrace;[}];
NewLine;[LF];
MarkupTextLiteral - [210..210)::0 - [] - Gen<Markup> - SpanEditHandler;Accepts:Any
Marker;[];
RazorMetaCode - [208..210)::2 - Gen<None> - SpanEditHandler;Accepts:Any
NewLine;[LF];
EndOfFile;[];
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
Markup span at (0:0,0 [2] ) (Accepts:Any) - Parent: Markup block at (0:0,0 [81] )
Transition span at (2:1,0 [1] ) (Accepts:None) - Parent: Directive block at (2:1,0 [79] )
MetaCode span at (3:1,1 [9] ) (Accepts:None) - Parent: Directive block at (2:1,0 [79] )
None span at (12:1,10 [1] ) (Accepts:AllWhitespace) - Parent: Directive block at (2:1,0 [79] )
MetaCode span at (13:1,11 [1] ) (Accepts:None) - Parent: Directive block at (2:1,0 [79] )
Code span at (14:1,12 [64] ) (Accepts:Any) - Parent: Directive block at (2:1,0 [79] )
MetaCode span at (78:3,0 [3] ) (Accepts:None) - Parent: Directive block at (2:1,0 [79] )
Markup span at (81:4,0 [0] ) (Accepts:Any) - Parent: Markup block at (0:0,0 [81] )
Transition span at (2:1,0 [1] ) (Accepts:None) - Parent: Directive block at (2:1,0 [77] )
MetaCode span at (3:1,1 [9] ) (Accepts:None) - Parent: Directive block at (2:1,0 [77] )
None span at (12:1,10 [1] ) (Accepts:AllWhitespace) - Parent: Directive block at (2:1,0 [77] )
MetaCode span at (13:1,11 [1] ) (Accepts:None) - Parent: Directive block at (2:1,0 [77] )
Code span at (14:1,12 [64] ) (Accepts:Any) - Parent: Directive block at (2:1,0 [77] )
MetaCode span at (78:3,0 [1] ) (Accepts:None) - Parent: Directive block at (2:1,0 [77] )
MetaCode span at (79:3,1 [2] ) (Accepts:Any) - Parent: Statement block at (2:1,0 [79] )
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,13 @@
MarkupTextLiteral - [0..2)::2 - [LF] - Gen<Markup> - SpanEditHandler;Accepts:Any
NewLine;[LF];
CSharpCodeBlock - [2..81)::79
RazorDirective - [2..81)::79 - Directive:{functions;CodeBlock;Unrestricted}
RazorDirective - [2..79)::77 - Directive:{functions;CodeBlock;Unrestricted}
CSharpTransition - [2..3)::1 - Gen<None> - SpanEditHandler;Accepts:None
Transition;[@];
RazorDirectiveBody - [3..81)::78
RazorDirectiveBody - [3..79)::76
RazorMetaCode - [3..12)::9 - Gen<None> - SpanEditHandler;Accepts:None
Identifier;[functions];
CSharpCodeBlock - [12..81)::69
CSharpCodeBlock - [12..79)::67
UnclassifiedTextLiteral - [12..13)::1 - [ ] - Gen<None> - SpanEditHandler;Accepts:AllWhitespace
Whitespace;[ ];
RazorMetaCode - [13..14)::1 - Gen<None> - AutoCompleteEditHandler;Accepts:None,AutoComplete:[<null>];AtEnd
Expand All @@ -32,9 +32,8 @@
StringLiteral;["<h3>@message</h3>"];
Semicolon;[;];
NewLine;[LF];
RazorMetaCode - [78..81)::3 - Gen<None> - SpanEditHandler;Accepts:None
RazorMetaCode - [78..79)::1 - Gen<None> - SpanEditHandler;Accepts:None
RightBrace;[}];
NewLine;[LF];
MarkupTextLiteral - [81..81)::0 - [] - Gen<Markup> - SpanEditHandler;Accepts:Any
Marker;[];
RazorMetaCode - [79..81)::2 - Gen<None> - SpanEditHandler;Accepts:Any
NewLine;[LF];
EndOfFile;[];
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
Markup span at (0:0,0 [2] ) (Accepts:Any) - Parent: Markup block at (0:0,0 [82] )
Transition span at (2:1,0 [1] ) (Accepts:None) - Parent: Directive block at (2:1,0 [80] )
MetaCode span at (3:1,1 [9] ) (Accepts:None) - Parent: Directive block at (2:1,0 [80] )
None span at (12:1,10 [1] ) (Accepts:AllWhitespace) - Parent: Directive block at (2:1,0 [80] )
MetaCode span at (13:1,11 [1] ) (Accepts:None) - Parent: Directive block at (2:1,0 [80] )
Code span at (14:1,12 [65] ) (Accepts:Any) - Parent: Directive block at (2:1,0 [80] )
MetaCode span at (79:3,0 [3] ) (Accepts:None) - Parent: Directive block at (2:1,0 [80] )
Markup span at (82:4,0 [0] ) (Accepts:Any) - Parent: Markup block at (0:0,0 [82] )
Transition span at (2:1,0 [1] ) (Accepts:None) - Parent: Directive block at (2:1,0 [78] )
MetaCode span at (3:1,1 [9] ) (Accepts:None) - Parent: Directive block at (2:1,0 [78] )
None span at (12:1,10 [1] ) (Accepts:AllWhitespace) - Parent: Directive block at (2:1,0 [78] )
MetaCode span at (13:1,11 [1] ) (Accepts:None) - Parent: Directive block at (2:1,0 [78] )
Code span at (14:1,12 [65] ) (Accepts:Any) - Parent: Directive block at (2:1,0 [78] )
MetaCode span at (79:3,0 [1] ) (Accepts:None) - Parent: Directive block at (2:1,0 [78] )
MetaCode span at (80:3,1 [2] ) (Accepts:Any) - Parent: Statement block at (2:1,0 [80] )
Loading