Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FUSE] Fix code folding #10459

Merged
merged 5 commits into from
Jun 17, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 61 additions & 0 deletions docs/Parsing.md
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. This was incredibly helpful for reviewing this PR.

Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# Introduction

This doc aims to keep a list of decisions that have been made around how razor syntax is parsed into a syntax tree.

## Whitespace handling

Whitespace handling is currently differently parsed depending on the chosen emit strategy (runtime or design time).

When in DesignTime whitespace between a CSharp and HTML node is generally parsed as an HTML node, whereas in Runtime the whitespace is parsed as part of a CSharp node. This ensures that at runtime arbitrary whitespace isn't incorrectly emitted as part of the HTML, but in design time the editor will only identify the actual code portion as being CSharp.

An example of this can be seen here: <https://github.com/dotnet/razor/blob/9f10012f7bbee0c17be26de048aee3e5adbc6c80/src/Compiler/Microsoft.CodeAnalysis.Razor.Compiler/src/Language/Legacy/CSharpCodeParser.cs#L743>

As part of the transition to use only runtime code generation, we had to make some subtle changes to the parsing of whitespace to ensure that the existing behavior in the editor continues to function as before.

Specifically we changed the parsing of trailing whitespace of razor code block directives (i.e. `@code`, `@function` and `@section`). Previously the whitespace was attached to a meta node that included the closing `}`

Using `^` to indicate whitespace:

```csharp
@code {
// code
}^^^

```

This would be previously be conceptually parsed as something like:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This would be previously be conceptually parsed as something like:
This would previously be conceptually parsed as something like:


```text
CSharpCode
RazorDirective
CSharpTransition
RazorDirectiveBody
RazorMetaCode
Identifier
CSharpCode
...
RazorMetaCode
Literal: }
Literal: ^^^\r\n
```

Thus when looking at the length of the RazorDirective, it includes the `^^^\r\n`. This causes issues with editor features like code folding. The user only want to fold the directive, not the directive and the following new line. (see <https://github.com/dotnet/razor/issues/10358>)

Instead, we now break the trailing whitespace into its own RazorMetaCode node, which is not a part of the directive itself. Conceptually something like:

```text
CSharpCode
RazorDirective
CSharpTransition
RazorDirectiveBody
RazorMetaCode
Identifier
CSharpCode
...
RazorMetaCode
Literal: }
RazorMetaCode
Literal: ^^^\r\n
```

In this way we keep the whitespace as belonging to the overall CSharpCode node, but don't make it part of the directive itself, ensuring the editor sees the correct length for the directive.
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
#nullable disable

using System;
using Microsoft.AspNetCore.Razor.Language.Components;
using Xunit;

namespace Microsoft.AspNetCore.Razor.Language.Legacy;
Expand Down Expand Up @@ -244,4 +245,104 @@ public void SupportsAllKindsOfImplicitMarkupInCodeBlock()
}
""");
}

[Fact]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider adding WorkItemAttributes linking to the issue

public void CodeBlocksTrailingWhitespace_01()
{
ParseDocumentTest("""
@code {
}

""", [ComponentCodeDirective.Directive]);
}

[Fact]
public void CodeBlocksTrailingWhitespace_02()
{
ParseDocumentTest("""
@code{
}

""", [ComponentCodeDirective.Directive]);
}

[Fact]
public void CodeBlocksTrailingWhitespace_03()
{
ParseDocumentTest("""
@code{
} @* comment *@

""", [ComponentCodeDirective.Directive]);
}

[Fact]
public void CodeBlocksTrailingWhitespace_04()
{
ParseDocumentTest("""
@code{
}
@* comment *@

""", [ComponentCodeDirective.Directive]);
}

[Fact]
public void CodeBlocksTrailingWhitespace_05()
{
ParseDocumentTest("""
@code {
}

@code {
}

""", [ComponentCodeDirective.Directive]);
}

[Fact]
public void CodeBlocksTrailingWhitespace_06()
{
ParseDocumentTest("""
@code {
}

<div></div>

""", [ComponentCodeDirective.Directive]);
}

[Fact]
public void CodeBlocksTrailingWhitespace_07()
{
ParseDocumentTest("""
@code {

}
<div></div>

""", [ComponentCodeDirective.Directive]);
}

[Fact]
public void CodeBlocksTrailingWhitespace_08()
{
ParseDocumentTest("""
@code {

} <div></div>

""", [ComponentCodeDirective.Directive]);
}

[Fact]
public void CodeBlocksTrailingWhitespace_09()
{
ParseDocumentTest("""
@code {

}<div></div>

""", [ComponentCodeDirective.Directive]);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
Markup span at (0:0,0 [0] ) (Accepts:Any) - Parent: Markup block at (0:0,0 [12] )
Transition span at (0:0,0 [1] ) (Accepts:None) - Parent: Directive block at (0:0,0 [10] )
MetaCode span at (1:0,1 [4] ) (Accepts:None) - Parent: Directive block at (0:0,0 [10] )
None span at (5:0,5 [1] ) (Accepts:AllWhitespace) - Parent: Directive block at (0:0,0 [10] )
MetaCode span at (6:0,6 [1] ) (Accepts:None) - Parent: Directive block at (0:0,0 [10] )
Code span at (7:0,7 [2] ) (Accepts:Any) - Parent: Directive block at (0:0,0 [10] )
MetaCode span at (9:1,0 [1] ) (Accepts:None) - Parent: Directive block at (0:0,0 [10] )
MetaCode span at (10:1,1 [2] ) (Accepts:Any) - Parent: Statement block at (0:0,0 [12] )
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
RazorDocument - [0..12)::12 - [@code {LF}LF]
MarkupBlock - [0..12)::12
MarkupTextLiteral - [0..0)::0 - [] - Gen<Markup> - SpanEditHandler;Accepts:Any
Marker;[];
CSharpCodeBlock - [0..12)::12
RazorDirective - [0..10)::10 - Directive:{code;CodeBlock;Unrestricted}
CSharpTransition - [0..1)::1 - Gen<None> - SpanEditHandler;Accepts:None
Transition;[@];
RazorDirectiveBody - [1..10)::9
RazorMetaCode - [1..5)::4 - Gen<None> - SpanEditHandler;Accepts:None
Identifier;[code];
CSharpCodeBlock - [5..10)::5
UnclassifiedTextLiteral - [5..6)::1 - [ ] - Gen<None> - SpanEditHandler;Accepts:AllWhitespace
Whitespace;[ ];
RazorMetaCode - [6..7)::1 - Gen<None> - AutoCompleteEditHandler;Accepts:None,AutoComplete:[<null>];AtEnd
LeftBrace;[{];
CSharpCodeBlock - [7..9)::2
CSharpStatementLiteral - [7..9)::2 - [LF] - Gen<Stmt> - CodeBlockEditHandler;Accepts:Any;CodeBlock
NewLine;[LF];
RazorMetaCode - [9..10)::1 - Gen<None> - SpanEditHandler;Accepts:None
RightBrace;[}];
RazorMetaCode - [10..12)::2 - Gen<None> - SpanEditHandler;Accepts:Any
NewLine;[LF];
EndOfFile;[];
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
Markup span at (0:0,0 [0] ) (Accepts:Any) - Parent: Markup block at (0:0,0 [31] )
Transition span at (0:0,0 [1] ) (Accepts:None) - Parent: Directive block at (0:0,0 [9] )
MetaCode span at (1:0,1 [4] ) (Accepts:None) - Parent: Directive block at (0:0,0 [9] )
MetaCode span at (5:0,5 [1] ) (Accepts:None) - Parent: Directive block at (0:0,0 [9] )
Code span at (6:0,6 [2] ) (Accepts:Any) - Parent: Directive block at (0:0,0 [9] )
MetaCode span at (8:1,0 [1] ) (Accepts:None) - Parent: Directive block at (0:0,0 [9] )
MetaCode span at (9:1,1 [22] ) (Accepts:Any) - Parent: Statement block at (0:0,0 [31] )
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
RazorDocument - [0..31)::31 - [@code{LF} LF]
MarkupBlock - [0..31)::31
MarkupTextLiteral - [0..0)::0 - [] - Gen<Markup> - SpanEditHandler;Accepts:Any
Marker;[];
CSharpCodeBlock - [0..31)::31
RazorDirective - [0..9)::9 - Directive:{code;CodeBlock;Unrestricted}
CSharpTransition - [0..1)::1 - Gen<None> - SpanEditHandler;Accepts:None
Transition;[@];
RazorDirectiveBody - [1..9)::8
RazorMetaCode - [1..5)::4 - Gen<None> - SpanEditHandler;Accepts:None
Identifier;[code];
CSharpCodeBlock - [5..9)::4
RazorMetaCode - [5..6)::1 - Gen<None> - AutoCompleteEditHandler;Accepts:None,AutoComplete:[<null>];AtEnd
LeftBrace;[{];
CSharpCodeBlock - [6..8)::2
CSharpStatementLiteral - [6..8)::2 - [LF] - Gen<Stmt> - CodeBlockEditHandler;Accepts:Any;CodeBlock
NewLine;[LF];
RazorMetaCode - [8..9)::1 - Gen<None> - SpanEditHandler;Accepts:None
RightBrace;[}];
RazorMetaCode - [9..31)::22 - Gen<None> - SpanEditHandler;Accepts:Any
Whitespace;[ ];
NewLine;[LF];
EndOfFile;[];
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
Markup span at (0:0,0 [0] ) (Accepts:Any) - Parent: Markup block at (0:0,0 [44] )
Transition span at (0:0,0 [1] ) (Accepts:None) - Parent: Directive block at (0:0,0 [9] )
MetaCode span at (1:0,1 [4] ) (Accepts:None) - Parent: Directive block at (0:0,0 [9] )
MetaCode span at (5:0,5 [1] ) (Accepts:None) - Parent: Directive block at (0:0,0 [9] )
Code span at (6:0,6 [2] ) (Accepts:Any) - Parent: Directive block at (0:0,0 [9] )
MetaCode span at (8:1,0 [1] ) (Accepts:None) - Parent: Directive block at (0:0,0 [9] )
Markup span at (9:1,1 [20] ) (Accepts:Any) - Parent: Markup block at (0:0,0 [44] )
Transition span at (29:1,21 [1] ) (Accepts:None) - Parent: Comment block at (29:1,21 [13] )
MetaCode span at (30:1,22 [1] ) (Accepts:None) - Parent: Comment block at (29:1,21 [13] )
Comment span at (31:1,23 [9] ) (Accepts:Any) - Parent: Comment block at (29:1,21 [13] )
MetaCode span at (40:1,32 [1] ) (Accepts:None) - Parent: Comment block at (29:1,21 [13] )
Transition span at (41:1,33 [1] ) (Accepts:None) - Parent: Comment block at (29:1,21 [13] )
Markup span at (42:1,34 [2] ) (Accepts:Any) - Parent: Markup block at (0:0,0 [44] )
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
RazorDocument - [0..44)::44 - [@code{LF} @* comment *@LF]
MarkupBlock - [0..44)::44
MarkupTextLiteral - [0..0)::0 - [] - Gen<Markup> - SpanEditHandler;Accepts:Any
Marker;[];
CSharpCodeBlock - [0..9)::9
RazorDirective - [0..9)::9 - Directive:{code;CodeBlock;Unrestricted}
CSharpTransition - [0..1)::1 - Gen<None> - SpanEditHandler;Accepts:None
Transition;[@];
RazorDirectiveBody - [1..9)::8
RazorMetaCode - [1..5)::4 - Gen<None> - SpanEditHandler;Accepts:None
Identifier;[code];
CSharpCodeBlock - [5..9)::4
RazorMetaCode - [5..6)::1 - Gen<None> - AutoCompleteEditHandler;Accepts:None,AutoComplete:[<null>];AtEnd
LeftBrace;[{];
CSharpCodeBlock - [6..8)::2
CSharpStatementLiteral - [6..8)::2 - [LF] - Gen<Stmt> - CodeBlockEditHandler;Accepts:Any;CodeBlock
NewLine;[LF];
RazorMetaCode - [8..9)::1 - Gen<None> - SpanEditHandler;Accepts:None
RightBrace;[}];
MarkupTextLiteral - [9..29)::20 - [ ] - Gen<Markup> - SpanEditHandler;Accepts:Any
Whitespace;[ ];
RazorComment - [29..42)::13
RazorCommentTransition;[@];
RazorCommentStar;[*];
RazorCommentLiteral;[ comment ];
RazorCommentStar;[*];
RazorCommentTransition;[@];
MarkupTextLiteral - [42..44)::2 - [LF] - Gen<Markup> - SpanEditHandler;Accepts:Any
NewLine;[LF];
EndOfFile;[];
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
Markup span at (0:0,0 [0] ) (Accepts:Any) - Parent: Markup block at (0:0,0 [26] )
Transition span at (0:0,0 [1] ) (Accepts:None) - Parent: Directive block at (0:0,0 [9] )
MetaCode span at (1:0,1 [4] ) (Accepts:None) - Parent: Directive block at (0:0,0 [9] )
MetaCode span at (5:0,5 [1] ) (Accepts:None) - Parent: Directive block at (0:0,0 [9] )
Code span at (6:0,6 [2] ) (Accepts:Any) - Parent: Directive block at (0:0,0 [9] )
MetaCode span at (8:1,0 [1] ) (Accepts:None) - Parent: Directive block at (0:0,0 [9] )
MetaCode span at (9:1,1 [2] ) (Accepts:Any) - Parent: Statement block at (0:0,0 [11] )
Transition span at (11:2,0 [1] ) (Accepts:None) - Parent: Comment block at (11:2,0 [13] )
MetaCode span at (12:2,1 [1] ) (Accepts:None) - Parent: Comment block at (11:2,0 [13] )
Comment span at (13:2,2 [9] ) (Accepts:Any) - Parent: Comment block at (11:2,0 [13] )
MetaCode span at (22:2,11 [1] ) (Accepts:None) - Parent: Comment block at (11:2,0 [13] )
Transition span at (23:2,12 [1] ) (Accepts:None) - Parent: Comment block at (11:2,0 [13] )
Markup span at (24:2,13 [2] ) (Accepts:Any) - Parent: Markup block at (0:0,0 [26] )
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
RazorDocument - [0..26)::26 - [@code{LF}LF@* comment *@LF]
MarkupBlock - [0..26)::26
MarkupTextLiteral - [0..0)::0 - [] - Gen<Markup> - SpanEditHandler;Accepts:Any
Marker;[];
CSharpCodeBlock - [0..11)::11
RazorDirective - [0..9)::9 - Directive:{code;CodeBlock;Unrestricted}
CSharpTransition - [0..1)::1 - Gen<None> - SpanEditHandler;Accepts:None
Transition;[@];
RazorDirectiveBody - [1..9)::8
RazorMetaCode - [1..5)::4 - Gen<None> - SpanEditHandler;Accepts:None
Identifier;[code];
CSharpCodeBlock - [5..9)::4
RazorMetaCode - [5..6)::1 - Gen<None> - AutoCompleteEditHandler;Accepts:None,AutoComplete:[<null>];AtEnd
LeftBrace;[{];
CSharpCodeBlock - [6..8)::2
CSharpStatementLiteral - [6..8)::2 - [LF] - Gen<Stmt> - CodeBlockEditHandler;Accepts:Any;CodeBlock
NewLine;[LF];
RazorMetaCode - [8..9)::1 - Gen<None> - SpanEditHandler;Accepts:None
RightBrace;[}];
RazorMetaCode - [9..11)::2 - Gen<None> - SpanEditHandler;Accepts:Any
NewLine;[LF];
RazorComment - [11..24)::13
RazorCommentTransition;[@];
RazorCommentStar;[*];
RazorCommentLiteral;[ comment ];
RazorCommentStar;[*];
RazorCommentTransition;[@];
MarkupEphemeralTextLiteral - [24..26)::2 - [LF] - Gen<None> - SpanEditHandler;Accepts:Any
NewLine;[LF];
EndOfFile;[];
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
Markup span at (0:0,0 [0] ) (Accepts:Any) - Parent: Markup block at (0:0,0 [26] )
Transition span at (0:0,0 [1] ) (Accepts:None) - Parent: Directive block at (0:0,0 [10] )
MetaCode span at (1:0,1 [4] ) (Accepts:None) - Parent: Directive block at (0:0,0 [10] )
None span at (5:0,5 [1] ) (Accepts:AllWhitespace) - Parent: Directive block at (0:0,0 [10] )
MetaCode span at (6:0,6 [1] ) (Accepts:None) - Parent: Directive block at (0:0,0 [10] )
Code span at (7:0,7 [2] ) (Accepts:Any) - Parent: Directive block at (0:0,0 [10] )
MetaCode span at (9:1,0 [1] ) (Accepts:None) - Parent: Directive block at (0:0,0 [10] )
MetaCode span at (10:1,1 [2] ) (Accepts:Any) - Parent: Statement block at (0:0,0 [12] )
Markup span at (12:2,0 [2] ) (Accepts:Any) - Parent: Markup block at (0:0,0 [26] )
Transition span at (14:3,0 [1] ) (Accepts:None) - Parent: Directive block at (14:3,0 [10] )
MetaCode span at (15:3,1 [4] ) (Accepts:None) - Parent: Directive block at (14:3,0 [10] )
None span at (19:3,5 [1] ) (Accepts:AllWhitespace) - Parent: Directive block at (14:3,0 [10] )
MetaCode span at (20:3,6 [1] ) (Accepts:None) - Parent: Directive block at (14:3,0 [10] )
Code span at (21:3,7 [2] ) (Accepts:Any) - Parent: Directive block at (14:3,0 [10] )
MetaCode span at (23:4,0 [1] ) (Accepts:None) - Parent: Directive block at (14:3,0 [10] )
MetaCode span at (24:4,1 [2] ) (Accepts:Any) - Parent: Statement block at (14:3,0 [12] )
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
RazorDocument - [0..26)::26 - [@code {LF}LFLF@code {LF}LF]
MarkupBlock - [0..26)::26
MarkupTextLiteral - [0..0)::0 - [] - Gen<Markup> - SpanEditHandler;Accepts:Any
Marker;[];
CSharpCodeBlock - [0..12)::12
RazorDirective - [0..10)::10 - Directive:{code;CodeBlock;Unrestricted}
CSharpTransition - [0..1)::1 - Gen<None> - SpanEditHandler;Accepts:None
Transition;[@];
RazorDirectiveBody - [1..10)::9
RazorMetaCode - [1..5)::4 - Gen<None> - SpanEditHandler;Accepts:None
Identifier;[code];
CSharpCodeBlock - [5..10)::5
UnclassifiedTextLiteral - [5..6)::1 - [ ] - Gen<None> - SpanEditHandler;Accepts:AllWhitespace
Whitespace;[ ];
RazorMetaCode - [6..7)::1 - Gen<None> - AutoCompleteEditHandler;Accepts:None,AutoComplete:[<null>];AtEnd
LeftBrace;[{];
CSharpCodeBlock - [7..9)::2
CSharpStatementLiteral - [7..9)::2 - [LF] - Gen<Stmt> - CodeBlockEditHandler;Accepts:Any;CodeBlock
NewLine;[LF];
RazorMetaCode - [9..10)::1 - Gen<None> - SpanEditHandler;Accepts:None
RightBrace;[}];
RazorMetaCode - [10..12)::2 - Gen<None> - SpanEditHandler;Accepts:Any
NewLine;[LF];
MarkupTextLiteral - [12..14)::2 - [LF] - Gen<Markup> - SpanEditHandler;Accepts:Any
NewLine;[LF];
CSharpCodeBlock - [14..26)::12
RazorDirective - [14..24)::10 - Directive:{code;CodeBlock;Unrestricted}
CSharpTransition - [14..15)::1 - Gen<None> - SpanEditHandler;Accepts:None
Transition;[@];
RazorDirectiveBody - [15..24)::9
RazorMetaCode - [15..19)::4 - Gen<None> - SpanEditHandler;Accepts:None
Identifier;[code];
CSharpCodeBlock - [19..24)::5
UnclassifiedTextLiteral - [19..20)::1 - [ ] - Gen<None> - SpanEditHandler;Accepts:AllWhitespace
Whitespace;[ ];
RazorMetaCode - [20..21)::1 - Gen<None> - AutoCompleteEditHandler;Accepts:None,AutoComplete:[<null>];AtEnd
LeftBrace;[{];
CSharpCodeBlock - [21..23)::2
CSharpStatementLiteral - [21..23)::2 - [LF] - Gen<Stmt> - CodeBlockEditHandler;Accepts:Any;CodeBlock
NewLine;[LF];
RazorMetaCode - [23..24)::1 - Gen<None> - SpanEditHandler;Accepts:None
RightBrace;[}];
RazorMetaCode - [24..26)::2 - Gen<None> - SpanEditHandler;Accepts:Any
NewLine;[LF];
EndOfFile;[];
Loading