[FUSE] Fix code folding

[chsienki]

When parsing code blocks, don't associate the trailing whitespace and newline with the actual directive, but as a meta node that follows it. That ensures that the directive itself only covers the space up to the final }, but we still class the trailing whitespace as C# so don't emit it as HTML either.

Fixes #10358

[chsienki]

@dotnet/razor-compiler for reviews please :)

[333fred]

Not sure I'm a huge fan of this without some driving principle behind it. In Roslyn, you can be confident that, when you look at a piece of text, you know where the whitespace will end up; I'm not certain that I can state such invariants here (before or after the change). Can we make such a statement, and then document it and adjust the code to follow it?

[chsienki]

@333fred I'm not sure I understand the ask totally; where would you want to document it?

[333fred]

Probably a simple markdown file in the docs/ folder. We can add things there in a future PR as well, like how the FindToken API works.

[jjonescz]

Can we add some unit tests for the folding issue (some test that would fail without this PR)? If that would not be easy, perhaps at least some compiler unit test that verifies the whitespace is attached to the correct nodes. The baselines themselves are not enough I think - we won't notice if we regress this in the future.

[chsienki]

We do have the integration tests (which is how we found this) but I'll see if I can cook up some parsing tests.

[jjonescz]

I see you added some parsing tests which are nice because they are narrow and test only the actual issue. But it doesn't address my concern - we usually regenerate lots of baselines and it's difficult to review them all and check they look as expected in all aspects - so it would be really easy to regress this fix in the future. Can we add some asserts that check the whitespace or spans of the nodes, i.e., something that would fail regardless of baselines?

[jjonescz]

Suggested change

	This would be previously be conceptually parsed as something like:
	This would previously be conceptually parsed as something like:

[jjonescz]

I see you added some parsing tests which are nice because they are narrow and test only the actual issue. But it doesn't address my concern - we usually regenerate lots of baselines and it's difficult to review them all and check they look as expected in all aspects - so it would be really easy to regress this fix in the future. Can we add some asserts that check the whitespace or spans of the nodes, i.e., something that would fail regardless of baselines?

[jjonescz]

Consider adding WorkItemAttributes linking to the issue

[333fred]

Thanks. This was incredibly helpful for reviewing this PR.