Explore moving source parsing to async open file operation

[tmat]

The idea is that when a file is open using new asynchronous open file API that the VS platform is working on we hook up syntax parser into the open file handler so that when the file is opened and user can start typing we already have the syntax tree available.

This would allow us to implement complex formatting command handlers entirely synchronously since they would operate on already parsed tree.

If our parser is not fast enough (TODO: measure) we could consider starting the parsing asynchronously as soon as the file opens but only enable the commands that require syntax trees when the tree becomes available. Until then the UI would indicate that the document is not ready yet and only basic text editing operations will be available (e.g. basic indentation but not smart indentation).

Since the formatting operations also require access to editorconfig options we would also need them to be available from the text snapshot. The editor already has them, so it just needs to expose them via an API:

• Make IVsTextView.IndentStyle available through ITextView.Options: VS #1412138
• Custom editorconfig properties are not exposed through editor options: VS #1429383
• Add free-threaded API that replaces IVsTextManager4.GetUserPreferences4: VS #1307188

Related: #57554

[tmat]

@CyrusNajmabadi @jasonmalinowski @sharwell RFC

[jasonmalinowski]

This would allow us to implement complex formatting command handlers entirely synchronously since they would operate on already parsed tree.

What happens if an edit requires us to reparse the file from scratch, say the classic add/remove a block comment case? The incremental parse will be just as expensive as a full parse there.

[tmat]

What do we do today? I would expect us to keep using incremental parsing while the file is open.

[jasonmalinowski]

@tmat, we do, but there's plenty of cases where an incremental parse is going to be as expensive as a full parse.

[jasonmalinowski]

That all said, doing some better async hookup here is absolutely something to look at. One thing we've had issues in the past with is file open also may have to collapse/expand previously collapsed regions; this has been a huge pain since we've sometimes needed a parse tree synchronously on file open and making that path more async would be a great help.

[CyrusNajmabadi]

@tmat, we do, but there's plenty of cases where an incremental parse is going to be as expensive as a full parse.

note: that's true, but we can think of things in an amortized fashion. overall incremental means the 99.9% case (making numbers up is fine) and we occasionally pay a slightly higher price. But the saving we get in that 99.9% means we have that extra time (and extra CPU available) to pay for the occasional more expensive work.

[CyrusNajmabadi]

Closing out as speculative. If this needs to happen, it needs to be assigned to a dev/milestone for that to happen in.