refactor: decouple diagnostics from handlers into a hook #714
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Background
Originally we would only publish diagnostics directly from the relevant (RPC request) handlers, such as from
textDocument/didOpen
,textDocument/didChange
ortextDocument/didClose
.This in principle introduced the requirement on the task scheduler to support blocking/synchronous execution as we didn't want to be publishing stale diagnostics before the parsing and decoding is actually finished. However, this execution combined with the amount of operations we need to run in different contexts makes this a complex implementation hard to reason about and harder to maintain. Consequently there is at least one bug 🐛 where duplicate operations are attempted to be scheduled and one of them will remain blocking forever, effectively causing a deadlock 🙀 . This bug was first discovered in #700 which adds more operations to ensure cross-module references are discovered and therefore makes the deadlock situation more likely to occur.
This PR therefore aims to remove the whole concept of blocking/sync execution and assume that all operations will be executed asynchronously.
UX Implications
This comes with a (subtle?) UX difference where we would publish diagnostics for all files as they are indexed (and changed), not just the ones which are open on the client. We could still provide the existing UX if we wanted to, but there's no clear benefit we'd gain by the additional complexity.
Also we now avoid publishing any diagnostics for non-autoloaded tfvars files, which more or less aligns the behaviour with gopls ' treatment of Go build tags. I will create a separate issue to track this problem.
See #715
LSP
There is currently no clear guidance in LSP spec around whether servers should publish diagnostics for unopened files. The only related part of the spec talks about clearing of diagnostics on close:
Language Clients & gopls
Although LSP doesn't provide clear guidance, it is clear that sending diagnostics for unopened files is a common practice in gopls. Clients account for this possibility differently, probably catering different user bases with different expectations:
Future TODOs
Versioned diagnostics
LSP allows us to send diagnostics with document versions, which presumably aids the client as it can just drop the ones which are associated with an old version of the document and effectively prevent ever publishing a stale diagnostic.
Attaching the document version at this point would not be trivial as we'd probably need to start tracking document versions in memdb alongside diagnostics. This PR therefore does increase the likelihood of us publishing outdated diagnostics, but they would always get updated/cleared as part of another module change hook execution anyway, so it's rather a "temporary blip".
See #716
Only publishing changed diagnostics
In an ideal scenario we would only publish diagnostics when there are different ones to publish and avoid publishing the same ones multiple times. In order to achieve this however we would need to check whether all diagnostics are equal. Currently we use the upstream
hcl.Diagnostic
to represent a diagnostic and the upstream implementation does not have an equality function itself. We could implement all this relatively easily downstream, but we can't actually take advantage of it until we figure out a better way of publishing and clearingvalidate
diagnostics, which currently get just cleared whenever other diagnostics are published, to avoid stale diagnostics.See #717