Skip to content

Commit

Permalink
Introduce cached document analysis (#457)
Browse files Browse the repository at this point in the history
This commit introduced analysis, a means to parse, analyze, and cache a document. Currently, analysis only tracks aliases, but it will be extended in the future to track additional scope information, such as imports, bindings, etc.

The idea for an analysis struct was borne from performance issues with the existing alias expansion, which would re-walk the AST each time an alias needed to be expanded. This was particularly egregious during indexing -- this new approach indexes the Lexical codebase about an order of magnitude faster.

Additionally, the document store (`Lexical.Document.Store`) now has a new feature: derived data. This allows us to lazily analyze documents as needed and cache the analysis result alongside the document in the store, invalidating the analysis whenever the document changes.

Selected summary of new APIs/changes:

* `Lexical.Ast.analyze(document)` - returns `%Analysis{}` which may or may not be valid (`analysis.valid?`). Used for whole-document analysis, e.g. indexing.
* `Lexical.Ast.reanalyze_to(analysis, position)` - returns `%Analysis{}` using the fragment of the original document up to the given position. Useful for things that may not require the whole document, e.g. resolving aliases. If the given analysis is already valid, this is a no-op.
* `Lexical.Ast.Aliases` has been removed in favor of analysis.
* Many `Lexical.Ast` functions have been refactored to also (or only) accept an analysis in place of a document.
* Added concept of derived data to `Lexical.Document.Store`. This allows for a derived analysis to be saved alongside the document and fetched using `{:ok, %Document{}, %Analysis{}} = Document.Store.fetch(uri, :analysis)`.
* Refactored a number of existing LS features to use analysis, including entity resolution, completions, references.
  • Loading branch information
zachallaun authored Nov 17, 2023
1 parent cf2a9fb commit face3c6
Show file tree
Hide file tree
Showing 42 changed files with 1,444 additions and 865 deletions.
314 changes: 147 additions & 167 deletions apps/common/lib/lexical/ast.ex

Large diffs are not rendered by default.

318 changes: 0 additions & 318 deletions apps/common/lib/lexical/ast/aliases.ex

This file was deleted.

66 changes: 66 additions & 0 deletions apps/common/lib/lexical/ast/analysis.ex
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
defmodule Lexical.Ast.Analysis do
@moduledoc """
A data structure representing an analyzed AST.
See `Lexical.Ast.analyze/1`.
"""

alias Lexical.Ast.Analysis.Analyzer
alias Lexical.Document
alias Lexical.Document.Position
alias Lexical.Document.Range

defstruct [:ast, :document, :parse_error, scopes: [], valid?: true]

@type t :: %__MODULE__{}

@doc false
def new(parse_result, document)

def new({:ok, ast}, %Document{} = document) do
scopes = Analyzer.traverse(ast, document)

%__MODULE__{
ast: ast,
document: document,
scopes: scopes
}
end

def new(error, document) do
%__MODULE__{
document: document,
parse_error: error,
valid?: false
}
end

@doc false
def aliases_at(%__MODULE__{} = analysis, %Position{} = position) do
case scopes_at(analysis, position) do
[%Analyzer.Scope{} = scope | _] ->
scope
|> Analyzer.Scope.alias_map(position)
|> Map.new(fn {as, %Analyzer.Alias{} = alias} ->
{as, Analyzer.Alias.to_module(alias)}
end)

[] ->
%{}
end
end

defp scopes_at(%__MODULE__{scopes: scopes}, %Position{} = position) do
scopes
|> Enum.filter(fn %Analyzer.Scope{range: range} = scope ->
scope.id == :global or Range.contains?(range, position)
end)
|> Enum.sort_by(
fn
%Analyzer.Scope{id: :global} -> 0
%Analyzer.Scope{range: range} -> {range.start.line, range.start.character}
end,
:desc
)
end
end
Loading

0 comments on commit face3c6

Please sign in to comment.