Indexing getting stuck

[JonasGruenwald]

I quite frequently run into a situation where lexical gets stuck indexing indefinitely, and sometimes it appears to spin up several indexing tasks which all end up stuck / don't leave the vscode notification center, this is a bit distracting:

Unfortunately reloading the editor does not help at all, once this starts happening, lexical always shows an indexing notification that never goes away when it starts, and the only thing that helps is deleting the indexes directory under .lexical and then restarting the editor.

After that, the indexing then completes immediately in around a second and everything works normally again.

It just happened to me again and I noticed the last line in the .lexical/project.log is this:

2025-02-02T00:05:02.596800+01:00 error: Task #PID<0.475.0> started from LXical.RemoteControl.Search.Store terminating, ** (FunctionClauseError) no function clause matching in :elixir_aliases.do_concat/2, (elixir 1.17.3) src/elixir_aliases.erl:183: :elixir_aliases.do_concat([{:unquote, [_scope_id: 144201440653480029, closing: [line: 72, column: 40], line: 72, column: 15], [{{:., [_scope_id: 144201440653480031, line: 72, column: 33], [{:CALLER, [_scope_id: 144201440653480032, line: 72, column: 23], nil}, :module]}, [_scope_id: 144201440653480030, no_parens: true, line: 72, column: 34], []}]}, :Consumer], "Elixir"), (elixir 1.17.3) src/elixir_aliases.erl:171: :elixir_aliases.concat/1, (lx_remote_control 0.7.2) lib/lexical/remote_control/analyzer/aliases.ex:14: anonymous fn/1 in LXical.RemoteControl.Analyzer.Aliases.at/2, (elixir 1.17.3) lib/map.ex:257: Map.do_map/2, (elixir 1.17.3) lib/map.ex:257: Map.do_map/2, (elixir 1.17.3) lib/map.ex:251: Map.new_from_map/2, (lx_remote_control 0.7.2) lib/lexical/remote_control/analyzer.ex:118: LXical.RemoteControl.Analyzer.expand_alias/3, (lx_remote_control 0.7.2) lib/lexical/remote_control/search/indexer/extractors/module.ex:258: LXical.RemoteControl.Search.Indexer.Extractors.Module.module/2, Function: &:erlang.apply/2, Args: [#Function<3.97721833/1 in LXical.RemoteControl.Search.Indexer.async_chunks/3>, [["/Users/jonas/Projects/g2-mono/backend/deps/patch/lib/patch/mock/code.ex", "/Users/jonas/Projects/g2-mono/backend/deps/mneme/test_integration/receive_test.exs", "/Users/jonas/Projects/g2-mono/backend/deps/phoenix/priv/templates/phx.gen.channel/channel_case.ex", "/Users/jonas/Projects/g2-mono/backend/deps/phoenix/priv/templates/phx.gen.auth/registration_controller_test.exs", "/Users/jonas/Projects/g2-mono/backend/_build/dev/lib/phoenix/priv/templates/phx.gen.context/access_no_schema.ex", "/Users/jonas/Projects/g2-mono/backend/_build/dev/lib/phoenix/priv/templates/phx.gen.json/changeset_json.ex", "/Users/jonas/Projects/g2-mono/backend/deps/telegex/lib/telegex/polling/gen_handler.ex", "/Users/jonas/Projects/g2-mono/backend/_build/dev/lib/phoenix/priv/templates/phx.gen.auth/routes.ex", "/Users/jonas/Projects/g2-mono/backend/deps/postgrex/lib/postgrex/default_types.ex", "/Users/jonas/Projects/g2-mono/backend/deps/mneme/benchmarks/diff.exs", "/Users/jonas/Projects/g2-mono/backend/_build/dev/lib/phoenix/priv/templates/phx.gen.html/controller_test.exs", "/Users/jonas/Projects/g2-mono/backend/_build/dev/lib/phoenix/priv/templates/phx.gen.auth/reset_password_live.ex", "/Users/jonas/Projects/g2-mono/backend/deps/phoenix/priv/templates/phx.gen.auth/settings_controller.ex", "/Users/jonas/Projects/g2-mono/backend/_build/dev/lib/phoenix/priv/templates/phx.gen.auth/registration_live.ex", "/Users/jonas/Projects/g2-mono/backend/_build/dev/lib/phoenix/priv/templates/phx.gen.auth/auth_test.exs", "/Users/jonas/Projects/g2-mono/backend/_build/dev/lib/phoenix/priv/templates/phx.gen.auth/test_cases.exs", "/Users/jonas/Projects/g2-mono/backend/deps/oban_web/lib/oban/web/resolver.ex", "/Users/jonas/Projects/g2-mono/backend/_build/dev/lib/phoenix/priv/templates/phx.gen.auth/session_controller.ex", "/Users/jonas/Projects/g2-mono/backend/deps/phoenix/priv/templates/phx.gen.json/json.ex", "/Users/jonas/Projects/g2-mono/backend/deps/phoenix/priv/templates/phx.gen.context/test_cases.exs", "/Users/jonas/Projects/g2-mono/backend/_build/dev/lib/phoenix/priv/templates/phx.gen.auth/conn_case.exs", "/Users/jonas/Projects/g2-mono/backend/deps/mneme/test/test_helper.exs", "/Users/jonas/Projects/g2-mono/backend/deps/phoenix/priv/templates/phx.gen.context/context.ex", "/Users/jonas/Projects/g2-mono/backend/deps/bandit/lib/bandit.ex", "/Users/jonas/Projects/g2-mono/backend/_build/dev/lib/phoenix/priv/templates/phx.gen.auth/confirmation_controller.ex"]]]

In the lexical.log there is this:

 [error] Child :undefined of Supervisor LXical.Server.TaskQueue.State.TaskSupervisor terminated
** (exit) :canceled
Pid: #PID<0.1553.0>
Start Call: Task.Supervised.start_link/?
Restart: :temporary
Shutdown: 5000
Type: :worker

Is there any way to avoid this?

[scohen]

This is interesting:

 started from LXical.RemoteControl.Search.Store terminating, ** (FunctionClauseError) no function clause matching in :elixir_aliases.do_concat/2, (elixir 1.17.3) src/elixir_aliases.erl:183: :elixir_aliases.do_concat([{:unquote, [_scope_id: 144201440653480029, closing: [line: 72, column: 40], line: 72, column: 15], [{{:., [_scope_id: 144201440653480031, line: 72, column: 33], [{:CALLER, [_scope_id: 144201440653480032, line: 72, column: 23], nil}, :module]}

This is likely what's killing indexing on your computer. Can we dig in there?

[JonasGruenwald]

I wouldn't know how tbh, the call to that is coming from lexical if I understand correctly?
Any steps I could take to debug this further?

[scohen]

This might be tedious, but on one of these files, Line 72 has something with :__CALLER__ on it. Seeing that file's source code would help.

["/Users/jonas/Projects/g2-mono/backend/deps/patch/lib/patch/mock/code.ex", 
"/Users/jonas/Projects/g2-mono/backend/deps/mneme/test_integration/receive_test.exs", 
"/Users/jonas/Projects/g2-mono/backend/deps/phoenix/priv/templates/phx.gen.channel/channel_case.ex", 
"/Users/jonas/Projects/g2-mono/backend/deps/phoenix/priv/templates/phx.gen.auth/registration_controller_test.exs", 
"/Users/jonas/Projects/g2-mono/backend/_build/dev/lib/phoenix/priv/templates/phx.gen.context/access_no_schema.ex", 
"/Users/jonas/Projects/g2-mono/backend/_build/dev/lib/phoenix/priv/templates/phx.gen.json/changeset_json.ex", 
"/Users/jonas/Projects/g2-mono/backend/deps/telegex/lib/telegex/polling/gen_handler.ex",
 "/Users/jonas/Projects/g2-mono/backend/_build/dev/lib/phoenix/priv/templates/phx.gen.auth/routes.ex", "/Users/jonas/Projects/g2-mono/backend/deps/postgrex/lib/postgrex/default_types.ex",
 "/Users/jonas/Projects/g2-mono/backend/deps/mneme/benchmarks/diff.exs", 
"/Users/jonas/Projects/g2-mono/backend/_build/dev/lib/phoenix/priv/templates/phx.gen.html/controller_test.exs", 
"/Users/jonas/Projects/g2-mono/backend/_build/dev/lib/phoenix/priv/templates/phx.gen.auth/reset_password_live.ex", 
"/Users/jonas/Projects/g2-mono/backend/deps/phoenix/priv/templates/phx.gen.auth/settings_controller.ex", 
"/Users/jonas/Projects/g2-mono/backend/_build/dev/lib/phoenix/priv/templates/phx.gen.auth/registration_live.ex", 
"/Users/jonas/Projects/g2-mono/backend/_build/dev/lib/phoenix/priv/templates/phx.gen.auth/auth_test.exs", 
"/Users/jonas/Projects/g2-mono/backend/_build/dev/lib/phoenix/priv/templates/phx.gen.auth/test_cases.exs", 
"/Users/jonas/Projects/g2-mono/backend/deps/oban_web/lib/oban/web/resolver.ex", 
"/Users/jonas/Projects/g2-mono/backend/_build/dev/lib/phoenix/priv/templates/phx.gen.auth/session_controller.ex", 
"/Users/jonas/Projects/g2-mono/backend/deps/phoenix/priv/templates/phx.gen.json/json.ex", 
"/Users/jonas/Projects/g2-mono/backend/deps/phoenix/priv/templates/phx.gen.context/test_cases.exs", 
"/Users/jonas/Projects/g2-mono/backend/_build/dev/lib/phoenix/priv/templates/phx.gen.auth/conn_case.exs", 
"/Users/jonas/Projects/g2-mono/backend/deps/mneme/test/test_helper.exs",
 "/Users/jonas/Projects/g2-mono/backend/deps/phoenix/priv/templates/phx.gen.context/context.ex", 
"/Users/jonas/Projects/g2-mono/backend/deps/bandit/lib/bandit.ex",
 "/Users/jonas/Projects/g2-mono/backend/_build/dev/lib/phoenix/priv/templates/phx.gen.auth/confirmation_controller.ex"]]

[scohen]

Though, now that I see this, is lexical being run in a monorepo? I'm not sure how well that'd work.

[JonasGruenwald]

It is, yeah – as in: the Elixir project is not at the root of my repo, but it is the only Elixir project in the repo.
I've set the project dir setting the vscode extension provides to the directory of the Elixir project, I thought that would suffice, is that not actually officially supported? Then that's probably the issue

[JonasGruenwald]

This might be tedious, but on one of these files, Line 72 has something with :__CALLER__ on it. Seeing that file's source code would help.

That would be this one:
https://github.com/telegex/telegex/blob/master/lib/telegex/polling/gen_handler.ex#L72
that library is pretty wild, very macro heavy.

[scohen]

Yeah, i see why that doesn't work, but it shouldn't crash things.
This is due to the fact that it's indexing your deps (though this is still a bug in lexical's AST code, which should be fixed), and that's due to being in a poncho project. One thing you can try is to open up each mix project separately, and things should work for now.

[JonasGruenwald]

Thanks for the hint, I've been opening the Elixir project separately while working on it and I have not encountered the issue anymore, so that does seem to be the cause, that's good enough for me.

I kind of expected the lexical.server.projectDir setting to basically be doing that, but I suppose that's not related to the language server but rather to the vscode extension – also just to reiterate for clarity: I do not have multiple Elixir projects (I think that's what poncho means though I'm not sure?), it's a single one, it's just that it's not at the root of the repository.