Graph.add: type already defined

[mikeshulman]

Using the version of merlin that's installed with OCaml 5.3.0 (I guess 5.4.1-503), I sometimes get an error message in Emacs that says

merlin-mode error: Graph.add: type already defined

and merlin stops working entirely for that buffer. I would love to post a MWE but I have no idea what might be causing this; it generally only happens with fairly large files, and removing small pieces of them at random tends to make the error go away. For example, I get the error in the file bin/olorin.ml in this branch. If you can tell me how to give more useful information to help debug the issue, I would be happy to try it.

[mikeshulman]

Can anyone help me try to debug this? It's starting to get very annoying. I suspect that it has something to do with GADTs, existential type variables, or polymorphically recursive functions.

[voodoos]

Hi @mikeshulman. Issues with the short paths graph are often quite tricky to track. It would be very useful if you could try to produce a smaller reproduction based on the one you already have. Looking at the linked PR, I understand this was a regression with the move to merlin for OCaml 5.3, is that right ? That is very useful information.

I might not be able to take a deep dive soon, but until then you can try disabling merlin's path shortening feature as a workaround. In a dune based project you can do that by subtracting the -short-paths flag from the standard set of flag in the concerned library / executable stanzas:

(library
  ...
  (flags :standard \ -short-paths))

[mikeshulman]

Thanks! Yes, this is a regression from OCaml 5.2.1 to 5.3.

I tried to make a smaller MWE, but I wasn't able to make it noticeably smaller and still reproduce the bug: removing small bits of the long file at random generally makes the error go away. I was hoping that someone with knowledge of merlin's internals could give me a pointer to some likely causes to help guide my minimization, or perhaps tell me how to enable some debug mode in merlin that would give more information about the source of the error.

[mikeshulman]

I tried your suggestion about subtracting -short-paths. Now I got the following error in my Emacs messages:

let: unknown answer: "exception":"File \"src/ocaml/typing/types.ml\", line 926, characters 15-21: Assertion failed
Raised at Ocaml_typing__Types.rev_log in file \"src/ocaml/typing/types.ml\", line 926, characters 15-27
Called from Ocaml_typing__Types.backtrack in file \"src/ocaml/typing/types.ml\", line 938, characters 20-37
Called from Ocaml_typing__Btype.backtrack in file \"src/ocaml/typing/btype.ml\" (inlined), line 640, characters 16-41
Called from Merlin_kernel__Mtyper.type_implementation in file \"src/kernel/mtyper.ml\", line 168, characters 2-23
Called from Merlin_kernel__Mtyper.run.(fun) in file \"src/kernel/mtyper.ml\", line 237, characters 35-78
Called from Merlin_utils__Misc.try_finally in file \"src/utils/misc.ml\", line 45, characters 8-15
Re-raised at Merlin_utils__Misc.try_finally in file \"src/utils/misc.ml\", line 62, characters 10-24
Called from Merlin_kernel__Mpipeline.process in file \"src/kernel/mpipeline.ml\", line 308, characters 22-49
Called from CamlinternalLazy.do_force_block in file \"camlinternalLazy.ml\", line 49, characters 17-27
Re-raised at CamlinternalLazy.do_force_block in file \"camlinternalLazy.ml\", line 56, characters 4-11
Called from CamlinternalLazy.force_lazy_block in file \"camlinternalLazy.ml\" (inlined), line 78, characters 27-67
Called from Merlin_kernel__Mpipeline.timed_lazy in file \"src/kernel/mpipeline.ml\", line 17, characters 11-23
Re-raised at Merlin_kernel__Mpipeline.timed_lazy in file \"src/kernel/mpipeline.ml\", line 23, characters 7-22
Called from CamlinternalLazy.do_force_block in file \"camlinternalLazy.ml\", line 49, characters 17-27
Re-raised at CamlinternalLazy.do_force_block in file \"camlinternalLazy.ml\", line 56, characters 4-11
Called from Merlin_kernel__Mpipeline.typer_result in file \"src/kernel/mpipeline.ml\" (inlined), line 133, characters 21-30
Called from Query_commands.dispatch in file \"src/frontend/query_commands.ml\", line 651, characters 16-47
Called from Merlin_commands__New_commands.run in file \"src/commands/new_commands.ml\", line 98, characters 15-53
Called from Merlin_utils__Std.let_ref in file \"src/utils/std.ml\", line 727, characters 8-12
Re-raised at Merlin_utils__Std.let_ref in file \"src/utils/std.ml\", line 733, characters 4-13
Called from Merlin_utils__Misc.try_finally in file \"src/utils/misc.ml\", line 45, characters 8-15
Re-raised at Merlin_utils__Misc.try_finally in file \"src/utils/misc.ml\", line 62, characters 10-24
Called from Stdlib__Fun.protect in file \"fun.ml\", line 34, characters 8-15
Re-raised at Stdlib__Fun.protect in file \"fun.ml\", line 39, characters 6-52
Called from Merlin_kernel__Mocaml.with_state in file \"src/kernel/mocaml.ml\", line 18, characters 8-38
Re-raised at Merlin_kernel__Mocaml.with_state in file \"src/kernel/mocaml.ml\", line 24, characters 4-15
Called from Dune__exe__New_merlin.run.(fun) in file \"src/frontend/ocamlmerlin/new/new_merlin.ml\", lines 118-119, characters 16-52

Running dune clean, rebuilding, and restarting Emacs makes this go away, but it comes back. Possibly the trigger has something to do with matching against GADTs, perhaps with existential types, maybe in the presence of holes (_) -- but that isn't sufficient, I've tried simple files that have all of that and the error doesn't appear.

[voodoos]

That's very surprising, I was not expecting the "workaround" to show a breakage at another place. That's probably good news however since it might be a clear lead on the root cause for the short-paths issue.

The failure happens when Merlin try to restore a previous version of the typer cache, so I expect you need at least these steps to reproduce from a restart:

• Run a command that types the buffer (like type-enclosing)
• Edit the buffer after the (unknown) location that triggers the issue
• Run the same first command, triggering the cache to restore the parts that are problematic

I think you can enable more debug output with the merlin-enable-debug. I need to know what is the command (or the chain of commands) that leads you to that failure, and with the debugging enabled you should be able to get that information.

(I probably won't have time to investigate more thoroughly in the coming 2 weeks)

[mikeshulman]

Here's the debug output. It appears that saving the buffer is sufficient as a command.

merlin-log.txt

[voodoos]

It appears that saving the buffer is sufficient as a command.

I built the oloring branch you provided and was able to reproduce, but not yet consistently with.
Do you get the bug simply by (starting from scratch) simply opening the norm.ml buffer, calling errors once ? or twice ?

[mikeshulman]

Try this branch, open the norm.ml buffer, run a command, modify the buffer and save it, maybe run another command. That does it very consistently for me.

[voodoos]

I am currently not able to reproduce the exception you get after deactivating -short-paths.
I tried running that script on the branch your suggested:

#!/bin/sh

FILE=lib/core/norm.ml

ocamlmerlin server stop-server
ocamlmerlin server errors -filename $FILE <$FILE
echo ";; let () = ()" >> $FILE
ocamlmerlin server errors -filename $FILE <$FILE
ocamlmerlin server stop-server

Does it trigger the exception for you ?

[mikeshulman]

No, that doesn't trigger it. Apparently not any old edit will work. Try editing the definition of fldmap on line 963, e.g. removing the unnecessary Term. qualifier on one of the constructors.