[Shapes] OCaml 4.14 + Shapes #1431

voodoos · 2022-01-14T14:59:08Z

Another Pr for early testing.

Based on Upgrade Merlin to OCaml 4.14 #1415, an up-to-date upgrade to OCaml 4.14
With the latest version of @gasche's strong call-by-need PR
With the locate code that uses shapes similar to others recent testing PRs
Most tests are improved, one is not (jumping to declaration instead of a functor argument).
Documentation tests are not passing at all yet.
Add tests illustrating module aliases issue
~~[ ] fix by patching the compiler~~
fix documentation and dot-pp-dot-ml-dune tests

The corresponding compiler (4.14 + @gasche's cbn) to be used with this merlin can be found here:
https://github.com/voodoos/ocaml/tree/shape-strong-call-by-need

voodoos · 2022-01-27T15:54:16Z

@trefis all tests are now passing, and jumping to the argument of a functor works again. This PR is ready for review !

To restore that behaviour I had to store the functor arguments uid in the uid_to_loc table of the local environment. See c5e6d07.
I suppose it is not necessary to upstream this as it only occurs in local buffers. But we might want to to reduce the divergence between Merlin and the compiler... What do you think ?

voodoos · 2022-02-03T14:44:20Z

@trefis : this now use uids to answer document queries when the target is in another comp_unit. It fallsback on the old heuristic when the target is in the current buffer or the previous algorithm found nothing (necessary for test unattached-comment).

I also reverted the changes for functor arguments. We now jump to the correct place in the argument's signature.

voodoos · 2022-02-04T11:15:01Z

I rebased both the 414 branch and this PR.
This led to the discovery of an issue with module aliases.
I will add more tests to illustrate the issue.

voodoos · 2022-02-10T11:56:00Z

@trefis all test fixed. This is ready for review. My original intent was to merge this into the 4.14 update PR but maybe we can first merge the update PR onto master and this one second ?

get_doc now use uids to get docstrings from external compilation units. I took that opportunity to fix Associate entry with toplevel comment and module comment #1421.

tests/test-dirs/locate/locate-stdlib.t

tests/test-dirs/locate/partial-cmt.t/run.t

Fix locate for documentation queries Previously the locate function would relies on a Typedtrie search and ocasionally return the documentation at the same time. With shape-based locate this is not the case anymore and we always rely on a call to `Ocamldoc.associate_comment`. This commit also fix file_switching log and simplifies locate's return type. Split locate and from_longident code in two steps - first identify the correct Uid by reducing the shape - then load the cmt to find the corresponding location

fmt: remove some trailing whitespaces

This is necessary to share the code with the new uid-based get_doc function

…ixes ocaml#1421

…naliasing goodness

Tests: OK (jump to functor arg sig) Test: document OK Test: update partial test Test: add a new test foir predefs Add test for jumping to a functor's argument Test: show issue with dune's module aliases Tests: module-doc foxed and improved Tests: module-aliases fixed and improved Tests: missing changes for module-aliases Tests: Add failing test for completion doc Tests: from-completion-query fixed + more cases Remove the `partial-cmt` test (The feature is not avalable anymorewith the shape-based backend.)

@ddickstein

CHANGES for 414: Tue Apr 5 20:51:42 CEST 2022 + merlin binary - don't reset the environment when running merlin in single mode so that the parent environement is forwarded the the child processes (ocaml/merlin#1425) - filter dups in source paths (ocaml/merlin#1218) - improve load path performance (ocaml/merlin#1323) - fix handlink of ppx's under Windows (ocaml/merlin#1413) - locate: look for original source files before looking for preprocessed files (ocaml/merlin#1219 by @ddickstein, fixes ocaml/merlin#894) - handle `=` syntax in compiler flags (ocaml/merlin#1409) - expose all destruct exceptions in the api (ocaml/merlin#1437) - fix superfluous break in error reporting (ocaml/merlin#1432) - recognise binding operators in locate and occurrences (ocaml/merlin#1398, @mattiase) - remove dependency on Result (ocaml/merlin#1441, @kit-ty-kate) - use the new "shapes" generated by the compiler to perform precise jump-to-definition (ocaml/merlin#1431) + editor modes - fix an issue in Neovim where the current line jumps to the top of the window on repeated calls to `MerlinTypeOf` (ocaml/merlin#1433 by @ddickstein, fixes ocaml/merlin#1221) - add module, module type, and class imenu items for emacs (ocaml/merlin#1244, @ivg) - add prefix argument to force or prevent opening in a new buffer in locate command (ocaml/merlin#1426, @panglesd) - add type-on-hover functionality for vim (ocaml/merlin#1439, @nilsbecker) - add a dedicated buffer `*merlin-errors*` containing the last viewed error (ocaml/merlin#1414, @panglesd) + test suite - make `merlin-wrapper` create a default `.merlin` file only when there is no `dune-project` to let tests use `dune ocaml-merlin` reader. (ocaml/merlin#1425) - cover locate calls on module aliases with and without dune - Add a test expliciting the interaction between locate and Dune's generated source files (ocaml/merlin#1444) CHANGES for 413: Tue Apr 5 20:59:42 CEST 2022 + merlin binary - don't reset the environment when running merlin in single mode so that the parent environement is forwarded the the child processes (ocaml/merlin#1425) - filter dups in source paths (ocaml/merlin#1218) - improve load path performance (ocaml/merlin#1323) - fix handlink of ppx's under Windows (ocaml/merlin#1413) - locate: look for original source files before looking for preprocessed files (ocaml/merlin#1219 by @ddickstein, fixes ocaml/merlin#894) - handle `=` syntax in compiler flags (ocaml/merlin#1409) - expose all destruct exceptions in the api (ocaml/merlin#1437) - fix superfluous break in error reporting (ocaml/merlin#1432) - recognise binding operators in locate and occurrences (ocaml/merlin#1398, @mattiase) - remove dependency on Result (ocaml/merlin#1441, @kit-ty-kate) + editor modes - fix an issue in Neovim where the current line jumps to the top of the window on repeated calls to `MerlinTypeOf` (ocaml/merlin#1433 by @ddickstein, fixes ocaml/merlin#1221) - add module, module type, and class imenu items for emacs (ocaml/merlin#1244, @ivg) - add prefix argument to force or prevent opening in a new buffer in locate command (ocaml/merlin#1426, @panglesd) - add type-on-hover functionality for vim (ocaml/merlin#1439, @nilsbecker) - add a dedicated buffer `*merlin-errors*` containing the last viewed error (ocaml/merlin#1414, @panglesd) + test suite - make `merlin-wrapper` create a default `.merlin` file only when there is no `dune-project` to let tests use `dune ocaml-merlin` reader. (ocaml/merlin#1425) - cover locate calls on module aliases with and without dune - Add a test expliciting the interaction between locate and Dune's generated source files (ocaml/merlin#1444) CHANGES for 412: Tue Apr 5 21:12:42 CEST 2022 + merlin binary - don't reset the environment when running merlin in single mode so that the parent environement is forwarded the the child processes (ocaml/merlin#1425) - locate: look for original source files before looking for preprocessed files (ocaml/merlin#1219 by @ddickstein, fixes ocaml/merlin#894) - fix handlink of ppx's under Windows (ocaml/merlin#1413) - handle `=` syntax in compiler flags (ocaml/merlin#1409) - fix superfluous break in error reporting (ocaml/merlin#1432) - recognise binding operators in locate and occurrences (ocaml/merlin#1398, @mattiase) - improve load path performance (ocaml/merlin#1323) - remove dependency on Result (ocaml/merlin#1441, @kit-ty-kate) + editor modes - fix an issue in Neovim where the current line jumps to the top of the window on repeated calls to `MerlinTypeOf` (ocaml/merlin#1433 by @ddickstein, fixes ocaml/merlin#1221) - add module, module type, and class imenu items for emacs (ocaml/merlin#1244, @ivg) - add prefix argument to force or prevent opening in a new buffer in locate command (ocaml/merlin#1426, @panglesd) - add type-on-hover functionality for vim (ocaml/merlin#1439, @nilsbecker) - add a dedicated buffer `*merlin-errors*` containing the last viewed error (ocaml/merlin#1414, @panglesd) + test suite - make `merlin-wrapper` create a default `.merlin` file only when there is no `dune-project` to let tests use `dune ocaml-merlin` reader. (ocaml/merlin#1425) CHANGES for 411: Tue Apr 5 21:17:21 PM CET 2022 + merlin binary - don't reset the environment when running merlin in single mode so that the parent environement is forwarded the the child processes (ocaml/merlin#1425) - locate: look for original source files before looking for preprocessed files (ocaml/merlin#1219 by @ddickstein, fixes ocaml/merlin#894) - fix handling of ppx's under Windows (ocaml/merlin#1413) - handle `=` syntax in compiler flags (ocaml/merlin#1409) - fix superfluous break in error reporting (ocaml/merlin#1432) - recognise binding operators in locate and occurrences (ocaml/merlin#1398, @mattiase) - remove dependency on Result (ocaml/merlin#1441, @kit-ty-kate) + editor modes - update quick setup instructions for emacs (ocaml/merlin#1380, @ScriptDevil) - fix an issue in Neovim where the current line jumps to the top of the window on repeated calls to `MerlinTypeOf` (ocaml/merlin#1433 by @ddickstein, fixes ocaml/merlin#1221) - add module, module type, and class imenu items for emacs (ocaml/merlin#1244, @ivg) - add prefix argument to force or prevent opening in a new buffer in locate command (ocaml/merlin#1426, @panglesd) - add type-on-hover functionality for vim (ocaml/merlin#1439, @nilsbecker) - add a dedicated buffer `*merlin-errors*` containing the last viewed error (ocaml/merlin#1414, @panglesd) + test suite - make `merlin-wrapper` create a default `.merlin` file only when there is no `dune-project` to let tests use `dune ocaml-merlin` reader. (ocaml/merlin#1425)

voodoos changed the base branch from master to 414 January 14, 2022 14:59

voodoos mentioned this pull request Jan 14, 2022

[WIP] Use OCaml 4.14 + shapes to ré-implement locate in Merlin #1416

Closed

3 tasks

voodoos force-pushed the 414-shapes-cbn branch 3 times, most recently from 245a140 to 111f2b9 Compare January 27, 2022 14:28

voodoos force-pushed the 414-shapes-cbn branch from ae1695d to c7e80d9 Compare February 3, 2022 14:41

voodoos force-pushed the 414 branch from 229a9b8 to a811fe0 Compare February 3, 2022 15:11

voodoos force-pushed the 414-shapes-cbn branch from 4de0c73 to cdde171 Compare February 4, 2022 11:11

voodoos force-pushed the 414 branch from a811fe0 to c8e2768 Compare February 5, 2022 11:19

voodoos force-pushed the 414-shapes-cbn branch from cdde171 to 88a7dd2 Compare February 7, 2022 16:20

voodoos force-pushed the 414 branch from c8e2768 to a811fe0 Compare February 10, 2022 11:17

voodoos force-pushed the 414-shapes-cbn branch from 8a74945 to 2722fda Compare February 10, 2022 11:28

voodoos force-pushed the 414 branch from a811fe0 to e1899a8 Compare February 10, 2022 11:44

voodoos force-pushed the 414-shapes-cbn branch 2 times, most recently from b216abe to c095af8 Compare February 10, 2022 11:51

let-def force-pushed the 414 branch from edb8c88 to 39e1b66 Compare February 17, 2022 04:00

voodoos force-pushed the 414 branch from e273cc7 to f8e1851 Compare March 28, 2022 19:06

voodoos changed the title ~~[Shapes] OCaml 4.14 + Shapes + Strong call by need~~ [Shapes] OCaml 4.14 + Shapes Mar 29, 2022

voodoos mentioned this pull request Mar 29, 2022

Fix #1228 #1331

Closed

voodoos force-pushed the 414 branch from f6464df to d76db1b Compare March 31, 2022 16:34

voodoos force-pushed the 414-shapes-cbn branch from 82723b8 to 6cb3d37 Compare March 31, 2022 17:18

trefis deleted the branch ocaml:master April 5, 2022 10:20

trefis closed this Apr 5, 2022

trefis reopened this Apr 5, 2022

trefis force-pushed the 414-shapes-cbn branch from a8a6fcf to 014f48d Compare April 5, 2022 11:15

trefis changed the base branch from 414 to master April 5, 2022 11:16

trefis reviewed Apr 5, 2022

View reviewed changes

tests/test-dirs/locate/locate-stdlib.t Outdated Show resolved Hide resolved

trefis reviewed Apr 5, 2022

View reviewed changes

tests/test-dirs/locate/partial-cmt.t/run.t Outdated Show resolved Hide resolved

voodoos force-pushed the 414-shapes-cbn branch 3 times, most recently from cb0a952 to 7d06a27 Compare April 5, 2022 16:35

voodoos and others added 16 commits April 5, 2022 13:37

Rename load_shape function and add log on fallback

4d97a2f

Use Uids to find docstrings when appropriate

ce2cf06

Add Envaux module from upstream

822b745

More logging

0dd75ac

Remove unused ~pos and ~config parameters

b7af068

fmt: remove some trailing whitespaces

Fix dot-pp-dot-ml-dune test

13488ee

Add function to detect module aliases

6d22389

Use alias detection to traverse Dune wrappings

0e0cf80

Make Locate.to_string return the uid along with the loc

f4696cc

This is necessary to share the code with the new uid-based get_doc function

Improve [doc_from_uid] to return top-level module doc when possible. F…

a301c13

…ixes ocaml#1421

Refactor [get_doc] so that it reuses [of_string] and thus share the u…

c781a1a

…naliasing goodness

Fix completion-triggered doc queries

ca70007

Fix iterator "super" call

f2551ae

Delete now unused Typedtrie module

9bf0b0d

locate: remove [Fallback], now dead code

73a01b9

voodoos force-pushed the 414-shapes-cbn branch 3 times, most recently from d16dc0d to 5f9ff06 Compare April 5, 2022 18:08

voodoos and others added 2 commits April 5, 2022 15:34

fallback: better logging, better test

77d0860

voodoos force-pushed the 414-shapes-cbn branch from dda950c to 77d0860 Compare April 5, 2022 18:34

voodoos merged commit 03563cd into ocaml:master Apr 5, 2022

voodoos mentioned this pull request Apr 5, 2022

[new release] merlin 4.5-414/413/412/411 and dot-merlin-reader 4.2 ocaml/opam-repository#21121

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Shapes] OCaml 4.14 + Shapes #1431

[Shapes] OCaml 4.14 + Shapes #1431

voodoos commented Jan 14, 2022 •

edited

Loading

voodoos commented Jan 27, 2022

voodoos commented Feb 3, 2022

voodoos commented Feb 4, 2022 •

edited

Loading

voodoos commented Feb 10, 2022

[Shapes] OCaml 4.14 + Shapes #1431

[Shapes] OCaml 4.14 + Shapes #1431

Conversation

voodoos commented Jan 14, 2022 • edited Loading

voodoos commented Jan 27, 2022

voodoos commented Feb 3, 2022

voodoos commented Feb 4, 2022 • edited Loading

voodoos commented Feb 10, 2022

voodoos commented Jan 14, 2022 •

edited

Loading

voodoos commented Feb 4, 2022 •

edited

Loading