Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix positionMapping in stale data #3920

Merged
merged 4 commits into from
Dec 26, 2023

Conversation

soulomoon
Copy link
Collaborator

@soulomoon soulomoon commented Dec 23, 2023

@soulomoon soulomoon marked this pull request as ready for review December 23, 2023 11:11
@michaelpj
Copy link
Collaborator

I have absolutely no idea how this stuff works. Maybe @wz1000 does?

@soulomoon soulomoon marked this pull request as draft December 23, 2023 15:37
Copy link
Collaborator Author

@soulomoon soulomoon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the problem lays in updatePositionMapping.
mapAccumRWithKey accumulate (key,val) decending keys, the delta should be the old ones, instead of new

@soulomoon soulomoon marked this pull request as ready for review December 24, 2023 00:18
@soulomoon
Copy link
Collaborator Author

soulomoon commented Dec 24, 2023

Since it contains version mapping between old versions and current version
then we need to share the new deltas and the old deltas get composed one by one

@soulomoon soulomoon force-pushed the FixStaleDataPositionMappings branch 2 times, most recently from 47ec09e to 8af6f0f Compare December 25, 2023 00:22
@soulomoon
Copy link
Collaborator Author

soulomoon commented Dec 26, 2023

In order to add tests on this, I have refactored the involved code into updatePositionMappingHelper from updatePositionMapping.
Just run cd ghcide; cabal test --test-option="-p /enumMapMappingTest/"

  1. Not many understand how is the bug a problem, the test case can serve as explaination.
  2. The behaviour of the bug is subtle, the test case can prevent re-introduction. I can see why no one have spotted this before.

I will explain how this is going on,
Suppose you have a text starting from (0,0) let x = 1 in 1(version 0, with x in posision (0, 4)).
For i in [1..5], you are inserting a space at (0, i) consecutively.
After the 5th insertion, the text is now l et x = 1 in 1(version 5, with x in position (0, 9)).
The bug would case the mapping of (0, 4)(version0) to (0, 8)(version5) which should be (0, 9)(version5)

The reason is that application of the mappings is reversed. The last insertion at (0,5), won't sent char at (0,4) to (0,5).

It could be easily reproduced by swtich addOldDelta back to addDelta in updatePositionMappingHelper

enumMapMappingTest :: TestTree
enumMapMappingTest = testCase "enumMapMappingTest" $ do
  let mkChangeEvent :: Range -> Text -> TextDocumentContentChangeEvent
      mkChangeEvent r t = TextDocumentContentChangeEvent $ InL $ #range .== r .+ #rangeLength .== Nothing .+ #text .== t
      mkCE :: UInt -> UInt -> UInt -> UInt -> Text -> TextDocumentContentChangeEvent
      mkCE l1 c1 l2 c2 = mkChangeEvent (Range (Position l1 c1) (Position l2 c2))
      events :: [(Int32, [TextDocumentContentChangeEvent])]
      events = map (second return) [(0, mkCE 0 0 0 0 ""), (1, mkCE 0 1 0 1 " "), (2, mkCE 0 2 0 2 " "), (3, mkCE 0 3 0 3 " "), (4, mkCE 0 4 0 4 " "), (5, mkCE 0 5 0 5 " ")]
      finalMap = Prelude.foldl (\m (i, e) -> updatePositionMappingHelper i e m) mempty events
  let updatePose fromPos = do
        mapping <- snd <$> EM.lookup 0 finalMap
        toCurrentPosition mapping fromPos
  updatePose (Position 0 4) @?= Just (Position 0 9)
  updatePose (Position 0 5) @?= Just (Position 0 10)

@soulomoon
Copy link
Collaborator Author

hi @michaelpj , I have added a test case, it should be easy to spot the bug now.

Copy link
Collaborator

@wz1000 wz1000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, great catch. All this is quite subtle!

ghcide/src/Development/IDE/Core/PositionMapping.hs Outdated Show resolved Hide resolved
ghcide/src/Development/IDE/Core/PositionMapping.hs Outdated Show resolved Hide resolved
@wz1000 wz1000 added the merge me Label to trigger pull request merge label Dec 26, 2023
@mergify mergify bot merged commit c2fcaae into haskell:master Dec 26, 2023
42 checks passed
soulomoon added a commit to soulomoon/haskell-language-server that referenced this pull request Dec 27, 2023
* Fix positionMapping in stale data

* add test for updatePositionMapping

* add comment to demonstrate addOldDelta
@michaelpj
Copy link
Collaborator

Thanks so much for doing the work to add a test!

michaelpj added a commit that referenced this pull request Jan 6, 2024
…tDocument/semanticTokens/full) (#3892)

* Implement semantic tokens lsp plugin draft

* SemanticTokens: combine information extracted from HieAst

* clean up

* map to default token types in lsp

* use lsp makeSemanticTokens to convert to lsp SemanticTokens type

* add test and cleanup

* refine semantic type to default one in lsp

* Use tokens from hieAst instead of renamedSource and add test

* use customize RefMap to get semantic type

* use refMap from useAsts

* Also compute imported names

* Also compute semantic type from TyThing

* Fix dependencies version

* fix version

* Retrieve nameSet from renamedSource to prevent names not visible(Such as by instance deriving) being handled

* add hlint config to ignore test data

* cean up test data

* revert flake.nix

* Rename query.hs to Query.hs

* Build: add semantic tokens to lts21

* Refactor and add README

* Semantic token, filter names in Ast

* CI: add consistancy check for wether semantic tokens computations is stable across different ghc versions

* Update documentation, cleanup test, remove default modifiers

* Fix: IO now classfied to TTypcon, add test for GADT and data family, Update documentation

* Restore stack.yaml

* fix stack build

* Refactor, move out ActualToken to Mappings and use ide logger

* Refactor: toLspTokenType should return Maybe type

* Stop use stale hieAst

* add getImportedNameSemanticRule rule to semantic tokens plugin

* do not retrieve hie in getImportedNameSemanticRule

* fix: add description for semantic tokens

* remove TValBind and TPaternBind and Use TFunction and TVariable instead

* cleanup

* Refactor useWithStaleMT and took care of the token range using position map

* fix build for 9.4

* refactor, use golden test

* refactor, use ExceptT for computeSemanticTokens

* Fix 9.2

* add persistentSemanticMapRule to prevent semantic tokens block on startup

* Fix, use hieKind instead of cast the type directly

* add options to turn semantic tokens on and off

* Disable stan plugin by default (#3917)

* Fix positionMapping in stale data (#3920)

* Fix positionMapping in stale data

* add test for updatePositionMapping

* add comment to demonstrate addOldDelta

* cleanup

* fix: for local variable, extract type from contextInfo instead of bind site, thus function in pattern binds can also be indentified

* clean up

* Update plugins/hls-semantic-tokens-plugin/src/Ide/Plugin/SemanticTokens/Query.hs

Co-authored-by: Michael Peyton Jones <me@michaelpj.com>

* refactor: remove TNothing and compact the test output

* refactor: rename SemanticTokenType to HsSemanticTokenType to avoid confusion with lsp' SemanticTokenTypes

* refactor: push the computation of semantic token type to getSemanticTokensRule

* update documentation

* cleanup hieAstSpanNames

* remove renamed source from getSemanticTokensRule and optimize query function for semantic token type

* try to exclude names that is not visible in hie and cleanup

* add HieFunMaskKind, it is to differ wether a type at type index is a function or non-function

* expose function flag to expose (=>, ->, -=>, ==>)

* 1. Relax GetDocMap kindMap to get TyThing for more than type variables.
2. Backport isVisibleFunArg

* use customize logger, add test for unicode

* fix: handle unicode in semantic tokens

* update KindMap to TyThingMap

* cleanup

* add realSrcSpanToCodePointRange, realSrcLocToCodePointPosition to Development.IDE.GHC.Error

* add Note [Semantic information from Multiple Sources]

* move recoverFunMaskArray to Mappings.hs

* fix test, data.Set might not appear

* fix: handle semantic tokens with more than one ast

* fix: instance PluginMethod Request Method_TextDocumentSemanticTokensFull

* clean up

* turn semantic tokens off by default

* fix doc

* clean up doc

---------

Co-authored-by: fendor <fendor@users.noreply.github.com>
Co-authored-by: Michael Peyton Jones <me@michaelpj.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merge me Label to trigger pull request merge
Projects
None yet
Development

Successfully merging this pull request may close these issues.

The current positionmappings is composing in the wrong way in lastValueIO
3 participants