feat: intellisense > prioritize column items over others #465

morgilad · 2024-06-09T12:06:39Z

This is a small workaround to prevent the Monaco editor from re-arranging the IntelliSense completion items based on the user's search term.

Firstly, I trigger provideCompletionItems on every key press by setting incomplete=true so I can change the filterText dynamically. To avoid calling the language server on each key press, I implemented a caching mechanism that prevents unnecessary calls.

Additionally, I prepend the user search term to column type items only when the search term is present in the filter text.

Since the Monaco editor re-arranges items where the search term appears first in the filterText, this ensures that the column items are brought up in the IntelliSense order.

** One thing missing in the implementation is that currently, the Monaco editor filters items using a fuzzy search, and I am not applying fuzzy matching to column names. We need to make a decision here: either implement fuzzy matching for column names and search terms as well, or remove the fuzzy search from the Monaco editor as Michal originally wanted.

maxburs · 2024-06-14T04:52:04Z

To avoid calling the language server on each key press, I implemented a caching mechanism that prevents unnecessary calls.

I don't think I understand. Doesn't the cache cause different suggestions when it's hit?

Additionally, I prepend the user search term to column type items only when the search term is present in the filter text.

Since the Monaco editor re-arranges items where the search term appears first in the filterText, this ensures that the column items are brought up in the IntelliSense order.

This hack makes me a bit uncomfortable... given where things are now, I'd rather push back on requirements that require us to do things like this.

If we were super on-top-of the current language server code adding a hack like this might be less risky, but as it is, I'd rather not add unnecessary complexity/risk.

That all said, I'm not really familiar enough with this code to fully understand how risky this really is. If you want to go over it more slowly I'd be happy to hop on a call.

package/tsconfig.json

maxburs · 2024-06-14T04:32:28Z

package/src/completionCacheManager/completionCacheManager.ts

+            const isIncluded = word?.includes(lastWord);
+            const shouldGetItems = !lastWord || !word || !isIncluded;
+            if (shouldGetItems) {
+                completionList = await getFromLanguageService(resource, position);


If we were on top of things I'd expect to see something after the wait point to check if we're still active or not.

I know our language server doesn't currently have much in the way of cancelation support, so I'm not expecting this PR to fix it.

Not sure I understand

Actually, I am sure I don't :)

For example, if we leave the page completion items are being fetched, we should stop progress.

Right now it's pretty easy to create exceptions by quickly closing the page after doing something to query text when the language server is active.

package/src/completionCacheManager/completionCacheManager.ts

maxburs · 2024-06-14T04:42:20Z

package/src/completionCacheManager/completionCacheManager.spec.ts

+    const mockGetFromLanguageService: jest.MockedFunction<GetFromLanguageService> = jest.fn();
+    const resource = {} as monaco.Uri;
+    const position = {} as ls.Position;


Nit: Do these need to be fake? Could we pass things that type-check to make the test more durable?

Fakes are test doubles that implement business rules. In contrast, dummies are objects that are passed around but never actually used. They typically fill parameter lists when no operations are needed on that object. This scenario is ideal for using dummies. Dummies help simplify the test setup by acting as placeholders for required objects that are not the focus of the test. In this case, the focus of the test is whether the languageService was called or not.

maxburs · 2024-06-14T04:45:23Z

package/src/completionCacheManager/completionCacheManager.ts

+            }
+
+            lastWord = word;


Suggested change

}

lastWord = word;

lastWord = word;

}

If we update the last work when the completion items haven't changed, don't we make it possible to not get completion items even when the new word make sense for the original completionList?

For example, if we call this with these words: "asdf", "a", "df". "df" won't hit the cache because "a" proceeded it, but would have if we omit "a".

Let's simulate what happens when we call it with these words in this order (I am mimicking it character by character as the user would type it on their keyboard. Copy-pasting doesn't trigger the CompletionAdapter at all):

"a" - miss - calls LS (language service)
"as" - hit - doesn't call LS
"asd" - hit
"asdf" - hit
"asd" - CompletionAdapter isn't triggered when deleting
"as" - not triggered
"a" - not triggered
"" - not triggered
"a" - miss as lastWord is "asdf" and it is not contained in "a"
"" - not triggered
"d" - miss as "a" is not contained in "d"
"df" - hit as "d" is included in "df"
So, if I understood you correctly, I think we are okay in this case. What do you think?

There's a few things I'm pretty sure that will happen that I think are pretty weird:

"as" - hit - doesn't call LS
"asd" - hit
"asdf" - hit

Are we sure it makes sense to use cached results with these? I'd expect "asdf" to return pretty different results from "a"

"a" - miss as lastWord is "asdf" and it is not contained in "a"

Even though we updated the last word, the last completion result was "a". If we only updated the word when we update this completion result, this would be a cache hit.

Are you expecting "asdf" to return completely different results or just a subset of the results returned from "a"? The Monaco editor filters the results, so as long as the results from "asdf" are a subset, we're fine. Previously, before setting incomplete: true, the Monaco editor would call provideCompletionItems only on the first character (and also on the fourth). I attempted to replicate this logic here (except for the fourth character as it produced strange results).

maxburs · 2024-06-14T04:54:10Z

package/src/completionCacheManager/completionCacheManager.spec.ts

How feasible would it be to initialize an annual language server instance, and write tests against that?

An annual language server? I am not sure I understand.

Sorry, meant to type “actual”

In this case, it is unnecessary since the test is only checking if the language service is called, not the specifics of how it is called.

I guess I'm a little worried the tests are a little too unit-y. It would be nice to have tests that run through full scenarios like you describe here [1].

[1] #465 (comment)

I'd try and figure out how we can snapshot completion results based on a list of user inputs, so we can see how the completion suggestions change when we update the code.

You are absolutely correct, these are unit tests. I love unit tests because they run quickly and test pure logic, which is exactly what I need in this case. I completely agree that having a full scenario test is also crucial, and we do have one. I began the task by writing an integration acceptance test, which you can see here.

sagivf · 2024-06-17T04:01:47Z

To avoid calling the language server on each key press, I implemented a caching mechanism that prevents unnecessary calls.

I don't think I understand. Doesn't the cache cause different suggestions when it's hit?

Additionally, I prepend the user search term to column type items only when the search term is present in the filter text.
Since the Monaco editor re-arranges items where the search term appears first in the filterText, this ensures that the column items are brought up in the IntelliSense order.

This hack makes me a bit uncomfortable... given where things are now, I'd rather push back on requirements that require us to do things like this.

If we were super on-top-of the current language server code adding a hack like this might be less risky, but as it is, I'd rather not add unnecessary complexity/risk.

That all said, I'm not really familiar enough with this code to fully understand how risky this really is. If you want to go over it more slowly I'd be happy to hop on a call.

@maxburs the hack here is defiantly not ideal, but I think it's worth the risk:

The need to always show tables over functions is fairly important. Monaco doesn't support this because their main focus is VS and that doesn't have this need. For a DB I think it's pretty important.
We have tests and will add more to make sure LS or Monaco don't break things.
The part that adds the most complexity here is the caching, which I'm not sure is a must. If we find issues, we could potentially just remove it.
we can try to work with the Monaco team in the meantime to try and get better support for this.

package/src/completionCacheManager/completionCacheManager.ts

sagivf · 2024-06-17T04:08:06Z

Do we have a test for count() before count_distinct?

`is it now easier to make it take presedent before the full work is available? (always give priority to single word over multiple)

package/src/completionCacheManager/completionCacheManager.ts

morgilad added 5 commits June 9, 2024 10:50

acceptance test

8b725f3

add jest types

4255e5d

add faker for tests

5aac254

completion items cache manager

cab8a31

completion items > table columns always appears first

44f3e82

morgilad requested review from sagivf and maxburs June 9, 2024 12:06

morgilad added 3 commits June 9, 2024 15:21

run unit tests on github actions

24856fb

refactor tests

413e365

refactor completion cache manager

329fb49

maxburs reviewed Jun 14, 2024

View reviewed changes

pr fixes

8842599

sagivf reviewed Jun 17, 2024

View reviewed changes

package/src/completionCacheManager/completionCacheManager.ts Show resolved Hide resolved

sagivf approved these changes Jun 17, 2024

View reviewed changes

morgilad added 2 commits June 17, 2024 11:45

sort completion items by match text while keeping kind order

8646c7c

add test case

a3a4439

morgilad merged commit e7deb72 into master Jun 17, 2024
4 checks passed

maxburs reviewed Jun 17, 2024

View reviewed changes

package/src/completionCacheManager/completionCacheManager.ts Show resolved Hide resolved

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: intellisense > prioritize column items over others #465

feat: intellisense > prioritize column items over others #465

morgilad commented Jun 9, 2024

maxburs commented Jun 14, 2024

maxburs Jun 14, 2024

morgilad Jun 16, 2024

morgilad Jun 16, 2024

maxburs Jun 17, 2024

maxburs Jun 14, 2024

morgilad Jun 16, 2024

maxburs Jun 14, 2024

morgilad Jun 16, 2024

maxburs Jun 17, 2024

morgilad Jun 18, 2024

maxburs Jun 14, 2024

morgilad Jun 16, 2024

maxburs Jun 16, 2024

morgilad Jun 17, 2024

maxburs Jun 17, 2024

morgilad Jun 18, 2024

sagivf commented Jun 17, 2024

sagivf commented Jun 17, 2024

feat: intellisense > prioritize column items over others #465

feat: intellisense > prioritize column items over others #465

Conversation

morgilad commented Jun 9, 2024

maxburs commented Jun 14, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sagivf commented Jun 17, 2024

sagivf commented Jun 17, 2024