Robotframework-lsp do not find keywords imported relatively

[martinRenou]

cc. @fabioz @avaissi @orlof

I have xeus-robot and robotframework-lsp installed in my environement. I have a keywords.robot file containing:

*** Settings ***
Library  String

*** Keywords ***
This Is A Dummy KeyWord
    Convert To Lower Case  "HELLO"

And a Notebook running xeus-robot next to it:

The problem is that it doesn't find the keywords defined in keywords.robot that is imported relatively.

Although we saw that there was a .virtual_documents directory created, that seem to contain files that were opened from JupyterLab and that are used by jupyterlab-lsp. It seems that once keywords.robot is added to this .virtual_documents directory it works and the keywords are found properly.

I cannot seem to be able to reproduce the issue in Python (having a local python module next to a Notebook), is there something special that python-language-server does?

[bollwyvl]

Does the discovery stuff work in the Editor?

I haven't been able to get #493 to even run rflsp, and not for lack of trying, but then it's been a couple releases.

The .virtual_documents stuff is a (glorious, to be sure) hack, and may need further customizing per language server. If someone was feeling generous with their time, and we could get #493 to pass some basic tests in the Editor, we could get look into more advanced features for Notebook...

[martinRenou]

Does the discovery stuff work in the Editor?

Are you talking about the Jupyterlab text editor? I can have a try. Is there a difference in behavior between the text editor and the notebook?

If someone was feeling generous with their time, and we could get #493 to pass some basic tests in the Editor, we could get look into more advanced features for Notebook...

I can definitely have a look. But I might need a bit more context. What does it mean to add robotframework-lsp? I suppose you are only adding it for testing?

[fabioz]

@bollwyvl

As an unrelated note, I see you have some comments related to Python 3.6 support... RFLS doesn't officially support it, but in practice it's still tested on the CI (https://github.com/robocorp/robotframework-lsp/blob/master/.github/workflows/tests-robotframework-lsp.yml#L32), so, it should actually still work there (so, if you fell that having that support is very important for your use case, I can consider making it official).

Also, in the integration, are you missing things from the language server itself (where I can probably help more) or is it just on the Editor (client) side?

[bollwyvl]

Thanks, @fabioz, thanks for chiming in, we're not trying to make extra work for you!

Python 3.6 support

as the conda-forge maintainer, I try to respect what the upstream packages declare. I'm not looking for more things to support, certainly not platforms that will sunset in the next couple months! Anyhow, as the bulk of our n-language test setup is set up with conda, there's no problem not supporting/testing 3.6 from our side.

missing things

I think on the server side, it's related to the requirement of many language servers (maybe rflsp, haven't looked too hard) to actually have real, honest-to-goodness files on disk, not just virtual files maintained via patches and a handshake from the client. Basically, no language servers i know of speaks the fifth cell of Foo.ipynb, so when a user is trying to write a (nested) language in a notebook, we write out an unpacked version of the file to a .virtual_documents folder, but this is known to break a number of things (#537), and we're still trying to figure out a way to do this that harmonizes with how the Lab debugger virtual file system works, and how languages that have a concept of relative search path can be made to work effectively here.

Again, I've not looked too hard at recent rflsp releases to see if the watch behavior still breaks everything on startup.

[bollwyvl]

@martinRenou Is there a difference in behavior between the text editor and the notebook?

Yeah: the .virtual_documents stuff isn't invoked at all for the Editor... unless you somehow sneak in sub-language transclusions i guess.

But I might need a bit more context.

Sorry, didn't mean to make you hunt for it: #332 has the broader story. We use and love robotframework on this project, and are always trying to get closer to dogfooding as much as we can. The idea would be:

• add an autodetection spec (like we have for python, r, julia, latex...) that will find the rflsp binary and launch it if a user opens a .robot or .resource file (or a text/x-robot notebook, or uses a %%robot magic)
• put rflsp under local unit/acceptance testing
• be able to use rflsp locally to help develop our existing "big" test suite
• eventually use a kernel and .ipynb files to make more interesting docs stuff (though this is somewhat fraught because of RTD, where I won't even try to setup browsers, etc.)

[fabioz]

I think the main issue is that you're seeing the use-case where the .robot is embedded in a notebook (where the notebook is usually standalone and thus the uri in the .virtual_documents may make sense) vs. a case where the user opens a .robot file directly, where the original uri should be kept.

Although even in the case where it's in a notebook, a concrete file in the .virtual_documents may not really be needed (just keeping the contents in memory and passing the original uri postfixed by .robot may be better as the language server can then search for other -- actual .robot files in the filesystem).

It's then the job of the language server to actually know how to work with partial contents in-memory or in-disk.

[fabioz]

Now seeing https://github.com/krassowski/jupyterlab-lsp/issues/537, it seems that's the same request as this...