No responses coming on STDOUT when starting jedi-language-server from cli

[jhrcek]

Hello. I'm building a backend of a web app that's supposed to provide python editor with autocomplete feature on the frontend.
I wanted to use jedi-language-server as (what seems to me to be) more lightweight alternative to python-language-server.
I'd like to start jedi-language-server process in the background and communicate with it using stdin/stdout.
However when trying on command line, I'm unable to elicit any responses on STDOUT.

My environment:
OS: Fedora 34
python --version: Python 3.9.5

Here's the steps demonstrating this issue:

$ pip install --upgrade jedi-language-server
...
$ jedi-language-server --version
0.32.0
$ jedi-language-server
INFO:pygls.server:Starting IO server

At this point I'm pasting some LSP messages on STDIN, like

Content-Length: 1062

{"jsonrpc":"2.0","id":0,"method":"initialize","params":{"capabilities":{"textDocument":{"hover":{"dynamicRegistration":true,"contentFormat":["plaintext","markdown"]},"synchronization":{"dynamicRegistration":true,"willSave":false,"didSave":false,"willSaveWaitUntil":false},"completion":{"dynamicRegistration":true,"completionItem":{"snippetSupport":false,"commitCharactersSupport":true,"documentationFormat":["plaintext","markdown"],"deprecatedSupport":false,"preselectSupport":false},"contextSupport":false},"signatureHelp":{"dynamicRegistration":true,"signatureInformation":{"documentationFormat":["plaintext","markdown"]}},"declaration":{"dynamicRegistration":true,"linkSupport":true},"definition":{"dynamicRegistration":true,"linkSupport":true},"typeDefinition":{"dynamicRegistration":true,"linkSupport":true},"implementation":{"dynamicRegistration":true,"linkSupport":true}},"workspace":{"didChangeConfiguration":{"dynamicRegistration":true}}},"initializationOptions":null,"processId":null,"rootUri":"file:///home/ubuntu/artifacts/","workspaceFolders":null}}

or

Content-Length: 50

{"jsonrpc": "2.0","method": "exit","params": null}

But there are no responses coming on STDOUT. Actually it seems that no matter what I enter on STDIN there are no responses on STDOU nor STDERR.
When I try similar steps with other language servers (like python-language-server) I'm getting responses on STDOUT.

Could you please advise if I'm doing something wrong or if this is a bug?

[Terseus]

I think jedi-language-server doesn't support input from stdin, see #51.

Maybe it's a good idea to add stdin support, even if only to do quick'n'dirty testing.

[pappasam]

For use-cases involving embedding Jedi-language-server in a web application, I believe you should use tcp instead of stdio. You can do this by passing the tcp option at the command line and then communicating with Jedi-language-server with tcp.

jedi-language-server --tcp

As far as specifics of what requests to use, it's somewhat beyond my current understanding. If you'd like to dig deeper, I suggest looking into pygls's tcp support and how that framework handles tcp.

[jhrcek]

Thank you for your responses.
We're still figuring out the architecture, but we have to support multiple programming languages and so far it seems that we'll want language servers to run in separate docker containers started with --network none so having the communication via stdin/stdout would be preferable.
I'm ok if you say that jedi-language-server doesn't support it, I can still use python-language-server instead which seems to support this way of usage just fine. The only thing I find a bit confusing is that the --help output is saying
use TCP server instead of stdio. That "instead" (+the fact that other language servers I already integrated support it) made me think that stdio was the default I/O mechanism. Feel free to close this issue if you're not planning to add support for this.

[pappasam]

Hmm, stdio is currently the default (used when you don't pass any options at the cli). Based on the linked issue above, looks like you might need "\r\n" at the end of your requests?

jedi-language-server relies on pygls, which provides the foundation for both stdio and tcp support. Any difference in behavior between jls and python-language-server regarding stdio probably involves pygls. @danixeee do you have any suggestions here for interactive usage at the command line over stdio?

[dimbleby]

The trick is to make sure that you are sending carriage returns; and you also need to remember to put an empty line between message headers and body.

eg non-interactively, you want a file like

Content-Length: 1062

{"jsonrpc":"2.0","id":0,"method":"initialize","params":{"capabilities":{"textDocument":{"hover":{"dynamicRegistration":true,"contentFormat":["plaintext","markdown"]},"synchronization":{"dynamicRegistration":true,"willSave":false,"didSave":false,"willSaveWaitUntil":false},"completion":{"dynamicRegistration":true,"completionItem":{"snippetSupport":false,"commitCharactersSupport":true,"documentationFormat":["plaintext","markdown"],"deprecatedSupport":false,"preselectSupport":false},"contextSupport":false},"signatureHelp":{"dynamicRegistration":true,"signatureInformation":{"documentationFormat":["plaintext","markdown"]}},"declaration":{"dynamicRegistration":true,"linkSupport":true},"definition":{"dynamicRegistration":true,"linkSupport":true},"typeDefinition":{"dynamicRegistration":true,"linkSupport":true},"implementation":{"dynamicRegistration":true,"linkSupport":true}},"workspace":{"didChangeConfiguration":{"dynamicRegistration":true}}},"initializationOptions":null,"processId":null,"rootUri":"file:///home/ubuntu/artifacts/","workspaceFolders":null}}

making sure that you have windows line endings, then:

$ jedi-language-server < example
INFO:pygls.server:Starting IO server
INFO:pygls.feature_manager:Registered "textDocument/didOpen" with options "None"
INFO:pygls.feature_manager:Registered "textDocument/didChange" with options "None"
INFO:pygls.feature_manager:Registered "textDocument/didSave" with options "None"
INFO:pygls.feature_manager:Registered "textDocument/hover" with options "None"
INFO:pygls.protocol:Language server initialized work_done_token=None process_id=None root_uri='file:///home/ubuntu/artifacts/' capabilities=ClientCapabilities(workspace=WorkspaceClientCapabilities(apply_edit=None, workspace_edit=None, did_change_configuration=DidChangeConfigurationClientCapabilities(dynamic_registration=True), did_change_watched_files=None, symbol=None, execute_command=None, workspace_folders=None, configuration=None, semantic_tokens=None, code_lens=None, file_operations=None), text_document=TextDocumentClientCapabilities(synchronization=TextDocumentSyncClientCapabilities(dynamic_registration=True, will_save=False, will_save_wait_until=False, did_save=False), completion=CompletionClientCapabilities(dynamic_registration=True, completion_item=CompletionItemClientCapabilities(snippet_support=False, commit_characters_support=True, documentation_format=[<MarkupKind.PlainText: 'plaintext'>, <MarkupKind.Markdown: 'markdown'>], deprecated_support=False, preselect_support=False, tag_support=None, insert_replace_support=None, resolve_support=None, insert_text_mode_support=None), completion_item_kind=None, context_support=False), hover=HoverClientCapabilities(dynamic_registration=True, content_format=[<MarkupKind.PlainText: 'plaintext'>, <MarkupKind.Markdown: 'markdown'>]), signature_help=SignatureHelpClientCapabilities(dynamic_registration=True, signature_information=SignatureHelpInformationClientCapabilities(documentation_format=[<MarkupKind.PlainText: 'plaintext'>, <MarkupKind.Markdown: 'markdown'>], parameter_information=None, active_parameter_support=None), context_support=None), declaration=DeclarationClientCapabilities(dynamic_registration=True, link_support=True), definition=DefinitionClientCapabilities(dynamic_registration=True, link_support=True), type_definition=TypeDefinitionClientCapabilities(dynamic_registration=True, link_support=True), implementation=ImplementationClientCapabilities(dynamic_registration=True, link_support=True), references=None, document_highlight=None, document_symbol=None, code_action=None, code_lens=None, document_link=None, color_provider=None, formatting=None, range_formatting=None, on_type_formatting=None, rename=None, publish_diagnostics=None, folding_range=None, selection_range=None, linked_editing_range=None, call_hierarchy=None, semantic_tokens=None, moniker=None), window=None, general=None, experimental=None) client_info=None locale=None root_path=None initialization_options=None trace=None workspace_folders=None
INFO:pygls.protocol:Sending data: {"jsonrpc": "2.0", "id": 0, "result": {"capabilities": {"textDocumentSync": {"openClose": true, "change": 2, "willSave": false, "willSaveWaitUntil": false, "save": true}, "completionProvider": {"triggerCharacters": [".", "'", "\""], "resolveProvider": true}, "hoverProvider": true, "signatureHelpProvider": {"triggerCharacters": ["(", ","]}, "definitionProvider": true, "referencesProvider": true, "documentHighlightProvider": true, "documentSymbolProvider": true, "codeActionProvider": {"codeActionKinds": ["refactor.inline", "refactor.extract"]}, "renameProvider": true, "executeCommandProvider": {"commands": []}, "workspaceSymbolProvider": true, "workspace": {"workspaceFolders": {"supported": true, "changeNotifications": true}, "fileOperations": {}}}}}
Content-Length: 758
Content-Type: application/vscode-jsonrpc; charset=utf-8

{"jsonrpc": "2.0", "id": 0, "result": {"capabilities": {"textDocumentSync": {"openClose": true, "change": 2, "willSave": false, "willSaveWaitUntil": false, "save": true}, "completionProvider": {"triggerCharacters": [".", "'", "\""], "resolveProvider": true}, "hoverProvider": true, "signatureHelpProvider": {"triggerCharacters": ["(", ","]}, "definitionProvider": true, "referencesProvider": true, "documentHighlightProvider": true, "documentSymbolProvider": true, "codeActionProvider": {"codeActionKinds": ["refactor.inline", "refactor.extract"]}, "renameProvider": true, "executeCommandProvider": {"commands": []}, "workspaceSymbolProvider": true, "workspace": {"workspaceFolders": {"supported": true, "changeNotifications": true}, "fileOperations": {}}}}}INFO:pygls.server:Shutting down the server
INFO:pygls.server:Closing the event loop.

Interactively, typing the carriage returns is not so easy. At first I thought that Ctrl-M would do it but, at least on my terminal, I need to go Ctrl-V Ctrl-M... well that's a whole different rabbit hole. (Also you probably want to redirect the logs, on stderr.) Then:

$ jedi-language-server 2>logs
Content-Length: 1062^M
^M
{"jsonrpc":"2.0","id":0,"method":"initialize","params":{"capabilities":{"textDocument":{"hover":{"dynamicRegistration":true,"contentFormat":["plaintext","markdown"]},"synchronization":{"dynamicRegistration":true,"willSave":false,"didSave":false,"willSaveWaitUntil":false},"completion":{"dynamicRegistration":true,"completionItem":{"snippetSupport":false,"commitCharactersSupport":true,"documentationFormat":["plaintext","markdown"],"deprecatedSupport":false,"preselectSupport":false},"contextSupport":false},"signatureHelp":{"dynamicRegistration":true,"signatureInformation":{"documentationFormat":["plaintext","markdown"]}},"declaration":{"dynamicRegistration":true,"linkSupport":true},"definition":{"dynamicRegistration":true,"linkSupport":true},"typeDefinition":{"dynamicRegistration":true,"linkSupport":true},"implementation":{"dynamicRegistration":true,"linkSupport":true}},"workspace":{"didChangeConfiguration":{"dynamicRegistration":true}}},"initializationOptions":null,"processId":null,"rootUri":"file:///home/ubuntu/artifacts/","workspaceFolders":null}}
Content-Length: 758
Content-Type: application/vscode-jsonrpc; charset=utf-8

{"jsonrpc": "2.0", "id": 0, "result": {"capabilities": {"textDocumentSync": {"openClose": true, "change": 2, "willSave": false, "willSaveWaitUntil": false, "save": true}, "completionProvider": {"triggerCharacters": [".", "'", "\""], "resolveProvider": true}, "hoverProvider": true, "signatureHelpProvider": {"triggerCharacters": ["(", ","]}, "definitionProvider": true, "referencesProvider": true, "documentHighlightProvider": true, "documentSymbolProvider": true, "codeActionProvider": {"codeActionKinds": ["refactor.inline", "refactor.extract"]}, "renameProvider": true, "executeCommandProvider": {"commands": []}, "workspaceSymbolProvider": true, "workspace": {"workspaceFolders": {"supported": true, "changeNotifications": true}, "fileOperations": {}}}}}

[pappasam]

Resolved by documenting steps here: befaecd

[danixeee]

You might consider adding an option to start jedi-language-server over websockets (openlawlibrary/pygls#129).