feat: Connect to synchronized kernel

[fecet]

I managed to implement an MVP for my feature request on #76. The implementation follows your suggested approach:

1. start sync
2. Request the connection info by executing javascript Jupyter.notebook.kernel.id
3. Open a terminal and connect to it by running jupyter console --existing.

This implementation supports connecting to a remote kernel by explicitly passing a hostname parameter and a naive approach(ssh host -t), and this require https://github.com/akinsho/toggleterm.nvim as a dependency

You can view the workflow here:

output.mp4

In my opinion, to improve this functionality, we may need to:

• Better function name
• Native terminal
• Expose user configuration interfaces to modify the running command and terminal style.
• Document
• Check windows support

Long-term plan

• Improve the process of connecting to a remote kernel to make it more intelligent.
• autocompletion Tab completion inside Jupyter QtConsole ipdb shell jupyter/qtconsole#99 Question: Completion in nvim's terminal window hrsh7th/nvim-cmp#1219
• Notebook 7 support Jupyter Notebook 7 compatibility (discussion) #74

As a beginner in JavaScript and Neovim plugin development, I would be grateful if you could share any additional ideas you may have to further improve this functionality. Additionally, I would appreciate your guidance on what else I need to do to merge this pull request.

[kiyoon]

Hi, thanks a lot for the PR! This looks awesome.

1. Is there any reason you used toggleterm? I haven't used the plugin but it's better not to have too many dependencies. Is it possible to just open a terminal natively and connect to the kernel? You can follow the implementation of :JupyniumStartAndAttachToServerInTerminal
2. To keep the naming consistent, we can change the name to :JupyniumKernelConnectInTerminal. Or, :JupyniumKernelOpenInTerminal may be a more user-friendly name?
3. Can you add this command in README?
4. I know this can be annoying, but Jupynium also supports Windows. Make sure the command runs on different types of shells.

[fecet]

Thank you for your suggestion. I will start making the changes immediately. The following points may require further input:

1. Toggleterm exposes an interface(https://github.com/akinsho/toggleterm.nvim#custom-terminals) for creating terminals, which makes it easier to configure and use. To balance this point and keep the dependencies minimal, it may be possible to fallback to Neovim's native terminal (just like Notify) when the user is detected not to have installed it.

2. Personally, I don't use Windows, so I would like to know which functions may not be compatible with Windows? As far as I can see, an incompatibility occurs with SSH because many people on Windows do not configure SSH directly but use some SSH management tools. However, I cannot think of a simple way to address this.

[kiyoon]

1. If the configuration is not absolutely game-changing, I think the native terminal still can do most. We already have quite many (optional) dependencies and things like this can be hard to maintain and test. I also know many people like to use floaterm, so toggleterm is not necessarily the one and only choice when it comes to terminal management?
2. Maybe it will work directly on PowerShell / CMD if your command is simple. If you have strings with quotes etc. you have to escape the commands in a slightly different way. You can assume that the command ssh is installed already, as I think it is installed by default on PowerShell? I also don't develop on Windows but I realised many people develop on Windows just because it's a company requirement related to software security etc. etc.

If you don't have Windows at all, I'll do some simple tests once all implementation is done. But do let me know what kind of example (complicated) shell commands should be executed, so I can see if they all work well.

[fecet]

OK, I will use the native terminal as default and expose the kernel id, so the user can define their own terminal interface if they want. As for Windows, the commands I am currently using are relatively simple, so they should be able to run without issue. However, I will keep in mind that I may need to use more complex commands in the future.

[kiyoon]

Let's first see how many people uses this feature. I think it's a niche functionality only a few people may use (but still very useful, thanks!), so if they ever wanted to customise terminal interface we can later add some more options without a dependency. First of all, is there any requirement for you or is horizontal split enough most of the time?

And the kernel ID is able to be achieved with the Lua API with one line of javascript. We don't need to really expose it I think.

[fecet]

And the kernel ID is able to be achieved with the Lua API with one line of javascript. We don't need to really expose it I think.

What I meant was to add a function similar to :lua Jupynium_connect_info(bufnr), which would only return the kernel ID. There is no need to create a Vim command for it, but it could be mentioned in the README file along with an example of how to require it to create a terminal. This way, users can configure it in their init file.

Or do we already have something like :lua Jupynium_execute_js(code)? Then that's definitely unnecessary.

[kiyoon]

:lua Jupynium_connect_info(bufnr)

This can be done with Jupynium_execute_javascript(bufnr, "Jupyter.notebook.kernel.id"). It's in the Lua API section in the README. So we can provide your functionality without needing an extra dependency, but for people who really want to customise it can still achieve that easily.

[kiyoon]

In the README, I think instead of long description you can just add in the Vim Commands section, add one line of description would be enough? I appreciate it though!

[fecet]

In the README, I think instead of long description you can just add in the Vim Commands section, add one line of description would be enough? I appreciate it though!

Good for me, I also feel that the long description now looks a bit inconsistent.

[kiyoon]

Do you also get this error?

Error executing Lua callback: [string "<nvim>"]:718: attempt to concatenate local 'kernel_id' (a userdata value)
stack traceback:
        [string "<nvim>"]:718: in function 'Jupynium_get_kernel_connect_shcmd'
        [string "<nvim>"]:726: in function <[string "<nvim>"]:723>

I think the kernel_id value is wrong. You should add return.

        kernel_id = driver.execute_script("return Jupyter.notebook.kernel.id")

[fecet]

Sorry I only test no-sync cases (connect to latest kernel) today since it's just implemented.

[kiyoon]

That's okay. I think it works alright now. Thanks a lot for your work. The only things hasn't been done are

1. Test on Windows
2. Mention pip install jupyter-console dependency in README

# jupyter-console is optional and used for `:JupyniumKernelOpenInTerminal`
pip install notebook jupyter-console

I think you can add that command in the README Jupyter Notebook version dependency area. Then we're good to go.

[fecet]

I see that you have requested someone else to do the testing work for Windows. If they lack the time, since this plugin does not require any additional installation, I should be able to find an opportunity to complete the testing on Windows.

And as for the document, it's suitable to mention an installation command in the requirements area? But I also don't find other where to place that.

[kiyoon]

If you can do that, that'd be helpful. He's actually using it on Windows so I thought it would be good to have it from an actual user. Once we make sure it runs on Windows it's good to go.

[kiyoon]

@sho-87 Hi, it would be really appreciated if you can test this feature on Windows. It runs a shell command so I think potentially it will be buggy on Windows shell.

Only if you care about this feature though!

You can just test :JupyniumKernelOpenInTerminal and see if you can interact with the Jupyter kernel of the opened notebook.

[sho-87]

A couple of things:

1. JupyniumStartAndAttachToServerInTerminal is not available to me without first starting/attaching to the server. I presume I should be running this command instead of the traditional startattach?
2. The command JupyniumKernelOpenInTerminal itself seems to work fine for me. I created a cell and executed the code, then running JupyniumKernelOpenInTerminal opens up an nvim terminal running ipython, and I have access through that to the variables I created. neat stuff

Isn't this basically a REPL window that is connected to the notebook kernel? Calling it a REPL might make it more obvious what this feature does

[fecet]

Basically, yes. But it would be a bit weird to use it as REPL since we don't know which kernel to connect without sync with jupynium. And jupynium already work as a better REPL.

Or we can let JupyniumStartAndAttachToServerInTerminal available without syncing/attaching (then it only connects to recent created kernel), and we can use it also a REPL if we cannot access GUI at some moment. But this seems inconsistent with current jupynium workflow

[kiyoon]

@sho-87

1. I think it's because you lazy load on command. However, you don't need to run that to be using this feature. :JupyniumStartAndAttachToServerInTerminal just shows the output logs of Jupynium, and I don't see a good way to combine this two features

2. I'm open for suggestions. I used the name kernel for consistency, but REPL might be a better name. If we do change the nameit would be JupyniumOpenREPLInTerminal

[sho-87]

1. I think it's because you lazy load on command. However, you don't need to run that to be using this feature. :JupyniumStartAndAttachToServerInTerminal just shows the output logs of Jupynium, and I don't see a good way to combine this two features

yup, I was just confused about the correct way to test this, but the JupyniumKernelOpenInTerminal by itself was working fine

2. I'm open for suggestions. I used the name kernel for consistency, but REPL might be a better name. If we do change the nameit would be JupyniumOpenREPLInTerminal

tbh I'd be ok with it either way. I was just trying to create a mental model for myself regarding what the feature would be for and REPL was the first thing that came to mind

[fecet]

To make it a real REPL we should also implement a method to send code cell to the terminal, currently it's just a plain terminal

[kiyoon]

Okay, I think it's ready and the current name sounds good enough. Thank you @fecet and @sho-87 for this cool feature and testing it!