Problem starting the leo-bridge server automatically with anaconda/miniconda on windows

[ar-jan]

Hi, it looks like you've been making great and steady progress over the last several months. Awesome! I just tried out the dev branch and I'v run into a problem loading the extension. I've installed all dependencies etc as described. Actually the first time I ran F5 to run the extension a welcome screen with Leo icon did appear, but I'd already clicked a VSCode message about window loading taking longer than 10 seconds, and trying a reload. Now I don't see the Leo welcome anymore.

When I click the "Connect to Leo Bridge server" icon in the Leo tab, I get Leo Bridge Connection Failed.

The terminal in the original window says this:

Executing task: npm run webpack <

leointeg@0.1.8 webpack C:\Programs\leointeg
webpack --mode development

Starting type checking service...
Using 1 worker with 2048MB memory limit
Child
Hash: f72f38eb55e39ace8f04
Version: webpack 4.41.2
Time: 3915ms
Built at: 01/12/2020 16:35:25
Asset Size Chunks Chunk Names
extension.js 271 KiB main [emitted] main
extension.js.map 357 KiB main [emitted] [dev] main
Entrypoint main = extension.js extension.js.map
[./src/constants.ts] 1.12 KiB {main} [built]
[./src/eamodioEditorManager/activeEditorTracker.ts] 3.56 KiB {main} [built]
[./src/eamodioEditorManager/comparers.ts] 1.6 KiB {main} [built]
[./src/eamodioEditorManager/documentManager.ts] 10.3 KiB {main} [built]
[./src/extension.ts] 12.1 KiB {main} [built]
[./src/leoBody.ts] 9.48 KiB {main} [built]
[./src/leoBridge.ts] 5.09 KiB {main} [built]
[./src/leoFiles.ts] 2.13 KiB {main} [built]
[./src/leoIntegration.ts] 53.2 KiB {main} [built]
[./src/leoNode.ts] 2.26 KiB {main} [built]
[./src/leoOutline.ts] 2.24 KiB {main} [built]
[./src/serverService.ts] 3.2 KiB {main} [built]
[./src/webviews/leoSettingsWebview.ts] 4.18 KiB {main} [built]
[path] external "path" 42 bytes {main} [built]
[vscode] external "vscode" 42 bytes {main} [built]
+ 27 hidden modules

WARNING in ./node_modules/ws/lib/buffer-util.js
Module not found: Error: Can't resolve 'bufferutil' in 'C:\Programs\leointeg\node_modules\ws\lib'
 @ ./node_modules/ws/lib/buffer-util.js
 @ ./node_modules/ws/lib/receiver.js
 @ ./node_modules/ws/index.js
 @ ./src/leoBridge.ts
 @ ./src/leoIntegration.ts
 @ ./src/extension.ts

WARNING in ./node_modules/ws/lib/validation.js
Module not found: Error: Can't resolve 'utf-8-validate' in 'C:\Programs\leointeg\node_modules\ws\lib'
 @ ./node_modules/ws/lib/validation.js
 @ ./node_modules/ws/lib/receiver.js
 @ ./node_modules/ws/index.js
 @ ./src/leoBridge.ts
 @ ./src/leoIntegration.ts
 @ ./src/extension.ts

Child
Time: 3904ms
Built at: 01/12/2020 16:35:25
Asset Size Chunks Chunk Names
main-styles.css 27.9 KiB main-styles [emitted] main-styles
main-styles.js 4.48 KiB main-styles [emitted] main-styles
settings.html 14.7 KiB [emitted]
settings.js 59.9 KiB settings [emitted] settings

Terminal will be reused by tasks, press any key to close it.

And the debug console says:

(node:11832) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
warning.js:27
(node:11832) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
warning.js:27
Launch with default command : py -3 c:\Programs\leointeg/leobridgeserver.py
extension.js:7226
leoInteg startup launched in 6 ms
extension.js:5532
websocket error: connect ECONNREFUSED 127.0.0.1:32125
extension.js:5921
websocket closed, code: 1006
extension.js:5925
websocket error: connect ECONNREFUSED 127.0.0.1:32125
extension.js:5921
websocket closed, code: 1006

I do have websockets installed in my default Python environment (which is an Anaconda env, Python 3.7.3). Any way to check if leointeg is using the correct Python environment? Also npm install from within VSCode terminal says all packages are installed. echo %PYTHONPATH% gives C:\Programs\Leo (correct). I'm on Windows 10. Leo is at version 6.0-final.

Any ideas to debug this further?

[ar-jan]

Ah, when I manually run python leobridgeserver.py first from a cmd prompt, and then F5 the extension, the Leo Bridge does work and I can open a Leo outline.

[boltex]

First off Thanks for being the first collaborator in this issues reporting section!
I have just extracted the automatic server startup into it's own file and have not tested it under windows yet. Strangely you come up with a bug related to this 🙄 so I'll test it out a bit later today and I'll see what happens and report in this thread. Thanks again for your testing and reporting.
Edit: typo.

[boltex]

Forgot to mention that the welcome screen will only open automatically on the first activation and on major version updates.

But of course you can open it anytime with the command palette: ctrl+shift+P , and start typing 'welcome' or 'Leo settings'

[boltex]

So I've re-read your console output more carefully and it seems that the server auto-start routine tried to start it and wrote in the log output: "Launch with default command : py -3 c:\Programs\leointeg/leobridgeserver.py".

But you seem to be able to start a python script on your system with the "python" command.

For now this extension just default to start a python script on windows with "py -3". (as opposed to "python" on other OSes) I should add more features to the automatic server startup service in order to have it detect the optimal way to start a python script on a given system.

To counter this lack of proper detection, the workaround I have put in place is a configuration option called "Python Command" near the bottom of the setting for leoInteg. (open leo settings via the command palette: Ctrl+Shift+P and start typing "leo settings") where you can specify a particular string command to start the leoBridge. (in your case "python")

Tell me if this makes sense.

If it does, I'll close this and create another issue about "detecting optimal way to start python script on given system/os" because obviously my detection procedure is way too simplistic.

Thanks! :) Edit: typos

[ar-jan]

I think a configuration option is fine, but I couldn't get it to work so far. This may be related to my using Anaconda, where there recommended way to use it is to activate the environment using Anaconda's activate.bat. This doesn't seem to work from VSCode. I also have Anaconda's python on my PATH (the old way), so I thought just python should also work, but alas.

With python as command I get:

Starting server with command: python
extension.js:7217
leoInteg startup launched in 7 ms
extension.js:5532
stderr: File "c:\Programs\leointeg/leobridgeserver.py", line 519
async def leoBridgeServer(websocket, path):
^
SyntaxError: invalid syntax
extension.js:7252
leoBridge exited with code 1

With C:\Programs\Anaconda3\envs\py3\python.exe configured I get:

Starting server with command: C:\Programs\Anaconda3\envs\py3\python.exe
extension.js:7217
leoInteg startup launched in 8 ms
extension.js:5532
leoBridge: Error importing leoApp.py
stderr: Traceback (most recent call last):
File "c:\Programs\leointeg/leobridgeserver.py", line 550, in
main()
File "c:\Programs\leointeg/leobridgeserver.py", line 516, in main
integController = leoBridgeIntegController()
File "c:\Programs\leointeg/leobridgeserver.py", line 26, in init
verbose=False) # True: prints messages that would be sent to the log pane.
File "C:\Programs\Leo\leo\core\leoBridge.py", line 73, in controller
verbose)
File "C:\Programs\Leo\leo\core\leoBridge.py", line 93, in init
self.initLeo()
File "C:\Programs\Leo\leo\core\leoBridge.py", line 124, in initLeo
assert(g.app)
AssertionError

[boltex]

Thank you @ar-jan , I've noted this issue as it being related to anaconda/miniconda environments, and will address this aspect of automatic leoBridge server startup.

In the meantime I guess someone would have to simply start an instance of the leoBridgeServer.py manually. I'll update the 'how to try this developpment version' part of the documentation to help other people who'd like to try and test out this extension.

[maphew]

My recipe for getting up and running with conda on a newish Win10 machine is here:https://groups.google.com/d/msg/leo-editor/lNoYr0rUVPw/UiKDsimWBAAJ

For me also the automatic start does not work, but it does from a vscode terminal. I tried putting python in the Leo Bridge Options for the Python command setting but it didn't work.

Debug Console output with the setting blank:

Starting server with command : py -3 c:\Users\mwilkie\code\leointeg/leobridgeserver.py
Websocket error: connect ECONNREFUSED 127.0.0.1:32125 extension.js:5565
Websocket closed, code: 1006

and with set to python:

Starting server with command: python extension.js:7461
Websocket error: connect ECONNREFUSED 127.0.0.1:32125

and set to python c:\Users\mwilkie\code\leointeg/leobridgeserver.py

Starting server with command: python c:\Users\mwilkie\code\leointeg/leobridgeserver.py
extension.js:7461
Websocket error: connect ECONNREFUSED 127.0.0.1:32125
extension.js:5565
Websocket closed, code: 1006

This is what the terminal output is when started from the original vscode window. Note the only part which was me is python leobridgeserver.py, all other text is automatic. From this I surmise the internal Leointeg launch process is missing the activate portions. I haven't yet tried to figure out what the recommended way if doing that is from within vscode. (it's likely in or linked from here https://code.visualstudio.com/docs/python/environments#_conda-environments)

Microsoft Windows [Version 10.0.18362.207]
(c) 2019 Microsoft Corporation. All rights reserved.

C:\Users\mhwilkie\code\leointeg>c:/tools/miniconda3/Scripts/activate

(base) C:\Users\mhwilkie\code\leointeg>conda activate vscode-leo

(vscode-leo) C:\Users\mhwilkie\code\leointeg>python leobridgeserver.py
Starting LeoBridge... (Launch with -h for help)
LeoBridge started at localhost on port: 32125 [ctrl+c] to break

[boltex]

@maphew

It's late & im tired so this may be a silly suggestion, but what about chaining commands? i.e. conda activate vscode-leo & python or conda activate vscode-leo && python

is it & or && to chain commands? ...i never remember

of course that's just a suggestion that came to mind... - i'll try some stuff and post more potent solutions here after a good nights sleep :)

oh! and many thanks for reporting ! it's things like this that make the project more easily installable to others until a real extension is released on the vscode market! :)

[ar-jan]

So, I just updated to the latest version on master (2020-07-04 692173d), and now connecting works using just python for Leo Bridge Options > Python Command!

[ar-jan]

Observation: I currently have "Start Server" unchecked, but when I F5 the leobridgeserver.py command is run.

[boltex]

There are 3 different ways to start the leobridge server in leoInteg. It seems to me you're mixing them up, maybe... not sure.

Before pressing F5, unfold the top dropdown to choose what to run and debug : 'run extension' only is the equivalent of running the extension as it's intended to when it's finally compiled and released.

Using "Python: leoBridge Server" or "Server and Extension" tells vscode you want it to start to run and debug the leobridge server as if it's the project you're working on, if that makes sense... so obviously you'll see a server start, but it's not leoInteg who's starting it, it's vscode debug panel.

Like I said on leo's forums, this is just quick 'thinking out loud' reply and I will soon cleanup those processes, and re-write more detailled documentation about this. (and I'll make sure to keep you updated)

Oh! and most importantly: Thanks again for reporting all those tryouts man! Very much appreciated !

Félix
edit: typos

[ar-jan]

Ah, looks like it. I'd just been running F5 without looking at the setting, which was set to "Server and Extension" for me. Anyway the big thing is that for some reason starting the bridge now works from leointeg, so I don't have to start it manually from cli anymore :).

[maphew]

is it & or && to chain commands? ...i never remember

& = run next command when first finishes
&& = run next command only if first exits without an error

I think it's the same for Windows CMD and 'nix Bash. In powershell I've read that it's different. My go-to place for things like this is SS64.com (https://ss64.com/nt/syntax-redirection.html).

I haven't had time to dig through the new stuff yet. It'll be a few days.

[ar-jan]

Upon trying this out a bit more, my previous message was premature/incomplete. Only the following combination seems to work well:

• VS Code's python.condaPath configured with c:\Anaconda3\Scripts\activate (or activate.bat; uses the base environment), or a copy of c:\Anaconda3\Scripts\activate with the environment name hardcoded in the last line
• Leointeg's Python Command configuration set to python
• Running the extension with the Server and Extension option

I had previously experimented extensively with chaining the activate.bat and python commands, without succes. Also tried out custom batch files which work when invoked manually, but not when configured as Leointeg Python Command.

[boltex]

@ar-jan and @maphew

something not right in what you're describing...

Leointeg's Python Command configuration set to python and Running the extension with the Server and Extension option are mutually exclusive: I have not explained those concepts clearly - and as I wanted to give more options for tester than less, it seems people are mix and matching those methods of starting the server in erroneous ways.

So to clarify: if you're running the extension with the "Server and Extension", then, the python command could be set to 'potato' it wouldn't matter because it's not used then. That would be using the debug profile in the .vscode/launch.json file. which I encourage you to open and see for youself what those 3 debugging profiles are,. those will not exist at all when the extension is compiled and distributed as a real extension.

So this is important to distinguish: to simulate a real, finished extension, select 'run extension' only in the run/debug menu. this will force you to either :
1- set leoInteg option to start server automatically with the provided python command, both being in the leoInteg settings.
2- start a server by typing leobridgeserver.py in a terminal.

If you instead select "server AND extension" in the run/debug panel, then, vscode will run and debug the server by itself, as if vscode itself was considering the server program as it's main project. and will use whichever python debug setting vscode's debugger is set to. Not the leoInteg options!

So...Either configure a python command line setting in leo Integs settings, and use the 'run extension' only in the run/debugger dropdown. OR , don't start server automatically, and use the "start both server and extension" in the run/debuger dropdown instead. (ok to connect automatically though, doesnt matter how the server was started to connect to it automatically)

Again, sorry English is not my native language. (i guess i should capture myself as a screencast while explaining this)

Félix

[ar-jan]

Ah yes of course, that's what you were also saying on the list. So scratch the Python Command part, the point is that only when using python.condaPath and Run Server and Extension do I get the bridge to work without executing the commands manually. None of the variations of Leointeg's Python Command have worked for me (I've just gone over a few again, making sure to only use Run Extension).

[boltex]

I wonder if this could be retested somehow?

[ar-jan]

I'm currently not using Windows and don't have an installation for testing. Interestingly I have other problems now on Linux, mainly with the Jupyter extension failing to load, but that seems related to the Python interpeter configuration. Hoping to try out leointeg again soon.

[boltex]

@ar-jan Hi again! :)

in leo-editor/leo-editor#2119, you state it somewhat works now, So I was wondering if this issue about starting the server automatically with anaconda/miniconda is still relevant...? Did the new 'Server options' have helped working around this problem? ...or are you not using anaconda/miniconda in your recent tests?

Anyways thanks again for your good suggestions and testing! Appreciated!

Félix

[maphew]

I think things have changed enough that this issue is no longer applicable and can be closed - or needs to be opened as new.

I am successfully using Leo Integ via a pipx venv with a setup very similar to my conda one above. Key to the solution was the new Path settings via #292 and LeoInteg v1.0.15.

For the record, here are the working paths for my system:

Leo‑Editor Path : C:\apps\pipx\venvs\leo\Lib\site-packages
Python Command : C:\apps\pipx\venvs\leo\Scripts\python.exe

Server Process Launch Command:
C:\apps\pipx\venvs\leo\Scripts\python.exe C:\apps\pipx\venvs\leo\Lib\site-packages/leo/core --port 32125

(Note: use pythonw.exe to note open a separate console window outside of vscode)