Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Slow completion for large Python packages when using the default pyls #272

Closed
jhettler opened this issue May 19, 2020 · 68 comments
Closed
Labels

Comments

@jhettler
Copy link

jhettler commented May 19, 2020

Description

Autocompletion - especially of bigger packages like pandas is slow.

Reproduce

  1. Go to Jupyterlab
  2. Write import pandas as pd
  3. Write pd.<tab>
  4. It takes seconds in Jupyterlab, in classic notebook it is immediately autocompleted.

Is it usual behavior or there is something wrong configured?

Expected behavior

Autocompletion finished nearly immediately :)

Context

  • Operating System and version: Linux b832f09e7bfa 5.4.0-7626-generic Code completion sometimes overwrites existing text  #30~1588169883~20.04~bbe668a-Ubuntu SMP x86_64 x86_64 x86_64 GNU/Linux - Docker
  • Browser and version: Chrome Version 81.0.4044.138 (Official Build) (64-bit)
  • JupyterLab version: Jupyterlab version 2.1.0 (same behavior was with 2.1.2)
Required: installed server extensions
config dir: /home/jovyan/.jupyter
    jupyterlab_git  enabled 
config dir: /opt/conda/etc/jupyter
    jupyter_lsp  enabled 
    jupyterlab  enabled 
    jupyterlab_git  enabled 
    nbdime  enabled 
    - Validating...
      jupyterlab_git 0.20.0 OK
    - Validating...
      jupyter_lsp 0.8.0 OK
    - Validating...
      jupyterlab 2.1.0 OK
    - Validating...
      jupyterlab_git 0.20.0 OK
    - Validating...
      nbdime 2.0.0 OK
Required: installed lab extensions
   app dir: /opt/conda/share/jupyter/lab
        @aquirdturtle/collapsible_headings v2.2.0  enabled  OK
        @bokeh/jupyter_bokeh v2.0.0  enabled  OK
        @jupyter-widgets/jupyterlab-manager v2.0.0  enabled  OK
        @jupyterlab/git v0.20.0  enabled  OK
        @krassowski/jupyterlab-lsp v1.0.0  enabled  OK
        jupyter-matplotlib v0.7.2  enabled  OK
        jupyterlab-plotly v1.5.4  enabled  OK
        jupyterlab-python-file v0.4.0  enabled  OK
        nbdime-jupyterlab v2.0.0  enabled  OK
        plotlywidget v1.5.4  enabled  OK
        qgrid2 v1.1.3  enabled  OK
Troubleshoot Output
$PATH:
	/opt/conda/bin
	/usr/local/sbin
	/usr/local/bin
	/usr/sbin
	/usr/bin
	/sbin
	/bin

sys.path:
/opt/conda/bin
/opt/conda/lib/python37.zip
/opt/conda/lib/python3.7
/opt/conda/lib/python3.7/lib-dynload
/opt/conda/lib/python3.7/site-packages

sys.executable:
/opt/conda/bin/python

sys.version:
3.7.6 | packaged by conda-forge | (default, Mar 5 2020, 15:27:18)
[GCC 7.3.0]

platform.platform():
Linux-5.4.0-7626-generic-x86_64-with-debian-buster-sid

which -a jupyter:
/opt/conda/bin/jupyter

pip list:
Package Version
-------------------------- -------------------
alembic 1.4.1
asn1crypto 1.3.0
astroid 2.4.1
async-generator 1.10
attrs 19.3.0
autopep8 1.5.2
azure-common 1.1.25
azure-storage-blob 1.5.0
azure-storage-common 1.4.2
backcall 0.1.0
beautifulsoup4 4.8.2
bleach 3.1.3
blinker 1.4
bokeh 1.4.0
boto3 1.13.12
botocore 1.16.12
certifi 2020.4.5.1
certipy 0.1.3
cffi 1.13.2
chardet 3.0.4
click 7.1.1
cloudpickle 1.3.0
colorama 0.4.3
conda 4.8.2
conda-package-handling 1.6.0
convertdate 2.2.1
cooked-input 0.5.2
coverage 5.1
cryptography 2.8
cycler 0.10.0
Cython 0.29.15
cytoolz 0.10.1
dask 2.11.0
dateparser 0.7.4
decorator 4.4.2
defusedxml 0.6.0
dill 0.3.1.1
distributed 2.12.0
docutils 0.15.2
entrypoints 0.3
enum34 1.1.10
et-xmlfile 1.0.1
flake8 3.8.1
Flask 1.1.2
Flask-Login 0.5.0
Flask-Mail 0.9.1
Flask-Migrate 2.5.3
Flask-Principal 0.4.0
Flask-SQLAlchemy 2.4.1
fsspec 0.6.2
future 0.18.2
gitdb 4.0.5
GitPython 3.1.2
gunicorn 20.0.4
h5py 2.10.0
HeapDict 1.0.1
holidays 0.10.2
idna 2.9
ijson 2.6.1
imageio 2.8.0
importlib-metadata 1.5.0
inflection 0.4.0
ipykernel 5.1.4
ipympl 0.5.3
ipython 7.13.0
ipython-genutils 0.2.0
ipython-sql 0.3.9
ipywidgets 7.5.1
isort 4.3.21
itsdangerous 1.1.0
jdcal 1.4.1
jedi 0.17.0
Jinja2 2.11.1
jmespath 0.10.0
joblib 0.14.1
json5 0.9.0
jsonschema 3.2.0
jupyter-client 6.0.0
jupyter-core 4.6.3
jupyter-lsp 0.8.0
jupyter-telemetry 0.0.5
jupyterhub 1.1.0
jupyterlab 2.1.0
jupyterlab-git 0.20.0
jupyterlab-server 1.1.4
kiwisolver 1.1.0
knowledge-repo 0.8.8
korean-lunar-calendar 0.2.1
lazy-object-proxy 1.4.3
llvmlite 0.31.0
locket 0.2.0
Mako 1.1.0
Markdown 3.2.2
MarkupSafe 1.1.1
matplotlib 3.2.1
mccabe 0.6.1
mistune 0.8.4
mpmath 1.1.0
msgpack 1.0.0
nbconvert 5.6.1
nbdime 2.0.0
nbformat 5.0.4
networkx 2.4
nose 1.3.7
notebook 6.0.3
numba 0.48.0
numexpr 2.7.1
numpy 1.18.1
oauthlib 3.0.1
olefile 0.46
openpyxl 3.0.3
oscrypto 1.2.0
packaging 20.1
pamela 1.0.0
pandas 1.0.2
pandocfilters 1.4.2
parso 0.7.0
partd 1.1.0
patsy 0.5.1
pexpect 4.8.0
pickleshare 0.7.5
Pillow 7.0.0
pip 20.0.2
plotly 4.5.4
pluggy 0.13.1
prettytable 0.7.2
prometheus-client 0.7.1
prompt-toolkit 3.0.4
protobuf 3.11.4
psutil 5.7.0
ptyprocess 0.6.0
pycodestyle 2.6.0
pycosat 0.6.3
pycparser 2.20
pycryptodomex 3.9.7
pycurl 7.43.0.5
pydocstyle 5.0.2
pyflakes 2.2.0
Pygments 2.6.1
PyJWT 1.7.1
pylint 2.5.2
PyMeeus 0.3.7
pyOpenSSL 19.1.0
pyparsing 2.4.6
PyPDF2 1.26.0
pyrsistent 0.15.7
PySocks 1.7.1
python-dateutil 2.8.1
python-editor 1.0.4
python-json-logger 0.1.11
python-jsonrpc-server 0.3.4
python-language-server 0.32.0
pytz 2019.3
PyWavelets 1.1.1
PyYAML 5.3
pyzmq 19.0.0
qgrid 1.3.1
regex 2020.5.14
requests 2.23.0
requests-oauthlib 1.3.0
retrying 1.3.3
rope 0.17.0
ruamel-yaml 0.15.80
ruamel.yaml 0.16.6
ruamel.yaml.clib 0.2.0
s3transfer 0.3.3
scikit-image 0.16.2
scikit-learn 0.22.2.post1
scipy 1.4.1
seaborn 0.10.0
Send2Trash 1.5.0
setuptools 46.0.0.post20200311
setuptools-git 1.2
six 1.14.0
smmap 3.0.4
snowballstemmer 2.0.0
snowflake-connector-python 2.2.6
snowflake-sqlalchemy 1.2.2
sortedcontainers 2.1.0
soupsieve 1.9.4
SQLAlchemy 1.3.15
sqlparse 0.3.1
statsmodels 0.11.1
sympy 1.5.1
tabulate 0.8.7
tblib 1.6.0
termcolor 1.1.0
terminado 0.8.3
testpath 0.4.4
toml 0.10.1
toolz 0.10.0
tornado 6.0.4
tqdm 4.43.0
traitlets 4.3.3
typed-ast 1.4.1
tzlocal 2.1
ujson 1.35
urllib3 1.25.7
validus 0.3.0
veryprettytable 0.8.1
vincent 0.4.4
Wand 0.6.1
wcwidth 0.1.8
webencodings 0.5.1
Werkzeug 1.0.1
wheel 0.34.2
widgetsnbextension 3.5.1
wrapt 1.12.1
xlrd 1.2.0
yapf 0.30.0
zict 2.0.0
zipp 3.1.0

conda list:
# packages in environment at /opt/conda:
#
# Name Version Build Channel
_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 0_gnu conda-forge
alembic 1.4.1 py_0 conda-forge
asn1crypto 1.3.0 pypi_0 pypi
astroid 2.4.1 pypi_0 pypi
async_generator 1.10 py_0 conda-forge
attrs 19.3.0 py_0 conda-forge
autopep8 1.5.2 pypi_0 pypi
azure-common 1.1.25 pypi_0 pypi
azure-storage-blob 1.5.0 pypi_0 pypi
azure-storage-common 1.4.2 pypi_0 pypi
backcall 0.1.0 py_0 conda-forge
beautifulsoup4 4.8.2 py37_0 conda-forge
blas 2.14 openblas conda-forge
bleach 3.1.3 pyh8c360ce_0 conda-forge
blinker 1.4 py_1 conda-forge
bokeh 1.4.0 py37_0 conda-forge
boto3 1.13.12 pypi_0 pypi
botocore 1.16.12 pypi_0 pypi
ca-certificates 2020.4.5.1 hecc5488_0 conda-forge
certifi 2020.4.5.1 py37hc8dfbb8_0 conda-forge
certipy 0.1.3 py_0 conda-forge
cffi 1.13.2 pypi_0 pypi
chardet 3.0.4 py37hc8dfbb8_1006 conda-forge
click 7.1.1 pyh8c360ce_0 conda-forge
cloudpickle 1.3.0 py_0 conda-forge
colorama 0.4.3 pypi_0 pypi
conda 4.8.2 py37_0 conda-forge
conda-package-handling 1.6.0 py37h8f50634_2 conda-forge
configurable-http-proxy 4.2.0 node13_he01fd0c_2 conda-forge
convertdate 2.2.1 pypi_0 pypi
cooked-input 0.5.2 pypi_0 pypi
coverage 5.1 pypi_0 pypi
cryptography 2.8 py37hb09aad4_2 conda-forge
cycler 0.10.0 py_2 conda-forge
cython 0.29.15 py37h3340039_1 conda-forge
cytoolz 0.10.1 py37h516909a_0 conda-forge
dask 2.11.0 py_0 conda-forge
dask-core 2.11.0 py_0 conda-forge
dateparser 0.7.4 pypi_0 pypi
decorator 4.4.2 py_0 conda-forge
defusedxml 0.6.0 py_0 conda-forge
dill 0.3.1.1 py37hc8dfbb8_1 conda-forge
distributed 2.12.0 py37_0 conda-forge
docutils 0.15.2 pypi_0 pypi
entrypoints 0.3 py37hc8dfbb8_1001 conda-forge
enum34 1.1.10 pypi_0 pypi
et-xmlfile 1.0.1 pypi_0 pypi
flake8 3.8.1 pypi_0 pypi
flask 1.1.2 pypi_0 pypi
flask-login 0.5.0 pypi_0 pypi
flask-mail 0.9.1 pypi_0 pypi
flask-migrate 2.5.3 pypi_0 pypi
flask-principal 0.4.0 pypi_0 pypi
flask-sqlalchemy 2.4.1 pypi_0 pypi
freetype 2.10.0 he983fc9_1 conda-forge
fsspec 0.6.2 py_0 conda-forge
future 0.18.2 pypi_0 pypi
giflib 5.2.1 h516909a_2 conda-forge
gitdb 4.0.5 pypi_0 pypi
gitpython 3.1.2 pypi_0 pypi
gunicorn 20.0.4 pypi_0 pypi
h5py 2.10.0 nompi_py37h513d04c_102 conda-forge
hdf5 1.10.5 nompi_h3c11f04_1104 conda-forge
heapdict 1.0.1 py_0 conda-forge
holidays 0.10.2 pypi_0 pypi
icu 64.2 he1b5a44_1 conda-forge
idna 2.9 py_1 conda-forge
ijson 2.6.1 pypi_0 pypi
imageio 2.8.0 py_0 conda-forge
importlib-metadata 1.5.0 py37hc8dfbb8_1 conda-forge
importlib_metadata 1.5.0 1 conda-forge
inflection 0.4.0 pypi_0 pypi
ipykernel 5.1.4 py37h5ca1d4c_0 conda-forge
ipympl 0.5.3 py_0 conda-forge
ipython 7.13.0 py37h5ca1d4c_0 conda-forge
ipython-sql 0.3.9 pypi_0 pypi
ipython_genutils 0.2.0 py_1 conda-forge
ipywidgets 7.5.1 py_0 conda-forge
isort 4.3.21 pypi_0 pypi
itsdangerous 1.1.0 pypi_0 pypi
jdcal 1.4.1 pypi_0 pypi
jedi 0.17.0 pypi_0 pypi
jinja2 2.11.1 py_0 conda-forge
jmespath 0.10.0 pypi_0 pypi
joblib 0.14.1 py_0 conda-forge
jpeg 9c h14c3975_1001 conda-forge
json5 0.9.0 py_0 conda-forge
jsonschema 3.2.0 py37hc8dfbb8_1 conda-forge
jupyter-lsp 0.8.0 pypi_0 pypi
jupyter_client 6.0.0 py_0 conda-forge
jupyter_core 4.6.3 py37hc8dfbb8_1 conda-forge
jupyter_telemetry 0.0.5 py_0 conda-forge
jupyterhub 1.1.0 py37_2 conda-forge
jupyterhub-base 1.1.0 py37_2 conda-forge
jupyterlab 2.1.0 pypi_0 pypi
jupyterlab-git 0.20.0 pypi_0 pypi
jupyterlab-server 1.1.4 pypi_0 pypi
kiwisolver 1.1.0 py37h99015e2_1 conda-forge
knowledge-repo 0.8.8 pypi_0 pypi
korean-lunar-calendar 0.2.1 pypi_0 pypi
krb5 1.16.4 h2fd8d38_0 conda-forge
lazy-object-proxy 1.4.3 pypi_0 pypi
ld_impl_linux-64 2.33.1 h53a641e_8 conda-forge
libblas 3.8.0 14_openblas conda-forge
libcblas 3.8.0 14_openblas conda-forge
libcurl 7.68.0 hda55be3_0 conda-forge
libedit 3.1.20170329 hf8c457e_1001 conda-forge
libffi 3.2.1 he1b5a44_1006 conda-forge
libgcc-ng 9.2.0 h24d8f2e_2 conda-forge
libgfortran-ng 7.3.0 hdf63c60_5 conda-forge
libgomp 9.2.0 h24d8f2e_2 conda-forge
liblapack 3.8.0 14_openblas conda-forge
liblapacke 3.8.0 14_openblas conda-forge
libllvm8 8.0.1 hc9558a2_0 conda-forge
libopenblas 0.3.7 h5ec1e0e_6 conda-forge
libpng 1.6.37 hed695b0_0 conda-forge
libprotobuf 3.11.4 h8b12597_0 conda-forge
libsodium 1.0.17 h516909a_0 conda-forge
libssh2 1.8.2 h22169c7_2 conda-forge
libstdcxx-ng 9.2.0 hdf63c60_2 conda-forge
libtiff 4.1.0 hc7e4089_5 conda-forge
libuv 1.34.0 h516909a_0 conda-forge
libwebp 1.1.0 h56121f0_2 conda-forge
libwebp-base 1.1.0 2 conda-forge
llvmlite 0.31.0 py37h5202443_1 conda-forge
locket 0.2.0 py_2 conda-forge
lz4-c 1.8.3 he1b5a44_1001 conda-forge
mako 1.1.0 py_0 conda-forge
markdown 3.2.2 pypi_0 pypi
markupsafe 1.1.1 py37h8f50634_1 conda-forge
matplotlib 3.2.1 pypi_0 pypi
mccabe 0.6.1 pypi_0 pypi
mistune 0.8.4 py37h516909a_1000 conda-forge
mpmath 1.1.0 py_0 conda-forge
msgpack-python 1.0.0 py37h99015e2_1 conda-forge
nbconvert 5.6.1 py37_0 conda-forge
nbdime 2.0.0 pypi_0 pypi
nbformat 5.0.4 py_0 conda-forge
ncurses 6.1 hf484d3e_1002 conda-forge
networkx 2.4 py_1 conda-forge
nodejs 13.10.1 hf5d1a2b_0 conda-forge
nose 1.3.7 pypi_0 pypi
notebook 6.0.3 py37_0 conda-forge
numba 0.48.0 py37hb3f55d8_0 conda-forge
numexpr 2.7.1 py37hb3f55d8_0 conda-forge
numpy 1.18.1 py37h95a1406_0 conda-forge
oauthlib 3.0.1 py_0 conda-forge
olefile 0.46 py_0 conda-forge
openpyxl 3.0.3 pypi_0 pypi
openssl 1.1.1g h516909a_0 conda-forge
oscrypto 1.2.0 pypi_0 pypi
packaging 20.1 py_0 conda-forge
pamela 1.0.0 py_0 conda-forge
pandas 1.0.2 py37h0da4684_0 conda-forge
pandoc 2.9.2 0 conda-forge
pandocfilters 1.4.2 py_1 conda-forge
parso 0.7.0 pypi_0 pypi
partd 1.1.0 py_0 conda-forge
patsy 0.5.1 py_0 conda-forge
pexpect 4.8.0 py37hc8dfbb8_1 conda-forge
pickleshare 0.7.5 py37hc8dfbb8_1001 conda-forge
pillow 7.0.0 py37h718be6c_1 conda-forge
pip 20.0.2 py_2 conda-forge
plotly 4.5.4 pypi_0 pypi
pluggy 0.13.1 pypi_0 pypi
prettytable 0.7.2 pypi_0 pypi
prometheus_client 0.7.1 py_0 conda-forge
prompt-toolkit 3.0.4 py_0 conda-forge
prompt_toolkit 3.0.4 0 conda-forge
protobuf 3.11.4 py37he1b5a44_0 conda-forge
psutil 5.7.0 py37h8f50634_1 conda-forge
ptyprocess 0.6.0 py_1001 conda-forge
pycodestyle 2.6.0 pypi_0 pypi
pycosat 0.6.3 py37h8f50634_1004 conda-forge
pycparser 2.20 py_0 conda-forge
pycryptodomex 3.9.7 pypi_0 pypi
pycurl 7.43.0.5 py37h16ce93b_0 conda-forge
pydocstyle 5.0.2 pypi_0 pypi
pyflakes 2.2.0 pypi_0 pypi
pygments 2.6.1 py_0 conda-forge
pyjwt 1.7.1 py_0 conda-forge
pylint 2.5.2 pypi_0 pypi
pymeeus 0.3.7 pypi_0 pypi
pyopenssl 19.1.0 py_1 conda-forge
pyparsing 2.4.6 py_0 conda-forge
pypdf2 1.26.0 pypi_0 pypi
pyrsistent 0.15.7 py37h8f50634_1 conda-forge
pysocks 1.7.1 py37hc8dfbb8_1 conda-forge
python 3.7.6 h357f687_4_cpython conda-forge
python-dateutil 2.8.1 py_0 conda-forge
python-editor 1.0.4 py_0 conda-forge
python-json-logger 0.1.11 py_0 conda-forge
python-jsonrpc-server 0.3.4 pypi_0 pypi
python-language-server 0.32.0 pypi_0 pypi
python_abi 3.7 1_cp37m conda-forge
pytz 2019.3 py_0 conda-forge
pywavelets 1.1.1 py37hc1659b7_0 conda-forge
pyyaml 5.3 py37h8f50634_1 conda-forge
pyzmq 19.0.0 py37hac76be4_1 conda-forge
qgrid 1.3.1 pypi_0 pypi
readline 8.0 hf8c457e_0 conda-forge
regex 2020.5.14 pypi_0 pypi
requests 2.23.0 pyh8c360ce_2 conda-forge
requests-oauthlib 1.3.0 pypi_0 pypi
retrying 1.3.3 pypi_0 pypi
rope 0.17.0 pypi_0 pypi
ruamel.yaml 0.16.6 py37h8f50634_1 conda-forge
ruamel.yaml.clib 0.2.0 py37h8f50634_1 conda-forge
ruamel_yaml 0.15.80 py37h8f50634_1001 conda-forge
s3transfer 0.3.3 pypi_0 pypi
scikit-image 0.16.2 py37hb3f55d8_0 conda-forge
scikit-learn 0.22.2.post1 py37hcdab131_0 conda-forge
scipy 1.4.1 py37h921218d_0 conda-forge
seaborn 0.10.0 py_1 conda-forge
send2trash 1.5.0 py_0 conda-forge
setuptools 46.0.0 py37hc8dfbb8_2 conda-forge
setuptools-git 1.2 pypi_0 pypi
six 1.14.0 py_1 conda-forge
smmap 3.0.4 pypi_0 pypi
snowballstemmer 2.0.0 pypi_0 pypi
snowflake-connector-python 2.2.6 pypi_0 pypi
snowflake-sqlalchemy 1.2.2 pypi_0 pypi
sortedcontainers 2.1.0 py_0 conda-forge
soupsieve 1.9.4 py37hc8dfbb8_1 conda-forge
sqlalchemy 1.3.15 py37h8f50634_1 conda-forge
sqlite 3.30.1 hcee41ef_0 conda-forge
sqlparse 0.3.1 pypi_0 pypi
statsmodels 0.11.1 py37h8f50634_1 conda-forge
sympy 1.5.1 py37hc8dfbb8_2 conda-forge
tabulate 0.8.7 pypi_0 pypi
tblib 1.6.0 py_0 conda-forge
termcolor 1.1.0 pypi_0 pypi
terminado 0.8.3 py37hc8dfbb8_1 conda-forge
testpath 0.4.4 py_0 conda-forge
tini 0.18.0 h14c3975_1001 conda-forge
tk 8.6.10 hed695b0_0 conda-forge
toml 0.10.1 pypi_0 pypi
toolz 0.10.0 py_0 conda-forge
tornado 6.0.4 py37h8f50634_1 conda-forge
tqdm 4.43.0 py_0 conda-forge
traitlets 4.3.3 py37hc8dfbb8_1 conda-forge
typed-ast 1.4.1 pypi_0 pypi
tzlocal 2.1 pypi_0 pypi
ujson 1.35 pypi_0 pypi
urllib3 1.25.7 py37hc8dfbb8_1 conda-forge
validus 0.3.0 pypi_0 pypi
veryprettytable 0.8.1 pypi_0 pypi
vincent 0.4.4 py_1 conda-forge
wand 0.6.1 pypi_0 pypi
wcwidth 0.1.8 py_0 conda-forge
webencodings 0.5.1 py_1 conda-forge
werkzeug 1.0.1 pypi_0 pypi
wheel 0.34.2 py_1 conda-forge
widgetsnbextension 3.5.1 py37_0 conda-forge
wrapt 1.12.1 pypi_0 pypi
xlrd 1.2.0 py_0 conda-forge
xz 5.2.4 h14c3975_1001 conda-forge
yaml 0.2.2 h516909a_1 conda-forge
yapf 0.30.0 pypi_0 pypi
zeromq 4.3.2 he1b5a44_2 conda-forge
zict 2.0.0 py_0 conda-forge
zipp 3.1.0 py_0 conda-forge
zlib 1.2.11 h516909a_1006 conda-forge
zstd 1.4.4 h3b9ef0a_1 conda-forge

Command Line Output
[D 19:50:19.900 LabApp] Searching ['/home/jovyan/.jupyter', '/home/jovyan/.jupyter', '/opt/conda/etc/jupyter', '/usr/local/etc/jupyter', '/etc/jupyter'] for config files
[D 19:50:19.900 LabApp] Looking for jupyter_config in /etc/jupyter
[D 19:50:19.900 LabApp] Looking for jupyter_config in /usr/local/etc/jupyter
[D 19:50:19.901 LabApp] Looking for jupyter_config in /opt/conda/etc/jupyter
[D 19:50:19.901 LabApp] Looking for jupyter_config in /home/jovyan/.jupyter
[D 19:50:19.901 LabApp] Looking for jupyter_config in /home/jovyan/.jupyter
[D 19:50:19.904 LabApp] Looking for jupyter_notebook_config in /etc/jupyter
[D 19:50:19.904 LabApp] Loaded config file: /etc/jupyter/jupyter_notebook_config.py
[D 19:50:19.905 LabApp] Looking for jupyter_notebook_config in /usr/local/etc/jupyter
[D 19:50:19.905 LabApp] Looking for jupyter_notebook_config in /opt/conda/etc/jupyter
[D 19:50:19.905 LabApp] Looking for jupyter_notebook_config in /home/jovyan/.jupyter
[D 19:50:19.906 LabApp] Loaded config file: /home/jovyan/.jupyter/jupyter_notebook_config.py
[D 19:50:19.906 LabApp] Loaded config file: /home/jovyan/.jupyter/jupyter_notebook_config.json
[D 19:50:19.906 LabApp] Looking for jupyter_notebook_config in /home/jovyan/.jupyter
[D 19:50:19.906 LabApp] Loaded config file: /home/jovyan/.jupyter/jupyter_notebook_config.py
[D 19:50:19.907 LabApp] Loaded config file: /home/jovyan/.jupyter/jupyter_notebook_config.json
[D 19:50:19.919 LabApp] Paths used for configuration of jupyter_notebook_config: 
        /etc/jupyter/jupyter_notebook_config.json
[D 19:50:19.920 LabApp] Paths used for configuration of jupyter_notebook_config: 
        /usr/local/etc/jupyter/jupyter_notebook_config.json
[D 19:50:19.921 LabApp] Paths used for configuration of jupyter_notebook_config: 
        /opt/conda/etc/jupyter/jupyter_notebook_config.d/jupyter-lsp-serverextension.json
        /opt/conda/etc/jupyter/jupyter_notebook_config.d/jupyterlab.json
        /opt/conda/etc/jupyter/jupyter_notebook_config.d/jupyterlab_git.json
        /opt/conda/etc/jupyter/jupyter_notebook_config.d/nbdime.json
        /opt/conda/etc/jupyter/jupyter_notebook_config.json
[D 19:50:19.922 LabApp] Paths used for configuration of jupyter_notebook_config: 
        /home/jovyan/.jupyter/jupyter_notebook_config.json
[I 19:50:20.255 LabApp] The port 8888 is already in use, trying another port.
[D 19:50:20.312 LabApp] Checking for /home/jovyan/.jupyter/node_modules/bash-language-server/bin/main.js
[D 19:50:20.312 LabApp] Checking for /opt/conda/share/jupyter/lab/staging/node_modules/bash-language-server/bin/main.js
[D 19:50:20.313 LabApp] Checking for /opt/conda/lib/node_modules/bash-language-server/bin/main.js
[D 19:50:20.313 LabApp] Checking for /opt/conda/node_modules/bash-language-server/bin/main.js
[D 19:50:20.313 LabApp] bash-language-server/bin/main.js not found in node_modules of [PosixPath('/home/jovyan/.jupyter'), PosixPath('/opt/conda/share/jupyter/lab/staging'), PosixPath('/opt/conda/lib'), PosixPath('/opt/conda')]
[D 19:50:20.313 LabApp] Checking for /home/jovyan/.jupyter/node_modules/dockerfile-language-server-nodejs/lib/server.js
[D 19:50:20.313 LabApp] Checking for /opt/conda/share/jupyter/lab/staging/node_modules/dockerfile-language-server-nodejs/lib/server.js
[D 19:50:20.313 LabApp] Checking for /opt/conda/lib/node_modules/dockerfile-language-server-nodejs/lib/server.js
[D 19:50:20.314 LabApp] Checking for /opt/conda/node_modules/dockerfile-language-server-nodejs/lib/server.js
[D 19:50:20.314 LabApp] dockerfile-language-server-nodejs/lib/server.js not found in node_modules of [PosixPath('/home/jovyan/.jupyter'), PosixPath('/opt/conda/share/jupyter/lab/staging'), PosixPath('/opt/conda/lib'), PosixPath('/opt/conda')]
[D 19:50:20.314 LabApp] Checking for /home/jovyan/.jupyter/node_modules/javascript-typescript-langserver/lib/language-server-stdio.js
[D 19:50:20.314 LabApp] Checking for /opt/conda/share/jupyter/lab/staging/node_modules/javascript-typescript-langserver/lib/language-server-stdio.js
[D 19:50:20.314 LabApp] Checking for /opt/conda/lib/node_modules/javascript-typescript-langserver/lib/language-server-stdio.js
[D 19:50:20.315 LabApp] Checking for /opt/conda/node_modules/javascript-typescript-langserver/lib/language-server-stdio.js
[D 19:50:20.315 LabApp] javascript-typescript-langserver/lib/language-server-stdio.js not found in node_modules of [PosixPath('/home/jovyan/.jupyter'), PosixPath('/opt/conda/share/jupyter/lab/staging'), PosixPath('/opt/conda/lib'), PosixPath('/opt/conda')]
[D 19:50:20.318 LabApp] Checking for /home/jovyan/.jupyter/node_modules/unified-language-server/src/server.js
[D 19:50:20.318 LabApp] Checking for /opt/conda/share/jupyter/lab/staging/node_modules/unified-language-server/src/server.js
[D 19:50:20.319 LabApp] Checking for /opt/conda/lib/node_modules/unified-language-server/src/server.js
[D 19:50:20.319 LabApp] Checking for /opt/conda/node_modules/unified-language-server/src/server.js
[D 19:50:20.319 LabApp] unified-language-server/src/server.js not found in node_modules of [PosixPath('/home/jovyan/.jupyter'), PosixPath('/opt/conda/share/jupyter/lab/staging'), PosixPath('/opt/conda/lib'), PosixPath('/opt/conda')]
[D 19:50:20.319 LabApp] Checking for /home/jovyan/.jupyter/node_modules/vscode-css-languageserver-bin/cssServerMain.js
[D 19:50:20.319 LabApp] Checking for /opt/conda/share/jupyter/lab/staging/node_modules/vscode-css-languageserver-bin/cssServerMain.js
[D 19:50:20.320 LabApp] Checking for /opt/conda/lib/node_modules/vscode-css-languageserver-bin/cssServerMain.js
[D 19:50:20.320 LabApp] Checking for /opt/conda/node_modules/vscode-css-languageserver-bin/cssServerMain.js
[D 19:50:20.320 LabApp] vscode-css-languageserver-bin/cssServerMain.js not found in node_modules of [PosixPath('/home/jovyan/.jupyter'), PosixPath('/opt/conda/share/jupyter/lab/staging'), PosixPath('/opt/conda/lib'), PosixPath('/opt/conda')]
[D 19:50:20.320 LabApp] Checking for /home/jovyan/.jupyter/node_modules/vscode-html-languageserver-bin/htmlServerMain.js
[D 19:50:20.320 LabApp] Checking for /opt/conda/share/jupyter/lab/staging/node_modules/vscode-html-languageserver-bin/htmlServerMain.js
[D 19:50:20.320 LabApp] Checking for /opt/conda/lib/node_modules/vscode-html-languageserver-bin/htmlServerMain.js
[D 19:50:20.321 LabApp] Checking for /opt/conda/node_modules/vscode-html-languageserver-bin/htmlServerMain.js
[D 19:50:20.321 LabApp] vscode-html-languageserver-bin/htmlServerMain.js not found in node_modules of [PosixPath('/home/jovyan/.jupyter'), PosixPath('/opt/conda/share/jupyter/lab/staging'), PosixPath('/opt/conda/lib'), PosixPath('/opt/conda')]
[D 19:50:20.321 LabApp] Checking for /home/jovyan/.jupyter/node_modules/vscode-json-languageserver-bin/jsonServerMain.js
[D 19:50:20.321 LabApp] Checking for /opt/conda/share/jupyter/lab/staging/node_modules/vscode-json-languageserver-bin/jsonServerMain.js
[D 19:50:20.321 LabApp] Checking for /opt/conda/lib/node_modules/vscode-json-languageserver-bin/jsonServerMain.js
[D 19:50:20.321 LabApp] Checking for /opt/conda/node_modules/vscode-json-languageserver-bin/jsonServerMain.js
[D 19:50:20.322 LabApp] vscode-json-languageserver-bin/jsonServerMain.js not found in node_modules of [PosixPath('/home/jovyan/.jupyter'), PosixPath('/opt/conda/share/jupyter/lab/staging'), PosixPath('/opt/conda/lib'), PosixPath('/opt/conda')]
[D 19:50:20.322 LabApp] Checking for /home/jovyan/.jupyter/node_modules/yaml-language-server/bin/yaml-language-server
[D 19:50:20.322 LabApp] Checking for /opt/conda/share/jupyter/lab/staging/node_modules/yaml-language-server/bin/yaml-language-server
[D 19:50:20.322 LabApp] Checking for /opt/conda/lib/node_modules/yaml-language-server/bin/yaml-language-server
[D 19:50:20.322 LabApp] Checking for /opt/conda/node_modules/yaml-language-server/bin/yaml-language-server
[D 19:50:20.323 LabApp] yaml-language-server/bin/yaml-language-server not found in node_modules of [PosixPath('/home/jovyan/.jupyter'), PosixPath('/opt/conda/share/jupyter/lab/staging'), PosixPath('/opt/conda/lib'), PosixPath('/opt/conda')]
[D 19:50:20.440 LabApp] [lsp] rootUri will be file:///home/jovyan/.jupyter
[D 19:50:20.440 LabApp] [lsp] virtualDocumentsUri will be file:///home/jovyan/.jupyter/.virtual_documents
[D 19:50:20.442 LabApp] [lsp] The following Language Servers will be available: {
      "pyls": {
        "argv": [
          "/opt/conda/bin/pyls"
        ],
        "config_schema": {
          "properties": {
            "pyls.configurationSources": {
              "default": [
                "pycodestyle"
              ],
              "description": "List of configuration sources to use.",
              "items": {
                "enum": [
                  "pycodestyle",
                  "pyflakes"
                ],
                "type": "string"
              },
              "type": "array",
              "uniqueItems": true
            },
            "pyls.executable": {
              "default": "pyls",
              "description": "Language server executable",
              "type": "string"
            },
            "pyls.plugins.jedi_completion.enabled": {
              "default": true,
              "description": "Enable or disable the plugin.",
              "type": "boolean"
            },
            "pyls.plugins.jedi_completion.include_params": {
              "default": true,
              "description": "Auto-completes methods and classes with tabstops for each parameter.",
              "type": "boolean"
            },
            "pyls.plugins.jedi_definition.enabled": {
              "default": true,
              "description": "Enable or disable the plugin.",
              "type": "boolean"
            },
            "pyls.plugins.jedi_definition.follow_builtin_imports": {
              "default": true,
              "description": "If follow_imports is True will decide if it follow builtin imports.",
              "type": "boolean"
            },
            "pyls.plugins.jedi_definition.follow_imports": {
              "default": true,
              "description": "The goto call will follow imports.",
              "type": "boolean"
            },
            "pyls.plugins.jedi_hover.enabled": {
              "default": true,
              "description": "Enable or disable the plugin.",
              "type": "boolean"
            },
            "pyls.plugins.jedi_references.enabled": {
              "default": true,
              "description": "Enable or disable the plugin.",
              "type": "boolean"
            },
            "pyls.plugins.jedi_signature_help.enabled": {
              "default": true,
              "description": "Enable or disable the plugin.",
              "type": "boolean"
            },
            "pyls.plugins.jedi_symbols.all_scopes": {
              "default": true,
              "description": "If True lists the names of all scopes instead of only the module namespace.",
              "type": "boolean"
            },
            "pyls.plugins.jedi_symbols.enabled": {
              "default": true,
              "description": "Enable or disable the plugin.",
              "type": "boolean"
            },
            "pyls.plugins.mccabe.enabled": {
              "default": true,
              "description": "Enable or disable the plugin.",
              "type": "boolean"
            },
            "pyls.plugins.mccabe.threshold": {
              "default": 15,
              "description": "The minimum threshold that triggers warnings about cyclomatic complexity.",
              "type": "number"
            },
            "pyls.plugins.preload.enabled": {
              "default": true,
              "description": "Enable or disable the plugin.",
              "type": "boolean"
            },
            "pyls.plugins.preload.modules": {
              "default": null,
              "description": "List of modules to import on startup",
              "items": {
                "type": "string"
              },
              "type": "array",
              "uniqueItems": true
            },
            "pyls.plugins.pycodestyle.enabled": {
              "default": true,
              "description": "Enable or disable the plugin.",
              "type": "boolean"
            },
            "pyls.plugins.pycodestyle.exclude": {
              "default": null,
              "description": "Exclude files or directories which match these patterns.",
              "items": {
                "type": "string"
              },
              "type": "array",
              "uniqueItems": true
            },
            "pyls.plugins.pycodestyle.filename": {
              "default": null,
              "description": "When parsing directories, only check filenames matching these patterns.",
              "items": {
                "type": "string"
              },
              "type": "array",
              "uniqueItems": true
            },
            "pyls.plugins.pycodestyle.hangClosing": {
              "default": null,
              "description": "Hang closing bracket instead of matching indentation of opening bracket's line.",
              "type": "boolean"
            },
            "pyls.plugins.pycodestyle.ignore": {
              "default": null,
              "description": "Ignore errors and warnings",
              "items": {
                "type": "string"
              },
              "type": "array",
              "uniqueItems": true
            },
            "pyls.plugins.pycodestyle.maxLineLength": {
              "default": null,
              "description": "Set maximum allowed line length.",
              "type": "number"
            },
            "pyls.plugins.pycodestyle.select": {
              "default": null,
              "description": "Select errors and warnings",
              "items": {
                "type": "string"
              },
              "type": "array",
              "uniqueItems": true
            },
            "pyls.plugins.pydocstyle.addIgnore": {
              "default": null,
              "description": "Ignore errors and warnings in addition to the specified convention.",
              "items": {
                "type": "string"
              },
              "type": "array",
              "uniqueItems": true
            },
            "pyls.plugins.pydocstyle.addSelect": {
              "default": null,
              "description": "Select errors and warnings in addition to the specified convention.",
              "items": {
                "type": "string"
              },
              "type": "array",
              "uniqueItems": true
            },
            "pyls.plugins.pydocstyle.convention": {
              "default": null,
              "description": "Choose the basic list of checked errors by specifying an existing convention.",
              "enum": [
                "pep257",
                "numpy"
              ],
              "type": "string"
            },
            "pyls.plugins.pydocstyle.enabled": {
              "default": false,
              "description": "Enable or disable the plugin.",
              "type": "boolean"
            },
            "pyls.plugins.pydocstyle.ignore": {
              "default": null,
              "description": "Ignore errors and warnings",
              "items": {
                "type": "string"
              },
              "type": "array",
              "uniqueItems": true
            },
            "pyls.plugins.pydocstyle.match": {
              "default": "(?!test_).*\\.py",
              "description": "Check only files that exactly match the given regular expression; default is to match files that don't start with 'test_' but end with '.py'.",
              "type": "string"
            },
            "pyls.plugins.pydocstyle.matchDir": {
              "default": "[^\\.].*",
              "description": "Search only dirs that exactly match the given regular expression; default is to match dirs which do not begin with a dot.",
              "type": "string"
            },
            "pyls.plugins.pydocstyle.select": {
              "default": null,
              "description": "Select errors and warnings",
              "items": {
                "type": "string"
              },
              "type": "array",
              "uniqueItems": true
            },
            "pyls.plugins.pyflakes.enabled": {
              "default": true,
              "description": "Enable or disable the plugin.",
              "type": "boolean"
            },
            "pyls.plugins.pylint.args": {
              "default": null,
              "description": "Arguments to pass to pylint.",
              "items": {
                "type": "string"
              },
              "type": "array",
              "uniqueItems": false
            },
            "pyls.plugins.pylint.enabled": {
              "default": true,
              "description": "Enable or disable the plugin.",
              "type": "boolean"
            },
            "pyls.plugins.rope_completion.enabled": {
              "default": true,
              "description": "Enable or disable the plugin.",
              "type": "boolean"
            },
            "pyls.plugins.yapf.enabled": {
              "default": true,
              "description": "Enable or disable the plugin.",
              "type": "boolean"
            },
            "pyls.rope.extensionModules": {
              "default": null,
              "description": "Builtin and c-extension modules that are allowed to be imported and inspected by rope.",
              "type": "string"
            },
            "pyls.rope.ropeFolder": {
              "default": null,
              "description": "The name of the folder in which rope stores project configurations and data.  Pass `null` for not using such a folder at all.",
              "items": {
                "type": "string"
              },
              "type": "array",
              "uniqueItems": true
            }
          },
          "title": "Python Language Server Configuration",
          "type": "object"
        },
        "display_name": "pyls",
        "env": {
          "PYTHONUNBUFFERED": "1"
        },
        "extend": [
          {
            "display_name": "pyls-mypy",
            "install": {
              "pip": "pip install pyls-mypy"
            }
          },
          {
            "display_name": "pyls-black",
            "install": {
              "pip": "pip install pyls-black"
            }
          },
          {
            "display_name": "pyls-isort",
            "install": {
              "pip": "pip install pyls-isort"
            }
          }
        ],
        "install": {
          "conda": "conda install -c conda-forge python-language-server",
          "pip": "pip install python-language-server[all]"
        },
        "languages": [
          "python"
        ],
        "mime_types": [
          "text/python",
          "text/x-ipython"
        ],
        "urls": {
          "home": "https://github.com/palantir/python-language-server",
          "issues": "https://github.com/palantir/python-language-server/issues"
        },
        "version": 2
      }
    }
[I 19:50:20.992 LabApp] JupyterLab extension loaded from /opt/conda/lib/python3.7/site-packages/jupyterlab
[I 19:50:20.992 LabApp] JupyterLab application directory is /opt/conda/share/jupyter/lab
[D 19:50:21.092 LabApp] Using default logger
[D 19:50:21.093 LabApp] Using default logger
[D 19:50:21.093 LabApp] Using default logger
[D 19:50:21.093 LabApp] Using default logger
[D 19:50:21.093 LabApp] Using default logger
[I 19:50:21.097 LabApp] Serving notebooks from local directory: /home/jovyan/.jupyter
[I 19:50:21.097 LabApp] The Jupyter Notebook is running at:
[I 19:50:21.097 LabApp] http://b832f09e7bfa:8889/?token=34c3ce50874939ad1fc2cc19e15f579329ddaf66dd9e0f90
[I 19:50:21.097 LabApp]  or http://127.0.0.1:8889/?token=34c3ce50874939ad1fc2cc19e15f579329ddaf66dd9e0f90
[I 19:50:21.097 LabApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 19:50:21.103 LabApp] 
Browser Output
console.js:34 LSP:  Will invoke completer after .
completion.js:121 [LSP][Completer] Token: {offset: 2, value: ".", type: undefined}
completion.js:189 [LSP][Completer] Merging completions: {start: 3, end: 3, matches: Array(1126), metadata: {…}} {start: 0, end: 3, matches: Array(500), metadata: {…}}
completion.js:203 [LSP][Completer] Removing kernel prefix:  np.

No errors

Thanks for help!

@ajacker
Copy link

ajacker commented May 22, 2020

me too, it usually takes long time

@krassowski
Copy link
Member

Does it take longer for you than in another editor using the same language server (e.g. vim with pyls/VS Code with pyls), or is this the same in all editors?

@jhettler
Copy link
Author

Hi @krassowski, it takes longer even in VS code with pyls, but just for the first time, if you try to write it again it is immediately autocompleted, but not in Jupyterlab. It is a bit faster in Jupyterlab after first try, but not as in VS Code.

@ajacker
Copy link

ajacker commented May 25, 2020

Hi, @krassowski ,i deploy it at my remote server (ubuntu 18.08) at the first time, it's too slow, about 10s+ for a hint. I thought it might be the network problem. But I deploy it at my PC(win10), it's still very slow(few secs), but better then my server.

@krassowski
Copy link
Member

Thank you for your feedback. I guess it might be hardware dependent but then I did see very slow completion of one laptop which had good specs but was running Windows. Maybe we could make use of caching in a way?

Please feel free to suggest your own solution ideas!

@Mbarki66
Copy link

Mbarki66 commented May 28, 2020

@krassowski First of all, great extension! Not perfect yet, but never thought all these features would be possible in jupyter notebooks!

The slow autocomplete is the only problem I encountered (on windows). I tried turning the jupyterlab-lsp extension off and as a result the autocomplete was much faster (using tab key of course). When I turn on the extension, the autocomplete becomes very slow again. Hope there is a solution to this.

@AMMJ93
Copy link

AMMJ93 commented Jun 2, 2020

Have the same issue sadly, so I understand it is not user related?

@sntgluca
Copy link

Hi,

I wish to share my experience here.
I do experience similar issues, and the following configuration is the only one I could come up with which gives a sufficient performance:

jupyter-lsp 1.0.0
jupyterlab  2.0.2  
jedi             0.15.2 
parso         0.5.2

Any upgrade to newer versions of these packages, i.e. jupyterlab==2.1.2 lead to backend errors slowing everything down, such as:

  File "<CONDA_ENV_DIR>/lib/python3.6/site-packages/parso/utils.py
", line 109, in python_bytes_to_unicode
    return unicode(source, encoding, errors)
LookupError: unknown encoding: 0

I hope this helps further investigation!

@krassowski, let me add that this extension is really useful and I am grateful it's available. Honestly, with JupyterLab it's still a bit laggy, I am sure it could get better, I hope you can continue improving it!

@aisensiy
Copy link

aisensiy commented Jul 8, 2020

I have the same issue, seems slower than the default autocomplete...

@bergkvist
Copy link

I have the same problem. Running JupyterLab on Linux here (i7 6700K), and autocomplete is taking 5-10 seconds.

@krassowski
Copy link
Member

krassowski commented Aug 29, 2020

There are several things that we can do to help:

  • implement completionItem/resolve so that the documentation does not have to be computed and sent over for every possible completion
  • on Linux use the ramdisk (when available) for the virtual documents
  • warm up jedi cache if pyls is detected (we already have a script for that in CI)
  • improve notebooks updates speed (see Virtual document updates are slow for large notebooks #336)

@krassowski
Copy link
Member

In addition to the list above (now updated to four ideas for tackling this issue), there is an upstream issue for recent pyls versions: palantir/python-language-server#823

It might be that some speed up will be seen once the upstream issue is resolved. If you are tracking this issue and would like to help but don't feel like learning typescript, head over to the python-language-server repo - it is pure Python :)

@quantum-booty
Copy link
Contributor

quantum-booty commented Sep 14, 2020

My completion is also pretty slow, on top of that dot does not trigger automatic completion. Is there a setting to switch this on? Also pressing tab does not work for dot and one char, but works for two or more characters:
np. tab = no completion
np.a tab = no completion
np.ar tab = yes completion

For matplotlib, I can autocomplete with one character:
plt. tab = no completion
plt.p = yes completion

@krassowski
Copy link
Member

krassowski commented Sep 14, 2020

@quantum-booty the continuous completion (a.k.a hinterland) is an opt-in feature and you need to enable it in the Advanced Settings Editor → Code Completion:

continues_hinting_and_docs

I suspect that you do not see auto-completion (i.e. completion just after typing dot) for np. just because there are so many candidate completions (599 when I just tried! numpy exports a huge namespace - though the 599 also includes kernel completions in it) that the request bails before the completer is fully loaded. This is a very valuable observation - thank you for the reproducible example @quantum-booty. We might wish to try to only render a subset (50?) completions at first and load more once the user went down the list, but this needs to wait until the renderer is available to extensions (JupyterLab 3.0).

Now to explain things a bit, it is not a surprise that the completions load slower with LSP:

  • we load both the kernel and LSP server completions so we are limited by whichever is slower for any particular request
  • the LSP completions carry the documentation which is heavy to transfer
    • some more advanced serves do not load documentation upfront but only after any of the completion items are activated; I plan to enable this for the servers which support it (e.g. R language server), but it has to wait until JupyterLab 3.0 is released with changes that are required to expose the completion renderer to extensions.
  • the first time the pyls (the Python server we recommend) loads completions for any specific package it has to analyse it which adds a bit overhead; the subsequent requests are faster as the underlying analyse (jedi) caches the results; however, as mentioned above there seems to be an upstream issue with the cache in the more recent versions of pyls.

Sorry that the experience is far from perfect! I do my best (with the great help of Nick!) but at the end of the day we are just doing this in our free time, and cannot move faster (although we would very much welcome any help!)

@krassowski krassowski pinned this issue Sep 14, 2020
@sntgluca
Copy link

Hi @krassowski ,
just wish to restate that your work is great and the outcome is much appreciated.

From my side, I just enjoy it and try to give feedback to make sure things get even better...

@krassowski
Copy link
Member

Thanks, @sntgluca! The feedback with a reproducible example like the one from @quantum-booty helps to narrow down what is the root cause fo the issues.

As for your comment on difficulty upgrading and parso errors - this is not something that has to do with this extension but with the pyls language server. they just had a new release hopefully fixing various errors (though there are some known issues with completion caching and diagnostics) - have a try upgrading and if you encounter any problems please do open a separate issue. I cannot help without detailed environment information.

@krassowski
Copy link
Member

I did some profiling today on the np.<tab> case, which brings up about 600 suggestions.

With the material icons enabled the action took ~3.5s, while with the icons turned off it took ~2.5s.

Icons

Screenshot from 2020-09-18 19-03-18

No icons

Screenshot from 2020-09-18 18-58-40

icons no icons
Screenshot from 2020-09-18 19-04-40 Screenshot from 2020-09-18 18-59-14

Even with icons disabled 0.77s is used for the actual rendering. The rendering code is upstream.

There are different approaches we could use:

  • only display some of the completions
  • use windowing
  • improve how icons are generated upstream (to avoid generating hundreds of SVG objects)

@krassowski
Copy link
Member

Would anyone here like to try running the jedi cache script and see if it makes a difference? Please adjust MODULES_TO_CACHE to include the modules you work with, and please let us know if this improved your experience, and which version of python-language-server and jedi you have installed.

@quantum-booty
Copy link
Contributor

quantum-booty commented Sep 19, 2020

@krassowski

default jedi environment <SameEnvironment: 3.8.5 in /home/henryw/virtualenvs/data_sci>
jedi environment <jedi.api.environment.InterpreterEnvironment object at 0x7f1236173130>
jedi cache already exists, aborting warm up!

@quantum-booty
Copy link
Contributor

quantum-booty commented Sep 22, 2020

0
@krassowski I deleted the jedi cache manually and ran the script, however it did not improve the speed. For numpy, the autocompete is still playing catch up after I finished typing np.array(). As you can see on the image, the autocompletion shows up after the documentation. It takes about 3 to 4 seconds for the autocompletion to appear. I am using the latest versions of pyls and jedi.

Regarding the icons, I can happily live without them, I'd be fine with texts inplace of the icons.

@sntgluca
Copy link

Hi @krassowski ,
I wish to continue on this thread with something borderline with the topic.

In my case I came to the conclusion that lsp autocompletion is strongly bound to I/O and networking (remote NFS + jupyterhub proxy + VPN) and I don't think there's much to do to drastically improve that...
On the other hand the "native" autocompletion is much faster, including if I attach a kernel to the editor.

I am thinking of some potential workarounds which could work for cases like mine:

  • Is it possible to disable autocompletion and let the "native" autocompletion run instead?
  • Have you been thinking of alternatives to jedi? Maybe I am too naive but could you think of starting a kernel automatically and bind the editor to that kernel?

Looking forward to your opinion

@sntgluca
Copy link

FYI,
I attempted rebuilding the cache with your script, but I only saw minor improvements (still taking 4-5 seconds to autocomplete).

Additionally I noticed a major difference with your snapshots above: in my case 'parsing' the instructions - as shown with highlighting - is really laggy and can take more than one second per statement. No error indication in the JS console. Just plain slow.

Hope it helps as a data point :)

@quantum-booty
Copy link
Contributor

quantum-booty commented Sep 23, 2020

Could disabling completion on dot improve performance? Maybe add a setting to trigger completion 1 or 2 or 3 characters after the dot. Maybe make dot trigger depending on library size, for a big library like numby, trigger 1 or 2 char after dot. For a small library, trigger completion on dot.

@sntgluca
Copy link

Hi @bollwyvl , @krassowski ,
I hope I didn't sound pushy, it wasn't my intention.
Based on your answers I agree it's not worth it and efforts should be rather spent on moving forward.

@g6ai
Copy link

g6ai commented Feb 11, 2021

@krassowski Thanks for the brilliant work! I tried https://github.com/krassowski/jupyterlab-lsp/issues/272#issuecomment-775726255 and it works well for local kernels, faster autocompletion, impressive.

However, with remote ipython kernels I installed with e.g. remote_ikernel, there's no autocompletion.

@krassowski
Copy link
Member

@g6ai please open a new bug report and provide all the details including the debug output from kernel.

@bollwyvl
Copy link
Collaborator

bollwyvl commented Feb 12, 2021

Yeah, once you go remote, the topology will get weird between what we know about, what your kernels know about, and what your language servers knows about. We're not really ready for that, partially see:

https://github.com/krassowski/jupyterlab-lsp/issues/184#issuecomment-777682465

@krassowski krassowski changed the title Slow autocompletion in Jupyterlab Slow completion for large Python packages when using the default pyls Feb 18, 2021
@krassowski krassowski added lang:python Python, pyls, etc and removed help wanted Extra attention is needed labels Feb 18, 2021
@chenshen03
Copy link

How do I turn off completion but keep the rest of jupyterlab-lsp working?

@autoscraper
Copy link

I also would like to know how do I turn off completion but keep the rest of jupyterlab-lsp working.
The completion experience have been awful. Lost many hours trying make it function properly, but always stop working at some point.

@sntgluca
Copy link

Hi @krassowski ,
I am experiencing some additional pyls configuration issues, which I consider in scope within this thread. I hope you could give me a hint.

It turns out pyls is constantly consuming more than 50% of CPU time, trying to index all python files in my home, which are plenty as I have several python files and separate environments, similarly to what is described in palantir/python-language-server#421

Do you have any configuration suggestion to limit this behaviour?

@krassowski
Copy link
Member

@sntgluca does it occur using my pyls fork, or the upstream? How do you know that it is indexing files? Is it coming from a specific pyls plugin? If this is an issue with pyls, then please open a detailed issue upstream, but chance is that it may be with a specific pyls plugin - then please open it in appropriate repository.

@sntgluca
Copy link

Dear @krassowski ,

I wanted to check with you first, maybe there was an obvious solution. Thanks for following up with me quickly anyway.
I opened a separate ticket: palantir/python-language-server#928

And to answer your questions:

I have been using strace -p $(pgrep pyls) to monitor the files accessed by pyls as described in palantir/python-language-server#421 (comment)

I can confirm the same behaviour occurs both with your fork python-language-server-0.36.2+49.g3536061 and with the official pip package python-language-server==0.36.2

I don't know which pyls plugin is involved or how to check it... sorry.

@krassowski
Copy link
Member

Thanks! FYI, we will be moving towards the https://github.com/python-lsp/python-lsp-server fork soon (once I get to send PRs that provide the performance improvements). At very least, you can check which plugins are installed using pip list | grep pyls.

@droctothorpe
Copy link

https://github.com/krassowski/jupyterlab-lsp/issues/272#issuecomment-774806496 brought my completion time for a Dask Gateway Gateway object down from 5 seconds to less than 0.5 seconds! Thank you so much!

@bollwyvl
Copy link
Collaborator

So now that all of the primary upstream pyls dependencies support python-lsp-server, perhaps we can close this?

@sntgluca
Copy link

sntgluca commented May 31, 2021

Hi!
Personally I think it's time to close the ticket... the progress has been tremendous and the quality of the package is pretty high. Of course that's only my 2 cents.

Just one more note, please. Can you guys comment on the status of the https://github.com/python-lsp/python-lsp-server integration, and your current suggestions on the current best practices / configuration?

Thanks for all the help!

@krassowski
Copy link
Member

No, none of my performance improvements got into pylsp yet. If anyone feels like picking up please do, I am happy to advise but at capacity right now.

@krassowski
Copy link
Member

Though as a side note, one can now easily checkout the pyright server and compare performance as this one does not use jedi. From what I saw we are now hitting the point with pyright where it is our rendering and not the server that is the culprit of the delay (especially icon SVG parsing which is - ridiculously - done each time rather than only once per icon type).

@krassowski
Copy link
Member

The big news is that it is now possible to disable completions from specific source (e.g. kernel or LSP servers) in the just released 3.7 version.

@potoo0
Copy link

potoo0 commented Jun 15, 2021

The big news is that it is now possible to disable completions from specific source (e.g. kernel or LSP servers) in the just released 3.7 version.

How to disable lsp for some specific language, and use kernel completions.
I want to use metals for scala, and disable pylsp for python.

@sntgluca
Copy link

Hi,

I wish to share one additional note which I think could be of interest:
I really got optimal results with @krassowski 's patched version of python-language-server, while using the default recommended python-lsp-server autocomplete is still pretty lagging.

This means, for me, that you're doing a wonderful job, but without integrating the latest features to the downstream packages, we won't get all the benefits. I'd love to be able to help out but I am not good enough to contribute...

@krassowski
Copy link
Member

How to disable lsp for some specific language, and use kernel completions.
I want to use metals for scala, and disable pylsp for python.

Hi @potoo0 sorry I missed your message earlier. You can disable completion for pylsp only by modifying its settings:

{
    "language_servers": {
        "pylsp": {
            "serverSettings": {
                "pylsp.plugins.jedi_completion.enabled": false,
                "pylsp.plugins.rope_completion.enabled": false
            }
        }
    }
}

@sntgluca it's on my todo-list later this month.

@krassowski
Copy link
Member

Quick update: the two pull requests with major performance improvements to python-lsp-server were merged yesterday. With the default settings you should get a substantial speed up. You can try it out by installing directly from GitHub:

pip install -U jupyterlab jupyterlab-lsp nbclassic
pip uninstall python-language-server python-lsp-server
pip install git+https://github.com/python-lsp/python-lsp-server.git@develop

There were also some improvements for the rendering of icons in #625.

@krassowski
Copy link
Member

krassowski commented Aug 2, 2021

python-lsp-server 1.2 is now available. I will close this issue for now as I believe it was solved - numerous performance improvements were added since it was created and most crucially the real culprit, this is very slow calls to jedi in the default pyls server has been eliminated. If you still struggle with performance please open a new issue with all the details needed to reproduce it.

For anyone reading this comment in future, please note that the fixes are available only in my personal pyls fork and in the community pylsp fork, but not in the original pyls which is no longer maintained (so you need python-lsp-server, not python-language-server); in short:

pip uninstall python-language-server
pip install -U jupyterlab jupyterlab-lsp nbclassic python-lsp-server

@krassowski krassowski unpinned this issue Aug 2, 2021
@hellojinwoo
Copy link

@krassowski Would it be better if I uninstall python-language-server and install -U jupyterlab jupyterlab-lsp nbclassic python-lsp-server, when I use your jupyter-lsp on jupyter desktop as well? In other words, is this needed for Jupyterlab Desktop also?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests