Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pylance taking very long to analyze files #9912

Closed
apoorv-s opened this issue Dec 13, 2024 · 7 comments
Closed

Pylance taking very long to analyze files #9912

apoorv-s opened this issue Dec 13, 2024 · 7 comments
Labels
as designed Not a bug, working as intended

Comments

@apoorv-s
Copy link

Environment data

  • Language Server version: 2024.12.1
  • OS and version: linux x64
  • Python version (and distribution if applicable, e.g. Anaconda): 3.10.15 (h4a871b0_2_cpython conda-forge)
  • python.analysis.indexing: true
  • python.analysis.typeCheckingMode: off

Code Snippet

import numpy as np
from numpy.linalg import solve
from scipy.optimize import root

import pickle

from sklearn.gaussian_process import GaussianProcessRegressor
from sklearn.gaussian_process.kernels import RBF
from sklearn.linear_model import LinearRegression

class AdvectivePumpingModel():
    def __init__(self) -> None:
        self.time_gp = None
        self.traj_x_gp = None
        self.traj_y_gp = None

Repro Steps

  1. Pylance is taking a lot of time to analyze the code, and I am getting notifications to raise the issue, which redirects me here. I am using vscode (Version: 1.93.1) on a remote Linux cluster. The yml file for conda environments is attached as well (It is in the .txt format since uploading a .yml file was not allowed).environment.txt

Expected behavior

Till last month, the pylance has been working great, and I have had no issues. However, after some extension updates, the behavior is very slow and takes a long time to analyze even a few lines of code. The logs presented are for a small code with less than 100 lines of code, but this behavior is consistent across different conda environments (with different Python versions) and different files.

I tried reinstalling the vscode and the extension, but it did not resolve anything. Apologies if I am something trivial.

Actual behavior

XXX

Logs

2024-12-13 14:05:43.001 [info] (Client) Pylance async client (2024.12.1) started with python extension (2024.14.1)
2024-12-13 14:05:43.879 [info] [Info  - 2:05:43 PM] (3281628) Server root directory: file:///home/darve/apoorv1/.vscode-server/extensions/ms-python.vscode-pylance-2024.12.1/dist
2024-12-13 14:05:43.882 [info] [Info  - 2:05:43 PM] (3281628) Pylance language server 2024.12.1 (pyright version 1.1.389, commit ce1325fc) starting
2024-12-13 14:05:43.882 [info] [Info  - 2:05:43 PM] (3281628) Starting service instance "DA4Dunes"
2024-12-13 14:05:43.936 [info] [Info  - 2:05:43 PM] (3281628) Setting environmentName for service "DA4Dunes": "3.10.15 (hypo conda)"
2024-12-13 14:05:43.936 [info] [Info  - 2:05:43 PM] (3281628) Setting pythonPath for service "DA4Dunes": "/home/darve/apoorv1/.conda/envs/hypo/bin/python"
2024-12-13 14:05:43.936 [info] [Info  - 2:05:43 PM] (3281628) No include entries specified; assuming /home/darve/apoorv1/DA4Dunes
2024-12-13 14:05:43.937 [info] [Info  - 2:05:43 PM] (3281628) Auto-excluding **/node_modules
2024-12-13 14:05:43.937 [info] [Info  - 2:05:43 PM] (3281628) Auto-excluding **/__pycache__
2024-12-13 14:05:43.938 [info] [Info  - 2:05:43 PM] (3281628) Auto-excluding **/.*
2024-12-13 14:05:43.983 [info] [Info  - 2:05:43 PM] (3281628) Assuming Python version 3.10.15.final.0
2024-12-13 14:05:44.891 [info] [Info  - 2:05:44 PM] (3281628) Found 4 source files
2024-12-13 14:05:44.907 [info] [Info  - 2:05:44 PM] (3281628) Starting service instance "<default>"
2024-12-13 14:05:44.936 [info] [Info  - 2:05:44 PM] (3281628) Background analysis(1) root directory: file:///home/darve/apoorv1/.vscode-server/extensions/ms-python.vscode-pylance-2024.12.1/dist
2024-12-13 14:05:44.936 [info] [Info  - 2:05:44 PM] (3281628) Background analysis(1) started
2024-12-13 14:05:44.967 [info] [Info  - 2:05:44 PM] (3281628) Setting environmentName for service "<default>": "3.10.15 (hypo conda)"
2024-12-13 14:05:44.967 [info] [Info  - 2:05:44 PM] (3281628) Setting pythonPath for service "<default>": "/home/darve/apoorv1/.conda/envs/hypo/bin/python"
2024-12-13 14:05:44.968 [info] [Info  - 2:05:44 PM] (3281628) No include entries specified; assuming /<default workspace root>
2024-12-13 14:05:44.968 [info] [Info  - 2:05:44 PM] (3281628) Auto-excluding **/node_modules
2024-12-13 14:05:44.968 [info] [Info  - 2:05:44 PM] (3281628) Auto-excluding **/__pycache__
2024-12-13 14:05:44.968 [info] [Info  - 2:05:44 PM] (3281628) Auto-excluding **/.*
2024-12-13 14:05:46.447 [info] [Info  - 2:05:46 PM] (3281628) Assuming Python version 3.10.15.final.0
2024-12-13 14:05:46.455 [info] [Error - 2:05:46 PM] (3281628) File or directory "/<default workspace root>" does not exist.
2024-12-13 14:05:46.456 [info] [Info  - 2:05:46 PM] (3281628) No source files found.
2024-12-13 14:05:46.463 [info] [Info  - 2:05:46 PM] (3281628) Indexer background runner(2) root directory: file:///home/darve/apoorv1/.vscode-server/extensions/ms-python.vscode-pylance-2024.12.1/dist (index)
2024-12-13 14:05:46.463 [info] [Info  - 2:05:46 PM] (3281628) Indexing(2) started
2024-12-13 14:05:46.463 [info] [Info  - 2:05:46 PM] (3281628) Background analysis(3) root directory: file:///home/darve/apoorv1/.vscode-server/extensions/ms-python.vscode-pylance-2024.12.1/dist
2024-12-13 14:05:46.465 [info] [Info  - 2:05:46 PM] (3281628) Background analysis(3) started
2024-12-13 14:05:54.947 [info] [Info  - 2:05:54 PM] (3281628) [IDX(2)] Long operation: scan packages file:///home/darve/apoorv1/DA4Dunes (8825ms)
2024-12-13 14:05:54.947 [info] [Info  - 2:05:54 PM] (3281628) scanned(2) 303 files over 1 exec env
2024-12-13 14:05:56.363 [info] [Info  - 2:05:56 PM] (3281628) indexed(2) 303 files over 1 exec env
2024-12-13 14:05:56.645 [info] [Info  - 2:05:56 PM] (3281628) Indexing finished(2).
2024-12-13 14:06:21.124 [info] [Info  - 2:06:21 PM] (3281628) [BG(1)] Long operation: checking: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (34906ms)
2024-12-13 14:06:21.124 [info] [Info  - 2:06:21 PM] (3281628) [BG(1)] Long operation: analyzing: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (34920ms)
2024-12-13 14:10:46.313 [info] [Info  - 2:10:46 PM] (3281628) [BG(1)] Long operation: checking: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (23454ms)
2024-12-13 14:10:46.313 [info] [Info  - 2:10:46 PM] (3281628) [BG(1)] Long operation: analyzing: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (23484ms)
2024-12-13 14:10:51.609 [info] [Info  - 2:10:51 PM] (3281628) [BG(1)] Long operation: checking: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (23185ms)
2024-12-13 14:10:51.610 [info] [Info  - 2:10:51 PM] (3281628) [BG(1)] Long operation: analyzing: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (23214ms)
2024-12-13 14:11:15.004 [info] [Info  - 2:11:15 PM] (3281628) [BG(1)] Long operation: checking: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (23334ms)
2024-12-13 14:11:15.004 [info] [Info  - 2:11:15 PM] (3281628) [BG(1)] Long operation: analyzing: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (23346ms)
2024-12-13 14:11:33.401 [info] [Info  - 2:11:33 PM] (3281628) [BG(1)] Long operation: checking: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (18357ms)
2024-12-13 14:11:33.401 [info] [Info  - 2:11:33 PM] (3281628) [BG(1)] Long operation: analyzing: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (18359ms)
2024-12-13 14:11:55.773 [info] [Info  - 2:11:55 PM] (3281628) [BG(1)] Long operation: checking: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (22317ms)
2024-12-13 14:11:55.773 [info] [Info  - 2:11:55 PM] (3281628) [BG(1)] Long operation: analyzing: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (22334ms)
2024-12-13 14:12:18.641 [info] [Info  - 2:12:18 PM] (3281628) [BG(1)] Long operation: checking: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (22825ms)
2024-12-13 14:12:18.641 [info] [Info  - 2:12:18 PM] (3281628) [BG(1)] Long operation: analyzing: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (22840ms)
2024-12-13 14:12:43.101 [info] [Info  - 2:12:43 PM] (3281628) [BG(1)] Long operation: checking: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (24425ms)
2024-12-13 14:12:43.101 [info] [Info  - 2:12:43 PM] (3281628) [BG(1)] Long operation: analyzing: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (24438ms)
2024-12-13 14:14:15.694 [info] [Info  - 2:14:15 PM] (3281628) [BG(1)] Long operation: checking: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (14595ms)
2024-12-13 14:14:15.694 [info] [Info  - 2:14:15 PM] (3281628) [BG(1)] Long operation: analyzing: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (14618ms)
2024-12-13 14:14:30.055 [info] [Info  - 2:14:30 PM] (3281628) [BG(1)] Long operation: checking: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (14334ms)
2024-12-13 14:14:30.055 [info] [Info  - 2:14:30 PM] (3281628) [BG(1)] Long operation: analyzing: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (14341ms)
2024-12-13 14:14:42.397 [info] [Info  - 2:14:42 PM] (3281628) [BG(1)] Long operation: checking: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (12267ms)
2024-12-13 14:14:42.397 [info] [Info  - 2:14:42 PM] (3281628) [BG(1)] Long operation: analyzing: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (12302ms)
2024-12-13 14:14:55.184 [info] [Info  - 2:14:55 PM] (3281628) Starting service instance "DA4Dunes"
2024-12-13 14:14:55.298 [info] [Info  - 2:14:55 PM] (3281628) Setting environmentName for service "DA4Dunes": "3.10.15 (hypo conda)"
2024-12-13 14:14:55.303 [info] [Info  - 2:14:55 PM] (3281628) Setting pythonPath for service "DA4Dunes": "/home/darve/apoorv1/.vscode-server/extensions/ms-python.vscode-pylance-2024.12.1/dist"
2024-12-13 14:14:55.303 [info] [Info  - 2:14:55 PM] (3281628) No include entries specified; assuming /home/darve/apoorv1/DA4Dunes
2024-12-13 14:14:55.304 [info] [Info  - 2:14:55 PM] (3281628) Auto-excluding **/node_modules
2024-12-13 14:14:55.304 [info] [Info  - 2:14:55 PM] (3281628) Auto-excluding **/__pycache__
2024-12-13 14:14:55.304 [info] [Info  - 2:14:55 PM] (3281628) Auto-excluding **/.*
2024-12-13 14:14:55.526 [info] [Info  - 2:14:55 PM] (3281628) Found 4 source files
2024-12-13 14:14:55.724 [info] [Info  - 2:14:55 PM] (3281628) [BG(1)] Long operation: checking: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (13208ms)
2024-12-13 14:14:55.724 [info] [Info  - 2:14:55 PM] (3281628) [BG(1)] Long operation: analyzing: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (13215ms)
2024-12-13 14:14:56.041 [info] [Info  - 2:14:56 PM] (3281628) Background analysis(4) root directory: file:///home/darve/apoorv1/.vscode-server/extensions/ms-python.vscode-pylance-2024.12.1/dist
2024-12-13 14:14:56.045 [info] [Info  - 2:14:56 PM] (3281628) Background analysis(4) started
2024-12-13 14:14:56.438 [info] [Info  - 2:14:56 PM] (3281628) Indexer background runner(5) root directory: file:///home/darve/apoorv1/.vscode-server/extensions/ms-python.vscode-pylance-2024.12.1/dist (index)
2024-12-13 14:14:56.438 [info] [Info  - 2:14:56 PM] (3281628) Indexing(5) started
2024-12-13 14:14:56.968 [info] [Info  - 2:14:56 PM] (3281628) scanned(5) 1 files over 1 exec env
2024-12-13 14:14:57.481 [info] [Info  - 2:14:57 PM] (3281628) indexed(5) 1 files over 1 exec env
2024-12-13 14:14:57.502 [info] [Info  - 2:14:57 PM] (3281628) Indexing finished(5).
2024-12-13 14:15:09.190 [info] [Info  - 2:15:09 PM] (3281628) [BG(1)] Long operation: checking: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (13500ms)
2024-12-13 14:15:09.190 [info] [Info  - 2:15:09 PM] (3281628) [BG(1)] Long operation: analyzing: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (13515ms)
2024-12-13 14:17:32.777 [info] [Info  - 2:17:32 PM] (3281628) [BG(1)] Long operation: checking: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (15940ms)
2024-12-13 14:17:32.777 [info] [Info  - 2:17:32 PM] (3281628) [BG(1)] Long operation: analyzing: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (15960ms)
2024-12-13 14:18:24.912 [info] [Info  - 2:18:24 PM] (3281628) [BG(1)] Long operation: checking: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (15052ms)
2024-12-13 14:18:24.913 [info] [Info  - 2:18:24 PM] (3281628) [BG(1)] Long operation: analyzing: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (15060ms)
2024-12-13 14:18:36.796 [info] [Info  - 2:18:36 PM] (3281628) [BG(1)] Long operation: checking: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (11838ms)
2024-12-13 14:18:36.796 [info] [Info  - 2:18:36 PM] (3281628) [BG(1)] Long operation: analyzing: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (11849ms)
2024-12-13 14:18:51.373 [info] [Info  - 2:18:51 PM] (3281628) [BG(1)] Long operation: checking: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (14560ms)
2024-12-13 14:18:51.373 [info] [Info  - 2:18:51 PM] (3281628) [BG(1)] Long operation: analyzing: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (14563ms)
2024-12-13 14:19:19.945 [info] [Info  - 2:19:19 PM] (3281628) [BG(1)] Long operation: checking: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (12876ms)
2024-12-13 14:19:19.945 [info] [Info  - 2:19:19 PM] (3281628) [BG(1)] Long operation: analyzing: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (12881ms)
2024-12-13 14:19:32.659 [info] [Info  - 2:19:32 PM] (3281628) [BG(1)] Long operation: checking: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (12670ms)
2024-12-13 14:19:32.659 [info] [Info  - 2:19:32 PM] (3281628) [BG(1)] Long operation: analyzing: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (12672ms)
2024-12-13 14:19:44.949 [info] [Info  - 2:19:44 PM] (3281628) [BG(1)] Long operation: SemanticTokens delta previousResultId:1734128331378 at file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (12287ms)
2024-12-13 14:19:59.862 [info] [Info  - 2:19:59 PM] (3281628) [BG(1)] Long operation: checking: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (14872ms)
2024-12-13 14:19:59.862 [info] [Info  - 2:19:59 PM] (3281628) [BG(1)] Long operation: analyzing: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (14884ms)
2024-12-13 14:20:13.098 [info] [Info  - 2:20:13 PM] (3281628) [BG(1)] Long operation: checking: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (13212ms)
2024-12-13 14:20:13.099 [info] [Info  - 2:20:13 PM] (3281628) [BG(1)] Long operation: analyzing: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (13225ms)
2024-12-13 14:20:26.214 [info] [Info  - 2:20:26 PM] (3281628) [BG(1)] Long operation: checking: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (13085ms)
2024-12-13 14:20:26.214 [info] [Info  - 2:20:26 PM] (3281628) [BG(1)] Long operation: analyzing: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (13092ms)
2024-12-13 14:20:39.480 [info] [Info  - 2:20:39 PM] (3281628) [BG(1)] Long operation: checking: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (13215ms)
2024-12-13 14:20:39.480 [info] [Info  - 2:20:39 PM] (3281628) [BG(1)] Long operation: analyzing: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (13233ms)
2024-12-13 14:20:52.230 [info] [Info  - 2:20:52 PM] (3281628) [BG(1)] Long operation: checking: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (12715ms)
2024-12-13 14:20:52.230 [info] [Info  - 2:20:52 PM] (3281628) [BG(1)] Long operation: analyzing: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (12734ms)
2024-12-13 14:21:05.118 [info] [Info  - 2:21:05 PM] (3281628) [BG(1)] Long operation: checking: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (12868ms)
2024-12-13 14:21:05.118 [info] [Info  - 2:21:05 PM] (3281628) [BG(1)] Long operation: analyzing: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (12875ms)
2024-12-13 14:21:18.043 [info] [Info  - 2:21:18 PM] (3281628) [BG(1)] Long operation: checking: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (12899ms)
2024-12-13 14:21:18.043 [info] [Info  - 2:21:18 PM] (3281628) [BG(1)] Long operation: analyzing: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (12906ms)
2024-12-13 14:21:34.045 [info] [Info  - 2:21:34 PM] (3281628) [BG(1)] Long operation: checking: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (15987ms)
2024-12-13 14:21:34.045 [info] [Info  - 2:21:34 PM] (3281628) [BG(1)] Long operation: analyzing: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (15991ms)
2024-12-13 14:22:20.898 [info] [Info  - 2:22:20 PM] (3281628) [BG(1)] Long operation: checking: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (22097ms)
2024-12-13 14:22:20.898 [info] [Info  - 2:22:20 PM] (3281628) [BG(1)] Long operation: analyzing: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (22101ms)
2024-12-13 14:22:20.899 [info] [Info  - 2:22:20 PM] (3281628) [BG(1)] Long operation: checking: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (22926ms)
2024-12-13 14:22:20.899 [info] [Info  - 2:22:20 PM] (3281628) [BG(1)] Long operation: analyzing: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (22928ms)
2024-12-13 14:22:41.680 [info] [Info  - 2:22:41 PM] (3281628) [BG(1)] Long operation: checking: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (22549ms)
2024-12-13 14:22:41.680 [info] [Info  - 2:22:41 PM] (3281628) [BG(1)] Long operation: analyzing: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (22559ms)
2024-12-13 14:23:03.632 [info] [Info  - 2:23:03 PM] (3281628) [BG(1)] Long operation: checking: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (21914ms)
2024-12-13 14:23:03.633 [info] [Info  - 2:23:03 PM] (3281628) [BG(1)] Long operation: analyzing: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (21921ms)
2024-12-13 14:23:27.755 [info] [Info  - 2:23:27 PM] (3281628) [BG(1)] Long operation: checking: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (24092ms)
2024-12-13 14:23:27.755 [info] [Info  - 2:23:27 PM] (3281628) [BG(1)] Long operation: analyzing: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (24098ms)
2024-12-13 14:23:51.970 [info] [Info  - 2:23:51 PM] (3281628) [BG(1)] Long operation: checking: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (24187ms)
2024-12-13 14:23:51.970 [info] [Info  - 2:23:51 PM] (3281628) [BG(1)] Long operation: analyzing: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (24190ms)
2024-12-13 14:24:13.104 [info] [Info  - 2:24:13 PM] (3281628) [BG(1)] Long operation: checking: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (21102ms)
2024-12-13 14:24:13.104 [info] [Info  - 2:24:13 PM] (3281628) [BG(1)] Long operation: analyzing: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (21112ms)
2024-12-13 14:24:36.549 [info] [Info  - 2:24:36 PM] (3281628) [BG(1)] Long operation: checking: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (23426ms)
2024-12-13 14:24:36.550 [info] [Info  - 2:24:36 PM] (3281628) [BG(1)] Long operation: analyzing: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (23430ms)
2024-12-13 14:24:55.858 [info] [Info  - 2:24:55 PM] (3281628) [BG(1)] Long operation: checking: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (19254ms)
2024-12-13 14:24:55.859 [info] [Info  - 2:24:55 PM] (3281628) [BG(1)] Long operation: analyzing: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py (19278ms)

@StellaHuang95
Copy link
Collaborator

Thanks for reaching out. Since you started experiencing slow operations after recent updates, this could be related to the new language server mode setting we introduced recently. The default mode for this setting is default, and you can find the default settings for each mode here.

Based on the logs you provided, it seems analyzing file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py is taking long time. Did you also see the long operation in previous versions? If not, it is possible that some settings were enabled, which triggers more in-depth analysis.

@apoorv-s
Copy link
Author

apoorv-s commented Jan 7, 2025

I have been experiencing this from updates I installed around October. I checked the settings, and the Diagnostic mode is set to openFilesOnly, and the Language Server Mode is set to default as well. I am not sure what's missing. Are there any other settings I should look out for?

@heejaechang
Copy link
Collaborator

can you share Long operation: analyzing: file:///home/darve/apoorv1/DA4Dunes/Core/Utils.py file? if you remove that file, does perf get better?

by the way, if you share Trace log as it described here - https://github.com/microsoft/pylance-release/wiki/Collecting-data-for-an-investigation.#collecting-trace-logs - rather than info log, that will give us a bit more info.

...

if that doesn't work, we might need more detail log such as https://github.com/microsoft/pylance-release/wiki/Collecting-data-for-an-investigation.#collecting-type-evaluation-related-data-in-the-log or/and https://github.com/microsoft/pylance-release/wiki/Collecting-data-for-an-investigation.#collecting-cpuprofiles

@heejaechang
Copy link
Collaborator

by the way, you mentioned I am using vscode (Version: 1.93.1) on a remote Linux cluster., and you are sure this is not environment issues such as slow network or slow remote machine and etc?

@apoorv-s
Copy link
Author

Hi, sorry for the delay in getting back to you. This behavior is consistent across files and environments. Every additional statement (as simple as x = self.state_dim) I write takes a significant time (>15s) to analyze. Here is the Utils.py file:

    import numpy as np
    from numpy.linalg import solve
    from scipy.optimize import root
    from scipy.stats import multivariate_normal
    from scipy.integrate import solve_ivp
    
    import pickle
    
    from sklearn.gaussian_process import GaussianProcessRegressor
    from sklearn.gaussian_process.kernels import RBF
    from sklearn.linear_model import LinearRegression
    
    from Configs.Configs import APMConfig
    
    class AdvectivePumpingModel():
        def __init__(self, config:APMConfig) -> None:
            self.time_gp = None
            self.traj_x_gp = None
            self.traj_y_gp = None
            
            self.state_dim = config.state_dim
            self.obs_dim = config.obs_dim
            self.forward_model_noise_dist = config.forward_noise_distribution
            self.obs_operator_noise_dist = config.obs_noise_distribution
    
            self.set_time_gp()
            self.set_xy_gp()
            
        def set_time_gp(self):
            # Use GPs notebooks to save the GP model
            # time_gp_kernel_optimized = "14.3**2 * RBF(length_scale=0.811) + WhiteKernel(noise_level=0.0709)"
            with open("SavedModels/saved_time_gp.pkl", "rb") as f:
                self.time_gp = pickle.load(f)
        
        def set_xy_gp(self):
            # Use GPs notebooks to save the GP models
            # traj_x_gp_kernel_optimized = "0.237**2 * RBF(length_scale=0.0366) + WhiteKernel(noise_level=1e-05)"
            # traj_y_gp_kernel_optimized = "0.0712**2 * RBF(length_scale=0.033) + WhiteKernel(noise_level=1e-05)"
            with open("SavedModels/saved_traj_x_gp_1.pkl", "rb") as f:
                self.traj_x_gp = pickle.load(f)
    
            with open("SavedModels/saved_traj_y_gp_1.pkl", "rb") as f:
                self.traj_y_gp = pickle.load(f)
        
        def root_finding(self, x_target, y_target):
            """Find the roots of GP based functions to find initial point and time corresponding to provided xy-coordinates.
    
            Args:
                x_target (float): target x-coordinate
                y_target (float): target y-coordinate
            """
            def helper_function(inp):
                inp = inp.reshape(1, -1)
                x_pred = self.traj_x_gp.predict(inp, return_std=False)
                y_pred = self.traj_y_gp.predict(inp, return_std=False)
                return np.array([x_target - x_pred, y_target - y_pred]).ravel()
            
            initial_guess = np.array([0.35, 0.5])
            sol = root(helper_function, initial_guess)
            
            if sol.success:
                return sol.x
            else:
                # raise ValueError("root not found")
                return [None, None]
        
        def forward_model(self, state):
            # TODO
            return state + self.forward_model_noise_dist.rvs()
        
        def observation_operator(self, state, obs_position):
            x_init, tau_norm = self.root_finding(obs_position[0], obs_position[1])
            t_total = self.time_gp.predict(np.array([x_init]), return_std=False)
            t_final = tau_norm*t_total
            
            return t_final
    
            
        
    if __name__=="__main__":
        import IPython
        IPython.embed()

The associated Config file:

    from scipy.stats import multivariate_normal
    
    class APMConfig():
        def __init__(self):
            self.dune_len = 1 #m
            self.hydraulic_conductivity = 172.8 #m/d
            self.dune_height = 0.09 #m
            self.avg_flow_depth = 0.3 #m
            self.avg_stream_velocity = 0.39 #m/s
            self.grav_const = 9.81 #m/s^2
            self.porosity = 0.38
            
            ## Data from excel file for 70cm dune
            # {'Hyd Conductivity': 0.002,
            # 'Slope': 0.003,
            # 'Depth': 0.3,
            # 'Dune Length': 0.7,
            # 'Water Surface Elevation': 0.1149}
            
            self.state_dim = 5
            self.obs_dim = 2
            self.forward_noise_distribution = multivariate_normal() # TODO
            self.obs_noise_distribution = multivariate_normal() # TODO
            
            self.obs_noise_description = "Gaussian"
            self.forward_noise_distribution_description = "Gaussian"

The trace log is in the attached txt file.

tracelog.txt

@erictraut
Copy link
Collaborator

Paring this down to a simplified sample that exhibits poor analysis performance:

from scipy.optimize import root
root(0, 0)

If someone from the pylance team would like to transfer this to the pyright project, I can dig into it and see if there's a way to improve the performance.

The scipy library is notoriously complex, and it has no inline type annotations. That means static type checkers like pyright need to infer type information from the library code.

@apoorv-s, one thing you can do to eliminate the performance problem is to install type stubs for scipy. You can do this by running pip install scipy-stubs within your local environment. If you don't want to do this, you can tell pyright to not attempt to infer type information from the scipy Python code by setting pyright.useLibraryCodeForTypes to false. However, that will disable all completion suggestions related to symbols that you import from this library.

@rchiodo rchiodo transferred this issue from microsoft/pylance-release Feb 13, 2025
@erictraut erictraut added the needs investigation Requires additional investigation to determine course of action label Feb 13, 2025
@erictraut
Copy link
Collaborator

I looked into this further. I don't think there's much I can do here. As we already knew, scipy is a very "thick" library written in Python (as opposed to a library like numpy, which tends to be a thin Python wrapper around a C++ or Fortran implementation). Scipy is also completely unannotated, so return types must be inferred from the implementation. Because the call chains are so deep, that involves analyzing hundreds or thousands of individual function implementations just to infer the return type of a top-level call like root in this example.

The solution here is type annotations — either inlined type information shipped as part of the scipy library or a separate stub library. Thankfully, there is an ongoing effort to create a type stub library for scipy. As mentioned above, you can install this into your Python environment using the command pip install scipy-stubs.

I'm closing the issue since there's nothing actionable for pyright at this point.

@erictraut erictraut closed this as not planned Won't fix, can't repro, duplicate, stale Feb 18, 2025
@erictraut erictraut added as designed Not a bug, working as intended and removed needs investigation Requires additional investigation to determine course of action labels Feb 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
as designed Not a bug, working as intended
Projects
None yet
Development

No branches or pull requests

4 participants