Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Black Formatter are hanging when trying to format on save #317

Open
maolike opened this issue Sep 20, 2023 · 23 comments
Open

Black Formatter are hanging when trying to format on save #317

maolike opened this issue Sep 20, 2023 · 23 comments
Assignees
Labels
bug Issue identified by VS Code Team member as probable bug info-needed Issue requires more information from poster triage-needed Issue is not triaged.

Comments

@maolike
Copy link

maolike commented Sep 20, 2023

Every time I open vscode and run it for a while, and then save the python file, a pop-up window will pop up and will not disappear, causing the python file to fail to format on save. This problem can be solved by restarting vscode.
image

@github-actions github-actions bot added the triage-needed Issue is not triaged. label Sep 20, 2023
@karthiknadig karthiknadig self-assigned this Sep 20, 2023
@karthiknadig
Copy link
Member

@maolike would it be possible to share a repro for this. we will need the type of environment, operating system, python versions, and if you can share your project itself it might help.

@karthiknadig karthiknadig added bug Issue identified by VS Code Team member as probable bug info-needed Issue requires more information from poster labels Sep 20, 2023
@chrisandrewmann
Copy link

Not sure if it's related but Black has stopped formatting on save on my system, seemingly after latest VSCode update.
I'm on VSCode 1.82.2 running on Windows 11 with Black Formatter v2023.4.1

@karthiknadig
Copy link
Member

@chrisandrewmann Can you share what you see in the logs? Output > Black Formatter for logs.

Also, are you using format on save or do you trigger formatting using one of the following shortcuts, if yes then which one:
image

@chrisandrewmann
Copy link

chrisandrewmann commented Sep 21, 2023

@karthiknadig I'm using format on save, not using shortcuts. Will share the logs shortly when I figure out how to.

@karthiknadig
Copy link
Member

@chrisandrewmann Please also share your settings.

@chrisandrewmann
Copy link

@karthiknadig Thanks, I should have checked the logs before. It's now working.
It was a typo in the --line-length parameter in my settings, which showed in the logs.

Previously I had:
"black-formatter.args": [
"--line-length 88",
],

Changed to:
"black-formatter.args": [
"--line-length", "88",
],

@maolike
Copy link
Author

maolike commented Sep 25, 2023

@maolike would it be possible to share a repro for this. we will need the type of environment, operating system, python versions, and if you can share your project itself it might help.

This problem only reappears in my local environment.My project is just a simple python file

Operating system:

版本 Windows 10 专业版
版本号 22H2
安装日期 ‎2023/‎2/‎3
操作系统内部版本 19045.3086
体验 Windows Feature Experience Pack 1000.19041.1000.0

Python version: Python 3.11.1

VS Code:

Version: 1.82.2 (system setup)
Commit: abd2f3db4bdb28f9e95536dfa84d8479f1eb312d
Date: 2023-09-14T05:55:25.390Z
Electron: 25.8.1
ElectronBuildId: 23779380
Chromium: 114.0.5735.289
Node.js: 18.15.0
V8: 11.4.183.29-electron.0
OS: Windows_NT x64 10.0.19045

@karthiknadig
Copy link
Member

@maolike In the logs do you see a formatting trigger? please share logs form Output > Black Formatter.

@maolike
Copy link
Author

maolike commented Sep 27, 2023

@karthiknadig
This is the latest log when the problem occurred.

2023-09-25 14:17:13.022 [info] [Trace - 2:17:13 PM] Received response 'textDocument/formatting - (2)' in 14ms.
2023-09-25 14:17:13.027 [info] [Trace - 2:17:13 PM] Sending notification 'textDocument/didChange'.
2023-09-27 10:18:47.875 [info] [Trace - 10:18:47 AM] Sending notification 'textDocument/didChange'.
2023-09-27 10:18:49.059 [info] [Trace - 10:18:49 AM] Sending request 'textDocument/formatting - (3)'.
2023-09-27 10:20:17.261 [info] [Trace - 10:20:17 AM] Sending notification '$/cancelRequest'.

@karthiknadig
Copy link
Member

karthiknadig commented Sep 27, 2023

@maolike I will need more details like the file contents that you were trying to format, virtual environments that you were using, your settings etc. Also, can you verify if black is able to format that file using command line? Last thing, is the encoding of the file UTF8?

You can try this command in windows"

> type <path-to-your-file> | python -m black --stdin-filename <path-to-your-file> -

Example:

> type test.py | python -m black --stdin-filename test.py -

@maolike
Copy link
Author

maolike commented Oct 7, 2023

This is my file contents:

import re

stylePattern = r"[\s\S]*</body>"
str = r"```html"
styleText = re.search(stylePattern, str)
if styleText is None:
   print("none")
else:
    print(styleText.group())

What does virtual environment mean?

My settings:

{
  "workbench.colorTheme": "GitHub Dark Dimmed",
  "JSON-zain.json.autorefresh": true,
  "security.workspace.trust.untrustedFiles": "open",
  "editor.formatOnSave": true,
  "editor.defaultFormatter": "esbenp.prettier-vscode",
  "window.zoomLevel": 1,
  "git.openRepositoryInParentFolders": "never",
  "extensions.ignoreRecommendations": true,
  "explorer.confirmDelete": false,
  "cSpell.userWords": [
    "autorefresh",
    "baiduv",
    "docpath",
    "easytrader",
    "esbenp",
    "hexin",
    "ksymproxy",
    "zain"
  ],
  "security.workspace.trust.enabled": false,
  "[python]": {
    "editor.defaultFormatter": "ms-python.black-formatter"
  },
  "cSpell.languageSettings": [],
  "terminal.integrated.defaultProfile.windows": "Command Prompt",
  "leek-fund.funds": [],
  "leek-fund.stocks": [
    "hf_CHA50CFD"
  ],
  "[xml]": {
    "editor.defaultFormatter": "DotJoshJohnson.xml"
  },
  "leek-fund.riseColor": "#888888",
  "leek-fund.fallColor": "#888888",
  "leek-fund.iconType": "arrow",
  "workbench.editor.untitled.hint": "hidden",
  "debug.onTaskErrors": "showErrors",
  "cmake.configureOnOpen": true,
  "cloudmusic.music.quality": 999000,
  "cloudmusic.account.autoCheck": true,
  "python.formatting.provider": "none"
}

The encoding of the file is UTF8.

I can format that file using command line.

image

@tumbl3w33d
Copy link

tumbl3w33d commented Nov 14, 2023

I have a similar behavior since the last update (running vscode 1.84.2, black formatter extension v2023.6.0) now. I have "format on save" active and what it does, no matter if I changed something in a file or not, is running black 57 times (at least I see the following block that often in the formatter output):

2023-11-14 14:46:42.296 [info] [Warn  - 2:46:42 PM] Black does not support range formatting. Formatting entire document.
2023-11-14 14:46:42.300 [info] /home/tumble/my/project/venv/bin/python -m black --line-length 100 --stdin-filename /home/tumble/my/project/roles/myrole/action_plugins/myplugin.py -
2023-11-14 14:46:42.301 [info] CWD formatter: /home/tumble/my/project
2023-11-14 14:46:42.401 [info] All done! ✨ 🍰 ✨
1 file left unchanged.

This takes about 11 seconds which makes saving a file a pain because it really blocks the saving so I need to wait a long to test small code changes.

Edit: the time used is not constant. After a "reload window" and saving again, it had 119 of these blocks in the output and took 52s.

Some more log output:

2023-11-14 15:01:15.727 [debug] Server State: Stopped
2023-11-14 15:01:17.823 [info] Name: Black Formatter
2023-11-14 15:01:17.823 [info] Module: black
2023-11-14 15:01:17.823 [debug] Configuration: {"name":"Black","module":"black"}
2023-11-14 15:01:17.823 [info] Default formatter is set to ms-python.black-formatter for workspace /home/tumble/projects/my/project
2023-11-14 15:01:17.823 [info] Python extension loading
2023-11-14 15:01:17.823 [info] Waiting for interpreter from python extension.
2023-11-14 15:01:17.865 [info] No interpreter found from setting black-formatter.interpreter
2023-11-14 15:01:17.865 [info] Getting interpreter from ms-python.python extension for workspace /home/tumble/projects/my/project
2023-11-14 15:01:17.865 [info] Python extension loaded
2023-11-14 15:01:17.869 [info] Interpreter from ms-python.python extension for /home/tumble/projects/my/project: /home/tumble/projects/my/project/venv/bin/python
2023-11-14 15:01:17.869 [info] No interpreter found from setting black-formatter.interpreter
2023-11-14 15:01:17.869 [info] Getting interpreter from ms-python.python extension for workspace /home/tumble/projects/my/project
2023-11-14 15:01:17.873 [info] Interpreter from ms-python.python extension for /home/tumble/projects/my/project: /home/tumble/projects/my/project/venv/bin/python
2023-11-14 15:01:17.882 [info] Server run command: /home/tumble/projects/my/project/venv/bin/python /home/tumble/.vscode/extensions/ms-python.black-formatter-2023.6.0/bundled/tool/lsp_server.py
2023-11-14 15:01:17.882 [info] Server: Start requested.
2023-11-14 15:01:17.882 [debug] Server State: Starting
2023-11-14 15:01:18.765 [info] CWD Server: /home/tumble/projects/my/project
2023-11-14 15:01:18.766 [info] Settings used to run Server:
[
    {
        "cwd": "/home/tumble/projects/my/project",
        "workspace": "file:///home/tumble/projects/my/project",
        "args": [
            "--line-length",
            "100"
        ],
        "path": [],
        "interpreter": [
            "/home/tumble/projects/my/project/venv/bin/python"
        ],
        "importStrategy": "useBundled",
        "showNotifications": "off"
    }
]

2023-11-14 15:01:18.766 [info] Global settings:
{
    "cwd": "/home/tumble",
    "workspace": "/home/tumble",
    "args": [
        "--line-length",
        "100"
    ],
    "path": [],
    "interpreter": [],
    "importStrategy": "useBundled",
    "showNotifications": "off"
}

2023-11-14 15:01:18.766 [info] sys.path used to run Server:
   /home/tumble/.vscode/extensions/ms-python.black-formatter-2023.6.0/bundled/libs
   /home/tumble/.vscode/extensions/ms-python.black-formatter-2023.6.0/bundled/tool
   /usr/lib/python311.zip
   /usr/lib/python3.11
   /usr/lib/python3.11/lib-dynload
   /home/tumble/projects/my/project/venv/lib/python3.11/site-packages
2023-11-14 15:01:18.766 [info] /home/tumble/projects/my/project/venv/bin/python -m black --version
2023-11-14 15:01:18.766 [info] CWD formatter: /home/tumble/projects/my/project
2023-11-14 15:01:18.803 [info] Version info for formatter running for /home/tumble/projects/my/project:
black, 23.10.1 (compiled: no)
Python (CPython) 3.11.5

2023-11-14 15:01:18.803 [info] SUPPORTED black>=22.3.0
FOUND black==23.10.1

2023-11-14 15:01:18.808 [debug] Server State: Running
2023-11-14 15:01:21.576 [info] [Trace - 3:01:21 PM] Sending request 'textDocument/rangeFormatting - (1)'.
2023-11-14 15:01:21.576 [info] Params: {
    "textDocument": {
        "uri": "file:///home/tumble/projects/my/project/roles/myrole/action_plugins/myplugin.py"
    },
    "range": {
        "start": {
            "line": 7,
            "character": 0
        },
        "end": {
            "line": 7,
            "character": 0
        }
    },
    "options": {
        "tabSize": 4,
        "insertSpaces": true
    }
}


2023-11-14 15:01:21.599 [info] [Trace - 3:01:21 PM] Received notification 'window/logMessage'.
2023-11-14 15:01:21.599 [info] Params: {
    "type": 2,
    "message": "Black does not support range formatting. Formatting entire document."
}


2023-11-14 15:01:21.600 [info] [Warn  - 3:01:21 PM] Black does not support range formatting. Formatting entire document.
2023-11-14 15:01:21.612 [info] [Trace - 3:01:21 PM] Received notification 'window/logMessage'.
2023-11-14 15:01:21.612 [info] Params: {
    "type": 4,
    "message": "/home/tumble/projects/my/project/venv/bin/python -m black --line-length 100 --stdin-filename /home/tumble/projects/my/project/roles/myrole/action_plugins/myplugin.py -"
}


2023-11-14 15:01:21.612 [info] /home/tumble/projects/my/project/venv/bin/python -m black --line-length 100 --stdin-filename /home/tumble/projects/my/project/roles/myrole/action_plugins/myplugin.py -
2023-11-14 15:01:21.612 [info] [Trace - 3:01:21 PM] Received notification 'window/logMessage'.
2023-11-14 15:01:21.613 [info] Params: {
    "type": 4,
    "message": "CWD formatter: /home/tumble/projects/my/project"
}

Edit 2: I tried running black on the command line, which I assumed to look something like this:

# resembles what you suggested above for windows, but I'm on linux, so I piped the filename in
$ echo roles/myrole/action_plugins/myplugin.py | python -mblack --stdin-filename roles/myrole/action_plugins/myplugin.py 
roles / myrole / action_plugins / myplugin.py
reformatted roles/myrole/action_plugins/myplugin.py

All done! ✨ 🍰 ✨

And while it looks like it succeeds, nothing seems to happen and the file remains unchanged. However, when I do this:

$ python -mblack roles/myrole/action_plugins/myplugin.py
reformatted roles/myrole/action_plugins/myplugin.py

All done! ✨ 🍰 ✨
1 file reformatted

I see significant reformatting taking place in the file.

@karthiknadig
Copy link
Member

karthiknadig commented Nov 14, 2023

@tumbl3w33d You cannot use echo, echo just prints what ever you type after it to stdout

When you run this command:

echo roles/myrole/action_plugins/myplugin.py | python -m black --stdin-filename roles/myrole/action_plugins/myplugin.py 

You are literally passing "roles/myrole/action_plugins/myplugin.py" to black which is why the out put from black is:
roles / myrole / action_plugins / myplugin.py

You should be using cat on (mac/linux) and type on windows.
try:

cat roles/myrole/action_plugins/myplugin.py | python -m black --stdin-filename roles/myrole/action_plugins/myplugin.py -

Note that depending on file sizes and content some times black itself can take a long time to parse. If black seems to show for you , you can try ruff (formatter) which is based on black but is implemented over rust, and is more performant.

@tumbl3w33d
Copy link

Ah, that makes sense. I didn't know what type does so I guessed wrong. With cat the behavior is the same though, in that it doesn't change anything about the file.

I don't think black is too slow for me, because the command line call is almost instant and does its job. It's just the extension doing something different that makes it run in circles and being called a hundred times without any effect and I can't tell yet why is that. 🤔

@karthiknadig
Copy link
Member

karthiknadig commented Nov 14, 2023

@tumbl3w33d When you run with cat or type you should see the output where black prints out the formatted content:
image

cat roles/myrole/action_plugins/myplugin.py | python -m black --stdin-filename roles/myrole/action_plugins/myplugin.py -

be sure to include the - at the end.

@tumbl3w33d
Copy link

Oh right, that's the case, indeed. I just misinterpreted the output. So we know there's no issue with calling black on command line either way. Is there any other info I could provide to let you know why the extension calls this multiple times without effect (although it says it worked)?

@karthiknadig
Copy link
Member

@tumbl3w33d For that I will need the entire log. The one you shared only contains the part where we call black, but not the response from black.

@tumbl3w33d
Copy link

I don't know what changed, but this looping effect is gone and so is the waiting time. I guess I'm fine then and I will get back in case this returns.

In that case I will try to bring the log then. Thanks for your responses!

@maolike
Copy link
Author

maolike commented Nov 17, 2023

I think it is a vscode bug.I open the file in a new vscode window,black formatter works,and output log normally.But when i back to old window,format the same file,there is no any output log about Black Formatter.Then,I use command to format file,it is ok.

@crucialfelix
Copy link

crucialfelix commented Dec 4, 2023

I get this often if I'm editing a python file that isn't in any project. ie shell scripts written in python

2023-12-04 09:35:19.774 [info] Settings used to run Server:
[
    {
        "cwd": "/Users/crucialfelix/org/bin",
        "workspace": "file:///Users/crucialfelix/org/bin",
        "args": [],
        "path": [],
        "interpreter": [
            "/opt/homebrew/bin/python3"
        ],
        "importStrategy": "useBundled",
        "showNotifications": "off"
    }
]

2023-12-04 09:35:19.776 [info] Global settings:
{
    "cwd": "/",
    "workspace": "/",
    "args": [],
    "path": [],
    "interpreter": [],
    "importStrategy": "useBundled",
    "showNotifications": "off"
}

2023-12-04 09:35:19.805 [info] sys.path used to run Server:
   /Users/crucialfelix/.vscode-insiders/extensions/ms-python.black-formatter-2023.7.13351006/bundled/libs
   /Users/crucialfelix/.vscode-insiders/extensions/ms-python.black-formatter-2023.7.13351006/bundled/tool
   /opt/homebrew/Cellar/python@3.11/3.11.6_1/Frameworks/Python.framework/Versions/3.11/lib/python311.zip
   /opt/homebrew/Cellar/python@3.11/3.11.6_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11
   /opt/homebrew/Cellar/python@3.11/3.11.6_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/lib-dynload
   /opt/homebrew/lib/python3.11/site-packages
2023-12-04 09:35:19.829 [info] /opt/homebrew/bin/python3 -m black --version
2023-12-04 09:35:19.861 [info] CWD formatter: /Users/crucialfelix/org/bin
2023-12-04 09:35:20.024 [info] Version info for formatter running for /Users/crucialfelix/org/bin:
black, 23.11.0 (compiled: no)
Python (CPython) 3.11.6

2023-12-04 09:35:20.028 [info] SUPPORTED black>=22.3.0
FOUND black==23.11.0

2023-12-04 09:35:46.265 [info] /opt/homebrew/bin/python3 -m black --stdin-filename /Users/crucialfelix/org/bin/move-inbox.py -
2023-12-04 09:35:46.268 [info] CWD formatter: /Users/crucialfelix/org/bin

There is no response because it never finishes.

If I run this manually:

bin❯ /opt/homebrew/bin/python3 -m black --stdin-filename /Users/crucialfelix/org/bin/move-inbox.py -
/opt/homebrew/opt/python@3.11/bin/python3.11: No module named black

And then I realize that it's using python 3.11
which is odd because VSCode has selected python 3.12

but the black formatter is defaulting to use /opt/homebrew/bin/python3

bin❯ /opt/homebrew/bin/python3 --version                                          
Python 3.11.6

My import strategy is the default "useBundled"

When set to useBundled, the extension will use the Black formatter binary that is shipped with the extension. When set to fromEnvironment, the extension will attempt to use the Black formatter binary and all dependencies that are available in the currently selected environment. Note: If the extension can't find a valid Black formatter binary in the selected environment, it will fallback to using the binary that is shipped with the extension.

black is shipped with the extension, but it's not using that for some reason.

bin❯ ll /Users/crucialfelix/.vscode-insiders/extensions/ms-python.black-formatter-2023.7.13351006/bundled/libs                             
drwxr-xr-x@    - crucialfelix  4 Dec 10:04 __pycache__
.rw-r--r--@   20 crucialfelix  1 Dec 12:21 _black_version.py
drwxr-xr-x@    - crucialfelix  4 Dec 09:35 black
drwxr-xr-x@    - crucialfelix  1 Dec 12:21 black-23.11.0.dist-info
drwxr-xr-x@    - crucialfelix  1 Dec 12:21 blackd

By switching VSCode to use the correct python (the default one used by by script) it works fine, because there is a black installed there.

@karthiknadig
Copy link
Member

@crucialfelix When you are running from terminal, the terminal does not include the bundled packages in its environment. This is intentional, to avoid interference from the packages that are only used for formatting from affecting anything that is production code.

If you want to run the bundled formatter all you need to do is export PYTHONPATH=/Users/crucialfelix/.vscode-insiders/extensions/ms-python.black-formatter-2023.7.13351006/bundled/libs Then run your command.

Third thing, if you want to manually run black using the mode that we use with the extension then you will need to do this:

> cat /Users/crucialfelix/org/bin/move-inbox.py | /opt/homebrew/bin/python3 -m black --stdin-filename /Users/crucialfelix/org/bin/move-inbox.py -

In this mode black receives content of your code as a string. The reason we have to do it this way is that the file itself on disk won't have your changes, as it is not yet saved. So, the content of the editor is sent to black.

I am not able to repro this locally, unfortunately, there can be several reasons why it appears to hang. Some of which may not be related to this extension at all. It really depends on if there were any other extensions doing something computation heavy. Can you look at Help > Toggle Developer Tools to see if any errors were reported.

@agsimmons
Copy link

agsimmons commented Feb 2, 2024

I'm also having this issue. I think it may be a vscode problem.

OS: Arch Linux

Relevant Config:

    "[python]": {
      "editor.formatOnSave": true,
      "editor.defaultFormatter": "ms-python.black-formatter",
    },
    "[javascript]": {
      "editor.formatOnSave": true,
      "editor.defaultFormatter": "esbenp.prettier-vscode",
      "editor.tabSize": 2,
      "editor.insertSpaces": true,
    },

This error began for me on VSCode. I tried using VSCodium to see if the issue was the same, and it reoccurred after some time. The issue seems to only occur after the editor has been opened for some time. This issue occurred for me on both 2023.6.0 and 2023.4.1 of vscode-black-formatter. I haven't specifically tested other versions, but I did not have this issue for the years before it started.

On save, the Saving xyz: Running 'Black Formatter' Formatter popup appears at the bottom right corner after a few seconds and remains forever. The file is never formatted. When this happens, the following log message appears in the dev tools. It consistently occurs each time this issue happens.

d

This is preceded by messages about the extension host becoming responsive/unresponsive.

image

Manually triggering the "Format Document" action does not work either. When this is done, the loading animation above the code window appears forever and the document is not formatted. No error is logged in this case.

I thought this was specific to vscode-black-formatter, but the issue is now happening for me with Javascript as well with the Prettier formatter with format-on-save and manual formatting.

After running the "Developer: Reload Window" action, format on save and manual formatting work again. I suspect that after some time, both will stop working again.

I'm not familiar with how vscode works, but it seems like the extension host becomes unresponsive, which causes formatter extensions to fail to run. Maybe they are "triggered" to run, but it never gets to the extension, which could explain the hanging popup. There are no errors in the output logs of either formatter, so that could back that theory up.

Please let me know if I can provide any more information

@patrickkidd
Copy link

patrickkidd commented Nov 6, 2024

This isn't exactly the same set of observations, but with save on format enabled, changes in the editor never get saved when doing command-S and then switching editor tabs before the black formatting completes. This is not an edge case for me, since I only ever save just before switching tabs as a rule, and do this many times every minute all day. I suspect others do the same.

Steps to reproduce:

  1. Enable Format on Save at any settings level
  2. Set the default formatter or python formatter to the black extension.
  3. Make changes to a python file of any decent length, hit command-s to save (macOS)
  4. Observe a 2-3 second delay for black to complete formatting and the file to save
  5. Switch editor tabs to a another file before save is complete
  6. Observe that the original editor tab is never saved.
  7. Switch back to the original edit tab, hit command-s to save again, but this time don't switch tabs.
  8. Observe that the file is formatted and saved after that 2-3 second delay. Sometimes it is < 1 second.

This also sometimes occurs without switching editor tabs. I see the 1 file left unchanged. message in the Black Formatter Output panel, but the editor tab is still marked as having unsaved changes and the changes never make it to disk. Looks like a fundamental bug in the extension's stdin/stdout/editor pipeline.

VSCode - October 2024 (version 1.95)
Black Formatter Extension - v2024.4.0
Python: 3.10 (via pyenv, then pipenv)
macOS 15.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Issue identified by VS Code Team member as probable bug info-needed Issue requires more information from poster triage-needed Issue is not triaged.
Projects
None yet
Development

No branches or pull requests

7 participants