efficient formatting

[noneofyourbusiness1415252]

Switch to ruff, which is much faster than yapf -- even though it can't tell whether the contents of the file have changed, causing extra IO/processing in the case they haven't, it's unbeaten by yapf in my tests. Or revert 3f6ede0 for the option which still uses yapf but without spawning an executable each time.
Also ~fixes https://ask.replit.com/t/replit-randomly-sets-all-tab-indententations-to-double-spaces-despite-preferences/57047.
There's a replit issue that indent disappears on format when the tab preference from the file settings differs from the user settings I noticed it was because I'd been passing a tuple to newText instead of a string. But still, it's kind of a replit issue for not validating the response and causing crazy behaviour.

[noneofyourbusiness1415252]

I've put this fork into a template -- except I realised that the pyright-extended used in repls is actually outdated and uses an old version of ruff from the Nix cache, so I had to modify the pyright-langserver.js to reverse the effect of 9520fac.

[jackyzha0]

Hey! Thanks for the contribution. Can we split this into two PRs?

1. Respect formatting (the one in packages/pyright-internal/src/languageServerBase.ts)
2. Switch to ruff via the WASM module. For this second PR, benchmarks would be nice to see

We also looked at two internally but didn't find a way to pass config in via flags but using WASM is a nice way to get around that! Feel free to ping me for a review once you do that :)

[noneofyourbusiness1415252]

@jackyzha0 not sure how to go about splitting it up: adding tab support in pyright-yapf/index.ts differs between using yapf and ruff. So I just made a PR from 68c5e44. And then do you want me to split that up so that there's one PR for respecting indent type preference, and one for the performance improvements?
As for benchmarks, all I can say is that, looking at the ?debug=1 lsp logs, the slowest cases were in the optimised yapf version, and the fastest cases were in ruff, formatting the same file multiple times on a Core plan repl. ruff best case 103ms, and worst case was 3404ms while I held Ctrl+S, spamming formatting requests. The yapf one simply fails if you spam it, and best case was 202ms.

[jackyzha0]

For the first part, I'm looking for just a PR to edit the CLI flags passed to yapf to respect tabs without all the WASM-ed module stuff. As for optimized yapf vs ruff, let's just go with Ruff and I can review that one separately (this should also include the tab formatting option that the first PR includes but for ruff)

The reason is that it'll likely take a few rounds of back and forth to while we review the latter PR but I think it's important to get a working patch in for tab formatting.

[noneofyourbusiness1415252]

the WASM-ed module stuff

to be clear, you mean the napi->cpython FFI stuff?

[jackyzha0]

I don't see any usages of napi ffi in your PR, just @wasm-fmt/ruff_fmt" and the header patching for fetching the wasm?

[noneofyourbusiness1415252]

by that I mean the use of node-calls-python in 68c5e44 for yapf

[jackyzha0]

theres a lot of package changes (esp at the root level), can you make sure your package-lock changes are scoped to your changes?

[github-actions]

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

[blast-hardcheese]

Hey there! In an effort to get some confidence around changes like this I added some LSP tests in #32 and #33 which cover completion, linting, and formatting.

I know this PR has been languishing for a while, but I think there's actually either a bug in the upstream ruff wrapper, or a bug in how we're loading the WASM bundle:

$ mkdir python_workspace
$ echo 'print("hello")' > python_workspace/main.py
$ node packages/pyright/dist/pyright.js ./python_workspace/
.../python_workspace/main.py
  .../python_workspace/main.py:1:1 - error: "print" is not defined (pyright[reportUndefinedVariable])
1 error, 0 warnings, 0 informations 

Additionally, in order to test this PR I had to delete

                {
                    test: /\.wasm$/,
                    type: 'asset/inline',
                },

from webpack.config.js, since if that's in there I get TypeError: The URL must be of scheme file, since the entire wasm file gets inlined as data:..., then passed into fs.readFile in recent versions of ruff_fmt. Just an FYI in case you rebase this and try to build.

To run the new test suite, make build test.

[blast-hardcheese]

I think this is moving in the right direction, but it can't be merged as is unfortunately. Going to mark this as "Request changes," but I'm also happy to either close this out due to the upstream bug(s?) or just switch to invoking ruff on the CLI instead of using the WASM bundle.

Please just let me know how you'd like to proceed, I'd like to get this in!

[noneofyourbusiness1415252]

I'd got rid of the ruff_fmt package on my end (and even the ruff CLI for ruff check) in favour of ruff_wasm, but I have yet to create a build script, and I've now got bigger priorities + a clumsy routine. I don't even know whether it works right now.

[rob32]

ruff as formatter would be really great. are there any news on this?

[noneofyourbusiness1415252]

Try my template. To keep the repl updated properly with the latest features from the replit python module, ruff would have to go directly in pyright-extended, hence this PR. (an alternative would be to patch the latest version of the replit module to use ruff similarly to https://replit.com/@UMARismyname/black?v=1#.config/bashrc, but that would be hard to implement especially if you're going for optimal formatting performance).
As you can see above, there are some upstream issues which mean that I have to rely on patching the ruff_wasm crate instead of simply using the ruff_fmt npm package. I happened to have already started this in the pyright-extended folder @ https://replit.com/@UMARismyname/black#pyright-extended/.git/hooks/post-merge while I was trying to optimise the linting. I've lost interest in programming on replit, so feel free to try and complete the work.