-
Notifications
You must be signed in to change notification settings - Fork 27.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Styling
] stylify using ruff
#27144
[Styling
] stylify using ruff
#27144
Conversation
The documentation is not available anymore as the PR was closed or merged. |
5335f89
to
0d678ad
Compare
examples/research_projects/movement-pruning/Saving_PruneBERT.ipynb
Outdated
Show resolved
Hide resolved
examples/research_projects/deebert/src/modeling_highway_bert.py
Outdated
Show resolved
Hide resolved
examples/research_projects/movement-pruning/Saving_PruneBERT.ipynb
Outdated
Show resolved
Hide resolved
examples/research_projects/movement-pruning/Saving_PruneBERT.ipynb
Outdated
Show resolved
Hide resolved
examples/research_projects/movement-pruning/Saving_PruneBERT.ipynb
Outdated
Show resolved
Hide resolved
examples/research_projects/movement-pruning/Saving_PruneBERT.ipynb
Outdated
Show resolved
Hide resolved
@@ -156,7 +155,7 @@ | |||
"rhoknp>=1.1.0,<1.3.1", | |||
"rjieba", | |||
"rouge-score!=0.0.7,!=0.0.8,!=0.1,!=0.1.1", | |||
"ruff>=0.0.241,<=0.0.259", | |||
"ruff>=0.1.5,<=0.2", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you elaborate a bit on this choice?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Latest version for better performance and unified api I'd say. While <=2 to have enough rolling in case they change things
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, I misread the numbers. Got it!
utils/check_copies.py
Outdated
async def run_ruff(code): | ||
command = ["ruff", "format", "-", "--config", "pyproject.toml", "--silent"] | ||
process = await asyncio.create_subprocess_exec( | ||
*command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE | ||
) | ||
stdout, _ = await process.communicate(input=code.encode()) | ||
return stdout.decode() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
would love to hear the motivation to make is async 🙏
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Followed what was done in huggingface hub, nor real reasons apart trying to make it faster. Though it won't be run in parallel. Did not benchmark both
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well, as here we don't really use the concurrency (which is the advantage of using coroutine
), it looks somehow overwhelming to use async
stuff here.
But my real concern is: here we spawn a new process whenever there is a # Copied from
statement, which is an expensive op. Previously, blackify
uses black.format_str
which is inside the same python process, despite black itself is slow.
I checked the run (check_copies) with this PR and on main
: 44 sec. on this PR and 32 sec. on main
.
See run with this PR and run with main.
I am afraid, at least for check_copies.py
, we lose the advantage of using ruff format
due to the many new processes being created each time.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well I can do not async, no worries. Black is just too complicated, and with ruff you don't need to use doc-builder on top of it.
Now differences can also come from modified files as well but let's remove black because if you take the make style
time into account it's already worth it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for this awesome work! Having a few question, but sure LGTM!
Remove asynch
* try to stylify using ruff * might need to remove these changes? * use ruf format andruff check * use isinstance instead of type comparision * use # fmt: skip * use # fmt: skip * nits * soem styling changes * update ci job * nits isinstance * more files update * nits * more nits * small nits * check and format * revert wrong changes * actually use formatter instead of checker * nits * well docbuilder is overwriting this commit * revert notebook changes * try to nuke docbuilder * style * fix feature exrtaction test * remve `indent-width = 4` * fixup * more nits * update the ruff version that we use * style * nuke docbuilder styling * leve the print for detected changes * nits * Remove file I/O Co-authored-by: charliermarsh <charlie.r.marsh@gmail.com> * style * nits * revert notebook changes * Add # fmt skip when possible * Add # fmt skip when possible * Fix * More ` # fmt: skip` usage * More ` # fmt: skip` usage * More ` # fmt: skip` usage * NIts * more fixes * fix tapas * Another way to skip * Recommended way * Fix two more fiels * Remove asynch Remove asynch --------- Co-authored-by: charliermarsh <charlie.r.marsh@gmail.com>
* try to stylify using ruff * might need to remove these changes? * use ruf format andruff check * use isinstance instead of type comparision * use # fmt: skip * use # fmt: skip * nits * soem styling changes * update ci job * nits isinstance * more files update * nits * more nits * small nits * check and format * revert wrong changes * actually use formatter instead of checker * nits * well docbuilder is overwriting this commit * revert notebook changes * try to nuke docbuilder * style * fix feature exrtaction test * remve `indent-width = 4` * fixup * more nits * update the ruff version that we use * style * nuke docbuilder styling * leve the print for detected changes * nits * Remove file I/O Co-authored-by: charliermarsh <charlie.r.marsh@gmail.com> * style * nits * revert notebook changes * Add # fmt skip when possible * Add # fmt skip when possible * Fix * More ` # fmt: skip` usage * More ` # fmt: skip` usage * More ` # fmt: skip` usage * NIts * more fixes * fix tapas * Another way to skip * Recommended way * Fix two more fiels * Remove asynch Remove asynch --------- Co-authored-by: charliermarsh <charlie.r.marsh@gmail.com>
What does this PR do?
Removes our dependency on
black
in favor of rust:make style
I'll try to get numbers but that one is pretty obvious