Skip to content

Commit

Permalink
Interlink/explain differences with lint, fmt and fix goals (#21864)
Browse files Browse the repository at this point in the history 
This does a few tweaks focused on the docs about the `lint`, `fmt` and
`fix` goals:

- explain the difference between them, particularly the "syntactic"
(`fmt`) vs. "semantic" (`fix`) changes, which I couldn't find expressed
anywhere else
- link between them
- expand the Python "linters and formatters" & goal docs to consider
`fix` too
- link to the various subsystems for each Python tool

---------

Co-authored-by: Benjy Weinberger <benjyw@gmail.com>
  • Loading branch information
@huonw @benjyw
huonw and benjyw authored Jan 22, 2025
1 parent 0ee9a16 commit 7cab9e3
Show file tree
Hide file tree
Showing 5 changed files with 98 additions and 35 deletions.
8 changes: 4 additions & 4 deletions docs/docs/python/goals/fmt.mdx
View file
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
---
title: fmt
title: fmt and fix
sidebar_position: 1
---

Autoformat source code.
Autoformat and autofix source code.

---

See [here](../overview/linters-and-formatters.mdx) for how to opt in to specific formatters, along with how to configure them:
See [here](../overview/linters-and-formatters.mdx) for how to opt in to specific formatters and fixers, along with how to configure them:

- Autoflake
- Black
Expand All @@ -16,4 +16,4 @@ See [here](../overview/linters-and-formatters.mdx) for how to opt in to specific
- Pyupgrade
- yapf

If you activate multiple formatters, Pants will run them sequentially so that they do not overwrite each other. You may need to update each formatter's config file to ensure that it is compatible with the other activated formatters.
If you activate multiple tools, Pants will run them sequentially so that they do not overwrite each other. You may need to update each tool's config file to ensure that it is compatible with the other activated tools.
52 changes: 26 additions & 26 deletions docs/docs/python/overview/linters-and-formatters.mdx
View file
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
---
title: Linters and formatters
title: Linters, formatters and fixers
sidebar_position: 4
---

How to activate and use the Python linters and formatters bundled with Pants.
How to activate and use the Python linters, formatters and fixers bundled with Pants.

---

:::tip Benefit of Pants: consistent interface
`pants lint` and `pants fmt` will consistently and correctly run all your linters and formatters. No need to remember how to invoke each tool, and no need to write custom scripts.
`pants lint`, `pants fmt` and `pants fix` will consistently and correctly run all your linters, formatters and fixers. No need to remember how to invoke each tool, and no need to write custom scripts.

This consistent interface even works with multiple languages, like running Python linters at the same time as Go, Shell, Java, and Scala.
:::
Expand All @@ -24,22 +24,22 @@ Pants does several things to speed up running formatters and linters:

## Activating linters and formatters

Linter/formatter support is implemented in separate [backends](../../using-pants/key-concepts/backends.mdx) so that they are easy to opt in to individually:

| Backend | Tool |
| :--------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------- |
| `pants.backend.python.lint.bandit` | [Bandit](https://bandit.readthedocs.io/en/latest/): security linter |
| `pants.backend.python.lint.black` | [Black](https://black.readthedocs.io/en/stable/): code formatter |
| `pants.backend.python.lint.docformatter` | [Docformatter](https://pypi.org/project/docformatter/): docstring formatter |
| `pants.backend.python.lint.flake8` | [Flake8](https://flake8.pycqa.org/en/latest/): style and bug linter |
| `pants.backend.python.lint.isort` | [isort](https://readthedocs.org/projects/isort/): import statement formatter |
| `pants.backend.python.lint.pydocstyle` | [Pydocstyle](https://pypi.org/project/pydocstyle/): docstring linter |
| `pants.backend.python.lint.pylint` | [Pylint](https://pylint.pycqa.org/): style and bug linter |
| `pants.backend.python.lint.yapf` | [Yapf](https://github.com/google/yapf): code formatter |
| `pants.backend.python.lint.autoflake` | [Autoflake](https://github.com/myint/autoflake): remove unused imports |
| `pants.backend.python.lint.pyupgrade` | [Pyupgrade](https://github.com/asottile/pyupgrade): automatically update code to use modern Python idioms like `f-strings` |
| `pants.backend.experimental.python.lint.ruff.check` | [Ruff (for linting)](https://docs.astral.sh/ruff/linter/): an extremely fast Python linter, written in Rust. |
| `pants.backend.experimental.python.lint.ruff.format` | [Ruff (for formatting)](https://docs.astral.sh/ruff/formatter/): an extremely fast Python code formatter, written in Rust. |
Linter/formatter/fixer support is implemented in separate [backends](../../using-pants/key-concepts/backends.mdx) so that they are easy to opt in to individually:

| Backend | Tool |
|:-------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------|
| [`pants.backend.python.lint.bandit`](../../../reference/subsystems/bandit) | [Bandit](https://bandit.readthedocs.io/en/latest/): security linter |
| [`pants.backend.python.lint.black`](../../../reference/subsystems/black) | [Black](https://black.readthedocs.io/en/stable/): code formatter |
| [`pants.backend.python.lint.docformatter`](../../../reference/subsystems/docformatter) | [Docformatter](https://pypi.org/project/docformatter/): docstring formatter |
| [`pants.backend.python.lint.flake8`](../../../reference/subsystems/flake8) | [Flake8](https://flake8.pycqa.org/en/latest/): style and bug linter |
| [`pants.backend.python.lint.isort`](../../../reference/subsystems/isort) | [isort](https://readthedocs.org/projects/isort/): import statement formatter |
| [`pants.backend.python.lint.pydocstyle`](../../../reference/subsystems/pydocstyle) | [Pydocstyle](https://pypi.org/project/pydocstyle/): docstring linter |
| [`pants.backend.python.lint.pylint`](../../../reference/subsystems/pylint) | [Pylint](https://pylint.pycqa.org/): style and bug linter |
| [`pants.backend.python.lint.yapf`](../../../reference/subsystems/yapf) | [Yapf](https://github.com/google/yapf): code formatter |
| [`pants.backend.python.lint.autoflake`](../../../reference/subsystems/autoflake) | [Autoflake](https://github.com/myint/autoflake): remove unused imports |
| [`pants.backend.python.lint.pyupgrade`](../../../reference/subsystems/pyupgrade) | [Pyupgrade](https://github.com/asottile/pyupgrade): automatically update code to use modern Python idioms like `f-strings` |
| [`pants.backend.experimental.python.lint.ruff.check`](../../../reference/subsystems/ruff) | [Ruff (for linting)](https://docs.astral.sh/ruff/linter/): an extremely fast Python linter, written in Rust. |
| [`pants.backend.experimental.python.lint.ruff.format`](../../../reference/subsystems/ruff) | [Ruff (for formatting)](https://docs.astral.sh/ruff/formatter/): an extremely fast Python code formatter, written in Rust. |

To enable, add the appropriate backends in `pants.toml`:

Expand All @@ -53,7 +53,7 @@ backend_packages = [
]
```

You should now be able to run `pants lint`, and possibly `pants fmt`:
You should now be able to run [`pants lint`](../../../reference/goals/lint), and possibly [`pants fmt`](../../../reference/goals/fmt) or [`pants fix`](../../../reference/goals/fix):

```
$ pants lint src/py/project.py
Expand All @@ -75,7 +75,7 @@ MyPy is run with the [check goal](../goals/check.mdx), rather than `lint`.

## Configuring the tools, for example, adding plugins

You can configure each formatter and linter using these options:
You can configure each formatter, fixer and linter using these options:

| Option | What it does |
| :------------------------ | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
Expand Down Expand Up @@ -131,15 +131,15 @@ For tools that autodiscover config files—such as Black, isort, Flake8, and Pyl
If your config file is in a non-standard location, you must instead set the `--config` option, e.g. `[isort].config`. This will ensure that the config file is included in the process's sandbox and Pants will instruct the tool to load the config.
:::

## Running only certain formatters or linters
## Running only certain formatters, fixers or linters

To temporarily skip a tool, use the `--skip` option for that tool. For example, run:

```bash
❯ pants --black-skip --flake8-skip lint ::
```

You can also use the `--lint-only` and `--fmt-only` options with the names of the tools:
You can also use the `--lint-only`, `--fmt-only` or `--fix-only` options with the names of the tools:

```bash
❯ pants lint --only=black ::
Expand Down Expand Up @@ -169,7 +169,7 @@ python_tests(
)
```

When you run `pants fmt` and `pants lint`, Pants will ignore any files belonging to skipped targets.
When you run `pants fmt`, `pants fix` and `pants lint`, Pants will ignore any files belonging to skipped targets.

## Tip: only run over changed files

Expand All @@ -191,9 +191,9 @@ Pants will find which files have changed and only run over those files. See [Adv

## Tips for specific tools

### Order of `backend_packages` matters for `fmt`
### Order of `backend_packages` matters for `fmt` and `fix`

Pants will run formatters in the order in which they appear in the `backend_packages` option.
Pants will run formatters and fixers in the order in which they appear in the `backend_packages` option.

For example, you likely want to put Autoflake (which removes unused imports) before Black and Isort, which will format your import statements.

Expand Down
24 changes: 22 additions & 2 deletions src/python/pants/core/goals/fix.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@
from pants.engine.unions import UnionMembership, UnionRule, distinct_union_type_per_subclass, union
from pants.option.option_types import BoolOption
from pants.util.collections import partition_sequentially
from pants.util.docutil import bin_name
from pants.util.docutil import bin_name, doc_url
from pants.util.logging import LogLevel
from pants.util.ordered_set import FrozenOrderedSet
from pants.util.strutil import Simplifier, softwrap
Expand Down Expand Up @@ -215,7 +215,27 @@ def did_change(self) -> bool:

class FixSubsystem(GoalSubsystem):
name = "fix"
help = "Autofix source code."
help = softwrap(
f"""
Autofix source code.
This goal runs tools that make 'semantic' changes to source code, where the meaning of the
code may change.
See also:
- [The `fmt` goal]({doc_url('reference/goals/fix')} will run code-editing tools that may make only
syntactic changes, not semantic ones. The `fix` includes running these `fmt` tools by
default (see [the `skip_formatters` option](#skip_formatters) to control this).
- [The `lint` goal]({doc_url('reference/goals/lint')}) will validate code is formatted, by running these
fixers and checking there's no change.
- Documentation about formatters for various ecosystems, such as:
[Python]({doc_url('docs/python/overview/linters-and-formatters')}), [JVM]({doc_url('jvm/java-and-scala#lint-and-format')}),
[SQL]({doc_url('docs/sql#enable-sqlfluff-linter')})
"""
)

@classmethod
def activated(cls, union_membership: UnionMembership) -> bool:
Expand Down
23 changes: 22 additions & 1 deletion src/python/pants/core/goals/fmt.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@
from pants.engine.goal import Goal, GoalSubsystem
from pants.engine.rules import Get, collect_rules, goal_rule
from pants.engine.unions import UnionMembership, UnionRule, union
from pants.util.docutil import doc_url
from pants.util.strutil import softwrap

logger = logging.getLogger(__name__)

Expand Down Expand Up @@ -51,7 +53,26 @@ def _get_rules(cls) -> Iterable:

class FmtSubsystem(GoalSubsystem):
name = "fmt"
help = "Autoformat source code."
help = softwrap(
f"""
Autoformat source code.
This goal runs tools that make 'syntactic' changes to source code, where the meaning of the
code doesn't (usually) change.
See also:
- [The `fix` goal]({doc_url('reference/goals/fix')}) will run code-editing tools that may make semantic
changes, not just syntactic ones.
- [The `lint` goal]({doc_url('reference/goals/lint')}) will validate code is formatted, by running these
formatters and checking there's no change.
- Documentation about formatters for various ecosystems, such as:
[Python]({doc_url('docs/python/overview/linters-and-formatters')}), [Go]({doc_url('docs/go#gofmt')}),
[JVM]({doc_url('jvm/java-and-scala#lint-and-format')}), [Shell]({doc_url('docs/shell#shfmt-autoformatter')}).
"""
)

@classmethod
def activated(cls, union_membership: UnionMembership) -> bool:
Expand Down
26 changes: 24 additions & 2 deletions src/python/pants/core/goals/lint.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@
from pants.engine.unions import UnionMembership, UnionRule, distinct_union_type_per_subclass, union
from pants.option.option_types import BoolOption
from pants.util.collections import partition_sequentially
from pants.util.docutil import bin_name
from pants.util.docutil import bin_name, doc_url
from pants.util.logging import LogLevel
from pants.util.meta import classproperty
from pants.util.strutil import Simplifier, softwrap
Expand Down Expand Up @@ -233,7 +233,29 @@ def _get_rules(cls) -> Iterable:

class LintSubsystem(GoalSubsystem):
name = "lint"
help = "Run linters/formatters/fixers in check mode."
help = softwrap(
f"""
Run linters/formatters/fixers in check mode.
This goal runs tools that check code quality/styling etc, without changing that code. This
includes running formatters and fixers, but instead of writing changes back to the
workspace, Pants treats any changes they would make as a linting failure.
See also:
- [The `fmt` goal]({doc_url('reference/goals/fix')} will save the the result of formatters
(code-editing tools that make only "syntactic" changes) back to the workspace.
- [The `fmt` goal]({doc_url('reference/goals/fix')} will save the the result of fixers
(code-editing tools that may make "semantic" changes too) back to the workspace.
- Documentation about linters for various ecosystems, such as:
[Python]({doc_url('docs/python/overview/linters-and-formatters')}), [Go]({doc_url('docs/go')}),
[JVM]({doc_url('jvm/java-and-scala#lint-and-format')}), [Shell]({doc_url('docs/shell')}),
[Docker]({doc_url('docs/docker#linting-dockerfiles-with-hadolint')}).
"""
)

@classmethod
def activated(cls, union_membership: UnionMembership) -> bool:
Expand Down

0 comments on commit 7cab9e3

Please sign in to comment.