Migrate flake8 config to pyproject.toml

[johnthagen]

If/when the flake8 developers decide to support pyproject.toml, should move flake8 configuration to that to consolidate configurations.

• pyproject.toml (PEP 518) support PyCQA/flake8#234 (locked)

Current Status

This comment lists blockers the Flake8 developers have listed:

• pyproject.toml (PEP 518) support PyCQA/flake8#234 (comment)

• "pip to change its behaviour so mere presence of the file does not change functionality"

  • Default to isolated, pyproject.toml-based builds pypa/pip#9175
  • Raise a warning when pip falls back to the "legacy" direct install code pypa/pip#8102

• "neither toml nor the python implementations are even 1.0"

  • The TOML language is 1.0

• "python to consider adding a standard library toml implementation"

  • tomllib has been added to Python 3.11

This comment lists a non-technical reason a Flake8 developer has against TOML (against the creator of TOML himself):

• Any interest in owning a flake8 extension for VS Code? PyCQA/flake8#1467 (comment)

Recent tweets:

• https://twitter.com/codewithanthony/status/1556976653208780805
  • Using TOML has the risk of breaking Flake8 plugins
• https://twitter.com/codewithanthony/status/1556979272534876160?s=20&t=ZnRLnfD4K4JIXmyqQq5zxg
  • Due to the complexity of Flake8's configuration system, adding TOML support would require a full rewrite of the plugin system

[drhagen]

Issue for flake8 pyproject.toml support was migrated and locked.

[Halkcyon]

@drhagen reading through that with the broken links is quite a mess. Has any progress been made in the year+ since that issue was closed on supporting pyproject.toml like a lot of other popular packages have?

[agronholm]

As a workaround, you can use pyproject-flake8.

[johnthagen]

bpo-40059 is tracking the discussion about adding TOML parsing to the stdlib, which has historically been one of the blockers that the flake8 maintainer has listed as a requirement for supporting pyproject.toml.

Also see:

• Please consider pushing tomli into stdlib hukkin/tomli#141
• Standardizing how to handle adding/removing modules from the stdlib python/steering-council#92
• https://discuss.python.org/t/adopting-recommending-a-toml-parser/4068/88
• Please add a co-maintainer uiri/toml#361 (comment)

[johnthagen]

tomli has been accepted into the stdlib for 3.11+ as tomllib

• Please consider pushing tomli into stdlib hukkin/tomli#141 (comment)

[agronholm]

It's a crutch until flake8 itself gets TOML support. Now that TOML support is in stdlib, there shouldn't be any excuse left for them not to add pyproject.toml support.

[johnthagen]

It appears that the recent acceptance of tomllib into the std library has not been enough to motivate flake8 devs to begin adding support at this time:

• pyproject.toml support PyCQA/flake8#1570
• pyproject.toml support blockers PyCQA/flake8#1571
• Any interest in owning a flake8 extension for VS Code? PyCQA/flake8#1467 (comment)

[johnthagen]

#91 uses pyproject-flake8 to move flake8 configuration into pyproject.toml. I am going to leave this issue open until flake8 adds native support for pyproject.toml, as there are certain edge cases that (e.g. editors shelling out to flake8) this does not fully solve.

[flying-sheep]

The problems with pyproject-flake8 are that it’s not upstream, and therefore needs more configuration than flake8, and is less integrated than it. E.g.:

• Its versions don’t reflect flake8 versions, so pre-commit autoupdate doesn’t bump its version, which means the cache will continue to hold the flake8 version that was downloaded when the newest pyproject-flake8 was released.
• IDE/Editor plugins like VS Code’s need to be configured to use pflake8 instead of flake8.

I’m very much a fan of having all development configuration in pyproject.toml and wonder how we can get every project to support it.

[agronholm]

Which is why it would be nice to get that support in upstream flake8 now that all of the blockers should be gone.

[johnthagen]

Summary of recent closed requests:

• Please unlock issue about pyproject.toml support PyCQA/flake8#1598
• Configuration file feature request revisited now that concerns addressed PyCQA/flake8#1632
• issue #1598 appears to have been locked by mistake PyCQA/flake8#1651
• With TOML part of the standard library in 3.11, can we have pyproject.toml support? PyCQA/flake8#1655

[flying-sheep]

I think there’s two facets to this:

1. The opaque approach the PyCQA people are taking here with closing & locking mostly* without even hinting at the reason. They don’t treat people with respect, they don’t acknowledge any of the arguments, they just cut off communication as if everyone was a bad faith actor or obviously wrong. I think that’s inexcusable. People write these issues because they want to make things better, not to spite Anthony or something.

   *The only hint I saw was in @NeilGirdhar’s issue Configuration file feature request revisited now that concerns addressed PyCQA/flake8#1632 (comment) where Anthony says “neither of the pip issues you linked are related”.

2. One of the things Anthony considers a hard blocker still remains: Pip is still branching behavior based on the presence of pyproject.toml, specifically build isolation.

   @NeilGirdhar’s issue above does indeed link unrelated pip issues, as can be glanced from Isolated builds when both pyproject.toml and setup.py co-exist pypa/pip#8437 (comment): Pip uses build isolation independently from PEP 517: --no-use-pep517 doesn’t turn it off.

To fix the bad blood, the PyCQA people need to start respecting people filing issues and explicitly link the pip issues that point out that their blocker is still not resolved.

To fix the underlying technical issue, pip would need to switch build isolation on or off based on a criterion other than Path('pyproject.toml').is_file(). Unconditionally on would work for the flake8 maintainers (as stated in an email), alternatively I assume switching based on the [build-system] table would also work.

[agronholm]

Hm, I was under the impression that nowadays pip only does an isolated build if the build-system section is found in pyproject.toml. Am I mistaken?

[flying-sheep]

You are! I checked the current main branch code:

1. The build_isolation config value is True by default and means “does the user allow pip to use build isolation?” (can be turned off via --no-build-isolation)
2. A requirement’s use_pep517 defaults to has_pyproject or not importlib.util.find_spec("setuptools") (can be specified via --use-pep517/--no-use-pep517)
3. pip will use build isolation if self.req.use_pep517 and build_isolation

Therefore without specifying any of the mentioned CLI options and with setuptools installed, build isolation depends on the existence of pyproject.toml, not its contents.

[agronholm]

Then it was at least discussed somewhere. Would seem like a good change to make in pip, yes?

[flying-sheep]

Seems like that’ll happen:

Once we kill setup.py install and setup.py develop, we should be good to flip the switch on build isolation and pyproject.toml-based builds as well.

See here: pypa/pip#9175 (comment)

[johnthagen]

Hey all, happy to use this as a forum to track status and discuss technical productive ways to improve the situation, but please refrain from personal attacks of any kind.

Thank you for your understanding.

[johnthagen]

I am migrating to this Flake8 pyproject.toml tool instead:

• https://github.com/john-hen/Flake8-pyproject
• Alternatives to this project? john-hen/Flake8-pyproject#2

The main advantages being:

• It still allows the user to run flake8 rather than pflake8
  • This removes one more thing to teach about the differences of how to run flake8 with pyproject.toml
  • This helps with some editor integration
• It doesn't monkey patch, so it might be a more stable way to get this functionality than pyproject-flake8

Disadvantages:

• It is more complex than pyproject-flake8

Edit, done in:

• Port flake8 pyproject.toml support plugin to flake8-pyproject #140

[flying-sheep]

I just came across https://github.com/charliermarsh/ruff, which seems like a great and fast alternative to flake8, isort, …

[johnthagen]

I just came across https://github.com/charliermarsh/ruff,

Yes, I've been keeping my eye on Ruff as well. It looks very promising.

[johnthagen]

I have decided to migrate from Flake8 to Ruff for a variety of reasons. Ruff natively supports pyproject.toml so this particular papercut is solved with it.

To look at how I choose to do the migration, see

• Migrate linting from Flake8/isort to Ruff #164