implement support for PEP-518 - the tool.pytest key in pyproject.toml

[RonnyPfannschmidt]

as per https://www.python.org/dev/peps/pep-0518/#specification

the file has a section for tools using the pypi name for the tool,

so this is the one file where we can properly and in a supported way put metadata

i propose adding support asap and deprecating the other config files for the 3.x series so we can possibly remove support starting with the 4.x series

[The-Compiler]

Big -1 on deprecating pytest.ini if you also suggest that.

My config files are pretty big, which is why I want to keep them in their own file rather than stuffing everything into one - line count:

   18 .coveragerc
   59 .flake8
   13 .pydocstylerc
   69 .pylintrc
  245 tox.ini
   41 pytest.ini

[RonnyPfannschmidt]

i see, good point!

i change my proposal to deprecating all files other than pytest.ini and pyproject.toml

[hynek]

Any plans on moving this forward? 😇

[RonnyPfannschmidt]

@hynek yes, but no time/volunteer

[tadeoos]

Hey guys, I gave it a go in #3686. It adds support for pyproject.toml but doesn't deprecate anything.

[nicoddemus]

Any ideas on how we the file would look like? #3686 was an attempt made by @tadeoos, but @RonnyPfannschmidt mentioned that we should take advantage of the additional layers offered by TOML.

I suggest we focus on the file format first, then after a consensus move to the API to get to that file format.

My quick take on this is that we don't need many sections other than separating built-in options from those defined by plugins. For brevity I think built-options could go to a root tool.pytest table, and plugin options in sub-tables named after the plugin.

Quick example:

[tool.pytest]
junit_family = "xunit2"
doctest_encoding = "UTF-8"

  [tool.pytest.qt]
  qt_api = "qt5"

  [tool.pytest.timeout]
  timeout = 10

  [tool.pytest.xprocess]
  timeout = 10

This avoids ini-options from different plugins to clashing.

[RonnyPfannschmidt]

that would intermix pytest and plugin tables in the most unforetunate manner,

tool.pytest.x nests inside tool.pytest

plugins should use tool.pytest_x, matching the distribution name with dashes replaced by underscore perhaps

the real problem isnt the new config file, the real problem is layering, integration of the old config options and transition of the apis

[nicoddemus]

that would intermix pytest and plugin tables in the most unforetunate manner,

Why would that be a problem?

The real problem isnt the new config file, the real problem is layering, integration of the old config options and transition of the apis

Oh I know, but I thought the discussion should start from the file format first. I'm well aware that the hard work will be the integration and backward compatibility. 😓

[njsmith]

Right, @nicodemus's example is equivalent to

[tool.pytest]
junit_family = "xunit2"
doctest_encoding = "UTF-8"
qt = { qt_api = "qt5" }
timeout = { timeout = 10 }
xprocess = { timeout = 10 }

If you wanted to avoid collisions you could use [tool.pytest.plugin.pytest-whatever], but that seems like it might be excessive, given that [tool.pytest-whatever] is already available.

with dashes replaced by underscore perhaps

Dashes are fine in toml keys, so if packages want to use [tool.pytest-whatever] then that's fine.

[RonnyPfannschmidt]

@nicoddemus i believe that without the integration strategy, we will end up with a massive mess

right now config initialization is pretty much fundamentally broken to begin with

[nicoddemus]

For illustration:

[tool.pytest]
junit_family = "xunit2"
doctest_encoding = "UTF-8"

  [tool.pytest-qt]
  qt_api = "qt5"

  [tool.pytest-timeout]
  timeout = 10

  [tool.pytest-xprocess]
  timeout = 10

i believe that without the integration strategy, we will end up with a massive mess

"massive mess" on the file format or the code to support the file format in the end? We can always can come back to the drawing board if we can't find a clean way to implement the desired file format.

Or do you want to change the discussion from the file format over to the API/integration?

[RonnyPfannschmidt]

the format is easy, the integration/semantics are hard

[marco-silva0000]

I would also like to see this feature added.

[nicoddemus]

What does the approach taken by tox to have a legacy session inside the toml file where it contains a plain string with the current pytest.ini contents? This would at least allow people to move their configuration there, while we try to brainstorm how to move this forward.

[RonnyPfannschmidt]

That one would be good, we could also support a mapping of options to strings or lists as the legacy value to take advantage of the file format for Syntax, which may be a help to the desired semantics

[nicoddemus]

we could also support a mapping of options to strings or lists as the legacy value to take advantage of the file format for Syntax, which may be a help to the desired semantics

Interesting, care to elaborate, perhaps with examples?

[RonnyPfannschmidt]

Format strings come to mind, and native lists

[RonnyPfannschmidt]

I won't make examples as I'm currently afk

[RonnyPfannschmidt]

[tool.pytest.LEGACY]
addopts = ["-p", "test"]

or

[tool.pytest]
LEGACY = """
addopts = -p test
"""

if a multiline string is used, we should allow a missing [pytest]
if a mapping is used, the type of each value should be enforced to exactly match whats used/declared (to avoid having to be unreasonably smart about the transformations in a format thats only supposed to help with transitions

(perhaps we should not support the mapping form to begin with)

[ofek]

Have we considered a plugin namespace to clean things up?

[tool.pytest]
addopts = "-x"

[tool.pytest.plugins.timeout]
timeout = 10

[tool.pytest.plugins.qt]
api = "pyside2"

[RonnyPfannschmidt]

@ofek we have not considered any completely new structure, as we yet have to figure how we want to unify arguments from clie, options from ini files and data from well structured toml data

we also haven't gotten't around investigating how it would actually be used - and i haven't found time to do more on my proposed api

so for now we are just going to support a legacy ini key where ini options can be put instead of the other files, better integration will come later with time or working external proposals

[iloveitaly]

For anyone else running into this issue, this is now supported via the tool.pytest.ini_options pyproject.toml key

[merwok]

Are you going to deprecate config in tox.ini?
Some of us prefer to have the config for lint/test tools in that file rather than many small files (or pyproject.toml, which may not be present for example for django web apps, or if present may be used for packaging only and not all other developement tasks)

[RonnyPfannschmidt]

This will stay for a longer while