Towards no-config packaging: Automatic discovery of packages, py_modules and name
#2894
Conversation
| print((root / "setup.cfg").read_text()) | ||
|
|
||
|
|
||
| def _get_sdist_members(sdist_path): |
There was a problem hiding this comment.
Both _get_sdist_members and _get_wheel_members could be added to the Archive test helper class in the case PR #2863 gets merged.
abravalheri
force-pushed
the
auto-discovery
branch
from
b1dbc83 to
85b1e66
Compare
|
PS: I am having some trouble with the CI. The macos instances seem to be suffering and throwing some unexpected errors when installing tox dependencies (which were kept unchanged regarding upstream/main), Hopefully the implementation is not too broken (I run the tests under Ubuntu 20.04), but some OS-specific adjustment might be required for Windows/OSX. |
| "site_scons", # SCons | ||
| # ---- Hidden directories/Private packages ---- | ||
| "[._]*", | ||
| ) |
There was a problem hiding this comment.
Is there any other directory that should be added here?
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
| "manage", # Django | ||
| # ---- Hidden files/Private modules ---- | ||
| "[._]*", | ||
| ) |
There was a problem hiding this comment.
Is there any other top-level Python file that should be added here?
abravalheri
force-pushed
the
auto-discovery
branch
2 times, most recently
from
41f160b to
f8453aa
Compare
abravalheri
force-pushed
the
auto-discovery
branch
from
f8453aa to
0274a30
Compare
| # and directories not meant for distribution (e.g. tool-related) | ||
|
|
||
|
|
||
| class FlatLayoutPackageFinder(PEP420PackageFinder): |
There was a problem hiding this comment.
Maybe to be less intrusive the FlatLayoutPackageFinder should inherit from PackageFinder?
…pa#2894) Adds automatic discovery of `packages`, `py_modules` and `name` for `flat-` and `src-` layouts.
Following up the discussion in pypa#2887 and pypa#2329, it seems that setuptools is moving towards more automatic discovery features. PackageFinder and PEP420PackageFinder are fundamental pieces of this puzzle and grouping together them togheter with the code implementing these new discovery features make a lot of sense.
abravalheri
force-pushed
the
auto-discovery
branch
from
1f93dd7 to
f39edae
Compare
It is desirable to have some configurations automatically derived. This is definitely possible for packages and py_modules, and (based on these two) also for name. This change adds a new class `setuptools.discovery.ConfigDiscovery` implementing the automatic discovery logic for packages, py_modules and name.
abravalheri
added this to the Add experimental support for project metadata via `pyproject.toml` milestone
As discussed in #2887 and #2888, it would be nice to have some configurations automatically derived.
This is definitely possible for
packagesandpy_modules, and (based on these two) also forname(as suggested by @jaraco).This implementation supersedes #2888 (closes #2888).
Summary of changes
setuptools.discovery.ConfigDiscoveryimplementing the automatic discovery logicPackageFinderandPEP420PackageFinderto thesetuptools.discoverymodule due to the implementation affinity.FlatLayoutPackageFinderandFlatLayoutModuleFinderclasses implementing a more careful discovery (more exclude patterns) since they are meant to be used directly with the project directory.ConfigDiscoverychanges theDistributionobject just before running commands.Unfortunately, it is not possible to call
ConfigDiscoveryinfinalize_options, since options coming from configuration files (such assetup.cfg) can be still missing (parsed in a later stage).Please notice all these class/module names can be changed upon suggestion. I just adopted them for the lack of better ones 😝.
Closes #2887
Differences from #2888
In #2888, I assumed that
namehave to be statically defined (this assumption follows the ideas in PEP 621). Therefore it is possible to implement a more restricted discovery, only finding modules/packages that match the distribution name when the flat-layout is employed.The implementation in this PR is "eager" (can find more files), since it does not rely on the given metadata.
Moreover this PR goes in the opposite direction and derives
nameforpackagesandpy_modulesas per the #2887 suggestion.How the user can "exclude" modules/packages not meant for distribution?
One of the main motivations of this PR is to simplify package configuration. Currently there is already a lot of options for configuring setuptools behaviour (
py_modules,packagesandpackage_dir). Something I do not wish to do is to complicate even more the understanding of the current existing options by introducing new ones or overloading the existing ones.Instead if the users want to change which files/folder are discovered they can manually specify them via the existing fields in
setup.cfg, or switch to the src-layout (which is supposed to solve most of these problems) when possible. In other words, they have to configure setuptools as it was done before the changes in this PR (so the worst case scenario: automatic discovery doesn't change anything).How about backward compatibility?
The same approach used in #2888 was adopted here: automatic values for
packagesandpy_moduleswill only be discovered if the user don't provide both (which would result, I believe, in an empty distribution prior to this PR -- in the majority of the cases, this would mean that some configuration was missing/wrong).Empty distributions are still possible (e.g. for those that wish to create "metapackages"), as long as they don't create Python module/packages in the project directory (files/folder associated with common tools are automatically excluded, e.g.
docs,fabfile.py,tasks.py-- as well as "private"/"hidden" ones, i.e. files/folders starting with an_or `.) -- which sound as a reasonable requirement for empty distributions.Pull Request Checklist
changelog.d/.(See documentation for details)