SPECPARTS dir in %_builddir/%buildsubdir is leaking to setuptools package discovery

[hroncok]

The SPECPARTS directory is leaking into setuptools package discovery. When upstream Python projects choose to use automatic Python package discovery by setuptools, SPECPARTS is considered a Python package (because empty directories actually are Python packages) and when not explicitly excluded, it makes setuptools die with:

...
discovered packages -- ['pgactivity', 'SPECPARTS', 'pgactivity.queries']
Traceback (most recent call last):
...
  File "/usr/lib/python3.11/site-packages/setuptools/discovery.py", line 441, in _analyse_flat_layout
    return self._analyse_flat_packages() or self._analyse_flat_modules()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/setuptools/discovery.py", line 447, in _analyse_flat_packages
    self._ensure_no_accidental_inclusion(top_level, "packages")
  File "/usr/lib/python3.11/site-packages/setuptools/discovery.py", line 477, in _ensure_no_accidental_inclusion
    raise PackageDiscoveryError(cleandoc(msg))
setuptools.errors.PackageDiscoveryError: Multiple top-level packages discovered in a flat-layout: ['SPECPARTS', 'pgactivity'].

(Full traceback at dalibo/pg_activity#378 (comment))

I suppose other upstreams might consider a new directory in $PWD something to automatically consider important.

---

Could this directory either be moved outside of %_builddir/%buildsubdir or at least be hidden (e.g. .SPECPARTS)? Thanks

[ffesti]

Yeah, I see this is a problem. The fact that we don't really have a proper directory for the build is really not great. We basically rely on the source tarball providing a directory (*). Having the directory beside the buildsubdir is probably the best solution for now until we move the whole thing one dir down (which is unlikely to happen in 4.19).

*) details apply

[pmatilai]

(see #2078 for further background)

[hroncok]

Another Fedora package is known to be impacted. python-quantities: https://bugzilla.redhat.com/show_bug.cgi?id=2213013

[pmatilai]

Repeating this here to attach it to the issue rather than a specific PR:

An easy solution would be letting a package override the path. That allows the handful of special cases to handle it on spec level while leaving it loud and clear for the others. Those overriding it are obviously aware of the magic it entails so that's not a problem.

Currently the macro thing is kinda backwards in that the directory is hardcoded in C and then exposed as a macro, when it should be the other way around.

[hroncok]

An easy solution would be letting a package override the path. That allows the handful of special cases to handle it on spec level while leaving it loud and clear for the others...

What would be the advice for packages affected by the setuptools package discovery issue? Putting %global specpartsdir .SPECPARTS in the spec? Do we want dozens of specs with that?

[dmnks]

I wonder if we could make it the default value for now (i.e. in the spirit of #2533 basically)?

[hroncok]

In our testing Copr for Python 3.12, the r"Multiple top-level packages discovered in a flat-layout: \[.*'SPECPARTS'" regex was found in:

• python-uc-micro-py
• python-linkify-it-py
• python-nose2

[pmatilai]

I don't know. Three affected packages is a handful where a workaround seems acceptable, if it's dozens of packages then less so. Unless it could be hidden in the surrounding helper macros.

[ffesti]

#2534 would solve that issue by moving the SPECPARTS dir one dir up. It's only one char different for moving it back in. But so far I don't quite see a drawback for doing it this way except there is now one more way how builds of the same package can step on each others toe.

[dlax]

Three affected packages is a handful where a workaround seems acceptable, if it's dozens of packages then less so.

"Automatic discovery" in setuptools is quite new (v61.0.0, 24 Mar 2022) and still in beta; that's probably why so few packages use it for now.

[pmatilai]

@ffesti's proposal is probably the least painful way out for now: it avoids polluting specs with crap that would become obsolete with #2078.

[jwakely]

Could the default be SPECPARTS-%{name} instead of %{name}-SPECPARTS, so that %{_builddir}/%{name}* doesn't change meaning?

[pmatilai]

That assumes %{_builddir}/%{name}* has a meaning to begin with. Any package making assumptions about has already stepped outside its boundaries.