With --use-pep517, file:/// and tar.gz sdists support setup.cfg only source trees, but implicit file paths do not

[graingert]

Description

Interestingly, while testing this I noticed that, assuming ./pkgdir contains setup.cfg but no setup.py:

pip install file://$PWD/pkgdir works, but pip install ./pkgdir complains that Neither 'setup.py' nor 'pyproject.toml' found..

The "fix" for that would be easy (in is_installable_dir()), but I'm not sure how far these ripples will go...

Originally posted by @sbidoul in #9945 (comment)

Expected behavior

$ echo '[metadata]\nname=foo' > setup.cfg && pip install --use-pep517 file:///$(pwd)
ERROR: Directory '.' is not installable. Neither 'setup.py' nor 'pyproject.toml' found.

pip version

pip 21.2.4 from /home/graingert/.virtualenvs/testing39/lib/python3.9/site-packages/pip (python 3.9)

Python version

Python 3.9.7

OS

Ubuntu 20.04.3 LTS

How to Reproduce

$ echo '[metadata]\nname=foo' > setup.cfg && pip install --use-pep517 file:///$(pwd)
Processing //home/graingert/projects/foo
  DEPRECATION: A future pip version will change local packages to be built in-place without first copying to a temporary directory. We recommend you use --use-feature=in-tree-build to test your packages with this new behavior before it becomes the default.
   pip 21.3 will remove support for this functionality. You can find discussion regarding this at https://github.com/pypa/pip/issues/7555.
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Building wheels for collected packages: foo
  Building wheel for foo (PEP 517) ... done
  Created wheel for foo: filename=foo-0.0.0-py3-none-any.whl size=933 sha256=b0ec4bd32d639456067a52c0297824efbc46d9a1f7b3894c63106ed25c9168fa
  Stored in directory: /tmp/pip-ephem-wheel-cache-qxna4rsa/wheels/4c/98/64/53a57a6326621aeffd6d95082ac196bdb3e2a8459e121fa09a
Successfully built foo
Installing collected packages: foo
  Attempting uninstall: foo
    Found existing installation: foo 0.0.0
    Uninstalling foo-0.0.0:
      Successfully uninstalled foo-0.0.0
Successfully installed foo-0.0.0

Output

No response

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[graingert]

there's also this on the setuptools end pypa/setuptools#2329

[sbidoul]

Hi @graingert

My comment you mentioned above has now been addressed: both pip install file://$PWD/pkgdir and pip install ./pkgdir fail when there is no pyproject.toml nor setup.py.

When enforcing the use of pep 517 in such situation, the spec says this:

If the pyproject.toml file is absent, or the build-backend key is missing,
the source tree is not using this specification, and tools should revert to the legacy behaviour
of running setup.py (either directly, or by implicitly invoking the setuptools.build_meta:__legacy__ backend).

So when --use-pep517 is set and there is no pyproject.toml, pip uses the default build system table and builds using the setuptools.build_meta:__legacy__ build backend, which does support setup.cfg-only configurations.

So I think the current behaviour is correct.

[graingert]

@sbidoul do you know when this was fixed because I can still reproduce this

[sbidoul]

I just tested with the pip main branch. It might have been fixed as part of #8212

[graingert]

I tried it against pip installed via git and got this:

 ✘  graingert@onomastic  testing39  ~/projects/foo  pip install -U git+https://github.com/pypa/pip.git
Collecting git+https://github.com/pypa/pip.git
  Cloning https://github.com/pypa/pip.git to /tmp/pip-req-build-m1oeci5v
  Running command git clone -q https://github.com/pypa/pip.git /tmp/pip-req-build-m1oeci5v
  Resolved https://github.com/pypa/pip.git to commit 7616583dbb2dcbda5a19d78873642d6751fbf017
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Building wheels for collected packages: pip
  Building wheel for pip (PEP 517) ... done
  Created wheel for pip: filename=pip-21.3.dev0-py3-none-any.whl size=1719972 sha256=f575d0ae1f0c372b5be154eeaa58063d01aefd086f528a50ebba8a089f6fe54c
  Stored in directory: /tmp/pip-ephem-wheel-cache-dsnjyudn/wheels/fb/cc/15/cf3753138466b998bef3949351cf6218eb857a9f27aa0be84a
Successfully built pip
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 21.2.4
    Uninstalling pip-21.2.4:
      Successfully uninstalled pip-21.2.4
Successfully installed pip-21.3.dev0
 graingert@onomastic  testing39  ~/projects/foo  echo '[metadata]\nname=foo' > setup.cfg && pip install --use-pep517 file:///$(pwd)                                         
Processing //home/graingert/projects/foo
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing wheel metadata (pyproject.toml) ... done
Building wheels for collected packages: foo
  Building wheel for foo (pyproject.toml) ... done
  Created wheel for foo: filename=foo-0.0.0-py3-none-any.whl size=933 sha256=ac036b06383d468d783199debe6aade285a5f9f806540901297c422e89b738c9
  Stored in directory: /tmp/pip-ephem-wheel-cache-xxfz2l75/wheels/4c/98/64/53a57a6326621aeffd6d95082ac196bdb3e2a8459e121fa09a
Successfully built foo
Installing collected packages: foo
  Attempting uninstall: foo
    Found existing installation: foo 0.0.0
    Uninstalling foo-0.0.0:
      Successfully uninstalled foo-0.0.0
Successfully installed foo-0.0.0
 ✘  graingert@onomastic  testing39  ~/projects/foo  ls
build  foo-0.0.0-py3-none-any.whl  foo.egg-info  setup.cfg  _setup.py
 graingert@onomastic  testing39  ~/projects/foo  echo '[metadata]\nname=foo' > setup.cfg && pip install --use-pep517 .
ERROR: Directory '.' is not installable. Neither 'setup.py' nor 'pyproject.toml' found.

eg the issue is still present and setup.py-less builds are supported with file:/// uris and rejected with "looks-like-path" args

[sbidoul]

Ok, I see, I had not tried the combination of --use-pep517 and a local directory URL. Surprising. I'll have a look.

[layday]

So when --use-pep517 is set and there is no pyproject.toml, pip uses the default build system table and builds using the setuptools.build_meta:__legacy__ build backend, which does support setup.cfg-only configurations.

I think you're misinterpreting the PEP here. The fallback applies only to legacy projects. A project is considered legacy only if it has a setup.py. If a project is legacy you can either (a) invoke setup.py directly or (b) invoke the __legacy__ backend via PEP 517. The quoted paragraph is poorly worded and the information is scattered between the two PEPs, but I don't think it's the intention that the build-backend should default to setuptool's __legacy__ backend in the absence of both pyproject.toml and setup.py.

Worth nothing that pypa/build rejects pyproject.toml-less projects without a setup.py, so pip is able to build projects that build can't.

[layday]

Edited my message above since I'm not so sure myself anymore after rereading the PEP...

[sbidoul]

Edited my message above since I'm not so sure myself anymore after rereading the PEP...

Yep, same with me, it's not obvious.

When we don't say --use-pep517, it's fine to reject it, I agree.
When we enforce the use of PEP 517, my instinct would be to use the default build-system and delegate to setuptools.build_meta:__legacy__ to build, leaving the responsibility to do the right thing to it.

I don't have a strong opinion on this, though.

[sbidoul]

This test is currently done in constructors.py, _get_url_from_path. Purely from a pip internals perspective I'd say this function should not be responsible to do such a check, if only looking at it's name.

[FFY00]

Relevant: #9945 (comment)

[layday]

When we don't say --use-pep517, it's fine to reject it, I agree.
When we enforce the use of PEP 517, my instinct would be to use the default build-system and delegate to setuptools.build_meta:__legacy__ to build, leaving the responsibility to do the right thing to it.

My thinking is that the fallback is there to help with the transition from the setuptools way of doing things to PEP 517. We don't want to elevate one specific backend above the rest for all eternity. If there's no setup.py and build-backend is absent, and pip (and build) does not error when forcing PEP 517, people will inevitably come to depend on PEP 517 defaulting to setuptools, which would be rather counterproductive. The wording's a bit muddled in the PEP but we should be able to agree on the intent.

[sbidoul]

But then if setuptools.build_meta:__legacy__ must be interpreted strictly as a way to invoke setup.py, then why does it work without setup.py ?

So I'd be inclined to remove the check from pip's _get_url_from_path. And if it is the consensus that the __legacy__ backend must only supports setup.py, then enforce that in setuptools.build_meta:__legacy__, or alternatively in pep517 (that we vendor and is the part that creates the default build-system table) [updated: we don't rely on pep517 to set the default build-system table].

[sbidoul]

Also, testing for the presence of pyproject.toml is not sufficient ? Because a pyproject.toml without a build-system table also means using the default backend ?

[FFY00]

No, the project should have a build-system table. pyproject.toml is not only used by packages, it is also used for tool configurations. It is not tied to packages.

[sbidoul]

I'm aware that pyproject.toml can be used for tool configuration. But its mere presence does trigger the pep517 build code path in pip, whether there is a build-system table in it or not.

[sbidoul]

For that matter, pip install --use-pep517 of an empty directory does also trigger the __legacy__ build backend, which happily builds UNKNOWN-0.0.0 :) So yeah I'm now quite convinced that a fix has to be done in setuptools.

[FFY00]

But then if setuptools.build_meta:__legacy__ must be interpreted strictly as a way to invoke setup.py, then why does it work without setup.py ?

That is merely coincidental.

For that matter, pip install --use-pep517 of an empty directory does also trigger the __legacy__ build backend, which happily builds UNKNOWN-0.0.0 :)

Which sounds like a bug to me 😅

So yeah I'm now quite convinced that a fix has to be done in setuptools.

I disagree, setuptools could change the behavior in order to mitigate this, but nobody should be relying on this in the first place.

[pfmoore]

For that matter, pip install --use-pep517 of an empty directory does also trigger the legacy build backend, which happily builds UNKNOWN-0.0.0 :)

Which sounds like a bug to me 😅

A bug in setuptools, yes. The user asked for PEP 517 behaviour, which explicitly states that if pyproject.toml is missing, the setuptools legacy backend should be used. It's up to that backend to check that it's being invoked in a directory which has whatever it needs to build a wheel.

I disagree, setuptools could change the behavior in order to mitigate this, but nobody should be relying on this in the first place.

It's not a matter of whether people rely on it, it's a matter of backends having decent error handling. I consider "I did something wrong and the tool produced garbage rather than spotting it and telling me" a legitimate bug report (against setuptools in this case).

I'm not sure where this discussion is going any more. And I don't think that abstract arguments about what is whose issue are that helpful. The title of the issue says "with --use-pep517", but the report is talking about cases where --use-pep517 is not specified (and hence pip works out whether to use PEP 517 itself). Can someone please confirm what the actual behaviour is that is being claimed to be incorrect here? Then we can focus on that.

[graingert]

For that matter, pip install --use-pep517 of an empty directory does also trigger the legacy build backend, which happily builds UNKNOWN-0.0.0 :)

Which sounds like a bug to me 😅

A bug in setuptools, yes. The user asked for PEP 517 behaviour, which explicitly states that if pyproject.toml is missing, the setuptools legacy backend should be used. It's up to that backend to check that it's being invoked in a directory which has whatever it needs to build a wheel.

I disagree, setuptools could change the behavior in order to mitigate this, but nobody should be relying on this in the first place.

It's not a matter of whether people rely on it, it's a matter of backends having decent error handling. I consider "I did something wrong and the tool produced garbage rather than spotting it and telling me" a legitimate bug report (against setuptools in this case).

I'm not sure where this discussion is going any more. And I don't think that abstract arguments about what is whose issue are that helpful. The title of the issue says "with --use-pep517", but the report is talking about cases where --use-pep517 is not specified (and hence pip works out whether to use PEP 517 itself). Can someone please confirm what the actual behaviour is that is being claimed to be incorrect here? Then we can focus on that.

I ran this echo '[metadata]\nname=foo' > setup.cfg && pip install --use-pep517 file:///$(pwd) and it should behave same as echo '[metadata]\nname=foo' > setup.cfg && pip install --use-pep517 .`

[sbidoul]

I think I found the next best place to fix this (#10534) - the best being in setuptools, IMO.

[layday]

It's not a matter of whether people rely on it, it's a matter of backends having decent error handling. I consider "I did something wrong and the tool produced garbage rather than spotting it and telling me" a legitimate bug report (against setuptools in this case).

This is not generic to all backends having "decent error handling" - it's specific to setuptools, because only setuptools is invoked implicitly. I think we're struggling to stay on top of when or not to invoke the legacy setuptools backend. There's at least six different variables to consider and all of their combinations: has setup.py, has pyproject.toml, has build-table, has requires, has build-backend, and --[no-]use-pep517. Something has to give - either we invoke setuptools implicitly in fewer situations, or if we can remove the first variable, by delegating the setup.py existence check to the setuptools legacy backend, then we should probably do that.

[pfmoore]

This is not generic to all backends having "decent error handling" - it's specific to setuptools, because only setuptools is invoked implicitly.

OK, it's related to setuptools having decent error handling. Is there any non-contrived reason why setuptools shouldn't be telling the user that they made a mistake if they invoke setuptools on an empty directory?

We seem to be assuming that setuptools don't want to fix this - and I don't know why. Has anyone tried reporting this as a bug to them?

I think we're struggling to stay on top of when or not to invoke the legacy setuptools backend.

Agreed.

Something has to give - either we invoke setuptools implicitly in fewer situations, or if we can remove the first variable, by delegating the setup.py existence check to the setuptools legacy backend, then we should probably do that.

Or we focus on getting the things that matter right, and don't worry about weird corner cases unless someone hits them in a real-world situation 🙂

To be clear, though, the whole --use-pep517 business was a bit of a hack when I first implemented it. There are a lot of different cases, and they are messy. I'd welcome something cleaner. But I think if we want to do that, we should write down the rules we're following, and not just fix things as we find them - otherwise, as you say, we just end up getting bogged down in confusion, and we'll keep going round this whole loop.

[layday]

Or we focus on getting the things that matter right, and don't worry about weird corner cases unless someone hits them in a real-world situation 🙂

Well, it did start out as a weird corner case, but it wasn't clear what to do when passing --use-pep517 (which pypa/build always does!), hence the ensuing discussion. If we relax the check to let setup.py-less source trees through in less cornery cases, that will have some decently far-reaching consequences. But I agree, we should formalise these rules.

[pfmoore]

but it wasn't clear what to do when passing --use-pep517 (which pypa/build always does!)

I'd be curious to know why you do that. I suspect it's intended to make sure you always get PEP 517 behaviour, but that was never the original intention of that flag. The original point was that pip was applying the rule from PEP 517 itself, "If the pyproject.toml file is absent, or the build-backend key is missing, the source tree is not using this specification" and so in those cases we were doing "something else"¹. The --use-pep517 flag was simply a way to override that and try to use PEP 517 (and the other behaviours that it triggers in pip, like build isolation) in any case (by doing what PEP 517 says² and using the setuptools legacy backend)

¹ If the PEP says "this spec doesn't apply in this case" then you can do whatever you like, by definition.
² It's actually quite weird that PEP 517 says what you should do when PEP 517 doesn't apply. But ho hum, it's useful to have that clarification, so I'm not complaining 🙂

[graingert]

² It's actually quite weird that PEP 517 says what you should do when PEP 517 doesn't apply. But ho hum, it's useful to have that clarification, so I'm not complaining

The purpose of the system is what it does

[layday]

I'd be curious to know why you do that.

We do it because we don't pre-install setuptools in the isolated build environment so build dependencies without wheels which haven't "opted into" PEP 517 can't be built. This was brought up in pypa/build#231, where flit tried to install every single runtime dependency in the build environment, because it could not parse the AST of the module to extract the version, as I recall.

[pradyunsg]

Oh, wow. This has blown up. I'm surprised everyone is suddenly so concerned about this. Of all the things I was expecting would get attention this wekeend on this issue tracker, this was not on that list. XD

I don't think this is a particularly useful discussion, so I'm gonna keep my participation limited -- don't expect me to reply to any questions, sorry. What follows, is my perspective on this.

---

I am in favour of adding a check for setup.cfg in the is_installable_dir and changing nothing else, as well as the PR that's currently open.

For setuptools, even an empty directory is a valid project. It will package up literally any directory on the filesystem and ship that to the user. If you want to complain about that behaviour, go here: pypa/setuptools#2329

I definitely prefer that that pip continues to protect users from setuptools' shitty behaviour, whenever we can, without going overboard. The directory case is a simple one to do this in, given that we already do so.

As far as I'm concerned, is_installable_dir is a hueristic, to protect our users from trying to run pip in an inappropriate directory. This is especially relevant in the PEP 517 case, thanks to setuptools' shitty behavior -- see link above. If it were to model the reality of what setuptools accepts, it should basically "return True" in the PEP 517 case -- setuptools will happily try to package up any directory into a wheel.

If pip install . in a random directory packages up the whole directory, and dumps that into a Python's site-packages, that's... horrible. I don't think we should have that behaviour.

I care much less about pip install file:///blah/foo and pip install sdist.tar.gz -- there's only so much protection that we should provide from setuptools' shitty behaviour here and they're much less common. I could be convinced that we should have these checks there too -- and I'd be happy to merge such a PR -- I probably won't be authoring such a PR though.

[henryiii]

Nice to see consistency here. If it helps future readers (I just found out about this discussion from the changelog, I was part of some of the earlier ones), this is the argument that lead me to agree with @FFY00 that PEP 517 requires setup.py to be present if pyproject.toml is missing:

If the pyproject.toml file is absent, or the build-backend key is missing, the source tree is not using this specification, and tools should revert to the legacy behaviour of running setup.py (either directly, or by implicitly invoking the setuptools.build_meta:__legacy__ backend).

This says a valid build backend could either choose to run setup.py directly or invoke setuptools.build_meta:__legacy__. Therefore, a valid PEP 517 builder could exist that runs setup.py if pyproject.toml is missing. Most prefer setuptools.build_meta:__legacy__, but it's not required. So therefore, a pure setup.cfg project is invalid under PEP 517.

And yes, this is making statements about the PEP 517 validity of a project that does not follow PEP 517.¹

Footnotes

1. @pfmoore, FYI, since you are adding your own footnotes above, GitHub now allows real markdown footnotes. :) ↩

[FFY00]

The pyproject.toml presence does not really mean anything, the build-system table presence in pyproject.toml does 😄

[pradyunsg]

FYI, since you are adding your own footnotes above, GitHub now allows real markdown footnotes. :)

Woah.

The pyproject.toml presence does not really mean anything, the build-system table presence in pyproject.toml does 😄

Are you sure? ;)

[pfmoore]

GitHub now allows real markdown footnotes.

Does it? ¹

OK, enough silliness for today 🙂

Footnotes

1. Ooh, yes it does²! ↩

2. I'm quite attached to my autohotkey macro for "superscript 1", but having proper support is good³ ↩

3. Unfortunately, it looks like Discourse doesn't support footnotes, so I may stick with doing them manually for consistency. ↩

[pradyunsg]

https://discuss.python.org/t/footnote-markdown-plugin/7845/4 has been poked!

[FFY00]

Are you sure? ;)

I am. pip's implementation and what the PEP says are different things. pip does not follow the PEP in this detail, it says

If the pyproject.toml file is absent, or the build-backend key is missing, the source tree is not using this specification, and tools should revert to the legacy behaviour of running setup.py (either directly, or by implicitly invoking the setuptools.build_meta:legacy backend).

Which is clear about a pyproject.toml that does not specify the backend being a legacy project, but I am not a pip maintainer so 🤷

[FFY00]

Users might want to use configure a tool in pyproject.toml but still use a legacy project, in which case pip will misbehave. I don't think it's a big deal since this use-case is not common, but pip is not following the PEP.

[pradyunsg]

Feel free to read distutils-sig about this.

[pradyunsg]

FWIW, I tend to agree with you, but I couldn’t convince folks that this was a bad idea, and you seem to have a better success rate than I do about these things. :)

[henryiii]

I know some people (cough, @asottile, cough) who hate pyproject.toml for this exact reason. I didn't realize the wording was correct in the PEP, and it's only been the implementations that have been broken, though! Hopefully in the near future, once PEP 517 is on by default, it won't really matter, but interesting that it was specified correctly¹ in the PEP.

in which case pip will misbehave

This is one of the three reasons flake8 does not have a pyproject.toml config option; if it moved to only having a single pyproject.toml file for config, it would break users who need the old behavior. The other two reasons are actually no longer true, AFAICK, there is a 1.0 release of the TOML standard, and tomli has become the "accepted" implementation (although not for very long at this point).

Footnotes

1. Correctly meaning adding a file to configure Black shouldn't change your build system and cause it to do something different. ↩

[uranusjr]

See https://discuss.python.org/t/2725. If pip's behaviour has changed between then and now, it should be treated as a bug. But if it didn't, then either the proposed amendment or your interpretation is wrong, because the current text was written to exactly describe pip's behaviour (at the time).

[layday]

Yes, the quoted passage from 517 should be read in conjunction with the passage from PEP 518, and even then, it's not easy to follow without knowing the background.

[henryiii]

The code there looks about right, maybe adding a pyproject.toml without a build-backend now has no effect on the build process in pip 20.0+? I never have pyproject.toml's without build-backends, so it's not something I test.

[FFY00]

I think the damage is done at this point, even if we fix pip's behavior, nobody can rely on it as there will still be tons of users stuck with older versions of pip. We would be creating divergence and introducing breakage. It could be worth, but I am not certain that it would, and I don't have any personal stake in this to motivate me to drive it forward.

If someone else wants to try starting a discussion on this with the goal of fixing the behavior, I'd be supportive.