Run a command from a [pipx.run] entry point #615

uranusjr · 2021-01-29T04:33:03Z

Supercedes #602.

I have added an entry to docs/changelog.md

Summary of changes

Test plan

Tested by running

pipx run --spec="build @ https://github.com/uranusjr/build/archive/pipx-run.zip" build --version

uranusjr · 2021-01-29T04:36:51Z

TODO: Write documentation on this.

henryiii · 2021-01-29T16:32:19Z

src/pipx/venv.py

@@ -34,6 +40,16 @@

 logger = logging.getLogger(__name__)

+# This regex was only introduced into importlib.metadata in Python 3.9.


You could use sys.version_info < (3, 9) instead of the try/except above, then require importlib_metadata for < 3.9 instead of < 3.8. Just a thought.

IMO the regex pattern is well-specified and stable and requring importlib-metadata on Python 3.8 is not worth it.

itsayellow · 2021-01-30T01:50:19Z

Can you give an example of what a setup.cfg would look like to define a pipx run entry point?

uranusjr · 2021-01-30T01:52:52Z

This is the configuration I’m using to test this (against pypa/build):

https://github.com/uranusjr/build/blob/5bcd9f2fc6b3560e68e907352064466fe3253de2/setup.cfg#L39-L43

itsayellow · 2021-01-30T02:24:27Z

OK, thanks.

So the idea is that one would need to make sure that the packages PyPI Package Name is the name of the pipx.run endpoint? Like:

pipx.run =
    <pypi_package_name> = <package_name>.__main__:entrypoint

Is there a way to eliminate <pypi_package_name> from this mechanism? So that people have one less opportunity to make a mistake and make this stop working for them, when really there's only one possible value for that string?

uranusjr · 2021-01-30T02:39:57Z

Unfortunately, each entry point must have a group and a name, so it’s not possible to write

[options.entry_points]
pipx.run = <import-path>

The closest possible is something like (using pipx as the group name and run as the key)

[options.entry_points]
pipx =
   run = <import-path>

or (using an empty string as key)

[options.entry_points]
pipx.run =
    = <import-path>

henryiii · 2021-01-30T02:44:08Z

That's how entry points work. What you would like to do is write:

[options.entry_points]
console_scripts =
     build = build.__main__:entrypoint

But that would be too common of a name and would break things. So now you can write:

[options.entry_points]
console_scripts =
    pyproject-build = build.__main__:entrypoint
pipx.run =
    build = build.__main__:entrypoint

Now only "pyproject-build" shows up in the console, but pipx run build works. It's exactly the same thing as what you'd put in console_scripts. By the way, there's no requirement that a package only supply it's PyPI username as a package (even though very common - think of something like pkg_resources from setuptools), and you could even have a custom entrypoint that could only be accessed by run --spec build my-special-pipx-command or something.

henryiii · 2021-01-30T02:47:18Z

Using run or an empty string (or anything not unique to the package) would likely be a disaster. If you have a dependency that also had a pipx run entry point, the two would be merged into an unordered list (I believe - that's how Numba works). You want the similarity to console_scripts.

henryiii · 2021-01-30T02:55:08Z

You are also not limited to matching the package name; you could add multiple commands. For example, if a project did not want to provide console_scripts (say it only wanted to provide a python -m interface (cough, rich, cough, cough), it could provide a series of pipx.run entry points, not just it's package name, accessible via --spec <package_name> <pipx.run entrypoint name>.

uranusjr · 2021-01-30T02:56:15Z

importlib.metadata has the information which entry point comes from which package, so it's possible to choose the correct entry point when there are multiple matches. The implementation would be a bit more complicated than the current PR, but it's doable.

henryiii · 2021-01-30T02:58:03Z

I didn't know that; good to know! I still think that the current implementation is correct, since you are literally just proving a console_scripts for pipx run only, but some of my arguments are no longer valid. I still think the strongest point is the symmetry with console_scripts - authors don't need to learn something new.

uranusjr · 2021-01-30T03:20:39Z

Yeah, I think the current approach is more balanced, simple to implement while also easy enough for the user to use. The user experience is not perfect, but improvements would also bring drawbacks that may prove problematic.

itsayellow · 2021-01-30T04:12:17Z

Yeah, I think the current approach is more balanced, simple to implement while also easy enough for the user to use. The user experience is not perfect, but improvements would also bring drawbacks that may prove problematic.

That sounds right, I was just wondering if there was an easy way to eliminate what could be a possible gotcha for package creators. Naming it properly should probably be pointed out in the docs.

itsayellow · 2021-01-31T06:45:10Z

So, just to double-check on how the interface is working...

If build has a pipx.run endpoint, then I'm assuming the following will not work:

pipx run bUiLd --help

It will download and install build properly from PyPI (I believe) but it won't match the likely pipx.run endpoint name of "build".

I feel like we should canonicalize the package name and/or endpoint name before matching them because of this, to avoid trouble with any random capitalization or punctuation for example.

uranusjr · 2021-01-31T07:13:58Z

I believe the same is true with console scripts as well though? I agree canonicalisation is the right thing to do, but it is arguably a separate enhancement.

itsayellow · 2021-01-31T19:37:00Z

I believe the same is true with console scripts as well though? I agree canonicalisation is the right thing to do, but it is arguably a separate enhancement.

The difference to my mind is that console scripts are between the packager and the user only, but here pipx is part of it. And the other parts of pipx behave in a way that different capitalizations, punctuations don't matter if they are equivalent canonically.

EDIT: Oh I see, you mean console scripts in conjunction with pipx run would face the same issue. OK, yes that's true. I guess it is a general separate issue.

uranusjr · 2021-02-01T03:11:06Z

I filed #618 for the name normalisation issue.

itsayellow · 2021-02-04T02:02:59Z

TODO: Write documentation on this.

Does this mean you're working on documentation for this PR?

uranusjr · 2021-02-04T02:21:51Z

I’m not actively writing the documentation for this.

itsayellow · 2021-02-07T23:05:54Z

src/pipx/venv.py

+        if not self.python_path.exists():
+            return None
+        site_packages = get_site_packages(self.python_path)
+        for dist in Distribution.discover(path=[str(site_packages)]):


How about limiting our check for the pipx.run entrypoint to only the main package?

Otherwise if we found a dependency with a pipx.run entrypoint having the app name, we would match it instead. This sounds like a dumb thing to happen, but I've seen dumb things in the past (like a package using two dependencies that both use the same name for console scripts).

~~Good idea, added.~~ (Reverted, see below.)

Hmm, this does not really work well after I think about this more, because pipx run does not really know what the “main package” is in a cached venv. It’s wrong to use the app argument, that would make usages like pipx run tox4 --spec "tox>=4a0" fail.

Don't we have that information in the metadata? e.g. self.main_package_name.

You could even use pipx.venv_inspect.get_dist using self.main_package_name as the first argument.

Oh, I didn’t know that! Let me try it out.

That seems to work. PR updated.

Is importlib.metadata.Distribution.discover() documented anywhere? I'm only finding it by browsing the importlib.metadata source. It would probably be useful to use in pipx.venv_inspect.

Also, just out of curiosity, is there a way you would get multiple dists from from Distribution.discover() if you searched for the name of the package?

I don’t think Distribution.discover() is documented. The author told me about it when I was investigating the possibility of using importlib.metadata in pip (to replace pkg_resources).

Is there a way you would get multiple dists from from Distribution.discover() if you searched for the name of the package?

Yes, because it is possible to have multiple installations of a package in different sys.path entries. For example:

pip install build pip install --user build

would give you two build distributions, one under the system site-packages, and the other in the per-user site-packages. Another possible case is enabling system site-packages in a virtual environment. Neither applies for pipx run though.

itsayellow

LGTM. Of course still need to fix the conflict but otherwise, nice!

henryiii · 2021-03-03T00:30:14Z

Adding docs in "developing for pipx" would be nice.

ghost

Good

src/pipx/venv.py

uranusjr mentioned this pull request Jan 29, 2021

Run a command from a [pipx.run] entry point #602

Closed

1 task

uranusjr force-pushed the pipx-run branch from 0174f42 to 8e240d7 Compare January 29, 2021 04:41

henryiii reviewed Jan 29, 2021

View reviewed changes

uranusjr force-pushed the pipx-run branch from 5aca07f to 23f12b6 Compare January 30, 2021 01:30

uranusjr mentioned this pull request Feb 1, 2021

Normalise entry point and package name to make 'pip run' more permissive #618

Open

itsayellow reviewed Feb 7, 2021

View reviewed changes

uranusjr force-pushed the pipx-run branch from 015f1c7 to 23f12b6 Compare February 8, 2021 07:51

itsayellow approved these changes Feb 9, 2021

View reviewed changes

uranusjr added 4 commits February 9, 2021 16:01

Extract app existence check into function

3e4b86c

Implement [pipx.run] entry point support

3162d2c

Better description of what we want

cebdc53

Search entry points only in the main package

6ffe136

uranusjr force-pushed the pipx-run branch from dc0afb9 to 6ffe136 Compare February 9, 2021 08:02

uranusjr merged commit 7ce040f into pypa:master Feb 24, 2021

uranusjr deleted the pipx-run branch February 24, 2021 23:55

ghost reviewed Apr 2, 2021

View reviewed changes

This was referenced Apr 12, 2021

chore: use newer pipx run syntax scientific-python/cookie#11

Merged

[feature] Support pipx run with different name in entry point natively if a package wants to support it #600

Closed

chore: use newer pipx run syntax scikit-hep/scikit-hep.github.io#127

Merged

uranusjr mentioned this pull request Sep 9, 2021

Run console scripts whose name doesn't match package name #725

Closed

matthewfeickert mentioned this pull request Apr 24, 2024

Add documentation for [pipx.run] entry point #1371

Open

chrysle reviewed May 20, 2024

View reviewed changes

src/pipx/venv.py Show resolved Hide resolved

chrysle mentioned this pull request May 21, 2024

Fix pipx run entry point discovery with local path #1422

Merged

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Run a command from a [pipx.run] entry point #615

Run a command from a [pipx.run] entry point #615

uranusjr commented Jan 29, 2021 •

edited

Loading

uranusjr commented Jan 29, 2021

henryiii Jan 29, 2021

uranusjr Jan 30, 2021 •

edited

Loading

itsayellow commented Jan 30, 2021

uranusjr commented Jan 30, 2021

itsayellow commented Jan 30, 2021 •

edited

Loading

uranusjr commented Jan 30, 2021 •

edited

Loading

henryiii commented Jan 30, 2021 •

edited

Loading

henryiii commented Jan 30, 2021 •

edited

Loading

henryiii commented Jan 30, 2021

uranusjr commented Jan 30, 2021

henryiii commented Jan 30, 2021

uranusjr commented Jan 30, 2021

itsayellow commented Jan 30, 2021

itsayellow commented Jan 31, 2021 •

edited

Loading

uranusjr commented Jan 31, 2021

itsayellow commented Jan 31, 2021 •

edited

Loading

uranusjr commented Feb 1, 2021

itsayellow commented Feb 4, 2021

uranusjr commented Feb 4, 2021

itsayellow Feb 7, 2021

uranusjr Feb 8, 2021 •

edited

Loading

uranusjr Feb 8, 2021

itsayellow Feb 8, 2021

itsayellow Feb 8, 2021

uranusjr Feb 9, 2021

uranusjr Feb 9, 2021

itsayellow Feb 9, 2021

itsayellow Feb 9, 2021

uranusjr Feb 9, 2021 •

edited

Loading

itsayellow left a comment

henryiii commented Mar 3, 2021

ghost left a comment

		@@ -34,6 +40,16 @@

		logger = logging.getLogger(__name__)

		# This regex was only introduced into importlib.metadata in Python 3.9.

Run a command from a [pipx.run] entry point #615

Run a command from a [pipx.run] entry point #615

Conversation

uranusjr commented Jan 29, 2021 • edited Loading

Summary of changes

Test plan

uranusjr commented Jan 29, 2021

Choose a reason for hiding this comment

uranusjr Jan 30, 2021 • edited Loading

Choose a reason for hiding this comment

itsayellow commented Jan 30, 2021

uranusjr commented Jan 30, 2021

itsayellow commented Jan 30, 2021 • edited Loading

uranusjr commented Jan 30, 2021 • edited Loading

henryiii commented Jan 30, 2021 • edited Loading

henryiii commented Jan 30, 2021 • edited Loading

henryiii commented Jan 30, 2021

uranusjr commented Jan 30, 2021

henryiii commented Jan 30, 2021

uranusjr commented Jan 30, 2021

itsayellow commented Jan 30, 2021

itsayellow commented Jan 31, 2021 • edited Loading

uranusjr commented Jan 31, 2021

itsayellow commented Jan 31, 2021 • edited Loading

uranusjr commented Feb 1, 2021

itsayellow commented Feb 4, 2021

uranusjr commented Feb 4, 2021

Choose a reason for hiding this comment

uranusjr Feb 8, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

uranusjr Feb 9, 2021 • edited Loading

Choose a reason for hiding this comment

itsayellow left a comment

Choose a reason for hiding this comment

henryiii commented Mar 3, 2021

ghost left a comment

Choose a reason for hiding this comment

uranusjr commented Jan 29, 2021 •

edited

Loading

uranusjr Jan 30, 2021 •

edited

Loading

itsayellow commented Jan 30, 2021 •

edited

Loading

uranusjr commented Jan 30, 2021 •

edited

Loading

henryiii commented Jan 30, 2021 •

edited

Loading

henryiii commented Jan 30, 2021 •

edited

Loading

itsayellow commented Jan 31, 2021 •

edited

Loading

itsayellow commented Jan 31, 2021 •

edited

Loading

uranusjr Feb 8, 2021 •

edited

Loading

uranusjr Feb 9, 2021 •

edited

Loading