Replaced autocommand usage with Typer

[Avasam]

One option to address pypa/setuptools#5045 is to replace autocommand (LGPLv3) usage with Typer (MIT).

Also relates to pypa/setuptools#5049

Alternative to, and closes #24 + closes #26

This would also help with #23

[abravalheri]

(thanks @Avasam).

typer is a bit of a "heavy weight" in terms of transient dependencies (click, typing-extensions, shellingham and rich). From the point of view of setuptools, I suppose that #26 or #24 would make more sense...

[Avasam]

I totally agree, hence I opened #24 & #26
I still opened this for the sake of completeness since it was one option previously discussed.

[jaraco]

My preference is to use Typer. I've been porting commands in other projects from autocommand to Typer, so it aligns well with those efforts. If only our packaging ecosystem had negative dependencies (optional dependencies installed by default), the typer dependency could be excluded by setuptools when relying on jaraco.text. Unfortunately, we don't, so we're stuck with the heavy dependencies.

Another option could be to separate the scripts from the library - and expose the scripts in another package like jaraco.develop or jaraco.commands (does not exist). If it were possible to create jaraco.text.scripts and have it extend (and depend on) jaraco.text, I'd probably do that, but sadly, a module like jaraco.text can't both have behavior and be a namespace, so it wouldn't be possible to keep the same invocation py -m jaraco.text.to-qwerty.

I've dreamed of a unified import behavior that would allow any module or package to be a namespace (and have submodules), such that there's no difference between a package and a namespace package. I don't know that I'll ever have the energy to propose it and work through the issues.

Maybe the best thing to do is find a word out there that can host the core library behavior of jaraco.text.

[jaraco]

What I really want here is something that also accepts - and converts that to stdin. I know that's out of scope for this change, but if typer has a facility for that, I'd like to use it. Sounds like typer.FileText is the thing.

[jaraco]

I started working on a solution using typer.FileText. I had this:

diff --git a/jaraco/text/__init__.py b/jaraco/text/__init__.py
index e786f8a..b80e258 100644
--- a/jaraco/text/__init__.py
+++ b/jaraco/text/__init__.py
@@ -2,12 +2,23 @@ from __future__ import annotations
 
 import functools
 import itertools
+import os
 import re
 import sys
 import textwrap
 from collections.abc import Callable, Generator, Iterable, Sequence
 from importlib.resources import files
-from typing import TYPE_CHECKING, Literal, Protocol, SupportsIndex, TypeVar, overload
+from typing import (
+    TYPE_CHECKING,
+    Literal,
+    Protocol,
+    SupportsIndex,
+    TextIO,
+    TypeAlias,
+    TypeVar,
+    cast,
+    overload,
+)
 
 from jaraco.context import ExceptionTrap
 from jaraco.functools import compose, method_cache
@@ -18,7 +29,7 @@ else:  # pragma: no cover
     from importlib.abc import Traversable
 
 if TYPE_CHECKING:
-    from _typeshed import FileDescriptorOrPath, SupportsGetItem
+    from _typeshed import SupportsGetItem
     from typing_extensions import Self, TypeAlias, TypeGuard, Unpack
 
     _T_co = TypeVar("_T_co", covariant=True)
@@ -681,9 +692,18 @@ def join_continuation(lines: _GetItemIterable[str]) -> Generator[str]:
         yield item
 
 
+Openable: TypeAlias = str | bytes | os.PathLike[str] | os.PathLike[bytes] | int
+
+
+@functools.singledispatch
 def read_newlines(
-    filename: FileDescriptorOrPath, limit: int | None = 1024
-) -> str | tuple[str, ...] | None:
+    filename: Openable | TextIO, limit: int | None = 1024
+) -> tuple[str, ...]:
+    return ()  # unreachable
+
+
+@read_newlines.register
+def _(filename: Openable, limit: int = 1024) -> tuple[str, ...]:
     r"""
     >>> tmp_path = getfixture('tmp_path')
     >>> filename = tmp_path / 'out.txt'
@@ -698,8 +718,13 @@ def read_newlines(
     ('\r', '\n', '\r\n')
     """
     with open(filename, encoding='utf-8') as fp:
-        fp.read(limit)
-    return fp.newlines
+        return read_newlines(fp, limit=limit)
+
+
+@read_newlines.register
+def _(filename: TextIO, limit: int = 1024) -> tuple[str, ...]:
+    filename.read(limit)
+    return cast(tuple[str, ...], filename.newlines)
 
 
 def lines_from(input: Traversable) -> Generator[str]:
diff --git a/jaraco/text/show-newlines.py b/jaraco/text/show-newlines.py
index e618f78..2a91f7b 100644
--- a/jaraco/text/show-newlines.py
+++ b/jaraco/text/show-newlines.py
@@ -7,10 +7,7 @@ from more_itertools import always_iterable
 import jaraco.text
 
 
-# filename is technically a FileDescriptorOrPath, but Typer doesn't support Unions yet
-# and this file is script-only (not importable) anyway.
-# https://github.com/fastapi/typer/issues/461
-def report_newlines(filename: str) -> None:
+def report_newlines(input: typer.FileText) -> None:
     r"""
     Report the newlines in the indicated file.
 
@@ -24,7 +21,7 @@ def report_newlines(filename: str) -> None:
     >>> report_newlines(filename)
     newlines are ('\n', '\r\n')
     """
-    newlines = jaraco.text.read_newlines(filename)
+    newlines = jaraco.text.read_newlines(input)
     count = len(tuple(always_iterable(newlines)))
     engine = inflect.engine()
     print(

I pushed that change as b1ef37e to a different branch.

It falis with:

ImportError while loading conftest '/Users/jaraco/code/jaraco/jaraco.text/conftest.py'.
jaraco/text/__init__.py:705: in <module>
    @read_newlines.register
     ^^^^^^^^^^^^^^^^^^^^^^
/opt/homebrew/Cellar/python@3.13/3.13.11_1/Frameworks/Python.framework/Versions/3.13/lib/python3.13/functools.py:908: in register
    raise TypeError(
E   TypeError: Invalid annotation for 'filename'. str | bytes | os.PathLike[str] | os.PathLike[bytes] | int not all arguments are classes.
py: exit 4 (0.23 seconds) /Users/jaraco/code/jaraco/jaraco.text> pytest pid=28330
  py: FAIL code 4 (1.30=setup[1.07]+cmd[0.23] seconds)
  evaluation failed :( (1.36 seconds)

Can you figure out the incantation for read_newlines to allow singledispatch to accept a filepath or an open file?

[Avasam]

I can certainly take a look.

I'm currently applying the typing changes downstream in case there's an edge-case I missed, it could quickly be patched in and included in next release.

[Avasam]

You're gonna have to avoid subsrcipting os.PathLike at runtime for that error.

if TYPE_CHECKING:
    Openable: TypeAlias = str | bytes | os.PathLike[str] | os.PathLike[bytes] | int
else:
    Openable = str | bytes | os.PathLike | int

or even

if TYPE_CHECKING:
    from _typeshed import FileDescriptorOrPath
    Openable: TypeAlias = FileDescriptorOrPath
else:
    Openable = str | bytes | os.PathLike | int

I'm surprised this is an issue even in Python 3.14

[jaraco]

Yes! That worked beautifully. 0105a8c

[jaraco]

Unfortunately, reading from stdin, the newlines are never inferred.

 jaraco.text replace-autocommand-with-typer 🐚 echo 'foo\nbar'
foo
bar
 jaraco.text replace-autocommand-with-typer 🐚 echo 'foo\nbar' | .tox/py/bin/python -m jaraco.text.show-newlines -
newlines are None

(note, I'm using xonsh, so 'foo\nbar' does actually emit foo<newline>bar)

All that trouble to add support for - and it's basically useless (here, but the pattern will be useful elsewhere).

[jaraco]

Dang. I thought this was done, but my latest commit fails on Python 3.9.

I tried a few things, including

diff --git a/jaraco/text/__init__.py b/jaraco/text/__init__.py
index b355818..17abd6a 100644
--- a/jaraco/text/__init__.py
+++ b/jaraco/text/__init__.py
@@ -14,7 +14,6 @@ from typing import (
     Literal,
     Protocol,
     SupportsIndex,
-    TypeAlias,
     TypeVar,
     overload,
 )
@@ -35,8 +34,11 @@ if TYPE_CHECKING:
     # Same as builtins._GetItemIterable from typeshed
     _GetItemIterable: TypeAlias = SupportsGetItem[int, _T_co]
     Openable: TypeAlias = FileDescriptorOrPath
+    # https://docs.python.org/3/library/io.html#io.TextIOBase.newlines
+    NewlineSpec: TypeAlias = str | tuple[str, ...] | None
 else:
     Openable = str | bytes | os.PathLike | int
+    NewlineSpec = str | tuple[str, ...] | None
 
 _T = TypeVar("_T")
 
@@ -694,10 +696,6 @@ def join_continuation(lines: _GetItemIterable[str]) -> Generator[str]:
         yield item
 
 
-# https://docs.python.org/3/library/io.html#io.TextIOBase.newlines
-NewlineSpec: TypeAlias = str | tuple[str, ...] | None
-
-
 @functools.singledispatch
 def read_newlines(
     filename: Openable | io.TextIOWrapper, limit: int | None = 1024

But I can't figure out the right incantation for Openable and NewlineSpec on Python 3.9.

@Avasam can you help? (feel free to commit directly)

[Avasam]

Done !

singledispatch does a runtime evaluation of annotations, so this will have to be a bit stricter on what we can use than usual to support Python 3.9.

Similarly, isinstance didn't suport comparisons on unions yet:

• either type-cast it on 3.9 to make mypy happy and omit the additional runtime check on 3.9, which was a check to make the type-checking happy anyway.
• or rewrite the isinstance check in the 3.9 branch to use a tuple

It should also be easy enough to cleanup once Python 3.9 support is dropped

[jaraco]

typer is a bit of a "heavy weight" in terms of transient dependencies (click, typing-extensions, shellingham and rich).

One of the reasons I was working on jaraco.text today was because it's one of the dependencies that still has a build-time dependency on Setuptools, and thus must be vendored to avoid pypa/packaging-problems#342. I was hoping to convert jaraco.text to the coherent build system and thereby provide sdists that build from flit-core, removing that dependency on coherent.build or setuptools. But then I was looking at the transitive dependencies, and you're right. Typer is a big one from a build system perspective. typer itself uses pdm. click and typing-extensions use flit-core. rich uses poetry. But shellingham uses Setuptools 😦 . For transitive dependencies, pygments uses hatchling and mdurl and markdown-it-py use flit-core. So the aggregate build closure of backends is flit-core, pdm, poetry, Setuptools, and hatchling. So either shellingham would have to be vendored or made optional or would have to adopt a different build backend, if we want to use jaraco.text requiring typer in Setuptools.

[Avasam]

[...] if we want to use jaraco.text requiring typer in Setuptools.

@jaraco This suggestion would still allow using typer without having to pull it in setuptools: #25 (comment)

So either shellingham would have to be vendored or made optional or would have to adopt a different build backend

I see Ofek is a maintainer of shellingham. I'm ready to bet he might he'd be open to consider an update to shellingham to use hatchling ^^

[Avasam]

Checkout https://pypi.org/project/typer-slim/

If you don't want the extra standard optional dependencies, install typer-slim instead.

When you install with:

pip install typer

...it includes the same code and dependencies as:

pip install "typer-slim[standard]"

The standard extra dependencies are rich and shellingham.

Note: The typer command is only included in the typer package.

[Avasam]

Alternatively:

Suggested change

	filename = cast(Openable, filename)
	assert isinstance(filename, (str, bytes, os.PathLike, int))

[abravalheri]

(jaraco): For transitive dependencies, pygments uses hatchling and mdurl and markdown-it-py use flit-core. So the aggregate build closure of backends is flit-core, pdm, poetry, Setuptools, and hatchling. So either shellingham would have to be vendored or made optional or would have to adopt a different build backend, if we want to use jaraco.text requiring typer in Setuptools.

(Avasam): I see Ofek is a maintainer of shellingham. I'm ready to bet he might he'd be open to consider an update to shellingham to use hatchling ^^

Switching shellingham to a different backend might not resolve the cycle: hatchling has its own dependencies (I think some of them rely on setuptools), and similar concerns may apply to pdm/poetry (I haven't checked). So the loop can reappear elsewhere.

The deeper problem is that the Python build ecosystem still hasn't solved the fundamental issue (see pypa/packaging-problems#342). PEP 517/518 gave us standard interfaces, but they did not address the self‑referential nature of the build system. In practice, the ecosystem has been relying on flit-core because it's virtually dependency‑free (still uses vendoring if I am not wrong), but this results in an implicit "everyone depends on flit-core to avoid cycles" situation. That takes us back to the kind of backend monoculture PEP 517/518 was trying to prevent. And it's worth remembering the cost of that transition: developers had to relearn and change entire workflows, tooling changed across the board, and we're still dealing with the aftershocks today (without even mentioning all the maintainer/contributor effort that had to be poured into tools like pip, PyPI, setuptools, etc... to adequate to the new standards and solve bugs).

Given that PyPA is unlikely to solve the cycle problem any time soon (judging by the age and state of pypa/packaging-problems#342), if setuptools is going to take regular third‑party dependencies (which seems to be Jason's vision), then the most reliable way to avoid cycles is to split out a dependency‑free, in-tree, bootstrapping package. This is the same pattern used by flit/flit-core and by hatch/hatchling: a backend that can build the main project, while the main project is then free to depend on whatever it needs (flit-core submits to the cycle-free constraints, while hatchling forbids its dependencies to be use itself directly or indirectly).

# EITHER
[build-system]
requires = ["setuptools-bootstrap"]
build-backend = "setuptools_bootstrap.build"

# OR
[build-system]
requires = ["setuptools-bootstrap"]
build-backend = "setuptools._build_itself"

We have 2 options:

• A tiny stand‑alone bootstrap package: minimal, stable, and dependency‑free (which exploits the fact that can only be used to build setuptools to simplify things and keep being small - e.g. bypassing validations/normalisations/etc...)
• An in‑tree bootstrap "variant": shares most of the codebase with setuptools but has its own METADATA and no dependencies.

This would protect setuptools from cycles that can be introduced by organic changes in its own dependency tree (unforeseeable). Any third‑party library could add new dependencies without risking a build deadlock for setuptools.

For the second option, most parts of setuptools could likely be reused internally, but we'd need to be careful: many imports won't be available during bootstrapping, so the bootstrap backend must avoid pulling in optional machinery (e.g. validation code, plugin loading) or triggering some imports.

Jason, sorry to highjack this discussion to talk about a setuptools problem. If you are interested in continuing this discussion we can move to the setuptools repo.

[Avasam]

Using typer-slim sidesteps shellingham (setuptools) and rich (poetry) for now, and nicely reduces necessary dependencies anyway. But your entire comment about cycle problems is still valid.

Funny thing is I learned about the slim version from their developer survey.

sorry to highjack this discussion to talk about a setuptools problem. If you are interested in continuing this discussion we can move to the setuptools repo.

(please ping me in that discussion on setuptools side, as it could positively affect distutils stub generation)

This PR is about solving a setuptools dependency problem in the first place, so it's not too out of place :P

[yan12125]

The latest typer-slim released earlier today becomes just a shallow wrapper of typer and brings all the dependencies: https://github.com/fastapi/typer/releases/tag/0.22.0. Something should become optional IMO. How do you think?

[Avasam]

We were going to use typer anyway, so that doesn't change. But now @abravalheri 's stated concerns about heavy build dependency chain are fully back.

I still think commands-only dependencies could be moved to their own extra: #30