kci rework: Click or Typer?

[gctucker]

This is not a "Trick or Treat" question :) The kci command line tool is to go hand-in-hand with the new API and as such we still have a few weeks to refine it and come up with a user-friendly design. Ideally, the main choices should be made by the end of October so we can have a first stable version available in December at the same time as the API production release candidate.

As part of this work, I've read some advice and guidelines from several places including clig.dev. The legacy KernelCI command line tools are based on the standard library argparse which works well but has some limitations. In particular, nested sub-commands like we need for kci is a bit tedious to implement. Assuming we're keeping the code as Python, it seems like the best alternatives would be Click and Typer which itself is based on Click. They have a few dependencies but they would seem worth adding to the project as they're quite common and lightweight ones.

See also the mailing list discussion.

Roadmap issue: kernelci/kernelci-core#2125

Option 1: Click

Here's an RFC if you want to try it out: kernelci/kernelci-core#2111

Basically, a typical command using an API token can be implemented this way with a custom class KciSecrets to handle settings files and pass secrets to the function:

@kci.command(cls=KciSecrets, help="whoami with API authentication")
@click.option('--config', type=str, help="Path to the YAML config")
@click.option('--api', type=str, help="Name of the API config entry")
def whoami(config, api, secrets):
    configs = kernelci.config.load(config)
    api_config = configs['api_configs'][api]
    api = kernelci.api.get_api(api_config, secrets.api.token)
    data = api.whoami()
    click.echo(json.dumps(data, indent=2))

The help message looks like this, basically the same as with argparse:

$ ./kci-click whoami --help
Usage: kci whoami [OPTIONS]

  whoami with API authentication

Options:
  --config TEXT  Path to the YAML config
  --api TEXT     Name of the API config entry
  --help         Show this message and exit.

Then when running it, with a TOML settings file containing the API token:

$ ./kci-click whoami 
{
  "id": "64ef04e7391d44b7fa620d13",
  "active": true,
  "profile": {
    "username": "admin",
    "hashed_password": "<hashed-password>",
    "groups": [
      {
        "id": "6499aa9da02fef8143c1feb0",
        "name": "admin"
      }
    ],
    "email": "email@domain.com"
  }
}

In summary, this basically provides a similar user experience as with argparse but with a cleaner and simpler implementation. It makes the command line design easier to implement so it can remove some burden with nested commands and default values in TOML etc. It should also help users write their own commands or contribute to the code base.

The only downside found so far is that it adds some external package dependency, as argparse is in the Python 3 standard library but Click is not. This should be a very minor thing though. Most distros already have a Click package (Debian Bookworm has v8.1.3).

Option 2: Typer

Here's an RFC if you want to try it out: kernelci/kernelci-core#2112

The equivalent implementation for the whoami command looks like this:

@kci.command(cls=KciSecrets, help="whoami with API authentication")
def whoami(config=Args.config, api=Args.api, secrets=Args.secrets):
    configs = kernelci.config.load(config)
    api_config = configs['api_configs'][api]
    api = kernelci.api.get_api(api_config, secrets.api.token)
    data = api.whoami()
    rich.print(data)

So it's arguably even simpler than with Click without losing any of the flexibility. As Typer uses rich to format its output (help, errors etc.) it seemed worthwhile also using it to show the output here too. Here are some sample commands with screenshots to show the full colours (secrets is a special hidden argument added by KciSecrets automatically so it's not shown in the help message):

Then it has some nice ways to handle errors and even suggest what the user got wrong:

Then on top of this, it provides a straightforward way to install auto-completion:

$ ./kci-typer --install-completion
bash completion installed in /home/gtucker/.bash_completions/kci-typer.sh
Completion will take effect once you restart the terminal

$ ./kci-typer [TAB]
foo     hack    whoami  
$ ./kci-typer

This appears to be just expanding possibilities on top of the Click implementation with no big downsides. Potentially, some people might not like the formatted output with panels and ASCII art and lots of colours. It should be possible to configure it with a theme to address this. It does add a bit more dependencies such as rich but we might want to use that anyway to provide some more readable output even if we didn't choose Typer. They're all packaged in Debian as well. There are many other Typer features that aren't showcased here, and as it's developed by Tiangolo like FastAPI it could bring some familiarity for developers who contribute both to the KernelCI API and the kci command-line tool.

Time to vote!

We'll take a look at the results on Monday 2nd October and wrap up this discussion then.

[gctucker]

One of the main issues highlighted by the mailing list discussion is the dependencies added by such frameworks that aren't part of the standard library, and the recurring problems in general with Python modules that evolve without backwards-compatible support. If users need to set up a venv to run the tool or have no other option but to rely on a Docker image then it's going to cause extra hurdles for community adoption.

One approach of course to mitigate this is the 3rd option to stick to argparse, but we also know this makes the implementation more complex and to some extent the command line user experience is not as good. Arguably, Typer adds some usability and can help provide some user flows on the command line rather than with a web dashboard - or as a complement to it. It's interesting to also consider ways to use such frameworks without impacting the users with all the dependencies, such as including the dependencies directly in the KernelCI package or providing a "plain" tool based only on argparse and a "fancy" one with optional dependencies for users who have it installed. These ideas might not be very practical, a bit of investigation would need to be done to get to the bottom of pros/cons there.

Regardless of which implementation framework we choose, we can already start proposing a stable design for the kci tool syntax with commands and arguments. Then probably next week we'll have discussed the implementation options enough to come up with a proposal that is practical, doesn't require rewriting a whole framework and maximises community adoption of the tool. It will most likely have to be a compromise between these diverging requirements but let's see where this takes us.

[acerv]

Hi, my 2 cents on the project, since I developed https://github.com/linux-test-project/kirk which is quite similar in this case.

Adding external dependencies can be quite tricky, unless it's 100% needed. The reason is explained by @gctucker , who mentioned critical issues we might have.

Also, click/Typer add really advanced graphical features, but, for most of the cases, argparse is quite good and not that difficult.
If you really need to build a fancy UI, with bars, graphical toys, etc, click/Typer is probably the way to go. But if you need a normal user interface which prints come colors and text, formatting JSON/TOML data, etc, then argparse is the way to go.

[gctucker]

Note: Click doesn't have fancy output, it's very similar to argparse but just a bit clearer IMHO. For example:

$ ./kci-click --help
Usage: kci-click [OPTIONS] COMMAND [ARGS]...

  Entry point for the kci command line tool

Options:
  --settings TEXT  Path to the TOML settings file
  --help           Show this message and exit.

Commands:
  foo
  hack
  whoami  whoami with API authentication

[gctucker]

For completeness, I tried to write an equivalent PoC using argparse: kernelci/kernelci-core#2119

Handling the TOML settings for default values seems a lot more brittle there as it's replacing object attributes in the args namespace. Then the idea of having the subcommands path as a TOML section isn't really practical without a lot more work so the sections are called whoami and hack instead of kci.whoami and kci.foo.bar. Having an arbitrary number of subcommand levels is also not practical without a lot more work as described in the comments and Git commit. There are other issues, the --verbose flag doesn't have a --no-verbose counterpart so when setting a default verbose = true in TOML users can't disable it on the command line.

Basically, Click has huge benefits over argparse. It makes the implementation much simpler and provides a much nicer user experience. Then Typer just builds on top of Click so it just goes a bit further with flashy extra features.

The Click project was created nearly 10 years ago and has over 1 million projects listed on GitHub as users and 340 contributors. Several well-supported and broadly used tools depend on it such as sqlite-utils, flask, black, matplotlib... So it would appear to be the best compromise in terms of being well maintained and fully-featured.

The Typer project seems to be a lot less used and it's not in Debian, so while it might be really nice with auto-completion and improved error handling and auto-generated help, it might be harder to adopt due to the effort required to install it.

[JenySadadia]

Thanks for the information @gctucker. Looking at the number of users and contributors, Click looks more reliable than Typer.
We certainly should take into account the project's stability and how it is being maintained before making the decision.

Bdw there is a useful link for comparison of Typer with Click and argparse from the official documentation.

[gctucker]

Thanks for the link, I hadn't seen it before. It also mentions other API frameworks, but I think we're good now with FastAPI.

As Typer is based on Click we could argue that it benefits from the large user base around Click. But there's potentially a greater chance that Typer gets abandoned earlier than Click. I think I would use Typer for my personal use but for a community project that needs to provide continuity, backwards-compatibility and portability over several years such as KernelCI it seems like Click would be better suited.

[gctucker]

Oh and I see there's a plugin click-completion, also apparently Click has Bash auto-completion support already. So as an optional dependency we could have auto-completion using Click - not just Typer (Type has a one-line command to install it so that's the main added value I guess).

[gctucker]

As of today, if we include one additional vote for argparse from the mailing list discussion, we basically have 3 votes for each option: Click, Typer and argparse.

Here's a summary with pros / cons of each solution based on the discussions so far:

Click

Pros:

• Easy to implement and maintain with nested subcommands
• Traditional ASCII interface
• Lightweight dependencies
• Very large user base and 10 years of past support
• Packaged in Debian Stable (bookworm)

Cons:

• Arguments need to be repeated in decorators and function signatures
• Auto-completion is not built-in (but can be installed fairly easily)
• Not in standard Python 3 library

Typer

Pros:

• Optimal implementation with type inference like FastAPI
• Built-in auto-completion
• Improved error reporting to the user and suggestions for typos
• Based on Click so stable foundation

Cons:

• Lots of additional dependencies
• Fancy output style might not improve user experience within the KernelCI community
• Relatively new project and small user base
• Not packaged in Debian Stable (bookworm)

argparse

Pros:

• Part of the Python 3 standard library so no additional dependencies
• Classic ASCII interface (slightly less user-friendly than Click)

Cons:

• Not designed for advanced use of subcommands
• Cumbersome to implement custom features e.g. TOML default values
• Ambiguity between positional and named arguments
• No standard way to enable auto-completion
• Lots of additional code required at application level so bug-prone and hard to maintain

[gctucker]

Meanwhile we've started looking into the kci command line syntax (#263) and TOML settings (#261) which is closely related. So far, subcommands with a greater depth than 2 are rare and we could probably find a way around it to only have kci commnand subcommand as a standard depth, but this might incur a cost from a user experience point of view.

So what I would propose is to start with Click using the version currently in Debian Stable (v8.1.3) as it has so many advantages over argparse. Then if users find the Click dependencies too cumbersome, we may consider bundling it directly with the kernelci package, probably with a Git submodule or directly added to the repository under kernelci.click. As a fallback, if we later find that adopting Click in the community still requires a larger effort than maintaining an implementation based on argparse we might be able to convert the tool to argparse then. This scenario seems very unlikely as there aren't any actual signs that Click is more difficult to adopt than the existing dependencies (see requirements.txt in kernelci-core repository). So the risk seems very low, and if we did hit this kind of problem we would still have a way to mitigate it.

[yurinnick]

In my opinion Click would be the best middle ground between barebones argparser and fancy typer. We use Click for the most use cases and in general very happy with it, given it's mature and stable framework. typer is nicer, but adds quite a few new dependencies, but only (?) provides some nicer output. Personally, I don't think I've ever though "I wish this CLI had a nicer output" for any well-design CLI tool.

[gctucker]

Thanks for the feedback :)

Typer actually also provides a very convenient way to install auto-completion, it suggests arguments when mistyped by the user, has a default exception handler to make the output more readable and arguably generates more accessible help messages. Also it's really simpler to implement sub-commands and type inference avoids the need for repeating argument names.

So for a personal project I would probably use Typer as I think it's miles ahead in some respects, but for a community project where things need to be able to install and we know the user-base is more familiar with old-style tools then Click does seem like the best option.

[gctucker]

OK I think we can resolve this now, and re-open in case some more discussion is needed.