File and path completion again

[dimbleby]

This is almost entirely @gtristan's work from #782, but updated for changes that have been made to click in the meantime. fixes #780

In my experience, failing autocompletion on paths and filenames is by far the most annoying part about using otherwise lovely click CLIs. Which is why I have been motivated to put this together!

As at #782, the approach is:

• Add -o nospace to the bash completion options, so that spaces are not automatically added to completions. This is necessary so that nested paths can be completed without constantly having to delete the added space.
• Then completions are expected to add spaces themselves, so update all the code that does this for completions automatically generated by click
• Look for type-based completions, and implement this for choices, booleans and - especially! - files and paths

I expect that the possibly-controversial bit is that, as of today, any user-implemented autocompletions won't include spaces; and so the addition of -o nospace will cause their CLIs to behave slightly differently.

I suggest that we either:

• Just go with this; it's actually an improvement because now users have the ability to do more sophisticated completions (as we do for files and paths)
• Or, we could change the click code to make sure there's a space after anything that users give us. That denies them the ability to do cool things, but maintains backwards-compatibility.

I'm happy to implement the second of these if that's what maintainers want - especially if it helps this longstanding fix finally to get over the line!

[JustinTArthur]

I like that this adds an API to the types.

[JustinTArthur]

I added a PR against your PR to add variable and tilde expansion, @dimbleby.

[dimbleby]

So I see! Looks good, I'll hit the green button in a moment...

[davidism]

I've been trying to review this and get it ready for 7.1, and I'm starting to realize just how messy completion is. Documentation for completion systems is hard to find and hard to follow.

Bash doesn't quote completions, so filenames with spaces (or any other values with special characters) are invalid. ZSH does quote, but that means it also quotes the trailing space that this PR is manually adding now. But since ZSH uses a menu for completions, not adding a space makes it impossible to continue completing paths, as pressing tab or arrows scrolls through the menu instead of listing the next directory.

Just like this PR turns on nospace, we could also turn off quoting in ZSH, and take over doing that. We've already turned off sorting. argcomplete has a neat trick that turns off spaces only if the completion ends with certain characters, but I don't know how to apply that to ZSH, it doesn't appear to have the equivalent of Bash's compopt. But this just starts to feel silly, we're basically making ourselves (or users) recreate everything that the completion system itself is already supposed to provide. Bash and ZSH both have ways of telling their completion systems to complete files correctly, for example. ZSH already expands ~ and has a ton of other cool features

The problem is the limited way that Click implements completion right now. There's no reason we just have to return completions. In fact, we already support returning descriptions, presumably we could extend that more. Wouldn't it be cool if Click could indicate to the completion script that it should use other functions that Bash or ZSH provide, not just _describe?

I'm trying to figure out a way to add some of the enhancements here like the completions method and file completion, without making the ZSH experience worse or backing ourselves into a corner when we do want to refactor the whole thing.

[schollii]

I expect that the possibly-controversial bit is that, as of today, any user-implemented autocompletions won't include spaces; and so the addition of -o nospace will cause their CLIs to behave slightly differently.

How about an env var that determines default behaviour? And/or the bash completion code generated (the code meant to be sourced to get completion hooked into bash) could be slightly different based on some arg given. Eg

_FOO_BAR_COMPLETE=source foo-bar 

is current behavior vs

_FOO_BAR_COMPLETE=source _FOO_BAR_COMP_AUTO_SPACE=false foo-bar

would generate hook code that turns off autospace.

[davidism]

That wouldn't help the issue here. Completion is already pretty opaque as it is, adding configuration through more env vars is adding complexity. Users shouldn't have to care about that.

[schollii]

@davidism Then how about a setting that can be toggled in the CLI main script code eg in my_click_app.py I could add a line click.set_auto_spaces() to get the old behavior, or just keep current behavior by default but have click.set_no_auto_spaces() for those starting fresh.

Would also be nice to have an additional option on individual commands to override the global so that commands can be tweaked one at a time, as you can in a bash script, but maybe too difficult in click. At least a global for all-or-none would be better than current situation.

[davidism]

No, I don't think adding settings anywhere is the right idea. Also, turning on or off spaces doesn't make Zsh completion any better, it's broken either way. The whole system needs to be redesigned, not made configurable.

[davidism]

Thanks for working on this. We're rewriting the whole completion system, if this is still and issue after that we'll need to find a new solution that fits with the new system and doesn't have the issues I outlined above. See #1484 and #1622.