RFC: Fisher 4.0

[jorgebucaran]

Fisher 4.x is now available 🎉

• How do I upgrade to Fisher 4.x?
• Documentation

---

Let's talk about Fisher 4.0. What would you like to see? What would you like to go away?

_install, _update and _uninstall event system

We have _uninstall events, but not _install or _update. Starting with 4.0 plugin authors should be able to reliably know when their plugin is installed, updated, or uninstalled. Related #526 #527 #573.

Oh My Fish

I'd like to remove some OMF plugin support. Specifically, deprecate init.fish, uninstall.fish, etc. See #581. The irony is that I introduced these conventions back in the day to OMF. So long, and thanks for all the fish!

Less is more

I don't consider Fisher a big ball of mud (yet), but could it be simpler? Yes. Less features = less complexity && less bugs && more maintainability. I still consider some features essential such as concurrent downloads, but I'm curious to see what else could go away, e.g., plugin dependencies, private package hosts (#464), gitlab/bitbucket support. See also: #579.

Other

• Require fish 3.0, use newer fish features, e.g., use wait to implement concurrent downloads.
• Rely less on external tools. No awk, no sed, no basename/dirname. Just mv, rm, cp, and mkdir.
• Remove support for private package hosts.
• Deprecate fishfile in favor of fish_bundle → fish_plugins. See Rename fishfile to fish_plugins #524.
• Needless to say, but upgrading to fisher 4.0 should be smooth and painless.

Timeline

I'll probably have enough time to begin working on this sometime in 2021.

I've been working on this for a while now and I might be able to finish it sooner, before December.

Fisher 4.0.0 lands this week. 🎉

[IlanCosman]

Less awk if that's a possibility. The average fish user/scripter like myself isn't likely to know awk syntax etc. which makes it much harder to understand the code and possibly contribute. I know next to nothing about fisher's code, but are there really places where pure fish is insufficient? I think fisher 4.0 should do it's best to use no or very little awk.

EDIT: Also definitely fish 3.0. Even debian is on 3.0 and no one has any excuse to be older than debian 😂

[jorgebucaran]

@IlanCosman It's worse without awk for some of the most complex things like parsing the fishfile as the equivalent with fish built-ins could possibly 10-fold the amount of code, but part of the idea of upgrading to fish 3.0 is using more of the fish built-ins for simple cases, validating a path, etc. 👍

[IlanCosman]

I really don't feel strongly about this at all, but my brothers, both CS majors, have always cautioned me against brevity over clarity. Personally I would rather use 30 lines of step-by-step fish rather than 5 lines of dense awk, but I understand you may feel differently.

Then again my first programming language was Java so.... 😂

[jorgebucaran]

@IlanCosman Give it 5 minutes. The syntax is not all that different from Java. It's so easy it hurts. And if you already know JavaScript, then you already know AWK's grammar.

Fish built-ins may change in the future as well, new built-ins or built-in flags may be added, etc. AWK will just keep on working. It's less churn. Anyway, AWK or not is beside the point. I should be able to use less of it in Ⅳ for sure. 😂

[PatrickF1]

The irony is that I introduced these conventions back in the day when I still contributed to OMF.

Wow didn't know you were also a major OMF contributor. That's cool, and thank you for your contributions to that project as well even if you think they were misguided!

Less awk if that's a possibility.

I also agree that the parsing that happens should be done using awk. @IlanCosman awk is actually a dying but pretty commonly used tool in the software industry, so I as a software engineer also know how to read and write some awk. It's incorporated into major tools such as vim.

What would you like to see?

• Not specific to the code but I'd like to see a tiny bit more documentation around creating packages. e.g. naming conventions, guiding principles, maybe one or two packages that you uphold as good examples.
• Would you mind reconsidering this? fisher ls doesn't specify which package version #550
• Output which packages are being removed or added on executing fisher (after manually editing the fishfile), fisher rm or fisher add
• A command to force refresh the cache. I've run into bugs where the cache was preventing new versions from being downloaded.

[ghost]

* Deprecate `fishfile` in favor of `fish_bundle`. See #524. Maybe this is a good opportunity to figure out a way to make parsing and formatting this file more easily too.

For parsing & formatting, would an existing configuration file format be up for consideration? Since the fishfile is currently a list of repositories, an existing format could allow you to leverage existing packages/commands to parse the configuration file.

Some considerations:

• TOML: My preference due to simplicity
• YAML
• JSON
• INI: An option, but can be hard to maintain since it doesn't have a consistent spec.

[jorgebucaran]

@vyasknellutla Probably not. Yeah, I've thought about it this before many times, but I always wind up on "not worth it".

[mattgreen]

Biggest thing I'd like to see in a breaking release is defaulting to installing packages somewhere other than ~/.config/fish. I'd argue the behavior is surprising enough to warrant being changed even if it means a bit more code. Perhaps ~/.config/fisher or ~/.config/fish/packages?

fish 3.0? This means we could use more fish builtins

Totally fair.

[IlanCosman]

I would strongly argue against that @mattgreen. Why is this behavior surprising to you? To me this is the only logical way of doing it. When I create functions, I put them in the functions directory. When I "install" a fisher "package" it's just a bunch of fish functions that someone else made, and there's no reason they shouldn't go in the usual spot.

In an optimal world fish would support autoloading directories within functions, so that directory wouldn't get so crowded.

Alternatively, fisher could request sudo access and drop packages in the $__fish_data_dir/vendor_ directories, but I don't think that's a great idea either.

[mattgreen]

I advocate separating the user's custom fish config from package contents because a package can inadvertently clobber user-written functions/completions/config snippets of the same name. I've done this before, and I should've read the docs, but the Principle of Least Astonishment seems to be the fish ethos. (Thank goodness for git.) Moving them to another directory protects user-specific configuration. It doesn't fix the fact that packages can clobber each other's functions, but I see that as a much smaller problem.

Related: can't remember off-hand if this is true, but AFAIK all package repos are updated for every mutable invocation of fisher (e.g. adds/remove/update). This means that malicious fisher packages will propagate faster than they would if they were manually updated. This is a difficult problem to solve completely and may be out of scope, but IMO justifies code to handle add/remove/update separately, rather than starting from scratch each time. (Pinning versions could work too.)

[IlanCosman]

I see what you're saying about clobbering. Perhaps fisher should ask the user about any files that it would clobber?

[ghost]

Since fisher is installing plain-text fish scripts, I'm personally fine with it being in $XDG_CONFIG_HOME/fish (by default: ~/.config/fish).

But if it is something we're considering, how about storing it in $XDG_DATA_HOME/fisher (by default: ~/.local/share/fisher) alongside the rest of the data fisher stores?

I can see a case be made if people would like to check in $XDG_CONFIG_HOME as a part of their dotfiles, but leave out files that fisher can generate from the configuration file fishfile.

[jorgebucaran]

@mattgreen Biggest thing I'd like to see in a breaking release is defaulting to installing packages somewhere other than ~/.config/fish. I'd argue the behavior is surprising enough to warrant being changed even if it means a bit more code.

[...] I advocate separating the user's custom fish config from package contents because a package can inadvertently clobber user-written functions/completions/config snippets of the same name.

Not by default, no. Optionally? You can use $fisher_path, but I'm actually considering deprecating $fisher_path. What? This won't even allow changing the installation path. How am I going to separate my own user-written functions/completions/snippets with other stuff I install with Fisher?

Note: We will probably never deprecate $fisher_path, though, since users will be users and will want to customize their setup to their wit's end and we should not impede that. And that's good. Fisher is designed so you can handle your Fish user configuration, not to dictate where you should put it.

Fisher is designed so you can handle your Fish user configuration. That is functions, completions, and conf.d snippets in $__fish_config_dir. This is the default location. But what if I also want to write my own functions, completions, and conf.d snippets and put them there? Every time I run fisher I risk overwriting files that have the same name from some plugin. Unlikely. A good plugin will prefix files like _my_plugin_* to avoid this, but it's true, it could happen.

If you're afraid this could happen, I suggest you create your own plugin for those files. Put it anywhere you want, .e.g. ~/fish/plugins/utils. Then install it:

fisher install ~/fish/plugins/utils

Principle of Least Astonishment seems to be the fish ethos.

This argument can stifle creativity too!

Moving them to another directory protects user-specific configuration. It doesn't fix the fact that packages can clobber each other's functions, but I see that as a much smaller problem.

It doesn't fix the fact that in fish all functions are in the same scope. Using Fish is very much like being inside a REPL (like when you're inside the Node REPL, etc.)

@mattmc3 Maybe the issue here is that you want to manage some things directly yourself and use Fisher for others and expect both approaches to be 100% compatible. They are largely compatible, but Fisher is optimized for managing your stuff as plugins. Having said that, I'm working to improve unsavory surprises, e.g., prevent clobbering of files (#624), multiple prompts (#629), etc.

---

[jorgebucaran]

About deprecating $fisher_path.

So, $fisher_path lets you customize the installation path, but allowing you to work around Fisher's usage philosophy you only end up hating Fisher because it doesn't perfectly adjust to how you think of fish and you have to add some code to your config.fish.

If people are absolutely against Fisher's default behavior, then maybe Fisher is not for them? Anyway, this is just a thought. Notice I didn't even mention it here.

[mattgreen]

@jorgebucaran, your approach is out of the box but has promise IMO. It is certainly a conceptual shift from the file-centric approach taken by users and utilities (e.g. stow). I suspect more of the work will be around educating users than the code itself.

Otherwise it is very clean and I'll give it a try. It might or might not click, but that's not on you.

[IlanCosman]

Any updates on this? I'm excited to use fisher and trim down my code base 😂

[jorgebucaran]

@IlanCosman Unfortunately, none at the moment, but I really can't wait to start working on this. I'm already working on this and should be ready to publish very soon!

[andreiborisov]

What I would like from the new major Fisher release is the ability to reliably use non .fish files in packages. The case for the fish-only packages seems somewhat limited for me since it’s often useful to include at least some template or config files in the package.

Now, Fisher downloads and manages them just fine already. The problem is you can’t reliably access those files from fish scripts, since .fish files are symlinked/copied into functions directory while everything else is left behind under Fisher managed directories. You can determine where the original package is located, but it’s somewhat complicated. More importantly, since Fisher’s private directory structure is an implementation detail, it’s brittle.

So, question is, how is it possible to access non .fish files from a fisher package script?

I see 2 potential solutions.

Ideally, all the package files would be stored together in the functions directory, this implies some folder-per-package structure there. Which perhaps can help with potential naming conflicts.

The other solution is to provide a command like fisher package locate which would print the package original location. There is a big downside with this approach, however: it would require the package to explicitly use that command, which hurts package portability greatly.

@jorgebucaran I would love to hear your thoughts on this particular issue🙏

[IlanCosman]

@andreiborisov relevant: #581

[jorgebucaran]

@andreiborisov As @IlanCosman mentioned, #581 would allow you to create e.g., a directory inside your plugin's functions directory. During the install, fisher would copy this directory to your ~/.config/fish/functions (together with any *.fish functions therein as usual). You can now access this directory and whatever is inside (status -f)/directory, $fisher_path/functions/directory, or $__fish_config_dir/functions/directory.

[jorgebucaran]

Fisher 4.x is now available. 🎉

If you are using in Fisher 3.x, you can upgrade to 4.x by running

fisher self-update

For everything else, check out the new documentation.