Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Treat runtime files from installation as immutable #3346

Closed
the-mikedavis opened this issue Aug 6, 2022 · 5 comments · Fixed by #5411
Closed

Treat runtime files from installation as immutable #3346

the-mikedavis opened this issue Aug 6, 2022 · 5 comments · Fixed by #5411
Labels
A-helix-term Area: Helix term improvements A-packaging Area: Packaging and bundling A-tree-sitter Area: Tree-sitter C-enhancement Category: Improvements

Comments

@the-mikedavis
Copy link
Member

the-mikedavis commented Aug 6, 2022

We should treat the grammar sources, compiled parsers, and queries produced while building Helix as immutable defaults which can be overridden with the existence of files in ~/.config/helix/runtime. Effectively, $HELIX_RUNTIME should act like $PATH in unix.

Except when installing from source, these files almost always end up in some /usr directory that needs sudo to edit (which you probably shouldn't do). With Nix, these end up in the store which is immutable so you need to build from source to modify queries or themes locally. Instead you should just be able to override files in your ~/.config/helix/runtime directory.

See #3310 (comment)
See #2091 (comment)

@amtoine
Copy link
Contributor

amtoine commented Oct 7, 2022

as in #4128, i have this very issue 😮

from the Arch main repositories, helix comes installed with all its grammar in /var/lib/helix/runtime/ which needs sudo privileges to do anything...
thus hx --grammar fetch/build does not work and sudo helix --grammar fetch/build has to be used 😕

always using ~/.config/helix/runtime/ and $HELIX_RUNTIME would be amazing 🤩

@amtoine
Copy link
Contributor

amtoine commented Nov 30, 2022

hello there 😋

  • is anyone working on this somewhere? 😮
  • is there any chance i can help?
    i do not know where to start really, but i'd like to see such feature in helix.
    that would be amazing to avoid messing around with sudo 🤩

cheers 👋

@domust
Copy link

domust commented Dec 9, 2022

Hi there,

I would also love to have this feature in helix.

Currently I'm trying to add a new language to helix while implementing tree-sitter grammar for it as well,
but I can't test it on my machine since I'm using NixOS.

If anyone could give me any pointers, I would try to help implementing this feature.

@paul-scott
Copy link
Contributor

I had a look at this. What we could do is change helix_loader::runtime_dir and have it return a Vec of directories in a particular priority. I thought it was going to be an easy fix but unfortunately it is used in 10 places across 4 sub-crates so would take a little bit of tinkering. In addition / alternatively we could write a public function that, given a relative path, searches for the full path of the first file found according to the runtime directory priority (or else an error). This would make it easier for the call sites.

By the way, does anyone know why RUNTIME_DIR (lazy static call to runtime_dir) is used only for the helix_loader::grammar::load_runtime_file method, while runtime_dir is used dynamically in all other call sites?

One thing that is a bit inconsistent with this proposal is that the themes live under RUNTIME_DIR/themes, but local user themes are currently being found in ~/.config/helix/themes, not ~/.config/helix/runtime/themes. So either that would have to be changed for consistencies sake, or kept in for backwards compatibility.

@amtoine
Copy link
Contributor

amtoine commented Jan 5, 2023

I had a look at this. What we could do is change helix_loader::runtime_dir [...] while runtime_dir is used dynamically in all other call sites?

do not know the code base to answer all this 👀

One thing that is a bit inconsistent with this proposal is that the themes live under RUNTIME_DIR/themes, but local user themes are currently being found in ~/.config/helix/themes, not ~/.config/helix/runtime/themes. So either that would have to be changed for consistencies sake, or kept in for backwards compatibility.

maybe helix could look for both?
making ~/.config/helix/runtime/themes/ the new default for consistency and ~/.config/helix/themes/ the second fallback directory if the former does not exist, for backwards compatibility 😋

paul-scott added a commit to paul-scott/helix that referenced this issue Jan 8, 2023
This is an implementation for helix-editor#3346.

Previously, one of the following runtime directories were used:

1. `$HELIX_RUNTIME`
2. sibling directory to `$CARGO_MANIFEST_DIR`
3. subdirectory of user config directory
4. subdirectory of path to helix executable

The first directory provided / found to exist in this order was used as a
root for all runtime file searches (grammars, themes, queries).

After this change it is possible for all of these directories to be
searched for runtime files. If the same file name appears in multiple runtime
directories, the same priority order is used as above.

One exception to this rule is that a user can have a `themes`
directory directly in the user config directory that has higher piority
to `themes` directories in runtime directories. That behaviour has been
preserved.

As part of implementing this feature `theme::Loader` was simplified
and the cycle detection logic of the theme inheritance was improved to
cover more cases and to be more explicit.
paul-scott added a commit to paul-scott/helix that referenced this issue Jan 18, 2023
This is an implementation for helix-editor#3346.

Previously, one of the following runtime directories were used:

1. `$HELIX_RUNTIME`
2. sibling directory to `$CARGO_MANIFEST_DIR`
3. subdirectory of user config directory
4. subdirectory of path to helix executable

The first directory provided / found to exist in this order was used as a
root for all runtime file searches (grammars, themes, queries).

This change lowers the priority of `$HELIX_RUNTIME` so that the user
config runtime has higher priority. More significantly, all of these
directories are now searched for runtime files, enabling a user to override
default or system-level runtime files. If the same file name appears
in multiple runtime directories, the following priority is now used:

1. sibling directory to `$CARGO_MANIFEST_DIR`
2. subdirectory of user config directory
3. `$HELIX_RUNTIME`
4. subdirectory of path to helix executable

One exception to this rule is that a user can have a `themes`
directory directly in the user config directory that has higher piority
to `themes` directories in runtime directories. That behaviour has been
preserved.

As part of implementing this feature `theme::Loader` was simplified
and the cycle detection logic of the theme inheritance was improved to
cover more cases and to be more explicit.
paul-scott added a commit to paul-scott/helix that referenced this issue Jan 18, 2023
This is an implementation for helix-editor#3346.

Previously, one of the following runtime directories were used:

1. `$HELIX_RUNTIME`
2. sibling directory to `$CARGO_MANIFEST_DIR`
3. subdirectory of user config directory
4. subdirectory of path to helix executable

The first directory provided / found to exist in this order was used as a
root for all runtime file searches (grammars, themes, queries).

This change lowers the priority of `$HELIX_RUNTIME` so that the user
config runtime has higher priority. More significantly, all of these
directories are now searched for runtime files, enabling a user to override
default or system-level runtime files. If the same file name appears
in multiple runtime directories, the following priority is now used:

1. sibling directory to `$CARGO_MANIFEST_DIR`
2. subdirectory of user config directory
3. `$HELIX_RUNTIME`
4. subdirectory of path to helix executable

One exception to this rule is that a user can have a `themes`
directory directly in the user config directory that has higher piority
to `themes` directories in runtime directories. That behaviour has been
preserved.

As part of implementing this feature `theme::Loader` was simplified
and the cycle detection logic of the theme inheritance was improved to
cover more cases and to be more explicit.
archseer added a commit that referenced this issue Mar 9, 2023
* Generalised to multiple runtime directories with priorities

This is an implementation for #3346.

Previously, one of the following runtime directories were used:

1. `$HELIX_RUNTIME`
2. sibling directory to `$CARGO_MANIFEST_DIR`
3. subdirectory of user config directory
4. subdirectory of path to helix executable

The first directory provided / found to exist in this order was used as a
root for all runtime file searches (grammars, themes, queries).

This change lowers the priority of `$HELIX_RUNTIME` so that the user
config runtime has higher priority. More significantly, all of these
directories are now searched for runtime files, enabling a user to override
default or system-level runtime files. If the same file name appears
in multiple runtime directories, the following priority is now used:

1. sibling directory to `$CARGO_MANIFEST_DIR`
2. subdirectory of user config directory
3. `$HELIX_RUNTIME`
4. subdirectory of path to helix executable

One exception to this rule is that a user can have a `themes`
directory directly in the user config directory that has higher piority
to `themes` directories in runtime directories. That behaviour has been
preserved.

As part of implementing this feature `theme::Loader` was simplified
and the cycle detection logic of the theme inheritance was improved to
cover more cases and to be more explicit.

* Removed AsRef usage to avoid binary growth

* Health displaying ;-separated runtime dirs

* Changed HELIX_RUNTIME build from src instructions

* Updated doc for more detail on runtime directories

* Improved health symlink printing and theme cycle errors

The health display of runtime symlinks now prints both ends of the
link.

Separate errors are given when theme file is not found and when the
only theme file found would form an inheritence cycle.

* Satisfied clippy on passing Path

* Clarified highest priority runtime directory purpose

* Further clarified multiple runtime details in book

Also gave markdown headings to subsections.

Fixed a error with table indentation not building
table that also appears present on master.

---------

Co-authored-by: Paul Scott <paul.scott@anu.edu.au>
Co-authored-by: Blaž Hrastnik <blaz@mxxn.io>
sagnibak pushed a commit to sagnibak/helix that referenced this issue Mar 21, 2023
…itor#5411)

* Generalised to multiple runtime directories with priorities

This is an implementation for helix-editor#3346.

Previously, one of the following runtime directories were used:

1. `$HELIX_RUNTIME`
2. sibling directory to `$CARGO_MANIFEST_DIR`
3. subdirectory of user config directory
4. subdirectory of path to helix executable

The first directory provided / found to exist in this order was used as a
root for all runtime file searches (grammars, themes, queries).

This change lowers the priority of `$HELIX_RUNTIME` so that the user
config runtime has higher priority. More significantly, all of these
directories are now searched for runtime files, enabling a user to override
default or system-level runtime files. If the same file name appears
in multiple runtime directories, the following priority is now used:

1. sibling directory to `$CARGO_MANIFEST_DIR`
2. subdirectory of user config directory
3. `$HELIX_RUNTIME`
4. subdirectory of path to helix executable

One exception to this rule is that a user can have a `themes`
directory directly in the user config directory that has higher piority
to `themes` directories in runtime directories. That behaviour has been
preserved.

As part of implementing this feature `theme::Loader` was simplified
and the cycle detection logic of the theme inheritance was improved to
cover more cases and to be more explicit.

* Removed AsRef usage to avoid binary growth

* Health displaying ;-separated runtime dirs

* Changed HELIX_RUNTIME build from src instructions

* Updated doc for more detail on runtime directories

* Improved health symlink printing and theme cycle errors

The health display of runtime symlinks now prints both ends of the
link.

Separate errors are given when theme file is not found and when the
only theme file found would form an inheritence cycle.

* Satisfied clippy on passing Path

* Clarified highest priority runtime directory purpose

* Further clarified multiple runtime details in book

Also gave markdown headings to subsections.

Fixed a error with table indentation not building
table that also appears present on master.

---------

Co-authored-by: Paul Scott <paul.scott@anu.edu.au>
Co-authored-by: Blaž Hrastnik <blaz@mxxn.io>
wes-adams pushed a commit to wes-adams/helix that referenced this issue Jul 4, 2023
…itor#5411)

* Generalised to multiple runtime directories with priorities

This is an implementation for helix-editor#3346.

Previously, one of the following runtime directories were used:

1. `$HELIX_RUNTIME`
2. sibling directory to `$CARGO_MANIFEST_DIR`
3. subdirectory of user config directory
4. subdirectory of path to helix executable

The first directory provided / found to exist in this order was used as a
root for all runtime file searches (grammars, themes, queries).

This change lowers the priority of `$HELIX_RUNTIME` so that the user
config runtime has higher priority. More significantly, all of these
directories are now searched for runtime files, enabling a user to override
default or system-level runtime files. If the same file name appears
in multiple runtime directories, the following priority is now used:

1. sibling directory to `$CARGO_MANIFEST_DIR`
2. subdirectory of user config directory
3. `$HELIX_RUNTIME`
4. subdirectory of path to helix executable

One exception to this rule is that a user can have a `themes`
directory directly in the user config directory that has higher piority
to `themes` directories in runtime directories. That behaviour has been
preserved.

As part of implementing this feature `theme::Loader` was simplified
and the cycle detection logic of the theme inheritance was improved to
cover more cases and to be more explicit.

* Removed AsRef usage to avoid binary growth

* Health displaying ;-separated runtime dirs

* Changed HELIX_RUNTIME build from src instructions

* Updated doc for more detail on runtime directories

* Improved health symlink printing and theme cycle errors

The health display of runtime symlinks now prints both ends of the
link.

Separate errors are given when theme file is not found and when the
only theme file found would form an inheritence cycle.

* Satisfied clippy on passing Path

* Clarified highest priority runtime directory purpose

* Further clarified multiple runtime details in book

Also gave markdown headings to subsections.

Fixed a error with table indentation not building
table that also appears present on master.

---------

Co-authored-by: Paul Scott <paul.scott@anu.edu.au>
Co-authored-by: Blaž Hrastnik <blaz@mxxn.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-helix-term Area: Helix term improvements A-packaging Area: Packaging and bundling A-tree-sitter Area: Tree-sitter C-enhancement Category: Improvements
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants
@paul-scott @the-mikedavis @amtoine @domust and others