emacs: Add basic tree-sitter support

[clhodapp]

Description of changes

From the commit message:

This commit adds basic support for tree-sitter in the emacs build, such that (if the user opts into tree-sitter support), tree-sitter will be enabled and binary library files for tree-sitter can be included in the lib directory of packages passed to emacsWithPackages. The libraries will be aggregated and included in treesit-extra-load-path.

The previous pattern for this in the community was to add tree-sitter libaries by patching emacs's RUNPATH with patchelf in a post-fixup phase. However, this has the substantial drawback that two different emacs installations with different lists of available tree-sitter libraries must be entirely separate builds. By supplying the tree-sitter libraries in the wrapping layer of emacsWithpackages, it becomes possible to share a single, more-cacheable "core emacs".

This support defaults to "on" only in emacs 29 and up, since previous versions do not support tree-sitter out of the box.

Additional notes:

• I don't believe that this should cause a rebuild of anything currently published in nixpkgs, since nixpkgs doens't currently contain any emacs 29 packages directly. I checked the hashes of several top-level emacs packages and they appeared unchanged. So, at present, this is more in service of users who are using the infrastructure from nixpkgs to build their own emacs 29 packages out-of-tree.
• I believe that it's still appropriate to incrementally start putting stuff to support of emacs 29 into nixpkgs because it won't be too long before it's actually out. Also, there's already some stuff in nixpkgs to help support emacs 29.
• I have tested this with my own out-of-tree emacs 29 package and it seems to work !

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 23.05 Release Notes (or backporting 22.11 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

[marsam]

I think you also need to add TREE_SITTER_LIBS. See:
https://github.com/nix-community/emacs-overlay/blob/2efd7c8d60ce0750097bbd327ec083e3ce545b31/overlays/emacs.nix#L78-L104

[clhodapp]

Does setting that variable on the emacs mkDerivation args actually do anything?

I'm not aware of anything that reads it and everything seems to work just fine without it on my Linux system. Is it possibly relevant on Darwin?

[marsam]

Sorry, you are right, it's not needed. I was using the emacs overlay as reference, but it seems to be outdated.

[clhodapp]

No worries! I just followed the upstream instructions from Emacs.

[clhodapp]

@adisbladis Do you think you could review this?

It seems like some others are eager to see this go in ( ❤️'s on the PR) plus, selfishly, it would let me stop having to update my nixpkgs fork 😉.

[adisbladis]

This looks great!

[clhodapp]

Thanks for the merge!