Are you interested in yet another package manager?

[raxod502]

I've created a new package manager straight.el as an alternative to package.el/Quelpa/Cask/Pallet/el-get/Borg. It appears that some people like it enough to mess around with hacking it into Spacemacs: radian-software/straight.el#128.

Would you be interested in a pull request that adds an option to Spacemacs for using straight.el instead of package.el + Quelpa, much like the existing options for Ivy vs. Helm and Evil vs. One True Keybindings?

[JAremko]

@raxod502 Does it work with Windows ?

cc @syl20bnr

[raxod502]

No, see radian-software/straight.el#124.

[syl20bnr]

@raxod502 That's funny you opened this issue, we talked about your package last month. I'm considering it to back the layer system and to replace the current (not released) approach to create a spacelpa repository to freeze the version of packages for master branch. staight.el would be a far lighter solution than maintaining a whole elpa repository.

Would you be interested in a pull request that adds an option to Spacemacs for using straight.el instead of package.el + Quelpa

Yes of course but the current layer system relies a lot on package.el!
The layer system code is in core/code=configuration-layer.el: https://github.com/syl20bnr/spacemacs/blob/develop/core/core-configuration-layer.el

much like the existing options for Ivy vs. Helm

As it is in core we don't need to support several alternatives, we would simply replace the current backend by straight.el.

A few questions:

1. What are the performance of straight.el vs. package.el when installing several hundreds of packages ?

2. Do you use package.el behind the scene or are you reimplementing some package.el feature ?

3. What's your plan for the future of this project ? Do you consider to publish it to Gnu ELPA ?

[syl20bnr]

Also like @JAremko said, we need windows support ;-)

[raxod502]

1. What are the performance of straight.el vs. package.el when installing several hundreds of packages ?

Installing packages is more or less the same speed as package.el in my experience. You might think that doing full Git clones would be slower—and it is, for big repos like Org—but for the most part there's no observable difference since the deciding factor is network latency. (Since package.el relies on a central server, it could speed this up by using bulk calls… but it doesn't.)

If you'd like me to do an official benchmark, I can. It would be pretty trivial.

More relevant is activating hundreds of packages, since that's what affects standard init time. Here I think straight.el is slower than package.el, at least at the moment. See radian-software/straight.el#9, radian-software/straight.el#118, radian-software/straight.el#119, radian-software/straight.el#129. However, note that it's still pretty fast. For example, activating Spacemacs' 320+ packages takes 3.6 seconds for me; see radian-software/straight.el#128 (comment). And also, I think straight.el is actually going to be faster than package.el once those problems are resolved.

2. Do you use package.el behind the scene or are you reimplementing some package.el feature ?

No. straight.el is a fully deterministic package manager, while package.el is inherently nondeterministic (see the comparison). Thus, using package.el at all would defeat the purpose of straight.el.

straight.el has no dependencies. All code was written by me, specifically for straight.el. This includes byte-compilation, autoload generation, dependency management, package downloading, etc.

Honestly, writing a package manager is really darned easy—at least the core part (downloading packages, byte-compiling them, generating autoloads, and activating them). The complexities in straight.el are in wringing an immutable environment out of the mutable mess that is Emacs; in providing interactive workflows for managing hundreds of Git repositories efficiently; and in tracking and merging recipes intelligently.

3. What's your plan for the future of this project ? Do you consider to publish in Gnu ELPA ?

My immediate plan is to make a 1.0 release. Since I have just started school, development will be slowed, but I certainly do not intend to abandon the project, since no other package managers exist that do what I want.

Regarding GNU ELPA, there are two problems with this. Firstly, installing straight.el via package.el makes no sense. Secondly, I am not willing to assign copyright for my code to the FSF, and AFAIK this is a requirement for adding a package to GNU ELPA.

Also like @JAremko said, we need windows support ;-)

Well, I guess this is a blocker if we're going to swap out the backend entirely rather than adding an option. I'm all for adding Windows support to straight.el, but unfortunately it won't happen until someone submits a pull request, since I don't have a Windows machine.

[fiveNinePlusR]

seems like this could be a better package manager now that windows support is there... any insights @syl20bnr?

[raxod502]

I believe some issues still need to be closed before straight.el can be considered a drop-in replacement for package.el in Spacemacs:

• Install packages directly, then convert to Git repositories later. This improves installation performance to be on par with package.el.
• Allow running external commands when building packages. This allows some important big packages like AUCTeX to be installed correctly.

Maybe some others. Would be nice to hear from @syl20bnr what is most important, even if I don't have the time right at the moment. (Perhaps other people can pitch in on these issues in order to improve the package management situation for Spacemacs!)

[cyruseuros]

Is this still in the works? It sounds absolutely brilliant.

Just thought I'd say I have completely replaced my dotspacemacs-additional-packages with straight.el in my dotspacemacs/user-init for the sake of dependency management and working on a package of my own that needs to be loaded in early startup. It's remarkable how well it plays with Spacemacs. The only thing that broke was evil maps for magit which I solved by installing evil-magit (and it seems like it will be a fairly easy fix later on). Here is my dotspacemacs/user-init:

(defun dotspacemacs/user-init ()
  "Initialization for user code:
This function is called immediately after `dotspacemacs/init', before layer
configuration.
It is mostly for variables that should be set before packages are loaded.
If you are unsure, try setting them in `dotspacemacs/user-config' first."
  ;; fix font issue with server
  (spacemacs|do-after-display-system-init
   (spacemacs/set-default-font dotspacemacs-default-font))

  ;; straight
  (defvar bootstrap-version)
  (defvar straight-repository-branch "develop")
  (defvar straight-recipes-emacsmirror-use-mirror t)
  (let ((bootstrap-file (expand-file-name
                         "straight/repos/straight.el/bootstrap.el"
                         user-emacs-directory))
        (bootstrap-version 5))
    (unless (file-exists-p bootstrap-file)
      (with-current-buffer (url-retrieve-synchronously
                            "https://raw.githubusercontent.com/raxod502/straight.el/develop/install.el"
                            'silent 'inhibit-cookies)
        (goto-char (point-max))
        (eval-print-last-sexp)))
    (load bootstrap-file nil 'nomessage))
  (straight-use-package 'use-package)

  (use-package ewal-spacemacs-themes
    :straight t
    :defer nil
    :init (setq ewal-force-tty-colors nil
                ewal-force-tty-colors-daemon nil)
    :config (when (ewal-load-wal-colors)
              (load-theme 'ewal-spacemacs-modern t))
    :hook (spacemacs-post-user-config
           . (lambda () (enable-theme 'ewal-spacemacs-modern))))

  (use-package ewal-evil-cursors
    :straight t
    :defer nil
    :after (ewal-spacemacs-themes)
    :config (when (ewal-load-wal-colors)
              (ewal-evil-cursors-get-emacs-colors :apply t)
              (ewal-evil-cursors-get-spacemacs-colors :apply t)))

  ;; fixes emacs state bug
  (use-package evil-magit
    :straight t
    :after (magit))

  (use-package solidity-mode :straight t :defer t)
  (use-package python-docstring :straight t :defer t)
  (use-package sphinx-doc :straight t :defer t)
  (use-package nodejs-repl :straight t :defer t)
  (use-package xresources-theme :straight t :defer t)
  (use-package terminal-here :straight t :defer t)
  (use-package helm-recoll :straight t :defer t)
  (use-package langtool :straight t :defer t)
  )

[raxod502]

It's unlikely to happen soon without help from the community, as I am fairly busy. Pull requests will be reviewed and accepted, though.

[cyruseuros]

Where do you need more help? the Straight or Spacemacs side?

[raxod502]

The most important piece of work is radian-software/straight.el#149, i.e. "snapshot installation" as I like to call it. My best idea is at radian-software/straight.el#149 (comment). This is necessary because the time and disk space usage of straight.el is significantly greater than package.el, which would be unacceptable for end-user experience.

I think it would also be necessary to fix radian-software/straight.el#279, since Spacemacs users expect full compatibility with MELPA recipes. Similarly, radian-software/straight.el#265 is bad for end user experience.

There are many other issues which I think are important, but the above three are the only ones I would consider truly blocking at this time.

As for the Spacemacs side, no work at all has been done to my knowledge. The place to start would be to look at all the places where package.el is used in Spacemacs, and see if straight.el provides the functionality that is used. Presumably, doing this will produce a couple more action items on the straight.el side.

[cyruseuros]

@raxod502 I'll take a look to see where I can help. Days are packed too, but this is just so fascinating.
PS: @syl20bnr, I know the straight.el packages are probably not being tracked, but I have about 500 packages installed in Spacemacs. Startup was approximately 5s with 8 additional-packages. After porting them to straight my startup time is reportedly 1.8s less. Based on snappiness alone I can guarantee more than 50% of that is an actual increase in speed, and not just the fact that the replacement packages aren't being tracked.

This is probably not a straight.el > package.el thing, but an additional-packages thing, but still. Also, with straight, we could completely get rid of that variable.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Please let us know if this issue is still valid!

[progfolio]

not stale

[raxod502]

For what it's worth, I intend to put my support behind Doom Emacs rather than Spacemacs going forward, so it is unlikely that I will follow up on this issue. I would, of course, be happy to advise someone who wishes to take on the task.

[emacs18]

I hope that PR I just submitted helps move this discussion along. I would like to be able to use straight rather than package.el for some situations.