Please consider maintaining the Emacs-Lisp files in a separate repository

[tarsius]

This repository contains one or more Emacs-Lisp libraries, which are distributed on Melpa for easy installation using Emacs' official package manager (package.el) and are also mirrored on the Emacsmirror.

This project/repository is about much more than just the elisp libraries. These libraries are tucked away in some "contrib" or "utilities" directory. The maintainers of the project as a whole likely care little about these libraries, which makes it a less than optimal place to maintain them. The maintainers will always have something more important to deal with and potential contributors are likely scared away by the additional processes and conventions that such a large project needs, but which are overkill when looking at the elisp libraries in isolation.

I would like to propose that the Emacs-Lips libraries are maintained in a separate repository. Doing that would solve a few issues and have additional benefits:

• Distributing a package that originates in such a large repository puts additional strain on the Melpa server. Fetching the repository takes longer and they require more space. We are already trying to deal with the latter, but not very successfully and this leads to special-cases in the tooling, which come with additional maintenance costs. See Strategy for huge repositories melpa/melpa#5361, also for some past discussion.

• I also maintain the Emacsmirror and here the costs are much higher than for Melpa. The Emacsmirror does not just distribute installable snapshots of a package, as Melpa does.

  Instead it mirrors the upstream repository. But in cases like this, where the repository contains much more than just the elisp libraries, it filters the history, using tools such as git filter-repo and git subtree. Unfortunately these tools are not very reliable and highly inefficient when continuously rewriting large repositories like this one.

  Currently this affects just 33 out of 8485 on the Emacsmirror, and the disproportionate amount of work that it takes to maintain the tooling for these special-cases is just not worth it. So I have decided to phase out support, but not before making an effort to get as many of these Emacs package slit out of the repositories they are currently being maintained in.

• For maintainers of this repository any issue about the elisp libraries is likely just more noise. They have better things to do.

• On the other hand, the original authors of the elisp libraries and people who have contributed to them, likely have notifications for this repository disabled because they only contribute to the elisp parts and there would just be too much noise otherwise. So issues and pull-requests by users are likely to go unnoticed for a long time.

  The author and contributors likely have to be explicitly summoned, as I am doing now:
  @Fanael @evmar @riannucci @

• Potential contributors are discouraged when their pull-requests are being "ignored" or they might be discouraged by the additional requirements imposed by the project as a whole, and not even try to contribute as a result.

• Users who install elisp packages by manually cloning their repositories and then manually adding the appropriate directories to the load-path, are forced to clone a huge repository just to get their hands on a few elisp libraries in some subdirectory. (Or they can use the filtered repositories on the Emacsmirror, but as I mentioned, those are going away.)

So in summary, I think there are many downsides to maintaining the elisp libraries in this repository. I am not sure there are any upsides, except that it is the status quo.

Moving the elip libraries out of here is some work, that someone has to do, but most of that work is already done. You can just clone the respective repository from the Emacsmirror (at https://github.com/emacsmirror/ninja-mode) and publish it somewhere else. (Please check that no recent history is missing and if the mirror repository is not uptodate, then ping me.)

Once you have done that tell me about it here, and I will take care of updating Melpa and the Emacsmirror.

Thanks for considering!
Jonas

[evmar]

Looking at the history of the file
https://github.com/ninja-build/ninja/commits/master/misc/ninja-mode.el
It basically hasn't been touched in eight years, so I don't really think there's much "maintenance" going on.

I think your message was automated, but I wonder if there's a better home for this code? For example, is there a place where emacs modes for other languages are maintained?

[tarsius]

I did indeed go through a list of packages that are maintained in larger repositories, and used essentially the same message template for all of them.

In more than one instance the elisp library hasn't been touched in years and I was considering just freezing these packages to the current version on the Emacsmirror and to then change Melpa to get these packages from there. But just because a package hasn't been touched in years, that doesn't necessarily mean that it won't suddenly be worked on again, so I decided to contact the upstreams first.

is there a place where emacs modes for other languages are maintained?

There is no such place where a collection of prog-modes are maintained, except for Emacs itself of course. Emacs probably isn't the best place for this mode though, and in any case adding it there, would likely involve quite a bit of work, which I can't really ask you to do.

I still think it would be best if you created a separate repository in this organization, but since you don't appear to expect to make any changes, the Emacsorphanage would also be a good place.

If you would like to do that, I can take care of almost all the work. All that would be left for you to do is to remove the library from this repository, and to maybe add a note pointing users at the new home. I can give you write access to the repository in the orphanage, if you wish, or you can leave it to me to merge any pull-requests, should any arrive.

[jhasse]

@tarsius it's been a while, but is your offer still on the table?

All that would be left for you to do is to remove the library from this repository, and to maybe add a note pointing users at the new home.

I could do that :)

[tarsius]

Yes, I would be happy to help, but because work is currently popping up left and right, there could be delays.

The first step is to rewrite the history to limit to ninja-mode.el. I was hoping I could come up with that in less than ten minutes, but unfortunately there are issues. But try to do this on your machine anyway. There's a change you won't encounter these issues.

$ git filter-repo --path misc/ninja-mode.el --path-rename misc/ninja-mode.el:ninja-mode.el
Parsed 2182 commitsfatal: Encountered commit-specific encoding iso-8859-8 in commit d709a37d14dd37610eea32991c6df8a1a62a25ee; use --reencode=[yes|no] to handle it
Parsed 2734 commits
Error: fast-export failed; see above.
fatal: stream ends early
fast-import: dumping crash report to .git/fast_import_crash_15210

$ git filter-repo --path misc/ninja-mode.el --path-rename misc/ninja-mode.el:ninja-mode.el --reencode=yes
git-filter-repo: error: unrecognized arguments: --reencode=yes

Turns out --reencode=yes is an argument for the internally used git fast-import. I couldn't quickly figure out if and how one can get git filter-repo to pass additional arguments to git fast-import.

I spotted --preserve-commit-encoding. I would prefer to fix commit encoding, but the rewrite succeeding with unfixed encoding is still better than the rewrite failing altogether.

$ git filter-repo --preserve-commit-encoding --path misc/ninja-mode.el --path-rename misc/ninja-mode.el:ninja-mode.el
Error: need git >= 2.23.0
$ git version
git version 2.46.0

Eh, what?

It appears that on my system git filter-repo ends up trying to use a second older git version laying around somethere. I don't think that is actually the case and it doing that would still be a bug somewhere. That being said, maybe on your system it works.

Sadly this is exactly the sort of complication I was thinking of when starting to write this message but before trying to come up with the appropriate git filter-repo invocation.

The time I have allocated for this for today is already more than used up, and I will have to go work on other things now.

You could try using the older git filter-branch instead.

[tarsius]

You could of course also decide that preserving the history of this file is not actually worth it and start over with a new "initial commit".

[jhasse]

out of interest: What Linux distribution are you using?

git filter-repo --path misc/ninja-mode.el --path-rename misc/ninja-mode.el:ninja-mode.el worked: https://github.com/jhasse/ninja-emacs

[tarsius]

out of interest: What Linux distribution are you using?

Guix, which is wonderful. Don't take these woos as an indication that something is wrong with that. I'm juggling and cannot investigate anything but the most urgent issues. (Such as why did my keyboard start dropping out today, on top of everything else, and what exactly are the steps again to flash new firmware?) I might have installed some of the involved tools manually and am stuck on old buggy versions, I just don't have the time to investigate.

git filter-repo --path misc/ninja-mode.el --path-rename misc/ninja-mode.el:ninja-mode.el worked: https://github.com/jhasse/ninja-emacs

Great! It looks sane.

Will update melpa and mirror later.

[jhasse]

Guix, which is wonderful. Don't take these woos as an indication that something is wrong with that.

Fascinating! I've heard of that, but never met someone that uses it :)

Will update melpa and mirror later.

Thanks!

[tarsius]

Please add ;; SPDX-License-Identifier: Apache-2.0 to the header.

You might also want to change the value of URL and move ;; Copyright ... before the headers (but leave the permission statement were it currently is), as this is conventionally done (see any library that is part of Emacs).

You should also tag a release and/or add a Package-Version header. If you do both, then it is very important that you tag the very commit that bumps the Package-Version header, because otherwise different tools would disagree on what commit is the release.

[jhasse]

I've invited you to the repository so that you can do the changes yourself - I think that's less error-prone as I don't know anything emacs.

[tarsius]

I've made most of these changes (and started using setq-local).

I have not added the version header. If you decide to do that, I think it should be done like this:

 ;; Copyright 2011 Google Inc. All Rights Reserved.
 
+;; Package-Version: 2.0
 ;; Package-Requires: ((emacs "24.3"))
 ;; Keywords: languages

I recommend you use a version that is greater than the largest tag in ninja-build/ninja (v1.12.1).

Now that it is maintained in its own repository, you should remove misc/ninja-build.el from this repository.

I am giving up my commit access now. I already maintain a few dozen packages I have created myself and semi-maintain more than a hundred packages in the Emacsorphanage, and cannot take on any more work.

[jhasse]

Thx!

@evmar Can you import https://github.com/jhasse/ninja-emacs as a new repository on @ninja-build?

[evmar]

Better, I sent you an invite to own the ninja-build organization!

[jhasse]

Thanks :)

I've transferred the repository: https://github.com/ninja-build/ninja-emacs

I am giving up my commit access now. I already maintain a few dozen packages I have created myself and semi-maintain more than a hundred packages in the Emacsorphanage, and cannot take on any more work.

Okay. Still the new repository is a good idea because we can possibly find someone who is using emacs and wants to maintain the package.

Now that it is maintained in its own repository, you should remove misc/ninja-build.el from this repository.

Will do that next.

[tarsius]

I've updated Melpa and the Emacsmirror again.