magit ediff on unstaged file leads to emacs freeze.

[vibrys]

Description

magit ediff command on unstaged file leads to emacs freeze.
This does not happen at every ediff command. Roughly 30% visits lead to freeze. Remaining ones behave correctly.
It never happened when ediffing file change from an already commited change.
This also does not happen on my non-spacemacs configuration, where ediff is also within the same single window.

Reproduction guide

• Start Emacs
• visit project directory
• SPC g s - magit-status
• edit some project file
• go back go magit status buffer.
• g to refresh.
• point the unstaged file changed.
• e - ediff session will popup.
• visit next/previous differences with n/p

at some point emacs stops responding on n/p keypresses and it starts consuming 100% CPU.
Subsequent C-g makes emacs back from freeze, but cursor is left within most left source code window, not within ediff bottom line.

System Info

• OS: gnu/linux
• Emacs: 24.5.1
• Spacemacs: 0.105.9
• Spacemacs branch: master (rev. 0562f05)
• Distribution: spacemacs
• Editing style: emacs
• Completion: helm
• Layers:

(auto-completion emacs-lisp git syntax-checking python c-c++ cscope themes-megapack)

Backtrace

after toggle-debug-on-quit, C-g tells:

Debugger entered--Lisp error: (quit)
redisplay_internal\ (C\ function)()

[duerrp]

Ediff from magit is a nice feature, thanks for reminding me of it. I cannot reproduce your problem though - seems to work nicely...

[duerrp]

Could this be related to Issue #4474?

[vibrys]

I do confirm, #4474 gets reproduced on my setup.
There is one difference:
freeze happens at 2nd diff chunk, not after the 7th one.

best regards.

[magnetophon]

@vibrys I tested again on 8825a06, and it seems fixed here.

[deb0ch]

Checking on this for the Automnal cleanup (#7344), what is the status on this ?

@vibrys Can you please confirm if it is fixed or not so we can close it (or not) ?

[vibrys]

Sorry for late answer.

I switched to spacemacs devel a few months ago to make sure I'm working with right code.
Unfortunatelly I'm still observing freezes.

regards,
Mat

[vibrys]

to be more precise, the last spacemacs devel version I observed freezes on is cf951a5.

[d12frosted]

@vibrys could you please checkout on the latest Spacemacs version (v0.200.x) or on latest develop?

[vibrys]

I did happen today again.
I working with version 72d87de (develop branch). SHould it contain the fix?

Tomorrow, I'll use newest version (f0803bc) and test it.

regards

[vibrys]

I did happen again today on f0803bc develop.

regards,
Mat

[bmag]

In #7366, @randre03 has also been experiencing freezes with ediff, but from SPC f e D and not magit. @vibrys just to make sure, if you create two files with the same content as the two versions of the same file that magit-ediff choked on, and run a regular ediff (M-x ediff) on them, does it also freeze? (I suspect it will)
If you can upload a pair of files that cause ediff to freeze, and an up-to-date SPC h d s info, I'll try to repro it on my machine.

[CeleritasCelery]

I have also experienced the same issue (magit ediff freezes Spacemacs at 2nd diff hunk, Emacs CPU at 100%). However, as @bmag hypothesized, I have had this same issue with just regular ediff so I don't think it is a magit problem.

[vibrys]

I copied two files, the one before change and the one after change, then opened these two buffers within two adjacent emacs windows, then did M-x ediff-buffers, and no freeze happened.

Unfortunatelly I cannot send buffer contents. I have prepared branch which is 100% reproducible. I'll try to start magit without spacemacs. Then I'll try to see the difference with magit, but after change has been staged, then commited. Maybe I should send some signal to emacs while it is in feeze, with some debug-on-quit? Then see what is emacs stack trace?

[CeleritasCelery]

I am able to reproduce the problem on the two files located below. Seems to break after the 5th diff. Would be useful if someone could download these files and see if they can reproduce the same problem.
file1.txt
file2.txt

The files differ only in white space.
using 0.200.1 with Emacs 24.5.1

[bmag]

@PrgrmAtCeleritas thanks for the files. Unfortunately, for some reason I am still unable to reproduce the bug on Emacs 24.5.1 with Spacemacs 0.200.3. Since @vibrys discovered that the display engine hangs (in redisplay-internal), I suspect the combination of major mode's syntax highlighting, Ediff's highlighting, and maybe also Flycheck (from syntax-checking layer), somehow confuses the display engine.

This is the setup I tried to repro with:

System Info 💻

• OS: gnu/linux
• Emacs: 24.5.1
• Spacemacs: 0.200.3
• Spacemacs branch: master (rev. b7e51d7)
• Graphic display: t
• Distribution: spacemacs
• Editing style: vim
• Completion: helm
• Layers:

(helm emacs-lisp)

@PrgrmAtCeleritas can you try and repro with this minimal setup? If it doesn't repro, then bisecting your config should lead you to the problematic (combination of) layers/config. If you can share your system information (SPC h d s), that would be nice.

Also, are these really txt files when you repro, or do they actually have a different extension? (I didn't recognize what prog language that was)

[CeleritasCelery]

They are perl (It won't let me upload files with that extension). However, I still encounter the bug whether they are txt or pl. The major-mode doesn't seem to make a difference.

System Info 💻

• OS: gnu/linux
• Emacs: 24.5.1
• Spacemacs: 0.200.1
• Spacemacs branch: nil (rev. 3706a42)
• Graphic display: t
• Distribution: spacemacs
• Editing style: vim
• Completion: helm
• Layers:

(csv javascript helm auto-completion better-defaults tmux ranger vinegar emacs-lisp git
     (shell :variables shell-default-height 30 shell-default-position 'bottom)
     syntax-checking version-control)

I will try with a minimal setup and see what happens.

[CeleritasCelery]

I tried it with the minimal setup below.

System Info 💻

• OS: gnu/linux
• Emacs: 24.5.1
• Spacemacs: 0.200.3
• Spacemacs branch: nil (rev. b7e51d7)
• Graphic display: t
• Distribution: spacemacs
• Editing style: vim
• Completion: helm
• Layers:

(helm emacs-lisp)

I still encounter the same problem. I also tried it with stock Emacs 24.5.1 and it has no problems, so it is not something wrong with my Emacs itself.

Not sure if this has already been discovered, but given what @vibrys uncovered about the display engine hanging I tried running Spacemacs Ediff in the terminal with my full setup and it does not have any problems. This may provide a temporary workaround until this issue is resolved.

[CeleritasCelery]

I went ahead and used debug-on-quit and got a pretty detailed backtrace. spacemacs/linum-update-window-scale-fix seems to show up right before the call to the redisplay_internal, which freezes the program.

@bmag or @d12frosted is there someone within the Spacemacs project who knows enough to look at this backtrace and get more insight about what may have caused the problem?

backtrace file
(FYI the backtrace has a lot of unprintable characters)

[CeleritasCelery]

I have another update which will hopefully help resolve this issue. I did some more debugging (This issue is really painful) and I tracked down the source of the problem to the which-key package. If I add which-key to my dotspacemacs-excluded-packages then I do not encounter this bug and ediff works fine. I am using which-key version 20161005.1154.

That is one thing I found. The other has to with the function spacemacs-base/init-ediff in layers/+distributions/spacemacs-base/packages.el. If I remove the line ediff-window-setup-function 'ediff-setup-windows-plain from defaults, ediff works fine and I have no problems. So I can either use ediff-setup-windows-plain, or the which-key package, but not both or things break. There seems to be some conflict between which-key and ediff trying to control the windows.

I would be very interested to see if @vibrys can produce the same results with the same changes.

I am not sure what the Spacemacs Collaborators want to do about this issue since it is clearly not affecting everyone. However I think I have found a fix for those people who are encountering this problem. Just remove that one line and it should work, at least it does for me.

[bmag]

According to the backtrace provided by @PrgrmAtCeleritas (thank you!), C-g was pressed during the execution of spacemacs/linum-update-window-scale-fix, which is an :after advice of linum-update-window, inside linum-after-scroll inside the redisplay engine. This suggests that our advice might cause an infinite loop of calling the redisplay engine. spacemacs/linum-update-window-scale-fix calls set-window-margins, which calls apply_window_adjustment, which calls wset_redisplay, which might trigger a new redisplay (but I'm not familiar with the code, so take it with a grain of salt). However, I was not able to reproduce the issue even with dotspacemacs-line-numbers set to t or relative. @PrgrmAtCeleritas, can you confirm that the bug is reproducible with dotspacemacs-line-numbers set to nil, so we can rule it out?

If I remove the line ediff-window-setup-function 'ediff-setup-windows-plain from defaults

Instead, you can add this line to user-config: (it will override Spacemacs' settings)

(setq ediff-window-setup-function 'ediff-setup-windows-default)

Can anyone else who is facing this problem confirm that setting ediff-window-setup-function in user-config is a valid work-around?

I am not sure what the Spacemacs Collaborators want to do about this issue since it is clearly not affecting everyone.

We'd like to fix the issue, of course, but I fear it might take some serious expertise, especially regarding the display engine. Since we barely have a clue of what's wrong, our concern right now is to isolate the cause for the issue as much as possible. Unfortunately I still can't reproduce the issue, so I have to ask you to take on yourself most of the burden of debugging.

Can you reproduce it with spacemacs-base? Change dotspacemacs-distribution to spacemacs-base and add these layers to ``:

spacemacs-completion
spacemacs-layouts
spacemacs-editing
spacemacs-editing-visual
spacemacs-evil
spacemacs-language
spacemacs-misc
spacemacs-purpose ;; only if you're on develop, doesn't exist on master
spacemacs-ui
spacemacs-ui-visual
spacemacs-org

This has the same effect as using the default spacemacs distribution. Once you've done this, you can start disabling layers (by commenting them out) to test which ones are necessary to reproduce the bug. which-key and ediff packages belong to spacemacs-base, and evil-ediff belongs to spacemacs-evil, so you might be able to reproduce the bug with only the spacemacs-evil layer enabled. Be sure to change dotspacemacs-install-packages to used-but-keep-unused beforehand, so you don't have to reinstall packages every time you change the configuration.

You can also try to reproduce the issue with plain Emacs, which-key and evil-ediff. This ~/.emacs file might help:

(require 'package)
;; if you use a non-nil `dotspacemacs-elpa-subdirectory':
;; (setq package-user-dir "path/to/elpa")
(package-initialize)

(require 'which-key)
(require 'evil-ediff)

;; which-key configuration goes here
;; ediff configuration goes here
;; evil-ediff configuration goes here

@vibrys we would like to hear your feedback as well.

[vibrys]

@bmag, the following workaround works ok:
(setq ediff-window-setup-function 'ediff-setup-windows-default)

The following does not reproduce the problem. ~/.emacs

(require 'package)
(package-initialize)

  (add-to-list 'package-archives '("melpa" . "http://melpa.milkbox.net/packages/") t)
  (add-to-list 'package-archives '("marmalade" . "http://marmalade-repo.org/packages/") t)
  (add-to-list 'package-archives '("elpy" . "https://jorgenschaefer.github.io/packages/"))
(require 'which-key)   ;; I also installed this package of course
(require 'evil-ediff)     ;; the same here

Additionally I installed magit.
then

1. restarted emacs
2. started ediff session (from magit, because this one does the reproduction).

No reproduce.

additionally I did
2. M-x ediff-toggle-multiframe #. this makes ediff control window be part of main frame (ie not the separate one).

no reproduce.

what do You want me to do as next step?

regards

[bmag]

@vibrys the ~/.emacs needs to contain configuration for ediff, which-key and evil-ediff, similar to what Spacemacs has. For example, it needs to set ediff-window-setup-function to what Spacemacs sets it. By the way, in normal usage package-archives is modified before calling package-intialize, but in this case you don't need to change package-archives at all. (I assume the packages were already installed by Spacemacs)

If you can't repro with a ~/.emacs, I'd prefer you try the spacemacs-base approach: change dotspacemacs-install-packages to used-but-keep-unused, change dotspacemacs-distribution to spacemacs-base, add the spacemacs-* layers (they are listed above). This setup should reproduce the bug. Now start disabling (commenting-out) some of the spacemacs-* layers, so we know which parts of Spacemacs we can ignore. The goal is to reproduce with spacemacs-base and the minimal number of layers. It is possible that you only need the spacemacs-evil layer to reproduce the bug. (if this works we can start eliminating packages)

[vibrys]

@bmag,

spacemacs-ui-visual ;; problem appears after adding this layer.

[vibrys]

@PrgrmAtCeleritas, configuring the following does not help on my config:

dotspacemacs-excluded-packages '(which-key)

version used:
which-key-20161031.1056

There is one thing I don't like though.
Even after I ?disabled? which-key, it is still reachable through M-x find-library. Can it be like that?

[vibrys]

@bmag,

Additionally it is enough to enable spacemacs-ui-visual to have problem reproduced, ie. no other layers from the list above need to be added to spacemacs-base.

[CeleritasCelery]

I think it also depends on the font. Some fonts are fine with 1.1 and others are not. At least in my testing.

[ghost]

@PrgrmAtCeleritas I would be careful about assuming things like that. This bug is extremely random (probably a race condition somewhere). So you can go for hours without seeing it and then suddenly it happens again.

I was just editing a diff with magit and ediff and had no problem for 20 minutes, then it froze.

For now I will type "w c" (write buffer C to disk) a lot to save my work as I progress through the file.

[CeleritasCelery]

Very consistent for me. When I have created a condition where it will freeze I make it freeze every single time. Seems to be the same for the others as well. You are the only one that has mentioned inconsistency.

[ghost]

Well yeah it isn't consistent for me. <SPC f e D n> froze every time 90% of the time. ;)

[prestancedesign]

Hi !

I'm Spacemacs user about 1 month and the ediff freezing bug are the only things is annoying me a lot and that I had to take time to fix on my todo list .
Whenever I used ediff, I would squeeze the buttocks :)

I don't have tested more than 1 hour but toggle-mode-line-off seems the best workaround...this keeps same windows positions, doesn't touch modeline unicode, etc.

  (add-hook 'ediff-before-setup-hook 'spacemacs/toggle-mode-line-off)
  (add-hook 'ediff-quit-hook 'spacemacs/toggle-mode-line-on)

Thx !

[ghost]

@prestancedesign Cool. Is that 100% a solution against the hangs?

[prestancedesign]

@SteveJobzniak Yes that seems. Yesteday afternoon, only one freeze when I ediff a local file against this remote version via this package https://github.com/cjohansson/emacs-ssh-deploy.
I guess, I must adapt the hook when I ediff with this tool.

I'll see on a longer use this week to see if this fix fixes that at 100% and keep you up to date.
Have a nice day !

[ghost]

@prestancedesign Cool, thanks for the information! Have a nice day too! 👍

[Apakollaps]

I have been going crazy from ediff freezes for a long time now, too bad I failed to find this issue earlier. I work mainly with hg repos, but from what I've read so far it's not git-related.

Initial tests indicate that the toggling of mode line in the ediff hooks works, we'll see if it stands the test of time...

Cheers everyone for your efforts!

[Apakollaps]

Toggling the mode line did not do the trick, unfortunately. Locks became much more rare, but didn't take long for a lockup to happen. Disabling spaceline altogether (as described in one of the comments) seems to do the trick.

[agzam]

It definitely happens less often than it used to, but still happens. It just froze when I tried to search in ediff middle window. Oh, and that seems to be very reproducible. File even doesn't have to be unstaged. SPC g s, select a file, e, jump to a window, evil search... Boom - Emacs is a zombie...

[agzam]

goddamit... ediff has become just completely unusable... can't do anything anymore... try for example resizing one of the windows... sigh... whoever finds nice workaround (or guaranteed fix) I'll send them nice bottle of scotch.

[CeleritasCelery]

@agzam Does disabling spaceline not do the trick?

[duianto]

I'm unable to reproduce the crash or freezing in Ubuntu 16.10

System Info 💻

• OS: gnu/linux
• Emacs: 26.0.50
• Spacemacs: 0.200.9
• Spacemacs branch: develop (rev. a213729)
• Graphic display: t
• Distribution: spacemacs
• Editing style: vim
• Completion: helm
• Layers:

(helm emacs-lisp git org version-control)

• System configuration features: XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GCONF GSETTINGS NOTIFY LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 MODULES XWIDGETS LIBSYSTEMD

It's running in virtual box with windows 10 as the host, if that could be preventing the crash some how. It doesn't happen in windows 10 either, but the reports here are for Linux, so it might not be relevant, but it's another data point:

System Info 💻

• OS: windows-nt
• Emacs: 25.2.2
• Spacemacs: 0.200.9
• Spacemacs branch: develop (rev. a213729)
• Graphic display: t
• Distribution: spacemacs
• Editing style: vim
• Completion: helm
• Layers:

(autohotkey markdown ranger emacs-lisp git github helm org syntax-checking version-control)

• System configuration features: XPM JPEG TIFF GIF PNG RSVG SOUND NOTIFY ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS MODULES

[CeleritasCelery]

This is a "works fine for some people" issue. I think it is system dependent as best I can figure.

[agzam]

@CeleritasCelery yup, it might be affecting only Mac users. My Linux laptop not with me right now. Can't confirm if that breaks there.

[yatesco]

I am still seeing hard locks on macos High Sierra running latest develop.

[braham-snyder]

edit: I'd forgotten that a setq for dotspacemacs-mode-line-unicode-symbols (default t) already exists in dotspacemacs/init -- you're probably better off setting that to nil than adding an additional setq in dotspacemacs/user-init as described below (even though it would also work):

Have you tried adding the following to dotspacemacs/user-init (note that -- unintuitively -- setting this in dotspacemacs/user-config will succeed at setting this variable to nil, but will not have any other effects whatsoever)?

(setq dotspacemacs-mode-line-unicode-symbols nil)

In the unlikely event that that trick does not avoid the freezes, you might try double checking whether your mode-line includes any unicode after it has already frozen (since some unicode symbols are only conditionally displayed, like workspace numbers -- which incidentally also happens to make dotspacemacs-mode-line-unicode-symbols's above quirk more difficult to catch).

[marcoslhc]

Still happening to me 100% of the time. I've tried disabling spaceline, mode-line and setting dotspacemacs-mode-line-unicode-symbols to nil to no avail.
It happens in master and develop
Also Spacemacs won't load my settings well (like color scheme) Now it's practically unusable.

System Info 💻

• OS: darwin
• Emacs: 25.3.1
• Spacemacs: 0.300.0
• Spacemacs branch: develop (rev. f3b6a22)
• Graphic display: t
• Distribution: spacemacs
• Editing style: vim
• Completion: helm
• Layers:

(haskell helm emacs-lisp git markdown html react javascript clojure common-lisp csv php ruby typescript vimscript go yaml java nginx docker ansible vagrant osx colors git github themes-megapack)

• System configuration features: JPEG RSVG IMAGEMAGICK NOTIFY ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS NS MODULES

[braham-snyder]

@marcoslhc where did you change dotspacemacs-mode-line-unicode-symbols? Unintuitively -- setting this in dotspacemacs/user-config (or interactively, after starting Spacemacs) will succeed at setting the variable to nil, but will not fix the issue.

Did you already try changing the line in dotspacemacs/init that sets dotspacemacs-mode-line-unicode-symbols to t? If you change that existing line in your dotfile (or at least the default dotfile) to instead set the variable to nil, then it ought to work (because only then will it be modified early enough during startup to take effect).

If that does not work (after a restart), and toggle-debug-on-quit -> "C-g" still gives:

Debugger entered--Lisp error: (quit)
redisplay_internal\ (C\ function)()

My best guess is that you have unicode symbols somewhere else that are causing issues -- maybe a file tree plugin that displays file icons, or something?

[stew]

for me, the work around for this is (customize-variable ediff-window-setup-function) and chosing multi-frame

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Please let us know if this issue is still valid!