Clojure: Emacs hangs after moving point around a `comment` form

[kommen]

Describe the bug
Emacs hangs with 100% CPU usage and can only be force killed

To Reproduce
Move point around the comment form here: https://github.com/nextjournal/glogi/blob/ee8889e44d0b208ab4638d089c6724deb6d24324/src/lambdaisland/glogi.cljs#L39-L42

Expected behavior
Emacs not hanging 😅

Which Language Server did you use
clojure-lsp

OS
macOS 10.15.7 (19H524)

Error callstack

Captured when Emacs hangs with 100% CPU usage, debug-on-quit enabled, and hitting C-g multiple times (hitting it once doesn't trigger it):

Debugger entered--Lisp error: (quit)
  clojure-sexp-starts-until-position(9)
  clojure-beginning-of-defun-function(1)
  beginning-of-defun-raw(nil)
  beginning-of-defun()
  clojure-font-lock-def-at-point(1)
  clojure-font-lock-extend-region-def()
  font-lock-default-fontify-region(1 10 nil)
  apply(font-lock-default-fontify-region 1 10 nil)
  #f(compiled-function (beg end &rest rest) #<bytecode -0xbb14a32eb27e9a7>)(1 10 nil)
  font-lock-fontify-region(1 10)
  #f(compiled-function (beg end) #<bytecode -0x194faac06963b207>)(1 10)
  font-lock-ensure()
  markdown-fontify-code-block-natively("clojure" 12 20)
  markdown-fontify-code-blocks-generic(markdown-match-gfm-code-blocks 84)
  markdown-fontify-gfm-code-blocks(84)
  font-lock-fontify-keywords-region(1 84 nil)
  font-lock-default-fontify-region(1 84 nil)
  font-lock-fontify-region(1 84)
  #f(compiled-function (beg end) #<bytecode -0x194faac06963b207>)(1 84)
  font-lock-ensure()
  lsp--fontlock-with-mode("```clojure\ncomment\n```\n\n----\n*/Users/kommen/work/g..." lsp--render-markdown)
  lsp--render-string("```clojure\ncomment\n```\n\n----\n*/Users/kommen/work/g..." "markdown")
  lsp--render-element(#<hash-table equal 2/2 0x1ff232da4c1d>)
  #f(compiled-function (contents server-id) #<bytecode -0x1d172dfdaec9aeb6>)(#<hash-table equal 2/2 0x1ff232da4c1d> clojure-lsp)
  apply(#f(compiled-function (contents server-id) #<bytecode -0x1d172dfdaec9aeb6>) #<hash-table equal 2/2 0x1ff232da4c1d> clojure-lsp nil)
  lsp-clients-extract-signature-on-hover(#<hash-table equal 2/2 0x1ff232da4c1d> clojure-lsp)
  lsp--handle-rendered-for-echo-area(#<hash-table equal 2/2 0x1ff232da4c1d>)
  lsp--render-on-hover-content(#<hash-table equal 2/2 0x1ff232da4c1d> nil)
  #<subr F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_229>(#<hash-table equal 2/2 0x1ff232da4bcf>)
  apply(#<subr F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_229> #<hash-table equal 2/2 0x1ff232da4bcf>)
  #f(compiled-function (&rest args) #<bytecode 0x14e45237d1458c28>)(#<hash-table equal 2/2 0x1ff232da4bcf>)
  #f(compiled-function (result) #<bytecode -0xe144ab71b90da>)(#<hash-table equal 2/2 0x1ff232da4bcf>)
  #f(compiled-function (result) #<bytecode 0x80d87792919d3d8>)(#<hash-table equal 2/2 0x1ff232da4bcf>)
  lsp--parser-on-message(#<hash-table equal 3/3 0x1ff232da4b81> #s(lsp--workspace :ewoc nil :server-capabilities #<hash-table equal 17/17 0x1ff233bff2ef> :registered-server-capabilities (#s(lsp--registered-capability :id "id" :method "workspace/didChangeWatchedFiles" :options #<hash-table equal 1/1 0x1ff24ac5772b>)) :root "/Users/kommen/work/glogi" :client #s(lsp--client :language-id nil :add-on? nil :new-connection (:connect #f(compiled-function (filter sentinel name environment-fn) #<bytecode 0x15b624abad0f1b4>) :test\? #f(compiled-function () #<bytecode 0x23cf4977d89075>)) :ignore-regexps nil :ignore-messages nil :notification-handlers #<hash-table equal 0/65 0x1ff23745d57d> :request-handlers #<hash-table equal 0/65 0x1ff23745dc3f> :response-handlers #<hash-table eql 1/65 0x1ff23745dc5f> :prefix-function nil :uri-handlers #<hash-table equal 1/65 0x1ff23745d457> :action-handlers #<hash-table equal 0/65 0x1ff23745d59d> :major-modes (clojure-mode clojurec-mode clojurescript-mode) :activation-fn nil :priority 0 :server-id clojure-lsp :multi-root nil :initialization-options (:dependency-scheme "jar") :custom-capabilities nil :library-folders-fn lsp-clojure--library-folders :before-file-open-fn nil :initialized-fn nil :remote? nil :completion-in-comments? nil :path->uri-fn nil :uri->path-fn nil :environment-fn nil :after-open-fn nil :async-request-handlers #<hash-table equal 0/65 0x1ff23745e43d> :download-server-fn #f(compiled-function (client callback error-callback update\?) #<bytecode 0x13cfb0240e2ee1e3>) :download-in-progress? nil :buffers nil) :host-root nil :proc #<process clojure-lsp> :cmd-proc #<process clojure-lsp> :buffers (#<buffer glogi.cljs>) :semantic-tokens-faces nil :semantic-tokens-modifier-faces nil :extra-client-capabilities nil :status initialized :metadata #<hash-table equal 0/65 0x1ff24b27cf3b> :watches #<hash-table equal 0/65 0x1ff24b27cfdf> :workspace-folders nil :last-id 0 :status-string nil :shutdown-action nil :diagnostics #<hash-table equal 1/65 0x1ff24af76531> :work-done-tokens #<hash-table equal 0/65 0x1ff24af76551>))
  #f(compiled-function (proc input) #<bytecode -0x14f2bddd95ddb199>)(#<process clojure-lsp> "Content-Length: 241\15\n\15\n{\"jsonrpc\":\"2.0\",\"id\":4,\"re...")
=

(setq lsp-print-io t)

=> Nothing gets logged into *lsp-log* while Emacs is using 100% CPU
Nothing gets logged into the clojure-lsp log file

Emacs Version: GNU Emacs 28.0.50 (build 1, x86_64-apple-darwin19.6.0, NS appkit-1894.60 Version 10.15.7 (Build 19H524)) of 2021-03-05

lsp version: LSP :: lsp-mode 20210307.1909, Emacs 28.0.50, darwin

lsp-clojure-server-info

LSP :: {:project-root "file:///Users/kommen/work/glogi",
 :project-settings {},
 :client-settings
 {:dependency-scheme "jar",
  :source-paths #{"src" "test"},
  :macro-defs {},
  :project-specs nil,
  :cljfmt {:indents {}},
  :document-formatting? true,
  :document-range-formatting? true},
 :port "NREPL only available on :debug profile compiled binary",
 :server-version "2021.03.06-17.05.35",
 :clj-kondo-version "2021.03.03",
 :log-path
 "/var/folders/c0/hfmx64kj4dx3y8dp772l_s6w0000gn/T/clojure-lsp.2501204020497163620.out"}

lsp-doctor:

Checking for Native JSON support: OK
Check emacs supports `read-process-output-max': OK
Check `read-process-output-max' default has been changed from 4k: OK
Byte compiled against Native JSON (recompile lsp-mode if failing when Native JSON available): OK
`gc-cons-threshold' increased?: OK
Using gccemacs with emacs lisp native compilation (https://akrl.sdf.org/gccemacs.html): OK

Emacs config: Latest spacemacs develop branch (syl20bnr/spacemacs@b7cbcb5) with these layers enabled and all packages up to date as of today:

   dotspacemacs-configuration-layers
   '(ruby
     octave
     csv
     elixir
     scheme
     javascript
     html
     org
     (clojure :variables
              clojure-enable-linters      'clj-kondo
              clojure-enable-clj-refactor t)
     ;; ----------------------------------------------------------------
     ;; Example of useful layers you may want to use right away.
     ;; Uncomment some layer names and press `SPC f e R' (Vim style) or
     ;; `M-m f e R' (Emacs style) to install them.
     ;; ----------------------------------------------------------------
     helm
     auto-completion
     ;; better-defaults
     emacs-lisp
     git
     osx
     markdown
     multiple-cursors
     ;; (shell :variables
     ;;        shell-default-height 30
     ;;        shell-default-position 'bottom)
     ;; spell-checking
     syntax-checking
     version-control
     yaml
     github
     mu4e
     ivy
     lsp
     )

Unfortunately emacs -q -l lsp-start-plain.el doesn't reproduce the issue. With my setup it reproduces 100% of the time.

[ericdallo]

Thank you for the detailed report, checking the stacktrace, it seems an issue when emacs is applying the highlighting font-face.
@yyoncho Does that happen in spaceemacs for you?

[yyoncho]

lsp--fontlock-with-mode("clojure\ncomment\n\n\n----\n*/Users/kommen/work/g..." lsp--render-markdown)

This is what is causing trouble, can you find us the real value that is used for that call? You can find it using lsp-workspace-show-log in combination with setting lsp-print-io or with using M-x trace-function RET lsp--fontlock-with-mode

[kommen]

[Trace - 05:13:18 PM] Received response 'textDocument/hover - (15)' in 33ms.
Result: {
  "contents": {
    "kind": "markdown",
    "value": "```clojure\nlambdaisland.glogi/logger\n```\n```clojure\n[n]\n[n level]\n```\n\n----\n```clojure\nGet a logger by name, and optionally set its level. Name can be a string\nkeyword, or symbol. The special keyword :glogi/root returns the root logger.\n```\n----\n*/Users/kommen/work/glogi/src/lambdaisland/glogi.cljs*"
  },
  "range": {
    "start": {
      "line": 40,
      "character": 2
    },
    "end": {
      "line": 40,
      "character": 8
    }
  }
}

@yyoncho is this what you're looking for?

[ericdallo]

looks correct to me the server response

[yyoncho]

Can you confirm that evaluating the following blocks:

(lsp--fontlock-with-mode 
 "```clojure\nlambdaisland.glogi/logger\n```\n```clojure\n[n]\n[n level]\n```\n\n----\n```clojure\nGet a logger by name, and optionally set its level. Name can be a string\nkeyword, or symbol. The special keyword :glogi/root returns the root logger.\n```\n----\n*/Users/kommen/work/glogi/src/lambdaisland/glogi.cljs*"
 'lsp--render-markdown)

[kommen]

No, calling that from ielm doesn't block.

[yyoncho]

Actually, AFAICS the string from the blocking call starts like that clojure\ncomment\n so the log does not match the callstack.

[kommen]

Sorry, my bad:

[Trace - 05:51:32 PM] Received response 'textDocument/hover - (31)' in 36ms.
Result: {
  "contents": {
    "kind": "markdown",
    "value": "```clojure\ncomment\n```\n\n----\n*/Users/kommen/work/glogi/src/lambdaisland/glogi.cljs*"
  },
  "range": {
    "start": {
      "line": 39,
      "character": 1
    },
    "end": {
      "line": 39,
      "character": 8
    }
  }
}

And yes, this hangs when running it in ielm:

(lsp--fontlock-with-mode 
 "```clojure\ncomment\n```\n\n----\n*/Users/kommen/work/glogi/src/lambdaisland/glogi.cljs*"
 'lsp--render-markdown)

[yyoncho]

can you do this:

(f-write-text 
 "```clojure\ncomment\n```\n\n----\n*/Users/kommen/work/glogi/src/lambdaisland/glogi.cljs*"
 'utf-8
 "/tmp/foo.md")

and then open /tmp/foo.md?

[kommen]

Opening that /tmp/foo.md also results in Emacs hanging with 100% cpu usage.

[yyoncho]

I suggest you try upgrading all packages, even you may try to delete your whole elpa to force redownloading the packages. I don't think the issue is lsp-mode related it seems like it is clojure-mode related, probably some odd setting(or interaction with cider?). If you are unable to fix it with these steps I guess it will be better to ask in clojure-mode repo, most likely they will be able to help right away.

[yyoncho]

make sure that you revert all clojure-mode related changes in your config.

[kommen]

Thank you folks, the issue appears whenever I have this enabled (setq clojure-toplevel-inside-comment-form t).

However, I don't get how the interaction between this and lsp can cause the problem I'm seeing? When disabling lsp I don't have any issues working in that clojure project.

[yyoncho]

However, I don't get how the interaction between this and lsp can cause the problem I'm seeing?

lsp-mode uses markdown mode to render the eldoc help which uses clojure-mode to render clojure sections. So lsp-mode is affected by this clojure-mode bug.

[yyoncho]

Marking the bug as closed since there is nothing to address on lsp-mode side.