CIDER super-slow after connecting to nREPL

[skrat]

Environment & Version information

OpenJDK 1.8 (same issue with Oracle JDK), Arch Linux

CIDER version information

CIDER 13. (same issue with 14. and 14-SNAPSHOT middleware)
org.clojure/tools.nrepl "0.2.12"

I'm running the nREPL through Figwheel with [com.cemerick/piggieback] middleware

Lein/Boot version

Lein 2.6.1

Emacs version

25.1.50

Operating system

Arch Linux

-----

After connecting CIDER to nREPL provided by Figwheel, then doing any file changes (Figwheel watches for those and re-compiles), or evaling something in Emacs, the CPU goes wee. It eventually completes the task, but after few of those I'm getting OOM:

cljs.user=> ERROR: Unhandled REPL handler exception processing message {:op complete, :session 639a52db-32df-47af-bcc9-7b7063075f72, :ns user, :symbol figwheel-sidecar.repl-ap, :context :same, :id 47}
java.lang.OutOfMemoryError: GC overhead limit exceeded
    at java.util.Arrays.copyOf(Arrays.java:3332)
    at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124)
    at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:541)
    at java.lang.StringBuffer.append(StringBuffer.java:350)
    at java.util.regex.Matcher.appendTail(Matcher.java:911)
    at java.util.regex.Matcher.replaceAll(Matcher.java:958)
    at java.lang.String.replace(String.java:2240)
    at cider.inlined_deps.compliment.v0v3v0.compliment.utils$list_files$iter__46350__46354$fn__46355.invoke(utils.clj:101)
    at clojure.lang.LazySeq.sval(LazySeq.java:40)
    at clojure.lang.LazySeq.seq(LazySeq.java:49)
    at clojure.lang.RT.seq(RT.java:521)
    at clojure.core$seq__4357.invokeStatic(core.clj:137)
    at clojure.core$concat$cat__4448$fn__4449.invoke(core.clj:715)
    at clojure.lang.LazySeq.sval(LazySeq.java:40)
    at clojure.lang.LazySeq.seq(LazySeq.java:49)
    at clojure.lang.RT.seq(RT.java:521)
    at clojure.core$seq__4357.invokeStatic(core.clj:137)
    at clojure.core$seq__4357.invoke(core.clj:137)
    at cider.inlined_deps.compliment.v0v3v0.compliment.utils$fn__46373$iter__46376__46380$fn__46381.invoke(utils.clj:114)
    at clojure.lang.LazySeq.sval(LazySeq.java:40)
    at clojure.lang.LazySeq.seq(LazySeq.java:49)
    at clojure.lang.Cons.next(Cons.java:39)
    at clojure.lang.RT.next(RT.java:688)
    at clojure.core$next__4341.invokeStatic(core.clj:64)
    at clojure.core.protocols$fn__6755.invokeStatic(protocols.clj:168)
    at clojure.core.protocols$fn__6755.invoke(protocols.clj:124)
    at clojure.core.protocols$fn__6710$G__6705__6719.invoke(protocols.clj:19)
    at clojure.core.protocols$seq_reduce.invokeStatic(protocols.clj:31)
    at clojure.core.protocols$fn__6738.invokeStatic(protocols.clj:75)
    at clojure.core.protocols$fn__6738.invoke(protocols.clj:75)
    at clojure.core.protocols$fn__6684$G__6679__6697.invoke(protocols.clj:13)
    at clojure.core$reduce.invokeStatic(core.clj:6545)

[skrat]

Bump

[bbatsov]

This seems like some bug in compliment, but without some exact steps to reproduce the problem we can't come up with a fix. //cc @alexander-yakushev

[skrat]

@bbatsov I believe this is a bug in cider-nrepl middleware. I have the same behavior when I connect vim-fireplace to a REPL with cider-nrepl middleware.

[skrat]

I used Netbeans profiler to see what's up, it show huge stack trace, the memory usage just keeps growing. This surely is a bug with something in the nREPL middleware.

https://gfycat.com/BoringOptimisticCockroach

[alexander-yakushev]

So, compliment.utils$list_files scans all the things on the classpath for classes. I'm not sure it is necessary on Clojurescript, and I don't know what exactly happens when that is done on Clojurescript.

Is this the complete stacktrace you've posted? I can't see any frames from CIDER itself. What's the entrypoint?

[bbatsov]

@bbatsov I believe this is a bug in cider-nrepl middleware. I have the same behavior when I connect vim-fireplace to a REPL with cider-nrepl middleware.

The middleware uses compliment internally.

[skrat]

I think I found the culprit. I have symlinks in my resources that point to directories, that in turn have symlinks to my project. It therefore keeps building up something, recursively, following the symlinks, that's why the memory, CPU and stack blows up. I believe compliment.utils$list_files or whatever it is that scans the classpath, should be robust enough to avoid this problem.

[alexander-yakushev]

I agree, the patch is certainly welcome.

[skrat]

@alexander-yakushev should I then checkout https://github.com/alexander-yakushev/compliment , try adding a failing test and fixing it in there?

[alexander-yakushev]

Yes, that would be perfect.