No support for shadow-cljs

[dhleong]

Expected behavior

The call works as normal.

Actual behavior

The op returns with an err:

java.lang.NullPointerException: null
 at refactor_nrepl.core$build_artifact_QMARK_.invokeStatic (core.clj:84)
    refactor_nrepl.core$build_artifact_QMARK_.invoke (core.clj:80)
    clojure.core$complement$fn__5669.invoke (core.clj:1441)
    clojure.core$every_pred$epn__8626$fn__8627.invoke (core.clj:7430)
    clojure.core$every_QMARK_.invokeStatic (core.clj:2681)
    clojure.core$every_pred$epn__8626.invoke (core.clj:7428)
    clojure.core$filter$fn__5893.invoke (core.clj:2821)
    clojure.lang.LazySeq.sval (LazySeq.java:42)
    clojure.lang.LazySeq.seq (LazySeq.java:58)
    clojure.lang.RT.seq (RT.java:535)
    clojure.core$seq__5402.invokeStatic (core.clj:137)
    clojure.core$concat$fn__5493.invoke (core.clj:725)
    clojure.lang.LazySeq.sval (LazySeq.java:42)
    clojure.lang.LazySeq.seq (LazySeq.java:51)
    clojure.lang.RT.seq (RT.java:535)
    clojure.core$seq__5402.invokeStatic (core.clj:137)
    clojure.core$concat$cat__5495$fn__5496.invoke (core.clj:734)
    clojure.lang.LazySeq.sval (LazySeq.java:42)
    clojure.lang.LazySeq.seq (LazySeq.java:51)
    clojure.lang.RT.seq (RT.java:535)
    clojure.lang.RT.nthFrom (RT.java:981)
    clojure.lang.RT.nth (RT.java:940)
    clojure.core$distinct$step__6413$fn__6414$fn__6416.invoke (core.clj:5048)
    clojure.core$distinct$step__6413$fn__6414.invoke (core.clj:5048)
    clojure.lang.LazySeq.sval (LazySeq.java:42)
    clojure.lang.LazySeq.seq (LazySeq.java:51)
    clojure.lang.RT.seq (RT.java:535)
    clojure.core$seq__5402.invokeStatic (core.clj:137)
    clojure.core$map$fn__5866.invoke (core.clj:2746)
    clojure.lang.LazySeq.sval (LazySeq.java:42)
    clojure.lang.LazySeq.seq (LazySeq.java:51)
    clojure.lang.RT.seq (RT.java:535)
    clojure.core$seq__5402.invokeStatic (core.clj:137)
    clojure.core$map$fn__5866.invoke (core.clj:2746)
    clojure.lang.LazySeq.sval (LazySeq.java:42)
    clojure.lang.LazySeq.seq (LazySeq.java:51)
    clojure.lang.RT.seq (RT.java:535)
    clojure.core$seq__5402.invokeStatic (core.clj:137)
    clojure.core$apply.invokeStatic (core.clj:660)
    clojure.core$mapcat.invokeStatic (core.clj:2783)
    clojure.core$mapcat.doInvoke (core.clj:2783)
    clojure.lang.RestFn.invoke (RestFn.java:423)
    refactor_nrepl.ns.libspecs$aliases_by_frequencies.invokeStatic (libspecs.clj:18)
    refactor_nrepl.ns.libspecs$aliases_by_frequencies.invoke (libspecs.clj:16)
    refactor_nrepl.ns.libspecs$namespace_aliases.invokeStatic (libspecs.clj:50)
    refactor_nrepl.ns.libspecs$namespace_aliases.invoke (libspecs.clj:44)
    refactor_nrepl.middleware$namespace_aliases_reply.invokeStatic (middleware.clj:182)
    refactor_nrepl.middleware$namespace_aliases_reply.invoke (middleware.clj:180)
    refactor_nrepl.middleware$wrap_refactor$fn__11867.invoke (middleware.clj:214)
    nrepl.middleware$wrap_conj_descriptor$fn__6083.invoke (middleware.clj:16)
    cider.nrepl$wrap_profile$fn__12088.invoke (nrepl.clj:300)
    nrepl.middleware$wrap_conj_descriptor$fn__6083.invoke (middleware.clj:16)
    cider.nrepl$wrap_spec$fn__12112.invoke (nrepl.clj:375)
    nrepl.middleware$wrap_conj_descriptor$fn__6083.invoke (middleware.clj:16)
    cider.nrepl$wrap_version$fn__12160.invoke (nrepl.clj:457)
    nrepl.middleware$wrap_conj_descriptor$fn__6083.invoke (middleware.clj:16)
    cider.nrepl$wrap_format$fn__12038.invoke (nrepl.clj:167)
    nrepl.middleware$wrap_conj_descriptor$fn__6083.invoke (middleware.clj:16)
    cider.nrepl$wrap_clojuredocs$fn__12176.invoke (nrepl.clj:482)
    nrepl.middleware$wrap_conj_descriptor$fn__6083.invoke (middleware.clj:16)
    cider.nrepl$wrap_xref$fn__12168.invoke (nrepl.clj:467)
    nrepl.middleware$wrap_conj_descriptor$fn__6083.invoke (middleware.clj:16)
    nrepl.middleware.interruptible_eval$interruptible_eval$fn__6394.invoke (interruptible_eval.clj:144)
    nrepl.middleware$wrap_conj_descriptor$fn__6083.invoke (middleware.clj:16)
    cider.nrepl$wrap_info$fn__12046.invoke (nrepl.clj:181)
    nrepl.middleware$wrap_conj_descriptor$fn__6083.invoke (middleware.clj:16)
    cider.nrepl$wrap_slurp$fn__11988.invoke (nrepl.clj:95)
    nrepl.middleware$wrap_conj_descriptor$fn__6083.invoke (middleware.clj:16)
    nrepl.middleware.session$add_stdin$fn__6670.invoke (session.clj:326)
    nrepl.middleware$wrap_conj_descriptor$fn__6083.invoke (middleware.clj:16)
    cider.nrepl$wrap_debug$fn__12022.invoke (nrepl.clj:137)
    nrepl.middleware$wrap_conj_descriptor$fn__6083.invoke (middleware.clj:16)
    nrepl.middleware.load_file$wrap_load_file$fn__6559.invoke (load_file.clj:81)
    nrepl.middleware$wrap_conj_descriptor$fn__6083.invoke (middleware.clj:16)
    shadow.cljs.devtools.server.nrepl$shadow_init$fn__7152.invoke (nrepl.clj:27)
    nrepl.middleware$wrap_conj_descriptor$fn__6083.invoke (middleware.clj:16)
    cider.nrepl$wrap_content_type$fn__11980.invoke (nrepl.clj:83)
    nrepl.middleware$wrap_conj_descriptor$fn__6083.invoke (middleware.clj:16)
    cider.nrepl$wrap_apropos$fn__11996.invoke (nrepl.clj:103)
    nrepl.middleware$wrap_conj_descriptor$fn__6083.invoke (middleware.clj:16)
    cider.nrepl$wrap_stacktrace$fn__12120.invoke (nrepl.clj:391)
    nrepl.middleware$wrap_conj_descriptor$fn__6083.invoke (middleware.clj:16)
    cider.nrepl$wrap_test$fn__12128.invoke (nrepl.clj:401)
    nrepl.middleware$wrap_conj_descriptor$fn__6083.invoke (middleware.clj:16)
    cider.nrepl$wrap_ns$fn__12072.invoke (nrepl.clj:257)
    nrepl.middleware$wrap_conj_descriptor$fn__6083.invoke (middleware.clj:16)
    cider.nrepl$wrap_trace$fn__12136.invoke (nrepl.clj:420)
    nrepl.middleware$wrap_conj_descriptor$fn__6083.invoke (middleware.clj:16)
    cider.nrepl$wrap_refresh$fn__12096.invoke (nrepl.clj:337)
    nrepl.middleware$wrap_conj_descriptor$fn__6083.invoke (middleware.clj:16)
    cider.nrepl$wrap_classpath$fn__12004.invoke (nrepl.clj:111)
    nrepl.middleware$wrap_conj_descriptor$fn__6083.invoke (middleware.clj:16)
    cider.nrepl$wrap_macroexpand$fn__12064.invoke (nrepl.clj:244)
    nrepl.middleware$wrap_conj_descriptor$fn__6083.invoke (middleware.clj:16)
    cider.nrepl$wrap_resource$fn__12104.invoke (nrepl.clj:363)
    nrepl.middleware$wrap_conj_descriptor$fn__6083.invoke (middleware.clj:16)
    cider.nrepl$wrap_undef$fn__12152.invoke (nrepl.clj:449)
    nrepl.middleware$wrap_conj_descriptor$fn__6083.invoke (middleware.clj:16)
    cider.nrepl$wrap_enlighten$fn__12030.invoke (nrepl.clj:163)
    nrepl.middleware$wrap_conj_descriptor$fn__6083.invoke (middleware.clj:16)
    cider.nrepl$wrap_complete$fn__12012.invoke (nrepl.clj:117)
    nrepl.middleware$wrap_conj_descriptor$fn__6083.invoke (middleware.clj:16)
    cider.piggieback$wrap_cljs_repl$fn__6491.invoke (piggieback.clj:310)
    nrepl.middleware$wrap_conj_descriptor$fn__6083.invoke (middleware.clj:16)
    cider.nrepl$wrap_out$fn__12080.invoke (nrepl.clj:291)
    nrepl.middleware$wrap_conj_descriptor$fn__6083.invoke (middleware.clj:16)
    cider.nrepl$wrap_inspect$fn__12056.invoke (nrepl.clj:200)
    nrepl.middleware$wrap_conj_descriptor$fn__6083.invoke (middleware.clj:16)
    shadow.cljs.devtools.server.nrepl_impl$handle.invokeStatic (nrepl_impl.clj:293)
    shadow.cljs.devtools.server.nrepl_impl$handle.invoke (nrepl_impl.clj:278)
    shadow.cljs.devtools.server.nrepl$middleware$fn__7161.invoke (nrepl.clj:39)
    nrepl.middleware$wrap_conj_descriptor$fn__6083.invoke (middleware.clj:16)
    nrepl.middleware.caught$wrap_caught$fn__6340.invoke (caught.clj:97)
    nrepl.middleware$wrap_conj_descriptor$fn__6083.invoke (middleware.clj:16)
    nrepl.middleware.print$wrap_print$fn__6307.invoke (print.clj:234)
    nrepl.middleware$wrap_conj_descriptor$fn__6083.invoke (middleware.clj:16)
    cider.nrepl$wrap_tracker$fn__12144.invoke (nrepl.clj:438)
    nrepl.middleware$wrap_conj_descriptor$fn__6083.invoke (middleware.clj:16)
    nrepl.middleware.session$session$fn__6655.invoke (session.clj:272)
    nrepl.middleware$wrap_conj_descriptor$fn__6083.invoke (middleware.clj:16)
    shadow.cljs.devtools.server.nrepl$start$fn__7192.invoke (nrepl.clj:133)
    nrepl.server$handle_STAR_.invokeStatic (server.clj:18)
    nrepl.server$handle_STAR_.invoke (server.clj:15)
    nrepl.server$handle$fn__6692.invoke (server.clj:27)
    clojure.core$binding_conveyor_fn$fn__5754.invoke (core.clj:2030)
    clojure.lang.AFn.call (AFn.java:18)
    java.util.concurrent.FutureTask.run (FutureTask.java:266)
    java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1149)
    java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:624)
    java.lang.Thread.run (Thread.java:748)

Steps to reproduce the problem

1. Add refactor-nrepl as a global shadow dependency, then enabled the middleware in the project:

 :nrepl {:port 9888
         :middleware [refactor-nrepl.middleware/wrap-refactor]}

2. Connect to nrepl using vim-fireplace, and attach to the main cljs build
3. Call namespace-aliases op using Fireplace connection:

echo fireplace#message({'op':'namespace-aliases', 'serialization-format': 'bencode'}, v:t_dict)

Environment & Version information

clj-refactor.el and refactor-nrepl version information

refactor-nrepl: 2.4.0

CIDER version information

Don't have this string, but I'm using cider-nrepl 0.22.4

Leiningen or Boot version

N/A

Vim version

MacVim 8.1.1722 (157)

Operating system

macOS 10.15 (19A583)

[expez]

There's no support for shadow-cljs projects, so the failure here is expected.

A better failure mode would be nice, but actual support would be even better.

Is this anything you'd be interested in working on @dhleong?

[dhleong]

Aha, that would explain it.

I'm always happy to help where I can but I might need a nudge in the right direction with this one. I was having trouble running tests at the REPL before, and nothing seemed to run at all until I disabled the cider-nrepl and refactor-nrepl plugins in my leiningen profile, and my current tooling sort of expects cider-nrepl. I can give it another shot later to see if doing inline-deps with those plugins disabled lets me get things working with them enabled, or something, but then my question would be: how do I access the state from a real shadow-cljs env from a refactor-nrepl REPL to test things?

[expez]

how do I access the state from a real shadow-cljs env from a refactor-nrepl REPL to test things?

If you can get a REPL open in a shadow-cljs project (even if none of the ops work) you can start investigating.

Here's the failing snippet:

(defn namespace-aliases
  "Return a map of file type to a map of aliases to namespaces
  {:clj {util com.acme.util str clojure.string
   :cljs {gstr goog.str}}}"
  []
  {:clj (->> (core/find-in-project (some-fn core/clj-file? core/cljc-file?))
             (map (partial get-libspec-from-file-with-caching :clj))
             aliases-by-frequencies)
   :cljs (->> (core/find-in-project (some-fn core/cljs-file? core/cljc-file?))
              (map (partial get-libspec-from-file-with-caching :cljs))
              aliases-by-frequencies)})

In particular, this call is going to fail: (core/find-in-project (some-fn core/cljs-file? core/cljc-file?)) because refactor-nrepl doesn't really understand how a shadow-cljs project is structured.

At that point you can, in your REPL:

user> (refactor-nrepl.core/find-in-project dentity)
;;=> BAM!

The next step would probably be to do (in-ns 'refactor-nrepl.core) and to redefine the build-artifact? predicate with a println or something so you can find out exactly which file it chokes on and why.

[dhleong]

Aha, the problem is that project-root returns nil, so the NPE in build-artifact? is this:

        target-path (-> path-or-file project-root .getCanonicalPath  ; <-- NPE
                        normalize-to-unix-path
                        (str "/target"))

It looks like project-root does not fall back to user.dir; it just keeps going up to / at which point it gives up and returns nil. Is that intended?

The reason it gets that far is because it doesn't know about shadow-cljs.edn as a root file. I think the easiest fix regardless of the answer to my above question is to just add shadow-cljs.edn to names-at-root. There seems to be another issue in aliases-by-frequency with using a Symbol as a String; once I figure that out I can put up a PR.

Thanks for the pointers, it was really helpful! I've been sidetracked by other projects so only got to this today.

[aiba]

FWIW, I've been using shadow-cljs with refactor-nrepl, and as a hack I just touch deps.edn alongside the shadow-cljs.edn file. Without this, I was getting, "WARNING: No Clojure project was detected. The refactor-nrepl middleware was not enabled."

So yeah, I support adding shadow-cljs.edn to names-at-root.

I also wonder, do we need to update cljr--project-dir?

[expez]

I also wonder, do we need to update cljr--project-dir?

I can't remember off the top of my head which functionality that predicate affects, but it might be a good idea!