Accessing a fileset's resource files from tests

[kommen]

I'm trying to set up a test system where a test case is automatically generated based on test data present in the file system including the expected output. However, I can't figure out how to access those files from within the test runner.

A simplified example:

(deftask backend-test []
  (merge-env! :source-paths ["clj/backend/src" "clj/backend/test"]
              :resource-paths ["clj/backend/fixtures"])
  (test-cljs :js-env :node :exit? true))

Directory structure of clj/backend:

|- src
|- test
|- fixtures
  |- test-data
      |- sample-input-1.edn
      |- expected-output-1.html
      |- sample-input-2.edn
      |- expected-output-2.html

Is it possible to somehow make the contents of clj/backend/fixtures accessible from the test runner so I can generate test cases based on the contents of test-data?

I saw that e.g node_modules is manually moved into the tmp dir: https://github.com/crisptrutski/boot-cljs-test/blob/master/src/crisptrutski/boot_cljs_test.clj#L136

So I guess I would somehow have to do that with the files I want to use as well?

Thank you in advance!

[crisptrutski]

The principle of least surprise would probably be for boot-cljs-test to copy the resources to the runtime path also, but that cost may be prohibitive for some users. Cannot recall why the files are copied and not simply hard linked - perhaps that would keep the idea more feasible.

Alternately could parameter-creep it and add resources option to the task.

Happy to explore either - and in the short term you could "explode" the "porcelain" task into the lower level calls and insert the extra directory copy between tasks. Let me know if this is unclear and I can provide a sample.

[kommen]

Another alternative I see is to add another task, like maybe cljs-test-resources which adds meta data to files in the fileset with add-meta and the run-cljs-tests finds those marked files to copy them to the runtime path. But I don't have enough experience with boot yet to know if this is possible and/or idiomatic.

As for the short term exploding I tried to instead of this:

(cljs-test/test-cljs
   :js-env :node
   :exit?  true})

This:

  (comp
   (cljs-test/fs-snapshot)
   (cljs-test/prep-cljs-tests)
   (cljs :ids #{"cljs_test/generated_test_suite"})
   (cljs-test/run-cljs-tests :js-env :node :cljs-opts {:target :nodejs :hashbang false})
   (cljs-test/fs-restore)))

But it still looks I'm getting something wrong as the exploded version can't find the the needed node_modules yet. Anything obvious I'm missing?

[crisptrutski]

Have misspoken - the change needs to happen inside the run-cljs-tests task. Monkey-patching add-node-modules! is an option.. Will take a rough look at just cutting a SNAPSHOT that just handles resources quickly.

[crisptrutski]

Published two 0.3.1-hardlinks-SNAPSHOT releases (confusingly the newer one uses symlinks), let me know if these resolve your issue.

[kommen]

Using 0.3.1-hardlinks-SNAPSHOT I get an error when it tries to create the link:

 $ boot backend-test
Adding: ([doo "0.1.7"]) to :dependencies
npm WARN slate@0.17.4 requires a peer of react-dom@^0.14.0 || ^15.0.0 but none was installed.
Compiling ClojureScript...
• cljs_test/generated_test_suite.js
WARNING: bounded-count already refers to: #'clojure.core/bounded-count in namespace: clojure.core.async, being replaced by: #'clojure.core.async/bounded-count

;; ======================================================================
;; Testing with Node:

                              java.lang.Thread.run                       Thread.java:  745
java.util.concurrent.ThreadPoolExecutor$Worker.run           ThreadPoolExecutor.java:  617
 java.util.concurrent.ThreadPoolExecutor.runWorker           ThreadPoolExecutor.java: 1142
               java.util.concurrent.FutureTask.run                   FutureTask.java:  266
                                               ...                                        
               clojure.core/binding-conveyor-fn/fn                          core.clj: 2020
                                 boot.core/boot/fn                          core.clj: 1029
                               boot.core/run-tasks                          core.clj: 1019
                       boot.user/eval1204/fn/fn/fn  boot.user4709939754164070111.clj:   19 (repeats 2 times)
      crisptrutski.boot-cljs-test/eval706/fn/fn/fn                boot_cljs_test.clj:   97
                 adzerk.boot-cljs/eval209/fn/fn/fn                     boot_cljs.clj:  137
                 adzerk.boot-cljs/eval266/fn/fn/fn                     boot_cljs.clj:  217
      crisptrutski.boot-cljs-test/eval776/fn/fn/fn                boot_cljs_test.clj:  167
            crisptrutski.boot-cljs-test/run-tests!                boot_cljs_test.clj:  137
     crisptrutski.boot-cljs-test/add-node-modules!                boot_cljs_test.clj:  109
                               boot.file/hard-link                          file.clj:  165
                    java.nio.file.Files.createLink                        Files.java: 1086
      sun.nio.fs.UnixFileSystemProvider.createLink       UnixFileSystemProvider.java:  476
     sun.nio.fs.UnixException.rethrowAsIOException                UnixException.java:  102
   sun.nio.fs.UnixException.translateToIOException                UnixException.java:   86
java.nio.file.NoSuchFileException: /Users/kommen/.boot/cache/tmp/Users/kommen/work/editor/apps/journal/1ott/-ueasqk/cljs_test/clj/renderer/fixtures -> clj/renderer/fixtures
         file: "/Users/kommen/.boot/cache/tmp/Users/kommen/work/editor/apps/journal/1ott/-ueasqk/cljs_test/clj/renderer/fixtures"
    otherFile: "clj/renderer/fixtures"
       clojure.lang.ExceptionInfo: /Users/kommen/.boot/cache/tmp/Users/kommen/work/editor/apps/journal/1ott/-ueasqk/cljs_test/clj/renderer/fixtures -> clj/renderer/fixtures
    file: "/var/folders/c0/hfmx64kj4dx3y8dp772l_s6w0000gn/T/boot.user4709939754164070111.clj"
    line: 49

[kommen]

The error seems to happen when :resource-paths points to a subdirectory:

This errors:

:resource-paths ["clj/backend/fixtures"]

Moving fixtures up to the project root doesn't error:

:resource-paths ["fixtures"]

[crisptrutski]

Thanks for isolating the problem, the linking code is a bit naive about parent directories existing 🙂

[crisptrutski]

Published another SNAPSHOT including 96a46cb

[kommen]

@crisptrutski doesn't work as expected for me.

I've adapted link-resources! to work for me like this:

(defn link-resources! [dir]
  (when (.exists (io/file "node_modules"))
    (file/sym-link (.getAbsoluteFile (io/file "node_modules")) (io/file dir "node_modules")))
  (doseq [path (boot/get-env :resource-paths)
          :let [f (io/file path)]
          :when (.exists f)]
    (doseq [r (.listFiles f)]
      (println "sym link" (.getAbsolutePath r) "->" (.getAbsolutePath (io/file dir (.getName r))))
      (file/sym-link (.getAbsoluteFile r) (io/file dir (.getName r))))))

This:

• ensures absolute paths are symlinked. the relative symlinks are pointing into nowhere
• links into the runtime directory all the contents of all :resource-paths, so the directory structure looks like it would in boot's target dir

However, I'm running into an issue where the symlinked files are deleted after they are linked once and the tests are rerun. I guess hard linking would solve that. Will try hard links later.

[crisptrutski]

Let me know how it goes with hard links. Perhaps the most flexible approach here would be lifecycle hooks where tasks can be attached?

[kommen]

So I don't understand why run-tests! has to manage the resources itself:
In add-suite-ns!'s commit! the fileset is written to the directory already. And the fileset includes my fixture resources, they are just not written.

If I instead of using boot env's :resource-paths I add the fixtures with add-resource they are written to the runtime dir and thus available there without the need to copy/link anything.

So replacing

(deftask backend-test []
  (merge-env! :source-paths ["clj/backend/src" "clj/backend/test"]
              :resource-paths ["clj/backend/fixtures"])
  (test-cljs :js-env :node :exit? true))

with

(deftask backend-test []
 (merge-env! :source-paths ["clj/backend/src" "clj/backend/test"])
 (comp
   (with-pre-wrap fileset
     (-> fileset
         (add-resource (java.io.File. "clj/backend/fixtures"))))     
   (test-cljs :js-env :node :exit? true))

actually fixes my issue with the currently published 0.3.0 of boot-cljs-test.

I don't understand why though...