Better compilation error messages?

[lydell]

(Sorry if this has been discussed before. I searched, but couldn't find anything.)

I'm new to elm-test. I tried it out for the first time today. First off, I must say that it was a very pleasant experience!

I use the Atom text editor, and the Elm plugins for it are really good. Specifically, all errors are pointed out for me directly in the editor, so when I finally compile my Elm code it never fails.

Except today. Because I compiled code that I hadn't written: elm-verify-examples had. And I didn't compile the code either: elm-test did.

Have a look at this:

$ elm-verify-examples && elm-test
Running in generate mode..
Compilation failed while attempting to build /home/lydell/src/elm-minesweeper/tests/Doc/Matrix/CustomSpec.elm /home/lydell/src/elm-minesweeper/tests/TestGrid.elm

Hmm. So something caused the the compilation to fail. But why? Because of misconfiguration? Something that I had forgot to install? A bug in elm-test? Or – of course – it could be invalid code in my tests somewhere.

I soon figured out that I could look for error by compiling manually.

$ cd tests
$ elm-make Doc/Matrix/CustomSpec.elm 
-- NAMING ERROR -------------------------------------- Doc/Matrix/CustomSpec.elm

Cannot find variable `Char.isUpper`.

20|                             Maybe.map (all Char.isUpper) (fromList [['A', 'B']])
                                               ^^^^^^^^^^^^
No module called `Char` has been imported.

Aha! Turns out I had missed adding import Char in a comment so that elm-verify-examples could pick it up and add that import to its generated code.

Then my tests passed just fine! By the way, here's the repo where I added elm-test and elm-verify-examples: https://github.com/lydell/elm-minesweeper

So the question is: Can we make the compilation error message nicer?

• Showing elm-make errors (like above) seems like a good idea.
• Or at least format the message on several lines and suggest how to proceed.
• Perhaps elm-verify-examples could verify that its generated code compiles as well.

What do you think?

[mgold]

Thanks for the issue! elm-verify-examples is a newly reworked project (like, last few days new) and I'm very excited for it. There is a much larger conversation about whether we want the elm-test node runner to integrate with it more directly (some kind of pre-run hook system?)

That said, I can't reproduce this bad error message. I add this example to my file:

List.filter Char.isUpper ['a', 'A'] --> ['A']

and I see a reasonable compiler error when I run elm-verify-examples && elm-test.

[lydell]

@mgold Perhaps this repo can help you reproduce? https://github.com/lydell/elm-test-compilation-error-demo

[mgold]

Running npm test reproduces. Running elm-verify-examples && elm-test directly does not reproduce. npm issue?

[lydell]

Strange. For me, there's no difference running elm-verify-examples && elm-test (after running npm i -g elm-test elm-verify-examples elm).

[bobwhitelock]

I recently ran in to the same error message, similarly due to a compilation error in my tests. My particular error was unrelated to the elm-verify-examples package however, I just has some Elm code in my tests directory which failed to compile.

I may be missing something, but it looks like the underlying issue here is to do with the node-test-runner giving an uninformative error message when compiling test code, rather than the particular reason there is bad code in the tests directory.

I went looking in node-test-runner, and it looks like a fix to give a more informative error message was made in rtfeldman/node-test-runner#161. After installing this it does appear to fix this for me by showing the elm-make error when this occurs. This hasn't made it into a release yet, but you should be able to install node-test-runner directly from Github until it does.

[lydell]

@bobwhitelock Thanks!