Skip to content

Commit

Permalink
Update README to explain test suites
Browse files Browse the repository at this point in the history
And also fix some errors.
  • Loading branch information
deadfoxygrandpa committed Apr 28, 2014
1 parent 3acd683 commit 26beed7
Showing 1 changed file with 47 additions and 27 deletions.
74 changes: 47 additions & 27 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,30 +40,41 @@ assertNotEqual a b -- Returns an AssertNotEqual assertion
assertionList [a, b, c] [d, e, f] -- Shorthand for [assertEqual a d, assertEqual b e, assertEqual c f]
```

## Running Tests
## Grouping Tests

Writing many tests as a flat list quickly becomes unwieldy. For easier maintenance you can group tests into logical
units called test suites. The following function will create a test suite from a suite name and a list of tests:
```haskell
suite : String -> [Test] -> Test
```
The type of a test suite is simply `Test`, allowing use of all the test runners with either a single test or a suite of tests. Test suites can also contain subsuites, of course.

Running a test produces a result. A result is either a pass or a failure, so it is represented as `Maybe String`. `Nothing` denotes the test passed, and `Just reason` indicates the test failed, and gives a `String` with a simple failure message.
The other benefit of grouping tests into suites is that the test runners described in the following sections will greatly simplify the output, showing only detailed information in suites that contain failed tests, making it easier to quickly spot the failures instead of being flooded with irrelevant data.

The most basic way to run a test is the `run` function, which has the type signature `Test -> Result`.
## Running Tests

Running a test produces a result. A result is either a pass, a failure, or a report containing detailed results from
the tests or subsuites contained in a suite. All results contain the name of the test or suite that was run, and failures additionally contain a failure message giving a hint as to why the test failed.

A list of tests can be run all at once, producing a `Report` Given a list of tests `myTests`, you can write: `report myTests`.
The most basic way to run a test is the `run` function, which has the type signature `Test -> Result`. A test suite can also be run all at once, again with the `run` function.

A `Report` is of type `{results : [Result], passes : [Result], failures : [Result]}`.
There is no built-in way to display results or reports, but there are functions for running tests and immediately seeing the results.
There is no built-in way to display results, but there are functions for running tests and immediately seeing the results.

## Displaying Results

In `ElmTest.Runner.Element` lives `runDisplay : [Test] -> Element`, which is an easy way to run your tests and report the results in-browser, as a standard Elm module. A full example could be:
In `ElmTest.Runner.Element` lives `runDisplay : Test -> Element`, which is an easy way to run your tests and report the results in-browser, as a standard Elm module. A full example could be:
```haskell
-- Example.elm
import String

import ElmTest.Test (test, Test)
import ElmTest.Test (test, Test, suite)
import ElmTest.Assertion (assert, assertEqual)
import ElmTest.Runner.Element (runDisplay)

tests : [Test]
tests = [ test "Addition" (assertEqual (3 + 7) 10)
tests : Test
tests = suite "A Test Suite"
[ test "Addition" (assertEqual (3 + 7) 10)
, test "String.left" (assertEqual "a" (String.left 1 "abcdefg"))
, test "This test should fail" (assert False)
]
Expand All @@ -73,7 +84,7 @@ main = runDisplay tests
```
Compile this with `elm --make Example.elm` and open the resulting `build/Example.html` file in your browser, and you'll see the results.

Another method is the `runDispay : [Test] -> (Bool, String)` function in `ElmTest.Runner.String`. This is almost the same, but it returns a `(Bool, String)` instead of an `Element`. The `Bool` is `True` if all tests passed, otherwise it is `False`. The `String` is a summary of the overall test results. Here's the same example as before, but modified for `ElmTest.Runner.String`:
Another method is the `runDispay : Test -> String` function in `ElmTest.Runner.String`. This is almost the same, but it returns a `String` instead of an `Element`. The `String` is a summary of the overall test results. Here's the same example as before, but modified for `ElmTest.Runner.String`:
```haskell
-- Example.elm
import String
Expand All @@ -82,20 +93,21 @@ import ElmTest.Test (test, Test)
import ElmTest.Assertion (assert, assertEqual)
import ElmTest.Runner.String (runDisplay)

tests : [Test]
tests = [ test "Addition" (assertEqual (3 + 7) 10)
tests : Test
tests = suite "A Test Suite"
[ test "Addition" (assertEqual (3 + 7) 10)
, test "String.left" (assertEqual "a" (String.left 1 "abcdefg"))
, test "This test should fail" (assert False)
]

results : String
results = snd <| runDisplay tests
results = runDisplay tests

main : Element
main = plainText results
```

There is one more version of this function. `runDisplay : [Test] -> IO ()` which lives in `ElmTest.Runner.Console`. This is designed to work with [Max New's Elm IO library](https://github.com/maxsnew/IO/). See the below section on **Testing from the Command Line** for details.
There is one more version of this function. `runDisplay : Test -> IO ()` which lives in `ElmTest.Runner.Console`. This is designed to work with [Max New's Elm IO library](https://github.com/maxsnew/IO/). See the below section on **Testing from the Command Line** for details.

## Demo

Expand Down Expand Up @@ -133,13 +145,20 @@ Generating JavaScript ... Done
Making exe

$ node ScriptExample.js
4 tests executed
3 tests passed
1 tests failed
8 == 1: FAILED. Expected: 8; got: 1
3 == 3: passed.
True: passed.
test head: passed.
5 suites run, containing 17 tests
3 suites and 16 tests passed
2 suites and 1 tests failed

Test Suite: A Test Suite: FAILED
Test Suite: Some tests: all tests passed
Test Suite: Some other tests: FAILED
8 == 1: FAILED. Expected: 8; got: 1
3 == 3: passed.
True: passed.
test head: passed.
Test Suite: More tests!: all tests passed
3 == 3: passed.
Test Suite: Even more!!: all tests passed
```
And that's it! Once `elm-io` is set up like this, you can run tests on the command line from any directory, in any Elm project. Just make sure to pull in the `evancz/automaton` and `jsdom` dependencies. Two current restrictions are that the module name of the file you wish to compile must be `Main`, and the following boilerplate must be added to this `Main` module:
```haskell
Expand All @@ -154,7 +173,7 @@ port responses : Signal (Maybe String)
You can examine `ScriptExample.elm` to see exactly how these are required. For a detailed log capturing the entire
setup process for the command line example, see: [ScriptExample.md](https://github.com/deadfoxygrandpa/Elm-Test/blob/master/ScriptExample.md).

Since `elm-io` version 0.1, it's not possible to ignore most of this boilerplate in favor of defining a
Since `elm-io` version 0.1, it's possible to ignore most of this boilerplate in favor of defining a
`console : IO ()` function, which effectively replaces the `main : Signal Element` function in normal
graphical Elm programs. Compile with `elm-io --default-ports Tests.elm tests.js`, replacing `Tests.elm`
with the filename of your Elm source file, and `tests.js` with the desired output script name to be
Expand All @@ -165,8 +184,9 @@ module Main where
import ElmTest.Runner.Console (runDisplay)
import open ElmTest.Test

tests : [Test]
tests = [ 5 `equals` 5
tests : Test
tests = suite "A Test Suite"
[ 5 `equals` 5
, test "Addition" (assertEqual (3 + 7) 10)
]

Expand All @@ -175,9 +195,9 @@ console = runDisplay tests
That's it! Make sure `evancz/automaton` and `jsdom` are installed in the project directory, then
compile like `elm-io --default-ports Tests.elm tests.js`. Run `node tests.js` and you will get:
```
2 tests executed
2 tests passed
0 tests failed
1 suites run, containing 2 tests
All tests passed
```
with exit code 0. If any tests fail, the process will exit with exit code 1.

Expand Down

0 comments on commit 26beed7

Please sign in to comment.