Skip to content

Commit

Permalink
Update cabal-testsuite README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
9999years committed Nov 8, 2024
1 parent 60fbff5 commit e842e5d
Showing 1 changed file with 36 additions and 17 deletions.
53 changes: 36 additions & 17 deletions cabal-testsuite/README.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,21 @@
cabal-testsuite is a suite of integration tests for Cabal-based
frameworks.

How to run
----------
# How to run

1. Build `cabal-testsuite` (`cabal build cabal-testsuite:cabal-tests`)
2. Run the `cabal-tests` executable. It will scan for all tests
in your current directory and subdirectories and run them.
To run a specific set of tests, use `cabal-tests --with-cabal=CABALBIN PATH ...`.
(e.g. `cabal run cabal-testsuite:cabal-tests -- --with-cabal=cabal cabal-testsuite/PackageTests/TestOptions/setup.test.hs`)
You can control parallelism using the `-j` flag.

There are a few useful flags:

* To run a specific set of tests, pass the path to a `.test.hs` file to run or
use the `-p`/`--pattern` flag to filter tests.

See the ["Selecting tests"](#selecting-tests) section below for more details.

* `-j INT` controls the number of threads used for running tests.

* `--with-cabal PATH` can be used to specify the path of a
`cabal-install` executable. IF YOU DO NOT SPECIFY THIS FLAG,
CABAL INSTALL TESTS WILL NOT RUN.
Expand All @@ -28,6 +31,28 @@ There are a few useful flags:
* `--keep-tmp-files` can be used to keep the temporary directories that tests
are run in.

## Selecting tests

To run a specific set of tests, use `cabal-tests --with-cabal=CABALBIN PATH ...`, e.g.:

```
cabal run cabal-testsuite:cabal-tests -- \
--with-cabal=cabal \
cabal-testsuite/PackageTests/TestOptions/setup.test.hs
```

Alternatively, use `-p`/`--pattern` to select tests dynamically:

```
cabal run cabal-testsuite:cabal-tests -- \
--with-cabal=cabal \
--pattern "/TestOptions/"
```

See [the documentation for Tasty pattern
syntax](https://hackage.haskell.org/package/tasty#patterns) for more
information.

## Which Cabal library version do cabal-install tests use?

By default the `cabal-install` tests will use the `Cabal` library which comes with
Expand Down Expand Up @@ -74,8 +99,7 @@ components have broken doctests
our CI currently checks that `Cabal-syntax` and `Cabal` doctests pass via
`make doctest-install && make doctest` (you can use this `make`-based workflow too).

How to write
------------
# How to write

If you learn better by example, just look at the tests that live
in `cabal-testsuite/PackageTests`; if you `git log -p`, you can
Expand Down Expand Up @@ -155,8 +179,7 @@ allow multiple tests to be defined in one file but run in parallel;
at the moment, these just indicate long running tests that should
be run early (to avoid straggling).

Frequently asked questions
--------------------------
# Frequently asked questions

For all of these answers, to see examples of the functions in
question, grep the test suite.
Expand Down Expand Up @@ -223,8 +246,7 @@ long before editing a file, in order for file system timestamp
resolution to pick it up. Use `withDelay` and `delay` prior to
making a modification.

Hermetic tests
--------------
# Hermetic tests

Tests are run in a fresh temporary system directory. This attempts to isolate the
tests from anything specific to do with your directory structure. In particular
Expand All @@ -235,8 +257,7 @@ tests from anything specific to do with your directory structure. In particular
* You must `git add` all files which are relevant to the test, otherwise
they will not be copied.

Design notes
------------
# Design notes

This is the second rewrite of the integration testing framework. The
primary goal was to use Haskell as the test language (letting us take
Expand Down Expand Up @@ -296,8 +317,7 @@ figure out how to get out the threading setting, and then spawn
that many GHCi servers to service the running threads. Improvements
welcome.

Expect tests
------------
# Expect tests

An expect test (aka _golden test_)
is a test where we read out the output of the test
Expand Down Expand Up @@ -366,8 +386,7 @@ Some other notes:
on the output for the string you're looking for. Try to be
deterministic, but sometimes it's not (easily) possible.

Non-goals
---------
# Non-goals

Here are some things we do not currently plan on supporting:

Expand Down

0 comments on commit e842e5d

Please sign in to comment.