Test.Tasty

The name of a test or a group of tests.

The main data structure defining a test suite.

It consists of individual test cases and properties, organized in named groups which form a tree-like hierarchy.

There is no generic way to create a test case. Instead, every test provider (tasty-hunit, tasty-smallcheck etc.) provides a function to turn a test case into a TestTree.

Groups can be created using testGroup.

Create a named group of test cases or other groups. Tests are executed in parallel. For sequential execution, see sequentialTestGroup.

Create a named group of test cases or other groups. Tests are executed in order. For parallel execution, see testGroup.

Parse the command line arguments and run the tests.

When the tests finish, this function calls exitWith with the exit code that indicates whether any tests have failed. Most external systems (stack, cabal, travis-ci, jenkins etc.) rely on the exit code to detect whether the tests pass. If you want to do something else after defaultMain returns, you need to catch the exception and then re-throw it. Example:

import Test.Tasty
import Test.Tasty.HUnit
import System.Exit
import Control.Exception

test = testCase "Test 1" (2 @?= 3)

main = defaultMain test
  `catch` (\e -> do
    if e == ExitSuccess
      then putStrLn "Yea"
      else putStrLn "Nay"
    throwIO e)

Parse the command line arguments and run the tests using the provided ingredient list.

When the tests finish, this function calls exitWith with the exit code that indicates whether any tests have failed. See defaultMain for details.

List of the default ingredients. This is what defaultMain uses.

At the moment it consists of listingTests and consoleTestReporter.

This ingredient doesn't do anything apart from registering additional options.

The option values can be accessed using askOption.

Locally adjust the option value for the given test subtree.

Locally set the option value for the given test subtree.

Customize the test tree based on the run-time options.

Acquire the resource to run this test (sub)tree and release it afterwards.

These are the two ways in which one test may depend on the others.

This is the same distinction as the hard vs soft dependencies in TestNG.

The after combinator declares dependencies between tests.

If a TestTree is wrapped in after, the tests in this tree will not run until certain other tests («dependencies») have finished. These dependencies are specified using an AWK pattern (see the «Patterns» section in the README).

Moreover, if the DependencyType argument is set to AllSucceed and at least one dependency has failed, this test tree will not run at all.

Tasty does not check that the pattern matches any tests (let alone the correct set of tests), so it is on you to supply the right pattern.

Examples

The following test will be executed only after all tests that contain Foo anywhere in their path finish.

after AllFinish "Foo" $
   

testCase "A test that depends on Foo.Bar" $ ...

Note, however, that our test also happens to contain Foo as part of its name, so it also matches the pattern and becomes a dependency of itself. This will result in a Test.Tasty.DependencyLoop exception. To avoid this, either change the test name so that it doesn't mention Foo or make the pattern more specific.

You can use AWK patterns, for instance, to specify the full path to the dependency.

after AllFinish "$0 == \"Tests.Foo.Bar\"" $
   

testCase "A test that depends on Foo.Bar" $ ...

Or only specify the dependency's own name, ignoring the group names:

after AllFinish "$NF == \"Bar\"" $
   

testCase "A test that depends on Foo.Bar" $ ...

Like after, but accepts the pattern as a syntax tree instead of a string. Useful for generating a test tree programmatically.

Examples

Only match on the test's own name, ignoring the group names:

after_ AllFinish (EQ (Field NF) (StringLit "Bar")) $
   

testCase "A test that depends on Foo.Bar" $ ...