Hedgehog

A property test, along with some configurable limits like how many times to run the test.

The property monad transformer allows both the generation of test inputs and the assertion of expectations.

A named collection of property tests.

The name of a property.

Should be constructed using OverloadedStrings:

  "apples" :: PropertyName

The name of a group of properties.

Should be constructed using OverloadedStrings:

  "fruit" :: GroupName

Creates a property with the default configuration.

Lift a test in to a property.

Because both TestT and PropertyT have MonadTest instances, this function is not often required. It can however be useful for writing functions directly in TestT and thus gaining a MonadTransControl instance at the expense of not being able to generate additional inputs using forAll.

An example where this is useful is parallel state machine testing, as executeParallel requires MonadBaseControl IO in order to be able to spawn threads in MonadTest.

Generates a random input for the test by running the provided generator.

Generates a random input for the test by running the provided generator.

This is a the same as forAll but allows the user to provide a custom rendering function. This is useful for values which don't have a Show instance.

Discards the current test entirely.

Check a property.

Check a property using a specific size and seed.

Discover all the properties in a module.

Functions starting with prop_ are assumed to be properties.

Check a group of properties in parallel.

Warning: although this check function runs tests faster than checkSequential, it should be noted that it may cause problems with properties that are not self-contained. For example, if you have a group of tests which all use the same database table, you may find that they interfere with each other when being run in parallel.

Using Template Haskell for property discovery:

tests :: IO Bool
tests =
  checkParallel $$(discover)

With manually specified properties:

tests :: IO Bool
tests =
  checkParallel $ Group "Test.Example" [
      ("prop_reverse", prop_reverse)
    ]

Check a group of properties sequentially.

Using Template Haskell for property discovery:

tests :: IO Bool
tests =
  checkSequential $$(discover)

With manually specified properties:

tests :: IO Bool
tests =
  checkSequential $ Group "Test.Example" [
      ("prop_reverse", prop_reverse)
    ]

The acceptable occurrence of false positives

Example, Confidence 10^9 would mean that you'd accept a false positive for 1 in 10^9 tests.

Make sure that the result is statistically significant in accordance to the passed Confidence

Set the number of times a property should be executed before it is considered successful.

If you have a test that does not involve any generators and thus does not need to run repeatedly, you can use withTests 1 to define a property that will only be checked once.

The number of successful tests that need to be run before a property test is considered successful.

Can be constructed using numeric literals:

  200 :: TestLimit

Set the number of times a property is allowed to discard before the test runner gives up.

The number of discards to allow before giving up.

Can be constructed using numeric literals:

  10000 :: DiscardLimit

Set the number of times a property is allowed to shrink before the test runner gives up and prints the counterexample.

The number of shrinks to try before giving up on shrinking.

Can be constructed using numeric literals:

  1000 :: ShrinkLimit

Set the number of times a property will be executed for each shrink before the test runner gives up and tries a different shrink. See ShrinkRetries for more information.

The number of times to re-run a test during shrinking. This is useful if you are testing something which fails non-deterministically and you want to increase the change of getting a good shrink.

If you are doing parallel state machine testing, you should probably set shrink retries to something like 10. This will mean that during shrinking, a parallel test case requires 10 successful runs before it is passes and we try a different shrink.

Can be constructed using numeric literals:

  0 :: ShrinkRetries

Set the target that a property will skip to before it starts to run.

Where to start running a property's tests.

Generator for random values of a.

Monad transformer which can generate random values of a.

Class of monads which can generate input data for tests.

A range describes the bounds of a number to generate, which may or may not be dependent on a Size.

The constructor takes an origin between the lower and upper bound, and a function from Size to bounds. As the size goes towards 0, the values go towards the origin.

Tests are parameterized by the size of the randomly-generated data. The meaning of a Size value depends on the particular generator used, but it must always be a number between 0 and 99 inclusive.

A splittable random number generator.

A test monad allows the assertion of expectations.

A test monad transformer allows the assertion of expectations.

Annotates the source code with a message that might be useful for debugging a test failure.

Annotates the source code with a value that might be useful for debugging a test failure.

Logs a message to be displayed as additional information in the footer of the failure report.

Logs a value to be displayed as additional information in the footer of the failure report.

Another name for pure ().

Causes a test to fail.

Fails the test if the condition provided is False.

Fails the test and shows a git-like diff if the comparison operation evaluates to False when applied to its arguments.

The comparison function is the second argument, which may be counter-intuitive to Haskell programmers. However, it allows operators to be written infix for easy reading:

  diff y (<) 87
  diff x (<=) r

This function behaves like the unix diff tool, which gives a 0 exit code if the compared files are identical, or a 1 exit code code otherwise. Like unix diff, if the arguments fail the comparison, a /diff is shown.

Fails the test if the two arguments provided are not equal.

Fails the test if the two arguments provided are equal.

Test that a pair of encode / decode functions are compatible.

Given a printer from some type a -> b, and a parser with a potential failure case b -> f a. Ensure that a valid a round trips through the "print" and "parse" to yield the same a.

For example, types should have tripping Read and Show instances:

trippingShowRead :: (Show a, Read a, Eq a, MonadTest m) => a -> m ()
trippingShowRead a = tripping a show readEither

Fails the test if the value throws an exception when evaluated to weak head normal form (WHNF).

Fails the test if the value throws an exception when evaluated to normal form (NF).

Fails the test if the action throws an exception.

The benefit of using this over simply letting the exception bubble up is that the location of the closest evalM will be shown in the output.

Fails the test if the IO action throws an exception.

The benefit of using this over liftIO is that the location of the exception will be shown in the output.

Fails the test if the Either is Left, otherwise returns the value in the Right.

Fails the test if the action throws an exception, or if the Either is Left, otherwise returns the value in the Right.

Fails the test if the ExceptT is Left, otherwise returns the value in the Right.

Fails the test if the Maybe is Nothing, otherwise returns the value in the Just.

Fails the test if the action throws an exception, or if the Maybe is Nothing, otherwise returns the value in the Just.

The name of a classifier.

Should be constructed using OverloadedStrings:

  "apples" :: LabelName

Records the proportion of tests which satisfy a given condition.

   prop_with_classifier :: Property
   prop_with_classifier =
     property $ do
       xs <- forAll $ Gen.list (Range.linear 0 100) Gen.alpha
       for_ xs $ \x -> do
         classify "newborns" $ x == 0
         classify "children" $ x > 0 && x < 13
         classify "teens" $ x > 12 && x < 20

Require a certain percentage of the tests to be covered by the classifier.

   prop_with_coverage :: Property
   prop_with_coverage =
     property $ do
       match <- forAll Gen.bool
       cover 30 "True" $ match
       cover 30 "False" $ not match

The example above requires a minimum of 30% coverage for both classifiers. If these requirements are not met, it will fail the test.

Add a label for each test run. It produces a table showing the percentage of test runs that produced each label.

Like label, but uses Show to render its argument for display.

The specification for the expected behaviour of an Action. These are used to generate sequences of actions to test.

This is the main type you will use when writing state machine tests. gen is usually an instance of MonadGen, and m is usually an instance of MonadTest. These constraints appear when you pass your Command list to sequential or parallel.

Optional command configuration.

An instantiation of a Command which can be executed, and its effect evaluated.

A sequence of actions to execute.

A sequential prefix of actions to execute, with two branches to execute in parallel.

Executes a list of actions sequentially, verifying that all post-conditions are met and no exceptions are thrown.

To generate a sequence of actions to execute, see the sequential combinator in the Hedgehog.Gen module.

Executes the prefix actions sequentially, then executes the two branches in parallel, verifying that no exceptions are thrown and that there is at least one sequential interleaving where all the post-conditions are met.

To generate parallel actions to execute, see the parallel combinator in the Hedgehog.Gen module.

Variables are the potential or actual result of executing an action. They are parameterised by either Symbolic or Concrete depending on the phase of the test.

Symbolic variables are the potential results of actions. These are used when generating the sequence of actions to execute. They allow actions which occur later in the sequence to make use of the result of an action which came earlier in the sequence.

Concrete variables are the actual results of actions. These are used during test execution. They provide access to the actual runtime value of a variable.

The state update Callback for a command needs to be polymorphic in the type of variable because it is used in both the generation and the execution phase.

The order of arguments makes Var FunctorB and TraversableB, which is how Symbolic values are turned into Concrete ones.

Take the value from a concrete variable.

Take the value from an opaque concrete variable.

Symbolic values: Because hedgehog generates actions in a separate phase before execution, you will sometimes need to refer to the result of a previous action in a generator without knowing the value of the result (e.g., to get the ID of a previously-created user).

Symbolic variables provide a token to stand in for the actual variables at generation time (and in Require/Update callbacks). At execution time, real values are available, so your execute actions work on Concrete variables.

See also: Command, Var

Concrete values: At test-execution time, Symbolic values from generation are replaced with Concrete values from performing actions. This type gives us something of the same kind as Symbolic to pass as a type argument to Var.

Opaque values.

Useful if you want to put something without a Show instance inside something which you'd like to be able to display.

For example:

  data State v =
    State {
        stateRefs :: [Var (Opaque (IORef Int)) v]
      } deriving (Eq, Show)

Distribute one monad transformer over another.

Barbie-types that can be mapped over. Instances of FunctorB should satisfy the following laws:

bmap id = id
bmap f . bmap g = bmap (f . g)

There is a default bmap implementation for Generic types, so instances can derived automatically.

Barbie-types that can be traversed from left to right. Instances should satisfy the following laws:

 t . btraverse f   = btraverse (t . f)  -- naturality
btraverse Data.Functor.Identity = Data.Functor.Identity           -- identity
btraverse (Compose . fmap g . f) = Compose . fmap (btraverse g) . btraverse f -- composition

There is a default btraverse implementation for Generic types, so instances can derived automatically.

Lifting of the Eq class to unary type constructors.

Any instance should be subject to the following law that canonicity is preserved:

liftEq (==) = (==)

This class therefore represents the generalization of Eq by decomposing its main method into a canonical lifting on a canonical inner method, so that the lifting can be reused for other arguments than the canonical one.

Lift the standard (==) function through the type constructor.

Lifting of the Ord class to unary type constructors.

Any instance should be subject to the following law that canonicity is preserved:

liftCompare compare = compare

This class therefore represents the generalization of Ord by decomposing its main method into a canonical lifting on a canonical inner method, so that the lifting can be reused for other arguments than the canonical one.

Lift the standard compare function through the type constructor.

Lifting of the Show class to unary type constructors.

Any instance should be subject to the following laws that canonicity is preserved:

liftShowsPrec showsPrec showList = showsPrec

liftShowList showsPrec showList = showList

This class therefore represents the generalization of Show by decomposing it's methods into a canonical lifting on a canonical inner method, so that the lifting can be reused for other arguments than the canonical one.

Lift the standard showsPrec and showList functions through the type constructor.

Higher-order traversable functors.

Deprecated in favor of TraversableB which can be derived using GHC.Generics