Test.Hspec.Hedgehog

Embed a Hedgehog PropertyT IO () in an hspec test.

spec :: Spec
spec =
  describe "my great test" $ do
    it "generates stuff" $
      hedgehog $ do
        a <- forAll generator
        a === expected

This function is only used to fix the type of the PropertyT monad transformer. The functions in Hedgehog are typically abstract in a MonadTest, and it's easy to get ambiguous type errors if you leave this out.

Use modified Args for given spec.

Use a modified maxSuccess for given spec.

Use a modified maxDiscardRatio for given spec.

Use a modified maxShrinks for given spec.

Add a label for each test run. It produces a table showing the percentage of test runs that produced each label.

A property test, along with some configurable limits like how many times to run the test.

The number of discards to allow before giving up.

Can be constructed using numeric literals:

  10000 :: DiscardLimit

The number of shrinks to try before giving up on shrinking.

Can be constructed using numeric literals:

  1000 :: ShrinkLimit

The number of successful tests that need to be run before a property test is considered successful.

Can be constructed using numeric literals:

  200 :: TestLimit

A splittable random number generator.

A test monad allows the assertion of expectations.

The property monad transformer allows both the generation of test inputs and the assertion of expectations.

Creates a property with the default configuration.

Fails the test if the condition provided is False.

Distribute one monad transformer over another.

Generator for random values of a.

Class of monads which can generate input data for tests.

Monad transformer which can generate random values of a.

Higher-order traversable functors.

Deprecated in favor of TraversableB which can be derived using GHC.Generics

Opaque values.

Useful if you want to put something without a Show instance inside something which you'd like to be able to display.

For example:

  data State v =
    State {
        stateRefs :: [Var (Opaque (IORef Int)) v]
      } deriving (Eq, Show)

Fails the test if the two arguments provided are equal.

Fails the test if the two arguments provided are not equal.

The acceptable occurrence of false positives

Example, Confidence 10^9 would mean that you'd accept a false positive for 1 in 10^9 tests.

A named collection of property tests.

The name of a group of properties.

Should be constructed using OverloadedStrings:

  "fruit" :: GroupName

The name of a classifier.

Should be constructed using OverloadedStrings:

  "apples" :: LabelName

The name of a property.

Should be constructed using OverloadedStrings:

  "apples" :: PropertyName

The number of times to re-run a test during shrinking. This is useful if you are testing something which fails non-deterministically and you want to increase the change of getting a good shrink.

If you are doing parallel state machine testing, you should probably set shrink retries to something like 10. This will mean that during shrinking, a parallel test case requires 10 successful runs before it is passes and we try a different shrink.

Can be constructed using numeric literals:

  0 :: ShrinkRetries

Where to start running a property's tests.

A test monad transformer allows the assertion of expectations.

Annotates the source code with a message that might be useful for debugging a test failure.

Annotates the source code with a value that might be useful for debugging a test failure.

Records the proportion of tests which satisfy a given condition.

   prop_with_classifier :: Property
   prop_with_classifier =
     property $ do
       xs <- forAll $ Gen.list (Range.linear 0 100) Gen.alpha
       for_ xs $ \x -> do
         classify "newborns" $ x == 0
         classify "children" $ x > 0 && x < 13
         classify "teens" $ x > 12 && x < 20

Like label, but uses Show to render its argument for display.

Require a certain percentage of the tests to be covered by the classifier.

   prop_with_coverage :: Property
   prop_with_coverage =
     property $ do
       match <- forAll Gen.bool
       cover 30 "True" $ match
       cover 30 "False" $ not match

The example above requires a minimum of 30% coverage for both classifiers. If these requirements are not met, it will fail the test.

Fails the test and shows a git-like diff if the comparison operation evaluates to False when applied to its arguments.

The comparison function is the second argument, which may be counter-intuitive to Haskell programmers. However, it allows operators to be written infix for easy reading:

  diff y (<) 87
  diff x (<=) r

This function behaves like the unix diff tool, which gives a 0 exit code if the compared files are identical, or a 1 exit code code otherwise. Like unix diff, if the arguments fail the comparison, a /diff is shown.

Discards the current test entirely.

Fails the test if the value throws an exception when evaluated to weak head normal form (WHNF).

Fails the test if the Either is Left, otherwise returns the value in the Right.

Fails the test if the action throws an exception, or if the Either is Left, otherwise returns the value in the Right.

Fails the test if the ExceptT is Left, otherwise returns the value in the Right.

Fails the test if the IO action throws an exception.

The benefit of using this over liftIO is that the location of the exception will be shown in the output.

Fails the test if the action throws an exception.

The benefit of using this over simply letting the exception bubble up is that the location of the closest evalM will be shown in the output.

Fails the test if the Maybe is Nothing, otherwise returns the value in the Just.

Fails the test if the action throws an exception, or if the Maybe is Nothing, otherwise returns the value in the Just.

Fails the test if the value throws an exception when evaluated to normal form (NF).

Causes a test to fail.

Logs a message to be displayed as additional information in the footer of the failure report.

Logs a value to be displayed as additional information in the footer of the failure report.

Generates a random input for the test by running the provided generator.

Generates a random input for the test by running the provided generator.

This is a the same as forAll but allows the user to provide a custom rendering function. This is useful for values which don't have a Show instance.

Another name for pure ().

Lift a test in to a property.

Because both TestT and PropertyT have MonadTest instances, this function is not often required. It can however be useful for writing functions directly in TestT and thus gaining a MonadTransControl instance at the expense of not being able to generate additional inputs using forAll.

An example where this is useful is parallel state machine testing, as executeParallel requires MonadBaseControl IO in order to be able to spawn threads in MonadTest.

Make sure that the result is statistically significant in accordance to the passed Confidence

Set the number of times a property is allowed to discard before the test runner gives up.

Set the number of times a property will be executed for each shrink before the test runner gives up and tries a different shrink. See ShrinkRetries for more information.

Set the number of times a property is allowed to shrink before the test runner gives up and prints the counterexample.

Set the target that a property will skip to before it starts to run.

Set the number of times a property should be executed before it is considered successful.

If you have a test that does not involve any generators and thus does not need to run repeatedly, you can use withTests 1 to define a property that will only be checked once.

A range describes the bounds of a number to generate, which may or may not be dependent on a Size.

The constructor takes an origin between the lower and upper bound, and a function from Size to bounds. As the size goes towards 0, the values go towards the origin.

Tests are parameterized by the size of the randomly-generated data. The meaning of a Size value depends on the particular generator used, but it must always be a number between 0 and 99 inclusive.

Check a property.

Check a group of properties in parallel.

Warning: although this check function runs tests faster than checkSequential, it should be noted that it may cause problems with properties that are not self-contained. For example, if you have a group of tests which all use the same database table, you may find that they interfere with each other when being run in parallel.

Using Template Haskell for property discovery:

tests :: IO Bool
tests =
  checkParallel $$(discover)

With manually specified properties:

tests :: IO Bool
tests =
  checkParallel $ Group "Test.Example" [
      ("prop_reverse", prop_reverse)
    ]

Check a group of properties sequentially.

Using Template Haskell for property discovery:

tests :: IO Bool
tests =
  checkSequential $$(discover)

With manually specified properties:

tests :: IO Bool
tests =
  checkSequential $ Group "Test.Example" [
      ("prop_reverse", prop_reverse)
    ]

Check a property using a specific size and seed.

An instantiation of a Command which can be executed, and its effect evaluated.

Optional command configuration.

The specification for the expected behaviour of an Action. These are used to generate sequences of actions to test.

This is the main type you will use when writing state machine tests. gen is usually an instance of MonadGen, and m is usually an instance of MonadTest. These constraints appear when you pass your Command list to sequential or parallel.

Concrete values: At test-execution time, Symbolic values from generation are replaced with Concrete values from performing actions. This type gives us something of the same kind as Symbolic to pass as a type argument to Var.

A sequential prefix of actions to execute, with two branches to execute in parallel.

A sequence of actions to execute.

Symbolic values: Because hedgehog generates actions in a separate phase before execution, you will sometimes need to refer to the result of a previous action in a generator without knowing the value of the result (e.g., to get the ID of a previously-created user).

Symbolic variables provide a token to stand in for the actual variables at generation time (and in Require/Update callbacks). At execution time, real values are available, so your execute actions work on Concrete variables.

See also: Command, Var

Variables are the potential or actual result of executing an action. They are parameterised by either Symbolic or Concrete depending on the phase of the test.

Symbolic variables are the potential results of actions. These are used when generating the sequence of actions to execute. They allow actions which occur later in the sequence to make use of the result of an action which came earlier in the sequence.

Concrete variables are the actual results of actions. These are used during test execution. They provide access to the actual runtime value of a variable.

The state update Callback for a command needs to be polymorphic in the type of variable because it is used in both the generation and the execution phase.

The order of arguments makes Var FunctorB and TraversableB, which is how Symbolic values are turned into Concrete ones.

Take the value from a concrete variable.

Executes the prefix actions sequentially, then executes the two branches in parallel, verifying that no exceptions are thrown and that there is at least one sequential interleaving where all the post-conditions are met.

To generate parallel actions to execute, see the parallel combinator in the Hedgehog.Gen module.

Executes a list of actions sequentially, verifying that all post-conditions are met and no exceptions are thrown.

To generate a sequence of actions to execute, see the sequential combinator in the Hedgehog.Gen module.

Take the value from an opaque concrete variable.

Discover all the properties in a module.

Functions starting with prop_ are assumed to be properties.

Test that a pair of encode / decode functions are compatible.

Given a printer from some type a -> b, and a parser with a potential failure case b -> f a. Ensure that a valid a round trips through the "print" and "parse" to yield the same a.

For example, types should have tripping Read and Show instances:

trippingShowRead :: (Show a, Read a, Eq a, MonadTest m) => a -> m ()
trippingShowRead a = tripping a show readEither

Lift the standard compare function through the type constructor.

Lift the standard (==) function through the type constructor.

Lift the standard showsPrec and showList functions through the type constructor.

Barbie-types that can be mapped over. Instances of FunctorB should satisfy the following laws:

bmap id = id
bmap f . bmap g = bmap (f . g)

There is a default bmap implementation for Generic types, so instances can derived automatically.

Barbie-types that can be traversed from left to right. Instances should satisfy the following laws:

 t . btraverse f   = btraverse (t . f)  -- naturality
btraverse Data.Functor.Identity = Data.Functor.Identity           -- identity
btraverse (Compose . fmap g . f) = Compose . fmap (btraverse g) . btraverse f -- composition

There is a default btraverse implementation for Generic types, so instances can derived automatically.

Lifting of the Eq class to unary type constructors.

Any instance should be subject to the following law that canonicity is preserved:

liftEq (==) = (==)

This class therefore represents the generalization of Eq by decomposing its main method into a canonical lifting on a canonical inner method, so that the lifting can be reused for other arguments than the canonical one.

Lifting of the Ord class to unary type constructors.

Any instance should be subject to the following law that canonicity is preserved:

liftCompare compare = compare

This class therefore represents the generalization of Ord by decomposing its main method into a canonical lifting on a canonical inner method, so that the lifting can be reused for other arguments than the canonical one.

Lifting of the Show class to unary type constructors.

Any instance should be subject to the following laws that canonicity is preserved:

liftShowsPrec showsPrec showList = showsPrec

liftShowList showsPrec showList = showList

This class therefore represents the generalization of Show by decomposing it's methods into a canonical lifting on a canonical inner method, so that the lifting can be reused for other arguments than the canonical one.