Test.ChasingBottoms.ContinuousFunctions

A framework for generating possibly non-strict, partial, continuous functions.

The functions generated using the standard QuickCheck Arbitrary instances are all strict. In the presence of partial and infinite values testing using only strict functions leads to worse coverage than if more general functions are used, though.

Using isBottom it is relatively easy to generate possibly non-strict functions that are, in general, not monotone. For instance, using

type Cogen a = forall b. a -> Gen b -> Gen b

integer :: Gen Integer
integer = frequency [ (1, return bottom), (10, arbitrary) ]

coBool :: CoGen Bool
coBool b | isBottom b = variant 0
coBool False          = variant 1
coBool True           = variant 2

function :: Cogen a -> Gen b -> Gen (a -> b)
function coGen gen = promote (\a -> coGen a gen)

we can generate possibly non-strict functions from Bool to Integer using function coBool integer. There is a high likelihood that the functions generated are not monotone, though. The reason that we can get non-monotone functions in a language like Haskell is that we are using the impure function isBottom.

Sometimes using possibly non-monotone functions is good enough, since that set of functions is a superset of the continuous functions. However, say that we want to test that x <=! y implies that f x <=! f y for all functions f (whenever the latter expression returns a total result). This property is not valid in the presence of non-monotone functions.

By avoiding isBottom and, unlike the standard coarbitrary functions, deferring some pattern matches, we can generate continuous, possibly non-strict functions. There are two steps involved in generating a continuous function using the framework defined here.

1. First the argument to the function is turned into a PatternMatch. A PatternMatch wraps up the pattern match on the top-level constructor of the argument, plus all further pattern matches on the children of the argument. Just like when coarbitrary is used a pattern match is represented as a generator transformer. The difference here is that there is not just one transformation per input, but one transformation per constructor in the input. PatternMatches can be constructed generically using match.

2. Then the result is generated, almost like for a normal Arbitrary instance. However, for each constructor generated a subset of the transformations from step 1 are applied. This transformation application is wrapped up in the function transform.

The net result of this is that some pattern matches are performed later, or not at all, so functions can be lazy.

Here is an example illustrating typical use of this framework:

data Tree a
  = Branch (Tree a) (Tree a)
  | Leaf a
    deriving (Show, Typeable, Data)

finiteTreeOf :: MakeResult a -> MakeResult (Tree a)
finiteTreeOf makeResult = sized' tree
  where
  tree size = transform $
    if size == 0 then
      baseCase
     else
      frequency' [ (1, baseCase)
                 , (1, liftM2 Branch tree' tree')
                 ]
    where
    tree' = tree (size `div` 2)

    baseCase =
      frequency' [ (1, return bottom)
                 , (2, liftM Leaf makeResult)
                 ]

Note the use of transform. To use this function to generate functions of type Bool -> Tree Integer we can use

forAll (functionTo (finiteTreeOf flat)) $
  \(f :: Bool -> Tree Integer) ->
    ...