Test.Syd

A synonym for xit

A synonym for xitWithOuter

A synonym for xitWithBoth

A synonym for xitWithAll

Declare a test that has not been written yet.

Declare a test that has not been written yet for the given reason.

Test that the given text is the same as what we find in the given golden file.

Test that the produced text is the same as what we find in the given golden file.

Test that the given bytestring is the same as what we find in the given golden file.

Test that the produced bytestring is the same as what we find in the given golden file.

Test that the given lazy bytestring is the same as what we find in the given golden file.

Note: This converts the lazy bytestring to a strict bytestring first.

Test that the produced bytestring is the same as what we find in the given golden file.

Note: This converts the lazy bytestring to a strict bytestring first.

Test that the given lazy bytestring is the same as what we find in the given golden file.

Note: This converts the builder to a strict bytestring first.

Test that the produced bytestring is the same as what we find in the given golden file.

Note: This converts the builder to a strict bytestring first.

Test that the given string is the same as what we find in the given golden file.

Test that the produced string is the same as what we find in the given golden file.

Test that the show instance has not changed for the given value.

Test that the show instance has not changed for the given value, via ppShow.

The golden test context for adding context to a golden test assertion:

goldenTestCompare = \actual expected ->
  if actual == expected
    then Nothing
    else Just $ Context (stringsNotEqualButShouldHaveBeenEqual actual expected) (goldenContext fp)

A golden test for output of type a.

The purpose of a golden test is to ensure that the output of a certain process does not change even over time.

Golden tests can also be used to show how the output of a certain process changes over time and force code reviewers to review the diff that they see in the PR.

This works by saving a golden output in the repository somewhere, committing it, and then compare that golden output to the output that is currently being produced. You can use `--golden-reset` to have sydtest update the golden output by writing the current output.

Define a test for each file in the given directory.

Example:

  scenarioDir "test_resources/even" $ \fp ->
    it "contains an even number" $ do
      s <- readFile fp
      n <- readIO s
      (n :: Int) `shouldSatisfy` even

Define a test for each file in the given directory, recursively.

Example:

  scenarioDirRecur "test_resources/odd" $ \fp ->
    it "contains an odd number" $ do
      s <- readFile fp
      n <- readIO s
      (n :: Int) `shouldSatisfy` odd

Assert that two values are equal according to ==.

Assert that two values are not equal according to ==.

Assert that a value satisfies the given predicate.

Assert that a value satisfies the given predicate with the given predicate name.

Assert that a value does not satisfy the given predicate.

Assert that a value does not satisfy the given predicate with the given predicate name.

Assert that computation returns the given value (according to ==).

Assert that computation returns the given value (according to ==).

Assert that the given list has the given prefix

Assert that the given list has the given suffix

Assert that the given list has the given infix

Assert that the given list contains all elements from the other given list and only them, perhaps in a different order.

Make a test fail

Note that this is mostly backward compatible, but it has return type a instead of () because execution will not continue beyond this function. In this way it is not entirely backward compatible with hspec because now there could be an ambiguous type error.

Annotate a given action with a context, for contextual assertions

This is a completely different function from the function with the same name in hspec. In hspec, context is a synonym for describe, but in sydtest, context is used for contextual failures.

For easy hspec migration

Assert that a given IO action throws an exception that matches the given exception

For easy hspec migration

Assert that two Strings are equal according to ==.

Note that using function could mess up the colours in your terminal if the Texts contain ANSI codes. In that case you may want to show your values first or use shouldBe instead.

Assert that two Texts are equal according to ==.

Note that using function could mess up the colours in your terminal if the Texts contain ANSI codes. In that case you may want to show your values first or use shouldBe instead.

An assertion that says two Strings should have been equal according to ==.

Note that using function could mess up the colours in your terminal if the Texts contain ANSI codes. In that case you may want to show your values first or use shouldBe instead.

An assertion that says two Texts should have been equal according to ==.

Note that using function could mess up the colours in your terminal if the Texts contain ANSI codes. In that case you may want to show your values first or use shouldBe instead.

An assertion that says two ByteStrings should have been equal according to ==.

A special exception that sydtest knows about and can display nicely in the error output

This is exported outwards so that you can define golden tests for custom types.

You will probably not want to use this directly in everyday tests, use shouldBe or a similar function instead.

Run a custom action before all spec items in a group, to set up an outer resource a.

Run a custom action before all spec items in a group without setting up any outer resources.

Run a custom action before all spec items in a group, to set up an outer resource b by using the outer resource a.

Run a custom action after all spec items, using the outer resource a.

Run a custom action after all spec items, using all the outer resources.

Run a custom action after all spec items without using any outer resources.

Run a custom action before and/or after all spec items in group, to provide access to a resource a.

See the FOOTGUN note in the docs for around_.

Run a custom action before and/or after all spec items in a group without accessing any resources.

FOOTGUN

This combinator gives the programmer a lot of power. In fact, it gives the programmer enough power to break the test framework. Indeed, you can provide a wrapper function that just _doesn't_ run the function like this:

spec :: Spec
spec = do
   let don'tDo :: IO () -> IO ()
       don'tDo _ = pure ()
   aroundAll_ don'tDo $ do
     it "should pass" True

During execution, you'll then get an error like this:

thread blocked indefinitely in an MVar operation

The same problem exists when using around_.

Something even more pernicious goes wrong when you run the given action more than once like this:

spec :: Spec
spec = do
   let doTwice :: IO () -> IO ()
       doTwice f = f >> f
   aroundAll_ doTwice $ do
     it "should pass" True

In this case, the test will "just work", but it will be executed twice even if the output reports that it only passed once.

Note: If you're interested in fixing this, talk to me, but only after GHC has gotten impredicative types because that will likely be a requirement.

Run a custom action before and/or after all spec items in a group to provide access to a resource a while using a resource b

See the FOOTGUN note in the docs for around_.

Run a custom action before and/or after all spec items in a group to provide access to a resource a while using all outer resources.

Run a custom action before every spec item, to set up an inner resource inner.

Note that this function turns off shrinking. See https://github.com/nick8325/quickcheck/issues/331

Run a custom action before every spec item without setting up any inner resources.

Note that this function turns off shrinking. See https://github.com/nick8325/quickcheck/issues/331

Run a custom action after every spec item, using the inner resource c.

Note that this function turns off shrinking. See https://github.com/nick8325/quickcheck/issues/331

Run a custom action after every spec item without using any inner resources.

Note that this function turns off shrinking. See https://github.com/nick8325/quickcheck/issues/331

Run a custom action before and/or after every spec item, to provide access to an inner resource c.

See the FOOTGUN note in the docs for around_.

Note that this function turns off shrinking. See https://github.com/nick8325/quickcheck/issues/331

Run a custom action before and/or after every spec item without accessing any inner resources.

It is important that the wrapper function that you provide runs the action that it gets _exactly once_.

FOOTGUN

This combinator gives the programmer a lot of power. In fact, it gives the programmer enough power to break the test framework. Indeed, you can provide a wrapper function that just _doesn't_ run the function like this:

spec :: Spec
spec = do
   let don'tDo :: IO () -> IO ()
       don'tDo _ = pure ()
   around_ don'tDo $ do
     it "should pass" True

During execution, you'll then get an error like this:

thread blocked indefinitely in an MVar operation

The same problem exists when using Test.Syd.Def.Around.aroundAll_.

The same thing will go wrong if you run the given action more than once like this:

spec :: Spec
spec = do
   let doTwice :: IO () -> IO ()
       doTwice f = f >> f
   around_ doTwice $ do
     it "should pass" True

Note: If you're interested in fixing this, talk to me, but only after GHC has gotten impredicative types because that will likely be a requirement.

Note that this function turns off shrinking. See https://github.com/nick8325/quickcheck/issues/331

Run a custom action before and/or after every spec item, to provide access to an inner resource c while using the inner resource d.

See the FOOTGUN note in the docs for around_.

Note that this function turns off shrinking. See https://github.com/nick8325/quickcheck/issues/331

A function that can provide a resource.

You can think of this as a potentially-resource-aware version of 'IO resource'. In other words, it's like an 'IO resource' that can clean up after itself.

This type has a monad instance, which means you can now compose setup functions using regular do-notation. This works together nicely with most supplier functions. Some examples:

• Network.Wai.Handler.Warp.testWithApplication

• Path.IO.withSystemTempDir

Note that these examples already have functions defined for them in sydtest companion libraries.

Use around with a SetupFunc

Use aroundWith with a SetupFunc

Use aroundWith' with a SetupFunc

Use aroundAll with a SetupFunc

Use aroundAllWith with a SetupFunc

Use aroundAllWithAll with a SetupFunc

Declare that all tests below must be run sequentially

Declare that all tests below may be run in parallel. (This is the default.)

Annotate a test group with Parallelism.

Declare that the order of execution of all tests below may be randomised.

Declare that the order of execution of all tests below must not be randomised.

Annotate a test group with ExecutionOrderRandomisation.

Modify the test timeout

Turn off timeouts

Turn off timeouts

Modify the number of retries to use in flakiness diagnostics.

Turn off retries

Make the number of retries this constant

Mark a test suite as "potentially flaky" with a given number of retries.

This will retry any test in the given test group up to the given number of tries, and pass a test if it passes once. The test output will show which tests were flaky.

WARNING: This is only a valid approach to dealing with test flakiness if it is true that tests never pass accidentally. In other words: tests using flaky must be guaranteed to fail every time if an error is introduced in the code, it should only be added to deal with accidental failures, never accidental passes.

Like flaky, but also shows the given message to the user whenever the test is flaky.

You could use it like this:

>>> flakyWith 3 "Something sometimes goes wrong with the database, see issue 6346" ourTestSuite

Mark a test suite as "must not be flaky".

This is useful to have a subgroup of a group marked as flaky that must not be flaky afteral.

Mark a test suite as 'potentially flaky', such that it will not fail if it is flaky but passes at least once.

Like potentiallyFlaky, but with a message.

Annotate a test group with FlakinessMode.

Mark a test suite as 'should pass'

Mark a test suite as 'should fail'

Annotate a test suite with ExpectationMode

Run a test suite during test suite definition.

This function only exists for backward compatibility. You can also just use liftIO instead.

The test definition monad

This type has three parameters:

• outers: A type-level list of the outer resources. These are resources that are prived once, around a group of tests. (This is the type of the results of aroundAll.)

• inner: The inner resource. This is a resource that is set up around every test, and even every example of a property test. (This is the type of the result of around.)

• result: The result (TestDefM is a monad.)

In practice, all of these three parameters should be () at the top level.

A synonym for a test suite definition

A tree of tests

This type has three parameters:

• outers: A type-level list of the outer resources. These are resources that are prived once, around a group of tests. (This is the type of the results of aroundAll.)

• inner: The inner resource. This is a resource that is set up around every test, and even every example of a property test. (This is the type of the result of around.)

• result: The result (TestDefM is a monad.)

In practice, all of these three parameters should be () at the top level.

When you're just using sydtest and not writing a library for sydtest, you probably don't even want to concern yourself with this type.

A synonym for easy migration from hspec

A synonym for easy migration from hspec

A synonym for easy migration from hspec