This module is used for defining new types of rules for Shake build systems, e.g. to support values stored in a database.
Most users will find the built-in set of rules sufficient. The functions in this module are designed for high-performance,
not ease of use or abstraction. As a result, they are difficult to work with and change more often than the other parts of Shake.
Before writing a builtin rule you are encouraged to use addOracle or addOracleCache if possible.
With all those warnings out the way, read on for the grungy details.
Shake "Builtin" rules are ones map keys to values - e.g. files to file contents. For each builtin rule you need to think:
What is the key type, which uniquely identifies each location, e.g. a filename.
What is the value type. The value is not necessarily the full value, but is the result people can get if they ask
for the value associated with the key. As an example, for files when you need a file you don't get any value back from
the file, so a simple file rule could have () as its value.
What information is stored between runs. This information should be sufficient to check if the value has changed since last time,
e.g. the modification time for files.
Typically a custom rule will define a wrapper of type Rules that calls addBuiltinRule, along with a type-safe wrapper over
apply so users can introduce dependencies.
Extensions
Once you have implemented the basic functionality there is more scope for embracing additional features of Shake, e.g.:
Shake provides a very comprehensive file rule which currently runs to over 500 lines of code, and supports lots of features
and optimisations. However, let's imagine we want to define a simpler rule type for files. As mentioned earlier, we have to make some decisions.
A key will just be the file name.
A value will be () - when the user depends on a file they don't expect any information in return.
The stored information will be the contents of the file, in it's entirety. Alternative choices would be the modtime or a hash of the contents,
but Shake doesn't require that. The stored information in Shake must be stored in a ByteString, so we pack and
unpack to convert.
We will allow user rules to be defined saying how to build any individual file.
First we define the type of key and value, deriving all the necessary type classes. We define a newtype over FilePath so we can
guarantee not to conflict with anyone else. Typically you wouldn't export the File type, providing only sugar functions over it.
Since we have decided we are also going to have user rules, we need to define a new type to capture the information stored by the rules.
We need to store at least the file it is producing and the action, which we do with:
data FileRule = FileRule File (Action ())
With the definitions above users could call apply and addUserRule directly, but that's tedious and not very type safe. To make it easier
we introduce some helpers:
These helpers just add our type names, providing a more pleasant interface for the user. Using these function we can
exercise our build system with:
example = do
fileRule "a.txt" $ pure ()
fileRule "b.txt" $ do
fileNeed "a.txt"
liftIO $ writeFile "b.txt" . reverse =<< readFile "a.txt"
action $ fileNeed "b.txt"
This example defines rules for a.txt (a source file) and b.txt (the reverse of a.txt). At runtime this example will
complain about not having a builtin rule for File, so the only thing left is to provide one.
addBuiltinFileRule :: Rules ()
addBuiltinFileRule = addBuiltinRule noLint noIdentity run
where
fileContents (File x) = do b <- IO.doesFileExist x; if b then IO.readFile' x else pure ""
run :: BuiltinRun File ()
run key old mode = do
now <- liftIO $ fileContents key
if mode == RunDependenciesSame && fmap BS.unpack old == Just now then
pure $ RunResult ChangedNothing (BS.pack now) ()
else do
(_, act) <- getUserRuleOne key (const Nothing) $ \(FileRule k act) -> if k == key then Just act else Nothing
act
now <- liftIO $ fileContents key
pure $ RunResult ChangedRecomputeDiff (BS.pack now) ()
We define a wrapper addBuiltinFileRule that calls addBuiltinRule, opting out of linting and cached storage.
The only thing we provide is a BuiltinRun function which gets the previous state, and whether any dependency has changed,
and decides whether to rebuild. If something has changed we call getUserRuleOne to find the users rule and rerun it.
The RunResult says what changed (either ChangedNothing or ChangedRecomputeDiff in our cases), gives us a new stored value
(just packing the contents) and the value which is ().
To execute our example we need to also call addBuiltinFileRule, and now everything works.
Defining builtin rules
9 declarations
Functions and types for defining new types of Shake rules.
Before looking at this function, you should read the warnings at the top of this module.
This function is not often necessary in build systems.
Define a builtin rule, passing the functions to run in the right circumstances.
The key and value types will be what is used by apply.
As a start, you can use noLint and noIdentity as the first two functions,
but are required to supply a suitable BuiltinRun.
Raises an error if any other rule exists at this type.
The action performed by --lint for a given key/value pair.
At the end of the build the lint action will be called for each key that was built this run,
passing the value it produced. Return Nothing to indicate the value has not changed and
is acceptable, or Just an error message to indicate failure.
For builtin rules where the value is expected to change, or has no useful checks to perform.
use Development.Shake.Rules.noLint.
Produce an identity for a value that can be used to do direct equality. If you have a custom
notion of equality then the result should return only one member from each equivalence class,
as values will be compared for literal equality.
The result of the identity should be reasonably short (if it is excessively long, hash it).
For rules where the value is never compatible use Development.Shake.Rules.noIdentity, which
returns Nothing. This will disable shared caches of anything that depends on it.
A suitable BuiltinIdentity that always fails with a runtime error, incompatible with shakeShare.
Use this function if you don't care about shakeShare, or if your rule provides a dependency that can
never be cached (in which case you should also call historyDisable).
Define a rule between key and value. As an example, a typical BuiltinRun will look like:
run key oldStore mode = do
...
pure $ RunResult change newStore newValue
Where you have:
key, how to identify individual artifacts, e.g. with file names.
oldStore, the value stored in the database previously, e.g. the file modification time.
mode, either RunDependenciesSame (none of your dependencies changed, you can probably not rebuild) or
RunDependenciesChanged (your dependencies changed, probably rebuild).
change, usually one of either ChangedNothing (no work was required) or ChangedRecomputeDiff
(I reran the rule and it should be considered different).
newStore, the new value to store in the database, which will be passed in next time as oldStore.
newValue, the result that apply will return when asked for the given key.
Execute a rule, returning the associated values. If possible, the rules will be run in parallel.
This function requires that appropriate rules have been added with addBuiltinRule.
All key values passed to apply become dependencies of the Action.
Apply a single rule, equivalent to calling apply with a singleton list. Where possible,
use apply to allow parallelism.
User rules
4 declarations
Define user rules that can be used by builtin rules.
Absent any builtin rule making use of a user rule at a given type, a user rule will have on effect -
they have no inherent effect or interpretation on their own.
Get the user rules that were added at a particular type which return Just on a given function.
Return all equally applicable rules, paired with the version of the rule
(set by versioned). Where rules are specified with alternatives or priority
the less-applicable rules will not be returned.
Save a value to the history. Record the version of any user rule
(or 0), and a payload. Must be run at the end of the rule, after
any dependencies have been captured. If history is enabled, stores the information
in a cache.
This function relies on produces to have been called correctly to describe
which files were written during the execution of this rule.