Text.Regex.Pcre2

Introduction

Atop the low-level binding to the C API, we present a high-level interface to add regular expressions to Haskell programs.

All input and output strings are strict Text, which maps directly to how PCRE2 operates on strings of 8-bit-wide code units.

The C API requires pattern strings to be compiled and the compiled patterns to be executed on subject strings in discrete steps. We hide this procedure, accepting pattern and subject as arguments in a single function, essentially:

pattern -> subject -> result

The implementation guarantees that, when partially applied to pattern but not subject, the resulting function will close on the underlying compiled object and reuse it for every subject it is subsequently applied to.

Likewise, we do not require the user to know whether a PCRE2 option is to be applied at pattern compile time or match time. Instead we fold all possible options into a single datatype, Option. Most functions have vanilla and configurable variants; the latter have "Opt" in the name and accept a value of this type.

Similar to how head :: [a] -> a sacrifices totality for type simplicity, we represent user errors as imprecise exceptions. Unlike with head, these exceptions are typed (as SomePcre2Exceptions); moreover, we offer Template Haskell facilities that can intercept some of these errors before the program is run. (Failure to match is not considered a user error and is represented by empty; see below.)

There's more than one way to do it with this library. The choices between functions and traversals, poly-kinded Captures and plain lists, string literals and quasi-quotations, quasi-quoted expressions and quasi-quoted patterns...these are left to the user. She will observe that advanced features' extra safety, power, and convenience entail additional language extensions, cognitive overhead, and (for lenses) library dependencies, so it's really a matter of finding the best trade-offs for her case.

Definitions

Pattern

The string defining a regular expression. Refer to syntax

here

.

Subject

The string the compiled regular expression is executed on.

Regex

A function of the form

Text -> result

, where the argument is the subject. It is "compiled" via partial application as discussed above. (Lens users: A regex has the more abstract form

Lens.Micro.Traversal' Text result

, but the concept is the same.)

Capture (or capture group)

Any substrings of the subject matched by the pattern, meaning the whole pattern and any parenthesized groupings. The PCRE2 docs do not refer to the former as a "capture"; however it is accessed the same way in the C API, just with index 0, so we will consider it the 0th capture for consistency. Parenthesized captures are implicitly numbered from 1.

Unset capture

A capture considered unset as distinct from empty. This can arise from matching the pattern

(a)?

to an empty subject—the 0th capture will be set as empty, but the 1st will be unset altogether. We represent both as empty

Text

for simplicity. See below for discussions about how unset captures may be detected or substituted using this library.

Named capture

A parenthesized capture can be named like this:

(?<foo>...)

. Whether they have names or not, captures are always numbered as described above.

Performance

Each API function is designed such that, when a regex is obtained, the underlying C data generated from the pattern and any options is reused for that regex's lifetime. Care should be taken that the same regex is not recreated ex nihilo and discarded for each new subject:

isEmptyOrHas2Digits :: Text -> Bool
isEmptyOrHas2Digits s = Text.null s || matches "\\d{2}" s -- bad, fully applied

Instead, store it in a partially applied state:

isEmptyOrHas2Digits = (||) <$> Text.null <*> matches "\\d{2}" -- OK but abstruse

When in doubt, always create regexes as top-level values:

has2Digits :: Text -> Bool
has2Digits = matches "\\d{2}"

isEmptyOrHas2Digits s = Text.null s || has2Digits s -- good

Note: Template Haskell regexes are immune from this problem and may be freely inlined; see below.

Also of note is the optimization that, for each capture that's more than half the length of the subject, a zero-copy Text is produced in constant time and space. This can yield a large performance boost in many cases, for example when splitting lines into key-value pairs as in the teaser. A downside, however, is that retaining these slices in memory will carry the overhead of the dead portions of the subject (still guaranteed to be less than the slices in total size).

Handling results and errors

In contrast to other APIs where there are separate functions to request single versus global matching, we accomplish this (since 2.0.0) in a unified fashion using the Alternative typeclass. Typically the user will choose from two instances, Maybe and []:

b2 :: (Alternative f) => Text -> f Text
b2 = match "b.."

-- Zero or one match
findB2 :: Text -> Maybe Text
findB2 = b2

-- Zero or more matches
findAllB2s :: Text -> [Text]
findAllB2s = b2

Other instances exist for niche uses, notably STM, that of optparse-applicative, and those of parser combinator libraries such as megaparsec.

By contrast, user errors are thrown purely. If a user error is to be caught, it must be at the site where the match or substitution results are evaluated. As a particular consequence, pattern compile errors are deferred to match sites.

>>> broken = match "*">>> :t brokenbroken :: Alternative f => Text -> f Text>>> broken "foo"*** Exception: pcre2_compile: quantifier does not follow a repeatable item                    *                    ^

evaluate comes in handy to force results into the IO monad in order to catch errors reliably:

>>> :set -XTypeApplications>>> handle @SomePcre2Exception (\_ -> return Nothing) $ evaluate $ broken "foo"Nothing

Despite whatever virtues, the API thus far has some fragility arising from various scenarios:

• pattern malformation such as mismatched parentheses (runtime error)

• out-of-bounds indexing of a capture group list (runtime error)

• out-of-bounds Lens.Micro.ixing of a Lens.Micro.Traversal' target (spurious failure to match)

• case expression containing a Haskell list pattern of the wrong length (spurious failure to match)

• regex created and discarded inline (suboptimal performance)

• precariously many backslashes in a pattern. Matching a literal backslash requires the sequence "\\\\"!

Using a combination of language extensions and pattern introspection features, we provide a Template Haskell API to mitigate these scenarios. To make use of it these must be enabled:

The inspiration for this portion of the library is Ruby, which supports regular expressions with superior ergonomics.