HORIZON HASKELLDocslts/ghc-9.10.x248f8f02026-10-05Search names, modules, packages, or :: a typeCtrl K

GHC 9.10.3 · lts/ghc-9.10.x · 248f8f0 · 2026-10-05

Modulebytestring-0.12.2.0Haskell2010

Data.ByteString.Lazy

A time and space-efficient implementation of lazy byte vectors using lists of packed Word8 arrays, suitable for high performance use, both in terms of large data quantities, or high speed requirements. Lazy ByteStrings are encoded as lazy lists of strict chunks of bytes.

A key feature of lazy ByteStrings is the means to manipulate large or unbounded streams of data without requiring the entire sequence to be resident in memory. To take advantage of this you have to write your functions in a lazy streaming style, e.g. classic pipeline composition. The default I/O chunk size is 32k, which should be good in most circumstances.

Some operations, such as concat, append, reverse and cons, have better complexity than their Data.ByteString equivalents, due to optimisations resulting from the list spine structure. For other operations lazy ByteStrings are usually within a few percent of strict ones.

The recomended way to assemble lazy ByteStrings from smaller parts is to use the builder monoid from Data.ByteString.Builder.

This module is intended to be imported qualified, to avoid name clashes with Prelude functions. eg.

import qualified Data.ByteString.Lazy as B

Original GHC implementation by Bryan O'Sullivan. Rewritten to use UArray by Simon Marlow. Rewritten to support slices and use ForeignPtr by David Roundy. Rewritten again and extended by Don Stewart and Duncan Coutts. Lazy variant by Duncan Coutts and Don Stewart.

  • 2 types
  • 110 values

Lazy ByteString

2 declarations
datadata ByteString
#

A space-efficient representation of a Word8 vector, supporting many efficient operations.

A LazyByteString contains 8-bit bytes, or by using the operations from Data.ByteString.Lazy.Char8 it can be interpreted as containing 8-bit characters.

Instances12IsList, Eq, Data, Ord, Read, Show, …
  • IsList ByteStringDefined in bytestring-0.12.2.0 · Data.ByteString.Lazy.Internal
  • Eq ByteStringDefined in bytestring-0.12.2.0 · Data.ByteString.Lazy.Internal
  • Data ByteStringDefined in bytestring-0.12.2.0 · Data.ByteString.Lazy.Internal
  • Ord ByteStringDefined in bytestring-0.12.2.0 · Data.ByteString.Lazy.Internal
  • Read ByteStringDefined in bytestring-0.12.2.0 · Data.ByteString.Lazy.Internal
  • Show ByteStringDefined in bytestring-0.12.2.0 · Data.ByteString.Lazy.Internal
  • IsString ByteStringDefined in bytestring-0.12.2.0 · Data.ByteString.Lazy.Internal

    Beware: fromString truncates multi-byte characters to octets. e.g. "枯朶に烏のとまりけり秋の暮" becomes �6k�nh~�Q��n�

  • Semigroup ByteStringDefined in bytestring-0.12.2.0 · Data.ByteString.Lazy.Internal
  • Monoid ByteStringDefined in bytestring-0.12.2.0 · Data.ByteString.Lazy.Internal
  • NFData ByteStringDefined in bytestring-0.12.2.0 · Data.ByteString.Lazy.Internal
  • Lift ByteStringDefined in bytestring-0.12.2.0 · Data.ByteString.Lazy.Internal
  • type Item ByteString = Word8Defined in bytestring-0.12.2.0 · Data.ByteString.Lazy.Internal

Introducing and eliminating ByteStrings

10 declarations

Basic interface

12 declarations
valuecons' :: Word8 -> ByteString -> ByteString
#

O(1) Unlike cons, cons' is strict in the ByteString that we are consing onto. More precisely, it forces the head and the first chunk. It does this because, for space efficiency, it may coalesce the new byte onto the first 'chunk' rather than starting a new 'chunk'.

So that means you can't use a lazy recursive contruction like this:

let xs = cons' c xs in xs

You can however use cons, as well as repeat and cycle, to build infinite lazy ByteStrings.

valuelast :: HasCallStack => ByteString -> Word8
#

O(n/c) Extract the last element of a ByteString, which must be finite and non-empty.

This is a partial function, consider using unsnoc instead.

Transforming ByteStrings

5 declarations

Reducing ByteStrings (folds)

8 declarations
valuefoldl :: (a -> Word8 -> a) -> a -> ByteString -> a
#

foldl, applied to a binary operator, a starting value (typically the left-identity of the operator), and a ByteString, reduces the ByteString using the binary operator, from left to right.

valuefoldr :: (Word8 -> a -> a) -> a -> ByteString -> a
#

foldr, applied to a binary operator, a starting value (typically the right-identity of the operator), and a ByteString, reduces the ByteString using the binary operator, from right to left.

Special folds

Building ByteStrings

0 declarations

Scans

valuescanl
  1. :: (Word8 -> Word8 -> Word8)

    accumulator -> element -> new accumulator

  2. -> Word8

    starting value of accumulator

  3. -> ByteString

    input of length n

  4. -> ByteString

    output of length n+1

#

scanl is similar to foldl, but returns a list of successive reduced values from the left.

scanl f z [x1, x2, ...] == [z, z `f` x1, (z `f` x1) `f` x2, ...]

Note that

head (scanl f z xs) == z
last (scanl f z xs) == foldl f z xs
valuescanr
  1. :: (Word8 -> Word8 -> Word8)

    element -> accumulator -> new accumulator

  2. -> Word8

    starting value of accumulator

  3. -> ByteString

    input of length n

  4. -> ByteString

    output of length n+1

#

scanr is similar to foldr, but returns a list of successive reduced values from the right.

scanr f z [..., x{n-1}, xn] == [..., x{n-1} `f` (xn `f` z), xn `f` z, z]

Note that

head (scanr f z xs) == foldr f z xs
last (scanr f z xs) == z

Accumulating maps

valuemapAccumL
  1. :: acc -> Word8 -> (acc, Word8)
  2. -> acc
  3. -> ByteString
  4. -> (acc, ByteString)
#

The mapAccumL function behaves like a combination of map and foldl; it applies a function to each element of a ByteString, passing an accumulating parameter from left to right, and returning a final value of this accumulator together with the new ByteString.

valuemapAccumR
  1. :: acc -> Word8 -> (acc, Word8)
  2. -> acc
  3. -> ByteString
  4. -> (acc, ByteString)
#

The mapAccumR function behaves like a combination of map and foldr; it applies a function to each element of a ByteString, passing an accumulating parameter from right to left, and returning a final value of this accumulator together with the new ByteString.

Infinite ByteStrings

Unfolding ByteStrings

valueunfoldr :: (a -> Maybe (Word8, a)) -> a -> ByteString
#

O(n) The unfoldr function is analogous to the List 'unfoldr'. unfoldr builds a ByteString from a seed value. The function takes the element and returns Nothing if it is done producing the ByteString or returns Just (a,b), in which case, a is a prepending to the ByteString and b is used as the next element in a recursive call.

Substrings

0 declarations

Breaking strings

valuetakeEnd :: Int64 -> ByteString -> ByteString
#

O(c) takeEnd n xs is equivalent to drop (length xs - n) xs. Takes n elements from end of bytestring.

Example3 expressions
takeEnd 3 "abcdefg""efg"takeEnd 0 "abcdefg"""takeEnd 4 "abc""abc"
valuedropEnd :: Int64 -> ByteString -> ByteString
#

O(n) dropEnd n xs is equivalent to take (length xs - n) xs. Drops n elements from end of bytestring.

Example3 expressions
dropEnd 3 "abcdefg""abcd"dropEnd 0 "abcdefg""abcdefg"dropEnd 4 "abc"""
valuedropWhile :: (Word8 -> Bool) -> ByteString -> ByteString
#

Similar to Prelude.dropWhile, drops the longest (possibly empty) prefix of elements satisfying the predicate and returns the remainder.

valuespanEnd :: (Word8 -> Bool) -> ByteString -> (ByteString, ByteString)
#

Returns the longest (possibly empty) suffix of elements satisfying the predicate and the remainder of the string.

spanEnd p is equivalent to breakEnd (not . p) and to (dropWhileEnd p &&& takeWhileEnd p).

We have

spanEnd (not . isSpace) "x y z" == ("x y ", "z")

and

spanEnd (not . isSpace) ps
   ==
let (x, y) = span (not . isSpace) (reverse ps) in (reverse y, reverse x)
valuegroup :: ByteString -> [ByteString]
#

The group function takes a ByteString and returns a list of ByteStrings such that the concatenation of the result is equal to the argument. Moreover, each string in the result contains only equal elements. For example,

group "Mississippi" = ["M","i","ss","i","ss","i","pp","i"]

It is a special case of groupBy, which allows the programmer to supply their own equality test.

Breaking into many substrings

valuesplit :: Word8 -> ByteString -> [ByteString]
#

O(n) Break a ByteString into pieces separated by the byte argument, consuming the delimiter. I.e.

split 10  "a\nb\nd\ne" == ["a","b","d","e"]   -- fromEnum '\n' == 10
split 97  "aXaXaXa"    == ["","X","X","X",""] -- fromEnum 'a' == 97
split 120 "x"          == ["",""]             -- fromEnum 'x' == 120
split undefined ""     == []                  -- and not [""]

and

intercalate [c] . split c == id
split == splitWith . (==)

As for all splitting functions in this library, this function does not copy the substrings, it just constructs new ByteStrings that are slices of the original.

valuesplitWith :: (Word8 -> Bool) -> ByteString -> [ByteString]
#

O(n) Splits a ByteString into components delimited by separators, where the predicate returns True for a separator element. The resulting components do not contain the separators. Two adjacent separators result in an empty component in the output. eg.

splitWith (==97) "aabbaca" == ["","","bb","c",""] -- fromEnum 'a' == 97
splitWith undefined ""     == []                  -- and not [""]

Predicates

2 declarations

Search for arbitrary substrings

Searching ByteStrings

0 declarations

Searching by equality

Searching with a predicate

valuefind :: (Word8 -> Bool) -> ByteString -> Maybe Word8
#

O(n) The find function takes a predicate and a ByteString, and returns the first element in matching the predicate, or Nothing if there is no such element.

find f p = case findIndex f p of Just n -> Just (p ! n) ; _ -> Nothing
valuepartition :: (Word8 -> Bool) -> ByteString -> (ByteString, ByteString)
#

O(n) The partition function takes a predicate a ByteString and returns the pair of ByteStrings with elements which do and do not satisfy the predicate, respectively; i.e.,

partition p bs == (filter p xs, filter (not . p) xs)

Indexing ByteStrings

10 declarations
valueelemIndexEnd :: Word8 -> ByteString -> Maybe Int64
#

O(n) The elemIndexEnd function returns the last index of the element in the given ByteString which is equal to the query element, or Nothing if there is no such element. The following holds:

elemIndexEnd c xs = case elemIndex c (reverse xs) of
  Nothing -> Nothing
  Just i  -> Just (length xs - 1 - i)
valuecount :: Word8 -> ByteString -> Int64
#

count returns the number of times its argument appears in the ByteString

count = length . elemIndices

But more efficiently than using length on the intermediate list.

Zipping and unzipping ByteStrings

4 declarations
valuezip :: ByteString -> ByteString -> [(Word8, Word8)]
#

O(n) zip takes two ByteStrings and returns a list of corresponding pairs of bytes. If one input ByteString is short, excess elements of the longer ByteString are discarded. This is equivalent to a pair of unpack operations.

valuezipWith :: (Word8 -> Word8 -> a) -> ByteString -> ByteString -> [a]
#

zipWith generalises zip by zipping with the function given as the first argument, instead of a tupling function. For example, zipWith (+) is applied to two ByteStrings to produce the list of corresponding sums.

Ordered ByteStrings

0 declarations

Low level conversions

0 declarations

Copying ByteStrings

valuecopy :: ByteString -> ByteString
#

O(n) Make a copy of the ByteString with its own storage. This is mainly useful to allow the rest of the data pointed to by the ByteString to be garbage collected, for example if a large string has been read in, and only a small part of it is needed in the rest of the program.

I/O with ByteStrings

0 declarations

⚠ Using lazy I/O functions like readFile or hGetContents means that the order of operations such as closing the file handle is left at the discretion of the RTS. Hence, the developer can face some issues when:

  • The program reads a file and writes the same file. This means that the file may be locked because the handler has not been released when writeFile is executed.

  • The program reads thousands of files, but due to lazy evaluation, the OS's file descriptor limit is reached before the handlers can be released.

Why?

Consider the following program:

import qualified Data.ByteString.Lazy as BL
main = do
  _ <- BL.readFile "foo.txt"
  BL.writeFile "foo.txt" mempty

Generally, in the IO monad side effects happen sequentially and in full. Therefore, one might reasonably expect that reading the whole file via readFile executes all three actions (open the file handle, read its content, close the file handle) before control moves to the following writeFile action. This expectation holds for the strict Data.ByteString API. However, the above LazyByteString variant of the program fails with openBinaryFile: resource busy (file is locked).

The reason for this is that Data.ByteString.Lazy is specifically designed to handle large or unbounded streams of data incrementally, without requiring all the data to be resident in memory at the same time. Incremental processing would not be possible if readFile were to follow the usual rules of IO: evaluating all side effects would require reading the file in full and closing its handle before returning from readFile. This is why readFile (and hGetContents in general) is implemented via unsafeInterleaveIO, which allows IO side effects to be delayed and interleaved with subsequent processing of the return value. That's exactly what happens in the example above: readFile opens a file handle, but since the content is not fully consumed, the file handle remains open, allowing the content to read on demand (never in this case, since the return value is ignored). So when writeFile is executed next, foo.txt is still open for reading and the RTS takes care to avoid simultaneously opening it for writing, instead returning the error shown above.

How to enforce the order of effects?

If the content is small enough to fit in memory, consider using strict readFile, potentially applying fromStrict afterwards. E. g.,

import qualified Data.ByteString as BS
import qualified Data.ByteString.Lazy as BL
main = do
  _ <- BS.readFile "foo.txt"
  BL.writeFile "foo.txt" mempty

If you are dealing with large or unbounded data streams, consider reaching out for a specialised package, such as conduit, machines-bytestring, pipes-bytestring, streaming-bytestring, streamly-bytestring, etc.

Standard input and output

valueputStr :: ByteString -> IO ()
#

Write a ByteString to stdout.

The chunks will be written one at a time. Other threads might write to the stdout in between, and hence putStr alone is not suitable for concurrent writes.

valueinteract :: (ByteString -> ByteString) -> IO ()
#

The interact function takes a function of type ByteString -> ByteString as its argument. The entire input from the standard input device is passed to this function as its argument, and the resulting string is output on the standard output device.

Files

I/O with Handles

valuehGetContents :: Handle -> IO ByteString
#

Read entire handle contents lazily into a ByteString. Chunks are read on demand, using the default chunk size.

File handles are closed on EOF if all the file is read, or through garbage collection otherwise.

valuehGetNonBlocking :: Handle -> Int -> IO ByteString
#

hGetNonBlocking is similar to hGet, except that it will never block waiting for data to become available, instead it returns only whatever data is available. If there is no data available to be read, hGetNonBlocking returns empty.

Note: on Windows and with Haskell implementation other than GHC, this function does not work correctly; it behaves identically to hGet.

valuehPut :: Handle -> ByteString -> IO ()
#

Outputs a ByteString to the specified Handle.

The chunks will be written one at a time. Other threads might write to the Handle in between, and hence hPut alone is not suitable for concurrent writes.

valuehPutNonBlocking :: Handle -> ByteString -> IO ByteString
#

Similar to hPut except that it will never block. Instead it returns any tail that did not get written. This tail may be empty in the case that the whole string was written, or the whole original string if nothing was written. Partial writes are also possible.

Note: on Windows and with Haskell implementation other than GHC, this function does not work correctly; it behaves identically to hPut.