Streaming.ByteString

A space-efficient representation of a succession of Word8 vectors, supporting many efficient operations.

An effectful ByteStream contains 8-bit bytes, or by using the operations from Streaming.ByteString.Char8 it can be interpreted as containing 8-bit characters.

A type alias for back-compatibility.

O(1) The empty ByteStream -- i.e. return () Note that ByteStream m w is generally a monoid for monoidal values of w, like ().

O(1) Yield a Word8 as a minimal ByteStream.

O(n) Convert a monadic stream of individual Word8s into a packed byte stream.

O(n) Converts a packed byte stream into a stream of individual bytes.

O(c) Transmute a pseudo-pure lazy bytestring to its representation as a monadic stream of chunks.

>>> Q.putStrLn $ Q.fromLazy "hi"hi>>> Q.fromLazy "hi"Chunk "hi" (Empty (()))  -- note: a 'show' instance works in the identity monad>>> Q.fromLazy $ BL.fromChunks ["here", "are", "some", "chunks"]Chunk "here" (Chunk "are" (Chunk "some" (Chunk "chunks" (Empty (())))))

O(n) Convert an effectful byte stream into a single lazy ByteString with the same internal chunk structure, retaining the original return value.

This is the canonical way of breaking streaming (toStrict and the like are far more demonic). Essentially one is dividing the interleaved layers of effects and bytes into one immense layer of effects, followed by the memory of the succession of bytes.

Because one preserves the return value, toLazy is a suitable argument for mapped:

S.mapped Q.toLazy :: Stream (ByteStream m) m r -> Stream (Of L.ByteString) m r

>>> Q.toLazy "hello""hello" :> ()>>> S.toListM $ traverses Q.toLazy $ Q.lines "one\ntwo\nthree\nfour\nfive\n"["one","two","three","four","five",""]  -- [L.ByteString]

O(n) Convert an effectful byte stream into a single lazy ByteStream with the same internal chunk structure. See toLazy which preserve connectedness by keeping the return value of the effectful bytestring.

O(c) Convert a monadic stream of individual strict ByteString chunks into a byte stream.

O(c) Convert a byte stream into a stream of individual strict bytestrings. This of course exposes the internal chunk structure.

O(1) Yield a strict ByteString chunk.

O(n) Convert a monadic byte stream into a single strict ByteString, retaining the return value of the original pair. This operation is for use with mapped.

mapped R.toStrict :: Monad m => Stream (ByteStream m) m r -> Stream (Of ByteString) m r

It is subject to all the objections one makes to Data.ByteString.Lazy toStrict; all of these are devastating.

O(n) Convert a byte stream into a single strict ByteString.

Note that this is an expensive operation that forces the whole monadic ByteString into memory and then copies all the data. If possible, try to avoid converting back and forth between streaming and strict bytestrings.

Perform the effects contained in an effectful bytestring, ignoring the bytes.

Make the information in a bytestring available to more than one eliminating fold, e.g.

>>> Q.count 'l' $ Q.count 'o' $ Q.copy $ "hello\nworld"3 :> (2 :> ())

>>> Q.length $ Q.count 'l' $ Q.count 'o' $ Q.copy $ Q.copy "hello\nworld"11 :> (3 :> (2 :> ()))

>>> runResourceT $ Q.writeFile "hello2.txt" $ Q.writeFile "hello1.txt" $ Q.copy $ "hello\nworld\n">>> :! cat hello2.txthelloworld>>> :! cat hello1.txthelloworld

This sort of manipulation could as well be acheived by combining folds - using Control.Foldl for example. But any sort of manipulation can be involved in the fold. Here are a couple of trivial complications involving splitting by lines:

>>> let doubleLines = Q.unlines . maps (<* Q.chunk "\n" ) . Q.lines>>> let emphasize = Q.unlines . maps (<* Q.chunk "!" ) . Q.lines>>> runResourceT $ Q.writeFile "hello2.txt" $ emphasize $ Q.writeFile "hello1.txt" $ doubleLines $ Q.copy $ "hello\nworld">>> :! cat hello2.txthello!world!>>> :! cat hello1.txthelloworld

As with the parallel operations in Streaming.Prelude, we have

Q.effects . Q.copy       = id
hoist Q.effects . Q.copy = id

The duplication does not by itself involve the copying of bytestring chunks; it just makes two references to each chunk as it arises. This does, however double the number of constructors associated with each chunk.

Perform the effects contained in the second in an effectful pair of bytestrings, ignoring the bytes. It would typically be used at the type

ByteStream m (ByteStream m r) -> ByteStream m r

Reconceive an effect that results in an effectful bytestring as an effectful bytestring. Compare Streaming.mwrap. The closest equivalent of

>>> Streaming.wrap :: f (Stream f m r) -> Stream f m r

is here consChunk. mwrap is the smart constructor for the internal Go constructor.

O(n) map f xs is the ByteStream obtained by applying f to each element of xs.

for xs f applies f to each chunk in the stream, and concatenates the resulting streams.

Generalised in 0.2.4 to match streaming: the callback's (ignored) return value can be of any type.

O(n) The intercalate function takes a ByteStream and a list of ByteStreams and concatenates the list after interspersing the first argument between each element of the list.

The intersperse function takes a Word8 and a ByteStream and `intersperses' that byte between the elements of the ByteStream. It is analogous to the intersperse function on Streams.

O(1) cons is analogous to (:) for lists.

O(1) Unlike cons, 'cons'' is strict in the ByteString that we are consing onto. More precisely, it forces the head and the first chunk. It does this because, for space efficiency, it may coalesce the new byte onto the first 'chunk' rather than starting a new 'chunk'.

So that means you can't use a lazy recursive contruction like this:

let xs = cons\' c xs in xs

You can however use cons, as well as repeat and cycle, to build infinite byte streams.

O(n/c) Append a byte to the end of a ByteStream.

O(n/c) Append two ByteStrings together.

O(n) filter, applied to a predicate and a ByteStream, returns a ByteStream containing those characters that satisfy the predicate.

O(1) Extract the head and tail of a ByteStream, or its return value if it is empty. This is the 'natural' uncons for an effectful byte stream.

The same as uncons, will be removed in the next version.

Take a builder constructed otherwise and convert it to a genuine streaming bytestring.

>>> Q.putStrLn $ Q.toStreamingByteString $ stringUtf8 "哈斯克尔" <> stringUtf8 " " <> integerDec 98哈斯克尔 98

This benchmark shows its performance is indistinguishable from toLazyByteString

Take a builder and convert it to a genuine streaming bytestring, using a specific allocation strategy.

A simple construction of a builder from a ByteString.

>>> let aaa = "10000 is a number\n" :: Q.ByteString IO ()>>>  hPutBuilder  IO.stdout $ toBuilder  aaa10000 is a number

Concatenate a stream of builders (not a streaming bytestring!) into a single builder.

>>> let aa = yield (integerDec 10000) >> yield (string8 " is a number.") >> yield (char8 '\n')>>> hPutBuilder IO.stdout $ concatBuilders aa10000 is a number.

foldr, applied to a binary operator, a starting value (typically the right-identity of the operator), and a ByteStream, reduces the ByteStream using the binary operator, from right to left.

fold keeps the return value of the left-folded bytestring. Useful for simultaneous folds over a segmented bytestream.

fold_, applied to a binary operator, a starting value (typically the left-identity of the operator), and a ByteStream, reduces the ByteStream using the binary operator, from left to right. We use the style of the foldl library for left folds

O(c) Extract the first element of a ByteStream, if there is one. Suitable for use with mapped:

S.mapped Q.head :: Stream (Q.ByteStream m) m r -> Stream (Of (Maybe Word8)) m r

O(1) Extract the first element of a ByteStream, which must be non-empty.

Extract the last element of a ByteStream, if possible. Suitable for use with mapped:

S.mapped Q.last :: Streaming (ByteStream m) m r -> Stream (Of (Maybe Word8)) m r

O(n/c) Extract the last element of a ByteStream, which must be finite and non-empty.

O(n/c) length returns the length of a byte stream as an Int together with the return value. This makes various maps possible.

>>> Q.length "one\ntwo\three\nfour\nfive\n"23 :> ()>>> S.print $ S.take 3 $ mapped Q.length $ Q.lines "one\ntwo\three\nfour\nfive\n"384

Like length, report the length in bytes of the ByteStream by running through its contents. Since the return value is in the effect m, this is one way to "get out" of the stream.

Test whether a ByteStream is empty, collecting its return value; to reach the return value, this operation must check the whole length of the string.

>>> Q.null "one\ntwo\three\nfour\nfive\n"False :> ()>>> Q.null ""True :> ()>>> S.print $ mapped R.null $ Q.lines "yours,\nMeredith"FalseFalse

Suitable for use with mapped:

S.mapped Q.null :: Streaming (ByteStream m) m r -> Stream (Of Bool) m r

O(1) Test whether a ByteStream is empty. The value is of course in the monad of the effects.

>>> Q.null "one\ntwo\three\nfour\nfive\n"False>>> Q.null $ Q.take 0 Q.stdinTrue>>> :t Q.null $ Q.take 0 Q.stdinQ.null $ Q.take 0 Q.stdin :: MonadIO m => m Bool

O1 Distinguish empty from non-empty lines, while maintaining streaming; the empty ByteStrings are on the right

>>> nulls  ::  ByteStream m r -> m (Sum (ByteStream m) (ByteStream m) r)

There are many (generally slower) ways to remove null bytestrings from a Stream (ByteStream m) m r (besides using denull). If we pass next to

>>> mapped nulls bs :: Stream (Sum (ByteStream m) (ByteStream m)) m r

then can then apply Streaming.separate to get

>>> separate (mapped nulls bs) :: Stream (ByteStream m) (Stream (ByteStream m) m) r

The inner monad is now made of the empty bytestrings; we act on this with hoist , considering that

>>> :t Q.effects . Q.concatQ.effects . Q.concat  :: Monad m => Stream (Q.ByteStream m) m r -> m r

we have

>>> hoist (Q.effects . Q.concat) . separate . mapped Q.nulls  :: Monad n =>  Stream (Q.ByteStream n) n b -> Stream (Q.ByteStream n) n b

Similar to null, but yields the remainder of the ByteStream stream when an answer has been determined.

Returns the number of times its argument appears in the ByteStream. Suitable for use with mapped:

S.mapped (Q.count 37) :: Stream (Q.ByteStream m) m r -> Stream (Of Int) m r

Returns the number of times its argument appears in the ByteStream.

count = length . elemIndices

Like uncons, but yields the entire first ByteString chunk that the stream is holding onto. If there wasn't one, it tries to fetch it. Yields the final r return value when the ByteStream is empty.

The same as unconsChunk, will be removed in the next version.

Yield-style smart constructor for Chunk.

Consume the chunks of an effectful ByteString with a natural right fold.

Consume the chunks of an effectful ByteString with a left fold. Suitable for use with mapped.

chunkFold is preferable to foldlChunks since it is an appropriate argument for Control.Foldl.purely which permits many folds and sinks to be run simultaneously on one bytestream.

chunkFoldM is preferable to foldlChunksM since it is an appropriate argument for impurely which permits many folds and sinks to be run simultaneously on one bytestream.

Instead of mapping over each Word8 or Char, map over each strict ByteString chunk in the stream.

Like chunkMap, but map effectfully.

Like chunkMapM, but discard the result of each effectful mapping.

Resolve a succession of chunks into its Church encoding; this is not a safe operation; it is equivalent to exposing the constructors

Construct a succession of chunks from its Church encoding (compare GHC.Exts.build)

Given a byte stream on a transformed monad, make it possible to 'run' transformer.

Zip a list and a stream-of-byte-streams together.