Streaming.ByteString.Internal

A space-efficient representation of a succession of Word8 vectors, supporting many efficient operations.

An effectful ByteStream contains 8-bit bytes, or by using the operations from Streaming.ByteString.Char8 it can be interpreted as containing 8-bit characters.

A type alias for back-compatibility.

Smart constructor for Chunk.

The memory management overhead. Currently this is tuned for GHC only.

The chunk size used for I/O. Currently set to 32k, less the memory management overhead

Construct a succession of chunks from its Church encoding (compare GHC.Exts.build)

Resolve a succession of chunks into its Church encoding; this is not a safe operation; it is equivalent to exposing the constructors

Consume the chunks of an effectful ByteString with a natural right fold.

Consume the chunks of an effectful ByteString with a left fold. Suitable for use with mapped.

Consume the chunks of an effectful ByteString with a natural right monadic fold.

Like foldlChunks, but fold effectfully. Suitable for use with mapped.

chunkFold is preferable to foldlChunks since it is an appropriate argument for Control.Foldl.purely which permits many folds and sinks to be run simultaneously on one bytestream.

chunkFoldM is preferable to foldlChunksM since it is an appropriate argument for impurely which permits many folds and sinks to be run simultaneously on one bytestream.

Instead of mapping over each Word8 or Char, map over each strict ByteString chunk in the stream.

Like chunkMap, but map effectfully.

Like chunkMapM, but discard the result of each effectful mapping.

Given some continual monadic action that produces strict ByteString chunks, produce a stream of bytes.

Like unfoldMChunks, but feed through a final r return value.

Convert a vanilla Stream of characters into a stream of bytes.

Note: Each Char value is truncated to 8 bits.

Packing and unpacking from lists packBytes' :: Monad m => [Word8] -> ByteString m () packBytes' cs0 = packChunks 32 cs0 where packChunks n cs = case B.packUptoLenBytes n cs of (bs, []) -> Chunk bs (Empty ()) (bs, cs') -> Chunk bs (packChunks (min (n * 2) BI.smallChunkSize) cs') -- packUptoLenBytes :: Int -> [Word8] -> (ByteString, [Word8]) packUptoLenBytes len xs0 = accursedUnutterablePerformIO (createUptoN' len $ p -> go p len xs0) where go !_ !n [] = return (len-n, []) go !_ !0 xs = return (len, xs) go !p !n (x:xs) = poke p x >> go (p plusPtr 1) (n-1) xs createUptoN' :: Int -> (Ptr Word8 -> IO (Int, a)) -> IO (B.ByteString, a) createUptoN' l f = do fp <- B.mallocByteString l (l', res) withForeignPtr fp $ p - f p assert (l' <= l) $ return (B.PS fp 0 l', res) {-# INLINABLE packBytes' #-}

Convert a Stream of pure Word8 into a chunked ByteStream.

The reverse of packChars. Given a stream of bytes, produce a Stream individual bytes.

Yield-style smart constructor for Chunk.

The recommended chunk size. Currently set to 4k, less the memory management overhead

Reconceive an effect that results in an effectful bytestring as an effectful bytestring. Compare Streaming.mwrap. The closest equivalent of

>>> Streaming.wrap :: f (Stream f m r) -> Stream f m r

is here consChunk. mwrap is the smart constructor for the internal Go constructor.

Internal utility for unfoldr.

Stream chunks from something that contains m (Maybe ByteString) until it returns Nothing. reread is of particular use rendering io-streams input streams as byte streams in the present sense.

import qualified Data.ByteString as B
import qualified System.IO.Streams as S
Q.reread S.read            :: S.InputStream B.ByteString -> Q.ByteStream IO ()
Q.reread (liftIO . S.read) :: MonadIO m => S.InputStream B.ByteString -> Q.ByteStream m ()

The other direction here is

S.unfoldM Q.unconsChunk    :: Q.ByteString IO r -> IO (S.InputStream B.ByteString)

Copied from Data.ByteString.Unsafe for compatibility with older bytestring.

Copied from Data.ByteString.Unsafe for compatibility with older bytestring.

Make the information in a bytestring available to more than one eliminating fold, e.g.

>>> Q.count 'l' $ Q.count 'o' $ Q.copy $ "hello\nworld"3 :> (2 :> ())

>>> Q.length $ Q.count 'l' $ Q.count 'o' $ Q.copy $ Q.copy "hello\nworld"11 :> (3 :> (2 :> ()))

>>> runResourceT $ Q.writeFile "hello2.txt" $ Q.writeFile "hello1.txt" $ Q.copy $ "hello\nworld\n">>> :! cat hello2.txthelloworld>>> :! cat hello1.txthelloworld

This sort of manipulation could as well be acheived by combining folds - using Control.Foldl for example. But any sort of manipulation can be involved in the fold. Here are a couple of trivial complications involving splitting by lines:

>>> let doubleLines = Q.unlines . maps (<* Q.chunk "\n" ) . Q.lines>>> let emphasize = Q.unlines . maps (<* Q.chunk "!" ) . Q.lines>>> runResourceT $ Q.writeFile "hello2.txt" $ emphasize $ Q.writeFile "hello1.txt" $ doubleLines $ Q.copy $ "hello\nworld">>> :! cat hello2.txthello!world!>>> :! cat hello1.txthelloworld

As with the parallel operations in Streaming.Prelude, we have

Q.effects . Q.copy       = id
hoist Q.effects . Q.copy = id

The duplication does not by itself involve the copying of bytestring chunks; it just makes two references to each chunk as it arises. This does, however double the number of constructors associated with each chunk.

findIndexOrEnd is a variant of findIndex, that returns the length of the string if no element is found, rather than Nothing.

Like bracket, but specialized for ByteString.

This is similar to withForeignPtr but comes with an important caveat: the user must guarantee that the continuation does not diverge (e.g. loop or throw an exception). In exchange for this loss of generality, this function offers the ability of GHC to optimise more aggressively.

Specifically, applications of the form: unsafeWithForeignPtr fptr (forever something)

See GHC issue #17760 for more information about the unsoundness behavior that this function can result in.