Convert a lazy ByteString into a Producer of strict ByteStrings
Modulepipes-bytestring-2.1.7Haskell2010
Pipes.ByteString
This module provides pipes utilities for "byte streams", which are
streams of strict ByteStrings chunks. Use byte streams to interact
with both Handles and lazy ByteStrings.
To stream to or from Handles, use fromHandle or toHandle. For example, the following program copies data from one file to another:
import Pipes
import qualified Pipes.ByteString as P
import System.IO
main =
withFile "inFile.txt" ReadMode $ \hIn ->
withFile "outFile.txt" WriteMode $ \hOut ->
runEffect $ P.fromHandle hIn >-> P.toHandle hOutYou can stream to and from stdin and stdout using the predefined stdin and stdout pipes, like in the following "echo" program:
main = runEffect $ P.stdin >-> P.stdoutYou can also translate pure lazy ByteStrings to and from pipes:
import qualified Data.ByteString.Lazy.Char8 as BL
main = runEffect $ P.fromLazy (BL.pack "Hello, world!\n") >-> P.stdoutIn addition, this module provides many functions equivalent to lazy ByteString functions so that you can transform or fold byte streams. For example, to stream only the first three lines of stdin to stdout you would write:
import Lens.Family (over)
import Pipes
import qualified Pipes.ByteString as PB
import Pipes.Group (takes)
main = runEffect $ over PB.lines (takes 3) PB.stdin >-> PB.stdoutThe above program will never bring more than one chunk (~ 32 KB) into memory, no matter how long the lines are.
Note that functions in this library are designed to operate on streams that are insensitive to chunk boundaries. This means that they may freely split chunks into smaller chunks and discard empty chunks. However, they will never concatenate chunks in order to provide strict upper bounds on memory usage.
- 4 types
- 69 values
- Packagepipes-bytestring-2.1.7
- Exports73
- LanguageHaskell2010
- LicenceBSD-3-Clause
- SourceByteString.hs
Producers
7 declarationsStream bytes from stdin
Convert a Handle into a byte stream using a default chunk size
Convert a handle into a byte stream using a maximum chunk size
hGetSome forwards input immediately as it becomes available, splitting the input into multiple chunks if it exceeds the maximum chunk size.
Convert a handle into a byte stream using a fixed chunk size
Similar to hGet except that it will never block waiting for data to become available.
Convert a handle into a byte stream using a fixed chunk size
hGet waits until exactly the requested number of bytes are available for each chunk.
Like hGet but with an extra parameter specifying an initial handle offset
Servers
2 declarationsLike hGetSome, except you can vary the maximum chunk size for each request
Like hGet, except you can vary the chunk size for each request
Consumers
2 declarationsConvert a byte stream into a Handle
p >-> toHandle handle = for p (liftIO . hPutStr handle)Pipes
8 declarationsApply a transformation to each Word8 in the stream
Map a function over the byte stream and concatenate the results
(take n) only allows n bytes to pass
Take bytes until they fail the predicate
Only allows Word8s to pass if they satisfy the predicate
Stream all indices whose elements match the given Word8
Stream all indices whose elements satisfy the given predicate
Strict left scan over the bytes
Folds
19 declarationsFold a pure Producer of strict ByteStrings into a lazy ByteString
Fold an effectful Producer of strict ByteStrings into a lazy ByteString
Note: toLazyM is not an idiomatic use of pipes, but I provide it for
simple testing purposes. Idiomatic pipes style consumes the chunks
immediately as they are generated instead of loading them all into memory.
Fold an effectful Producer of strict ByteStrings into a lazy ByteString alongside the return value
Note: toLazyM' is not an idiomatic use of pipes, but I provide it for
simple testing purposes. Idiomatic pipes style consumes the chunks
immediately as they are generated instead of loading them all into memory.
Reduce the stream of bytes using a strict left fold
Note: It's more efficient to use folds from Control.Foldl.ByteString in
conjunction with Pipes.Prelude.fold when possible
Retrieve the first Word8
Retrieve the last Word8
Determine if the stream is empty
Count the number of bytes
Fold that returns whether M.Any received Word8s satisfy the predicate
Fold that returns whether M.All received Word8s satisfy the predicate
Return the maximum Word8 within a byte stream
Return the minimum Word8 within a byte stream
Determine whether any element in the byte stream matches the given Word8
Determine whether all elements in the byte stream do not match the given Word8
Find the first element in the stream that matches the predicate
Index into a byte stream
Find the index of an element that matches the given Word8
Store the first index of an element that satisfies the predicate
Store a tally of how many elements match the given Word8
Parsing
5 declarationsThe following parsing utilities are single-byte analogs of the ones found
in pipes-parse.
peekByte checks the first Word8 in the stream, but uses unDrawByte to push the Word8 back
peekByte = do
x <- drawByte
case x of
Nothing -> return ()
Just w8 -> unDrawByte w8
return xCheck if the underlying Producer has no more bytes
Note that this will skip over empty ByteString chunks, unlike
isEndOfInput from pipes-parse.
isEndOfBytes = liftM isNothing peekByteParsing Lenses
8 declarationsImproper lens that splits a Producer after the given number of bytes
Improper lens that splits after the longest consecutive group of bytes that satisfy the given predicate
Improper lens that splits after the longest consecutive group of bytes that fail the given predicate
Improper lens that splits at the first occurrence of the pattern.
Improper lens that splits after the first group of matching bytes, as defined by the given equality predicate
group :: Monad m => Lens' (Producer ByteString m x) (Producer ByteString m (Producer ByteString m x))word :: Monad m => Lens' (Producer ByteString m x) (Producer ByteString m (Producer ByteString m x))line :: Monad m => Lens' (Producer ByteString m x) (Producer ByteString m (Producer ByteString m x))Improper lens that splits a Producer after the first line
Unlike lines, this does not consume the newline marker, which is stored within the inner Producer
Note: This function is purely for demonstration purposes since it assumes a
particular encoding. You should prefer the Text equivalent of
this function from the pipes-text library.
Transforming Byte Streams
6 declarations(drop n) drops the first n bytes
Drop bytes until they fail the predicate
Intersperse a Word8 in between the bytes of the byte stream
Improper lens from unpacked Word8s to packaged ByteStrings
Improper lens from packed ByteStrings to unpacked Word8s
Group byte stream chunks into chunks of fixed length
Note: This is the only function in this API that concatenates ByteString chunks, which requires allocating new ByteStrings
FreeT Transformations
10 declarationsSplit a byte stream into FreeT-delimited byte streams of fixed size
Split a byte stream into groups separated by bytes that satisfy the predicate
Split a byte stream into groups separated by the given byte
Split a byte stream into groups separated by the given ByteString
Isomorphism between a byte stream and groups of identical bytes using the supplied equality predicate
Improper lens between a bytestream and its lines
Note: This function is purely for demonstration purposes since it assumes a
particular encoding. You should prefer the Text equivalent of
this function from the pipes-text library.
Improper lens between lines and a bytestream
Note: This function is purely for demonstration purposes since it assumes a
particular encoding. You should prefer the Text equivalent of
this function from the pipes-text library.
Convert a bytestream to delimited words
Note: This function is purely for demonstration purposes since it assumes a
particular encoding. You should prefer the Text equivalent of
this function from the pipes-text library.
Convert delimited words back to a byte stream
Note: This function is purely for demonstration purposes since it assumes a
particular encoding. You should prefer the Text equivalent of
this function from the pipes-text library.
Re-exports
6 declarationsData.ByteString re-exports the ByteString type.
Data.Word re-exports the Word8 type.
Pipes.Parse re-exports Parser.
Pipes.Group re-exports concats, intercalates, and FreeT
(the type).
A space-efficient representation of a Word8 vector, supporting many efficient operations.
A ByteString contains 8-bit bytes, or by using the operations from Data.ByteString.Char8 it can be interpreted as containing 8-bit characters.
Instances15IsList, Eq, Data, Ord, Read, Show, …
IsList ByteStringDefined in bytestring-0.12.2.0 · Data.ByteString.Internal.TypeEq ByteStringDefined in bytestring-0.12.2.0 · Data.ByteString.Internal.TypeData ByteStringDefined in bytestring-0.12.2.0 · Data.ByteString.Internal.TypeOrd ByteStringDefined in bytestring-0.12.2.0 · Data.ByteString.Internal.TypeRead ByteStringDefined in bytestring-0.12.2.0 · Data.ByteString.Internal.TypeShow ByteStringDefined in bytestring-0.12.2.0 · Data.ByteString.Internal.TypeIsString ByteStringDefined in bytestring-0.12.2.0 · Data.ByteString.Internal.TypeBeware: fromString truncates multi-byte characters to octets. e.g. "枯朶に烏のとまりけり秋の暮" becomes �6k�nh~�Q��n�
Semigroup ByteStringDefined in bytestring-0.12.2.0 · Data.ByteString.Internal.TypeMonoid ByteStringDefined in bytestring-0.12.2.0 · Data.ByteString.Internal.TypeNFData ByteStringDefined in bytestring-0.12.2.0 · Data.ByteString.Internal.TypeBinary ByteStringDefined in binary-0.8.9.3 · Data.Binary.ClassHashable ByteStringDefined in hashable-1.4.7.0 · Data.Hashable.ClassSubstitution ByteStringDefined in stringsearch-0.3.6.6 · Data.ByteString.Search.SubstitutionLift ByteStringDefined in bytestring-0.12.2.0 · Data.ByteString.Internal.Typetype Item ByteString = Word8Defined in bytestring-0.12.2.0 · Data.ByteString.Internal.Type
8-bit unsigned integer type
Instances22Bounded, Enum, Eq, Integral, Data, Num, …
Bounded Word8Defined in ghc-internal-9.1003.0 · GHC.Internal.WordEnum Word8Defined in ghc-internal-9.1003.0 · GHC.Internal.WordEq Word8Defined in ghc-internal-9.1003.0 · GHC.Internal.WordIntegral Word8Defined in ghc-internal-9.1003.0 · GHC.Internal.WordData Word8Defined in ghc-internal-9.1003.0 · GHC.Internal.Data.DataNum Word8Defined in ghc-internal-9.1003.0 · GHC.Internal.WordOrd Word8Defined in ghc-internal-9.1003.0 · GHC.Internal.WordRead Word8Defined in ghc-internal-9.1003.0 · GHC.Internal.ReadReal Word8Defined in ghc-internal-9.1003.0 · GHC.Internal.WordShow Word8Defined in ghc-internal-9.1003.0 · GHC.Internal.WordIx Word8Defined in ghc-internal-9.1003.0 · GHC.Internal.WordBits Word8Defined in ghc-internal-9.1003.0 · GHC.Internal.WordFiniteBits Word8Defined in ghc-internal-9.1003.0 · GHC.Internal.WordStorable Word8Defined in ghc-internal-9.1003.0 · GHC.Internal.Foreign.StorablePrintfArg Word8Defined in base-4.20.2.0 · Text.PrintfNFData Word8Defined in deepseq-1.5.0.0 · Control.DeepSeqBinary Word8Defined in binary-0.8.9.3 · Data.Binary.ClassHashable Word8Defined in hashable-1.4.7.0 · Data.Hashable.ClassLift Word8Defined in template-haskell-2.22.0.0 · Language.Haskell.TH.SyntaxIArray UArray Word8Defined in array-0.5.8.0 · Data.Array.BaseMArray IOUArray Word8 IODefined in array-0.5.8.0 · Data.Array.IO.InternalsMArray (STUArray s) Word8 (ST s)Defined in array-0.5.8.0 · Data.Array.Base
The "free monad transformer" for a functor f
Instances29MonadError, MonadReader, MonadState, MonadWriter, MonadBase, MonadFree, …
(Functor f, MonadError e m) => MonadError e (FreeT f m)Defined in free-5.2 · Control.Monad.Trans.Free(Functor f, MonadReader r m) => MonadReader r (FreeT f m)Defined in free-5.2 · Control.Monad.Trans.Free(Functor f, MonadState s m) => MonadState s (FreeT f m)Defined in free-5.2 · Control.Monad.Trans.Free(Functor f, MonadWriter w m) => MonadWriter w (FreeT f m)Defined in free-5.2 · Control.Monad.Trans.Free(Functor f, MonadBase b m) => MonadBase b (FreeT f m)Defined in free-5.2 · Control.Monad.Trans.Free(Functor f, Monad m) => MonadFree f (FreeT f m)Defined in free-5.2 · Control.Monad.Trans.FreeFunctor f => MonadTrans (FreeT f)Defined in free-5.2 · Control.Monad.Trans.Free(Functor f, Monad m) => Monad (FreeT f m)Defined in free-5.2 · Control.Monad.Trans.Free(Functor f, Functor m) => Functor (FreeT f m)Defined in free-5.2 · Control.Monad.Trans.Free(Functor f, MonadFail m) => MonadFail (FreeT f m)Defined in free-5.2 · Control.Monad.Trans.Free(Functor f, Monad m) => Applicative (FreeT f m)Defined in free-5.2 · Control.Monad.Trans.Free(Foldable m, Foldable f) => Foldable (FreeT f m)Defined in free-5.2 · Control.Monad.Trans.Free(Monad m, Traversable m, Traversable f) => Traversable (FreeT f m)Defined in free-5.2 · Control.Monad.Trans.Free(Functor f, MonadPlus m) => Alternative (FreeT f m)Defined in free-5.2 · Control.Monad.Trans.Free(Functor f, MonadPlus m) => MonadPlus (FreeT f m)Defined in free-5.2 · Control.Monad.Trans.Free(Functor f, MonadIO m) => MonadIO (FreeT f m)Defined in free-5.2 · Control.Monad.Trans.Free(Eq1 f, Eq1 m) => Eq1 (FreeT f m)Defined in free-5.2 · Control.Monad.Trans.Free(Ord1 f, Ord1 m) => Ord1 (FreeT f m)Defined in free-5.2 · Control.Monad.Trans.Free(Read1 f, Read1 m) => Read1 (FreeT f m)Defined in free-5.2 · Control.Monad.Trans.Free(Show1 f, Show1 m) => Show1 (FreeT f m)Defined in free-5.2 · Control.Monad.Trans.Free(Functor f, MonadCont m) => MonadCont (FreeT f m)Defined in free-5.2 · Control.Monad.Trans.Free(Functor f, MonadCatch m) => MonadCatch (FreeT f m)Defined in free-5.2 · Control.Monad.Trans.Free(Functor f, MonadThrow m) => MonadThrow (FreeT f m)Defined in free-5.2 · Control.Monad.Trans.Free(Functor f, Monad m) => Apply (FreeT f m)Defined in free-5.2 · Control.Monad.Trans.Free(Functor f, Monad m) => Bind (FreeT f m)Defined in free-5.2 · Control.Monad.Trans.Free(Eq1 f, Eq1 m, Eq a) => Eq (FreeT f m a)Defined in free-5.2 · Control.Monad.Trans.Free(Ord1 f, Ord1 m, Ord a) => Ord (FreeT f m a)Defined in free-5.2 · Control.Monad.Trans.Free(Read1 f, Read1 m, Read a) => Read (FreeT f m a)Defined in free-5.2 · Control.Monad.Trans.Free(Show1 f, Show1 m, Show a) => Show (FreeT f m a)Defined in free-5.2 · Control.Monad.Trans.Free