Streamly.Internal.Data.Parser

takeBetween m n takes a minimum of m and a maximum of n input elements and folds them using the supplied fold.

Stops after n elements. Fails if the stream ends before m elements could be taken.

Examples: -

>>> :{
  takeBetween' low high ls = Stream.parse prsr (Stream.fromList ls)
    where prsr = Parser.takeBetween low high Fold.toList
:}

>>> takeBetween' 2 4 [1, 2, 3, 4, 5]Right [1,2,3,4]

>>> takeBetween' 2 4 [1, 2]Right [1,2]

>>> takeBetween' 2 4 [1]Left (ParseError "takeBetween: Expecting alteast 2 elements, got 1")

>>> takeBetween' 0 0 [1, 2]Right []

>>> takeBetween' 0 1 []Right []

takeBetween is the most general take operation, other take operations can be defined in terms of takeBetween. For example:

>>> take n = Parser.takeBetween 0 n>>> takeEQ n = Parser.takeBetween n n>>> takeGE n = Parser.takeBetween n maxBound

Pre-release

Stops after taking exactly n input elements.

• Stops - after consuming n elements.

• Fails - if the stream or the collecting fold ends before it can collect exactly n elements.

>>> Stream.parse (Parser.takeEQ 2 Fold.toList) $ Stream.fromList [1,0,1]Right [1,0]

>>> Stream.parse (Parser.takeEQ 4 Fold.toList) $ Stream.fromList [1,0,1]Left (ParseError "takeEQ: Expecting exactly 4 elements, input terminated on 3")

Take at least n input elements, but can collect more.

• Stops - when the collecting fold stops.

• Fails - if the stream or the collecting fold ends before producing n elements.

>>> Stream.parse (Parser.takeGE 4 Fold.toList) $ Stream.fromList [1,0,1]Left (ParseError "takeGE: Expecting at least 4 elements, input terminated on 3")

>>> Stream.parse (Parser.takeGE 4 Fold.toList) $ Stream.fromList [1,0,1,0,1]Right [1,0,1,0,1]

Pre-release

Takes at-most n input elements.

• Stops - when the collecting parser stops.

• Fails - when the collecting parser fails.

>>> Stream.parse (Parser.takeP 4 (Parser.takeEQ 2 Fold.toList)) $ Stream.fromList [1, 2, 3, 4, 5]Right [1,2]

>>> Stream.parse (Parser.takeP 4 (Parser.takeEQ 5 Fold.toList)) $ Stream.fromList [1, 2, 3, 4, 5]Left (ParseError "takeEQ: Expecting exactly 5 elements, input terminated on 4")

Internal

Match the input sequence with the supplied list and return it if successful.

>>> listEq = Parser.listEqBy (==)

Match the given sequence of elements using the given comparison function. Returns the original sequence if successful.

Definition:

>>> listEqBy cmp xs = Parser.streamEqBy cmp (Stream.fromList xs) *> Parser.fromPure xs

Examples:

>>> Stream.parse (Parser.listEqBy (==) "string") $ Stream.fromList "string"Right "string"

>>> Stream.parse (Parser.listEqBy (==) "mismatch") $ Stream.fromList "match"Left (ParseError "streamEqBy: mismtach occurred")

Like listEqBy but uses a stream instead of a list and does not return the stream.

Match if the input stream is a subsequence of the argument stream i.e. all the elements of the input stream occur, in order, in the argument stream. The elements do not have to occur consecutively. A sequence is considered a subsequence of itself.

Collect stream elements until an element fails the predicate. The element on which the predicate fails is returned back to the input stream.

• Stops - when the predicate fails or the collecting fold stops.

• Fails - never.

>>> Stream.parse (Parser.takeWhile (== 0) Fold.toList) $ Stream.fromList [0,0,1,0,1]Right [0,0]

>>> takeWhile cond f = Parser.takeWhileP cond (Parser.fromFold f)

We can implement a breakOn using takeWhile:

breakOn p = takeWhile (not p)

Like takeWhile but uses a Parser instead of a Fold to collect the input. The combinator stops when the condition fails or if the collecting parser stops.

Other interesting parsers can be implemented in terms of this parser:

>>> takeWhile1 cond p = Parser.takeWhileP cond (Parser.takeBetween 1 maxBound p)>>> takeWhileBetween cond m n p = Parser.takeWhileP cond (Parser.takeBetween m n p)

Stops: when the condition fails or the collecting parser stops. Fails: when the collecting parser fails.

Pre-release

Like takeWhile but takes at least one element otherwise fails.

>>> takeWhile1 cond p = Parser.takeWhileP cond (Parser.takeBetween 1 maxBound p)

Drain the input as long as the predicate succeeds, running the effects and discarding the results.

This is also called skipWhile in some parsing libraries.

>>> dropWhile p = Parser.takeWhile p Fold.drain

takeEndBy cond parser parses a token that ends by a separator chosen by the supplied predicate. The separator is also taken with the token.

This can be combined with other parsers to implement other interesting parsers as follows:

>>> takeEndByLE cond n p = Parser.takeEndBy cond (Parser.fromFold $ Fold.take n p)>>> takeEndByBetween cond m n p = Parser.takeEndBy cond (Parser.takeBetween m n p)

>>> takeEndBy = Parser.takeEndByEsc (const False)

See also "Streamly.Data.Fold.takeEndBy". Unlike the fold, the collecting parser in the takeEndBy parser can decide whether to fail or not if the stream does not end with separator.

Pre-release

Like takeEndBy but the separator is dropped.

See also "Streamly.Data.Fold.takeEndBy_".

Pre-release

Like takeEndBy but the separator elements can be escaped using an escape char determined by the first predicate. The escape characters are removed.

pre-release

Parse a token that starts with an element chosen by the predicate. The parser fails if the input does not start with the selected element.

• Stops - when the predicate succeeds in non-leading position.

• Fails - when the predicate fails in the leading position.

>>> splitWithPrefix p f = Stream.parseMany (Parser.takeStartBy p f)

Examples: -

>>> p = Parser.takeStartBy (== ',') Fold.toList>>> leadingComma = Stream.parse p . Stream.fromList>>> leadingComma "a,b"Left (ParseError "takeStartBy: missing frame start")...>>> leadingComma ",,"Right ",">>> leadingComma ",a,b"Right ",a">>> leadingComma ""Right ""

Pre-release

Like takeStartBy but drops the separator.

>>> takeStartBy_ isBegin = Parser.takeFramedByGeneric Nothing (Just isBegin) Nothing

Take either the separator or the token. Separator is a Left value and token is Right value.

Unimplemented

Like splitOn but strips leading, trailing, and repeated separators. Therefore, ".a..b." having . as the separator would be parsed as ["a","b"]. In other words, its like parsing words from whitespace separated text.

• Stops - when it finds a word separator after a non-word element

• Fails - never.

>>> wordBy = Parser.wordFramedBy (const False) (const False) (const False)

S.wordsBy pred f = S.parseMany (PR.wordBy pred f)

Given an input stream [a,b,c,...] and a comparison function cmp, the parser assigns the element a to the first group, then if a `cmp` b is True b is also assigned to the same group. If a `cmp` c is True then c is also assigned to the same group and so on. When the comparison fails the parser is terminated. Each group is folded using the Fold f and the result of the fold is the result of the parser.

• Stops - when the comparison fails.

• Fails - never.

>>> :{ runGroupsBy eq =     Stream.fold Fold.toList         . Stream.parseMany (Parser.groupBy eq Fold.toList)         . Stream.fromList:}

>>> runGroupsBy (<) [][]

>>> runGroupsBy (<) [1][Right [1]]

>>> runGroupsBy (<) [3, 5, 4, 1, 2, 0][Right [3,5,4],Right [1,2],Right [0]]

Unlike groupBy this combinator performs a rolling comparison of two successive elements in the input stream. Assuming the input stream is [a,b,c,...] and the comparison function is cmp, the parser first assigns the element a to the first group, then if a `cmp` b is True b is also assigned to the same group. If b `cmp` c is True then c is also assigned to the same group and so on. When the comparison fails the parser is terminated. Each group is folded using the Fold f and the result of the fold is the result of the parser.

• Stops - when the comparison fails.

• Fails - never.

>>> :{ runGroupsByRolling eq =     Stream.fold Fold.toList         . Stream.parseMany (Parser.groupByRolling eq Fold.toList)         . Stream.fromList:}

>>> runGroupsByRolling (<) [][]

>>> runGroupsByRolling (<) [1][Right [1]]

>>> runGroupsByRolling (<) [3, 5, 4, 1, 2, 0][Right [3,5],Right [4],Right [1,2],Right [0]]

Pre-release

Like groupByRolling, but if the predicate is True then collects using the first fold as long as the predicate holds True, if the predicate is False collects using the second fold as long as it remains False. Returns Left for the first case and Right for the second case.

For example, if we want to detect sorted sequences in a stream, both ascending and descending cases we can use 'groupByRollingEither (<=) Fold.toList Fold.toList'.

Pre-release

Like wordBy but treats anything inside a pair of quotes as a single word, the quotes can be escaped by an escape character. Recursive quotes are possible if quote begin and end characters are different, quotes must be balanced. Outermost quotes are stripped.

>>> braces = Parser.wordFramedBy (== '\\') (== '{') (== '}') isSpace Fold.toList>>> Stream.parse braces $ Stream.fromList "{ab} cd"Right "ab">>> Stream.parse braces $ Stream.fromList "{ab}{cd}"Right "abcd">>> Stream.parse braces $ Stream.fromList "a{b} cd"Right "ab">>> Stream.parse braces $ Stream.fromList "a{{b}} cd"Right "a{b}"

>>> quotes = Parser.wordFramedBy (== '\\') (== '"') (== '"') isSpace Fold.toList>>> Stream.parse quotes $ Stream.fromList "\"a\"\"b\""Right "ab"

Quote and bracket aware word splitting with escaping. Like wordBy but word separators within specified quotes or brackets are ignored. Quotes and escape characters can be processed. If the end quote is different from the start quote it is called a bracket. The following quoting rules apply:

• In an unquoted string a character may be preceded by an escape character. The escape character is removed and the character following it is treated literally with no special meaning e.g. e.g. h e l l o is a single word, n is same as n.

• Any part of the word can be placed within quotes. Inside quotes all characters are treated literally with no special meaning. Quoting character itself cannot be used within quotes unless escape processing within quotes is applied to allow it.

• Optionally escape processing for quoted part can be specified. Escape character has no special meaning inside quotes unless it is followed by a character that has a escape translation specified, in that case the escape character is removed, and the specified translation is applied to the character following it. This can be used to escape the quoting character itself within quotes.

• There can be multiple quoting characters, when a quote starts, all other quoting characters within that quote lose any special meaning until the quote is closed.

• A starting quote char without an ending char generates a parse error. An ending bracket char without a corresponding bracket begin is ignored.

• Brackets can be nested.

We should note that unquoted and quoted escape processing are different. In unquoted part escape character is always removed. In quoted part it is removed only if followed by a special meaning character. This is consistent with how shell performs escape processing.

wordWithQuotes without processing the quotes and escape function supplied to escape the quote char within a quote. Can be used to parse words keeping the quotes and escapes intact.

>>> wordKeepQuotes = Parser.wordWithQuotes True (\_ _ -> Nothing)

wordWithQuotes with quote processing applied and escape function supplied to escape the quote char within a quote. Can be ysed to parse words and processing the quoting and escaping at the same time.

>>> wordProcessQuotes = Parser.wordWithQuotes False (\_ _ -> Nothing)

takeFramedBy_ isBegin isEnd fold parses a token framed by a begin and an end predicate.

>>> takeFramedBy_ = Parser.takeFramedByEsc_ (const False)

takeFramedByEsc_ isEsc isBegin isEnd fold parses a token framed using a begin and end predicate, and an escape character. The frame begin and end characters lose their special meaning if preceded by the escape character.

Nested frames are allowed if begin and end markers are different, nested frames must be balanced unless escaped, nested frame markers are emitted as it is.

For example,

>>> p = Parser.takeFramedByEsc_ (== '\\') (== '{') (== '}') Fold.toList>>> Stream.parse p $ Stream.fromList "{hello}"Right "hello">>> Stream.parse p $ Stream.fromList "{hello {world}}"Right "hello {world}">>> Stream.parse p $ Stream.fromList "{hello \\{world}"Right "hello {world">>> Stream.parse p $ Stream.fromList "{hello {world}"Left (ParseError "takeFramedByEsc_: missing frame end")

Pre-release

Parse a block enclosed within open, close brackets. Block contents may be quoted, brackets inside quotes are ignored. Quoting characters can be used within quotes if escaped. A block can have a nested block inside it.

Quote begin and end chars are the same. Block brackets and quote chars must not overlap. Block start and end brackets must be different for nesting blocks within blocks.

>>> p = Parser.blockWithQuotes (== '\\') (== '"') '{' '}' Fold.toList>>> Stream.parse p $ Stream.fromList "{msg: \"hello world\"}"Right "msg: \"hello world\""

span p f1 f2 composes folds f1 and f2 such that f1 consumes the input as long as the predicate p is True. f2 consumes the rest of the input.

> let span_ p xs = Stream.parse (Parser.span p Fold.toList Fold.toList) $ Stream.fromList xs

> span_ (< 1) 1,2,3

> span_ (< 2) 1,2,3

> span_ (< 4) 1,2,3

Pre-release

Break the input stream into two groups, the first group takes the input as long as the predicate applied to the first element of the stream and next input element holds True, the second group takes the rest of the input.

Pre-release

Like spanBy but applies the predicate in a rolling fashion i.e. predicate is applied to the previous and the next input elements.

Pre-release

sequence f p collects sequential parses of parsers in a serial stream p using the fold f. Fails if the input ends or any of the parsers fail.

Pre-release

count n f p collects exactly n sequential parses of parser p using the fold f. Fails if the input ends or the parser fails before n results are collected.

>>> count n = Parser.countBetween n n>>> count n p f = Parser.manyP p (Parser.takeEQ n f)

Unimplemented

countBetween m n f p collects between m and n sequential parses of parser p using the fold f. Stop after collecting n results. Fails if the input ends or the parser fails before m results are collected.

>>> countBetween m n p f = Parser.manyP p (Parser.takeBetween m n f)

Unimplemented

Like many but uses a Parser instead of a Fold to collect the results. Parsing stops or fails if the collecting parser stops or fails.

Unimplemented

Collect zero or more parses. Apply the supplied parser repeatedly on the input stream and push the parse results to a downstream fold.

Stops: when the downstream fold stops or the parser fails. Fails: never, produces zero or more results.

>>> many = Parser.countBetween 0 maxBound

Compare with many.

Collect one or more parses. Apply the supplied parser repeatedly on the input stream and push the parse results to a downstream fold.

Stops: when the downstream fold stops or the parser fails. Fails: if it stops without producing a single result.

>>> some p f = Parser.manyP p (Parser.takeGE 1 f)>>> some = Parser.countBetween 1 maxBound

Compare with some.

Apply two parsers alternately to an input stream. The input stream is considered an interleaving of two patterns. The two parsers represent the two patterns. Parsing starts at the first parser and stops at the first parser. It can be used to parse a infix style pattern e.g. p1 p2 p1 . Empty input or single parse of the first parser is accepted.

>>> p1 = Parser.takeWhile1 (not . (== '+')) Fold.toList>>> p2 = Parser.satisfy (== '+')>>> p = Parser.deintercalate p1 p2 Fold.toList>>> Stream.parse p $ Stream.fromList ""Right []>>> Stream.parse p $ Stream.fromList "1"Right [Left "1"]>>> Stream.parse p $ Stream.fromList "1+"Right [Left "1"]>>> Stream.parse p $ Stream.fromList "1+2+3"Right [Left "1",Right '+',Left "2",Right '+',Left "3"]

Apply two parsers alternately to an input stream. The input stream is considered an interleaving of two patterns. The two parsers represent the two patterns. Parsing starts at the first parser and stops at the first parser. It can be used to parse a infix style pattern e.g. p1 p2 p1 . Empty input or single parse of the first parser is accepted.

>>> p1 = Parser.takeWhile1 (not . (== '+')) Fold.toList>>> p2 = Parser.satisfy (== '+')>>> p = Parser.deintercalate1 p1 p2 Fold.toList>>> Stream.parse p $ Stream.fromList ""Left (ParseError "takeWhile1: end of input")>>> Stream.parse p $ Stream.fromList "1"Right [Left "1"]>>> Stream.parse p $ Stream.fromList "1+"Right [Left "1"]>>> Stream.parse p $ Stream.fromList "1+2+3"Right [Left "1",Right '+',Left "2",Right '+',Left "3"]

Like deintercalate but the entire input must satisfy the pattern otherwise the parser fails. This is many times faster than deintercalate.

>>> p1 = Parser.takeWhile1 (not . (== '+')) Fold.toList>>> p2 = Parser.satisfy (== '+')>>> p = Parser.deintercalateAll p1 p2 Fold.toList>>> Stream.parse p $ Stream.fromList ""Right []>>> Stream.parse p $ Stream.fromList "1"Right [Left "1"]>>> Stream.parse p $ Stream.fromList "1+"Left (ParseError "takeWhile1: end of input")>>> Stream.parse p $ Stream.fromList "1+2+3"Right [Left "1",Right '+',Left "2",Right '+',Left "3"]

Like sepBy but requires at least one successful parse.

Definition:

>>> sepBy1 p1 p2 f = Parser.deintercalate1 p1 p2 (Fold.catLefts f)

Examples:

>>> p1 = Parser.takeWhile1 (not . (== '+')) Fold.toList>>> p2 = Parser.satisfy (== '+')>>> p = Parser.sepBy1 p1 p2 Fold.toList>>> Stream.parse p $ Stream.fromList ""Left (ParseError "takeWhile1: end of input")>>> Stream.parse p $ Stream.fromList "1"Right ["1"]>>> Stream.parse p $ Stream.fromList "1+"Right ["1"]>>> Stream.parse p $ Stream.fromList "1+2+3"Right ["1","2","3"]

Apply two parsers alternately to an input stream. Parsing starts at the first parser and stops at the first parser. The output of the first parser is emiited and the output of the second parser is discarded. It can be used to parse a infix style pattern e.g. p1 p2 p1 . Empty input or single parse of the first parser is accepted.

Definitions:

>>> sepBy p1 p2 f = Parser.deintercalate p1 p2 (Fold.catLefts f)>>> sepBy p1 p2 f = Parser.sepBy1 p1 p2 f <|> Parser.fromEffect (Fold.extractM f)

Examples:

>>> p1 = Parser.takeWhile1 (not . (== '+')) Fold.toList>>> p2 = Parser.satisfy (== '+')>>> p = Parser.sepBy p1 p2 Fold.toList>>> Stream.parse p $ Stream.fromList ""Right []>>> Stream.parse p $ Stream.fromList "1"Right ["1"]>>> Stream.parse p $ Stream.fromList "1+"Right ["1"]>>> Stream.parse p $ Stream.fromList "1+2+3"Right ["1","2","3"]

Non-backtracking version of sepBy. Several times faster.

Like manyTill but uses a Parser to collect the results instead of a Fold. Parsing stops or fails if the collecting parser stops or fails.

We can implemnent parsers like the following using manyTillP:

countBetweenTill m n f p = manyTillP (takeBetween m n f) p

Unimplemented

manyTill chunking test f tries the parser test on the input, if test fails it backtracks and tries chunking, after chunking succeeds test is tried again and so on. The parser stops when test succeeds. The output of test is discarded and the output of chunking is accumulated by the supplied fold. The parser fails if chunking fails.

Stops when the fold f stops.

manyThen f collect recover repeats the parser collect on the input and collects the output in the supplied fold. If the the parser collect fails, parser recover is run until it stops and then we start repeating the parser collect again. The parser fails if the recovery parser fails.

For example, this can be used to find a key frame in a video stream after an error.

Unimplemented

Apply a collection of parsers to an input stream in a round robin fashion. Each parser is applied until it stops and then we repeat starting with the the first parser again.

Unimplemented

Keep trying a parser up to a maximum of n failures. When the parser fails the input consumed till now is dropped and the new instance is tried on the fresh input.

Unimplemented

Like retryMaxTotal but aborts after n successive failures.

Unimplemented

Keep trying a parser until it succeeds. When the parser fails the input consumed till now is dropped and the new instance is tried on the fresh input.

Unimplemented

Zip the input of a fold with a stream.

Pre-release

Pair each element of a fold input with its index, starting from index 0.

Pre-release

makeIndexFilter indexer filter predicate generates a fold filtering function using a fold indexing function that attaches an index to each input element and a filtering function that filters using @(index, element) -> Bool) as predicate.

For example:

filterWithIndex = makeIndexFilter indexed filter
filterWithAbsTime = makeIndexFilter timestamped filter
filterWithRelTime = makeIndexFilter timeIndexed filter

Pre-release

sampleFromthen offset stride samples the element at offset index and then every element at strides of stride.

Pre-release