HORIZON HASKELLDocslts/ghc-9.10.xc74966e2026-09-27Search names, modules, packages, or :: a typeCtrl K

GHC 9.10.3 · lts/ghc-9.10.x · c74966e · 2026-09-27

Moduleos-string-2.0.7Haskell2010

System.OsString.Data.ByteString.Short

A compact representation suitable for storing short byte strings in memory.

In typical use cases it can be imported alongside Data.ByteString, e.g.

import qualified Data.ByteString       as B
import qualified Data.ByteString.Short as B
         (ShortByteString, toShort, fromShort)

Other ShortByteString operations clash with Data.ByteString or Prelude functions however, so they should be imported qualified with a different alias e.g.

import qualified Data.ByteString.Short as B.Short
  • 1 type
  • 72 values
  • Packageos-string-2.0.7
  • Exports74
  • LanguageHaskell2010
  • LicenceBSD-3-Clause
  • SourceShort.hs

The ShortByteString type

2 declarations
newtypenewtype ShortByteString
#

A compact representation of a Word8 vector.

It has a lower memory overhead than a ByteString and does not contribute to heap fragmentation. It can be converted to or from a ByteString (at the cost of copying the string data). It supports very few other operations.

Instances14IsList, Eq, Data, Ord, Read, Show, …

Memory overhead

With GHC, the memory overheads are as follows, expressed in words and in bytes (words are 4 and 8 bytes on 32 or 64bit machines respectively).

  • B.ByteString unshared: 8 words; 32 or 64 bytes.

  • B.ByteString shared substring: 4 words; 16 or 32 bytes.

  • ShortByteString: 4 words; 16 or 32 bytes.

For the string data itself, both ShortByteString and B.ByteString use one byte per element, rounded up to the nearest word. For example, including the overheads, a length 10 ShortByteString would take 16 + 12 = 28 bytes on a 32bit platform and 32 + 16 = 48 bytes on a 64bit platform.

These overheads can all be reduced by 1 word (4 or 8 bytes) when the ShortByteString or B.ByteString is unpacked into another constructor.

For example:

data ThingId = ThingId {-# UNPACK #-} !Int
                       {-# UNPACK #-} !ShortByteString

This will take 1 + 1 + 3 words (the ThingId constructor + unpacked Int + unpacked ShortByteString), plus the words for the string data.

Heap fragmentation

With GHC, the B.ByteString representation uses pinned memory, meaning it cannot be moved by the GC. This is usually the right thing to do for larger strings, but for small strings using pinned memory can lead to heap fragmentation which wastes space. The ShortByteString type (and the Text type from the text package) use unpinned memory so they do not contribute to heap fragmentation. In addition, with GHC, small unpinned strings are allocated in the same way as normal heap allocations, rather than in a separate pinned area.

Introducing and eliminating ShortByteStrings

6 declarations

Basic interface

12 declarations

O(1) Extract the last element of a ShortByteString, which must be finite and non-empty. An exception will be thrown in the case of an empty ShortByteString.

This is a partial function, consider using unsnoc instead.

O(n) Extract the elements after the head of a ShortByteString, which must be non-empty. An exception will be thrown in the case of an empty ShortByteString.

This is a partial function, consider using uncons instead.

Note: copies the entire byte array

O(1) Extract the first element of a ShortByteString, which must be non-empty. An exception will be thrown in the case of an empty ShortByteString.

This is a partial function, consider using uncons instead.

Transforming ShortByteStrings

3 declarations

Reducing ShortByteStrings (folds)

8 declarations
valuefoldl :: (a -> Word8 -> a) -> a -> ShortByteString -> a
#

foldl, applied to a binary operator, a starting value (typically the left-identity of the operator), and a ShortByteString, reduces the ShortByteString using the binary operator, from left to right.

valuefoldr :: (Word8 -> a -> a) -> a -> ShortByteString -> a
#

foldr, applied to a binary operator, a starting value (typically the right-identity of the operator), and a ShortByteString, reduces the ShortByteString using the binary operator, from right to left.

Special folds

Generating and unfolding ByteStrings

valueunfoldr :: (a -> Maybe (Word8, a)) -> a -> ShortByteString
#

O(n), where n is the length of the result. The unfoldr function is analogous to the List 'unfoldr'. unfoldr builds a ShortByteString from a seed value. The function takes the element and returns Nothing if it is done producing the ShortByteString or returns Just (a,b), in which case, a is the next byte in the string, and b is the seed value for further production.

This function is not efficient/safe. It will build a list of [Word8] and run the generator until it returns Nothing, otherwise recurse infinitely, then finally create a ShortByteString.

If you know the maximum length, consider using unfoldrN.

Examples:

   unfoldr (\x -> if x <= 5 then Just (x, x + 1) else Nothing) 0
== pack [0, 1, 2, 3, 4, 5]

Substrings

0 declarations

Breaking strings

Similar to Prelude.dropWhile, drops the longest (possibly empty) prefix of elements satisfying the predicate and returns the remainder.

Note: copies the entire byte array

O(n) Break a ShortByteString into pieces separated by the byte argument, consuming the delimiter. I.e.

split 10  "a\nb\nd\ne" == ["a","b","d","e"]   -- fromEnum '\n' == 10
split 97  "aXaXaXa"    == ["","X","X","X",""] -- fromEnum 'a' == 97
split 120 "x"          == ["",""]             -- fromEnum 'x' == 120
split undefined ""     == []                  -- and not [""]

and

intercalate [c] . split c == id
split == splitWith . (==)

Note: copies the substrings

O(n) Splits a ShortByteString into components delimited by separators, where the predicate returns True for a separator element. The resulting components do not contain the separators. Two adjacent separators result in an empty component in the output. eg.

splitWith (==97) "aabbaca" == ["","","bb","c",""] -- fromEnum 'a' == 97
splitWith undefined ""     == []                  -- and not [""]

Predicates

3 declarations

Search for arbitrary substrings

valuebreakSubstring
  1. :: ShortByteString

    String to search for

  2. -> ShortByteString

    String to search in

  3. -> (ShortByteString, ShortByteString)

    Head and tail of string broken at substring

#

Break a string on a substring, returning a pair of the part of the string prior to the match, and the rest of the string.

The following relationships hold:

break (== c) l == breakSubstring (singleton c) l

For example, to tokenise a string, dropping delimiters:

tokenise x y = h : if null t then [] else tokenise x (drop (length x) t)
    where (h,t) = breakSubstring x y

To skip to the first occurrence of a string:

snd (breakSubstring x y)

To take the parts of a string before a delimiter:

fst (breakSubstring x y)

Note that calling `breakSubstring x` does some preprocessing work, so you should avoid unnecessarily duplicating breakSubstring calls with the same pattern.

Searching ShortByteStrings

0 declarations

Searching by equality

Searching with a predicate

valuefind :: (Word8 -> Bool) -> ShortByteString -> Maybe Word8
#

O(n) The find function takes a predicate and a ShortByteString, and returns the first element in matching the predicate, or Nothing if there is no such element.

find f p = case findIndex f p of Just n -> Just (p ! n) ; _ -> Nothing

Indexing ShortByteStrings

8 declarations

Low level conversions

0 declarations

Packing CStrings and pointers

O(n). Construct a new ShortByteString from a CString. The resulting ShortByteString is an immutable copy of the original CString, and is managed on the Haskell heap. The original CString must be null terminated.

O(n). Construct a new ShortByteString from a CStringLen. The resulting ShortByteString is an immutable copy of the original CStringLen. The ShortByteString is a normal Haskell value and will be managed on the Haskell heap.

Using ShortByteStrings as CStrings

valueuseAsCString :: ShortByteString -> (CString -> IO a) -> IO a
#

O(n) construction. Use a ShortByteString with a function requiring a null-terminated CString. The CString is a copy and will be freed automatically; it must not be stored or used after the subcomputation finishes.

valueuseAsCStringLen :: ShortByteString -> (CStringLen -> IO a) -> IO a
#

O(n) construction. Use a ShortByteString with a function requiring a CStringLen. As for useAsCString this function makes a copy of the original ShortByteString. It must not be stored or used after the subcomputation finishes.

Beware that this function does not add a terminating NUL byte at the end of CStringLen. If you need to construct a pointer to a null-terminated sequence, use useAsCString (and measure length independently if desired).