GHC.Parser.Annotation

Exact print annotations exist so that tools can perform source to source conversions of Haskell code. They are used to keep track of the various syntactic keywords that are not otherwise captured in the AST.

The wiki page describing this feature is https://gitlab.haskell.org/ghc/ghc/wikis/api-annotations https://gitlab.haskell.org/ghc/ghc/-/wikis/implementing-trees-that-grow/in-tree-api-annotations

Note: in general the names of these are taken from the corresponding token, unless otherwise noted See Note [exact print annotations] above for details of the usage

A token stored in the syntax tree. For example, when parsing a let-expression, we store EpToken "let" and EpToken "in". The locations of those tokens can be used to faithfully reproduce (exactprint) the original program text.

With UnicodeSyntax, there might be multiple ways to write the same token. For example an arrow could be either -> or →. This choice must be recorded in order to exactprint such tokens, so instead of EpToken "->" we introduce EpUniToken "->" "→".

Layout information for declarations.

Certain tokens can have alternate representations when unicode syntax is enabled. This flag is attached to those tokens in the lexer so that the original source representation can be reproduced in the corresponding EpAnnotation

Convert a normal annotation into its unicode equivalent one

Some template haskell tokens have two variants, one with an e the other not:

 [| or [e|
 [|| or [e||

This type indicates whether the e is present or not.

Captures an annotation, storing the AnnKeywordId and its location. The parser only ever inserts EpaLocation fields with a RealSrcSpan being the original location of the annotation in the source file. The EpaLocation can also store a delta position if the AST has been modified and needs to be pretty printed again. The usual way an AddEpAnn is created is using the mj ("make jump") function, and then it can be inserted into the appropriate annotation.

The anchor for an AnnKeywordId. The Parser inserts the EpaSpan variant, giving the exact location of the original item in the parsed source. This can be replaced by the EpaDelta version, to provide a position for the item relative to the end of the previous item in the source. This is useful when editing an AST prior to exact printing the changed one. The list of comments in the EpaDelta variant captures any comments between the prior output and the thing being marked here, since we cannot otherwise sort the relative order.

Used in the parser only, extract the RealSrcSpan from an EpaLocation. The parser will never insert a DeltaPos, so the partial function is safe.

Tokens embedded in the AST have an EpaLocation, unless they come from generated code (e.g. by TH).

Spacing between output items when exact printing. It captures the spacing from the current print position on the page to the position required for the thing about to be printed. This is either on the same line in which case is is simply the number of spaces to emit, or it is some number of lines down, with a given column offset. The exact printing algorithm keeps track of the column offset pertaining to the current anchor position, so the deltaColumn is the additional spaces to add in this case. See https://gitlab.haskell.org/ghc/ghc/wikis/api-annotations for details.

Smart constructor for a DeltaPos. It preserves the invariant that for the DifferentLine constructor deltaLine is always > 0.

The exact print annotations (EPAs) are kept in the HsSyn AST for the GhcPs phase. We do not always have EPAs though, only for code that has been parsed as they do not exist for generated code. This type captures that they may be missing.

A goal of the annotations is that an AST can be edited, including moving subtrees from one place to another, duplicating them, and so on. This means that each fragment must be self-contained. To this end, each annotated fragment keeps track of the anchor position it was originally captured at, being simply the start span of the topmost element of the ast fragment. This gives us a way to later re-calculate all Located items in this layer of the AST, as well as any annotations captured. The comments associated with the AST fragment are also captured here.

The ann type parameter allows this general structure to be specialised to the specific set of locations of original exact print annotation elements. So for HsLet we have

type instance XLet GhcPs = EpAnn AnnsLet data AnnsLet = AnnsLet { alLet :: EpaLocation, alIn :: EpaLocation } deriving Data

The spacing between the items under the scope of a given EpAnn is normally derived from the original Anchor. But if a sub-element is not in its original position, the required spacing can be directly captured in the anchor_op field of the entry Anchor. This allows us to freely move elements around, and stitch together new AST fragments out of old ones, and have them still printed out in a precise way.

An Anchor records the base location for the start of the syntactic element holding the annotations, and is used as the point of reference for calculating delta positions for contained annotations. It is also normally used as the reference point for the spacing of the element relative to its container. If the AST element is moved, that relationship is tracked in the anchor_op instead.