This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
How Layout Works
What layout-text does, step by step. You do not need this to use
cl-textrender; it is here to explain results that surprise, and for anyone
changing the code. The pipeline lives in src/layout.lisp, hit testing in
src/hit-test.lisp.
Libraries
cl-textrender does no Unicode or font work of its own. It asks:
- ICU, through cl-icu, for paragraphs and bidi runs (UAX #9), script runs, grapheme boundaries (UAX #29), line break opportunities (UAX #14), and character properties: bidi class, general category, white space, default-ignorable, and vertical orientation (UTR #50).
- HarfBuzz, through cl-harfbuzz, for shaping, the character map, glyph advances, font metrics and glyph outlines.
1. Styles
Every character gets a style: a font fallback list, a feature list and
data. The base style comes from the font argument. Spans are checked, then
applied in order, each overriding only the keys it gives. Characters whose
styles are equal, compared with equal, share one style object, so equal
styles never split a run.
2. Paragraphs
ICU splits the text into paragraphs at characters of bidi class B and
resolves each paragraph's base direction, from :direction or its first
strong character. The separator is cut off the paragraph's content but
stays in its range. A text ending in a separator gets a final empty
paragraph. In vertical text, every paragraph's direction is left to right:
columns read top to bottom.
For each paragraph:
- Graphemes. ICU finds its grapheme boundaries.
- Fonts. Each grapheme takes a font from its style's fallback list: the previous grapheme's font, if it is in the list and has glyphs for every character of this grapheme that needs one; otherwise the first font in the list that does; otherwise the list's first font. Default-ignorable and control characters need no glyph.
- Orientation. In vertical text, each character is upright, unless its Vertical_Orientation is R, rotated.
- Scripts. ICU finds its script runs.
3. Items and measuring
The paragraph is cut into items: ICU's bidi runs, cut again at script runs and wherever the font, style or orientation changes. Each item is one HarfBuzz shaping call. The whole paragraph's text goes to HarfBuzz with the item's range, so shaping sees the context around the item (Arabic joining across a style change, for one). Upright vertical items are shaped top to bottom; the rest in their bidi direction. Common, Inherited and Unknown script runs let HarfBuzz infer the script.
Shaping happens at the font's own scale, in font units, without hinting. Advances and offsets are scaled to layout units afterwards, in double-floats.
To measure for line breaking, each glyph's advance is added to the first character of its cluster, giving every character a width.
4. Line breaking
Lines are filled greedily, at ICU's break opportunities. A line's width is the sum of its characters' widths up to its last non-whitespace character, so trailing whitespace hangs; tabs are expanded to their stops as they are measured. When the text up to the next opportunity does not fit:
- break at the last opportunity that fit, if there was one;
- otherwise break at the last grapheme boundary that fits;
- otherwise break after the first grapheme, which gets a line to itself.
A mandatory break (U+2028, or the end of the paragraph) always ends a line,
and is remembered as hard for justification. With :wrap nil, a line
breaks only there.
5. Planning lines
Each line is then shaped on its own:
- ICU reorders the line's bidi runs into visual order, for this line alone.
- Its items are shaped again, still with the whole paragraph as context.
- Tab glyphs become the space glyph, with their advance stretched to the next stop, measured from the line's start edge.
- The line's metrics are the largest ascent, descent and line gap of the fonts its items use.
With :max-lines, planning stops once there are enough lines, and the last
line is replaced by a version cut at a grapheme boundary and ended with the
ellipsis. The ellipsis takes the style of the last character kept, and each
of its graphemes takes a font from that style's fallback list by the same
rule as text, preferring the last character's font; each font's piece is an
item of its own. With
:align :justify and a width, each eligible line's spaces are widened
equally.
With :wrap nil, a line longer than the width is cut short. The cut is
first estimated from the advances the paragraph was shaped with: the last
grapheme boundary where the text and the ellipsis fit. The line cut there is
then shaped and measured, and the cut moves back a grapheme at a time while
it does not fit, so a long line is shaped a few times rather than once for
every grapheme it loses. The result is the cut that trying every boundary
from the line's end would find.
6. Placing
Lines are stacked along the block axis. Each line box is
line-spacing × (ascent + descent + line gap) tall, with the extra beyond
ascent and descent split above and below; columns in vertical text are
placed from the right. Alignment offsets each line's content within the
width, and trailing whitespace is placed past the end edge.
Glyphs are then placed one after another along the inline axis, by their advances, with HarfBuzz's offsets applied:
- horizontal glyphs at the pen, on the line's baseline;
- upright vertical glyphs on the column's central baseline;
- rotated vertical glyphs on a baseline that centers their em box in the column, with their offsets turned with them.
Every y is converted to the layout's convention, and positions are handed out as single-floats.
7. Caret data
While placing glyphs, each line records, for every caret stop on it, the stretch of the line its grapheme occupies and its direction:
- A cluster's extent runs from its first glyph's start to its last glyph's end.
- A cluster that covers several graphemes, a ligature, is divided equally between them, in reading order. In right-to-left runs the division is mirrored.
- An ellipsis covers no text and records nothing.
The layout also keeps a bit vector over the text marking every grapheme boundary of every paragraph, separators included, so CR LF is one step.
Hit testing works from these tables alone, in an inline coordinate along
the line and a block coordinate across the lines, and converts to and from
the layout's axes at its edges. caret-position reads the leading edge of
the grapheme at an index; hit-test finds the line by its block extent and
the nearest grapheme by its inline extent; selection-rectangles merges the
extents of the selected graphemes into contiguous stretches; move-caret
sorts a line's caret stops by position to step visually.