cl-harfbuzz

• Usage
  • With cl-ft2
  • Color glyphs
  • Variable fonts
  • Lifetimes
  • Errors
• API overview
• Not covered
• Development
• License

Common Lisp bindings for HarfBuzz, the text shaping engine.

cl-harfbuzz comes in two layers, like cl-ft2:

• cl-harfbuzz: a Lisp-idiomatic API. OpenType tags, scripts and languages are strings ("kern", "Arab", "ar"). Directions and flags are keywords, and colors are lists (r g b a). Every failure signals a harfbuzz-error condition. The examples give it the local nickname hb.
• cl-harfbuzz/ffi: the raw layer underneath, on cl-ffi, named after the C API (hb_buffer_add_codepoints → hb-buffer-add-codepoints).

An optional system, cl-harfbuzz/ft2, builds HarfBuzz fonts on cl-ft2 faces, so shaping and rendering use the same FreeType data. Plain cl-harfbuzz does not load cl-ft2.

Only the libharfbuzz shared library (7.0 or newer) is needed at run time; if it cannot be loaded, loading cl-harfbuzz signals a harfbuzz-error that says what to install. Struct layouts are written by hand, so building needs no C compiler.

On Windows there is nothing to install: cl-harfbuzz carries its own native/windows-x64/harfbuzz.dll and loads it by its full path, so no other harfbuzz.dll on PATH gets in the way. An application saved from an image, without cl-harfbuzz's source tree, loads harfbuzz.dll by name instead: ship the DLL next to the executable, with cl-ft2's freetype.dll if it uses cl-harfbuzz/ft2. native/build-windows.sh rebuilds the DLL from source, reproducibly, and native/build-windows.sh --check verifies the committed one. That build includes FreeType support (hb-ft) but not glib, ICU, Graphite or the Windows shapers.

Usage

(asdf:load-system :cl-harfbuzz)

(defpackage #:example
  (:use #:cl)
  (:local-nicknames (#:hb #:cl-harfbuzz)))
(in-package #:example)

(hb:with-face (face "/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf")
  (hb:with-font (font face)                  ; scale = units per EM, so font units
    (hb:shape-text font "office")
    ;; => #(#S(SHAPED-GLYPH :INDEX 82   :CLUSTER 0 :X-ADVANCE 1253 ...)
    ;;      #S(SHAPED-GLYPH :INDEX 5044 :CLUSTER 1 ...)   ; the "ffi" ligature
    ;;      ...)
    (hb:shape-text font "office" :features '("-liga"))   ; no ligature
    (hb:shape-text font "سلام")              ; direction and script guessed
    (hb:glyph-outline font (hb:font-char-glyph font #\O))))

shape-text is the convenient form. For more control, use a buffer:

(hb:with-buffer (buffer)
  (hb:buffer-add-text buffer text :start 10 :end 20)   ; the rest is shaping context
  (setf (hb:buffer-direction buffer) :rtl
        (hb:buffer-script buffer) "Arab"
        (hb:buffer-language buffer) "ar")
  (hb:shape font buffer :features '("kern=0" "ss01"))
  (hb:buffer-glyphs buffer))

Each shaped-glyph has an index, the glyph's index in the font, and a cluster, the index into the text of the first character it came from. It also has advances and offsets with y up, and flags such as :unsafe-to-break. Positions and metrics are in the font's units: font units for a plain font, unless font-scale is set, and pixels for an ft2-font. Features use HarfBuzz's own string syntax: "-liga", "kern=0", "aalt=2", "smcp[3:5]".

Language-specific forms are used only when asked for: :language to shape-text, or buffer-language on a buffer, such as "sr" for Serbian letter shapes. Neither shape-text nor guess-buffer-properties takes the language from the process locale, so the result does not depend on the machine.

With cl-ft2

(asdf:load-system :cl-harfbuzz/ft2)

(defpackage #:example/ft2
  (:use #:cl)
  (:local-nicknames (#:hb #:cl-harfbuzz) (#:hb-ft2 #:cl-harfbuzz/ft2) (#:ft2 #:cl-ft2)))
(in-package #:example/ft2)

(ft2:with-library (lib)
  (ft2:with-face (face lib "/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf")
    (setf (ft2:face-size face) 32)
    (hb-ft2:with-ft2-font (font face)
      ;; Each glyph's pen position and bitmap.  Positions are in pixels, as
      ;; in cl-ft2.
      (let ((x 0))
        (loop for g across (hb:shape-text font "Hello")
              collect (list (+ x (hb:shaped-glyph-x-offset g))
                            (ft2:glyph-index-bitmap face (hb:shaped-glyph-index g)))
              do (incf x (hb:shaped-glyph-x-advance g)))))))
;; => ((0 #S(BITMAP :WIDTH 18 ...)) (385/16 #S(BITMAP ...)) (175/4 ...) ...)

An ft2-font is a hb:font that gets its glyph data from the cl-ft2 face:

• It works in pixels. Shaped positions, metrics, outlines and paint coordinates are pixels at the face's size, as exact rationals, the same units cl-ft2 uses.
• It follows the face. When the face's size, transform or variation coordinates change, the font picks the change up.
• It is destroyed with the face. destroy-face or destroy-library destroys it first, so it never points at a freed FT_Face.
• Variations go through cl-ft2. Setting font-design-coordinates, font-axis-coordinate or font-named-instance on an ft2-font changes the cl-ft2 face, so the two never disagree. Both libraries number named instances from 0.
• Shaping reuses the glyph slot. Shaping loads glyphs into the face's glyph slot. ft2:glyph-index-bitmap loads the glyph it renders, so it is unaffected. If you read the slot yourself, load the glyph after shaping.

Color glyphs

glyph-paint returns the paint operations for a glyph, in order: transforms, clips to glyph outlines, solid fills, linear, radial and sweep gradients, images, and compositing groups. It works for COLRv1, COLRv0, embedded PNG and SVG, and plain glyphs alike, and leaves rasterizing to a 2D graphics library. The COLR, CPAL, SVG and PNG data is also available directly: face-color-palettes, glyph-color-layers, glyph-svg, glyph-png.

Variable fonts

face-variation-axes and face-named-instances describe a font's axes. The variation is set with setf:

(setf (hb:font-axis-coordinate font "wght") 700)  ; one axis, by tag
(setf (hb:font-design-coordinates font) '(700 85)) ; every axis, in axis order
(setf (hb:font-design-coordinates font) '())       ; back to the named instance
(setf (hb:font-named-instance font) 2)             ; 0-based, or NIL for none

Values are clamped to each axis's range. Axes left off the end of a font-design-coordinates list take their default.

Likewise (setf (hb:font-scale font) 1024): font-scale and font-ppem read as two values (x and y) and are set from a number or an (x y) list.

Lifetimes

Faces, fonts and buffers are released explicitly with destroy-face, destroy-font and destroy-buffer, or by the with- macros. HarfBuzz reference counts its objects, so a font stays usable after the face it was made from is destroyed, and font-face still returns a usable face for it, which the font owns. Using a destroyed object signals harfbuzz-error. Faces and fonts can be shared between threads; buffers cannot.

Errors

(hb:with-buffer (buffer)
  (hb:buffer-add-text buffer "abc")
  (handler-case (hb:buffer-glyphs buffer)
    (hb:harfbuzz-error (e)
      (princ-to-string e))))
;; => "The buffer holds text, not glyphs: it has not been shaped.
;;     Shape it with SHAPE first, or shape text in one call with SHAPE-TEXT."

Every failure signals harfbuzz-error. Its report says what went wrong, then what to do; harfbuzz-error-message and harfbuzz-error-hint return the two parts. harfbuzz-error-function names the HarfBuzz function that failed, or is nil when cl-harfbuzz caught the problem itself.

An argument of the wrong type signals a standard type-error from the function it was passed to, such as a face where a font belongs or a keyword that is not one of the documented choices. The functions declare their argument types, so SBCL also warns at compile time about a mistake it can see, such as a negative glyph index.

API overview

Functions named face-* take a face, font-* a font, buffer-* a buffer, and glyph-* a font and a glyph index.

• Errors: harfbuzz-error, harfbuzz-error-message, harfbuzz-error-hint, harfbuzz-error-function.
• Types, which the functions declare and programs can check with: glyph-index, a glyph's index in its font, below 2^32; direction, :ltr, :rtl, :ttb or :btt; color, a list (r g b a) of octets; font-source, a filename, pathname or octet vector; cluster-level, :monotone-graphemes, :monotone-characters or :characters; buffer-flag, one of the keywords of buffer-flags; layout-table, :gsub or :gpos; metric, a keyword of font-metric; name-id, a name table ID or a standard name's keyword, such as :font-family; tag, an OpenType tag of one to four characters, such as "kern"; and script, an ISO 15924 code of four letters, such as "Latn".
• Library: harfbuzz-version, shapers, script-direction.
• Faces: make-face (from a filename, pathname or octet vector of a TrueType or OpenType font or collection; not WOFF or WOFF2), destroy-face, with-face, face-live-p, face-count, face-index, face-units-per-em, face-glyph-count, face-table-tags, face-table.
• Fonts: make-font, destroy-font, with-font, font-live-p, font-face, font-scale, font-ppem, font-point-size (all three setf-able), font-char-glyph, font-glyph-advance, font-glyph-extents (the ink box: xmin, ymin, xmax, ymax), font-glyph-name, font-glyph-from-name, font-extents, font-metric.
• Buffers and shaping: make-buffer, destroy-buffer, with-buffer, buffer-live-p, clear-buffer, reset-buffer, buffer-add-text, buffer-length, buffer-content-type, buffer-direction, buffer-script, buffer-language, guess-buffer-properties, buffer-cluster-level, buffer-flags, shape, buffer-glyphs, buffer-glyph, shape-text, shaped-glyph-*.
• Outlines and paint: glyph-outline, glyph-paint.
• OpenType data: face-name, face-name-entries, face-script-tags, face-feature-tags, face-variable-p, face-variation-axes (→ var-axis), face-named-instances (→ named-instance), font-design-coordinates, font-axis-coordinate, font-normalized-coordinates, font-named-instance (the first two and the last setf-able), face-color-palettes, face-color-layers-p, face-color-paint-p, face-svg-p, face-png-p, glyph-color-layers, glyph-color-paint-p, glyph-svg, glyph-png.
• Escape hatch: face-handle, font-handle, buffer-handle.
• cl-harfbuzz/ft2: make-ft2-font, with-ft2-font, ft2-font, ft2-font-face.

Not covered

• Subsetting (hb-subset). It is a separate library.
• Custom font functions, and HarfBuzz's own Unicode and AAT APIs.
• Bidi resolution, script itemization and line breaking. Use ICU to split text into runs before shaping each run.

Development

Dependencies are pinned in the stiftfile with stift, and tests use parachute.

stift test

The tests need libharfbuzz, libfreetype and DejaVu Sans (Debian/Ubuntu: libharfbuzz0b libfreetype6 fonts-dejavu-core; Windows: DejaVu Sans only). To use DejaVu Sans from another location, set CL_HARFBUZZ_TEST_FONT to its path. The color tests use tests/fonts/cl-harfbuzz-test-color.ttf, built by tests/fonts/make-test-color-font.py (needs fontTools). The variable-font tests use cl-ft2's test font.

License

MIT. See LICENSE.