This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

Scripts

Every character belongs to a Unicode script (Latin, Arabic, Han, …) or is shared between scripts. cl-icu names scripts by their ISO 15924 codes, the form HarfBuzz's :script takes:

Code	Script
"Latn", "Cyrl", "Grek"	Latin, Cyrillic, Greek
"Arab", "Hebr"	Arabic, Hebrew
"Deva", "Thai"	Devanagari, Thai
"Hani", "Hira", "Kana", "Hang"	Han, Hiragana, Katakana, Hangul
"Zyyy"	Common: used with many scripts (spaces, digits, punctuation, most symbols and emoji)
"Zinh"	Inherited: takes the script of the base it attaches to (most combining marks, variation selectors, ZWJ)

char-script char → string

The script of char (a character or code point). The string is kept and given back for every character of that script, so asking again allocates nothing: do not change it.

(icu:char-script #\a)                  ; => "Latn"
(icu:char-script (code-char #x627))    ; => "Arab"
(icu:char-script #\1)                  ; => "Zyyy"
(icu:char-script (code-char #x301))    ; => "Zinh"   combining acute accent
(icu:char-script (code-char #x1F600))  ; => "Zyyy"   emoji are Common

char-script-extensions char → list of strings

Every script char is used with (its Script_Extensions property). Many Common and Inherited characters are in fact used with a few specific scripts:

(icu:char-script-extensions (code-char #x30FC))   ; ー  => ("Hira" "Kana")
(icu:char-script-extensions (code-char #x64B))    ; Arabic fathatan => ("Arab" "Syrc")
(icu:char-script-extensions (code-char #x3001))   ; 、 => ("Bopo" "Hani" "Hang" "Hira" "Kana" "Mong" "Yiii")
(icu:char-script-extensions #\a)                  ; => ("Latn")
(icu:char-script-extensions #\1)                  ; => ("Zyyy")

A character without extensions gives a one-element list of its script.

script-name script → string or nil

The long name of an ISO 15924 code, matched case-insensitively, or nil if ICU does not know it:

(icu:script-name "Arab")   ; => "Arabic"
(icu:script-name "Zyyy")   ; => "Common"
(icu:script-name "latn")   ; => "Latin"
(icu:script-name "Qaaa")   ; => NIL   (reserved for private use)

script-runs text &key (start 0) end → list of script-run

Splits text from start to end into runs of one script each, in logical order, covering the whole range. This is the unit a shaper works on.

Reader	Value
script-run-start, script-run-end	character indexes into text
script-run-script	an ISO 15924 code; "Zyyy" only if the whole run is Common characters

(icu:script-runs "Hello Мир 日本語")
;; => (#S(SCRIPT-RUN :START 0 :END 6 :SCRIPT "Latn")    ; "Hello "
;;     #S(SCRIPT-RUN :START 6 :END 10 :SCRIPT "Cyrl")   ; "Мир "
;;     #S(SCRIPT-RUN :START 10 :END 13 :SCRIPT "Hani")) ; "日本語"

The rules, which are those of ICU's own script run iterator (usc_nextRun):

• Common and Inherited characters join the run around them. Spaces, digits and punctuation stay with the preceding script; Common characters at the start take the script that follows; combining marks stay with their base.

  (icu:script-runs "(1) abc")        ; => one run, 0–7, "Latn"
  (icu:script-runs "123 !!")         ; => one run, 0–6, "Zyyy"
  (icu:script-runs "abc 😀 мир")     ; => "Latn" 0–6 (emoji included), "Cyrl" 6–9
  

• Paired brackets go together. A closing bracket takes the script of the run its opening bracket was in, not of the text just before it:

  (icu:script-runs "Hello мир (سلام) 123 ")
  ;; => (… "Latn" 0–6, "Cyrl" 6–11, "Arab" 11–15, "Cyrl" 15–21)
  ;;    the ")" at 15 returns to the Cyrillic run that "(" opened in
  

  Unclosed and stray brackets are harmless.

• Each script gets its own run. Japanese mixes Han, Hiragana and Katakana, and these become separate runs:

  (icu:script-runs "東京はーラーメン")
  ;; => (… "Hani" 0–2, "Hira" 2–4, "Kana" 4–8)
  

  Runs follow the Script property, not Script_Extensions: the Common ー at 3 joined the preceding Hiragana run. HarfBuzz shapes Japanese correctly either way.

An empty range gives (). With start and end, only that part of text is split, and positions stay indexes into text.

With cl-harfbuzz

Pass each run's script to hb:shape-text. Directional runs (from bidi-runs) and script runs are independent; intersect them to get pieces with one direction and one script, as the Cookbook does. Compute script-runs once per paragraph and reuse it for every line (see Core concepts).

A "Zyyy" run (text that is all digits, punctuation or symbols) can be passed as it is: HarfBuzz shapes Common text with its default shaper, which suits it.