Development

Layout

cl-icu.asd             two systems: cl-icu and cl-icu/test
src/
  package.lisp         cl-icu/ffi and cl-icu packages
  conditions.lisp      icu-error, first, since loading ICU can signal it
  ffi-library.lisp     loading libicuuc, probing the symbol suffix, DEFICU, LOAD-ICU
  ffi-functions.lisp   the ICU functions and C constants
  error-codes.lisp     UErrorCode checking
  utf16.lisp           Lisp strings to and from UTF-16, offset mapping
  version.lisp         icu-version, unicode-version
  properties.lisp      char-property, names, mirroring
  bidi.lisp            bidi-paragraphs, bidi-levels, bidi-runs
  script.lisp          scripts and script-runs
  break.lisp           text-boundaries, graphemes, line-breaks, words, sentences
  transform.lisp       normalization and case mapping
tests/                 parachute tests

The two layers follow the other libraries of the stack: a raw /ffi package that transcribes C, and an idiomatic package on top. cl-icu has no implementation-specific code: that is cl-ffi's.

Dependencies

Pinned in the stiftfile with stift: cl-ffi, at a commit of its main branch, and Parachute, with the four libraries it needs, for the tests.

Running the tests

stift test

asdf:test-system signals an error if any test fails, so the command exits unsuccessfully.

The suite (cl-icu/test, parachute) has 37 tests and 323 checks. It covers versions and library loading, single return values, properties and names, argument types, bidi levels, runs, lines, paragraphs and visual order, script runs including bracket pairing, all four boundary types including Thai and Japanese dictionaries, normalization and case mapping, ICU's error codes, and the types deficu declaims.

Three tests are about loading:

• library calls load-icu again, as a starting image does, and checks that every binding still works. On Windows it also checks that icu.dll was loaded with an empty suffix.
• saved-image saves an image with cl-icu, starts it in a fresh process and checks that it finds ICU.
• unusable-icu checks the errors for an ICU that cannot be loaded, one that is no ICU cl-icu knows, and a deficu of a function ICU lacks.

The tests need only ICU: libicu-dev or a versioned runtime package on Debian and Ubuntu, nothing on Windows. Some expectations depend on Unicode data and may need adjusting for very old or very new ICU versions.

CI

Forgejo Actions, .forgejo/workflows/ci.yml, on pushes to main and on pull requests: fakehacker/actions/stift-ci@v1, the action's major tag, which follows its compatible releases, on SBCL 2.6.9 in a debian:forky container, installing libicu-dev. Debian names the runtime package after the ICU version, so the development package is installed to get whichever one the image has.

Conventions

• New ICU functions go in ffi-functions.lisp, defined with deficu under their C name without the version suffix, and exported from cl-icu/ffi. Use only functions present in ICU 52, or raise the stated minimum.
• Every failure is an icu-error whose message says what went wrong in terms of the Lisp call and whose hint says what to do. An ICU call that takes a UErrorCode goes inside (with-checked-error-code (error-code "ubrk_open" "find the line boundaries") (cl-icu/ffi:ubrk-open ... error-code)), which binds the variable named first to a fresh UErrorCode and signals with the task in user terms. Argument type problems are type-errors: declaim an ftype for every exported function and check-type its arguments. See Errors.
• Every position returned to the user is a character index; convert ICU's UTF-16 offsets with the tables in utf16.lisp.
• Never pass NULL as a locale to ICU, which would select the process's locale: pass "", the root locale, when the user gives none.