Strings

Encodings

Keyword	Terminator
:utf-8, the default	one zero byte
:utf-16le, :utf-16be	two zero bytes, at an even distance
:latin-1	one zero byte

Both directions are strict. A character an encoding cannot represent, a code point above U+00FF in Latin-1 or a surrogate in UTF-8 and UTF-16, signals an encoding-error that names the character and its index. Bytes that are no valid string, a malformed, truncated, overlong or surrogate UTF-8 sequence or a lone UTF-16 surrogate, signal a decoding-error that gives the byte where the string goes wrong.

Reading

(ffi:foreign-string-to-lisp pointer)                       ; to the terminator
(ffi:foreign-string-to-lisp pointer :count 12)             ; exactly 12 bytes
(ffi:foreign-string-to-lisp pointer :encoding :utf-16le :max-chars 10)

foreign-string-to-lisp returns the string and the number of bytes it took, or nil for a null pointer. It reads to the terminator, or :count bytes, zero bytes among them, from :offset bytes past the pointer, and at most :max-chars characters.

Writing

• foreign-string-alloc copies a string to malloc'd memory and returns the pointer and the bytes it takes, the terminator included, or none with :null-terminated-p nil. foreign-string-free frees it.
• with-foreign-string does both around a body: (with-foreign-string ((pointer size) text :encoding :utf-16le) ...).
• lisp-string-to-foreign writes into a buffer of yours as many whole characters as fit, and the terminator.
• with-foreign-pointer-as-string gives C a buffer to write a string into, and returns the string it holds after the body.

In calls

A :string argument of defcfun, foreign-funcall and the others takes a Lisp string, copied as UTF-8 for the call and freed after it, or a pointer as it is. A :string result is a Lisp string, or nil for a null pointer; the memory stays C's. nil is no argument: pass (ffi:null-pointer) for C's NULL.

(:string :encoding encoding) is a string type in another encoding, with the same rules: an argument is copied in that encoding, ending in its terminator, and a result or a callback's parameter is read in it. The wide strings of Windows' W functions are (:string :encoding :utf-16le):

(ffi:defctype wide-string (:string :encoding :utf-16le)
  "A null-terminated UTF-16 string, LPCWSTR.")

(ffi:defcfun ("SetWindowTextW" set-window-text) :int32
  (window :pointer) (text wide-string))

(set-window-text window "Grüße, ☃ 𝄞")

(:string) is :string. An encoding cl-ffi does not know is refused: in a type specifier with an ffi-error that lists the ones it knows, and as the :encoding of a string function with a type-error, whose expected type is the four of them:

The value :UCS-2 is not of type (MEMBER :LATIN-1 :UTF-8 :UTF-16LE :UTF-16BE) when binding :ENCODING

A callback cannot return a string type, and struct slots and mem-ref take none: read the pointer as :pointer and give it to foreign-string-to-lisp with its :encoding.