cl-ffi

• For cffi users
• Types
• Memory
• Functions and callbacks
• Strings
• Libraries and saved images
• Errors
• Threads and allocation
• Tests
• License

A foreign function interface for SBCL on 64-bit platforms, in cffi's vocabulary: C types, memory, structs and unions, enums and bitfields, functions, callbacks, strings and shared libraries, which load again by themselves when a saved image starts. It depends on nothing outside the Lisp. It is the foreign function interface of the Pyssel engine, whose bindings all use it.

(asdf:load-system :cl-ffi)

(defpackage #:example
  (:use #:cl)
  (:local-nicknames (#:ffi #:cl-ffi)))
(in-package #:example)

;; A C function, with its types: the Lisp string is copied for the call.
(ffi:defcfun ("strlen" c-strlen) :size (string :string))
(c-strlen "hello")
;; => 5

;; A struct, laid out as C lays it out, here on the stack.
(ffi:defcstruct point (x :int32) (y :int32))
(ffi:with-foreign-object (p '(:struct point))
  (ffi:with-foreign-slots ((x y) p (:struct point))
    (setf x 3 y 4)
    (+ x y)))
;; => 7

;; C calling Lisp: the C library's qsort sorts with a Lisp comparison.
(ffi:defcallback compare :int ((a :pointer) (b :pointer))
  (- (ffi:mem-ref a :int32) (ffi:mem-ref b :int32)))
(ffi:with-foreign-object (numbers :int32 4)
  (loop for n in '(3 1 4 1)
        for i from 0
        do (setf (ffi:mem-aref numbers :int32 i) n))
  (ffi:foreign-funcall "qsort" :pointer numbers :size 4 :size 4
                               :pointer (ffi:callback compare) :void)
  (loop for i below 4 collect (ffi:mem-aref numbers :int32 i)))
;; => (1 1 3 4)

For cffi users

The names are cffi's, and so is what they do, for what cl-ffi has. A cffi pointer is a cl-ffi pointer, so libraries on either can share them. Where cl-ffi differs:

• mem-ref, mem-aref and foreign-slot-value read and write scalars only. A struct, union or array slot has no value to read: take foreign-slot-pointer or mem-aptr for a pointer to it. With a constant type, using one anyway is refused at compile time.
• There is no :convention: x86-64 and AArch64 have one C calling convention, Windows included. No struct passes by value, and there are no foreign variables and no type translators of your own.
• foreign-funcall-pointer takes no list of options, so a call is (foreign-funcall-pointer pointer :int x :int).
• foreign-symbol-pointer looks in every library loaded, and takes no :library.
• Strings are strict both ways: what an encoding cannot represent signals an encoding-error, and bytes that are no valid string a decoding-error. The encodings are UTF-8, the default, UTF-16LE, UTF-16BE and Latin-1, and (:string :encoding :utf-16le) is a string type in one, as in cffi. There is no *default-foreign-encoding* and no *foreign-library-directories*.
• A library clause's condition tests cl-ffi's platform keywords, :windows, :unix, :linux and :darwin, as well as *features*; :windows holds on Windows, though SBCL sets only :win32 itself.
• defcfun declaims the function's ftype, so SBCL checks the types of its arguments, at compile time where it can.
• Libraries load again from their definitions when a saved image starts, and add-image-start-hook runs code then.
• Errors are conditions of type ffi-error.

Types

A type specifier is a keyword for a C scalar type, or one of the forms below. A constant specifier is parsed at compile time, so code that uses it holds only the memory access and the conversion.

Specifier	Size	Lisp values
:int8 to :uint64	1 to 8	integers
:char, :short, :int, :long-long, unsigned forms	1 to 8	integers; :char is signed
:long, :unsigned-long or :ulong	8, or 4 on Windows	integers
:size, :ssize, :intptr, :uintptr, :ptrdiff	8	integers
:float, :double	4, 8	single and double floats
:pointer	8	pointers
:string	8	strings, in calls only
(:string :encoding encoding)	8	strings, in calls only
:boolean, (:boolean type)	as type	generalized booleans
(:struct name), (:union name)	the layout's	none: use pointers
names defctype, defcenum and defbitfield define	the base	as the base, keywords, lists

• (defctype name base-type) names a type that converts as base-type does.
• (defcenum name item...) and (defcenum (name base-type) item...) define an enum: each item a keyword or (keyword value), counting on from the previous value. Its values are keywords to Lisp; an integer passes as itself, and a value no keyword has signals an ffi-error when read. foreign-enum-value, foreign-enum-keyword and foreign-enum-keyword-list convert by hand.
• (defbitfield name item...) defines flags: each item a symbol or (symbol mask). Its values are lists of symbols, one for each single bit set. foreign-bitfield-value, foreign-bitfield-symbols and foreign-bitfield-symbol-list convert by hand.
• (defcstruct name (slot type) (slot type :count n)...) and defcunion define structs and unions with C's layout. A struct slot holds another struct by value; :count makes an array.
• foreign-type-size and foreign-type-alignment give a type's size and alignment, and foreign-slot-offset and foreign-slot-type a slot's offset and type.
• (foreign-type-lisp-type type) gives the Lisp type of the values an argument of type takes, and (foreign-type-lisp-type type :result) the values type of what a call returns as a result of type, both as defcfun declaims them in its functions' ftype: so a binding made by other means can declaim the same.

Memory

• Pointers: null-pointer, null-pointer-p, pointerp, pointer-eq, pointer-address, make-pointer and inc-pointer, and the type foreign-pointer.
• (mem-ref pointer type &optional offset) reads a scalar, and setf writes it; mem-aref indexes an array of them, and mem-aptr gives a pointer to any element, structs included.
• foreign-slot-value reads and writes a scalar slot, and with-foreign-slots makes variables of slots: in (with-foreign-slots ((x (height y)) p (:struct point)) ...), x stands for slot x and height for slot y. foreign-slot-pointer points into an array or struct slot.
• with-foreign-object and with-foreign-objects give memory for the extent of their body: on the stack when the type and count are constants. with-foreign-pointer does the same for a number of bytes.
• foreign-alloc allocates with malloc, optionally filled with :initial-element, and foreign-free frees.
• with-pointer-to-vector-data lends C the elements of a specialized Lisp vector, such as one of (unsigned-byte 8) or single-float, which stays in place for the extent of its body.

Functions and callbacks

(ffi:defcfun ("labs" c-labs) :long
  "Return the absolute value of VALUE."
  (value :long))

(ffi:foreign-funcall "abs" :int -3 :int)                ; => 3
(ffi:foreign-funcall-pointer (ffi:foreign-symbol-pointer "abs") :int -4 :int)

• defcfun defines a function from a C function's name, result type and arguments, with a docstring of your own or one that names the C function. foreign-funcall calls by name without defining anything, and foreign-funcall-pointer through a pointer; the result type comes last, and is :void if left out, for no values.
• foreign-funcall-varargs calls a variadic function, the fixed arguments in a list and the variable ones after it, promoted as C promotes them: (foreign-funcall-varargs "snprintf" (:pointer buffer :size size :string "%d") :int 42 :int). On Windows, a variable float among the first four arguments goes in the integer register the function reads it from, as the platform asks: SBCL does not put it there, so cl-ffi passes its bits there itself.
• defcallback defines a function C can call, and callback and get-callback return the pointer to give C. A callback converts its arguments and result as their types say. Nothing may unwind through C frames, so a callback must not let a condition or a non-local exit out of its body: it records what it was told and returns.
• errno returns the errno the last foreign call left. Read it right after the call, since any other foreign call may change it. On Windows it is the errno of msvcrt, the C library calls by name reach there.
• with-foreign-float-environment masks the floating-point traps SBCL unmasks, as C code expects. Call C code that computes with floats, such as a GPU driver, inside it.

Strings

• foreign-string-to-lisp reads a string to its terminator, or :count bytes, at most :max-chars characters, and returns it with the number of bytes read; a null pointer gives nil.
• foreign-string-alloc copies a Lisp string to malloc'd memory, and foreign-string-free frees it; with-foreign-string does both around a body.
• lisp-string-to-foreign writes into a buffer of yours as many characters as fit whole, and the terminator.
• with-foreign-pointer-as-string gives C a buffer to write a string into, and returns it.
• In defcfun and foreign-funcall, a :string argument takes a Lisp string, copied for the call and freed after it, or a pointer; a :string result is a Lisp string, or nil for a null pointer.

• (:string :encoding encoding) does the same in another encoding than UTF-8: :utf-16le, :utf-16be or :latin-1. It is what the wide strings of Windows' W functions are, so a binding of one takes and returns Lisp strings:

  (ffi:defctype wide-string (:string :encoding :utf-16le))
  (ffi:defcfun ("SetWindowTextW" set-window-text) :int32
    (window :pointer) (text wide-string))

  A callback's parameter of a string type is read in its encoding too. Struct slots and mem-ref take no string type: read the pointer as :pointer and give it to foreign-string-to-lisp with its :encoding.

Libraries and saved images

(ffi:define-foreign-library libz
  (:windows "zlib1.dll")
  (:unix (:or "libz.so.1" "libz.so")))

(ffi:use-foreign-library libz)

(ffi:defcfun ("zlibVersion" zlib-version) :string)
(zlib-version)
;; => "1.3.2", or whichever zlib is installed

• define-foreign-library names the files to look for: each clause (condition spec), the first whose condition holds being used. A spec is a file name the system searches for, a full path, (:or spec...) to try one after another, or (:default "name") for the platform's suffix.
• use-foreign-library loads a library when its file is loaded, and load-foreign-library when it is called; foreign-library-loaded-p and foreign-library-pathname say whether and from where.
• load-foreign-library also takes a spec instead of a name, for a library of no name, such as one whose file name is known only at run time: (load-foreign-library (format nil "libicui18n.so.~D" 78)).
• A library that cannot be loaded signals a load-foreign-library-error, which lists each file tried and the system's reason.
• When an image saved with cl-ffi starts, it loads each library again, in order, from its definition or spec, rather than from the path it was loaded from, which on another machine may not exist. Then it runs the functions add-image-start-hook added, for state such as pointers looked up in a library. What fails then is reported as a warning.

Errors

Every error cl-ffi signals is an ffi-error, whose report says what was wrong and what to write instead:

• load-foreign-library-error, with load-foreign-library-error-library and load-foreign-library-error-attempts;
• encoding-error, with encoding-error-encoding and encoding-error-character;
• decoding-error, with decoding-error-encoding and decoding-error-position, the byte where the string goes wrong;
• and plain ffi-error conditions for unknown types, slots, enum keywords and callbacks, and for memory malloc cannot give.

An argument of the wrong kind is a type-error before cl-ffi looks at it: each function declaims the full domain of its arguments, such as the encodings for :encoding and the shape of a type specifier, so SBCL checks them when the function is called, and warns of a constant one when it compiles the call:

The value :UCS-2 is not of type (MEMBER :LATIN-1 :UTF-8 :UTF-16LE :UTF-16BE) when binding :ENCODING

Threads and allocation

• Definitions are made at load time, and any thread may read them. Pointers are values any thread may use; the memory they point to has the rules of whatever owns it. A callback runs on the thread the C code calls it on, which SBCL allows for threads C started.
• What code that runs every frame needs allocates nothing, which the tests measure: mem-ref, mem-aref, foreign-slot-value and their setf forms with constant types, with-foreign-object of a constant size, calls through foreign-funcall and foreign-funcall-pointer of numbers and pointers, and reading through a pointer that make-pointer made from an address. On SBCL, passing a pointer to stack memory to a function that is not inlined boxes it, so a binding's per-frame functions should be declared inline.

Tests

stift test

The suite calls the C library, so it needs nothing installed beyond the Lisp. It holds cl-ffi's layouts to a C program's, whose output for Linux and for Windows is committed (see tests/layout.c), and saves and starts an image, in a Lisp of its own, to check that libraries, calls, callbacks and hooks come back. It runs on Linux and on Windows.

License

MIT; see LICENSE.