This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

Internals

How cl-wl moves messages between Lisp and libwayland. This page is for contributors. Users of cl-wl do not need any of it.

Source layout

File	Contents
ffi/	cl-wl/ffi, the raw layer
src/package.lisp	the cl-wl package
src/conditions.lisp	wl-error and its subclasses
src/enums.lisp	the fixed and fd types, define-enum, enum-value, enum-keywords
src/interfaces.lisp	the descriptions of interfaces, messages and arguments; introspection; define-interface, define-request
src/tables.lisp	building the foreign wl_interface tables
src/proxy.lisp	proxy, display, event handlers, destroy-proxy
src/dispatch.lisp	the C dispatcher, the event queue, delivery, the dispatch functions
src/requests.lisp	marshalling and sending requests
src/display.lisp	connect, disconnect, with-display
src/shared-memory.lisp	shared-memory and read-keymap
protocols/wayland.lisp	the generated core protocol
src/globals.lisp	bind and friends; loads after the core protocol, whose requests it uses
scanner/	cl-wl/scanner
test-compositor/	cl-wl/test-compositor, which uses only cl-wl's exported API
tests/	the Parachute suite

Descriptions

Loading generated bindings fills three tables:

• *interfaces* maps a proxy class name to an interface-info, which holds the wire name, the version, and the requests and events as vectors of message indexed by opcode.
• *interfaces-by-wire-name* is the same table keyed by protocol name. find-proxy-class reads it.
• *enums* maps an enum type name to an enum-info: its entries and whether it is a bitfield.

A message holds the Lisp name, wire name, opcode, since version, destructor flag, and a list of argument. An argument holds the name, wire type, interface, nullability, enum and array element type. Both are exported, with their readers, for introspection; interface-info and enum-info stay internal, and interface-requests and interface-events hand out fresh lists of their messages.

The tables are written only while bindings load, and only read afterwards, so any thread may read them.

define-interface registers the events, including their :destructor option. Each define-request adds its message to the interface's request vector, so redefining an interface alone keeps its requests.

Interface tables

libwayland describes each interface with a struct wl_interface: its name, version, and arrays of wl_message, each of which holds a name, a signature string and an array of wl_interface pointers for its object arguments. C bindings compile these structs in.

cl-wl builds them from the descriptions in foreign memory, per display, on first use (info-interface-pointer in src/tables.lisp), and frees them on disconnect. Interfaces refer to each other, often in cycles, so each struct is registered before the messages that may refer back to it are built. Because no foreign memory is global:

• nothing needs rebuilding after an image is saved and restarted;
• no lock is needed, because each display owns its tables.

The signature string follows wayland-util.h. It starts with the since version if that is above 1. Each argument is then one character (i u f s o n a h), preceded by ? if nullable. A new_id without an interface is written sun (interface name, version, id).

The exported interface-tables, interface-pointer and free-interface-tables give the same tables to other code. The test compositor builds its tables with them, so both ends describe protocols identically.

Sending a request

define-request expands into a function that:

1. checks each argument's type, the proxy's liveness, and its version against since;
2. stack-allocates a wl_argument array with cl-ffi:with-foreign-object;
3. stores each argument at offset 8 × index. Enums are converted with enum-value, fixed numbers are rounded to 24.8, and strings and arrays are copied to dynamic-extent foreign memory. Pointers are stored as integer addresses so that no pointer object is made;
4. calls send-request, which is inlined so that the stack array is never boxed. send-request calls wl_proxy_marshal_array_flags, passing the new object's interface table if the request creates one and WL_MARSHAL_FLAG_DESTROY if it is a destructor;
5. wraps a returned wl_proxy in a new proxy of the right class, and registers it with the display and with the dispatcher, or forgets the proxy if it was destroyed.

A request that creates no object allocates nothing.

destroy-proxy looks for a destructor request among the interface's requests that takes no arguments and whose since the proxy's version reaches. It calls that request's function if there is one, and otherwise destroys the wl_proxy on the client side.

Receiving an event

Every proxy has the same C dispatcher, dispatcher, installed with wl_proxy_add_dispatcher. libwayland calls it from inside wl_display_dispatch_pending and friends. cl-wl's dispatch functions bind *dispatching-display* around those calls so that the dispatcher can find the display.

The dispatcher:

1. looks the target proxy up by address in the display's proxy table, and drops the event if the proxy is unknown;
2. takes the next free event-record from the display's ring buffer, growing it if full;
3. decodes each argument into the record's argument vector. Enums become keywords, with bitfield lists cached per display. Fixed numbers become single-floats, and strings are decoded as UTF-8. Objects are looked up by address, and new objects are adopted as new proxies. Arrays become vectors or lists as their argument says;
4. returns 0.

Everything happens inside handler-case. A condition is kept in the display's event-error slot and the event is dropped. Nothing unwinds through libwayland's frames. The callback receives addresses as integers, and arguments are read at constant offsets through pointers cl-ffi:make-pointer makes and only reads through, so decoding allocates no pointer objects.

After libwayland returns, deliver:

1. signals event-error if one is held;
2. pops records oldest first. Each record leaves the queue before its handler runs, so an unwinding handler leaves the remaining events queued, and a handler may dispatch re-entrantly;
3. calls the handler through call-handler, a case over the argument count up to ten that passes arguments without consing a list. It clears the record's arguments first, so the ring keeps nothing alive;
4. for a destructor event, destroys the proxy on the client side in the cleanup of an unwind-protect, so it is destroyed even if the handler unwinds. The compositor has destroyed the object already, and libwayland frees its id when the delete_id that follows arrives.

Dispatch, step by step

dispatch follows libwayland's documented read protocol:

1. wl_display_prepare_read. If it refuses because events are queued, dispatch_pending them (they are queued as Lisp events) and retry;
2. wl_display_flush. A failure is fatal only if the display reports an error; a full socket is not;
3. poll the fd until readable or the timeout passes, resuming after EINTR with the time left;
4. wl_display_read_events, or wl_display_cancel_read on timeout or unwind;
5. wl_display_dispatch_pending, which runs dispatcher for each event;
6. deliver.

Steps 2 to 4 are one function, finish-read, which read-events shares. read-events makes one wl_display_prepare_read: if it refuses, events are waiting to be delivered and read-events returns true at once; otherwise it finishes the read and returns whether events came, without dispatching. Because it touches only libwayland's socket and queues, it may run while another thread uses the display.

Errors from libwayland

A negative result from a display function makes cl-wl ask wl_display_get_error. If a protocol error is recorded (wl_display_get_protocol_error gives a non-null interface), the result is a compositor-error. Otherwise it is a connection-error with the errno and its strerror text. display-error builds and returns that condition, and the dispatch functions signal it.

Shared memory

make-shared-memory calls memfd_create with MFD_CLOEXEC, sizes the file with ftruncate and maps it shared, readable and writable. The memory keeps its fd and mapping; the compositor maps its own duplicate of the fd when a pool is created. write-shared-memory copies with an inlined memcpy whose destination is an integer address and whose source comes from cl-ffi:with-pointer-to-vector-data, so a write makes no pointer object. read-keymap maps the keymap privately, reads up to the first null byte, unmaps it and closes the fd in unwind-protect cleanups.

Allocation tests

tests/runtime-test.lisp counts the bytes allocated over many events, requests and shared-memory writes, and requires zero in steady state. SBCL's counter only advances when an allocation region fills, which is why many iterations are counted.

Conventions

The code follows the workspace's standards. Every exported function declaims an ftype that gives each argument its full type, which SBCL checks on entry; the bodies check only what a type cannot say. Every slot has a :type. Every definition has a docstring. Implementation-specific code lives only in cl-ffi. Foreign callbacks only queue. Safety is never lowered.