cl-wl

• Usage
  • Binding globals
  • Names and values
  • Events
  • Allocation
  • Lifetimes
  • Shared memory and keymaps
  • Pipes
  • Introspection
  • Errors
  • Threads
  • Cursors
• Testing code built on cl-wl
• The scanner
• Protocol files
• Development
• License

Common Lisp bindings for the Wayland client library, libwayland-client.

cl-wl is made of these systems:

• cl-wl: the Lisp-idiomatic API. It covers displays, proxies and event dispatch, and includes the bindings for the core protocol (package cl-wl/wayland). It returns Lisp values, turns errors into conditions, and has foreign callbacks only queue events.
• cl-wl/xdg-shell: bindings for xdg-shell (package cl-wl/xdg-shell).
• cl-wl/xdg-decoration: bindings for xdg-decoration, with which a client asks the compositor to draw its windows' title bars (package cl-wl/xdg-decoration).
• cl-wl/fractional-scale and cl-wl/viewporter: bindings for fractional-scale-v1, with which the compositor tells a client the scale to draw a surface at, such as 1.5, and viewporter, with which the client shows a larger buffer at the surface's size (packages cl-wl/fractional-scale and cl-wl/viewporter).
• cl-wl/cursor-shape: bindings for cursor-shape-v1, with which a client asks the compositor to draw a cursor by name, such as :text (package cl-wl/cursor-shape). It refers to tablet tools, so it loads cl-wl/tablet, the bindings for tablet-v2 (package cl-wl/tablet).
• cl-wl/text-input: bindings for text-input-v3, through which input methods, such as those for Chinese and Japanese, send the text being composed and the text chosen (package cl-wl/text-input).
• cl-wl/pointer-constraints: bindings for pointer-constraints-v1, which locks the pointer in place or confines it to a surface, as games and drawing programs want (package cl-wl/pointer-constraints).
• cl-wl/relative-pointer: bindings for relative-pointer-v1, the pointer's motion as it is, with and without acceleration, which goes on while the pointer is locked (package cl-wl/relative-pointer).
• cl-wl/xdg-dialog: bindings for xdg-dialog-v1, which tells the compositor that a window with a parent is a dialog of it, and whether it is modal (package cl-wl/xdg-dialog).
• cl-wl/cursor: cursor themes through libwayland-cursor, for clients that set their own cursor images where the compositor does not offer cursor-shape-v1 (package cl-wl/cursor); see "Cursors" below. Its raw layer is cl-wl/cursor-ffi.
• cl-wl/ffi: the raw layer underneath, on cl-ffi, a direct transcription of the libwayland-client C API. It does no error checking and no memory management.
• cl-wl/test-compositor: an in-process compositor for testing code built on cl-wl without a real compositor (package cl-wl/test-compositor); see "Testing" below.
• cl-wl/scanner: a build-time tool that reads Wayland protocol XML and generates the bindings. The generated code is committed, so loading cl-wl needs neither the scanner nor any XML.

Only the libwayland-client shared library (1.20 or newer) is needed at run time, and libwayland-cursor, which comes with it, for cl-wl/cursor. No C compiler or Wayland headers are needed to build.

Usage

(asdf:load-system :cl-wl/xdg-shell)

(defpackage #:example
  (:use #:cl)
  (:local-nicknames (#:wl #:cl-wl)
                    (#:wlp #:cl-wl/wayland)
                    (#:xdg #:cl-wl/xdg-shell)))
(in-package #:example)

(wl:with-display (display)
  (let* ((compositor (wl:bind display 'wlp:wl-compositor))
         (wm-base (wl:bind display 'xdg:xdg-wm-base))
         (surface (wlp:wl-compositor-create-surface compositor))
         (xdg-surface (xdg:xdg-wm-base-get-xdg-surface wm-base surface))
         (toplevel (xdg:xdg-surface-get-toplevel xdg-surface)))
    (wl:set-event-handlers xdg-surface
                           :configure (lambda (xdg-surface serial)
                                        (xdg:xdg-surface-ack-configure xdg-surface serial)))
    (wl:set-event-handlers wm-base
                           :ping (lambda (wm-base serial)
                                   (xdg:xdg-wm-base-pong wm-base serial)))
    (wl:set-event-handlers toplevel
                           :configure (lambda (toplevel width height states)
                                        (declare (ignore toplevel))
                                        (format t "~&Configure: ~Dx~D ~A~%" width height states))
                           :close (lambda (toplevel)
                                    (declare (ignore toplevel))
                                    (format t "~&Close requested~%")))
    (xdg:xdg-toplevel-set-title toplevel "Hello")
    (wlp:wl-surface-commit surface)
    (wl:dispatch display :timeout 1)))
;; Configure: 0x0 NIL

Binding globals

bind binds the compositor's global of an interface, at the highest version that both the compositor and the bindings support. Pass :version to cap it, and :errorp nil to get NIL for an interface the compositor does not offer. bind-all binds every global of an interface, such as each wl_output. global-version and list-globals tell what the compositor offers, supported-version what the bindings support, and find-proxy-class maps a protocol name from list-globals to its class.

Globals come and go, as displays are plugged in and out. follow-globals binds every global of an interface now and each one announced later, and hands it over; when the compositor withdraws one, it hands the proxy back, for you to destroy:

(wl:follow-globals display 'wlp:wl-output
                   :added (lambda (output) (watch-output output))
                   :removed (lambda (output)
                              (forget-output output)
                              (wl:destroy-proxy output)))

Globals announced later arrive while events are dispatched. stop-following ends it.

Names and values

Protocol names become Lisp names: interface wl_surface is the proxy class wl-surface, its request attach is the function wl-surface-attach, its event enter is the keyword :enter, and its enum error is the type wl-surface-error. Each class's documentation lists its events and their arguments, and each request function's documentation lists its arguments.

Wire type	Lisp value
int, uint	integer
enum	keyword, or a list of keywords for a bitfield; an unknown value arrives as an integer
fixed	single-float in events, exact below 65536; any real in requests
string	string, or NIL where the protocol allows null
object	proxy, or NIL where the protocol allows null
new_id	the new proxy, returned by the request
array	list of keywords for an array of enum values, (vector (unsigned-byte 32)) for one of integers, otherwise (vector (unsigned-byte 8))
fd	integer; one received belongs to the receiver, who closes it with close-fd

The protocol format cannot say what an array holds, so the bindings decode the arrays whose contents the protocols describe: xdg_toplevel's configure states arrive as a list such as (:maximized :activated), its wm_capabilities likewise, and wl_keyboard's enter keys as a vector of keycodes. Other arrays arrive as bytes. cl-wl/scanner:shipped-array-elements returns the list.

A request that creates an object returns its proxy. A request newer than the proxy's version signals version-error before anything is sent.

Events

Each event of a proxy gets its own handler, set with set-event-handlers or (setf event-handler). A handler is called with the proxy and the event's arguments, in the order the class documentation lists them. Events without a handler are dropped. Setting a handler for an event the interface does not have signals an error that lists the events it does have.

Events are decoded inside libwayland's callback and queued. dispatch, dispatch-pending and roundtrip then call the handlers once libwayland has returned, so handlers may signal and unwind freely. If a handler unwinds, the events after it stay queued for the next dispatch.

dispatch reads and delivers events. It waits until some arrive, or for :timeout seconds (0: don't wait).

For a program with an event thread that shares the display with other threads, dispatch splits in two. read-events waits for events and reads them without delivering them, and is the one function that may run while another thread uses the display. dispatch-pending then delivers them. Guard the display with a lock; call read-events without holding it, and everything else holding it.

Allocation

Events and requests run every frame, so in steady state they allocate nothing: not to queue an event, not to call its handler, and not to send a request. The exceptions are values that have to be new Lisp objects: strings, arrays, new proxies, and the first occurrence of each bitfield value (its keyword list is cached per display afterwards, so do not modify it). A test checks this.

Lifetimes

destroy-proxy destroys any proxy. It sends the interface's destructor request if the proxy's version has one, such as wl_output.release from version 3, and otherwise destroys the proxy on the client side only, as for wl_registry or a version-2 wl_output. The destructor requests, such as wl-surface-destroy, may also be called directly. An event that ends its object, such as wl_callback's :done, destroys the proxy after its handler has run. disconnect destroys every proxy of the display. Using a destroyed proxy signals destroyed-proxy-error. It never touches freed memory. proxy-pointer returns the raw wl_proxy, or the wl_display for a display, for code such as Vulkan surface creation.

borrow-proxy goes the other way: it wraps a wl_proxy that other C code made and owns, such as a cursor image's wl_buffer in libwayland-cursor, so that it can be passed to requests. A borrowed proxy receives no events, and its destructor requests signal wl-error; destroy-proxy and disconnect forget it without destroying the object. Forget it before its owner destroys the object.

Shared memory and keymaps

wl_shm buffers live in a file both sides map. make-shared-memory makes one, write-shared-memory copies octets or 32-bit pixels into it without allocating, and destroy-shared-memory (or with-shared-memory) frees it:

(wl:with-shared-memory (memory (* 4 width height))
  (wl:write-shared-memory memory pixels)   ; (simple-array (unsigned-byte 32) (*))
  (let* ((pool (wlp:wl-shm-create-pool shm (wl:shared-memory-fd memory)
                                       (wl:shared-memory-size memory)))
         (buffer (wlp:wl-shm-pool-create-buffer pool 0 width height (* 4 width) :xrgb8888)))
    (wlp:wl-shm-pool-destroy pool)
    …))

shared-memory-pointer is the escape hatch for drawing straight into it. read-keymap turns the fd and size of wl_keyboard's :keymap event into the keymap's text, for cl-xkbcommon, and closes the fd. An fd an event delivers is yours to close, so close one you do not read, such as a keymap in a format you do not know, with close-fd.

Pipes

The clipboard's data moves through pipes. A client reads the clipboard by passing the write end of one to wl_data_offer.receive; the client whose text it is gets that end in its wl_data_source :send event, writes the text and closes it. make-pipe makes the pipe, receive-text reads until the writer closes its end and send-text writes, each closing its file descriptor and giving up after :timeout seconds:

(multiple-value-bind (in out) (wl:make-pipe)
  (wlp:wl-data-offer-receive offer "text/plain;charset=utf-8" out)
  (wl:flush display)
  (wl:close-fd out)
  (wl:receive-text in :timeout 1))       ; => the text, or NIL after a second

(wl:set-event-handlers source
                       :send (lambda (source type fd)
                               (declare (ignore source type))
                               (wl:send-text fd text :timeout 1)))

send-text returns false if the reader closed its end early or did not read in time, so a reader that never reads cannot hold up the event loop. Do not wait in receive-text for text your own client sends: it is sent only while your client dispatches events.

Introspection

interface-requests and interface-events describe an interface's messages and their arguments, as message and argument objects. interface-pointer builds the foreign wl_interface tables libwayland needs, for code that talks to libwayland directly, such as the test compositor.

Errors

All conditions are of type wl-error:

• compositor-error: the compositor raised a fatal protocol error.
• connection-error: a connection failed or broke.
• version-error: a request needs a newer interface version.
• destroyed-proxy-error: a request was sent on a destroyed proxy.
• event-error: an event could not be decoded, for example invalid UTF-8.

A connection that has failed stays failed. display-error returns the compositor-error or connection-error that broke it, or nil while it works, without signalling: use it to tell a broken connection from other failures, for example when releasing objects of a display that may be gone.

Threads

A display, and every proxy created on it, may be used by one thread at a time, except that read-events may wait on the display while another thread uses it. Different displays may be used on different threads.

Cursors

A client sets the pointer's cursor after each wl_pointer :enter, with the enter's serial. Where the compositor offers wp_cursor_shape_manager_v1, it draws the cursor from a name:

(let ((device (cl-wl/cursor-shape:wp-cursor-shape-manager-v1-get-pointer manager pointer)))
  (cl-wl/cursor-shape:wp-cursor-shape-device-v1-set-shape device serial :text))

Elsewhere the client attaches the cursor's image to a surface of its own. cl-wl/cursor loads the images from the user's cursor theme:

(asdf:load-system :cl-wl/cursor)

(cl-wl/cursor:with-cursor-theme (theme shm)            ; the user's theme and size
  (let* ((cursor (or (cl-wl/cursor:find-cursor theme "text")
                     (cl-wl/cursor:find-cursor theme "xterm")))
         (image (svref (cl-wl/cursor:cursor-images cursor) 0))
         (surface (wlp:wl-compositor-create-surface compositor)))
    (wlp:wl-surface-attach surface (cl-wl/cursor:cursor-image-buffer image) 0 0)
    (wlp:wl-surface-commit surface)
    (wlp:wl-pointer-set-cursor pointer serial surface
                               (cl-wl/cursor:cursor-image-hotspot-x image)
                               (cl-wl/cursor:cursor-image-hotspot-y image))
    …))

load-cursor-theme reads the user's theme and size from $XCURSOR_THEME and $XCURSOR_SIZE, as other toolkits do, and takes :name and :size. Load a theme at twice the size for a surface of buffer scale 2. A theme that is not installed falls back to the default theme, then to a few cursors built into libwayland-cursor ("default", "text", "pointer", "wait" and some more), so some cursors are always found. Themes name cursors as CSS does ("text", "ew-resize"), and older ones as X did ("xterm", "sb_h_double_arrow"): look for both. An animated cursor has several images; cursor-frame says which to show when, and for how long. The theme owns the buffers: never destroy them, and destroy the theme, with destroy-cursor-theme, before disconnecting the display.

Testing code built on cl-wl

cl-wl/test-compositor runs a minimal compositor in the test process, on libwayland-server, connected to a client display over a socketpair. It advertises the globals a test asks for, records every request, and sends the events the test posts. It runs only when told to, so tests are deterministic.

(asdf:load-system :cl-wl/test-compositor)

(use-package :cl-wl/test-compositor)

(with-test-compositor (compositor display (cl-wl/wayland:wl-compositor 6))
  (let* ((wl-compositor (bind-global compositor display 'cl-wl/wayland:wl-compositor 6))
         (surface (cl-wl/wayland:wl-compositor-create-surface wl-compositor)))
    (cl-wl/wayland:wl-surface-commit surface)
    (pump compositor display)
    (mapcar #'request-name (compositor-requests compositor))))
;; => ("bind" "create_surface" "commit")

pump moves messages both ways until nothing moves: it sends the client's requests, lets the compositor handle them and dispatches the client's events, again and again, so requests that handlers send in answer arrive too. add-global and remove-global advertise and withdraw a global while the client runs, as when a display is plugged in and out. compositor-requests returns what arrived, as request objects with a name, the resource it was sent to, its arguments and the resource it created; compositor-last-target and compositor-last-created find those resources. Values come and go as cl-wl delivers them, with enum values as keywords: post-event sends (post-event compositor toplevel :configure 640 480 '(:activated)). post-error raises a protocol error, and disconnect-client drops the client, for testing how code copes with a broken connection. compositor-errors lists errors raised in the compositor's callbacks; a test should check that it is empty. It needs libwayland-server (Debian/Ubuntu: libwayland-server0).

The scanner

The scanner works in two stages:

1. The reader (scanner/reader.lisp) parses a protocol XML file into a protocol model of plain, read-only structs: protocol, interface, message, arg, enum, entry and description. It checks the file against the protocol format and signals a protocol-error that says where the problem is.
2. The generator (scanner/generator.lisp) reads only the model. It writes define-enum, define-interface and define-request forms, which cl-wl's macros expand into classes, functions and the tables libwayland needs.

The reader is the only part of the scanner that uses an XML library (xmls), so swapping that library means rewriting one file. The reader tests in tests/reader-test.lisp define what any replacement must produce.

To regenerate the committed bindings after changing a protocol file or the generator:

(asdf:load-system :cl-wl/scanner)
(cl-wl/scanner:regenerate-bindings)

A test checks that the committed files match what the scanner generates. The output is laid out by hand rather than by the pretty printer, whose layout can change between SBCL releases, so every release produces the same bytes.

Protocol files

protocols/ holds the protocol XML that the bindings are generated from, so regenerating gives the same output on every machine. The generated bindings sit next to them:

• wayland.xml: the core protocol, from Wayland 1.26.0.
• xdg-shell.xml: from wayland-protocols 1.49.
• xdg-decoration.xml: unstable/xdg-decoration/xdg-decoration-unstable-v1.xml from wayland-protocols 1.49.
• viewporter.xml: stable/viewporter/viewporter.xml from wayland-protocols 1.49.
• fractional-scale.xml: staging/fractional-scale/fractional-scale-v1.xml from wayland-protocols 1.49.
• tablet.xml: stable/tablet/tablet-v2.xml from wayland-protocols 1.49.
• cursor-shape.xml: staging/cursor-shape/cursor-shape-v1.xml from wayland-protocols 1.49.
• text-input.xml: unstable/text-input/text-input-unstable-v3.xml from wayland-protocols 1.49.
• pointer-constraints.xml: unstable/pointer-constraints/pointer-constraints-unstable-v1.xml from wayland-protocols 1.49.
• relative-pointer.xml: unstable/relative-pointer/relative-pointer-unstable-v1.xml from wayland-protocols 1.49.
• xdg-dialog.xml: staging/xdg-dialog/xdg-dialog-v1.xml from wayland-protocols 1.49.

Each file keeps its own copyright and licence notice.

Development

Dependencies are pinned in the stiftfile with stift, and tests use parachute.

stift test

The tests need libwayland-client, libwayland-server and libwayland-cursor (Debian/Ubuntu: libwayland-client0 libwayland-server0 libwayland-cursor0). They run against a small compositor built on libwayland-server inside the test process, so they need no running compositor.

License

MIT. See LICENSE. The files in protocols/ are under their own licences, stated in each file.